I recently partook in an inspiring, thought-provoking Minnesota Evaluation Association panel called “What is evaluation?”
The discussion quickly became profound, reaching beyond simply defining what we all do. I found it to be a brave, loving space where people with different perspectives leaned in and listened deeply to each other. Since evaluation is as much about asking questions as it is about providing answers, I want to share a little about the conversation and leave you with some questions to prompt you to look at your relationship with evaluation in a new way.
The energy of Dr. Nicky Bowman (Waapalaneexkweew, Mohican/Lunaape) was very present in the discussion; she shared her experiences and questions she is facing, and I also remembered some of the stories she shares in her paper with Carolee Dodge-Francis, Ed.D., “Culturally Responsive Indigenous Evaluation and Tribal Governments: Understanding the Relationship.” In the paper, they consider Indigenous evaluation from four perspectives: Ktanaxkihlaak—Eastern Door, or the origin story and beginnings; Shaawaneewang—Southern Door, or balanced development and Indigenous contributions; Wsihkaang—Western Door, or perseverance and unknown potentiality; and Loowaneewang—Northern Door, or turning to elder wisdom to guide new beginnings.
The door framework helped me think of other doors that were opened during the conversation: the doors that lead away from what evaluation is and toward other questions about who does it, how, when, where, and why.
Who
The title of “evaluator” can mean so many things. At The Improve Group, we see everyone as evaluators with different levels of experience or ways of seeing their work. Our consultants bring to our organization many different degrees, lived experiences, and areas of expertise. Is everyone an evaluator? Or is it a professional title or distinction? Does someone need to identify as an evaluator to be in an evaluative practice?
It’s also important to consider these questions within the context of how white supremacy and oppression have shaped the field and its makeup. The research of Vidhya Shanker, Ph.D., provides insight into how systemic racism and oppression have influenced how we think about and practice evaluation and also who we uplift and leave out of the founding narrative about evaluation.
How
I have seen definitions of evaluation that emphasize a systemic process (see Better Evaluation and Canadian Evaluation Society). There are also numerous methods used in evaluation to try to balance knowledge, burden, and bias. During the conversation, I heard a passion for evaluation that centers community, equity, and adaptability.
When
At the MNEA event, we talked about what evaluation has been in the past, what it is now, and what we want it to be in the future. This framework helps me organize my thoughts—what I like and do not like from the history of evaluation; what we are learning about evaluation today in a challenging time; and how we could nurture evaluation to be more equitable in the future.
Where
If evaluation takes place in institutions or even whole sectors, which themselves have a history grounded in oppression and white supremacy (see 'Decolonizing Wealth' Addresses Philanthropy's White Supremacy Problem, Offers Solutions'), what is the evaluation’s responsibility? During the conversation, I was inspired by hearing how people overcame the constraints of their organizations to do meaningful evaluation.
Why
For what purpose do you evaluate—are you hoping to learn? Be responsive to a funder? Better engage stakeholders? More importantly, are these purposes mutually exclusive? There was a time in evaluation when purposes centered around finding value and merit were emphasized in professional evaluation training; more recently, evaluation has been coupled with learning and decision-making.
I hope these questions raise questions of your own and prompt you to explore your relationship with evaluation, the role of evaluation in social change, and the intersection of the two.