By Thomas Archibald
Associate Professor, Department of Agricultural, Leadership, and Community Education, Virginia Tech
To achieve the Sustainable Development Goals (SDGs) and recover from the devastating impacts of COVID-19, business as usual is unacceptable. Paradigmatic transformations are needed as civil society, industry, and governments alike rethink how they work and how they define success in relation to social, economic, and ecological impacts. In parallel, conventional evaluation approaches are inadequate. Reductionist, linear, technical-rationalistic modes of thinking and evaluating are ill-equipped to face the complex dynamic challenges confronting us. As Michael Quinn Patton and the Blue Marble Evaluation (BME) community suggest, evaluating transformation requires transforming evaluation.
One way to foster transformation in evaluation is to intentionally and explicitly focus on critical and evaluative thinking skills. Specifically, evaluative thinking can broaden recognition of evaluation as a powerful tool to improve public accountability and good governance, while simultaneously bolstering evaluation systems and capacities among evaluators and programme implementors alike.
Evaluating transformation requires transforming evaluation.
The needed transformations of evaluation
One clarion call about transformation is from the BME approach, with its three overarching principles: global thinking, Anthropocene[1] as context and transformative engagement.
The premise of the third principle is that, “Global, anthropogenic problems are so severe, threatening the future sustainability of the planet and humanity, that major and rapid systems transformations are needed.”[2]
Another call for transformation is from Thomas Schwandt[3], reflected on thoughtfully by Zenda Ofir.[4] Against the backdrop and predominance of ‘normal evaluation,’ which is “wedded to notions of scientific rationality, social progress, effectiveness and efficiency in social programming, and the broad ideology of modernization,” post-normal evaluation draws on:
Resilience thinking as a rationality of governing
The return of politics to the people
A recovery of practical reasoning
Co-production to redefine the relationship between professionals and citizens, and
Ethical accountability.
Yet another call for transformation is the call for equitable evaluation, as expressed via the principles of the Equitable Evaluation Initiative, including that evaluation should be in service of equity, should be multiculturally valid and orientated toward participant ownership, and should answer critical questions about the historical and structural entanglements of societal conditions and the strategies designed to address them.
One final call for transformation with clear implications for the evaluation of the SDGs is the movement towards decolonizing evaluation—which requires that evaluation itself be decolonized while simultaneously acting as a decolonizing force in society. This topic is discussed beautifully in a Twende Mbele webinar featuring Candice Morkel, Mjiba Frehiwot, and Mokgophana Ramasobana.[5]
Taken together, Blue Marble Evaluation, post-normal evaluation, equitable evaluation, and decolonizing evaluation can guide evaluation towards the radical reimagining required by the challenges of evaluating the SDGs and COVID-19 recovery well. All four of these trends also highlight the importance of critical and evaluative thinking.
Blue Marble Evaluation, post-normal evaluation, equitable evaluation, and decolonizing evaluation can guide evaluation towards the radical reimagining required by the challenges of evaluating the SDGs and COVID-19 recovery well.
Evaluative thinking for transformative evaluation
What is evaluative thinking? It must be the thinking we (or any one does) while evaluating, you might say. In a sense, that is true, but in recent years, a number of evaluation scholars and practitioners have taken a deeper dive into this elusive concept. According to Patton,
“Evaluation is an activity. Evaluative thinking is a way of doing business. This distinction is critical. It derives from studies of evaluation use. Evaluation is more useful—and actually used—when the programme and organizational culture manifests evaluative thinking.”[6]
My colleagues and I have defined it as follows, “Evaluative thinking is critical thinking applied in the context of evaluation, motivated by an attitude of inquisitiveness and a belief in the value of evidence, that involves identifying assumptions, posing thoughtful questions, pursuing deeper understanding through reflection and perspective taking, and informing decisions in preparation for action.” (p. 384)[7]
Expanding the horizon of what worldviews frame our conceptualizations of evaluative thinking and reasoning, Nan Wehipeihana and Kate McKegg also consider the ethical imperative of acknowledging and upholding the place of indigenous knowledge systems in evaluative thinking.[8]
Evaluative thinking is both a fundamental philosophical foundation of evaluation and a way to develop evaluation capacity of individuals and organizations—to unleash the power of inquiry. It is similar to critical thinking and reflective practice, but also distinct, in that it centers the four-step logic of evaluation to make value judgments.
Evaluative thinking is both a fundamental philosophical foundation of evaluation and a way to develop evaluation capacity of individuals and organizations—to unleash the power of inquiry.
Here are some ways evaluative thinking relates to transforming evaluation to evaluate transformation:
Evaluative thinking democratizes and decentralizes evaluative inquiry. Therefore, sensitive to power dynamics, citizens from all walks of life can find their place in collecting, analyzing, and using data.
Evaluative thinking taps into practical wisdom and a plurality of ways of knowing and reasoning. Due to the messiness of “wicked problems,” practitioners don’t just apply technical solutions to technical problems. We are not robots. Rather, we engage in “reflection-in-action, a kind of ongoing experimentation, as a means to finding a viable solution to such problems” leading to “a particular kind of craft knowledge (or the wisdom of practice).”[9]
Evaluative thinking is systems and equity thinking. By constantly identifying assumptions, taking multiple perspectives, and exploring relationships, evaluative thinking helps do the systems-oriented ‘glocal’ work BME requires, including the attention to power, positionality, and privilege that equitable evaluation necessitates.
Evaluative thinking balances intuition and rationality. In a time where we must re-center values and facts in the face of the dangerous erosion of public discourse for good governance, evaluative thinking admits a plurality of ways of knowing while maintaining a critical eye on the credibility of claims and evidence.
In these four ways, among others, evaluative thinking has the potential to effect functioning cultures of transformed evaluation needed to evaluate the SDGs well for the betterment of all.
[1] The current geological age, viewed as the period during which human activity has been the dominant influence on climate and the environment. https://www.nationalgeographic.org/encyclopedia/anthropocene/.
[3] Schwandt, T. A. (2019). Post-normal evaluation? Evaluation, 25(3), 317-329.
[7] Buckley, J., Archibald, T., Hargraves, M., & Trochim, W. M. (2015). Defining and teaching evaluative thinking: Insights from research on critical thinking. American Journal of Evaluation, 36(3), 375-388. doi:10.1177/1098214015581706
[8] Wehipeihana, N., & McKegg, K. (2018). Values and culture in evaluative thinking: Insights from Aotearoa New Zealand. In A. T. Vo &T. Archibald (Eds.), Evaluative Thinking. New Directions for Evaluation. 158, 93–107.
[9] Schwandt, T. (2015). Evaluation foundations revisited: Cultivating a life of the mind for practice. Stanford University Press.
Thomas Archibald is an Associate Professor and Extension Specialist in the Department of Agricultural, Leadership, and Community Education at Virginia Tech, where he also directs the Feed the Future Senegal Youth in Agriculture project. Thomas Archibald is winner of the American Evaluation Association Marcia Guttentag Promising New Evaluator Award, he serves on the Board of Directors of the Eastern Evaluation Research Society and is an Associate Editor of Evaluation and Program Planning. He received his PhD from Cornell University in 2013. Follow Thomas on Twitter and LinkedIn.
