Introduction
Evaluations are part of our everyday lives. And yet, so few know and implement program and project evaluations in a logical and meaningful way. This four-part blog series aims to expand our collective understanding on the definitions, kinds, and implementations of evaluation and evaluation research. This part focuses on what evaluation is and how it differs from research. Part 1 can be accessed by clicking here.
Evaluation vs. Research
My understanding of evaluation has been a journey. I've always understood that my internal evaluations are involved with everything from what to wear for the weather, to how to act in various situations, to how to accommodate change. As I have progressed through life, my evaluative techniques have become more complex and focused. During the early part of my career, I confronted evaluation through trial and error and came to understand evaluation as a kind of reflective process. Was it research or something else? I wasn't sure. Student evaluations, annual professional reviews, and program reviews helped me see ways to implement evaluation, but they did not help me clarify whether evaluation was research or not. When I started writing National Science Foundation (NSF) Improving Undergraduate STEM Education (IUSE) grants, I began to realize that evaluation was a field of study that shared much in common with research but was not research itself. If one conducted a research program, then one needed to conduct a detailed evaluation on that program to see if it worked in the ways the principle investigator (PI) hoped it would. I discovered that evaluation must be the umbrella that I had heard about for years which included concepts like "closing the loop", "theory of change" and "logic models".
It was not until this class that I understood some fine distinctions between evaluation and research. Russ-Eft and Preskill (2009) distinguish between the modes for evaluation and research. They state that the two modes of inquiry differ by:
- their purpose and scope;
- their audience and stakeholders, including those groups' questions, needs, and relative buy-in; and
- their communication, both during the process and at the end "in different ways and to different groups" (p. 8).
What then is evaluation?
In terms of defining evaluation, I really liked Russ-Eft & Preskill's (2009) definition: "Evaluation, therefore, collects data, which is turned into information that, when used, becomes knowledge at the individual level. If shared with others in the organization, that knowledge may then lead to organization-level learning" (p. 5). I think this definition simplifies evaluation, but hints at the iterative and personal nature of evaluation as well. I also think that this definition helps clarify some of the defining characteristics of formal evaluations. Evaluation is a "process of determining something about the merit, worth, or significance of an entity" (Scriven, 2013, p. 170). In order for evaluative judgements to be made fairly, evaluations must conduct systematic assessment of the evaluand compared with some standard to provide unbiased and accurate information (Weiss, 2013, p. 131).
Even though research and evaluation differ in many ways, integrating evaluation into research through practices like member-checking, getting buy-in from the group being analyzed, and communicating with stakeholders throughout the research process strengthens the overall research enterprise in dynamic and potentially transformative ways. Integrating research into evaluation by asking broader research questions and conducting evaluation research simultaneously with the evaluand's evaluation also strengthens overall evaluation activity in dynamic and potentially transformative ways.
The Potential of Evaluations
I envision regular, iterative, ongoing evaluations as part of a transformative process for higher education. I think we, the higher education stakeholders, need to regularly reflect and assess where our programs are and whether they are serving well our students as well as the faculty, staff, administration, and governmental entities. Participatory/collaboration evaluation, where the goal is to evaluate the needs of those closest to the program, as well as empowerment and culturally responsive evaluation, where typically oppressed groups are given a voice, particularly to those in power, seem like excellent evaluation methods to use to help begin that transformative process. These evaluation methods may even encourage more varied and in depth evaluation processes to explore the deep racial undercurrents and colonial practices used in much of education. I think these evaluation methods may allow us to better assess where we've been, where we are and where we are going in our higher education classes, programs, schools, and universities.
References:
Russ-Eft, D. & Preskill, H. (2009). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change (2nd ed). Basic Books.
Scriven, M. (2013). Conceptual revolutions in evaluation: Past, present, and future. In M.C. Alkin (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences (2nd ed., pp. 167-179). Sage Publications. https://doi.org/10.1177/1098214013479152
Weiss, C. H. (2013). Rooting for evaluation: Digging into beliefs. In M.C. Alkin (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences (2nd ed., pp. 130-143). Sage Publications. https://doi.org/10.1177/1098214013479152