Amplifying the experiences of organisational actors enacting evaluation policy for social equity programmes: Examining the influences of internal evaluation on practice and strategy decision-making

Abstract Information: Evaluation use is one of the most researched areas of evaluation. However, this research is mainly descriptive, and typically neglects exploring how actors within organizations respond to and use evaluation when it has been imposed on them in the context of an evaluation system. It is often assumed that evaluation should be used instrumentally to inform decision-making. But evaluation can have influences that stretch far beyond intentions for positive instrumental use. For example, we know that systems of evaluation can end up shaping the organizational policies, practice, and behaviours they originally intended to assess. In addition, evaluation can become captured by organizations which can affect the extent to which it can be used to speak truth to power. These broader effects of evaluation systems have prompted calls for more critical reflection on evaluation. Learning from the experiences of organizational actors responsible for enacting policies that drive social equity agendas in addition to navigating systems of evaluation is crucial if we are to understand the myriad influences of evaluation. To add value to existing research on evaluation, this paper presents a study which utilized Q-methodology and semi-structured interviews to explore how a diversity of staff from administrators to vice-chancellors, based within substantively different English higher education providers, experience evaluation and its influence on their widening participation practice and strategy decision-making intending to advance social equity. Findings from this study highlight how evaluation in widening participation has become systematized. Accordingly, evaluation practices have become embedded into the organizational structures of higher education providers, affecting how staff responsible for enacting widening participation policy evaluate their programmes. At an organizational level, evaluation is largely being used to legitimize the activities of widening participation departments, indicating evaluation capture. At an individual level evaluation is contributing to the increased evaluative thinking, and attitudinal and behavioural changes of staff. Of note, the findings emphasize the role of social, professional, and scientific values of organizational actors in informing how evaluation is practiced and how it influences practice and strategy decision-making. Consequently, this study calls for future research to continue to explore the way multiple and intersecting values affect how internal evaluation conducted in the context of an evaluation system either contributes to or hinders learning to advance social equity through practice and organizational strategy.