From Researching “On” to Researching “With”: Insights from Collaborative Reflective Practice on DEI in Evaluation

Abstract Information: Since 2018, our research has focused on how United States (US) National Science Foundation (NSF)-funded Advanced Technological Education (ATE) projects define and measure Diversity, Equity, and Inclusion (DEI) to enhance broadening participation in STEM goals. The US NSF ATE program focuses on educating technicians for technology fields through partnerships with two-year academic institutions, secondary schools, and industry (NSF, 2018). We discovered that ATE evaluators gathered the most evidence related to diversity and less for equity and inclusion (Boyce et al., 2022). Feedback from the ATE community highlighted that they were less interested in being “researched on” and more interested in being “researched with.” This led us to adopt a participatory case study approach in this phase of our research to explore how to develop and implement an action plan to measure and engage DEI within the evaluation.

In this session, we will be discussing the shift in our methodological approach. Since 2022, we have been collaborating with three diverse ATE projects (project principal investigator and external evaluators) across the U.S., utilizing the DATA-DATA model for reflective practice and action research as our methodological foundation (Peters, 2009; Tovey & Skolits, 2022; Tovey & Archibald, 2023). This approach emphasized describing problems, analyzing their causes, theorizing solutions, and planning action, with a focus on reflective practice to prevent premature solution-seeking.

Our methodology involved intensive qualitative data collection through written reflections, project meetings, and interviews, culminating in an action plan. Through facilitated reflective practice, project teams engaged in generative dialogue to understand DEI within their projects and evaluations. Our findings underscore the necessity of a guiding framework, the value of active DEI engagement by facilitators, and the influence of context, relationships, evaluator values, client preferences, and funder commitments on DEI integration.

Importantly, our research demonstrated how reflective practice and a structured framework can empower project teams to develop effective resolutions and action plans. This session highlights how our methodology enabled the ‘doing’ of DEI in evaluation and provided space and time for synergy between the PI and the evaluator regarding what the DEI-related challenges/issues were and what they could do about it.