130 - Beyond Counting Numbers: Redefining Success in Broadening Participation in STEM Programs

Abstract Information: Federally funded STEM programs, including those supported by organizations like the Department of Defense (DoD), National Science Foundation (NSF), and NASA, are tasked with broadening participation in STEM education and careers, particularly among historically marginalized populations. Evaluators play a pivotal role in documenting this participation, often relying on quantitative indicators. However, a sole focus on numbers can overlook crucial aspects influencing program effectiveness and long-term impact. This session challenges the overreliance on quantitative metrics. Participation rates paint an incomplete picture. They fail to reveal the motivations behind participation, the effectiveness of program design for historically marginalized students, or the lived experiences of program participants. We advocate for an approach that integrates quantitative data with qualitative data to offer a richer understanding of program influence. Through an interactive presentation, will explore how evaluators, program implementers, and funders can transcend numerical metrics and adopt a holistic approach that aligns with the AEA's Guiding Principles for Evaluators. We will explore critical gaps in current practices and address them through presentation, discussion, and illustrative examples from real-world STEM programs. This session aims to unlock the full potential of evaluations of STEM programs – not just measuring program reach but assessing how effectively these programs achieve their goals of equity and inclusion.