44 - Transforming STEM Outcomes for Women: Findings from a 10-year follow-up study of the FIRST® Robotics Program

Abstract Information: Over the past two decades, educators and policymakers have expressed concerns over the low levels of math and science achievement among U.S. students and their gradual decline moving into STEM careers in which women are especially underrepresented. This one of a kind 10-year post program participation comparison group evaluation investigates the role that an after school hands-on robotics program, FIRST®, can play in promoting STEM for young women. FIRST® (For Inspiration and Recognition of Science and Technology) operates after-school robotics programs for young people ages 4-18. The mission of FIRSTis to provide life-changing robotics programs that give young people the skills, confidence, and resilience to build a better world.
Study Goals: The mixed methods evaluation examines the impacts of FIRST on all participants' interests in STEM and their likelihood to pursue STEM in their academic and professional careers. Expanding on the consistently higher impacts of FIRST for young women found in the 10 years of the post-program participation survey, an interview study with 42 FIRST female alumni was added in the final year to include their voices reflecting on the especially strong program impact for women.

Methods: Longitudinal survey data collected in annual follow-up surveys of intervention and a matched comparison groups frame this analysis. Response rates have been consistently high, with 72% responding in year 10. In-depth interview data with a diverse sample of 42 FIRST female participants were collected during the final year to better understand why the impact of FIRST is especially strong for women. Interview participants differed by race/ethnicity, family economic positions at program participation, and locations (urban, suburban, and rural). 
Measures: The quantitative analyses framing the interview study uses validated scales that measure interest in STEM and STEM careers, involvement in STEM activities, STEM identity, and understanding STEM, and college course taking and majors. Control variables include race/ethnicity, family income at baseline, parental support for STEM, and geography of participants schools (urban, suburban, rural). Qualitative interview data with FIRST female participants focused on post program participation topics such as STEM coursework in high school and college, the fields females were pursuing in their current careers and why, their experiences in FIRST, and how they believed FIRST influenced their post-FIRST academic and work experiences.
Preliminary

Results: Analyses of the 10^th year survey data reveal that FIRST college students reported significantly higher rates of STEM interest, attitudes, and college level behaviors than comparison students. These findings are strongest for female comparisons. Preliminary analysis of the qualitative data underscore these impacts for women, who discussed how they got interested in joining FIRST and how their FIRST experience impacted their college majors. They also shared the challenges women experience when participating in FIRST and working in STEM fields, and how they persisted in STEM. 
This multi-method evaluation provides insights for longitudinal evaluation approaches and implication for the role of after-school STEM programs for young women’s decision-making regarding their STEM college and career choices. The evaluation also underscores the importance of including program participants' voices in evaluations.