163 - Evaluation of a Peer Mentoring Employee Development Program in Supplemental Instruction

Abstract Information: Supplemental Instruction (SI) is a peer led program which aims to improve students’ grades in higher education courses with historically high rates of D’s and F’s. Using an internationally applied program model, SI sessions employ active learning principles and are led by an undergraduate student (SI Leader) who has passed the course with an A. The SI Model has been widely evaluated and shows strong evidence of improving course performance. However, the process of training SI Leaders to facilitate the sessions is not manualized and varies between institutions. A large university in the Southeastern U.S created an SI Leader development program using a peer mentoring model to improve SI Leader facilitation skills through feedback from an experienced SI Leader (Peer Mentor). Mentors attend three sessions each semester and give standardized observation reports providing targeted feedback on active learning and peer-to-peer engagement within sessions. Initiated in the Fall of 2019, this peer mentoring program has received positive feedback from SI Leaders, however the evaluation presented here is the first systematic investigation of the impact of the peer mentoring program. Two emerging evaluators carried out this evaluation which started as a course project. We worked with SI Leadership to develop evaluation questions, data collection schedules, and dissemination plans. The purpose became to assess changes in and the associations between SI Leader session facilitation confidence and facilitation skills, Peer Mentor comfort with providing feedback, and dyadic relationship quality over the course of a semester then use that information to investigate the effectiveness of the Peer Mentor Model. The initial phase of the evaluation took place over the Fall 2023 semester. Despite a small population, recruitment was highly successful. The final samples for analysis were 35 participants (71.4%) at the pre-semester data collection and 27 participants (55.1%) for post-semester, with 23 participants completing both surveys. Taking a mixed-method approach, quantitative data were analyzed using a combination of descriptive mean comparisons, t-tests, and correlations and qualitative data were analyzed using thematic analysis. Data revealed general growth in Leader facilitation confidence and relationship quality over the semester. Further, relationship quality between an SI Leader and their Peer Mentor may have a greater impact on Leader confidence than characteristics of the Peer Mentor (i.e., comfort with providing feedback). Qualitative feedback indicated the program can aim to create stronger partnerships through continued clear and open feedback covering strengths and areas of improvement for SI Leaders. Even with promising results, we recommended a second semester of data collection as each analysis had a small number of observations due to the small population size and investigating pattern replication the next semester would add rigor to the results. The stakeholders were very satisfied with the evaluation process which parsimoniously met the evaluation goals and shared the desire to conduct a second semester of data collection which is underway in the Spring 2024 semester.