Harry Potter and the AI Evaluation: Insights from Incorporating ChatGPT and Hogwarts into an Undergraduate Program Evaluation Course

Abstract Information: In this session, we present a refreshing and fun approach to teaching program evaluation, particularly through the engaging universe of Harry Potter’s Hogwarts School of Witchcraft and Wizardry (Rowling, 1998; HSWW). This undergraduate course, designed to impart foundational knowledge in evaluative thinking and credible evidence, used the HSWW as a unique case study for creating an evaluation plan. Through this lens, students explored diverse evaluation scenarios, enhancing their understanding and skills in a creative, inclusive environment. A key feature of the class project was the integration of AI technology, ChatGPT, to explore the HSWW, and really the Harry Potter universe. Through the integration of this technology into hands-on activities in-class, students were able to conduct mock interviews with the HSWW stakeholders—Snape, Dumbledore, Hagrid, Moaning Myrtle, etc.—understanding their diverse viewpoints and perspectives. We believe that this use of a fictional universe and incorporation of AI into the classroom fostered a deeper comprehension of and connection with the intricacies involved in program evaluation and the art and practice of evaluative thinking (Tovey & Archibald, 2023). In this session, we will explore how integrating case-centered teaching (e.g., Kallemeyn et al., 2022; Montrosse-Moorhead et al., 2022) with elements of popular culture and artificial intelligence can enrich learning experiences and promote the inclusion of diverse perspectives in program evaluation. The incorporation of cases in evaluation education is essential. Using cases in teaching and learning equips future professionals, academics, and consumers of evaluation with visceral experiences that enhance their understanding beyond abstract concepts, highlighting the fundamental role of context in effective evaluation practice (Linfield & Tovey, 2022; Tovey & Greene, 2022). Using the HSWW as an intervention offers the opportunity to explore program evaluation from multiple layers and perspectives, facilitating deeper learning in students—even if they think Harry Potter is “a bit millennial” (undergraduate student in class, Spring 2024). In this session, we will share our approach, prompts, insights, lessons learned, and reflections from engaging students in a whole class project dedicated to creating an evaluation plan for the HSWW.