Generative Artificial Intelligence (Gen AI): Guiding Innovative and Responsible Qualitative Data Analysis in Evaluation Practice

Abstract Information: Qualitative data analysis plays a pivotal role in evaluation, facilitating understanding of interventions, advancing social justice goals, and amplifying the voices of often marginalized participants. To analyze qualitative data, evaluators use diverse analytical strategies like thematic analysis, narrative analysis, and critical discourse analysis. While traditional methods are prevalent, generative artificial intelligence (Gen AI) tools such as ChatGPT and Co-Pilot offer exciting prospects for enhancing evaluators' analytic strategies. Gen AI's proficiency in processing natural language, conducting automatic coding, and generating descriptive responses, presents promising avenues for analyzing qualitative data, potentially offering novel perspectives (Morgan, 2023; Sabarre et al., 2023). However, using generative AI tools for qualitative data analysis raises various ethical and practical concerns (Azzam, 2023; Mason, 2023; Schlagwein & Willocks, 2023). Ethical issues encompass algorithmic biases, interpretive adequacy, data handling, privacy, hallucinations, and result trustworthiness—each impacting evaluation quality, including the representation of diverse voices, advancement of social justice, and credible findings (Reid, 2023). Practical challenges include evaluator competencies in using Gen AI tools (Mason, 2023); many lack the skills to prompt appropriate analytical outputs from Gen AI tools (Ferretti, 2023). Evaluators’ competencies to handle data inaccurate or ambiguous data further complicate the usage of Gen AI tools (Fui-Hoon Nah, 2023). Hence, our paper explores integrating Gen AI tools like ChatGPT for qualitative data analysis within evaluation, discussing their generative potential for supporting evaluative qualitative data analysis and the implications of Gen Al for responsible evaluation. We aim to contribute to the field of qualitative research by guiding innovative and responsible uses of Gen AI technology to deepen understandings of interventions, promote equity, and amplify voices.