The relevance of typology development to evaluation: a whole person approach to coding

Abstract Information: This demonstration introduces a qualitative analytic approach to developing a typology of participant experience in program and/or policy evaluation. This session begins by introducing the core differences between traditional approaches to qualitative analysis and 'ideal type' analysis for evaluation research. Typical thematic analysis involves splitting data into small, discrete, coded parts; and then analyzing code distributions and frequency to identify patterns across participants. While such approaches can be useful in determining what might be working most and/or least effectively within a program or policy, this approach risks losing the most salient aspects of an individual participant’s experience. In ideal type analysis, the evaluator reviews data associated with each participant to prepare a unique case summary that reflects what the participant emphasized as the most significant or salient aspects of their experience. The evaluator then systematically compares these case summaries to determine categories based on shared characteristics of participant's experience (Stapley et al., 2022). The resulting set of categories forms a typology of participant experience. After introducing the core elements of ideal type analysis and typology development, this demonstration will draw on actual case study data from a policy evaluation to walk participants through utilizing this method. The purpose of the evaluation was to understand the effects of an innovative housing policy on resident well-being. The policy, implemented by the City of Portland, was the first of its kind in the nation to address past displacement by recreating access to affordable housing in a gentrifying, historically Black neighborhood. The City funded the construction of seven multi-unit rental buildings and created a preference policy to prioritize access to residents with historic ties to the neighborhood. Evaluators collected interview data (N: 61) in this convergent mixed method (quant = QUAL) longitudinal study. Ultimately, evaluators used an ideal type analysis to create a typology of resident experiences with the policy: those who were now “stable”, those who saw their housing as a “stepping-stone”, those who were “dissatisfied”, and those who were “insecure” in their housing. This analysis contributed critical insight into how the housing is being experienced, who it is serving well, and in what ways it is falling short of its intended goals. In the demonstration, participants will review interview excerpts from this case, view corresponding case summaries, and see how these summaries were used to determine distinct categories of resident experience. Participants will gain an understanding of the contributions of ideal-type analysis to evaluation research, learn the methodological approach to creating holistic case summaries and the steps involved in creating a typology, and consider applications to their own evaluation studies.

Stapley, E., O’Keeffe, S., & Midgley, N. (2022). Developing typologies in qualitative research: The use of ideal-type analysis. International Journal of Qualitative Methods, 21, 16094069221100633.