Theoretical and Philosophical Justification for Measurement of Subjective Experiences in Evaluation with Applied Examples

Abstract Information: Relational developmental systems metatheory postulates that human development occurs in interactions between individuals and their environments, rather than solely within individuals or solely due to environmental influence (Overton, 2013). A key element of the individual-environment relationship is the individual’s subjective interpretation of the environment. When seeking to evaluate the effect of programs or institutions, we may identify differences in participation or outcomes that are not clearly explainable by differences in dosage or reach. Measuring the subjective experiences of program participants, such as satisfaction, sense of belonging, or meaningfulness, can elucidate explanatory mechanisms for why different people have different outcomes in ways that simple demographics cannot. The challenge then becomes how to measure the subjective in a precise way. Even though individual experiences can differ dramatically, precise measurement requires identifying the similarities between people at the level of constructs (e.g., meaningfulness) and the typical ways we communicate about them. Panelists will discuss the philosophical commitments involved in precisely measuring ontologically subjective experiences and present examples of such measurement in the context of a cultural institution. The panel chair will open the presentation with a brief description of relational developmental theory as a justification for the theoretical importance of measuring subjective experience and present a call to panelists to discuss the requirements and merits of doing so. The first panelist will present on the philosophical commitments required to make the ontologically subjective (i.e., internal experience) knowable in a way that is scientifically valid and reliable. The second panelist will provide an example of how this can be accomplished by describing the development of a scale designed to measure the construct of meaningfulness in museum visitors (funded by IMLS Museums Empowered, Measuring What Matters: Building Capacity to Evaluate Meaningfulness in the Visitor Experience: ME-249540-OMS-21, Principle Investigator: Ellen Roth). The third panelist will provide an example illustrating the importance of developing culturally responsive measures. Session attendees will then be invited to participate in a dialog about the implications for best practices in evaluation.