79 - Negotiating tradeoffs between truth and justice in international development impact evaluation

Abstract Information: Evaluators engaged in international development impact studies confront a series of tradeoffs that influence their ability to achieve two crucial evaluation objectives: establishing a dependable cause-effect relationship (causal validity), and ensuring equity in knowledge production (epistemic justice). These decisions require the delicate balance of practical constraints, methodological rigor, and ethical considerations to achieve meaningful and equitable outcomes. This paper presents the findings of a qualitative dissertation research study employing narrative inquiry to understand how causal validity and epistemic justice manifest in international development impact evaluation. It explores elements of practice that might foster or impede epistemic justice or causal validity and investigates how evaluators negotiate tradeoffs between the two. By analyzing the real-world experiences of evaluators across diverse settings, we find that contextual challenges, selection bias, power imbalances, and limited access to resources pose obstacles to causal validity and epistemic justice. On the other hand, contextual sensitivity, inclusive research practices, continuous learning and improvement, and robust methodological approaches all bolster both epistemic justice and causal validity. However, certain factors such as established research paradigms and disciplinary norms, quality assurance and academic peer review, and methodological standardization promote causal validity while limiting epistemic justice. Conversely, a focus on community empowerment, attention to diverse perspectives, and qualitative and contextual understanding are considered to promote epistemic justice while potentially limiting causal validity. Overall, in addressing these constraints and navigating the inherent tradeoffs, evaluators aspiring to both epistemic justice and causal validity in their evaluation practice must prioritize contextual understanding, methodological flexibility, the ability to balance rigor with real world constraints, and multiple causal perspectives.