Challenges in Assessing Efficiency in Development Evaluation: Insights from Evaluation Practices in Korea’s Development Cooperation

Abstract Information: Efficiency serves as a fundamental aspect of program evaluation, regarded as one of the six key criteria in development cooperation evaluation guidelines. However, it appears that the assessment of efficiency within development projects has been overlooked, if not neglected. The purpose of this study is to identify the challenges associated with evaluating efficiency of development cooperation projects, drawing insights from current evaluation practices in Korea. The study will first introduce the concept and methodologies of efficiency assessment within program evaluation theories and expected results in ideal scenarios. Then it will present an analysis of the current evaluation practices of Korea’s development cooperation projects, by reviewing 188 evaluations published between 2020 and 2022 by the Korea International Cooperation Agency (KOICA), the foremost official development aid agency in Korea. These evaluations will be scrutinized to address three primary questions: (1) What evaluation questions are posed under the “efficiency” criterion?; (2) What methods are employed to address these questions?; and (2) Do the findings and judgments align with the purpose and expectations of assessing efficiency? The preliminary analysis reveals that the majority of evaluations employ standardized efficiency assessment questions outlined in the guidelines. Some of these questions, however, may not always be relevant to the efficiency dimension within specific project contexts. Moreover, methodologies employed are often lacking in rigor, and evidence presented tends to be weak or often irrelevant to the questions. Additionally, there appears to be a discrepancy in the understanding of “efficiency,” with some evaluations using concepts divergent from those in the evaluation literature. A worrisome consequence observed is that these practices are replicated by new evaluators, as they often refer to the published evaluation reports. To identify the root causes of these observations, the study will conduct in-depth interviews with evaluators and evaluation managers within the agency to explore the underlying factors influencing efficiency assessment practices. Assessment of efficiency in program evaluation requires supporting information and knowledge drawn from other aspects of evaluation including program's needs, implementation process, and outcomes. Based on this argument, the discussion will explore possible ways to enhance efficiency assessment in development evaluation, paying particular attention to the factors influencing the current practices and unintended consequences of mechanically replicating past practices.