40 - Use-focused Evaluation: The Case of Developmental Evaluation in Ethiopia's Complex Operating Environment

When evaluations are overly complex, expensive, and rigid, they are less useful to humanitarian and development stakeholders. Managers cannot get real-time information to make decisions; program staff view participating in an evaluation as adding more to their busy schedule; and the rapidly changing operating environment in humanitarian work makes setting and measuring results extremely difficult. To acknowledge and address these challenges, Headlight Consulting Services, LLC (Headlight) implemented a Developmental Evaluation (DE) in 2020 with USAID/Ethiopia’s Disaster Risk Management team and the Strategic Advisory Group for Emergencies, in the face of multiple and concurrent shocks, including COVID-19, desert locust, drought, flooding, and one of the most bloody wars in Ethiopia’s history. The project supported the client team in being data-driven and flexible in the changing context. As per the evidence from the Developmental Evaluation evidence and annual reports, several USAID partners had opportunities to reflect, learn, and adapt with a better understanding of what worked and what did not. The project built local capacities, from hiring all local Developmental Evaluation Leads to providing tools, clear SOPs, and capacity-building opportunities. Despite such a challenging operating environment, the flexible nature of the Developmental Evaluation approach, coupled with the strong collaboration between USAID/Ethiopia and the project team, led to increased buy-in to the DE approach. In 2023, USAID/Ethiopia increased the scope of the Developmental Evaluation from the Disaster Risk Management Team in the Office of Humanitarian Assistance to USAID/Ethiopia and USAID/Djibouti mission-wide. While acknowledging the strong buy-in, it is evident that Developmental Evaluation is not a one-size-fits-all approach. The Developmental Evaluators continued to wrestle with the ever-changing context and the client's need to provide use-focused support by generating evidence and actionable recommendations, facilitating strategy and activity design, and, most importantly, supporting continuous and intentional learning and adaptation. This roundtable discussion will highlight the successes, challenges, and learnings from years of practicing this non-traditional evaluation approach in one of the most complex contexts for humanitarians, development actors, and evaluators alike. The discussion will share examples of evaluative efforts conducted under the DE, the Developmental Evaluators' embedded support model, how we used real-time feedback, and approaches for facilitating adoption of CLA practices. The session will include a robust conversation on providing evidence and supporting adaptations in a complex environment, to improve development outcomes.