53 - Strengthening Staff Evaluation Capacity to Enable Continuous Improvement: The Experience of Education Development Center and FEMA National Exercise Division

Abstract Information: Monitoring, evaluation, and learning (MEL) activities are critical to enabling the continuous improvement of policies and interventions. Without effective MEL practices, organizations will struggle to evaluate, learn from, and adapt their policies and interventions. Yet many organizations find it difficult to recruit, train, and resource MEL staff to perform MEL activities well. Many also struggle to ensure programmatic staff understand MEL activities well enough to support MEL staff and utilize their work to improve policies and interventions. In this Birds of a Feather Presentation, MEL staff from Education Development Center (EDC) and the Federal Emergency Management Agency (FEMA) National Exercise Division (NED) propose discussing with session attendees how they have worked to strengthen the MEL capacity of their staff and partners. Presenters will bring examples from EDC’s international workforce development programs, as well as FEMA NED’s work in domestic emergency management.  Presenters will also bring questions to guide the discussion and hear from fellow evaluators, such as: • How do you get programmatic staff to understand the importance of MEL activities? • How do you get programmatic and leadership staff to dedicate time and resources to MEL activities? • How do you build the capacity of MEL staff? • How do you facilitate exchanges between MEL and non-MEL staff to improve programming? The goal of the session is to explore together and share best practices and potential issues associated with MEL capacity-strengthening in both domestic and international contexts.

Relevance Statement: This Birds of a Feather Presentation relates to the 2024 AEA conference theme “Amplifying and Empowering Voices in Evaluation” because MEL capacity-building is a prime example of how organizations can empower MEL and programmatic teams by equipping them with the skills needed to continuously improve strategies and interventions. This session seeks to initiate a conversation on the strategies favored by fellow practitioners to build MEL capacity, challenges they faced and solutions they identified.
For example, EDC’s international projects empower local staff and partners by equipping them with basic MEL skills so that they understand and implement evaluation and learning strategies with minimal support from headquarters MEL staff. Capacity strengthening also empowers local staff to adapt international MEL standards to their own context and helps strengthen the sustainability of MEL practices beyond the project. On the other hand, capacity-building opportunities bring programmatic teams on the same page as MEL staff. Empowering the voice of program staff who have an intimate knowledge of interventions and policies ensures that MEL systems fit with the program goals and learning needs.
Similarly, FEMA NED is ensuring all staff speak the same language when it comes to conducting and learning from evaluations. This approach empowers both programmatic staff, who now have better understanding of the purpose and uses of evaluations, and evaluation staff, who can now better engage the programmatic staff on the development of evaluations that will help NED learn and improve. To implement this, the organization developed and led custom training on MEL concepts, organized individual functional group sessions to discuss the application of MEL to disparate functions, and validated understanding through individual section engagements. Establishing a common framework and terminology around program evaluation not only helps to ensure adoption, but also establishes a culture of improvement most necessary in the public sector for real program impact. Additionally, orienting public sector culture toward a framework that better aligns with the international development sector will provide a common lens through which to understand challenges and advancements of interventions in both spheres, while also contributing to the understanding of interconnected interventions in a global sense.  
To help generate conversation, EDC will share concrete examples of how it has worked toward linking MEL and project staff work to strengthen projects in Djibouti, Liberia, and Senegal. FEMA will also share examples from its domestic results framework in establishing the connection between emergency management exercises and community preparedness by linking programs across the division through a common set of indicators.
Participants are invited to share their own examples, and consider questions such as: What are other organizations doing to empower MEL staff? What are they doing to amplify the voice of programmatic and leadership in MEL activities? Or how do they amplify the voices of MEL staff in learning and adaptive management activities?