258 - Developing a standard-but-flexible operating procedure for program staff conducting evaluations

Abstract Information: Internal evaluators are often stretched thin by a multitude of projects. Meanwhile, project staff often wish to contribute more to evaluations, but lack the necessary expertise. The Global Health & Clinical Skills (GHCS) department in the American Academy of Pediatrics sought to create a “standard operating procedure” that program staff could follow to conduct evaluations with limited, strategic support from a single internal evaluator. The challenge: GHCS’ projects are highly diverse, ranging from dozens to hundreds of participants, from Latin America to Africa, from monitoring well-evidenced interventions to experimenting with new innovations. To evaluate these programs, staff needed a standard operating procedure that is not too standard. In response, a “standard-flexible operating procedure” (SFOP) for measurement, evaluation and dissemination was developed. Interviews were conducted with staff and collaborators to determine current evaluation practices and strengths, aspirations for enhanced evaluation approaches, and opportunities for improvement. Three evaluation principles emerged on which to build the SFOP: utilization-focus (Patton 2003), collaboration and cultural responsiveness (Woodson 2021), and complexity awareness (Walton 2014). Utilizing a strengths-based approach to the SFOP, steps were outlined that each staff member could follow to complete an evaluation (e.g., create a theory of change, craft evaluation questions, collect data, disseminate results and iterate on program implementation, and more). Theory of change templates, a guide to budgeting an evaluation, and several other resources were created to systematize the approach across program staff. Customization options were also included to be adaptable to varied program needs. Implementation of the SFOP is currently ongoing on a project-by-project basis, engaging program staff in team-wide skill trainings as well as one-on-one consultation to adapt the SFOP to a given program. As a team, we are engaged in a collaborative learning and growth process that has resulted in evaluation tools and strategies that can be applied to diverse projects while ensuring that best practices are utilized. This poster will discuss the process of creating the SFOP and rolling it out to program staff, including specific examples of both SFOP elements and program staff engagement in the process.