118 - Empowering project stakeholders through engagement in action-oriented participatory methods: Experiences from multi-country project evaluations

Abstract Information:

Introduction: The evaluators often design evaluation projects proposing some traditional data collection methods such as survey, interview that are often blamed for the top-down approach. The main drawback of applying traditional methods in evaluation is that it losses the participation’s active participation and collaboration of stakeholders in the evaluation process. Orbis International, a humanitarian nonprofit working in Asia, Africa and Latin America and Caribbean since 1982 to reduce avoidable blindness through capacity building, eye care system strengthening, and long-term country programs. Annually Orbis conduct around 20 project evaluations using some traditional data collection methods that lack the active involvement and participation of proejct stakeholders including implementing partners and beneficiaries as well. To empower the stakeholders through their involvement into evaluation process, Orbis has recently applied some action-orientated participatory methods in different countries and tested the effectiveness of these methods.

Methods: Orbis international’s Global Monitoring and Evaluation (MEL) team has been applying various participatory methods of its evaluation in different program countries since 2019. The team first applied the Theory of Action (TOA) methods in Peru involving project partners, government and non-government organizations, and beneficiaries. The evaluation team also used participatory workshop, Flow Diagram, Venn/Network Diagram to understand more about project management, partnerships, and collaboration. In 2021, during the pandemic, Orbis Global MEL team used TOA method along with other participatory methods such as matrix ranking, participatory group discussion and validation workshops with various stakeholders to evaluate project in Zambia. In 2023, TOA and other participatory methods were used to evaluate one of the Orbis Mongolia projects. Lastly, in 2023-24, we applied some participatory methods in two evaluation projects in Bangladesh that include pictorial diary, health facility and resource mapping, matrix ranking and TOA to engage project stakeholders in design, evaluation and findings sharing phases.

Results: Orbis’s Global MEL team wanted to understand the effectiveness of the participatory approaches that they used in the project evaluations in different countries. It was observed that the stakeholders who were engaged in these methods appreciated the evaluation team to involve them in the evaluation process. Most of the participants did feel connected and self-esteemed as they thought they first time were able to discuss about project performance freely and openly. Unlike other structured methods, the participatory methods ensured flexibility to adjust of the tools even during data collection phase. One of the major strengths of these methods was participants were able to challenge any ideas or opinions and ask for clarification on the spots. Other strengths of these participatory methods include stakeholders were involved in analyzing and dissemination of findings, unlike the bottom-up approaches. The approaches also ensured shared decision making, empowering people to take decisions, prompting participants to share their experiences. It was also observed that visualization helped participants to understand the discussion points and could contribute accordingly.

Conclusion: Participatory approaches have been very effective and relevant to the context of Orbis project evaluation process. It created enthusiasm and self-esteem among the project stakeholders, and it ensured shared learning and taking actions collaboratively.