93 - Elevating Methodological and Ethical Standards in Monitoring

Abstract Information: In the domain of evaluation, well-established frameworks exist that provide methodological and ethical standards. However, the practice of monitoring, integral to understanding and improving program implementation and outcomes, has not been subjected to the same level of scrutiny and standardization. Third-party monitoring, a critical component for assessing program performance in conflict and crisis contexts, particularly within humanitarian assistance, often requires rapid data production and lacks standardized procedures. Reports of monitoring data often lack detailed methodology descriptions, including sampling, participant recruitment and selection procedures, and data analysis strategies. The common rationale for this disparity is the urgency of quick data collection, especially in third-party monitoring, a speed that evaluations are perceived not to achieve.

Traditionally, monitoring's primary role has been to verify the alignment of program activities and early results to planned objectives. Increasingly, however, funders are requesting that monitoring data capture longer term outcomes, which often includes the collection of beneficiary and other stakeholder feedback and perceptions. This trend necessitates a reevaluation of the ethical rigor of monitoring practices. For instance, obtaining consent during beneficiary interviews and ensuring the representativeness and appropriate selection of informants must be carefully considered.

In this think tank, we aim to challenge the perception that monitoring cannot or does not need to incorporate methodological and ethical rigor due to its inherent mandate. We aim to explore effective strategies to integrate stringent methodological and ethical standards into monitoring practices while still ensuring the timeliness of data collection for quick program adaptation. This session will engage participants in critically examining the current state of monitoring practices and identifying potential gaps in methodological and ethical standards, drawing upon evaluation standards. Attendees will delve into these gaps' implications for program accountability, stakeholder risk, and the quality of monitoring outcomes. They will work together to identify the areas of rigor from evaluation that can be applied to monitoring and share examples of where this has been done.

The facilitators will provide an overview of the issue, outlining the historical context of monitoring practices, the evolution of its functions especially in the context of conflict and crisis, and the existing disparity in standards compared to evaluation. They will share insights from their recent leadership roles in third-party monitoring projects and on institutional review boards (IRBs), emphasizing how they successfully integrated methodological rigor and upheld human subject protection standards. Participants will then be divided into small groups to explore specific aspects of the issue. One group will address methodological challenges in monitoring, while the other will discuss the integration of ethical standards. Post-discussion, the groups will regroup to share recommendations, followed by a facilitated session to synthesize insights and develop strategies for improving monitoring's rigor and ethics.

By the session's end, participants will have collaboratively developed a deep understanding of aligning monitoring with the evaluation's rigorous standards. The session aims to culminate in concrete recommendations that can guide practitioners in strengthening the methodological and ethical foundations of monitoring, thereby enhancing its credibility in the field of evaluation.