Designing Surveys in the Shadow of Violence: Balancing Rigor, Safety, and Utility in Conflict-Affected Contexts

Amidst the complexities of ongoing humanitarian emergencies, program evaluation plays a crucial role in ensuring programs effectively serve vulnerable populations. Effective program evaluation in conflict-affected settings often necessitates supplementing traditional survey methods with strategies that address the unique challenges of these contexts. This workshop delves into the challenges and ethical considerations involved in collecting data from populations experiencing trauma, displacement, and limited resources. The central question this workshop will address is: How can we ethically and effectively adapt survey methods to gather reliable data from vulnerable populations affected by conflict, considering the challenges and limitations of these contexts?

Drawing on a case study of an education program serving participants in a conflict-affected region, primarily targeting women and girls, we explore the practical dimensions of designing safe, ethical, and impactful surveys. This case study allows us to examine the amplification and empowerment of these historically underrepresented voices within the context of program evaluation. We emphasize participant engagement throughout the data collection process. Program participants receive the findings in aggregate form, allowing them to see the collective voice of their community reflected in the results. Additionally, participants have the opportunity to share feedback on the data collection methods themselves, ensuring the approach is culturally sensitive and minimizes any potential burden on respondents. We share practical solutions for addressing challenges such as internet access limitations and ensuring respondent safety. We discuss the use of technology, such as internet safety tools, and the ethical dilemmas of intervention when participant responses suggest danger. Participants will engage in interactive exercises, including practicing decision-making around ethical survey design, specific item design and analyzing de-identified survey responses to determine how participants actually respond to specific survey items.

The workshop promotes critical reflection of monitoring and evaluation in conflict-affected contexts, considering both the responsibility to protect participants and the value of generating actionable data. In this workshop, participants will: •Focus on specific challenges such as: o Limited internet access: Developing data collection methods within the context of limited, unpredictable internet access and considering their feasibility and ethical implications. o Ensuring participant safety: Discussing strategies like informed consent procedures, anonymity measures, and support mechanisms for potential distress. •Brainstorm practical solutions tailored to the specific challenge. •Consider ethical considerations throughout the solution-finding process. This workshop is relevant to evaluation professionals working in conflict-affected areas and researchers interested in ethical survey design within vulnerable populations. It contributes to the field by: •Highlighting the specific challenges of conducting surveys in violent conflict zones, particularly for vulnerable populations like women and girls. •Sharing practical strategies for ensuring participant safety and empowerment through ethical data collection, including incorporating participant feedback into instrument and program design through methods like applicant surveys. •Offering a space for dialogue on the philosophical complexities of M&E in conflict, emphasizing the importance of amplifying the voices of the most affected communities, and actively incorporating their feedback into instrument and program design. •Empowering participants, particularly women and girls, to design surveys that are both rigorous and sensitive to the context.