1041 - Letting the Data Speak: Effectively integrating and synthesizing your data to tell the story

Abstract Information: Over the years, we have developed an approach, the Data Analysis, Integration, and Synthesis (DAIS) process, that efficiently and rigorously integrates and interprets data, aligning with the 2024 conference theme of fostering innovative practices in evaluation. This participatory approach allows us to integrate complex data streams, often collected and analyzed by multiple evaluators ensuring that diverse data are brought together to tell a coherent story. The DAIS process is interactive and iterative. During a DAIS process, we display analyzed data visually to the evaluation team (using cards or sticky notes), and we work together to integrate it into initial findings. We have found that this process can overcome the challenge of moving from analysis to report writing and can help lead to a well-structured report with strong findings, a clear narrative reflected in the conclusions, and recommendations grounded in data.

In this workshop, we present the DAIS process and invite the participants to practice each step, providing them with an opportunity to experience participatory strategies that enable effective data synthesis. We begin by presenting the approach with a brief lecture and discussion on organizing a DAIS workshop -- What is it? What are the benefits? Who needs to be there? How will it be formatted? We will explain how to develop data analysis summaries, which comprise early emerging theme statements with supporting evidence and data sources. These data analysis summaries are the key inputs to a DAIS.



We will present a mock study with four key evaluation questions and data. Participants work in small groups to practice developing emerging theme statements for each question. We will then practice the workshop steps, focusing mostly on how an evaluation team can affinity map emerging themes into higher-level findings during a DAIS workshop. We will discuss the difference between findings, conclusions, and recommendations and practice developing strong conclusions and recommendations from the emerging findings. Finally, we will discuss good practices and strategies to move from the DAIS to drafting a cohesive narrative in an evaluation report. Thus, through experiential learning, participants will experience how DAIS processes produce traceable, contextualized and supportable evidence-based findings and conclusions and recommendations appropriate for the intended audience, with a confident team standing behind the story the data tell. Post-workshop, attendees will receive access to digital resources and tools to implement DAIS in their evaluations, ensuring continuous learning and application of the workshop insights.

Relevance Statement: The DAIS workshop addresses a critical gap in evaluation practice – the transition from data collection to impactful reporting through effective integration and synthesis of diverse data streams. Often in evaluation, there is immense focus on data collection without fully considering the crucial analysis and synthesis part of the process. All evaluators have probably, at one time or other, found themselves with a mountain of evidence from various data streams, and struggle to figure out how to tell a strong story in an evaluation report. This can especially be an issue if multiple analysts and report writers are on the team, often in different locations. Some might also have concerns about bias if checks are not established along the way, or if the team moves from data collection to writing without fully synthesizing the data. The consequence can be a poorly written evaluation report that is not fully informed by the evidence. This workshop introduces the DAIS process, a method we have refined over the years to enhance evaluators’ ability to construct narratives that are rigorously grounded in evidence. The DAIS process gives evaluation teams a structured and deeply participatory approach to pause after data collection and analysis, and work together in a workshop setting to integrate evidence and construct a meaningful story – including findings, conclusions, and recommendations- directly linked to the evidence.

In this workshop, participants will learn about the DAIS process and have an opportunity to practice each step of the process using mock data. They will learn to collaboratively analyze and integrate data, enhancing their ability to develop nuanced and evidence-based findings. They will gain skills in synthesizing diverse data streams into coherent narratives, ensuring their evaluations are directly linked to the underlying evidence. This learning experience will empower evaluators to confidently derive and support their conclusions and recommendations, enriching their evaluation practice with a method that emphasizes teamwork and data-driven insights. The workshop aligns with the 2024 conference theme by emphasizing innovation in evaluation practices and encouraging evaluators to adopt collaborative and participatory approaches to the analysis process. By participating in this workshop, evaluators will gain valuable skills that reinforce quality standards in evaluation theory and practice, enhancing their ability to produce evaluations that truly inform and influence decision-making. The workshop format offers a platform for both new and seasoned evaluators to share knowledge, learn from each other, and contribute to the evolution of evaluation practices, promoting diversity, sustainability, and the collective advancement of the field.