89 - How do you measure and evaluate city systems change? Conversations around multiple methods and lessons learned from a trauma-informed transformation initiative in Chicago.

Abstract Information: Evaluating systems change is no easy task, and evaluators both new and seasoned can be challenged with questions related to defining the system, measuring change, and where to even start. Yet, working upstream to better understand and address root causes of downstream outcomes remains critically important; collective efforts to change inequitable and racist systems supports changing social and structural determinants that undergird communities’ opportunities to thrive and be healthy.

Across the United States, one promising and advancing area of work around changing city systems has been that of establishing trauma-informed and healing centered cities, as the impact of trauma on individuals and communities has been increasingly recognized across public health institutions, social service agencies, educational systems, government, and community and youth advocacy groups. In Chicago, the Department of Public Health (CDPH) has long been on their own journey in advancing trauma-informed systems change, and partnered with Education Development Center (EDC) evaluators to help answer these critical questions – how should we measure and evaluate systems change, i.e. a trauma-informed transformation initiative in Chicago? In service of building CDPH’s knowledge and capacity, EDC conducted a mixed-methods landscape scan of city-wide trauma-informed systems change efforts to enhance community wellbeing, which then informed the development of a pilot survey designed to assess trauma-informed systems change efforts in Chicago.

For this Think Tank session on evaluating systems change, we will first ground the conversation by briefly sharing our experiences in endeavoring to answer these questions specific to a trauma-informed transformation initiative in Chicago. We will share examples of mixed methods we used to help us understand measuring systems change in our context and importantly uplift and amplify critical community voices. Most importantly, we will facilitate peer learning in small group discussions to foster collective emerging ideas and innovations with fellow evaluators around evaluating city systems change.

Our Think Tank small groups will cover each of the multiple methods used along our journey, offer our lessons learned, and elicit discussion around peer experiences and emerging strategies. We will facilitate small group discussions related to (1) learning from the empirical literature and using case studies to learn about peer city efforts, (2) designing survey measures to assess city systems-level change, (3) using interviews to center and amplify critical voices, and (4) uplifting innovative strategies from the field. Following the breakout groups, we will reconvene to highlight collective key insights across the multiple methods to inform comprehensive emerging practices and innovations for how to measure and evaluate city systems change. We hope to spark ideas and create momentum for new evaluators’ ongoing learning and practice.