185 - Evaluation in a New Age: Developing and Evaluating a Data Dashboard using an Iterative Process with Multidisciplinary Partners

Abstract Information: Interactive data dashboards are being used more and more in the social sector to monitor projects and build trust; they can be powerful evaluation tools when deployed effectively. In a top-ranked research institution, evidence, scientific validity, rigor, and academic publishing has long been valued. Data dashboards are part of a new wave of evaluation tools, fueled by the passion and innovation of a new generation of evaluation researchers. This poster will demonstrate how a published framework for creating, using, and maintaining dashboards as an evaluation tool was used in practice by evaluators at the University of Wisconsin to develop a public-facing data dashboard for the Wisconsin 988 Suicide and Crisis Lifeline. The poster will also demonstrate how passion for a new way of evaluation persevered in multiple systems laden with rules and operations. This poster will explore how data sharing, ownership, and visualization best practices were incorporated into the dashboard development process. Emphasis will be put on how the evaluators prioritized user design and transparency in data sharing, and collaborated with university, state, and call center partners on all aspects of the process. This included determining the need, building buy-in, solidifying the data measures, designing the dashboard, and launching the dashboard. While leading with values of integrity, innovation, transparency, and inclusion, the evaluators facilitated monthly meetings from June 2023 to December 2023 with partners to create the data dashboard product, addressing barriers inherent in long-existing systems and prioritizing feedback from those with lived experiences. Several approval processes were required before the final product could be implemented, and conversations and collaboration were needed to ensure that the aforementioned values were upheld. During the dashboard launch in February 2024, rapid and responsive evaluation was used to address incoming feedback from users and to improve the functionality of the dashboard. Launch also resulted in several communications about the new data dashboard, including listserv emails, social media posts, and a news story. Information will also be shared regarding how the evaluators are improving the dashboard using results from the user feedback survey. The 988 evaluation team at the university-based evaluation research center consisted of three evaluators with varying levels of professional experience, ranging from early career to senior. This project led to skill building and professional development for everyone on the team. Throughout the demonstration, the new evaluators will share lessons learned from working on this project and experiences of bringing new evaluation tools to multiple long-standing systems. Additionally, the senior evaluator will share reflections from championing this work, mentoring a new evaluator, and mentoring a new Co-Principal Investigator.