228 - Intersection of evaluation and data analytics: Case study from a community-based organization

Abstract Information: In recent years, the landscape of evaluation practices within community settings has undergone significant evolution, driven by advancements in data analytic technologies. The emergence of machine learning, natural language processing, artificial intelligence, and enhanced visualization tools, coupled with the increased accessibility of data from diverse management information systems (e.g. electronic health record systems and robust customer relational management systems), has reshaped how community-based organizations approach evaluation, performance monitoring, and organizational improvements. Within a community setting, evaluation has been valuable in determining program impact, facilitating program improvement, and communicating program impact to stakeholders. However, the evaluation process can often take considerable amount of time to obtain actionable insights. The growing demand for more real-time data to assess program performance, make programmatic improvements, and to inform strategic decisions necessitates a shift toward leveraging data analytic tools. This shift is especially pertinent to internal evaluators, who are asked to frequently engage in both performance monitoring and evaluation projects. To effectively meet these evolving demands, internal evaluators need to cultivate proficiency in data analytic tools and techniques. This entails developing skills not only in evaluation methodology, data analysis, reporting but also incorporating data analytics skills, such as data programming, dashboard design and visualization capabilities. Through a case study presentation and guided discussion, this roundtable session aims to explore key questions and anticipated challenges for evaluators navigating the intersection of evaluation and data analytics in community settings. Participants will delve into topics like identifying the new skill sets required for internal evaluators, strategies for building capacity in data analytics, and effective communication of the value and necessity of evaluation and data analytics. Additionally, discussions will explore various models for integrating evaluation and data analytics within internal evaluation teams, considering the pros, cons, challenges, and considerations associated with each approach. By engaging in this collaborative exploration, participants will gain insights to inform their evaluation practices, navigate challenges, and maximize the potential of data analytics in evaluation efforts.