289 - Evaluation Science is from Mars; Healthcare Quality Improvement Science is from Venus: What Devotees of These Disciplines Can Teach Each Other

Abstract Information: Program evaluators are often tasked with developing evaluation designs for clinical research programs and other health system initiatives. These efforts often engage collaborating physicians, nurses, pharmacists, and other clinicians. Many healthcare professionals have completed some level of coursework in healthcare quality improvement (QI) science during their academic training, and clinicians also gain experience with healthcare QI methods through involvement in performance improvement initiatives within their work settings. As such, program evaluators operating within healthcare environments are likely to encounter clinical collaborators who are knowledgeable about healthcare QI science, but who are less familiar with program evaluation methods. This presents both opportunities and risks for program evaluation. Opportunely, the concepts and goals of these two fields have many areas of alignment, yet each discipline offers something different and a diversity of ways of thinking can be an asset to program evaluation. On the other hand, there are also potential risks as the alternative vocabularies and ways of modeling may lead to miscommunication, confusion, and disengagement. This poster provides a graphic representation/translation of the overlap and contrasts that illustrate what evaluators need to know to leverage the synergies while minimizing the risks. For evaluation scientists, an appreciation of common frameworks used in healthcare QI science can aid in understanding the mindset of clinician collaborators, and may enhance the eagerness of clinicians to contribute to program evaluation efforts. Clinicans will also benefit from evaluation processes that incorporate their perspective and that link to familiar themes, including elucidating relationships between activities and outcomes, and continuous learning and adaptation. This poster considers two foremost healthcare QI frameworks: the Donabedian Quality of Care Model, and the Model for Improvement, and how each model can inform program evaluation efforts. The poster highlights how the Donabedian model shares similarities with logic modeling, which defines program resources, activities, outputs, and intended outcomes, and also explains how the purposes and contexts of these models can differ. The Model for Improvement is another extensively utilized healthcare QI method. Its hallmark feature is its iterative Plan-Do-Study-Act (PDSA) cycles that test the results of planned interventions. This poster illustrates how the Model for Improvement’s PDSA cycle approach contrasts with program evaluation methods for defining outcome and impact, including the use of qualitative feedback and prospective data collection to inform goal achievement. For clinicians who have program management responsibilities evaluation science has additional merits that can promote positive transfer. These include the use of formative and summative assessments of program activities, strategies for gauging the needs and perspectives of stakeholders, methods for developing an evaluation approach for complex and multi-faceted initiatives, and considering impacts beyond the institution. The poster also discusses program evaluation methods as complementary and supplementary to the Donabedian framework and the Model for Improvement.