120 - Lessons Learned Adapting a Health Impact Assessment Tool Using Cognitive Interviews

Abstract Information:

Introduction: The National Institutes of Health (NIH) supported the Rapid Acceleration of Diagnostics-Underserved Populations (RADx-UP) program, which aims to ensure that all Americans have access to COVID-19 testing and promotes health equity, especially in underserved communities. Under RADx-UP, there are two pilot programs — Community Collaboration Grants (C2G) and Rapid Research Pilot Program (RP2) — that fund projects to conduct community-engaged outreach and research to address COVID-19 testing disparities. Our evaluation team used cognitive interviews with project representatives to revise an existing Health Impact Assessment tool to measure health equity impacts within these community-centered programs. We highlight our process of adapting the original tool and share lessons learned that other evaluators may apply to their work.

Methods: From August to September 2023, we conducted online cognitive interviews among six representatives purposively sampled from 3 C2G and 3 RP2 projects. We developed a semi-structured interview guide to assess participants' opinions about an equity measurement tool developed by Heller et al. (2014). During interviews, we asked participants to use a think-aloud approach to elicit feedback related to comprehension, clarity, structure, and difficulty level on the items and response options within the original tool. A team of three coders deductively developed a codebook and analyzed data using Dedoose, a mixed-methods web application. After analysis, we revised the original tool based on the summarized feedback. We re-engaged participants to provide additional feedback on how the tool improved from the original to the revised version. Once finalized, the final version will be administered to assess the pilot programs’ health equity impacts. Result: All participants provided feedback on the tool indicating that we should make changes to improve the tool’s suitability for measuring pilot projects’ health equity impacts. For example, participants indicated that we needed to improve the clarity and readability of key terms and phrases and provided suggestions for how to tailor the survey to community partners. Participants also recommended we change statements to questions and make questions and response options concise and mutually exclusive, reducing the tool length and response burden. Significance: This project highlights the importance of cognitive interviewing as a critical methodology to: 1) tailor an existing, pilot-tested tool to a community-based audience, and 2) increase comprehension by improving the readability, applicability, and clarity of the existing tool. We adapted the tool for C2G and RP2 program participants, bridging a gap in the availability of tools to assess health equity impacts in community-based programs. Through the cognitive interview process, we realized the importance of prioritizing face validity and tailoring tools to target audiences. We also had to consider balancing trade-offs, such as time, staff capacity, relevance, and participant burden, between creating a new tool versus adapting an existing, published tool. Our process and lessons learned can be used as starting points for other community-engaged research practitioners or evaluators looking to adapt the health equity impact assessment tool or others like it.