43 - TIPS for Evaluating a Trauma-Informed School-Based Program: Challenges and Lessons

Abstract Information: Stress and trauma are important contributors to the youth mental health crisis in the U.S. today. The prevalence of potentially traumatic experiences among American youth has become well known in recent years, and trauma associated with the COVID-19 pandemic increased the threat to young people’s psychological well-being. Studies have found that females, children with preexisting internalizing disorders, and youth whose families were most affected by the pandemic were more susceptible to experiencing psychological trauma symptoms. School is a place where effective trauma-informed programs can make a positive impact on students experiencing difficulties that present barriers to learning. Whole-school trauma-informed approaches are still relatively new, and evidence supporting them remains limited. This paper describes the evaluation of the Trauma-Informed Program for Schools (TIPS) (a pseudonym), a support model that assists schools in creating and implementing a school-specific action plan to implement and monitor trauma-responsive and healing-centered practices. Specifically, we focus on challenges engaging schools in evaluation activities and strategies to address them.

We use a mixed-methods design to understand the implementation and outcomes of TIPS. Interviews and focus groups are conducted with regional support providers and school-based staff and administrators. We use survey data to assess constructs such as fidelity of training, changes in school climate, perceived program effects, and attitudes towards trauma-informed education.

Evaluations in school settings can be challenging because school staff are busy and experience high rates of burnout and turnover. To address these challenges, we use a systematic approach to recruitment, which includes cash incentives, repeated recruitment attempts, and replacement of nonresponsive schools. Relationship building is integral to our approach. We also make sure to express our appreciation to service providers through phone calls and holiday gifts. Another strategy we use is flexibility in scheduling interviews and focus groups with schools, including taking advantage of differing time zones to widen our availability. In our presentation, we will discuss the strategies we use to navigate recruitment and engagement challenges in detail. We will offer lessons learned that other evaluators could apply to their own work. For example, we will discuss the importance of identifying a key program stakeholder who can serve as an evaluation champion and encourage school staff to participate. Additionally, we recommend that program staff set the expectation of participation in evaluation activities in their communications with schools from a program’s outset. This way, school staff will not be surprised by requests for participation and will see the evaluation as another part of the program.