197 - Considering Heterogenous Effects in the Evaluation of a New Teacher Coaching Program

Abstract Information:

Objective: This evaluation considered aspects of a heterogeneity in the effects of an instructional coaching support program (North Carolina New Teacher Support Program (NTSP)) on teacher attrition. Some subgroups of teachers are more vulnerable to workplace attrition than others. Our study focuses on beginning teachers. They represent a fairly small proportion of the teacher workforce but have greater rates of attrition. Professional development is one known option for reducing teacher attrition (Bastian & Marks, 2017; DeJong & Campoli, 2018; Ronfeldt & McQueen, 2017; Smith & Ingersoll, 2004). Yet, evaluations of professional development programs are often inconclusive. The evaluations, programs, and even coach training may be the root of these inconsistencies underscoring the need for evaluating coaching strategies while considering individual and contextual factors. Most evaluations consider broad features of coaching programs (e.g., Brown et al., 2020; Kang & Berliner, 2012; Smith & Ingersoll, 2004) and do not consider heterogeneity in these experiences across teacher characteristics and contextual factors. As part of our evaluation, we sought to answer: (1) What coaching practices best support teacher retention? and (2) Do these coaching practices vary across teacher, coach, and school characteristics?

Methods: This study used NTSP data from 2014-15 through 2020-21, and administrative data from the North Carolina Department of Public Instruction (NCDPI). There were 2,966 teachers who met the program participation criteria of a minimum 12 coaching sessions and at least six total hours of coaching interaction. This binary indicator of NTSP participation was our primary independent variable of interest. Our outcome of interest was teacher retention which was operationalized as those who completed a March-to-September employment cycle. The analysis used a fixed–effects model with a combination of OLS linear probability modeling and fixed effects logit modeling. To consider effect heterogeneity, we conduced sub-group analyses with categorical variables that did not change over time and included interaction terms in the fixed-effects models for continuous variables that changed over time.

Results: First, NTSP demonstrated a significant and positive impact on teacher retention when considering all participants. This is an important result but further subgroup analysis informs the generalizability of these results. Black teachers and teachers in cities, towns, and suburbs experienced greater effects while significant effects were not detected for teachers in rural schools. We also found significant interactions between several school level factors: (a) percent of economically disadvantaged students, (b) percent of minority students, and (c) percent of teachers with three or less years of experience. The positive effect of NTSP on teacher retention comes primarily from its ability to maintain and even increase retention rates in schools with more economically disadvantage and minority students and magnify retention rates in schools with a high percentage of new teachers.

Conclusions: The evaluations using comprehensive analytic approaches detail for whom and in what contexts positive effects are applicable. The results detailed here can be codified into recommendations to guide evidence-based decisions for increasing teacher retention and therefore accelerating learning for all students.