Practical Considerations on How the Teacher ICCs Vary/Co-Vary with School Characteristics: Implications for Equity in Educational Evaluation

Abstract Information: Impact evaluations using cluster randomized trials (CRTs) or multisite cluster randomized trials (MSCRTs) are commonly used in evaluating educational intervention efficacy. Considering the nested structure of the educational system with students nested in teachers nested in schools, studies sometimes involve the teachers, which requires teacher-level design parameters. These parameters are likely not universal across schools. The existing literature lacks a systematic exploration of how teacher intra-class correlation coefficients (ICCs) vary across school characteristics, which is crucial for accurate power analysis and study planning. This study investigates this variability across three states – Michigan, Kentucky, and North Carolina, revealing nuanced patterns in teacher ICCs. Notably, urban and non-urban schools exhibit opposing teacher ICCs across states, while lower teacher-to-student ratios are consistently associated with higher teacher ICCs. The relationship between free or reduced lunch eligibility (FRL) and teacher ICCs also varies across states. These findings inform a more accurate estimation of design parameters for CRTs and MSCRTs, emphasizing the need to account for school-specific characteristics in educational impact evaluations. Practical examples will illustrate how researchers and evaluators can utilize these insights to enhance study planning.