Challenges and opportunities: Learnings from the development and administration of a state-wide web-based survey

Abstract Information: This presentation shares the valuable learnings from the development and administration processes of a state-wide survey on understanding school communities’ student support and resource landscape. As part of a larger research project funded by the Institute of Education Sciences that studies the impact of City Connects, a comprehensive student support, on promoting student learning and equity, this survey targeted Pk-12 school principals in a Midwest state to learn about their student needs and resources that were in place and were needed. Among the more than 2000 school principals across the state, the project team utilized stratified sampling and selected a subset of this population (about 300), whose schools were able to reflect the state’s overall student demographic characteristics. Customized survey invitations were sent to their work emails, and by the end of the data collection phase, a total of 295 responses were collected, which yielded rich data for our subsequent data analyses. During this process, the project team encountered a number of different challenges associated with instrument development and survey administration. For example, how to ensure the validity and reliability of the instrument, how to interpret pilot testing data, and how to better engage school principals to complete our survey. To address these challenges, a variety of strategies were identified, developed, and carried out. The sharing of lessons learned and the creative thinking that the project team did related to large-scale survey development and administration could greatly benefit evaluators in the field.