Amplifying the Voices of “Everyday Evaluators”: Building Local, Community-Wide Evaluation Capacity

Abstract Information: The San Antonio Evaluator Pipeline (SAEP) project grew out of a growing realization: Our community does not have enough evaluators to fulfill the needs of local nonprofits, foundations, higher education, K-12 school districts, private sector companies, and government. Even though San Antonio is one of the largest cities in the nation, we do not have many experienced evaluators or a strong pipeline for recruiting and training emerging evaluators. The focus of this session is on an additional realization: many nonprofit organizations in San Antonio lack the capacity to have full-time, trained, experienced evaluators on staff. As part of SAEP’s work to strengthen San Antonio’s evaluation ecosystem, Dr. Leigh Rauk and Dr. Kurt Steuck are conducting a landscape analysis to better understand the San Antonio nonprofit landscape with a specific focus on identifying “everyday evaluators”, challenges and barriers nonprofit organizations face related to engaging in evaluation, and identifying their professional development needs. We have defined everyday evaluators as professionals working within nonprofit organizations who take on evaluation-related roles and responsibilities but do not necessarily identify themselves as an evaluator nor do they have significant, formal training in evaluation. Through this landscape analysis we have conducted interviews with nonprofit stakeholders working in San Antonio utilizing a snowball sampling approach. Interviewees have represented various organizations including staff members from foundations and service providers as well as experts with years of experience working directly with nonprofit organizations in San Antonio. Interview results have revealed several key findings regarding everyday evaluators’ roles as well as challenges and barriers nonprofit organizations face with respect to formal evaluation capacity. Everyday evaluators serve a variety of roles with the most commonly reported roles being program manager, grants manager, and data analyst. Only one organization we interviewed had a staff member with a clear title related to learning and evaluation. While many grants require a some type of evaluation component, many nonprofit organizations do not have the capacity to conduct a robust evaluation or the funding to hire a dedicated evaluator on staff. For some organizations, this common grant requirement was actually a barrier to applying for much needed funding due to their inability to fulfill that requirement of the grant.