Co-Creating Evaluation Frameworks with Nonprofits and Funders

Abstract Information: Evaluation is useful in demonstrating the effectiveness of strategies aimed to improve programs, policies, and practices. However, many community organizations lack the knowledge, skills, and resources to engage in impactful evaluations. In an attempt to address this lack of capacity, the Evaluation Capacity Network (ECN) partnered with the Alberta Mentoring Partnership (AMP) to develop community-based evaluation capacity building (ECB) initiatives and provide continued support in evaluation processes of AMP’s mentoring programs. This paper will discuss qualitative data drawn from interviews and focus groups from an ECB initiative, and process data gathered during the building of an evaluation framework for a new province-wide mentoring program that provides services to youth and young adults ages 16-24. To support AMP and their partnered mentoring agencies, the ECN created and facilitated a 6-month experiential ECB initiative called E-Eval, which aimed to develop individual and organizational evaluation capacity. With a community-based and culturally sensitive approach to ECB, the purpose of E-Eval was to enhance the participating mentoring agencies' evaluation capacity and to improve the quality and cultural relevance of their mentoring programming. Through collaboration between the ECN, the participating mentoring agencies, and the funders, a single joint evaluation framework for a new mentoring program was developed. Feedback sessions and post-initiative interviews were conducted by the ECN to examine if the processes and practices of E-Eval were effective community-based ECB strategies. From the perspective of funders and participating agencies, it was clear that funder involvement was beneficial to the overall culture of evaluation for the new mentoring program. Additionally, the co-creation of the evaluation framework provided an opportunity for all participants to share data collection tools and culturally relevant resources. Due to the positive outcomes of the collaborative E-Eval approach, AMP and the ECN continued their partnership and currently support agencies with the implementation of the mentoring program evaluation framework. The first year of the mentoring program (April 2023 to March 2024) served as a trial period for the evaluation process as agencies faced unforeseen circumstances and program changes in response to community needs. The ECN staff created positive working relationships through open dialogue with all parties involved in the mentoring program which allowed the ECN to flag issues in the data collection process and make changes to the evaluation processes accordingly. Therefore, all agencies and funders actively took part in the modification of the evaluation framework and data collection methods. In doing so, the voices of frontline staff were major contributors to how the mentoring program was to be evaluated in a way that considered the youth being served. The communicative working relationships formed between the ECN and current mentoring agencies has helped the ECN gain a better understanding of the mentoring program thereby improving the ECN’s ability to support the evaluation needs of future agencies that join the mentoring program. We look forward to sharing our evaluation process with the goal of contributing tangible resources for other community groups who wish to bring more collaborative approaches to their program evaluation.