131 - Connecting People and Points of Entry to Conditions of Collective Impact: Evaluation of AGEP Alliances Focused on Improving Representation of Underserved Communities in the Professoriate

Abstract Information: Over the past decade, the Collective Impact (CI) framework has gained popularity as a structured approach to making cross-sector collaboration initiatives achieve significant and lasting social change, tackling social and environmental problems from childhood obesity and homelessness to college graduation rates. CI is an agreement among a collective of important stakeholders from multiple sectors to work towards a common goal, usually solving a particular social problem (Kania & Kramer, 2011). The current report highlights the functioning of two AGEP alliances, each evaluated using the conditions of CI as a guide in a higher education, multi-institutional change partnership model housed within a state university system. AGEP alliances have the expressed intention of systemwide reform to increase the representation of underserved communities of color in the professoriate. CI is intended to maximize the outcomes of such efforts through the evaluation and reflection around the five conditions of collective impact. In the list below, there are two terms included for each condition of CI. These terms reflect the evolution of CI from a management-focused model for partnership programs to a movement-making paradigm for community initiatives (Weaver, 2015). • Common Agenda / Shared Aspiration • Continuous Communication / Authentic Community Engagement • Backbone Organization / Containers for Change • Shared Measurements / Strategic Learning • Mutually Reinforcing Activities / High Leverage Activities Current reviews on CI in the literature provide a starting point in considering how to evaluate its implementation in a partnership program. Practitioners have reported challenges in evaluating CI, with the complexity of CI often cited as the most salient challenge for evaluation. Although researchers have suggested practical recommendations to support the evaluation of CI initiatives, theoretical models that can conceptually scaffold the navigation of the complexity of CI remain scarce. We drew on an ecological approach designed for evaluating collaborative practice in NSF partnership projects, namely the SPARC model (Burr et al., 2021). It proposes five engaged in an AGEP alliance: • Sponsors of funded programs (S), • Program partners and participants (P), • Assessment and evaluation professionals (A), • academic researchers (R), • and the national and global Community (C) A collaboration theory, the Collaboration Evaluation and Improvement Framework (CEIF, Woodland & Hutton, 2012) provides five specific entry points for the assessment of key collaborative practices: • Operationalize Collaboration • Identify and Map Communities of Practice, • Monitor Stages of Development, • Assess Levels of Integration • Assess Cycles of Inquiry We present an integrated model connecting the stakeholders groups and entry points for evaluation with the conditions of CI. This integration helps practitioners navigate the complexity of evaluating the conditions of CI by scaffolding the design and implementation of the evaluation with the SPARC and CEIF models. The integration further serves to amplify the voices of all stakeholders across the academic ecology of the partnership and empowers partners to be change agents guided by CI. There is a key role for evaluators as change agents in CI, who both facilitate collaborative practice and evaluate it.