Reflections on a collaborative design of impact monitoring indicators for racial justice in a civil rights advocacy organization working in the Deep South.

Abstract Information: This session presents reflections on creating organizational indicators to measure a legal advocacy nonprofit’s progress toward long-term goals that focus on racial justice in the Southern United States. The organization works primarily in states considered part of the Deep South. In late 2022 the organization launched a strategic plan that contained five areas of impact: reducing poverty, strengthening voting rights, removing white supremacy from the mainstream, decreasing incarceration and criminalization of Black and Brown people, and increasing power and capacity for multi-racial democracy in the South. The internal evaluation team was tasked with creating indicators that enabled measuring the organization’s contribution to its five impact areas. The goal was to generate indicators that align with the strategic framework so that reports on indicator tracking would be useful to departments and project teams. The evaluation team faced several challenges: The organization had no prior monitoring measures; this was the first strategic plan for this organization. The evaluation team was new to the organization and still establishing monitoring and evaluation processes. As a legal advocacy organization focusing on civil rights issues, much of the organization’s work is reactive and defensive. To address these challenges and the need for creating useful and meaningful indicators, the evaluation team took a participatory and collaborative approach. It formed a working group with 30 members representing all levels of staff from nearly all departments. The working group formed five sub-committees to focus on each of the impact areas. Over the course of eight months, the evaluators facilitated an iterative process to reflect on the content of the strategic plan, identify preferences for outcomes, consider the capacity to collect data, and refine a set of proposals to a final list of 24 indicators. The next phase of the process was to implement the indicator data collection. The evaluation team designed a pilot for the first year of implementation. The pilot sampled eight of the 24 indicators. The sample focused on indicators that were the most feasible to collect with the organization’s limited capacity and which could yield useful data during the first year. Other indicators require long-term multi-year research that will take multiple years to set up. A new working group was formed with members who have subject matter expertise in the topics of the selected indicators. Working group members also were selected from teams who aimed to be primarily involved in collecting data and integrating it into project planning and decisions. Another goal of the pilot phase is to generate learning questions linked to the strategic plan that focus on key research and evaluation priorities of executive leadership, the board, and other organizational stakeholders.