The DHS Evidence Overview: Amplifying collaboration of evaluators and program staff, evaluative thinking, and equity to strengthen program design, planning, and management for homeland security

Abstract Information: As a “newcomer” to evaluation, the U.S. Department of Homeland Security (DHS) has not historically leveraged evaluation in the design, planning, or management of its financial Assistance Programs. In 2022, with support of the Office of Management and Budget (OMB) DHS piloted a process to facilitate the use of evidence among this group of programs, with the goal of strengthening program decision making. The pilot process helped program staff to use evaluative thinking as they use evidence and stakeholder perspectives to define the problem (or need) to be addressed; determine how the pro-posed activities are expected to lead to intended outcomes and overarching goal; describe what evidence exists to support the program logic and where it is weak or lacking; and determine what indicators and evaluations are needed to understand if results are achieved and to fill evidence gaps. This annual DHS planning exercise is a collaboration among the evaluation offices, financial assistance poli¬cy and oversight offices, grants administration offices, and program offices; many of whom were not previously engaged in evaluation. Building this into standard operating procedures is instrumental in identifying agency Learning Agenda questions, areas where evaluation is needed, and where performance management and reporting on results can be strength¬ened. In 2023 DHS expanded the process to include all financial assistance programs across the Federal Emergency Management Agency (FEMA), US Citizenship and Immigration Services (USCIS), Cybersecurity and Infrastructure Security Agency (CISA), US Coast Guard, Countering Weapons of Mass Destruction Office, the Science and Technology Directorate, and for policy areas where build¬ing evidence on what works is crucial. Recent Executive Orders on Advancing Racial Equity motivate additions to the process and create opportunities for using the Evidence Overview to unify other agency planning efforts as part of the federal government’s “comprehensive approach to advancing equity for all, including people of color and others who have been historically underserved, marginalized, and adversely affected by persistent poverty and inequality.” In 2024 DHS introduced and is piloting an alternative tool—the Equity Evidence Overview—which uses best practices of equity analysis to guide teams in considering and being explicit about how equity factors into the program, measurement, and evaluation planning. The tool guides exploration of inequities and barriers to equitable access, quality of processes, and outcomes. Equity-related purpose must be explicit in stated goals and objectives. Program activities include outreach to and meet the specific needs of underserved groups to ensure they can access programs and services, and that those experiences meet their expectations, enabling them to derive intended benefits. Performance indicators and evaluation must position the program to understand changes to the populations being served, what led to those changes, and what promising insights resulted that should be considered for further advancement to better reach underserved groups next time. This paper discusses the Equity/Evidence Overview approach, findings and lessons learned from implementation, and emerging opportunities to use the approach to support collaboration of evaluation and program staff for customer-, results-, and equity-focused program design, planning, and management.