1009 - Cost-Inclusive Evaluation: You Can Do It, I'm Doing It, and Here's How

Abstract Information: This workshop teaches cost-inclusive evaluation (CIE) with examples from the presenter's 50 years of experience conducting CIE for apprentice-training programs in health care, in emergency assistance programs for LGBTQI+ and other human rights defenders in international settings, consumer-centered programs for suicide prevention, substance use disorders, and mental health services for youth and adults. Illustrations of problems and solutions in CIE are drawn from the presenter's work in international assistance, and treatment and prevention programs for drug abuse, depression, weight control, and adolescent behavioral health. Qualitative and quantitative methods covered include evaluation of costs from multiple interest group perspectives, cost studies, cost-effectiveness analysis, and cost-benefit analysis. Social Return On Investment is viewed critically, along with traditional economic evaluation. The workshop includes examples of the potential for CIE to reveal hidden inequities, to inadvertently maintain or exacerbate those inequities, and to reduce and remove those inequities. After each topic is taught and illustrated, workshop participants apply their new knowledge to a cost-inclusive evaluation of their own choosing. Spreadsheets and tables to structure CIE are provided via a micro website developed by the presenter

Relevance Statement: Insights that cost-inclusive evaluation (CIE) generates for evaluators and researchers are illustrated with 50 years of program evaluations in health and human services, including child, parent, and adolescent interventions, consumer-operated mental health services, randomized clinical trials for suicide prevention, and emergency assistance programs for human rights defenders. Re-conceptualizations of _costs_ and _outcomes_ are introduced first, as these have proven to make CIE more approachable, understandable, do-able, and useful for program evaluations. _Costs_ are conceptualized not as simply sums of money spent, but more usefully as the types and amounts, and current monetary values, of resources actually consumed by the program to engage participants in the specific activities that are responsible for program outcomes. Adding to the “ingredients” approach to measuring costs, this resource-activity or _recipe_ approach to evaluating costs enables better understanding of how a program delivers its services, and what a program requires in budgets. In addition, dissemination of successful programs is aided by quantitative resource x activity tables for planning expenditures in different communities and countries. Also in this workshop, _outcomes_ are conceptualized not only as traditional _effectiveness_, but also as hoped-for monetary outcomes of savings in government services that are no longer needed after program participation, such as some forms of health care, income support, and criminal justice services. Another monetary outcome of importance families and communities is possible increased income for recipients of program services. Next, cost, cost-effectiveness, and cost-benefit studies are recast as attempts not to just describe, but to understand and optimize, relationships between program costs and outcomes. Findings from the presenter’s evaluations are used to show that CIE works best if conducted using multiple interest group perspectives to collect and analyze data from individual participants. These new ways of understanding costs, outcomes, cost-effectiveness, and cost-benefit go beyond quantifying cost-outcome relationships to understanding resource → activity, activity → process, and process → outcome relationships. Accessible graphic methods of portraying these relationships are used to empower engagement of participants, providers, and community members in deciding how to best use precious public resources to provide the best services to the most people in need; more succinctly, _the best to the most for the least_. This focus in CIE can guide public health efforts to raise up as many people as possible, a distinct from maximizing the average improvement for relatively few individuals. Finally, CIE as presented and trained via this workshop uses experiential _evidence _, rather than tenuously generalized abstract research findings, as the foundation for understanding cost → outcome relationships. Complex mathematical models of costs, cost-effectiveness, and cost-benefit are rejected in favor of simpler, observation-raised relationships between the types and amounts of resources used to attain outcomes of value to the communities served by the programs. The latter approach has proven to be more acceptable and more understandable for funders, providers, and recipients of programs in health and human services.