What: The Pew-MacArthur Results First Initiative, a project of The Pew Charitable Trusts and the John D. and Catherine T. MacArthur Foundation, is working with Massachusetts state officials to implement an innovative approach to evidence-based policymaking. The process helps to determine which policies and programs are worthwhile investments—and which are not.

The Results First approach can be used to assess the effectiveness of programs and number of interventions in a growing range of social policy areas including adult and juvenile criminal justice, child welfare, education, mental health, and substance abuse. Since 2011, 21 states have partnered with Results First to implement the approach, including a sophisticated, customized cost-benefit model, which guides decisions on what programs to invest in to achieve key public policy goals, such as reducing recidivism.

“The Results First model gives us a proven way to collect and use data to make sound evidence-based policy decisions,” says Trial Court Chief Justice Paula Carey. “The model will, in turn, enable us to work more effectively with our many stakeholders across multiple agencies to reduce recidivism and reach our public safety goals in cost-effective and measurable ways.”

Who: The Special Commission to Study the Commonwealth’s Criminal Justice System and the Executive Office of Public Safety and Security (EOPSS) lead the state’s Results First efforts. A cross-agency technical work group supports data collection and analysis for the model’s criminal justice piece. This group includes members from the Trial Court, Probation, EOPSS Research and Policy Analysis Division, the Department of Correction, the Department of Youth Services, the Parole Board, the Hampden County Sheriff’s Department, and a Results First technical expert.

Results so far: In June 2014, the technical work group customized the adult criminal justice piece of the Results First model and calculated the cost-benefit ratio for 17 evidence-based programs and practices in total: four in probation, seven for corrections, and six in parole. For Probation, the group determined net benefits ranging from $424 to $14,205 per probationer, with expected reductions in recidivism of up to 23.3 percent. The expected reductions in recidivism are based on a meta-analysis of national evaluations of similar programs.   

Study highlights include:

Probation Program/PracticeBenefits per ParticipantCost per ParticipantNet Benefit per ParticipantCost-Benefit RatioChange in Crime
Risk, Need, and Responsivity Principles with ORAS*$12,377($81)$12,296$152.54-21.2%
Electronic Monitoring/GPS: Probation Population$14,969($759)$14,205$19.66-23.3%
HOPE/MORR**$14,666($2,921)$11,745$5.03-22.8%
Employment Training/Job Assistance in the Community: Probation Population$4,023($3,599)$424$1.12-6.7%

*Ohio Risk Assessment System
**Hawaii Opportunity Probation Enforcement Program / Massachusetts Offender Recidivism Reduction
Source: Massachusetts’ Evidence-Based Approach to Reducing Recidivism, pewtrusts.org/resultsfirst as of 12/29/15.
 

To customize the cost-benefit model, the technical work group completed a six- and seven-year recidivism analysis of Massachusetts offenders starting a term of probation or parole, or released from incarceration in the Department of Correction or houses of correction, in 2005. The group also looked at the most serious new offenses in that period. The chart below shows that for most of the offenders, the most serious new offense was a misdemeanor or felony property crime:

pie chart


 

Next steps: Since initially implementing the Results First model, Massachusetts has made noticeable strides towards identifying and expanding programs that reduce recidivism while controlling costs. Participating state agencies, and the Trial Court including Probation, will use the model to routinely evaluate an expanding inventory of current and new criminal justice programs implemented using a standard set of procedures (“best practices”), and to make data-based budget and policy choices. State leaders from all three branches of government may also expand the Results First Initiative to inform other policy areas.
 

What Cost-Benefit Analysis Is, and Why It Matters

Even in the best of economic times, state leaders are often forced to make difficult budget choices. Allocating the right amount of resources to support public services can be an ongoing challenge. Policymakers need proven ways to determine which programs and policies yield the greatest benefits in the most cost-effective way. Cost benefit analysis evaluates policy decisions (or programs) in terms of their outcomes, or costs and benefits. Cost benefit analysis is most often used to evaluate public sector projects1.

Cost-Benefit Analysis Can Help State Leaders…

  • Identify which programs work and which do not in a systematic way;
  • Determine which programs could be expanded, targeted for further evaluation, or eliminated;
  • Calculate return on investment statistics for various programs;
  • Rank programs based on their projected benefits, costs, and investment potential; and,
  • Predict the impact of alternative policies and programs.

Creating a Template for Informed Decision-Making: How the Results First Cost-Benefit Model Works2

  1. Results First provides participating states with access to the model, which uses rigorous national research to analyze program effectiveness.  Additionally, the Results First Clearinghouse Database3 compiles information from eight national research clearinghouses, enabling state policymakers to easily identify and compare which programs are effective and which are not.
  2. Massachusetts adds and analyzes its own state-specific population and cost data. Applying state-specific data to national results helps predict the effects that different program and policy approaches would have on programs here in Massachusetts.
  3. State Results First teams use their customized Results First model to calculate the long-term benefits and costs for each program being assessed.
  4. The state-specific model ranks each program according to its expected return on investment (ROI), the relative value it would generate for taxpayers and stakeholders.
  5. During the budget process, policymakers use the information and program rankings to consider the viability of programs and determine funding allocations.

_________________________

1 London School of Economics, Handbook of Public Economics, vol. II, Chapter 14, The Theory of Cost-Benefit Analysis, Jean Dreze and Nicholas Stern, pp. 909-911.

2 Source: pewstates.org/resultsfirst as of 12/29/15

3 Visit http://www.pewtrusts.org/en/multimedia/data-visualizations/2015/results-first-clearinghouse-database