Overview
Although the Northwestern District Attorney’s Office (NWDA) effectively administers many aspects of its Juvenile Diversion Program (JDP), including assessing the needs of participants, defining deliverables and expectations, monitoring participants’ progress, and confirming completion of agreed-upon program requirements, it does not collect data or conduct post-completion outcome evaluations to determine the program’s effectiveness. As a result, NWDA is limited in its ability to measure the success of the program and determine whether any changes to it are necessary.
Best Practices
The Juvenile Diversion Guidebook prepared by the Models for Change2 Juvenile Diversion Workgroup states,
Every diversion program should have a way to determine whether it is meeting its goals and objectives. . . . A program can select its most important objective(s) and focus on obtaining data related to their evaluation.
The guidebook suggests evaluating outcomes in the following areas: “Reduction in Recidivism,” “Provision of Services,” “Reduction in System Costs,” “Increased Successful Outcomes for the Child,” “Increased Accountability,” “Reduction in Labeling and Its Effects on Delinquency,” and “Reduction in Unnecessary Social Control.”
The goal of the JDP, as stated in NWDA’s JDP Policies and Procedures, is as follows:
To offer intervention at an early stage to eligible juvenile offenders in order to increase safety, accountability, competency development, improve outcomes, and decrease the likelihood of further offenses.
To determine whether this goal is being met, the JDP would need to collect and analyze data. In fact, the Juvenile Diversion Guidebook states, “Program evaluation typically requires a systematic way of collecting data throughout some period of time of the diversion program’s operation.”
Reasons for Issues
NWDA officials stated that there are no data collection efforts to measure outcomes because NWDA lacks the time and staff to perform these tasks.
Recommendation
NWDA should identify the relevant data that could be used to assess the effectiveness of the JDP, establish a formal process to collect and evaluate this information, and use it to make necessary program enhancements.
Auditee’s Response
We appreciate the Auditor’s recognition that the Northwestern District Attorney’s Office Juvenile Diversion Program (NWDA-JDP) effectively assesses the needs of participants, monitors participants’ progress, and confirms completion of agreed-upon program requirements.
We disagree, however, with the Auditor’s finding that the NWDA-JDP does not collect data or conduct post-completion outcome evaluations to determine the effectiveness of the program and as a result, the NWDA is limited in its ability to measure the success of the program.
The central goal of the NWDA-JDP as stated in its Policies and Procedures Manual is:
To offer intervention at an early stage to eligible juvenile offenders in order to increase safety, accountability, competency development, improve outcomes, and decrease the likelihood of further offenses.
We believe that we meet the objectives contained in this goal, by effectively measuring outcomes for juveniles who participate in the program; and by maintaining data on these outcomes. Our central criteria for measuring a successful outcome is focused on the development and completion of an individualized diversion plan that is restorative for victims, juveniles, their families, and their communities. Our approach incorporates the core principles contained in The Juvenile Diversion Guidebook—Models for Change which the Auditor presents as Best Practices for Juvenile Diversion Programs:
Fairness (for youth, families, victims, and communities), the recognition of developmental difference between youth and adults, individual differences (in development, culture, strengths and needs), youth potential, responsibility, and safety (rights of individuals and communities to feel safe). (Juvenile Diversion Guidebook—Models for Change at p. 69)
It should be noted that the Juvenile Diversion Guidebook—Models for Change suggests that a program can select its most important objective(s) and focus on obtaining data related to those objectives.
The NWDA-JDP collects data and assesses its effectiveness in meeting its core goals and objectives in many ways, including: comprehensive assessments of a juvenile and development of an individualized diversion plan that is monitored and the outcomes are tracked; collection and review of parent and child post-completion surveys; regular assessment and review of the program within the organizational structure of our office; staff participation in professional development in best practices (e.g. Motivational Interviewing, Restorative Justice, Mental Health First Aid, [Adverse Childhood Experiences] and Trauma Informed Practices, Recovery Coach Training, and [Screening, Brief Intervention and Referral to Treatment]; collaborative work with community partners; and through data collection within internal office databases.
That being said, we view the Auditor’s report as an opportunity to explore the expansion of the NWDA-JDP’s definition of successful outcomes to include the collection of more long term data beyond the date on which a diversion case is closed. More specifically, we would like to investigate data collection options which will help us improve the diversion program’s ability to assess our effectiveness in “decreasing the likelihood of further offenses.” Our concern in this area lies in the extraordinary nuance of this type of data. For example, the duration of the diversion program is relatively short and once the child has successfully completed the diversion program, their involvement with our office is over. If the juvenile reoffends, there is no way to determine what factors may have influenced his or her return to the criminal justice system. We expect that the new Juvenile Justice Policy and Data Board created by the Criminal Justice Reform Act of 2018 will provide guidance regarding effective data collection methods.
Auditor’s Response
We do not dispute that NWDA’s JDP is structured so that program participants are effectively assessed and provided with individualized programs to better ensure that they complete the program. Although participants’ successful completion of the program is certainly an important measure of how effectively it is administered, we believe that NWDA should also consider collecting data and measuring the outcomes achieved by the program, as suggested in the Juvenile Diversion Guidebook.
Based on its response, NWDA is taking measures to address our recommendation.
| Date published: | November 29, 2018 |
|---|