Overview
Currently, the Norfolk District Attorney’s Office (NDAO) collects Diversion Program case data, such as participants’ names and offenses; the completion dates of any services or classes; and the reasons participants did not complete the program, if applicable. These data could be used to measure the results of the program and determine whether any changes to it are necessary. However, NDAO is not evaluating these data; it is only using the data as a record of participation and completion of the program-specific requirements.
Consequently, NDAO cannot determine whether this program effectively and efficiently provides a tangible benefit to the community it serves. Had NDAO evaluated these Diversion Program data, it could have used them to consider potential improvements to the program and to support requests to the Legislature for program-specific funding.
Authoritative Guidance
Although there are no specific laws or requirements for data collection and evaluation for the Diversion Program, there are state and national publications that encourage data collection as a way to ensure that diversion programs achieve their intended purposes. For example, Models for Change is a multistate initiative focused on promoting the advancement of juvenile justice reform, funded by the John D. and Catherine T. MacArthur Foundation. Models for Change has its own juvenile diversion workgroup that has prepared a Juvenile Diversion Guidebook, which states,
To ensure the diversion program is meeting its objectives and goals, a record-keeping and data collection system should be in place to assist in providing periodic evaluations.
Additionally, ICF International, a global consulting firm, has received grant funds to conduct an assessment study of pretrial juvenile diversion practices within District Attorneys’ Offices throughout Massachusetts. Section 4.1.2 of ICF’s Massachusetts Juvenile Diversion Assessment Study, published in January 2015, states,
In order to ensure that diversion programs are meeting their stated goals and objectives, it is critical for [District Attorneys’] offices to implement a standard record keeping and data collection system. This will allow offices to assess the need for program adjustments over time; identify whether program goals and objectives are being met, for whom, and why; and provide justification for additional resources and supports.
As illustrated in the quotations, just collecting the data is not helpful. However, periodic evaluations of the data will help District Attorneys’ Offices identify the need for program adjustments over time and determine whether programs are meeting their intended purposes. Therefore, collecting and evaluating data for measuring program performance can be considered a best practice.
Additionally, although the above criteria discuss juvenile diversion, there is support for using the same processes for young adult diversion that are used for juvenile diversion. For instance, the Justice Policy Institute’s report Improving Approaches to Serving Young Adults in the Justice System states,
The evidence that adolescence extends well beyond 18 could, theoretically, be applied to the justice system to reduce long-term negative consequences for young people and take advantage of a significant opportunity to give young adults the best chance to succeed.
Reasons for Issues
NDAO officials explained that success is measured by a case not being prosecuted and the participant being held responsible by having to complete the program-specific requirements (such as writing a letter of apology, performing community service, and/or completing classes in person or online), thereby avoiding a criminal record that negatively affects their future employment status. Therefore, each participant who completes the program is considered a success.
Recommendation
NDAO should consider ways to evaluate the Diversion Program data that it currently collects to help identify measures of program success other than participants’ successful completion of the program, identify areas where program improvements may be needed, and support requests to the Legislature for any necessary program-specific funding.
Auditee’s Response
- The Norfolk District Attorney’s Office has not established a process to measure the success of its Diversion Program.
The Norfolk District Attorney’s Office (NDAO) is proud of its Diversion Program.
Consistent with Legislative intent, the NDAO seeks to divert appropriate individuals who have committed minor offenses in an effort to preserve their lack of criminal record so as not to hinder future employment, military service or other important opportunities in the individual’s future.
As concluded by the [Office of the State Auditor (OSA)], the NDAO “ensures that all program requirements are fulfilled and documented for participants who have successfully completed its Diversion Program.”
The OSA also recognizes that the NDAO’s Diversion Program is discretionary and [was] not governed by statute [during the audit period], and that the program does not receive separate funding from the Legislature. As such, the NDAO maintains that measures of the program’s success are entirely and appropriately determined by the criminal justice professionals involved in its administration (NDAO).
At the commencement of the audit period, the OSA attempted to assess all diversion programs using a predetermined standard for success; i.e., to measure recidivism. It was not until a meeting with the Massachusetts District Attorneys Association and State Auditor Suzanne Bump on December 11, 2017, and a later telephone conference call on January 30, 2018, that the OSA communicated it was “no longer going the route of recidivism.”
The OSA has acknowledged that it reviewed the NDAO diversion data, which showed participants’ successes or failures. The NDAO also offered the OSA documentation showing the NDAO’s internal assessment of its diversion program at the end of the Audit.
Finally, the NDAO verbally communicated to the OSA that its diversion programs had changed throughout the years based on analyzing its programs over time. In particular, the NDAO discussed at great length with the OSA how it had recently started a new Buyer Diversion Program because the NDAO recognized that the drug and alcohol educational component in its existing Diversion Program was missing a much-needed clinical component.
In summary, the Norfolk District Attorney’s Office (NDAO) is proud of its Diversion Program and its success rate, which was 82.05% for 2017 at the time the Audit was completed.
- The NDAO Victim Witness Advocate Program
The NDAO feels it provided adequate access, information and assistance to OSA for it to determine whether the NDAO’s Victim Witness Advocate Program provides assistance throughout the court process to victims and witnesses of crimes as required by Section 5 of Chapter 258B of the General Laws.
On January 30, 2018, in a telephone conference call with the NDAO and OSA, the OSA Audit Manager indicated that she did not want to read the notes in the selected sample of Victim Witness case files; she just wanted to know the date the phone call was made to show contact was made. The OSA Audit Manager suggested that someone from the NDAO sit with OSA staff, cover the notes in the file, and let the OSA see that the phone call was made.
In response to that conversation, the NDAO subsequently sat with the OSA over a number of days and went through each case file in the audit sample. The OSA observed each file and saw notes of calls made and copies of letters sent to victims on each case. The type of victim contact and the date of such contact were written down by two members of the OSA. On a number of occasions, an OSA member later asked the NDAO to clarify a particular date/contact to ensure its notes were accurate for each case file audited. Appropriately, the victim’s personal information was not provided, as the NDAO takes very seriously its mandate to protect victims’ identities, especially sexual assault and domestic violence victims.
- Audit Constraints
The OSA identified two particular audit constraints attributed to the NDAO. OSA cites a delay on NDAO’s behalf and confidentiality concerns.
- Confidentiality
The NDAO takes very seriously its mandate to protect victims’ identities, especially sexual assault and domestic violence victims, and maintains that it complied with those mandates while providing the OSA with the information it needed to conduct its Audit. See, e.g., G.L. c. 258B § 3(h); G.L. c. 41 § 97D; G.L. c. 6A § 18N; and G.L. c. 9A § 1.
- NDAO Purported Delay
The OSA contacted the NDAO in April of 2017 [regarding] its decision to audit the NDAO’s Diversion and Victim Witness Programs. At that time, the audit period was identified as July 1, 2014 to December 31, 2016. There was a subsequent postponement of the starting date of the Audit by the OSA due to a delay in completing their previous audit. The OSA was finally able to start NDAO’s Audit on June 15, 2017.
In an effort to comply with the OSA requests, a number of conversations occurred between the two agencies concerning the disclosure of confidential information contained in NDAO diversion and criminal case files. The two agencies disagreed on each’s legal obligation concerning confidentiality. The issue resulted with the Massachusetts District Attorneys Association meeting directly with State Auditor Suzanne Bump on December 11, 2017, to discuss these issues.
On January 30, 2018, by letter the OSA advised the NDAO that it would reengage its Audit on the NDAO Diversion and Victim Witness Program “on or about February 5, 2018.” In that same letter, the OSA changed the Audit period to January 1, 2017 through December 31, 2017.
In its response, NDAO also provided a timeline that listed 18 dates between March 1, 2018 and May 30, 2018 when NDAO asserts that it provided information to OSA and answered questions OSA had regarding NDAO’s Diversion and Victim Witness Programs.
Auditor’s Reply
OSA acknowledges in our report that NDAO collects data that could be used to better evaluate the effectiveness/success of its Diversion Program and that the program has changed throughout the years. However, as noted in our report, our concern is that NDAO does not effectively evaluate all the data it collects; it only uses the data as a record of participation and completion of program-specific requirements. During the audit, OSA asked NDAO officials how they track or measure the success of the program. They explained that it was done on a case-by-case basis, but that there is no limit to how many times a person can participate in the program. They further explained that the Diversion Program coordinator tracks the completion of program-specific requirements for each case; NDAO does not measure or track the program as a whole. Finally, NDAO acknowledged that it did not share any program data collected with other District Attorneys. Such sharing would be useful in monitoring the extent to which participants might have offended in other districts.
We do not dispute that appropriately diverting people who have committed minor offenses in an effort to help them avoid a criminal record is a positive goal of the program. However, we believe that NDAO could better administer this program if it analyzed all the program data it collected, or collected information such as why some participants could not successfully complete the program, and used this information to make improvements to the program as necessary. Therefore, we again urge NDAO to consider ways to evaluate the Diversion Program data that it currently collects to help identify measures of program success other than participants’ successful completion of the program and identify areas where program improvements may be needed.
We do not agree that NDAO provided “adequate access, information and assistance to OSA for it to determine whether the NDAO’s Victim Witness Advocate Program provides assistance throughout the court process to victims and witnesses of crimes as required by Section 5 of Chapter 258B of the General Laws.” OSA’s audit manager did tell NDAO that she did not want to read the notes in the selected sample of Victim Witness Program case files and that instead, she just wanted to know the dates of phone calls to show that contact was made. However, although OSA agreed that it did not need to review any confidential notes in these files, we did need to confirm through visual inspection that required contacts with victims and witnesses were made, and the dates they were made, to assess compliance with Chapter 258B of the Massachusetts General Laws. We acknowledge that NDAO attempted to accommodate our requests to review the information in these files, but it did not give us access to this information in a timely manner and did not allow us to review or handle Victim Witness Program case files containing victim witness advocates’ notes and dates when contact was made with victims or witnesses. Rather, the contents of the selected files were read to us by the First Assistant District Attorney; we were not allowed to examine the files. Because we were not permitted to read the source documents ourselves, we had to rely on testimonial evidence regarding the contents of the files. Therefore, it is necessary and appropriate for OSA to report this limitation and our inability to conclude on this audit objective in our audit report.
NDAO is correct in stating that OSA eventually told NDAO personnel that it was no longer seeking information about program participants’ recidivism rate. OSA made this decision because of the audit constraints discussed in this report.
In its response, NDAO asserts that it offered OSA documentation showing NDAO’s internal assessment of its Diversion Program at the end of the audit. However, this information was provided after the conclusion of the audit fieldwork, and therefore OSA did not have the opportunity to assess its authenticity and accuracy.
In its response, NDAO provided a timeline detailing how it cooperated with OSA’s requests for information from March 2018 through May 2018. OSA appreciated this cooperation, but as previously noted, the information provided did not allow us to conclude on all of our audit objectives. Moreover, although NDAO began providing this information in March 2018, our audit began in June 2017. The delays that occurred during this audit were the result of the audit constraints imposed by NDAO. Had the office cooperated with OSA and provided information in a timelier manner, the audit could have been completed significantly sooner.
Finally, although we appreciate NDAO’s concerns about giving us access to certain information, we believe we have the authority to obtain and analyze this information during audits. OSA’s enabling statute grants it access to these records. Specifically, Section 12 of Chapter 11 of the General Laws states,
The department of the state auditor shall audit the accounts, programs, activities and functions directly related to the aforementioned accounts of all departments, offices, commissions, institutions and activities of the commonwealth, including those of districts and authorities created by the general court and including those of the income tax division of the department of revenue and, for such purposes, the authorized officers and employees of the department of the state auditor shall have access to such accounts at reasonable times and the department may require the production of books, documents, vouchers and other records relating to any matter within the scope of an audit conducted under this section . . . except tax returns.
| Date published: | October 24, 2018 |
|---|