Overview
During our review of the PSI Matrix submitted by MSA for the audit period, we identified several data integrity issues within the submitted templates. Specifically, we found numerous instances of incorrect data formats, such as text entered where numerical values were required, text fields meant for in-depth background information containing only unexplained numbers, and required cells that were left blank.
We focused on the following fields across all services and programs for calendar years 2023 and 2024: Persons Served, Seat Capacity, Persons Waitlist, Session Frequency, Program Intensity, Program Duration, License or Certification, and Annual Time Costs. For calendar year 2023, we found that 591 (6%) out of 9,568 total cells were left blank across these eight data fields, while for calendar year 2024, we found that 493 (5%) out of 10,024 total cells were left blank. The Program Intensity field accounted for the majority of blank entries, with 334 blank cells in 2023 and 257 in 2024. The other seven fields averaged 37 blank cells in 2023 and 34 in 2024. The graph below shows the number of blank cells in the PSI Matrix in the fields listed for all services and programs for calendar years 2023 and 2024.
Number of Blanks Cells Found in the PSI Matrix for Calendar Years 2023 and 2024
These formatting and data entry errors compromise the accuracy and consistency of the PSI Matrix. As the PSI Matrix is intended to provide standardized program information to key stakeholders for review and guidance, such discrepancies hinder effective analysis, contribute to inconsistent reporting across agencies, and reduce the overall reliability of the data. This can, in turn, impair the ability of stakeholders to make informed decisions based on the report.
Authoritative Guidance
The Commonwealth’s approved budget line item 8910-7110—Massachusetts Sheriffs’ Association Main Appropriation for fiscal years 2023 and 2024 states,
For the operation of the Massachusetts Sheriffs Association, Inc.; provided, that the sheriffs shall appoint persons to serve as executive director, assistant executive director, research director and other staff positions as necessary for the coordination and standardization of services and programs, the collection and analysis of data related to incarceration, recidivism and generation of reports, [and] technical assistance and training to ensure standardization in organization, operations and procedures.
Reasons for Issue
MSA did not have sufficient policies and procedures to ensure that data submitted in the PSI Matrix was standardized, complete, and properly formatted. While MSA officials stated that guidance is embedded within the PSI Matrix to assist sheriffs’ offices with data entry, they also noted that MSA has limited authority to enforce consistent or accurate completion of the templates. Additionally, the design of the PSI Matrix allows for too much flexibility in how data is entered, contributing to inconsistent reporting across sheriffs’ offices.
Recommendations
- MSA should strengthen its policies and procedures to ensure that data entered into the PSI Matrix is complete, properly formatted, and standardized across sheriffs’ offices. This should include implementing validation controls within the template itself to restrict improper entries (e.g., preventing text from being entered into numeric fields, requiring that mandatory fields be completed, and using dropdown options where appropriate).
- MSA should develop, document, and implement a process for reviewing and verifying submissions before final compilation to ensure accuracy and consistency.
- MSA should consider issuing more detailed guidance and offering regular training to sheriff’s office staff members who are responsible for data entry.
Auditee’s Response
To address the issues identified, the MSA will implement the following actions:
While the MSA acknowledges the auditor’s recommendations with respect to data entry practices, it should be noted that a formal MSA data reporting policy was created and implemented in 2022 in response to Audit 2021–1447.31. In addition, recognizing the scope and complexity of data reporting requirements, the MSA recruited and hired a Director of Analytics and Research. With this significant personnel investment, the MSA continues to enhance fidelity of data aggregation, analysis and reporting and we are appreciative of the previous audit recommendations contained in Audit 2021–1447.31.
The MSA continually endeavors to strengthen its internal policies to support standardized data entry practices across all Sheriff’s Offices. This includes issuing formal documentation outlining required formats, mandatory fields, and submission protocols. Formal accountability measures will be introduced to ensure compliance with these standards.
In response the MSA will deploy the following protocols:
The PSI Matrix template will be updated to include embedded validation controls for the 40,000+ data points collected. These controls will:
- Prevent text from being entered into numeric fields
- Require completion of mandatory fields
- Utilize dropdown menus for standardized selections
The MSA will implement a structured process for reviewing and verifying PSI Matrix submissions prior to final compilation. This process will include automated checks and manual reviews to ensure accuracy and consistency across all entries.
The MSA will enhance upon the current detailed guidance materials and conduct quarterly training sessions for Sheriff’s Office staff responsible for data entry. These sessions will focus on proper use of the PSI Matrix template, common data entry issues, and best practices for maintaining data integrity.
The MSA will engage existing committees and working groups to establish an ongoing dialogue to monitor data quality and provide support to the Sheriff’s Offices. This includes periodic audits, performance dashboards, and enhanced communication channels to address reporting challenges and shared reporting improvements.
Auditor’s Reply
Based on its response, MSA is taking measures to address our concerns regarding this matter. As part of our post-audit review process, we will follow up on this matter in approximately six months.
| Date published: | November 26, 2025 |
|---|