Audit of the Department of Children and Families Objectives, Scope, and Methodology

An overview of the purpose and process of auditing of the Department of Children and Families

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor (OSA) has conducted a performance audit of the Department of Children and Families (DCF) for the period January 1, 2014 through December 31, 2015.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient and appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer, the conclusion we reached regarding each objective, and where each objective is discussed in the audit findings.

Objective	Conclusion
Does DCF properly report critical incidents to the Office of the Child Advocate (OCA) when required?	No; see Findings 1, 2, and 3
Is the fatality review report completed, approved, and disseminated on request as required by Section 13.02(4) of Title 110 of the Code of Massachusetts Regulations (CMR)?	No; see Finding 4

 

To achieve our audit objectives, we gained an understanding of the internal controls we determined to be relevant to our audit objectives and tested the controls’ operating effectiveness over fatality review reports.

In a previous project, OSA assessed the reliability of the MassHealth data in the Medicaid Management Information System (MMIS), which is maintained by the Executive Office of Health and Human Services. As part of that assessment, OSA reviewed existing information, tested selected system controls, and interviewed knowledgeable agency officials about the data. Additionally, we performed validity and integrity tests on all claim data, including (1) testing for missing data, (2) scanning for duplicate records, and (3) looking for dates outside specific time periods. Based on the analysis conducted, we determined that the data obtained were sufficiently reliable for the purposes of this report.

We performed data validity and integrity tests on data received from DCF in various spreadsheets, including testing for missing data and scanning for duplicate records and hidden rows, columns, and formulas. We also ensured that data from MMIS were recorded in i-FamilyNet before including the data in our populations. Based on the analyses conducted, we determined that the data obtained were sufficiently reliable for the purposes of this audit. We relied on the hardcopy source documents for other data needs.

Unreported Incidents

During our audit, we extracted from MMIS information about 617 unique medical incidents. The information included medical billing and procedure codes involving children receiving DCF care. The incidents included physical assaults, injuries that resulted from the use of weapons, drug overdoses or poisonings, suicide attempts, fire-related injuries, and severe burns or bone fractures. We then matched the entire population of 617 incident details from MMIS to each child’s case file in i-FamilyNet. The purpose of this testing was to determine whether DCF was aware of these incidents and, if necessary, performed an investigation to determine the child’s current health and welfare.

We obtained a population of 781⁴ incidents that involved DCF children in residential facilities and homes from January 1, 2014 through December 31, 2015 and that in OSA’s opinion, based on information including medical billing and procedure codes, represented a serious bodily injury as outlined in the definition of a critical incident. These incidents included fatalities, near fatalities, suicide attempts, and gunshot wounds. From this population, we selected a nonstatistical judgmental sample of 40 incidents to determine whether DCF properly notified OCA of critical incidents as required by Sections 1 and 5 of Chapter 18C of the General Laws.

Additionally, we ran a query from MMIS that identified a population of 75 incidents involving children in DCF care during the audit period that might have required district attorney (DA) referrals according to DCF officials. Two of the 75 incidents occurred outside Massachusetts, but OSA sent the remaining 73 incidents to the appropriate DAs’ offices to obtain evidence of whether DCF had referred them as required by Section 51B of Chapter 119 of the General Laws.

Fatalities

We requested the complete list of children who died in DCF care during our audit period and should have had corresponding fatality review reports.⁵ Because we determined that controls over the fatality review reports were not designed effectively, we tested the full population of 73 fatalities that occurred during the audit period. We requested the fatality review reports to determine whether each report was completed on time,⁶ contained the required elements, and evidenced the DCF commissioner’s approval as required by 110 CMR 13. We also determined whether each completed fatality review report was made available to OCA on request in accordance with Section 6 of Chapter 18C of the General Laws.

Whenever sampling was used, we applied a nonstatistical approach, and as a result, we were not able to project our results to the entire populations.

4. This population of 781 incidents was obtained from DCF via i-FamilyNet and from MassHealth via the MMIS database.

5. To be considered reportable under DCF’s policy during the audit period, a fatality had to involve a child under the age of 18 who had, or whose family had, an open case or a case that had closed within six months before the date of the fatality.

6. According to 110 CMR 13.02(4), a fatality review report must be sent to the commissioner within 30 calendar days unless the death was determined to be of natural causes, in which case the report must be sent within 60 calendar days.

Date published:	December 7, 2017