Audit of the Office of the Child Advocate Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Office of the Child Advocate.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Office of the Child Advocate (OCA) for the period July 1, 2020 through March 31, 2022.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
1. Did OCA advise the public and government stakeholders about how the Commonwealth may improve its services to and for children and their families as required by Section 2(d) of Chapter 18C of the General Laws?	Not always; see Findings 1, 2, and 3
2. Did OCA develop formal policies and procedures for how it develops temporary cost share agreements as required by Section 2(e) of Chapter 18C of the General Laws?	No; see Other Matters
3. Did OCA meet the requirements of its interdepartmental service agreements (ISAs) relevant to the Commonwealth’s enacted budgetary language for fiscal years 2020, 2021, and 2022?	No; see Findings 4 and 5

To accomplish our audit objectives, we gained an understanding of OCA’s internal control environment relevant to our objectives by examining OCA’s internal control plan; by reviewing OCA’s seven ISAs relevant to the Commonwealth’s enacted budgetary language that were with the University of Massachusetts (UMass) Chan Medical School, the UMass Boston Behavioral Health Integrated Resources for Children (BIRCh) Project, and the Executive Office of Health and Human Services (EOHHS); and by interviewing OCA officials.

We reviewed and tested the operating effectiveness of the internal controls related to the ISAs by determining whether OCA’s director signed and dated the seven ISAs relevant to the Commonwealth’s enacted budgetary language for fiscal years 2020 through 2022.

To obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures described below.

External Communications

To determine whether OCA advised the public and government stakeholders about how the Commonwealth may improve its services to and for children and their families as required by Section 2(d) of Chapter 18C of the General Laws, we took the following actions.

We obtained OCA’s annual reports and annual Advisory Council meeting minutes published on its website—which, according to its internal control plan, OCA should post consistently for every fiscal year—and we determined whether any of these documents that should have been posted during the audit period were missing. We reviewed all two of OCA’s annual reports for the audit period to determine whether the content was related to advising the public and/or government stakeholders about how the Commonwealth may improve its services to and for children and their families.

Based on the results of our testing, we determined that OCA did not always advise the public and government stakeholders about how the Commonwealth may improve its services to and for children and their families during the audit period. See Findings 1, 2, and 3 for more information.

Cost Share Agreements

To determine whether OCA developed formal policies and procedures for how it develops temporary cost share agreements as required by Section 2(e) of Chapter 18C of the General Laws, we took the following actions:

• We requested OCA’s formal policies and procedures on developing cost share agreements that were in effect during the audit period.
• We also reviewed the unified planning team’s (UPT’s) annual reports for fiscal years 2020 and 2021 to verify that none of the UPT cases required OCA to intervene.

Based on the results of our testing, we determined that OCA did not develop formal policies and procedures for how it develops temporary cost share agreements. See Other Matters for more information.

Enacted Budget Responsibilities

To determine whether OCA met the requirements of its ISAs relevant to the Commonwealth’s enacted budgetary language for fiscal years 2020, 2021, and 2022, we took the following actions:

• We reviewed all seven ISAs that OCA created with UMass Chan Medical School, the UMass Boston BIRCh Project, and EOHHS for fiscal years 2020 through 2022.
• We obtained all expense transactions related to all seven ISAs from the Massachusetts Management Accounting and Reporting System (MMARS). We used this data to verify the population of these seven ISAs.
• We tested all 39 deliverables listed in these seven ISAs by reviewing supporting documentation, such as project reports, learning modules, survey forms, project timelines, and job descriptions, to ensure that OCA sufficiently satisfied the requirements of the ISAs.

Based on the results of our testing, we determined that OCA did not meet the requirements of its ISAs relevant to the Commonwealth’s enacted budgetary language during the audit period. See Findings 4 and 5 for more information.

Data Reliability Assessment

In 2018 and 2022, the Office of the State Auditor performed data reliability assessments for MMARS. These assessments focused on reviewing selected system controls, including access, security awareness, audit and accountability, configuration management, identification and authentication, and personnel security.

As part of the current audit, we performed direct queries of MMARS data using the Commonwealth Information Warehouse⁴ database for all OCA expenses incurred during the audit period. We tested this data for blank fields, duplicate records, and dates outside of the audit period. We then filtered the data to include only expenses related to the seven ISAs. In addition, we compared our query results obtained from the Commonwealth Information Warehouse to OCA’s query results.

Based on the results of the data reliability assessment procedures described above, we determined that the information we obtained from MMARS was sufficiently reliable for the purposes of our audit.

4.    The Commonwealth Information Warehouse contains budget, human resources, and payroll information, as well as financial transaction data from MMARS.

Date published:	October 7, 2024