Audit of the Military Division of the Commonwealth Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Military Division of the Commonwealth.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Military Division of the Commonwealth (MIL) for the period July 1, 2019 through December 31, 2021.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
Did MIL administer reimbursement for tuition and fee waivers as required by the state budget for fiscal years 2020, 2021, and 2022 and the policies of the Massachusetts Army and Air Force National Guard Tuition and Fee Reimbursement Program?	No; see Finding 1
Did MIL perform the terms of the contracts to which it agreed in its interdepartmental service agreements (ISAs) with the Executive Office for Administration and Finance (A&F) and the Department of Elementary and Secondary Education (DESE) for missions and purchases related to COVID-19?	No; see Finding 2

 

To achieve our objectives, we gained an understanding of the internal controls relevant to the objectives by interviewing MIL officials, as well as by reviewing MIL’s internal control plan and agency policies and procedures.

Scope Impairments: Reimbursements for the Massachusetts Army and Air Force National Guard Tuition and Fee Reimbursement Program

Section 9.12 of the United States Government Accountability Office’s Government Auditing Standards states, “Auditors should . . . report any significant constraints imposed on the audit approach by information limitations or scope impairments.” During our audit of MIL, we determined that we could not rely on the information in the Massachusetts National Guard Tuition System, which was necessary to conduct this audit, because of multiple control deficiencies regarding the system’s data (see Finding 1).

To determine whether MIL administered reimbursements for tuition and fee waivers according to its final budgetary language for fiscal years 2020, 2021, and 2022 and the policies of the Massachusetts Army and Air Force National Guard Tuition and Fee Reimbursement Program, we interviewed agency officials who were knowledgeable about the tuition program and MIL’s Massachusetts National Guard Tuition System. We interviewed employees from the information system company that hosts the system. We reviewed the tuition system data dictionary and the user manual. We obtained data from the tuition system. We also performed a gap analysis for the certificate of eligibility identification numbers and found that there were 464 certificate transactions that had been deleted from the tuition system. We identified at least 450 occurrences in which MIL approved two certificates for a Guard member for a single semester and at least 30 occurrences in which MIL approved three certificates for a Guard member for a single semester. We followed up with MIL management on each of these specific issues.

Further, we reviewed the Massachusetts National Guard Tuition System’s controls by reviewing the System and Organization Control reports² of the external information system company hosting the tuition system.

The information from the Massachusetts National Guard Tuition System included lists of the state colleges and universities that Guard members attended, the estimated costs of tuition and fees, the names of the Guard members, the Guard members’ certificates of eligibility, the semesters in which the Guard members attended, the number of credits required for their desired courses, and the approval status of the certificates. We interviewed knowledgeable MIL officials about the system and tested the information for duplicate records, dates outside of our audit period, and gaps in certificate identification numbers (related to the 464 certificate transactions that had been deleted from the tuition system). We vouched³ a random sample of 25 records from the tuition system to original source documents (such as the hardcopy, approved certificates and reimbursement forms that state colleges or universities sent to MIL). We selected a random sample of 25 certificates / reimbursement forms and traced these source documents to the information in the tuition system. We determined from our tracing and vouching that the information obtained from the tuition system for our audit period was not sufficiently reliable for our audit work. (See Finding 1 for additional information.)

COVID-19 Funding

To determine whether MIL complied with the terms of its contracts with other state agencies, we obtained the ISAs between MIL and A&F and between MIL and DESE for deployment of uniformed personnel for missions related to COVID-19. We interviewed MIL officials to understand the management of the ISAs, the procedures used to document the performance of ISA services, and the deployment of National Guard members to address COVID-19.

To verify that MIL spent the COVID-19 funding it received from A&F and DESE to satisfy the terms of the ISAs, we acquired a dataset of expenditures related to the ISAs from the Massachusetts Management Accounting and Reporting System (MMARS). We then compared the budget from each Attachment B to the actual expenditures made from MMARS data. As part of this comparison, we verified that the object codes (which indicate the type of goods or services) in the Attachment B were used in the MMARS data and that the amount expended in MMARS did not exceed the budgets in the ISAs’ Attachment Bs. In fiscal year 2020, A&F and MIL entered into an ISA with a budgeted amount of $15,220,000 and later into an amended ISA with an increased budget of $26,656,216. The two parties entered into another ISA in fiscal year 2021 with a budget of $6,572,645. In fiscal year 2022, DESE and MIL entered into an ISA with a budgeted amount of $3,385,973 and later into an amended ISA with a reduced budget of $3,091,165. Please see Appendix B for the five Attachment Bs from MIL’s ISAs with A&F and DESE.

To determine whether MIL fulfilled the performance terms of the ISAs, we requested documentation that MIL had performed the actions required as outlined in the ISAs. However, MIL was unable to provide documentation to support that it had performed the actions outlined in the ISAs. Some of the actions outlined in the ISA with A&F included ensuring the safety and readiness of emergency service personnel, supporting the increased demand of medical services and COVID-19 testing sites, and providing logistics for personal protective equipment and cleaning supplies for local emergency service and government needs. According to the ISA with DESE, MIL would provide 90 Guard members to serve as school bus drivers in Chelsea, Lawrence, Lowell, and Lynn. Please see Appendix A for the Attachment As from MIL’s ISAs with A&F and DESE.

We requested the biweekly reports MIL sent to A&F (approximately 30 reports during the audit period) to determine whether MIL could demonstrate that it complied with the reporting requirements of the ISAs between these two agencies. However, MIL was unable to provide evidence that it submitted the biweekly reports to A&F. (See Finding 2 for additional information.)

We received and reviewed the one expenditure report that MIL was required to send, and sent, to DESE in accordance with the ISA between these two agencies.

Data Reliability Assessment

In 2018 and 2022, the Office of the State Auditor performed data reliability assessments of MMARS. The assessments focused on reviewing selected system controls (access, security awareness, audit and accountability, configuration management, identification and authentication, and personnel security).

MIL uses MMARS to approve and track payments to the state colleges or universities for the Massachusetts Army and Air Force National Guard Tuition and Fee Reimbursement Program. We performed a data reliability assessment by reviewing MIL’s access control policy and procedures. We tested whether MIL employees had received security awareness training and background checks before they were given access to MMARS. We tested whether MIL had evidence of periodic supervisory reviews of access to MMARS for its employees.

Based on the results of the procedures described above, we determined that the data we obtained from MMARS was sufficiently reliable for the purpose of our second objective.

2.   A System and Organization Control report examines controls over an organization’s systems relevant to security, availability, processing integrity, confidentiality, or privacy issued by an independent contractor.

3.   Vouching is the inspection of supporting documentation to corroborate data.

Date published:	January 10, 2024