Audit of the Department of Labor Relations Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Department of Labor Relations.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Department of Labor Relations (DLR) for the period July 1, 2020 through June 30, 2022.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
Did DLR meet its case stage timeline goals for closed cases in accordance with the DLR Case Processing Goals document?	No; see Finding 1
Did DLR submit its annual reports to the Legislature as required by Section 9U of Chapter 23 of the General Laws?	Yes
Did DLR ensure that its employees completed cybersecurity awareness training in accordance with Sections 6.2.3 and 6.2.4 of the Executive Office of Technology Services and Security’s (EOTSS’s) Information Security Risk Management Standard IS.010?	Yes

To accomplish our audit objectives, we gained an understanding of the aspects of DLR’s internal control environment relevant to our objectives by reviewing applicable policies and procedures and by interviewing DLR management.

To obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures described below.

Case Stage Timeline Goals

To determine whether DLR met its case stage timeline goals for closed cases in accordance with the DLR Case Processing Goals document, we took the following actions. We gathered the following information regarding closed cases, all of which originated from DLR’s case management database:

• one Microsoft Excel spreadsheet of all 1,163 cases that were closed during the audit period (which we downloaded directly from the case management database) and
• one Microsoft Excel spreadsheet of all 48,075 unique events that related to the 1,163 cases that were closed during the audit period (which DLR provided to us).

We then took these two spreadsheets and merged them to create a list of 9,251 combined case/event data points for cases closed during the audit period.

For each of the 1,163 cases closed during the audit period, we identified a unique starting event and a unique concluding event using the list of 9,251 combined closed case/event data points. Then, we compared these events to 23 of the case stage timeline goals¹ from the DLR Case Processing Goals document by taking the following actions (based on the two steps from this document):

• First, we calculated the actual number of days between each corresponding starting event and concluding event and compared these actual numbers to the number of days allowed according to the DLR Case Processing Goals document.
• Second, we calculated the actual percentages for the combined closed case/event data points that met the corresponding case stage timeline goals and compared these actual percentages to the corresponding case stage timeline goal percentages documented in the DLR Case Processing Goals document.

See Finding 1 for information regarding issues we identified with DLR meeting its case stage timeline goals.

Annual Reports

To determine whether DLR submitted its annual reports to the Legislature as required by Section 9U of Chapter 23 of the General Laws, we took the following actions. We requested, and DLR provided us with, copies of the annual reports DLR filed for each fiscal year within the audit period. We requested, and DLR provided us with, copies of emails related to the submissions of these annual reports to the Legislature. We then inspected these emails for the dates they were sent to determine whether DLR submitted the annual reports to the Legislature within 120 days of the close of the fiscal years.

We noted no exceptions in our testing; therefore, we concluded that DLR submitted its annual reports to the Legislature during the audit period.

Cybersecurity Awareness Training

To determine whether DLR ensured that its employees completed cybersecurity awareness training in accordance with Sections 6.2.3 and 6.2.4 of EOTSS’s Information Security Risk Management Standard IS.010, we took the following actions.

We requested, and DLR provided us with, a list of all employees who were active during the audit period. This list of DLR employees contained all 32 employees, which included 28 active for the whole audit period and 4 terminated who were active during a portion of the audit period. For all 32 employees, we reviewed cybersecurity awareness training records that we obtained from DLR employees and Executive Office of Labor and Workforce Development management. These records came from the two cybersecurity awareness training platforms that DLR used during the audit period. We also inspected completion transcripts and reminder emails from both cybersecurity awareness training platforms to determine whether each employee completed cybersecurity awareness training in a timely manner.

We noted no exceptions in our testing; therefore, we concluded that DLR ensured that its employees completed cybersecurity awareness training during the audit period.

Data Reliability Assessment

Case Management Database

To determine the reliability of the information in the case management database, we tested certain information system controls (e.g., account management, security training, personnel screening, user identification and authentication, session locks, and unsuccessful login attempts). To test the accuracy of the Microsoft Excel spreadsheet of the 1,163 cases that were closed during the audit period, we selected a judgmental sample² of 20 closed cases from the list and compared its information (e.g., the case charging party,³ the case number,⁴ the case starting date, and the case closing date) to the information in source documents that were scanned and stored in the case management database.

To test the completeness of the data we received from DLR’s case management database, we selected a judgmental sample of 20 source documents (e.g., emails and settlement agreements) from DLR’s case management database and traced the information in these documents back to the information (e.g., the case charging party, the case number, the case starting date, and the case closing date) recorded in the Microsoft Excel spreadsheet of 1,163 closed cases.

To test the reliability of the list of 9,251 combined case/event data points for cases closed during the audit period, we selected a judgmental sample of 20 events from the list, and compared the information corresponding to them (e.g., the case number, the event starting date, and the event closing date) to source documents (e.g., emails and case notes), doing so by accessing the DLR case management database directly. Furthermore, we judgmentally selected 20 source documents and traced the information in them (e.g., the case number, the case starting date, and the case closing date) to the information in the list of 9,251 combined case/event data points.

Cybersecurity Awareness Training

To determine the reliability of the cybersecurity awareness training records, we verified that the list of all 32 employees who were active during the audit period was complete and accurate by tracing the names on this list to the timesheets for all employees who were active for two weeks (that we judgmentally selected) out of each year of the audit period. We also reviewed System and Organization Control reports⁵ for both of the cybersecurity awareness training platforms that DLR used during the audit period and ensured that an independent contractor performed information system control tests (e.g., security management, access control, configuration management, segregation of duties, and contingency planning) on the platforms.

Based on the results of the data reliability assessment procedures described above, we determined that the information obtained was sufficiently reliable for the purposes of our audit.

1.    We only tested 23 of the case state timeline goals listed in the DLR Case Processing Goals document because some of the possible goals were not represented by cases that were closed during the audit period.

2.   Auditors use judgmental sampling to select items for audit testing when a population is very small, the population items are not similar enough, or there are specific items in the population that the auditors want to review. Auditors use their knowledge and judgment to select the most appropriate sample. For example, an auditor might select items from areas of high risk. The results of testing using judgmental sampling cannot be used to make conclusions or projections about entire populations; however, they can be used to identify specific issues, risks, or weaknesses.

3.    The charging party is the name of the entity that filed the claim.

4.   The case number is a unique identifier that DLR assigns to each case for identification purposes.

5.   A System and Organization Control report is a report on controls about a service organization’s systems relevant to security, availability, processing integrity, confidentiality, or privacy issued by an independent contractor.

Date published:	October 30, 2024