Audit of the Department of Unemployment Assistance Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Department of Unemployment Assistance.

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Department of Unemployment Assistance (DUA) for the period July 1, 2022 through June 30, 2024. When analyzing DUA’s call log data, we revised the audit period to August 26, 2024 through August 26, 2025 because, according to conversations with DUA officials, DUA only retains call log data for the latest 13-month period, and this revised timeframe allowed us to review call log data that DUA had in its possession.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. 

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
To what extent did DUA establish formal metrics for timely responses to claimant inquiries and requests made via telephone calls?	To an insufficient extent; see Findings 1 and 4
Did DUA process initial unemployment insurance (UI) claims in a timely manner by issuing each claimant’s first benefit payment in accordance with the federal Employment and Training Administration’s (ETA’s) “UI PERFORMS Core Measures”?	No; see Finding 2
Did DUA ensure that its vendor, Fast Enterprises LLC, resolved system issues that were reported during the implementation of the Employment Modernization and Transformation (EMT) project before the EMT system’s full implementation for claimant use, in accordance with the “EMT Production Support Procedures”?	Yes
To what extent did DUA provide a minimum level of in-person assistance for individuals applying for unemployment compensation at its regional offices in accordance with the language of Section 62A(a) of Chapter 151A of the General Laws that was in effect during the audit period?	To an insufficient extent; see Finding 3

To accomplish our audit objectives, we gained an understanding of the DUA internal control environment relevant to our objectives by reviewing applicable agency procedures and by interviewing DUA management. We evaluated the design and implementation of the internal controls relevant to our objectives. In addition, to obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures described below.

Claimant Inquiries and Requests

To determine to what extent DUA established formal metrics for timely responses to claimant inquiries and requests made via telephone calls, we interviewed DUA management and asked them whether they had developed any formal performance standards or metrics to evaluate call handling performance. We found that DUA did not establish performance standards or monitoring procedures for timely responses to claimant telephone inquiries. 

We then requested, from DUA management, a list of records of all the calls received during the audit period. DUA informed us that it could not provide the data for the period July 1, 2022 through June 30, 2024 because it only retains call log data for the latest 13-month period, and the requested log data for the audit period had been deleted. Because of this, we extended our audit period to include August 26, 2024 through August 26, 2025 and requested the call log data for that time range. See Finding 4 for more information.

We performed analytical procedures on 100% of the call log data from August 26, 2024 through August 26, 2025 to determine the queue time (which comprises the time that the caller spent waiting after automatic voice prompts to speak with an agent), the total queue time (which is the period comprising the queue time and the queue callback time), and the total wait time (which is the period comprising the queue time, the queue callback wait time, and the automatic voice prompt time) to determine the length of time it took for a claimant to receive customer assistance.

Further, we calculated the monthly average call handling time⁹ and the monthly call abandonment rate¹⁰ and compared the results to industry best practice.¹¹ To calculate the monthly average call handling time, we added together the total talk time, total hold time, and total after-call work time. (See Figure 3.) We then divided this sum by the total number of calls that DUA handled during each month in the revised audit period. Finally, we converted the average call handling time from seconds to minutes. 

Figure 3. Call Handling Time Formula

To calculate the monthly abandonment rate, we divided the number of abandoned calls by the total number of incoming calls received during each month in the revised audit period.

For this objective, we found certain issues during our testing; namely, that DUA did not establish policies that defined formal metrics for timely responses to claimant inquiries and requests made via telephone calls, as would be best practice. See Finding 1 and Finding 4 for more information.

First Benefit Payments

To determine whether DUA processed initial UI claims in a timely manner by issuing each claimant’s first benefit payments in accordance with ETA’s “UI PERFORMS Core Measures,” we obtained from DUA management a list of UI claims submitted during the audit period. We performed analytical procedures on 100% of the population to determine the total number of first benefit payments for intrastate and interstate UI claims categorized as paid within 14 days, paid within 35 days, and paid after 35 days. (See Table 4, found in Finding 2, for more information.) We calculated the monthly percentage of UI claims paid within each of these categories by dividing the UI claims paid for the category in question by the total number of UI claims for the month. We compared these percentages to the acceptable performance levels set by ETA.

In addition, we selected a random, nonstatistical¹² sample of 25 UI claims that experienced delayed payments and reviewed with DUA officials the claims in this sample to identify the underlying reasons for delayed payments. 

For this objective, we found certain issues during our testing; namely, that DUA did not meet the acceptable levels of performance for first benefit payment promptness required by ETA’s standards. See Finding 2 for more information.

EMT Project Issue Resolution

To determine whether DUA ensured that its vendor, Fast Enterprises LLC, resolved EMT system issues that were reported during the implementation of the EMT project before the EMT system’s full implementation for claimant use, in accordance with the “EMT Production Support Procedures,” we requested a list of solution requests (SQRs) generated during the audit period.

We selected a random, statistical¹³ sample of 60 out of 1,112 SQRs, using a 95% confidence level,¹⁴ a 0% expected error rate,¹⁵ and a 5% tolerable error rate.¹⁶ For each SQR in our sample, we calculated the number of days to completion by comparing the dates of submission and resolution. Additionally, we determined whether each SQR had been tested and confirmed that the SQR was reviewed and approved before implementation. 

For this objective, we found no significant issues during our testing. Therefore, we concluded that, based on our testing, DUA ensured that SQRs were resolved by its vendor in accordance with the “EMT Production Support Procedures” before the system’s full implementation for claimant use.

DUA In-Person Assistance

To determine to what extent DUA provided a minimum level of in-person assistance for individuals applying for unemployment compensation at its regional offices in accordance with the language of Section 62A(a) of Chapter 151A of the General Laws that was in effect during the audit period, we performed the procedures outlined below.

We interviewed DUA officials to determine to what extent DUA provided in-person assistance for individuals applying for UI at its four regional offices during the audit period. Additionally, we conducted an in-person site visit in Springfield, one of DUA’s four regional offices, to observe whether the office currently provides UI services.

For this objective, we found certain issues during our testing, namely that in-person service was not available at three of the four regional offices that were open during the audit period: Brockton, Lawrence, and Springfield. Instead, DUA only offered in-person assistance at the Boston Re-Employment Center, and this service was available by appointment only. See Finding 3 for more information.

We used a combination of statistical and nonstatistical sampling methods for testing, and we did not project the results of our testing to the corresponding populations. 

Data Reliability Assessment

DUA’s Online Portal

To assess the reliability of the UI claim data that we obtained from DUA’s online portal, we interviewed DUA officials who were knowledgeable about the data. We reviewed the System and Organization Control 2 Type 2 Reports¹⁷ covering the audit period, including information about access controls, security management, configuration management, contingency planning, and segregation of duties. We also tested whether DUA had access and account management controls and personnel member screenings and whether DUA conducted cybersecurity awareness training for its employees.

We tested the UI claim data for any duplicates, gaps, and worksheet errors (hidden rows, headers, and missing data elements). We reviewed the parameters used in generating the list from the system to verify whether we received the data expected. Further, we compared the total number of records in the dataset we received to the ETA 5159¹⁸ report that DUA submitted to the US Department of Labor during the audit period.

Call Center Software

To assess the reliability of the call log data that we obtained for the period August 26, 2024 through August 26, 2025 from the call center software, we interviewed DUA officials who were knowledgeable about the data. We reviewed the System and Organization Control 2 Type 2 Reports covering the period November 1, 2023 through October 31, 2024, including information about access controls, security management, configuration management, contingency planning, and segregation of duties. We also tested whether DUA conducted access and account management controls, personnel member screenings, and cybersecurity awareness training. We also checked the data for any duplicates, gaps, and worksheet errors (hidden rows, headers, and missing data elements). 

EMT System

To assess the reliability of the SQR data that we obtained from the EMT system, we interviewed DUA officials who were knowledgeable about the data. We reviewed the System and Organization Control 2 Type 2 Reports covering the audit period, including information about access controls, security management, configuration management, contingency planning, and segregation of duties. We also checked the data for any duplicates, gaps, and worksheet errors (hidden rows, headers, and missing data elements). 

Based on the results of the data reliability assessment procedures described above, we determined that the information we obtained during our audit was sufficiently reliable for the purposes of our audit.

9.    The average call handling time measures the total time that a DUA call center agent spends on each call, including the conversation itself and any related tasks such as data entry or follow-up.

10.    The call abandonment rate measures the percentage of customers who hung up before reaching a live agent.

11.    For the purposes of this audit report, because Massachusetts did not have established standards, we used the New York State call center standards as a benchmark for evaluating call center efficiency. These standards, which apply broadly to New York State agencies delivering public services, suggested an average call handling time of approximately four to six minutes and a call abandonment rate below 5% of the total number of incoming calls. 

12.    Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items are not similar enough, or there are specific items in the population that the auditors determine are appropriate to review.

13.    Auditors use statistical sampling to select items for audit testing when a population is large (usually over 1,000) and contains similar items. Auditors generally use a statistics software program to choose a random sample when statistical sampling is used. The results of testing using statistical sampling, unlike those from judgmental sampling, can usually be used to make conclusions or projections about entire populations.

14.    Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are representative of the population (parameter), expressed as a percentage. A 95% confidence level means that 95 out of 100 times, the statistics accurately represent the larger population.

15.    Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the auditor’s knowledge of factors such as prior audit results, the understanding of controls gained in planning, or a probe sample. In this case, we are assuming there are no errors in the data provided to us by the auditee.

16.    The tolerable error rate (which is expressed as a percentage) is the maximum error in the population that is acceptable while still using the sample to conclude that the results from the sample have achieved the objective.

17.    A System and Organization Control report is a report, issued by an independent contractor, about a service organization’s systems relevant to security, availability, processing integrity, confidentiality, and/or privacy.

18.    According to Section B of ETA’s UI Reports Handbook No. 401, an “ETA 5159 report contains monthly information on claims activities and on the number and amount of payments under State unemployment insurance laws (State UI) and Federal unemployment insurance laws for Federal workers (UCFE) and for ex-service members (UCX).”

 

Date published:	September 21, 2026