• This page, Audit of the Chelsea Soldiers’ Home Objectives, Scope, and Methodology, is offered by
  • Office of the State Auditor

Audit of the Chelsea Soldiers’ Home Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Chelsea Soldiers’ Home.

Table of Contents


In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Chelsea Soldiers’ Home (CHE) for the period July 1, 2017 through June 30, 2019. For our audit of nursing overtime, we narrowed our scope to include only instances of premium overtime.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is our audit objective, indicating the question we intended our audit to answer, the conclusion we reached regarding the objective, and where the objective is discussed in the audit findings.



  1. Did CHE ensure that the need for Nursing Department premium overtime was documented and approved in accordance with CHE policy?

No; see Findings 1 and 2


To achieve our objectives, we gained an understanding of the internal controls we deemed significant to our audit objectives by reviewing Section 7.2 of the collective bargaining agreement (CBA) for Unit 7, Section 7.2 of the CBA for Unit 2, and agency policies and procedures, as well as conducting inquiries with CHE management. We evaluated the design of controls over CHE’s management review and approval of overtime.

We performed the following procedures to obtain sufficient, appropriate audit evidence to address the audit objective.

Data from the Human Resources Compensation Management System

We performed an inquiry to ask CHE management to provide us with Overtime Request Forms, which detail the need and prior authorization for overtime. However, CHE management told us that during our audit period, nurses did not complete them as required by its policy and procedures.

To facilitate our planned substantive tests, we queried a data set from the Human Resources Compensation Management System (HR/CMS) that included all Unit 2 and Unit 7 nursing staff members who were paid the premium overtime rate. The data included all premium overtime transactions, dates worked, numbers of hours, time reporting codes,1 overtime descriptions, employee names, employee identification numbers, job titles, event types, pay cycle start dates, pay cycle end dates, and dollar amounts.

We asked CHE management to provide us with the Daily Staffing Schedules and Weekly Overtime Summaries, which detail the approval for overtime, for our audit period. However, CHE management told us that during our audit period, 17 months of records (for the period August 1, 2017 through December 22, 2018) were shredded to create space.

We filtered data from HR/CMS by overtime event type to eliminate any non-associated overtime payments.2 We then divided the 20,664 overtime transactions (which totaled 90,623 worked hours and $3,320,033) into two strata. Stratum 1 is overtime with supporting documentation, which is 32% of the total amount paid (totaling 6,545 transactions, 28,577 worked hours, and $1,051,995). Stratum 2 is overtime without supporting documentation, which is 68% of the total amount paid (totaling 14,119 transactions, 62,046 worked hours, and $2,268,038).

For premium overtime testing, we took a statistical random sample of 102 overtime transactions out of a population of 6,545, using a confidence level of 95% with a 20% tolerable error rate and 50% expected error rate.3 We inspected Weekly Overtime Summary Forms to determine whether the director of nursing had approved the premium overtime hours. We inspected Daily Staffing Schedules to determine whether nursing staff members were listed as having received overtime.

Forty-Hour-Test and Eight-Hour-Test Methodology

We queried HR/CMS for the audit period and conducted an analysis of payroll data to identify instances of premium overtime paid to nurses who had not fulfilled the CBA criteria to be paid at the premium overtime rate. We examined all 20,664 overtime transactions to determine whether employees whose work week was less than 40 hours were paid at their regular rates before qualifying for the overtime rate. We also determined whether employees whose regular workday was more than eight hours were compensated at the rate of time and one-half their regular hourly rates for authorized overtime.

Incremental Overtime

Nursing staff members can incur incremental overtime by clocking in early for a shift, clocking out after the scheduled end of a shift, or not clocking out for an unpaid meal break. We conducted an analysis of overtime payroll data and determined that the majority of this overtime was less than 2 hours. We extracted all incremental overtime of 2 hours or less that was worked the same day as 8 or more regular hours and identified 9,804 (totaling $223,136) overtime transactions for the audit period.

Data Reliability Assessment

CHE uses HR/CMS to record employees’ time and attendance. We tested selected system controls (access controls, security management, configuration management, contingency planning, and segregation of duties). Further, we ensured the completeness and accuracy of the data from HR/CMS by judgmentally selecting 20 nurses from HR/CMS and verifying that they were on an employee list provided by CHE. Also, we judgmentally selected 20 nurses from the employee list to trace the employee identification numbers to HR/CMS data to determine whether they were employees. Based on the results of our data reliability assessments, we determined that the information obtained for our audit period was sufficiently reliable for the purpose of our audit objectives.

1.    According to the Massachusetts Human Resources Division website, “A time reporting code is a unique identifier to specify the type of time used (e.g., sick, vacation, overtime, etc.).”

2.    Non-associated overtime payment types include regular, sick (payment during leave under the Family and Medical Leave Act for the worker’s own illness, including pregnancy-related illness), and personal (payment for time off for family issues or personal needs).

3.    The confidence level is the measure of how confident we can be that our results reflect what we would have obtained if the entire population had been tested. The expected error rate is the anticipated rate of occurrence of the overtime not being authorized. The tolerable error rate is the maximum error in the population we would be willing to accept and still conclude that the result from the sample had achieved the audit objective.

Date published: March 30, 2021

Help Us Improve Mass.gov with your feedback