• This page, Audit of the Department of Labor Standards—Division of Apprentice Standards Objectives, Scope, and Methodology, is   offered by
  • Office of the State Auditor

Audit of the Department of Labor Standards—Division of Apprentice Standards Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Department of Labor Standards—Division of Apprentice Standards.

Table of Contents

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Department of Labor Standards’ (DLS’s) Division of Apprentice Standards (DAS) for the period July 1, 2015 through December 31, 2017.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer, the conclusion we reached regarding each objective, and where each objective is discussed in the audit findings.

Objective

Conclusion

  1. Did DAS conduct performance evaluations of registered apprentice programs as required by Sections 29 and 30 of Title 29 of the Code of Federal Regulations (CFR) and Section 26 of Title 454 of the Code of Massachusetts Regulations (CMR)?

No; see Findings 1 and 2

  1. Do DAS apprentice agreements contain all required terms, conditions, and other information in accordance with state standards?

No; see Finding 3

 

We gained an understanding of internal controls we deemed significant to our audit objectives through inquiries and observations. We evaluated the design and effectiveness of controls over apprentice agreement registration and assessed whether these controls operated as management intended during our audit period. In addition, we performed the following procedures to obtain sufficient, appropriate audit evidence to address our audit objectives.

  • We examined program sponsor files to determine whether DAS conducted quality assurance assessments (QAAs) of all new (i.e., provisionally registered) programs at the end of their first year of operation in compliance with 454 CMR 26.03(3) during the audit period. We selected a nonstatistical random sample of 35 provisionally registered program sponsors out of a population of 239. We reviewed the files for originally prepared QAAs and field reports.
  • We examined program sponsor files to determine whether QAAs of all permanently registered programs were conducted at least once every five years, as required by 454 CMR 26.03(4), during the audit period. We selected a nonstatistical random sample of 40 permanently registered program sponsors out of a population of 496. We reviewed the files for originally prepared QAAs and field reports.
  • We examined program sponsor files to determine whether DAS conducted equal employment opportunity (EEO) compliance reviews of all apprentice programs registered with DAS that had five or more apprentices as required by 29 CFR 30 during the audit period. We tested the entire population of 90 program sponsors that were required to comply with this regulation. We reviewed the files for the originally prepared compliance review checklists (29 CFR 30).
  • We selected a statistical random sample of apprentice agreements, using a 95% confidence level and a tolerable error rate of 10%. We sampled 30 apprentice agreements from the 7,437 total active agreements from the audit period. To determine whether the apprentice agreements were executed in accordance with 454 CMR 26.06, we inspected them and each apprentice’s file to determine whether they contained the following:
  • the occupational training approaches and required training hours and/or skills for time-based, competency-based, or hybrid training programs
  • the total number of hours of hands-on training required of the apprentice for completion of a time-based program
  • a description of the work processes in which the apprentice would participate and an approximation of the proportion of time to be spent in each major process
  • the contact information of the individual/s designated by the apprentice program sponsor to receive, investigate, and resolve complaints and concerns raised by apprentices about the apprentice program
  • an acknowledgment that the apprentice had been advised that DAS’s deputy director is also available to receive, investigate, and resolve complaints or concerns
  • the names and signatures of the sponsor’s representatives
  • the number of hours of instruction that the apprentice would receive in technical subjects related to his/her occupation
  • a statement of the graduated scale of wages to be paid to the apprentice
  • an acknowledgement that the apprentice had been advised that the apprentice program was governed by Sections 11E through 11L of Chapter 23 of the General Laws and 454 CMR 26, which describe the purpose and goals of the apprentice program
  • the amount of credit the apprentice had already earned toward program completion from previous related job training and instruction
  • an acknowledgment that the apprentice had received a complete copy of the apprentice agreement and a copy of the “Apprentice Rights and Responsibilities”

Whenever nonstatistical sampling was used, we could not project our results to the population.

Data Reliability

We determined the reliability of the DLS Apprentice Standards database by (1) performing electronic testing of the data elements, including testing for missing data, either entire missing records or missing values in key data elements, duplicate records, and values outside a designated range; (2) reviewing existing information about the data and the system that produced them; and (3) interviewing agency officials who were knowledgeable about the data. We verified the completeness and accuracy of three tables in the database: the Permanent Program table, the EEO Compliance table, and the Apprentice Agreement table.

We ensured the completeness of the Permanent Program table of active sponsors by judgmentally selecting 20 sponsorship approval letters and apprentice agreements from a DAS file cabinet and then verifying that the information they contained corresponded to the program name, sponsor number, and status in the Permanent Program table. In addition, we randomly selected 20 sponsors from the Permanent Program table and traced the program name, sponsor number, and status to the hardcopy sponsorship approval letter and apprentice agreement on file.

We ensured the completeness of the EEO Compliance table of active sponsors with at least five active apprentices by judgmentally selecting 10 sponsor files from a DAS file cabinet and verifying that the total number of apprentice agreements corresponded to the number listed in the EEO Compliance table. In addition, we randomly selected 10 sponsors from the EEO Compliance table and traced them to the hard copies of the apprentice agreements on file.

We ensured the completeness of the Apprentice Agreement table of apprentice agreements by judgmentally selecting 20 apprentice agreements from a DAS file cabinet and verifying that the apprentice identification number, apprentice name, sponsor name, and date of hire corresponded to the data in the Apprentice Agreement table. In addition, we randomly selected 20 apprentice agreements from the Apprentice Agreement table and traced the apprentice identification numbers, apprentice names, sponsor names, and dates of hire to the hardcopy apprentice agreements on file.

We determined that the three data tables from the DLS Apprentice Standards database were sufficiently reliable for the purposes of this audit.

Date published: August 15, 2019

Help Us Improve Mass.gov  with your feedback

Please do not include personal or contact information.
Feedback