Audit of the Commonwealth Corporation Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Commonwealth Corporation.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of Commonwealth Corporation (CommCorp) for the period July 1, 2020 through June 30, 2022.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
1. Did CommCorp ensure that YouthWorks grant recipients met the program participant eligibility and reporting requirements in Sections V and IX (for fiscal year 2021) and Sections III and VII (for fiscal year 2022) of CommCorp’s YouthWorks “Program Administration and Management Guide”?	No; see Finding 2
2. Did CommCorp track each YouthWorks program participant’s employment status after the completion of each program cycle to assess whether the program achieved its goal of helping program participants secure unsubsidized employment, in accordance with Section VIII (for fiscal year 2021) and Section VII (for fiscal year 2022) of CommCorp’s YouthWorks “Program Administration and Management Guide” and Section 116(b)(2)(A) of the Workforce Innovation and Opportunity Act of 2014?	No; see Finding 1
3. Did CommCorp ensure that its employees completed cybersecurity awareness training, in accordance with Sections 6.2.3 and 6.2.4 of the Executive Office of Technology Services and Security’s (EOTSS’s) Information Security Risk Management Standard IS.010?	No; see Finding 3

To accomplish our audit objectives, we gained an understanding of the aspects of CommCorp’s internal control environment relevant to our objectives by reviewing applicable policies and procedures and by interviewing officials at CommCorp, the Executive Office of Labor and Workforce Development, and four MassHire Workforce Development Boards.

In addition to the findings mentioned in the table above, CommCorp also had inadequate access controls for its YouthWorks database (see Findings 4a and 4b for more information).

To obtain sufficient, appropriate evidence to address our audit objectives, we performed the following procedures.

YouthWorks Program Participant Eligibility and Reporting

To determine whether CommCorp ensured that YouthWorks grant recipients met the program participant eligibility and reporting requirements in Sections V and IX (for fiscal year 2021) and Sections III and VII (for fiscal year 2022) of CommCorp’s YouthWorks “Program Administration and Management Guide,” we took the following actions.

CommCorp provided us with a list of all YouthWorks participant spots filled during the audit period (the population of which was 10,280), which originated from its YouthWorks database. We then selected a random, statistical⁷ sample of 100 participant spots filled, using a 95% confidence level,⁸ a 0% expected error rate,⁹ and a 5% tolerable error rate.¹⁰ We reviewed program participant case files to determine whether YouthWorks grant recipients maintained supporting documentation that verifies program participant eligibility and accurately recorded program participant information in the YouthWorks database. To verify program participants’ age eligibility as recorded in the list, we inspected certain documentation that YouthWorks grant recipients collected from each program participant (e.g., birth certificates, identification cards, driver’s licenses, and passports). To verify program participants’ income eligibility as recorded in the list, we inspected certain documentation that YouthWorks grant recipients collected from each program participant (e.g., self-attestation statements, parent or guardian pay stubs and tax returns, Supplemental Nutrition Assistance Program eligibility records, free or reduced-price school lunch eligibility records, and school records). To verify program participants’ risk-factor eligibility as recorded in the list, we inspected certain documentation that YouthWorks grant recipients collected from each program participant (e.g., school records, letters or records from the Executive Office of Health and Human Services, foster care letters or records, and court records). To verify program participants’ Social Security numbers as recorded in the list, we inspected certain documentation that YouthWorks grant recipients collected from each program participant (e.g., Social Security cards, school records, and employment forms). Lastly, we analyzed the entire list of YouthWorks participant spots filled to determine the quantity of participant spots filled that were missing a corresponding Social Security number in the YouthWorks database.

See Finding 2 for more information regarding the results of our testing of CommCorp’s adherence to YouthWorks program participant eligibility and reporting requirements.

YouthWorks Performance Assessment

We determined whether CommCorp tracked each YouthWorks program participant’s employment status after the completion of each program cycle to assess whether the program achieved its goal of helping program participants secure unsubsidized employment, in accordance with Section VIII (for fiscal year 2021) and Section VII (for fiscal year 2022) of CommCorp’s YouthWorks “Program Administration and Management Guide” and Section 116(b)(2)(A) of the Workforce Innovation and Opportunity Act of 2014. To do this, we took the following actions.

We reviewed the versions of CommCorp’s YouthWorks “Program Administration and Management Guide” that were in effect during the audit period. We interviewed the following CommCorp officials to discuss the intended purpose of YouthWorks: the director of youth employment, curriculum, and training; the associate director of YouthWorks; the vice president of legal affairs and administration; and the evaluation director. We also asked these officials whether CommCorp had developed any performance measures to evaluate the effectiveness of its YouthWorks program and, if the program did not achieve desired results, whether CommCorp management identified what CommCorp needed to do to improve the program’s performance. Lastly, we filtered the entire list of participant spots filled (10,280) by the “Any Employment Outcomes” data field to determine whether YouthWorks grant recipients tracked and reported on program participants’ success in securing unsubsidized employment after their time with YouthWorks.

See Finding 1 for more information regarding the results of our testing of CommCorp’s assessment of its YouthWorks program.

Cybersecurity Awareness Training

To determine whether CommCorp ensured that its employees completed cybersecurity awareness training, in accordance with Sections 6.2.3 and 6.2.4 of EOTSS’s Information Security Risk Management Standard IS.010, we interviewed CommCorp’s information technology and facilities manager and CommCorp’s vice president of legal affairs and administration to discuss whether CommCorp had established a cybersecurity awareness training program for its employees.

See Finding 3 for more information regarding the results of our testing of CommCorp’s cybersecurity awareness training.

Data Reliability Assessment

To determine the reliability of the list of YouthWorks participant spots filled that CommCorp provided to us, we took the following actions. We checked the list for consistent formatting, duplicate records, and missing data. We also ensured that the ages of program participants were within the required eligibility range. In addition, we tested information system general controls and policies related to access control over the YouthWorks database (for more information, see Findings 4a and 4b).

To test the accuracy of the list we received, we randomly selected a sample of 50 program participants from the list and compared program participant information to source documentation (e.g., YouthWorks program participant applications, birth certificates, identification cards, and school records) provided by YouthWorks grant recipients. To test the completeness of the data we received, we selected a different random sample of 50 program participant files we obtained from YouthWorks grant recipients and traced these back to the list of YouthWorks participant spots filled. Additionally, we reviewed the internal program participant lists and payroll records of three MassHire Workforce Development Boards and compared the total number of program participants found on them to the list of YouthWorks participant spots filled.

Based on the results of the data reliability assessment procedures described above, we determined that the information obtained for the audit period was sufficiently reliable for the purposes of our audit.

7.   Auditors use statistical sampling to select items for audit testing when a population is large (usually over 1,000) and contains similar items. Auditors generally use a statistics software program to choose a random sample when statistical sampling is used. The results of testing using statistical sampling, unlike those from judgmental sampling, can usually be used to make conclusions or projections about entire populations.

8.    Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are representative of the population (parameter), expressed as a percentage.

9.    Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the auditor’s knowledge of factors such as prior year results, the understanding of controls gained in planning, or a probe sample.

10.    The tolerable error rate (which is expressed as a percentage) is the maximum error in the population that is acceptable while still using the sample to conclude that the results from the sample have achieved the objective.

Date published:	July 9, 2024