Audit of the Human Resources Division Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Human Resources Division.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Human Resources Division (HRD) for the period July 1, 2020 through June 30, 2022. When designing the audit plan for affirmative action and diversity plan annual progress reports, we extended the audit period to December 31, 2022 to include the annual progress reports due in August 2022.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
1. Did HRD ensure that state agencies submitted an affirmative action plan, in accordance with Section 3.1(A) and bullet points 6 and 10 of Section 7 of HRD’s Guidelines for Implementing Executive Order 592?	No; see Finding 1
2. Did HRD ensure that state agencies submitted a diversity plan, in accordance with Section 3.1(B) and bullet points 6 and 10 of Section 7 of HRD’s Guidelines for Implementing Executive Order 592?	No; see Finding 1
3. Did HRD ensure that state agencies submitted an annual progress report that detailed actions these state agencies took and progress these state agencies made toward meeting their affirmative action and diversity goals, in accordance with Section 3.2 of HRD’s Guidelines for Implementing Executive Order 592?	No; see Findings 2 and 3

To accomplish our audit objectives, we gained an understanding of the aspects of HRD’s internal control environment relevant to our objectives by reviewing applicable policies and procedures and by interviewing HRD management. We evaluated the design of controls over HRD management’s review and approval of affirmative action plans, diversity plans, and the annual progress reports for each plan.

To obtain sufficient, appropriate evidence to address our audit objectives, we performed the following procedures.

Affirmative Action Plans

To determine whether HRD ensured that state agencies submitted an affirmative action plan, in accordance with Section 3.1(A) and bullet points 6 and 10 of Section 7 of HRD’s Guidelines for Implementing Executive Order 592, we took the following actions.

• We selected a sample of 20 state agencies by taking the following actions.
  • From the population of 62 state agencies that were operating during the audit period, we selected a targeted, nonstatistical sample of 13 state agencies that submitted affirmative action plans after December 31, 2021. (We chose this target date to capture any plans that were submitted three or more months late.)
  • From the remaining population of 49 state agencies that were operating during the audit period, we selected a judgmental,⁵ nonstatistical sample of 7 state agencies that submitted affirmative action plans before our chosen target date of December 31, 2021.
• We inspected each of the 20 affirmative action plans corresponding to the state agencies in our sample to determine whether each state agency developed and submitted its plan on time, in accordance with Section 3 of HRD’s Guidelines for Implementing Executive Order 592.
• We inspected each of the 20 affirmative action plans corresponding to the state agencies in our sample to determine whether each plan included the following required information, in accordance with Section 3.1(A) of HRD’s Guidelines for Implementing Executive Order 592:

                  (1)  Introduction and Policy Statement;

                  (2)  Designation of Affirmative Action and Equal Opportunity Designations and                                    responsibilities;

                  (3)  Organizational Profile;

                  (4)  Identification of Problem areas;

                  (5)  Action-Oriented Programs;

                  (6)  Internal Audit and Reporting Systems; and

                  (7)  Dissemination and Communication.

• We inspected each of the 20 affirmative action plans corresponding to the state agencies in our sample to determine whether HRD’s Office of Diversity and Equal Opportunity (ODEO) reviewed each plan and either approved it, returned it for amendment, or rejected it, in accordance with bullet point 6 of Section 7 of HRD’s Guidelines for Implementing Executive Order 592.
• We inspected follow-up emails that ODEO sent to state agencies that were noncompliant in terms of meeting their affirmative action plan submission deadline to determine whether ODEO sent follow-up communications, in accordance with ODEO’s “Accountability Policy for Affirmative Action Plans for Executive Branch Agencies,” and whether ODEO implemented any remedial courses of action, in accordance with bullet point 10 of Section 7 of HRD’s Guidelines for Implementing Executive Order 592.
• We interviewed HRD management about what remedial courses of action HRD had taken with noncompliant state agencies, if any.

See Finding 1 for more information regarding the results of our testing of affirmative action plans.

Diversity Plans

To determine whether HRD ensured that state agencies submitted a diversity plan, in accordance with Section 3.1(B) and bullet points 6 and 10 of Section 7 of HRD’s Guidelines for Implementing Executive Order 592, we took the following actions.

• We selected a sample of 20 state agencies by taking the following actions. (Note that this sample is different from that discussed in “Affirmative Action Plans” in the Audit Objectives, Scope, and Methodology section of this report.)
  • From the population of 62 state agencies that were operating during the audit period, we selected a targeted, nonstatistical sample of 12 state agencies that submitted diversity plans after December 1, 2021. (We chose this target date to capture any plans that were submitted two or more months late.)
  • From the remaining population of 48 state agencies that were operating during the audit period, we selected a judgmental, nonstatistical sample of 8 state agencies that submitted diversity plans before our chosen target date of December 1, 2021.
• We inspected each of the 20 diversity plans corresponding to the state agencies in our sample to determine whether each state agency developed and submitted its plan on time, in accordance with Section 3 of HRD’s Guidelines for Implementing Executive Order 592.
• We inspected each of the 20 diversity plans corresponding to the state agencies in our sample to determine whether each plan included the following required information, in accordance with Section 3.1(B) of HRD’s Guidelines for Implementing Executive Order 592: “an agency vision, agency mission statement, and diversity and inclusion goals with written strategy and measures.”
• We inspected each of the 20 diversity plans corresponding to the state agencies in our sample to determine whether ODEO reviewed each plan and either approved it, returned it for amendment, or rejected it, in accordance with bullet point 6 of Section 7 of HRD’s Guidelines for Implementing Executive Order 592.
• We inspected follow-up emails that ODEO sent to state agencies that were noncompliant in terms of meeting their diversity plan submission deadline to determine whether ODEO sent follow-up communications, in accordance with ODEO’s “Accountability Policy for Affirmative Action Plans for Executive Branch Agencies,” and whether ODEO implemented any remedial courses of action, in accordance with bullet point 10 of Section 7 of HRD’s Guidelines for Implementing Executive Order 592.
• We asked HRD management about what remedial courses of action HRD had taken with noncompliant state agencies, if any.

See Finding 1 for more information regarding the results of our testing of diversity plans.

Annual Progress Reports

To determine whether HRD ensured that state agencies submitted an annual progress report that detailed actions these state agencies took and progress these state agencies made toward meeting their affirmative action and diversity goals, in accordance with Section 3.2 of HRD’s Guidelines for Implementing Executive Order 592, we took the following actions.

Annual Progress Reports for Affirmative Action Plans

• We extended the audit period to December 31, 2022 to include the annual progress reports due on August 30, 2022, because the annual progress reports that included fiscal year 2022 affirmative action and diversity plan goals (which were submitted during the audit period) were due on this date.
• From the population of 62 state agencies that were operating during the audit period, we tested the same 20 state agencies that submitted affirmative action plans from our sample discussed in “Affirmative Action Plans” in the Audit Objectives, Scope, and Methodology section of this report. For these 20 state agencies, we reviewed their annual progress reports for their affirmative action plans from each of the fiscal years in the audit period (2020, 2021, and 2022), resulting in our review of 60 annual progress reports for affirmative action plans.
• We inspected each of the 60 annual progress reports for affirmative action plans from our sample to determine whether each state agency submitted its annual progress report on or before August 30 for each respective fiscal year.
• Additionally, we inspected the 20 annual progress reports for affirmative action plans for fiscal year 2022 to determine whether each state agency developed goals to address any areas of concern that each state agency self-identified in its affirmative action plans.

See Findings 2 and 3 for more information regarding the results of our testing of annual progress reports for affirmative action plans.

Annual Progress Reports for Diversity Plans

• We extended the audit period to December 31, 2022 to include the annual progress reports due on August 30, 2022, because the annual progress reports that included fiscal year 2022 affirmative action and diversity plan goals (which were submitted during the audit period) were due on this date.
• From the population of 62 state agencies that were operating during the audit period, we tested the same 20 state agencies that submitted diversity plans from our sample discussed in “Divesrsity Plans” in the Audit Objectives, Scope, and Methodology section of this report. For these 20 state agencies, we reviewed their annual progress reports for their diversity plans from each of the fiscal years in the audit period (2020, 2021, and 2022), resulting in our review of 60 annual progress reports for diversity plans.
• We inspected each of the 60 annual progress reports for diversity plans from our sample to determine whether each state agency submitted its annual progress report on or before August 30 for each respective fiscal year.
• Additionally, we inspected the 20 annual progress reports for diversity plans for fiscal year 2022 to determine whether each state agency developed goals to address any areas of concern that each state agency self-identified in its diversity plan.

See Finding 2 for more information regarding the results of our testing of annual progress reports for diversity plans.

We used nonstatistical sampling methods for testing and therefore did not project the results of our testing to any population.

Data Reliability Assessment

Affirmative Action Plans and Diversity Plans

To determine the reliability of the Microsoft Excel spreadsheet that ODEO used to track each of the affirmative action plans and the diversity plans for fiscal years 2022 and 2023 (which ODEO received during the audit period), we analyzed the spreadsheet for hidden information (i.e., data, columns, rows, worksheets, and other hidden information resulting from active filters). We assessed the reliability of the data obtained from ODEO’s web-based workflow tracking tool (i.e., the submission and approval dates for the affirmative action plans, diversity plans, and annual progress reports) by conducting interviews with ODEO staff members who were knowledgeable about the data and we tested certain general information system controls (e.g., access controls) related to this data. To confirm that the spreadsheet had a complete list of the state agencies that were required to complete affirmative action plans and diversity plans, we compared the state agencies from the spreadsheet to a list of state agencies found on the Massachusetts State Government Organizational Chart as of July 2021, which is available publicly on the Mass.gov website. To ensure the accuracy of the spreadsheet, we traced a random sample of 10 affirmative action plans and 10 diversity plans from the spreadsheet back to the source documents (i.e., the affirmative action plans and diversity plans).

Annual Progress Reports

To determine the reliability of the Microsoft Excel spreadsheets that ODEO used to track each annual progress report for affirmative action plans and diversity plans for fiscal years 2020, 2021, and 2022 (which ODEO received during the audit period and the extended period through December 31, 2022), we analyzed the spreadsheets for hidden information (i.e., data, columns, rows, worksheets, and other hidden information resulting from active filters). To confirm that each spreadsheet had a complete list of the state agencies that were required to complete annual progress reports for affirmative action plans and diversity plans, we compared the state agencies from the spreadsheets to a list of state agencies found on the Massachusetts State Government Organizational Chart as of July 2021, which is available publicly on the Mass.gov website. To ensure the accuracy of the spreadsheets, we traced a random sample of 10 annual progress reports for each fiscal year of the audit period from the spreadsheets back to the source documents (i.e., the annual progress reports for the affirmative action plans and diversity plans).

Based on the results the data reliability assessment procedures described above, we determined that the information obtained was sufficiently reliable for the purposes of our audit.

5.   Auditors use judgmental sampling to select items for audit testing when a population is very small, the population items are not similar enough, or there are specific items in the population that the auditors want to review. Auditors use their knowledge and judgment to select the most appropriate sample. For example, an auditor might select items from areas of high risk. The results of testing using judgmental sampling cannot be used to make conclusions or projections about entire populations; however, they can be used to identify specific issues, risks, or weaknesses.

 

Date published:	July 5, 2024