Audit of the Supplier Diversity Office Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Supplier Diversity Office.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of the Supplier Diversity Office (SDO) for the period January 1, 2021 through December 31, 2022.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
To what extent, if at all, did SDO meet the goal of maintaining consistent and measurable progress in expanding access and equity of opportunity in state contracting, in accordance with Section 61(a) of Chapter 7 of the General Laws?	To a sufficient extent
How, if at all, did SDO’s Compliance Unit monitor the purchasing performance of each state agency participating in the Supplier Diversity Program (SDP) to ensure that these agencies met benchmarks established in Section 61(m) of Chapter 7 of the General Laws?	To an insufficient extent; see Finding 1 and Other Matters 1 and 3
How, if at all, did SDO monitor contracts between statewide contractors and SDP partners to ensure that statewide contractors were in program compliance, as required by Section 61(m) of Chapter 7 of the General Laws?	To an insufficient extent; see Finding 2 and Other Matter 2
To what extent, if at all, did SDO devise equitable policies and procedures to provide training and other services for all SDO programs, as required by SDO’s final budgetary language for line items 1775–0200 for fiscal year 2021 and 1780–0100 for fiscal year 2022?	To a sufficient extent

In addition to the conclusions we reached regarding our audit objectives, we also identified issues not specifically addressed by our objectives. For more information, see Other Matters.

To accomplish our audit objectives, we gained an understanding of the aspects of the internal control environment relevant to our objectives by reviewing applicable SDO policies and procedures and by conducting inquiries with SDO officials who are responsible for the oversight of SDO’s programs. In addition, to obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures described below.

Progress in Expanding Access and Equity of Opportunity in State Contracting

To determine to what extent, if at all, SDO met the goal of maintaining consistent and measurable progress in expanding access and equity of opportunity to state contracting, in accordance with Section 61(a) of Chapter 7 of the General Laws, we took the following actions. We requested, and SDO provided, a list of business applicants that submitted applications during the audit period. From this list, we selected a random, nonstatistical sample of 50 businesses out of 592 businesses that submitted certification applications during the audit period.

	Number of Businesses	Percent of Businesses
Businesses That Applied After Attending a Precertification Workshop	592	55%
Businesses That Did Not Apply After Attending a Precertification Workshop	487*	45%
Total Number of Businesses That Attended a Precertification Workshop	1,079	100%

*  Note that 88 out of these 487 businesses that attended a precertification workshop during the audit period submitted applications after the audit period.

We requested the full application for each business applicant in our sample and determined whether the submitted application was accepted or rejected by SDO or whether a business withdrew its application by reviewing each application’s Certrak activity log.

• For applications that were accepted, we determined whether an intake coordinator performed an investigation as required by the document “Supplier Diversity Office (SDO) Standard Certification Policy & Procedures” and whether SDO sent a letter to the business, notifying it that SDO approved it for certification.
• For applications that were rejected, we determined whether SDO communicated the application issues to the business applicant and whether SDO allowed the business applicant to take the time allotted by the document “Supplier Diversity Office (SDO) Standard Certification Policy & Procedures” to fix these issues.
• For applications that were withdrawn, we requested and reviewed correspondence from the business applicants in question that communicated the reason why they withdrew their applications.

Furthermore, we calculated the amount of time it took SDO to complete both the full certification process and each step in the certification process.

Average Number of Business Days in Each Part of the Certification Process

	Average Number of Business Days to Complete
	Intake Coordinator’s Initial Decision	Intake Coordinator’s Transmittal to Investigator	Investigator’s Decision	Full Process	Number of Applications
Applications Initially Accepted	9	33	23	65	6
Applications Initially Rejected, Then Approved	11	77	28	116	33
Total Applications Approved	—	—	—	—	39*

*  This is the number of applications that were approved out of our sample of 50.

Point in the Certification Process 
When Applications Were Rejected and/or Withdrawn

	Number of Applications Rejected/Withdrawn During
	Intake Coordinator’s Step	Investigator’s Step	Number of Applications
Applications Ultimately Rejected by SDO	3	2	5
Applications Ultimately Withdrawn by Business Applicant	1	3	4
Total Applications Not Approved	—	—	9*

*  This is the number of applications that were rejected and/or withdrawn out of our sample of 50.

For this objective, we noted no significant issues during our testing. Therefore, we concluded that, during the audit period, SDO met the goal of maintaining consistent and measurable progress in expanding access and equity of opportunity to state contracting.

Benchmark Monitoring

To determine how, if at all, SDO’s Compliance Unit monitored the purchasing performance of each SDP-participating state agency to ensure that these agencies met benchmarks established in Section 61(m) of Chapter 7 of the General Laws, we obtained and reviewed a list of state agencies that participated in the SDP during the audit period. From this list, we selected a random, nonstatistical sample of 20 state agencies out of 73 SDP-participating state agencies.

The list included the SDP spending amount made by each SDP-participating state agency, which would indicate its progress toward meeting SDO’s spending benchmark(s) for years 2021 and 2022. We reviewed one progress report for calendar year 2022 for each state agency in our sample to determine whether SDO monitored the purchasing performance of each state agency.

Percent of SDP-Participating State Agencies That Met Spending Benchmarks

Certification Category	Percent of State Agencies That Met Benchmark
	Fiscal Year 2021	Fiscal Year 2022
Minority-Owned Business Enterprise	40%	50%
Women-Owned Business Enterprise	20%	35%
Veteran Business Enterprise	10%	10%

Percent of SDP-Participating State Agencies That Met Spending Benchmarks

For this objective, we noted certain issues during our testing. See Finding 1 and Other Matters 1 and 3 for more information regarding how SDO’s Compliance Unit monitors the purchasing performance of each SDP-participating state agency.

Contract Monitoring

To determine how, if at all, SDO monitored contracts between statewide contractors and SDP partners to ensure that statewide contractors were in program compliance, as required by Section 61(m) of Chapter 7 of the General Laws, we requested, and SDO provided, a list of SDP spending transactions by statewide contractors. From this list, we selected a random, nonstatistical sample of 60 contracts from a population of 817 SDP spending contracts for the period July 1, 2020 through June 30, 2022.⁸ The information for fiscal year 2023⁹ was not available during our audit because the verification process relied on previous fiscal year–end data. We reviewed the work SDO’s Compliance Unit performed during the indirect spending validation process. SDO’s Compliance Unit provided us with each statewide contractor’s self-reported indirect spending; emails SDO sent to each SDP partner requesting that it confirm whether the reported spending by statewide contractors was accurate, overstated, or understated; and the SDP partner’s response. Next, we determined whether SDO logged each SDP partner’s response correctly. Then, we compared the reported spending from each statewide contractor to each SDP partner’s response and determined whether each statewide contractor provided accurate, overstated, or understated spending with each SDP partner. Lastly, we requested and reviewed follow-up emails SDO sent to each statewide contractor that SDO determined to have overstated SDP spending by more than 10%.

For this objective, we noted certain issues during our testing. See Finding 2 and Other Matters 2 for more information regarding how SDO monitors contracts between statewide contractors and SDP partners.

SDO’s Equitable Policies and Procedures

To determine to what extent, if at all, SDO devised equitable policies and procedures to provide training and other services for all SDO programs, as required by SDO’s final budgetary language for line items 1775–0200 (fiscal year 2021) and 1780–0100 (fiscal year 2022), we took the following actions: In terms of our testing, we defined the term equitable as providing equal and fair training and other service opportunities to all entities across the Commonwealth. We interviewed SDO’s director of marketing, communications, and training. We reviewed SDO’s website to determine the types of training it offered as evidence of SDO’s outreach and training to support diverse and small businesses. Trainings offered included general topics for public purchasers, topics about the Supplier Diversity HUB system, and topics about how to find SDP partners (which is delivered through webcasts). In addition, we reviewed SDO’s Municipal Supplier Diversity Playbook (which provides a framework for the establishment and development of municipal SDPs) and the outcomes of SDO’s statewide advertising campaign, which it executed with the help of an external marketing firm.

For this objective, we noted no significant issues during our testing. Therefore, we concluded that, during the audit period, SDO devised equitable policies and procedures to provide training and other services for all SDO programs.

We used nonstatistical sampling methods for testing and therefore did not project the results of our testing to any populations.

Data Reliability Assessment

Progress in Expanding Access and Equity of Opportunity in State Contracting

To determine the reliability of the list of business applicants that submitted applications during the audit period, we interviewed SDO system management employees who were knowledgeable about the data in the list. We performed validity and integrity tests on the list of business applicants, which included (1) testing for duplicate entries; (2) testing for dates outside of the audit period; and (3) inspecting the data for hidden rows and columns, embedded data, and invisible content. Additionally, we selected 20 businesses from the list and vouched¹⁰ these to the hardcopy applications.

Benchmark Monitoring

To determine the reliability of the list of state agencies that participated in the SDP during the audit period, we interviewed SDO system management employees who were knowledgeable about the data in the list. We performed validity and integrity tests on the list of SDP-participating agencies, which included (1) testing for duplicate entries and (2) inspecting the data for hidden rows and columns, embedded data, and invisible content. Additionally, we selected 20 state agencies from the Executive Office for Administration and Finance’s budget list, which is available through the Office of the Comptroller of the Commonwealth, and traced them to SDO’s list of SDP-participating state agencies. Then we selected 20 state agencies from SDO’s list of SDP-participating state agencies and vouched them back to the Executive Office for Administration and Finance’s budget list.

Contract Monitoring

To determine the reliability of the list of SDP spending transactions by statewide contractors, we interviewed SDO system management employees who were knowledgeable about the data in the list. We performed validity and integrity tests on the list of SDP spending transactions by statewide contractors, which included (1) testing for duplicate entries and (2) inspecting the data for hidden rows and columns, embedded data, and invisible content. Additionally, we selected a random sample of 20 businesses from this list and vouched them back to SDO’s Directory of Certified Businesses, verifying the name of the SDP partner’s business and its certification category.

Based on the results of the data reliability assessment procedures described above, we determined that the information we obtained was sufficiently reliable for the purposes of our audit.

8.    SDO monitors SDP-participating state agency spending at the end of each fiscal year. Fiscal year 2021 began on July 1, 2020.

9.    Fiscal year 2023 encompassed the period July 1, 2022 through June 30, 2023. 

10.   Vouching is the inspection of supporting documentation to corroborate data.

Date published:	December 23, 2024