Audit of the Executive Office of Housing and Livable Communities—Emergency Shelter Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Executive Office of Housing and Livable Communities—Emergency Shelter.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Executive Office of Housing and Livable Communities (EOHLC) for the period July 1, 2021 through December 31, 2023. When designing the audit plan for the emergency no-bid contracts, we extended the audit period to June 30, 2024 to encompass certain expenditures related to emergency no-bid contracts during that timeframe.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
Did EOHLC pay the traditional shelters and hotels/motels—which provided housing to individuals who sought emergency assistance (EA)—according to their contracts with EOHLC?	No; see Finding 6 and Other Matters
Did EOHLC ensure that the emergency food and transportation services provided under the Emergency Assistance Family Shelter program were contracted in accordance with section 21.05(3) of Title 801 of the Code of Massachusetts Regulations (CMR)?	No; see Findings 1, 2, 3, 4, and 5

To accomplish our audit objectives, we gained an understanding of EOHLC’s internal control environment related to our objectives by reviewing EOHLC’s internal policies and procedures and by interviewing EOHLC officials. We reviewed policies and procedures related to the EA Family Shelter and Services Program, which included payments made to emergency shelter providers and the administration of emergency contracts. We evaluated the design, and tested the operating effectiveness, of internal controls related to payments made to shelter providers. Specifically, we examined the controls governing the processing of payments by EOHLC. We confirmed that the contracting manager signed a payment voucher request form before any payments were issued to shelter providers. Additionally, we verified that EOHLC EA Family Shelter Program staff members conducted reconciliations to ensure that shelter providers were billing the correct amounts for units and nightly rates, as established in the contracts. In addition, to obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures listed below.

EA Shelter Payments

To determine whether the EOHLC compensated contracted shelter providers—who assist those seeking emergency assistance—according to the terms of their contracts, we took the following actions:       

• We obtained and examined 183 contracts between EOHLC and shelter providers, including hotels and motels.
• We obtained and examined a spreadsheet of 3,379 payments, totaling $700,020,208, made to shelter providers, including hotels and motels, during the audit period from the Massachusetts Management Accounting and Reporting System (MMARS).
• We divided the payments into two categories: before and during the emergency. From a total of 2,516 payments, totaling $415,870,686, made to contracted shelter providers for the period, July 1, 2021 through June 30, 2023, we selected a random, statistical sample¹⁰ of 47 payments, totaling $9,493,014, at a 90% confidence level,¹¹ a 5% tolerable error rate,¹² and a 0% expected error rate.¹³ We used Teammate Analytics¹⁴ to select our sample. Additionally, for the period from July 1, 2023 through December 31, 2023, with a total population of 863 payments totaling $284,149,521, we selected a random, nonstatistical sample¹⁵ of 60 payments totaling $13,931,539.
• For the selected samples, we obtained and examined invoices to verify that the payments, including the nightly unit rates and number of units, were billed accurately according to contract terms and conditions.

Based on the results of our testing, we determined that EOHLC did not ensure that the payments made to shelter providers were in compliance with the terms and conditions of their contracts. See Finding 6.

Emergency Contracts

We sought to determine whether the EOHLC complied with 801 CMR 21.05(3) in contracting for emergency services under the EA Family Shelter Program. To determine whether EOHLC appropriately executed emergency no-bid contracts and whether the associated services, including meal and transportation deliveries, were provided in compliance with contract terms and financial requirements, we performed the following procedures:

• We obtained and reviewed all emergency no-bid contracts related to the food and transportation services within the EA program for the period ending June 30, 2024. These contracts included agreements with Spinelli and Mercedes Cab Company / Pilgrim Transit. Our review focused on assessing whether the contracts were appropriately awarded under the emergency provisions of the regulations.
• We examined the written justifications provided for the use of no-bid emergency contracts to determine whether the criteria outlined in 801 CMR 21.05(3) were met. Specifically, we examined whether no-bid emergency conditions and the need for immediate services were documented and consistent with regulatory guidelines.
• We obtained and reviewed internal communications (emails) from EOHLC from the period of time before the execution of the emergency contracts, which were assessed to understand the contracting process, decision-making, and rationale for awarding the emergency contract. Additionally, we reviewed email communications between EOHLC and Spinelli.
• Spinelli Invoices:
  • We obtained from EOHLC invoices sent by Spinelli covering the contract period from August 2023 through March 31, 2024. We analyzed a total of 34 invoices, totaling $9,446,700, to break down the costs associated with the 487,899 meals provided and 5,142 deliveries, displayed in the table on page 15, made to 30 hotels and motels served by Spinelli. We verified the list of 30 hotels and motels served by Spinelli against the list of 67 hotels¹⁶ contracted by EOHLC for use as emergency shelters, ensuring that there was no duplication of food deliveries or deliveries to non-hotels or motels under contract with EOHLC.
  • We compared the delivery rates charged by Spinelli for each location to its official delivery rate sheet (see Appendix A) to ensure consistency with contracted delivery rates.
• Mercedes Cab Company / Pilgrim Transit Invoices:
  • We obtained nine invoices, totaling $4,749,307, from Mercedes Cab Company / Pilgrim Transit, covering the period from October 2023 through June 2024 for each of the 15,294 rides completed. We analyzed all 15,294 rides completed and identified the 556 outliers examined below:
    • We target tested all 15 rides, totaling $7,833, that occurred outside of regularly scheduled service hours (6:00 a.m. to 8:00 p.m.) to assess whether the rides provided were to and from a hotel or motel and a medical facility. Using Google Maps, we verified the pickup and drop-off addresses to determine whether they were either a hotel or motel and medical facilities.
    • We target tested all 206 rides, totaling $36,339, that were requested for less than a mile to determine whether such rides were in accordance with the allowable costs set forth in the standard operating procedures. Using Google Maps, we verified the pickup and drop-off addresses to determine whether they were locations allowable according to the standard operating procedures.
    • We target tested all 320 rides, totaling $162,050, that had drop-off locations outside the state to determine whether these rides were reasonable for out-of-state travel. Using Google Maps, we verified the pickup and drop-off addresses to determine whether they were locations allowable according to the standard operating procedures.
    • We target tested all 15 rides, totaling $13,262, that had 10 or more seats requested. Using Google Maps, we verified the pickup and drop-off addresses to determine whether they were locations allowable according to the standard operating procedure.

Based on the results of our testing, we determined that EOHLC did not ensure compliance with 801 CMR 21.05(3) when contracting emergency services under the EA Family Shelter Program. See Findings 1, 2, 3, 4, and 5.

We used a combination of statistical and nonstatistical sampling methods for testing, and we did not project the results of our testing to the corresponding populations.

Data Reliability Assessment

MMARS

The financial data from MMARS constitutes the official accounting records of the Commonwealth and forms the basis for the Commonwealth’s audited annual financial statements. In 2022, the Office of the State Auditor performed a data reliability assessment of MMARS that focused on testing selected system controls (access controls, application controls, configuration management, contingency planning, and segregation of duties). Additionally, in the current audit, we reviewed EOHLC’s policies and procedures and conducted tests regarding EOHLC employees who had access to MMARS during the audit period to determine whether those employees had completed cybersecurity awareness training.

Contracts

To determine the reliability of the 183 shelter provider contracts that EOHLC provided to us, we selected a random sample of 35 shelter provider contracts and divided them into three strata. The first stratum of 8 shelter provider contracts was for fiscal year 2022. The second stratum of 10 shelter provider contracts was for fiscal year 2023. The third stratum of 17 shelter provider contracts was for fiscal year 2024. We compared vendor names, MMARS document identification number, and dollar amounts per contract to the MMARS data to ensure that the payments did not exceed the total dollar amounts in the contracts. To determine the reliability of the emergency no-bid contracts for food and transportation, we compared the total dollar amount of the contracts to the total dollar amount of payments made by EOHLC in MMARS to ensure that payments did not exceed the total dollar amount of the contracts.

Shelter Provider Payments

To determine the reliability of shelter provider payment data that EOHLC provided to us, we compared the total number of invoices, and the total amounts paid to the information documented in MMARS for the periods July 1, 2021 through June 30, 2023 and July 1, 2023 through December 31, 2023.

Mercedes Cab Company / Pilgrim Transit Ride Data

To determine the reliability of the ride data that EOHLC provided to us, which contained a breakdown of the costs associated with the transportation services (number of riders, mileage fee, and extra person fee) contracted by Mercedes Cab Company / Pilgrim Transit, we analyzed the spreadsheet for hidden information (i.e., hidden data, columns, rows, worksheets, and other hidden information resulting from active filters), we compared the total amount spent from the spreadsheet to the amount spent in MMARS.

Based on the results of the data reliability assessment procedures described above, we determined that the information we obtained was sufficiently reliable for the purposes of our audit.

10.    Auditors use statistical sampling to select items for audit testing when a population is large and contains similar items. Auditors generally use a statistics software program to choose a random sample when statistical sampling is used. The results of testing using statistical sampling, unlike those from judgmental sampling, can usually be used to make conclusions or projections about entire populations.

11.    Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are representative of the population (parameter), expressed as a percentage.

12.    Tolerable error rate is the maximum error in the population that auditors would be willing to accept and still conclude that the result from the sample has achieved the audit objective.

13.    Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the auditor’s knowledge of factors such as prior year results, the understanding of controls gained in planning, or a probe sample.

14.    This is a Microsoft Excel–based data analytics tool that allows auditors to execute advanced data analysis.

15.    Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items are not similar enough, or there are specific items in the population that the auditors want to review.

16.    All contracted hotels and motels serving as emergency shelters, excluding those receiving food from Spinelli, were required to provide three meals a day: a continental breakfast, lunch at $16, and dinner at $21.

Date published:	May 20, 2025