• This page, Audit of the Office of Consumer Affairs and Business Regulation Objectives, Scope, and Methodology, is   offered by
  • Office of the State Auditor

Audit of the Office of Consumer Affairs and Business Regulation Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Office of Consumer Affairs and Business Regulation.

Table of Contents

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor (OSA) has conducted a performance audit of certain activities of the Office of Consumer Affairs and Business Regulation (OCABR) for the period July 1, 2017 through June 30, 2019.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective

Conclusion

  1. Does OCABR administer the home improvement contractor (HIC) arbitration process in compliance with Sections 14.05(1), 14.08(4), 14.08(6), 14.08(7), 14.17(1), and 14.17(3) of Title 201 of the Code of Massachusetts Regulations (CMR)?

Yes

  1. Does OCABR administer the Residential Contractor’s Guaranty Fund (RCGF) in compliance with Section 6 of Chapter 142A of the General Laws and 201 CMR 14.19(2), 14.20(1), 14.20(2), 14.20(3), 14.20(6), and 14.21(1)?

No; see Findings 1 and 2

 

In addition to concluding on our audit objectives, we identified an issue we believe warrants OCABR’s attention, which we have disclosed in the “Other Matters” section of this report.

To achieve our objectives, we gained an understanding of the internal controls we deemed significant to the objectives by reviewing OCABR’s policies and procedures, as well as conducting inquiries with OCABR management. We evaluated the design of controls over the operation of OCABR’s HIC Arbitration Program, RCGF cases, the HIC Fund, and the RCGF. We identified deficiencies within the HIC Fund and RCGF (see Finding 1 and Finding 2). We also evaluated the effectiveness of controls over the processes OCABR had established for establishing eligibility for RCGF cases and authorizing payments from the RCGF. We assessed whether these controls operated as intended during the audit period.

We performed the following procedures to obtain sufficient, appropriate audit evidence to address the audit objectives.

HIC Arbitration Program

We selected a nonstatistical random sample of 35 arbitration cases from the population of 172 that were filed with OCABR during the audit period. We reviewed each hardcopy case file and verified that each one contained certain documentation required by OCABR regulations. This documentation included (1) a date stamp indicating the date the application was received by OCABR, (2) a notation indicating the date the case was approved or accepted for a hearing by OCABR, (3) a notation indicating the date on which the hearing was scheduled, (4) evidence that OCABR notified the filer that the claim was accepted for a hearing within 90 business days after accepting the request for arbitration, (5) evidence that any monetary awards were paid within 21 business days of the award date, and (6) notice to the claimant of the arbitrator’s decision.

We compared the information in six fields in the HIC registration database for all 172 arbitration cases—the HIC registration number, case number, date the application was received, resolution, date paid, and amount paid—to the information about these cases on OCABR’s website to determine whether OCABR accurately disclosed the information on its website.

RCGF

We selected a nonstatistical random sample of 20 RCGF cases from a population of 102 that OCABR processed during our audit period and determined the following for each case: (1) whether the HIC was registered at the time of the execution of the contract with the claimant and was current on HIC registration fee payments; (2) whether a notice of payment was provided to the HIC before payment of the RCGF claim to the claimant; (3) whether OCABR received a reimbursement from the HIC for any claim paid from the RCGF to a claimant; (4) whether, when required, there was evidence of the revocation of an HIC’s registration; and (5) whether claimants who won their cases were awarded no more than the maximum $10,000. We also verified that OCABR maintained a balance in the RCGF that was sufficient to cover all of its anticipated RCGF cases for each fiscal year.

We compared fields in OCABR’s HIC registration database for all 102 RCGF cases (specifically, the HIC registration number; contractor business name; responsible party; HIC’s street address, including city, state, and ZIP code; claim number; date the claim was received; disposition; RCGF payment date; and amount paid) to the information on OCABR’s website to assess the accuracy of the information on the website.

We cross-referenced the “Home Improvement Contractor Delinquent List,” dated October 25, 2018, on OCABR’s website to OCABR’s “HIC Registration Lookup” search tool and checked and compared the HIC registration numbers and HIC names in the two sources for accuracy.

We compared OCABR’s payments from the RCGF for the audit period according to its financial records to the information in its HIC registration database and the Massachusetts Management Accounting and Reporting System (MMARS) and noted any discrepancies.

Other Analytical Procedures

We examined all the information in OCABR’s semiannual financial reports during the audit period, recalculated the totals in the reports, and compared this information to the information in MMARS.

To determine whether OCABR sent the information within the Office of the Comptroller of the Commonwealth’s (CTR’s) intercept timeline of 120 days, we used OCABR’s “Debt Collection List” of HICs who were delinquent on RCGF reimbursements during our audit period to calculate the number of days from the reimbursement due date to when OCABR sent the delinquent HIC’s information to CTR.

Using the “HIC Registration Lookup” search tool, we searched for HIC registration numbers equaling zero3 and extracted from the website 492 contractor registrations. We separated the 492 registration numbers into two strata: registration status of “not registered” (445 registration numbers) and registration status other than “not registered” (47 registration numbers). For the first stratum, we selected a random nonstatistical sample of 25 of the 445 numbers and confirmed that they were not registered. For the second stratum, we selected a random nonstatistical sample of 10 of the 47 numbers and determined their registration statuses.

We analyzed all 43,739 fee transactions that were recorded in OCABR’s HIC registration database during the audit period to determine whether OCABR charged the correct fee amounts in accordance with its policy.

When sampling, we used a nonstatistical sampling method and did not project the results to the entire population.

Data Reliability Assessment

HIC Registration Database

We tested certain application controls4 related to the calculation and recording of fees in OCABR’s HIC registration database. We also tested certain general controls5 regarding access to, and security of, the database. We confirmed that all 7,875 HIC applications in the database that were filed during our audit period were assigned distinct and sequential application numbers and that no gaps in numbers existed. We confirmed that all 58,625 of the total application numbers in the database were distinct and sequential numbers with no unexplained gaps.

We determined the reliability of the case information tables in the HIC registration database by completing the following tests:

  • We randomly selected 10 of 172 arbitration cases filed during the audit period from the case information tables and traced them to hardcopy case files.
  • We randomly selected an additional 10 arbitration cases from the hardcopy case files and traced them to the case information tables.
  • We randomly selected 10 of 102 RCGF cases filed during the audit period from the hardcopy case files and traced them to the case information tables.
  • We randomly selected an additional 10 RCGF cases from the case information tables and traced them to the hardcopy case files.
  • We checked the case information tables for duplicate case numbers and dates outside the audit period.

Based on the results of our data reliability assessment, we determined that the information obtained for our audit period was sufficiently reliable for the purposes of our audit work.

MMARS

In 2018, OSA performed a data reliability assessment of MMARS that focused on testing selected system controls (access controls, configuration management, and security management) for the period April 1, 2017 through March 31, 2018.

In our current audit, we tested security management controls at OCABR during the audit period to assess security awareness training and personnel screening. We randomly selected 10 of the 65 RCGF payments in MMARS disbursement reports and traced them to RCGF payment vouchers. We then randomly selected 10 RCGF payment vouchers and traced them to the MMARS disbursement reports.

Based on the results of our data reliability assessment, we determined that the information obtained for our audit period was sufficiently reliable for the purposes of our audit work.

N-court Payment System

We reviewed the System and Organization Control reports,6 bridge letters,7 and peer review reports issued on N-court, a system that is provided and supported by a vendor and is used by OCABR to process fee payments. We verified the accuracy of 4 of 24 reconciliations prepared by OCABR by comparing N-court statements to OCABR’s bank statements and its MMARS budget inquiry reports.

Based on the results of our data reliability assessment, we determined that the information obtained for our audit period was sufficiently reliable for the purposes of our audit work.

3.     OCABR assigns a registration number of zero to HICs in the HIC registration database who have been brought to OCABR’s attention—for instance, by a consumer complaint—but have never been registered. However, there were some HICs with statuses such as “revoked,” “expired,” and “suspended” who were incorrectly assigned registration numbers of zero in the database.

4.     According to the US Government Accountability Office’s publication Assessing Data Reliability, “Application controls . . . are incorporated directly into computer applications to help ensure the validity, completeness, accuracy, and confidentiality of transactions and data during application processing. They include controls over input, processing, output, master file, interface, and the data management system.”

5.     Assessing Data Reliability states, “General controls are the policies and procedures that apply to all or a large segment of an entity’s information systems and help ensure the proper operation of information systems. They include security management, configuration management, and logical and physical access controls, among others.”

6.     A System and Organization Control report about a service organization’s systems is issued by an independent contractor to provide assurance about a service organization’s security, processing integrity, confidentiality, and/or privacy controls.

7.     A bridge letter, also known as a gap letter, is made available by a service organization to cover the period between the reporting period end date of a System and Organization Control report and the release of a new report.

Date published: March 10, 2021

Help Us Improve Mass.gov  with your feedback

Please do not include personal or contact information.
Feedback