Audit of the Department of Public Utilities Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Department of Public Utilities.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Department of Public Utilities’ (DPU’s) Transportation Network Company Division for the period October 1, 2017 through August 31, 2019.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
Do the Transportation Network Company Division’s monitoring measures for rideshare drivers’ eligibility prevent ineligible drivers from providing rideshare services to the general public as required by Sections 274.06 and 274.13(1–4) of Title 220 of the Code of Massachusetts Regulations (CMR)? Specifically,
Does the division review and verify national background record checks of rideshare driver applicants performed by transportation network companies (TNCs)?	No see Findings 1a and 1b
Does the division review and verify mandatory six-month background record checks of existing rideshare drivers performed by TNCs?	No see Findings 1a and 1c
Does the division perform state background record checks of rideshare driver applicants, identifying all ineligible applicants?	Yes
Does the division perform state background record checks of rideshare drivers at least once a year to ensure that drivers remain eligible?	Yes
When the division deems rideshare drivers ineligible, does it notify the TNCs and ensure that the drivers stop operating rideshare vehicles?	No see Findings 1a and 1d
Does the Transportation Network Company Division oversee and monitor the rideshare complaint process to ensure that complaints are resolved and ensure public safety as required by 220 CMR 274.12(3)?	No see Finding 2

 

To achieve our objectives, we gained an understanding of the Transportation Network Company Division’s internal control environment related to the objectives by reviewing the division’s policies and procedures, as well as conducting inquiries with division personnel and management. We tested the controls’ operating effectiveness over TNCs’ national background record checks of rideshare drivers and complaint resolution reporting. We identified a control deficiency in the division’s oversight of TNCs. This oversight is intended to ensure that TNCs perform national background record checks at initial application and then every six months and promptly bar ineligible drivers from accessing their digital networks to pick up riders in accordance with 220 CMR 274.13(3). (See Finding 1a.)

Scope Limitation

We encountered a scope limitation with regard to Objective 1e. Specifically, when performing our testing to determine whether the Transportation Network Company Division effectively monitored the eligibility of rideshare drivers, we attempted to determine whether all rideshare drivers who had been suspended or had their Clearance Certificates revoked were denied access to their TNCs’ digital networks. We requested that the division provide us with documentation that TNCs barred rideshare drivers with suspended or revoked Clearance Certificates from their digital networks.

Transportation Network Company Division officials responded that they did not maintain this documentation. We requested that the division obtain from two TNCs (Uber and Lyft) evidence of the date the drivers’ access to the TNCs’ digital networks was turned off. Uber and Lyft stated that they would not provide the requested screenshot to show that rideshare drivers with suspended or revoked certificates were denied access to the digital networks. Although we were able to obtain an understanding of the division’s monitoring activities, we could not determine to what extent TNCs barred rideshare drivers with suspended or revoked Clearance Certificates from their digital networks. (See Finding 1d.)

TNCs’ Driver National Background Record Checks

To determine whether the Transportation Network Company Division ensured that TNCs performed national background record checks of potential rideshare drivers as required by 220 CMR
274.06(2)(a–e), we selected a statistical, random sample, with a 0% expected error rate, a 5% precision rate, and a 95% confidence level, of 53 of the 195,708 potential drivers who applied to TNCs during the audit period. To determine whether these 53 applicants had any disqualifying events, we obtained and reviewed their names; their driver’s license numbers; and copies of their criminal, sex offender, and driving records that indicated whether national background record checks had been performed.

To determine whether the Transportation Network Company Division monitored whether TNCs performed national background record checks of active rideshare drivers every six months, we selected a statistical, random sample, with a 0% expected error rate, a 5% precision rate, and a 95% confidence level, of 58 of the 851,334 active rideshare drivers during the audit period. For these 58 drivers, we obtained and reviewed names; driver’s license numbers; and copies of criminal, sex offender, and motor vehicle records to ensure that the checks had been performed.

To determine whether the Transportation Network Company Division notified TNCs when it identified rideshare drivers who did not meet qualifying standards and should be barred from accessing the TNCs’ digital networks, we interviewed management and observed an email generated by the DPU screener program. Emails are sent automatically from the screener program to both the TNC and the driver with the suspended or revoked certificate, notifying both parties that a Clearance Certificate has been suspended or revoked.

To determine whether the Transportation Network Company Division verified that TNCs barred access to their digital networks for rideshare drivers who did not meet qualifying standards for rideshare driving in the Commonwealth, we selected a statistical, random sample, with a 0% expected error rate, a 5% precision rate, and a 95% confidence level, of 60 of the 10,050 rideshare drivers with suspended or revoked Clearance Certificates during the audit period. We gave the division these 60 potential drivers’ names and driver’s license numbers and requested screenshots of TNC digital network files showing that the drivers no longer had access to the networks. The division did not have these documents and requested them from the TNCs. The TNCs would not provide the requested documents; this led to the previously mentioned scope limitation.

Division Driver State Background Record Checks

To determine whether the Transportation Network Company Division verified that potential rideshare drivers passed a state background record check as required by 220 CMR 274.06(3)(a–e), we selected a statistical, random sample, with a 0% expected error rate, a 5% precision rate, and a 95% confidence level, of 60 of the 154,234 potential rideshare drivers initially approved during the audit period. We gave the division these 60 drivers’ names and driver’s license numbers and requested record documentation of this verification. We examined copies of the potential rideshare drivers’ records from the Criminal Offender Record Information (CORI) database, the Sex Offender Registry Information (SORI) database, the Warrant Management System (WMS), and the Registry of Motor Vehicles’ (RMV’s) driver history database to determine whether any matches occurred. For records that did have matches, we inspected copies of the CORI, SORI, WMS, and RMV records to determine whether any of the matches was a disqualifying event that would prevent the potential rideshare driver from getting approval.

To determine whether eligible rideshare drivers passed annual state background record checks, we selected a statistical, random sample, with a 0% expected error rate, a 5% precision rate, and a 95% confidence level, of 60 of the 851,334 then-current rideshare drivers whom the Transportation Network Company Division deemed eligible during the audit period. We gave the division these 60 drivers’ names and driver’s license numbers and requested documentation verifying that successful subsequent annual state background record checks were performed for them. We examined copies of the rideshare drivers’ records from the CORI database, the SORI database, WMS, and the RMV driver history database to determine whether any matches occurred. We inspected records that did have matches to determine whether any of the matches was a disqualifying event that would prevent the rideshare driver from performing rideshare services.

Rideshare Complaint Process

To determine whether the Transportation Network Company Division monitored monthly complaint reports as required by 220 CMR 274.12(3), we interviewed division management and determined that the division only collected the reports from TNCs and had not established a process to monitor the information in them. We obtained copies of the monthly complaint reports submitted by TNCs and identified the dates the complaints were made to TNCs, the types of complaints, and whether the complaints had been resolved. We also determined through interviews with division management that the division had not requested documentation from TNCs showing what the complaints were or evidence related to the actions TNCs took to resolve them. Uber and Lyft provided monthly complaint reports from the period January 2019 through August 2019.⁵ They contained 156,127 rider complaints about rideshare drivers and 115,364 driver complaints about riders.

Data Reliability

To gain an understanding of the screener program data system and controls, we interviewed information technology personnel who were responsible for oversight of the system. To assess the system’s reliability, we tested the following general information technology controls: security management, access controls, configuration management, contingency planning, and segregation of duties.

To assess the completeness and accuracy of the applicant list and the driver list, we compared the list of applicants and drivers in the screener program data system to the list of applicants and drivers from the Transportation Network Company Division’s annual report. We determined that the list of applicants and drivers was sufficiently reliable for the purposes of this audit.

To assess the reliability of the Transportation Network Company Division’s rideshare driver background record checks in the screener program data system, we traced our samples of rideshare driver applicants and drivers from the screener program data system to source documents from the CORI database, the SORI database, WMS, and the RMV driver history database. We determined that the rideshare driver applicant and rideshare driver data from the screener program data system were sufficiently reliable for the purpose of this audit.

To assess the completeness of TNCs’ monthly complaint reports, we interviewed Transportation Network Company Division management, determined when the division started to receive the complaint reports, and followed up with requests to obtain missing monthly reports. We determined that the data were sufficiently reliable for the purpose of this audit.

 

5.     Driver and rider monthly complaint reports compiled and resolved by the TNCs were to be provided to the Transportation Network Company Division starting in October 2017. The division did not start receiving the reports until January 2019.

Date published:	November 23, 2021