Overview
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Montachusett Regional Transit Authority (MART) for the period October 1, 2019 through September 30, 2021.
We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.
|
Objective |
Conclusion |
|
No; see Finding 1 |
|
No; see Finding 2 |
To achieve our audit objectives, we gained an understanding of MART’s internal control environment related to the objectives by reviewing applicable agency policies and procedures, as well as conducting interviews with MART’s staff members and management. We evaluated the design of controls over ADA‑required paratransit services and ADA paratransit complaints.
We performed the following procedures to obtain sufficient, appropriate audit evidence to address the audit objectives.
Performance Standards for ADA-Required Paratransit Services
To determine whether MART complied with its standards for ADA-required paratransit services regarding on-time performance, missed trips, and untimely drop-offs, we obtained information regarding all trips that occurred during the audit period and extracted the 52,108 ADA-required paratransit rides that occurred during the audit period. We analyzed 100% of these 52,108 ADA-required paratransit rides to determine whether MART met the performance standard it established in its “ADA Paratransit Service Standards.” We compared the following information about all 52,108 ADA-required paratransit rides that occurred during the audit period to MART’s performance standards for on-time performance, missed trips, and untimely drop-offs.
On-Time Performance
To determine whether MART complied with its performance standard of picking up at least 96% of riders within the scheduled pick-up window (i.e., 20-minutes), we calculated the number of minutes between the ADA-required paratransit rider’s actual pickup time and scheduled pickup time for each of the 52,108 rides that occurred during the audit period. We calculated the percentage of pickups that were on time from every trip that occurred during the audit period and compared it to MART’s performance standard to determine whether MART met its on-time performance standard. We identified 1,476 late trips that were outside of the on-time pickup window and sorted these trips by the number of minutes that they were late. We then selected a sample by targeting the 20 trips with the largest number of minutes late. For the remaining 1,456 late trips, we randomly selected a sample of 27 trips, using a 90% confidence level, 10% tolerable error rate, and 0% error rate. For both of the samples (the 20 trips with the largest number of minutes late and the randomly selected 27 trips), we examined MART’s ADA transit manager’s monthly reconciliation notes on late trips, which included the cause of each late pickup.
Missed Trips
To determine whether MART complied with its performance standard of having a missed trip rate of no more than 5%, we calculated the number of minutes between the actual pickup time and the scheduled pickup time for each of the 52,108 rides that occurred during the audit period and filtered the trips that were classified as either canceled-on-arrival or no-shows. For trips classified as canceled-on-arrival or no-shows, we calculated the number of minutes between the driver’s arrival time and departure time to determine whether the driver waited five minutes before departing without the rider. We identified a total of 314 missed trips where the driver did not arrive within the 20-minute window and/or did not wait five minutes before departing without the rider. We calculated the percentage of missed trips and compared this to MART’s 5% performance standard to determine whether MART’s contracted paratransit provider met MART’s performance standard for missed trips. We selected a random, nonstatistical sample of 27 out of the 314 missed trips. For our sample, we examined MART’s ADA transit manager’s monthly reconciliation notes on missed trips, which included the cause of each missed trip.
Untimely Drop-Offs
To determine whether MART complied with its performance standard of having an untimely drop-off rate of no more than 3%, we calculated the number of minutes between the actual drop-off time and the rider’s scheduled drop-off time for each of the 52,108 rides that occurred during the audit period. We calculated the percentage of drop-offs that were untimely and compared this to MART’s performance standard to determine whether MART met its standard for untimely drop-offs. We then targeted the 20 untimely drop-offs with the largest number of minutes between scheduled drop-off time and actual drop-off time from the total population of 1,290 untimely drop-offs that occurred during the audit period. For the remaining 1,270 untimely drop-offs, we randomly selected a sample of 27 trips, using a 90% confidence level, 10% tolerable error rate, and 0% error rate. For our samples, we examined MART’s ADA transit manager’s monthly reconciliation notes on untimely drop-offs, which included the cause of each untimely drop-off.
ADA Paratransit Complaints
To determine whether MART investigated and documented its responses to ADA paratransit complaints, we analyzed 100% of the 34 ADA paratransit complaints that MART received during the audit period. We calculated the number of business days between the date MART received each complaint and the date MART’s contracted paratransit provider started the investigation to determine whether the investigation started within three business days of MART’s receipt of the complaint.
To determine whether MART issued its response within five business days of its receipt of the complaint, we calculated the business days between the date that MART received the complaint and the date that MART’s staff member(s) responded to the complainant. We examined the comments MART recorded in its Microsoft Access database for ADA paratransit complaints regarding all 10 of the investigations that took over five business days to verify the dates that complaints were submitted and the dates that complaints were resolved, and we determined the reason why the complaints were not resolved within five business days.
We used a combination of nonstatistical and statistical sampling methods for our audit objectives and did not project the results from the samples to the populations.
Data Reliability Assessment
Transit Scheduling and Dispatching System
To determine the reliability of the ADA-required paratransit rider data from the transit scheduling and dispatching system, we interviewed officials who were knowledgeable about the ADA-required paratransit rider data and tested selected information system controls (security management, configuration management, contingency planning, and segregation of duties). We then tested the ADA-required paratransit rider data for missing and duplicate records and for records that were outside of our audit period.
We obtained a list of 606 riders who were on MART’s list of approved ADA-required paratransit riders from the ADA transit manager, selected a random sample of 20 riders, and traced each of these 20 ADA-required paratransit rider records to hardcopy rider applications. We randomly selected 20 hardcopy ADA-required paratransit rider applications and traced these applications to the list of riders who were on MART’s ADA-required paratransit service list. We verified that all riders who received ADA-required paratransit services from MART during the audit period were on MART’s list of riders eligible for ADA-required paratransit services.
Microsoft Access Database for ADA Paratransit Complaints
To determine the reliability of the data from MART’s Microsoft Access database for ADA paratransit complaints, we interviewed officials about the data sets for ADA-required paratransit trips and ADA paratransit complaints. We tested the complaint data for missing and duplicate records and for records that were outside of our audit period. We also traced the names on all 34 of the complaints that were received during the audit period to our list of ADA-required paratransit riders to verify that ADA paratransit complaints were reported by ADA-required paratransit riders.
Based on the results of our data reliability procedures detailed above, we determined that the information obtained for our audit period was sufficiently reliable for the purpose of our audit.
| Date published: | August 30, 2023 |
|---|