Overview
The Berkshire Regional Transit Authority (BRTA) did not accurately calculate the pickup and drop‑off performance metrics for its paratransit services required by the Americans with Disabilities Act (ADA). BRTA also did not achieve the on-time performance targets outlined in its memorandum of understanding with the Massachusetts Department of Transportation (MassDOT).
BRTA inaccurately calculated its on-time performance by combining its pickup and drop-off percentages; the calculation for on-time performance should only include ride pickup percentages. This resulted in over-reporting of BRTA’s on-time performance percentages as shown below.
| Fiscal Year | BRTA’s On-time Performance Percentage | On-time Performance Percentage Using Federal Transit Administration Guidelines |
|---|---|---|
| Second Half of Fiscal Year 2021 | 91.2% | 88.63% |
| Fiscal Year 2022 | 89.8% | 83.57% |
| First Half of Fiscal Year 2023 | 89.1% | 82.91% |
In addition, the drop-off percentages were not calculated correctly because BRTA included a five‑minute window after a rider’s scheduled drop-off, which does not match the Federal Transit Administration’s (FTA’s) untimely drop-off guidelines. Using this method of including a five-minute window, BRTA calculated 1,383 untimely drop-offs for the audit period. When using FTA’s guidelines that consider any drop-off after the originally scheduled time as late, we calculated that BRTA had 8,454 untimely drop-offs during the audit period.
If BRTA does not accurately calculate its performance metrics, then it cannot identify how well its ADA‑required paratransit service is performing. Late ADA-required paratransit services, whether for pickups or drop-offs, may have a negative impact on those relying on these paratransit services.
Authoritative Guidance
Sections 5.3 and 5.6 of Chapter 8 of FTA’s Circular 4710.1 states,
When assessing the timelines of service, it is important to distinguish among on-time, early, and late pickups. . . . When calculating on-time performance, transit agencies often combine early pickups together with on-time pickups when documenting on-time performance. . . . For analysis purposes, transit agencies typically report this combined metric as “early arrivals plus on-time arrivals” and separately track the number and rate of early pickups, late pickups, and on-time pickups. . . .
FTA encourages establishing policies to drop off riders . . . no later than appointment times.
According to the Bilateral Memorandum of Understanding between BRTA and MassDOT, BRTA’s on-time performance targets are as follows:
On-Time Performance—Demand Response:
- Target ([fiscal year (FY) 2021]): 96.25%
- Interim Milestone (FY22): 96%
- Target (FY23) 96.25%
Reasons for Issue
BRTA management stated that, when the scheduling and dispatching system was implemented at BRTA, the system was programmed by the scheduling and dispatching system vendor to calculate performance metrics. The scheduling and dispatching system vendor has since changed ownership. However, BRTA did not have adequate policies and procedures, including a monitoring component, to ensure that it accurately calculated and updated performance metrics. Further, BRTA did not have adequate policies and procedures, including a monitoring component, to ensure that it met the performance targets it agreed to with MassDOT.
Recommendations
- BRTA should separately calculate its ADA-required paratransit service on-time pickup and drop-off performance metrics.
- BRTA should ensure that its scheduling and dispatching system is updated to accurately calculate its current performance metrics.
- BRTA should establish adequate policies and procedures, including a monitoring component, to ensure that it accurately calculates and updates performance metrics within its scheduling and dispatching system when necessary.
- BRTA should establish adequate policies and procedures, including a monitoring component, to ensure that it meets the performance targets it agreed to with MassDOT.
Auditee’s Response
BRTA disagrees with the methodology used to determine on-time performance with isolating pickup and drop-off percentages. BRTA riders are allowed to choose a scheduled time, either pickup or drop off. The rider is given the freedom to choose that critical timepoint for each leg of the trip. In order to effectively calculate on-time performance, percentages for pickup would need to be compared to those trips that were scheduled as customer choice-pickup time. The same logic would apply for drop off, if that time was chosen as critical to the rider. As noted in FTA Circular 4710.1, “the FTA encourages establishing policies to drop off riders no more than 30 minutes before appointment times and no later than appointment times.” BRTA believes the flexibility of riders’ choice in which time is critical (pickup or drop-off) allows for greater freedom and control by the ADA community we serve.
BRTA has the capacity to calculate on-time pickup as well as on-time drop off as outlined in the audit report. BRTA Administrator will pursue updating the data points reported in the [memorandum of understanding] with MassDOT.
Auditor’s Reply
As noted above, BRTA combined its pickup and drop-off percentages when calculating its on-time performance metric, which the FTA’s guidelines do not allow.
In addition, BRTA incorrectly included a five-minute window after a rider’s scheduled drop-off when calculating its timely drop-off percentages. Section 4.5 of Chapter 8 of FTA’s Circular 4710.1 allows for transit agencies to “establish a reasonable ‘window’ around the negotiated pickup time during which the vehicle may arrive and still be regarded as ‘on time,’ to account for day-to-day variability in the operation of complementary paratransit.” The on-time window allowed by FTA is solely related to pickup times, not drop-off times. As previously mentioned, FTA’s guidelines consider any drop-off after the originally scheduled time to be late.
We also want to note here that we have audited the topic of on-time performance of paratransit services at nine other RTAs across the Commonwealth over the past 20 months. No other RTA raised concerns regarding the methodology we used to determine on-time performance. We urge BRTA to fully implement our recommendations.
| Date published: | October 4, 2024 |
|---|