Overview
The MBTA did not ensure that visual inspections were completed at its subway station locations at the agreed-upon frequency of at least twice per contracted hour. Specifically, BBB only completed the agreed‑upon number of visual inspections 6% of the time during the audit period.
Based on the contracted days and hours, we determined that there were 43,727 contracted days during the audit period when subway station locations were staffed by BBB employees. These 43,727 contracted days resulted in 672,589 contracted hours of BBB coverage. Of these 43,727 contracted days for all subway station locations, there were only 2,794 (6%) days that met or exceeded the agreed-upon station check frequency of two station checks per contracted hour. Therefore, 40,933 (94%) contracted days did not meet the agreed-upon frequency of at least two station checks per contracted hour. Of the 672,589 contracted hours of BBB coverage for all subway station locations, there were only 335,473 (50%) hours that met or exceeded the agreed-upon station check frequency of two station checks per contracted hour. Therefore, 337,116 (50%) contracted hours did not meet the agreed-upon frequency.
A lack of regular station checks increases the risk that safety hazards and maintenance needs will not be identified in a timely manner, affecting the safety of the MBTA’s riders, employees, and subway stations.
If the MBTA does not ensure that the controls (such as visual inspections by BBB employees) it designed to prevent safety incidents from occurring are in place and working as intended, then its ability to protect its riders and employees will continue to be impaired.
Authoritative Guidance
MBTA officials told us in a meeting on July 11, 2023 that they expected BBB to conduct station checks of their assigned subway station locations at least twice per hour. BBB officials confirmed to us their expectation with this frequency of station checks. During a meeting on July 27, 2023, BBB’s director of operations told us that the MBTA had verbally communicated this frequency expectation on a continual basis since the beginning of the initial contract period in 2017.
Reasons for Issue
The MBTA has not developed, documented, or implemented monitoring controls to ensure that visual inspections of its stations are completed at the agreed-upon frequency. The MBTA did not establish a service‑level agreement and performance metrics in its contract with BBB to evaluate BBB’s performance in this area. (See the “Other Matters” section of the report for more information.)
Recommendations
- The MBTA should develop, document, and implement monitoring controls to ensure that visual inspections are completed at the agreed-upon frequency of at least twice per hour.
- The MBTA should establish service-level agreements and performance metrics in its contract with BBB to evaluate BBB’s performance in this area.
- The MBTA should ensure that BBB meets all of its contractual obligations for the entirety of its contract.
Auditee’s Response
The MBTA encourages and requests that TAs conduct 2–4 station checks per hour. This, however, is not a contractual requirement in the agreement with BBB. A station check is referred to in RFP No. 3-22 for In-Station Customer Service Operations as:
The MBTA expects all [relevant BBB employees] to be actively aware of potential safety or security risks. . . . [These BBB employees] are required to monitor the condition and operability of customer-facing features at stations, and when necessary, report, by radio or station phone and/or audit management system, the following and other matters:
a) Station maintenance issues, including but limited to, inoperable elevators and escalators, and automated door opener . . .
b) Tripping hazards and path of travel obstructions . . .
c) Cleaning needs, including elevators and restrooms . . .
d) [Automated fare collection] equipment malfunctions ([fare vending machines] and fare gates)[.]
The methodology used by the [OSA] negatively skews the data. For example, if a second station inspection was conducted one minute after the hour, the [OSA] nonetheless considered that hour to be a failure. BBB staffed approximately 646,000 base hours during the audit period at various stations. A total of 1,743,971 total station checks were conducted during that same period, averaging to approximately 2.4 station checks per hour staffed.
Additional information also supports that TAs were conducting routine and regular station checks. BBB reported that 44,584 maintenance requests were submitted during the audit period, which averages out to 61 maintenance requests submitted per day. This number reflects that TAs are serving in their proper function of identifying maintenance issues and by doing so, preventing potential accidents and incidents. Further, these checks can be impacted by service changes due to diversions, coverage moves to staff high-priority station posts, or station emergencies, closures, or evacuations.
Auditor’s Reply
Regarding the analysis performed to test whether visual inspections were completed at the agreed-upon frequency of at least twice per hour, we disagree with the MBTA’s statement that the methodology used by OSA negatively skews the data. Our audit team used the performance expectation that the MBTA and BBB both independently stated was in effect during the audit period; this should have been incorporated into the contract, as would be best practice. Documenting and enforcing this performance expectation may have helped address some of the concerns shared by the MBTA in its response.
The stated performance expectation was that each station be inspected at least twice per hour. The MBTA questions instances where the second inspection occurred shortly after the expiration of one hour—outside of the hour, but just slightly. These instances may appear reasonable to MBTA’s management, but they are violations of the performance expectation that the MBTA had established and that had already existed, as acknowledged by both the MBTA and BBB. In addition, the MBTA’s example of a second inspection falling just outside of the one hour timeframe ignores the fact that between 2 and 4 inspections were to be conducted and instead focuses on instances where BBB was meeting the lowest possible performance permitted under this agreed-upon frequency. The MBTA confirmed in its response, “The MBTA encourages and requests that [BBB employees] conduct 2–4 station checks per hour,” yet BBB failed to perform even the bare minimum number of 2 inspections per hour 94% of the time.
The MBTA also mentions in its reply that BBB reported maintenance requests during the audit period, which it believes supports that BBB employees were serving their proper function. However, maintenance requests were not one of our audit objectives and are not relevant to the performance expectation established by the MBTA and BBB that was considered during this audit.
Regarding MBTA’s response about “high-priority station posts,” we audited to the contract, which included Tier I and Tier II stations (considered fully covered) and Tier III stations (considered partially covered), and used the coverage hours noted in the contract to perform our analyses. In its response, the MBTA indicates that 1,743,971 total station checks were conducted during the audit period, “averaging to approximately 2.4 station checks per hour staffed.” The MBTA’s claim of 1.7 million station checks is inflated. The actual number of station check records, logged in accordance with the MBTA’s own standard of one station check per record, should be approximately 1.28 million. For instance, there were 72 station checks logged under two records by a single BBB employee on January 12, 2021 at 11:04 p.m. at the Back Bay station, 69 station checks logged under one record by a single BBB employee on January 30, 2021 at 12:23 p.m. at the Central North station, and 40 station checks logged under one record by a single BBB employee on June 26, 2021 at 5:01 p.m. at the Assembly North station.
This batching of station checks skews the data, appears to be intended to bypass the performance measure, and may not constitute completed safety inspections. While 1,743,971 total station checks does average to 2.7 checks per hour, this is highly misleading and is an inappropriate metric that may deceive the public regarding the performance under this contract.
We strongly encourage the MBTA to implement our recommendations in this area, as a lack of regular station checks increases the risk that safety hazards and maintenance needs will not be identified in a timely manner, affecting the safety of the MBTA’s riders, employees, and subway stations.
| Date published: | July 7, 2024 |
|---|