Overview
During the course of our audit, we also identified the below two issues that warrant attention.
1. The Massachusetts Bay Transportation Authority Did Not Implement Controls To Establish Accountability or Effectively Monitor Block by Block’s Use of Fare Access Cards.
The Massachusetts Bay Transportation Authority (MBTA) equips Block by Block (BBB) with fare access cards that can be used in certain situations to grant customers courtesy taps. As previously mentioned, BBB is required to record courtesy taps, and the reasons for the taps, in BBB’s Statistics Management and Ambassador Reporting Technology (SMART) system. According to an MBTA policy memorandum to all BBB employees, dated December 23, 2022, the following are approved reasons for BBB employees to issue a courtesy tap:
| Accessibility | Intoxicated customer |
| Amtrak employee | Keolis employee |
| BBB employee (self) | KONE employee* |
| [Rider] bought wrong ticket | Massachusetts Department of Transportation employee |
| Break coverage (BBB) | MBTA employee |
| [Rider] came out by mistake | Military |
| [Rider] came out to use the restroom | Monthly pass issue |
| Cleaners | Pass issue (general) |
| Disability pass issue | Police |
| Fare gate issue | Rafford Lane—Ashmont** |
| Firefighter | Senior pass issue |
| Forgot something | Short money for fare |
| Fare vending machine issues | Station check |
| Customer [experiencing homelessness] | Student pass issue |
| Supervisor (BBB) | Wrong station |
| Wrong side |
* These are the MBTA elevator and escalator repair staff members, who are not expected to pay in order to access the equipment in stations.
** This is specifically for riders transferring from the Mattapan light rail at Ashmont station because, while riders are not given new tickets for this transfer, they are not expected to pay a second time.
During the audit period, BBB employees recorded 683,908 courtesy taps in the SMART system. The chart below shows the numbers by reason for all courtesy taps recorded by BBB employees during the audit period.
SMART System Courtesy Tap Numbers by Reason
Beginning with the initial contract with BBB in 2017, the MBTA has provided its contracted service provider with fare access cards. Initially, these fare access cards were associated with a specific MBTA location, rather than with individual BBB employees. However, on April 1, 2019, the MBTA issued a memorandum to all BBB employees, replacing station-assigned fare access cards with fare access cards assigned to individual BBB employees. This memorandum stated the following:
Due to the recent significant increase in courtesy taps with no specific explanation and evidence that not all taps are being reported in the Smart System, the MBTA will be assigning a [fare access card (Charliecard)] to each [BBB employee]. This will enable the MBTA and BBB to track the number of courtesy taps and reporting by individual [BBB employees] and take action where necessary.
These Charliecards are to be used for work purposes ONLY and will be closely monitored via the MBTA [automated fare collection] system. The use of the Charliecard is non-transferrable and [BBB employees] will be allowed to use the Charliecard to travel to and from their work location during their assigned shift only. Each [BBB employee] will be required to sign a document acknowledging and understanding the policy and will be held solely responsible for the Charliecard for the duration of their employment with BBB. Upon resignation or termination, the [BBB employee] must return the Charliecard to BBB immediately. This Charliecard is not to be used as a BBB recruitment tool, as the MBTA is not providing free travel to BBB employees.
In interviews with the audit team, MBTA and BBB officials stated that this switch to fare access cards assigned to BBB employees was a pilot initiative that only lasted about three months. During this three-month period, which was outside of the audit period, both the MBTA and BBB stated that they closely monitored the use of these fare access cards. According to BBB officials, this initiative revealed signs of misuse of the courtesy taps (e.g., BBB employees letting people in for free and cards being used at locations not covered by those employees). Further, MBTA officials told us that the high turnover rate of BBB employees made it difficult to track these cards. As a result, after the three-month period, the MBTA made a decision to revert to station-assigned fare access cards for courtesy taps. During the audit period, the MBTA provided only station-assigned fare access cards to BBB.
Although BBB has used these cards since 2017, and despite continued signs of misuse of fare access cards, we found that the MBTA still has not implemented controls to establish accountability or effectively monitor use of these cards.
BBB provided us with a list of all courtesy taps it performed during the audit period from BBB’s SMART system. We also obtained data from the MBTA’s automated fare collection system for all of the fare access cards that the MBTA assigned to BBB during the audit period. We compared the two datasets and found that BBB was not always recording courtesy taps in the SMART system, as shown below.
As outlined in the chart above, of the 2,521,864 total transactions recorded in the MBTA’s automated fare collection system, only 669,547 (27%) had a corresponding entry documented in the SMART system. Therefore, BBB did not record 1,852,317 (73%) courtesy taps, and the reasons for these taps. Failure to properly record courtesy taps, at a rate of $2.40 per transaction, increases the risk for significant lost revenue.
Our analysis also identified 83,990 instances where these fare access cards were used at different MBTA locations than the ones where the cards were assigned. In 445 of these instances, the fare access cards were used on an MBTA bus, indicating a high probability that the card was used for personal use.
The MBTA and BBB did not have any formal written policies or procedures for the use of these fare access cards in place for the majority of the audit period. The MBTA issued a policy memorandum to all BBB employees, dated December 23, 2022, that stated the following:
The purpose of this policy memorandum is to define the proper use of [BBB employee] station assigned [fare access cards (passes)]. Effective immediately, all [BBB employees] must be made aware that these passes should only be used to assist customers who experience difficulties with their fare media by providing a “courtesy tap” upon customer request. [BBB employees] are mandated to report these taps (and reasons for the taps) via BBB’s Smart System. This will enable the MBTA and BBB to track the number of courtesy taps provided by [BBB employees] and act where necessary if/when reported taps are deemed to be excessive or in violation of this policy. Station access passes are NOT to be used in the absence of a legitimate customer request or to prevent a customer from missing a departing train. [BBB employees] are to use their “best judgment” with customers who exhibit behavior that is aggressive or belligerent in order to maintain personal safety.
Although this policy memorandum suggests that the MBTA tracks the number of courtesy taps provided by BBB employees, we found that the MBTA has not established a threshold for what it considers to be an excessive amount of use.
Although the MBTA had access to transactional data for the station-assigned fare access cards it provided to BBB during the audit period through its automated fare collection system and through its read-only access to SMART system data, we found no evidence that the MBTA used this data to monitor BBB’s use of these cards. In addition, BBB officials told us that it had been several years since the MBTA asked them to provide any data regarding the amount of and reasons for courtesy taps recorded in the SMART system. According to BBB officials, BBB’s courtesy taps have not been an area of focus for the MBTA.
Without proper controls in place, the MBTA cannot ensure that the use of these fare access cards is appropriate. Inappropriate use of fare access cards could result in potential lost revenue for the MBTA.
The MBTA should develop, document, and implement policies and procedures that ensure the proper use of fare access cards by its contracted service providers. These policies and procedures should establish a threshold for what the MBTA considers to be an excessive amount of use. The MBTA should use the automated fare collection system transaction data and available BBB SMART system reports to consistently review the use of these cards. The MBTA should hold BBB accountable for inappropriate use of these cards.
Auditee’s Response
Both the MBTA and BBB have accepted responsibility for not closely tracking the taps made and reviewed in the [Office of the State Auditor’s (OSA’s)] Report. However, the MBTA does not agree with the [OSA’s] finding that the additional taps were offering free travel to customers and therefore resulted in lost fare revenue. Beyond the station checks discussed earlier, BBB conducted 2,006,961 elevator checks and 147,314 management observations, all of which rely on the use of BBB management team-issued passes. If a [transit ambassador (TA)] is conducting a station or elevator check or responding to an incident or emergency, it will most likely be logged into the system as such, rather than also being logged as a courtesy tap. In sum, BBB’s daily operations and BBB employee’s performing work functions but not properly recording their tap do not result in lost revenue for the MBTA.
The MBTA tracks usage of assigned BBB passes monthly and monitors usage to ensure that the assigned passes are being used at the correct station or by the appropriate BBB management member. Passes found to be out of compliance are deactivated and cannot be used on the system. In addition, the passes are attached to station-issued iPads via metal zip ties. The iPads do not leave the stations as they are secured in the station booth. If a pass is lost or misplaced, only a few hours could pass before BBB management is made aware of it as the next TA coming on-shift would require a pass to conduct their job. When BBB management is made aware, the MBTA is immediately notified that the pass can be deactivated, then a replacement is sent out either by the manager on duty or a shift supervisor. There are circumstances, however, when an assigned station pass may also be used at other locations for a variety [of] legitimate purposes, such as work during a diversion, replacing a non-functional iPad with a functional one or replacing a pass at a station if a station is without a pass with the correct label.
It would be very difficult to develop a specific threshold for courtesy taps for a system as large as the MBTA given the number of incidents that occur each day requiring a TA to tap. The MBTA tracks usage by station pass and works closely with our [automated fare collection] fraud department to investigate any instances brought to our attention where a tap pass is being used at an abnormally high rate. It has not been the practice to share these monthly reports with BBB unless the MBTA is alerted to instances of abnormal tapping, such as multiple taps in a short period of time, or excessive tapping by a particular TA.
Auditor’s Reply
MBTA states in its response that “BBB conducted 2,006,961 elevator checks and 147,314 management observations, all of which rely on the use of BBB management team-issued passes” (emphasis added). We specifically requested data on station-assigned fare access cards. We did not include management-issued passes in our data. We were able to report on the reasons for all the courtesy taps in the audit period, as mentioned above within our audit report (see this chart). MBTA officials informed us that cleaners, MBTA employees, and BBB employees should have their own cards or service gate keys to enter the subway system and should not be using courtesy taps to enter. Although the MBTA is correct in stating that cleaners and the previously mentioned employees used courtesy taps for station access, this represents a failure of MBTA oversight regarding this contract and access to its stations, as these employees, according to the MBTA, are supposed to use their own access cards or service gate keys to access stations.
The MBTA indicates that the iPads are secured in station booths. This is not the case while BBB employees are on a shift, as these are what the BBB employees use to log station checks and courtesy taps. During our site visits, we noticed that they are usually on shoulder straps around BBB employees’ necks or shoulders.
The MBTA indicates in its response that it “[tracks] usage by station pass and work closely with [its automated fare collection] fraud department to investigate any instances brought to [its] attention where a tap pass is being used at an abnormally high rate.” During our audit, however, we were told by MBTA’s deputy director of fraud detection and analytics that their team does not perform any fraud monitoring of courtesy taps, because these passes are not registered to individuals.
Furthermore, the MBTA asserts in its response that “It would be very difficult to develop a specific threshold for courtesy taps for a system as large as the MBTA.” We disagree, as we feel establishing a specific threshold, and monitoring to ensure that this threshold is not exceeded, is a necessary and important part of the MBTA’s contract with BBB. In addition, it is also important to investigate instances that exceed the established threshold or when station-assigned fare access cards are used at locations other than where they are assigned. For example, as part of our data analysis, we noted that on December 31, 2022, one station-assigned fare access card was used 40 times at the Orient Heights station when it was assigned to the Wood Island station. The use of this pass 40 times in one day in a station where it was not assigned—and certainly not secured in the station booth—and the MBTA’s response to this issue indicates a lack of monitoring and control and a lack of awareness by MBTA management about how its operations are being conducted.
2. The Massachusetts Bay Transportation Authority’s Monitoring of Vendor Performance Needs Improvement.
The MBTA did not have sufficient performance measures in place to promote accountability and ensure that BBB achieved the MBTA’s desired outcomes related to safety. Although the MBTA’s Request for Proposals No. 41-17 for In-Station Customer Service stated that service-level agreements12 would be used to measure the quality of service provided by the vendor, the MBTA did not create any service-level agreements in its initial contract with BBB. Service-level agreements are a critical component of any outsourcing because they clearly define the level of service expected of the vendor; how the services provided will be measured; and the penalties, if any, should the agreed-upon service levels not be achieved.
The MBTA’s contract with BBB—which, through multiple extensions, totaled approximately $48 million between July 31, 2017 and September 30, 2022, with another estimated $28 million spent since that time—did not include any performance metrics to measure the quality of the services being provided by BBB. The contract also did not contain any penalties or incentives that may have compelled BBB to perform at a high level. Because the MBTA’s contract with BBB did not include performance metrics, the MBTA’s ability to evaluate the quality of service provided by BBB was limited. In addition, there was a higher-than-acceptable risk of unsatisfactory performance by BBB which may have resulted in wasted taxpayer and rider money.
Although the MBTA’s most recent (seventh) contract with BBB, effective October 1, 2022, includes service‑level agreements and penalties related to safety ($25–$50 per incident), this still requires the MBTA to detect noncompliance. As currently designed, the MBTA does not have the capacity to detect, and therefore enforce against, noncompliance. The service-level agreements also do not describe the specific services to be provided by BBB. The service-level agreements state the following:
[BBB] is expected to regularly report to the MBTA potential tripping and slipping hazards, or other safety hazards, located in areas of MBTA stations where [BBB’s] duties are performed. Failure to report will incur a penalty. Examples of potential hazards include but are not limited to:
- Station flooring with large sections of concrete or individual bricks missing
- Leaking water or ice on walking surfaces on station platforms and in lobbies
- Raised floor edges on stairways, ramps, platforms and in lobbies
- Debris, fixtures or signage falling from a wall or ceiling
As previously mentioned in this report, the MBTA refers to the above visual inspections of subway station locations as station checks. The MBTA expects these station checks to be completed by BBB at least twice per hour, yet it has still not incorporated this requirement into its service-level agreement. This lack of detail makes it difficult for the MBTA to enforce this requirement and increases the risk of misunderstanding between the MBTA and BBB regarding what the expectations are under the contract.
These are critical elements for management of outsourced contracts. The MBTA should strengthen its contract monitoring processes by ensuring that service-level agreements, measurable performance metrics, penalties, and incentives are included in all of its vendor contracts. The MBTA should also ensure that it develops comprehensive written contracts with vendors that clearly describe the entirety of the parties’ agreement.
Auditee’s Response
The MBTA disagrees with the Auditor’s statement that the MBTA did not have sufficient performance measures in place to promote accountability and ensure that BBB achieved the MBTA’s desired outcomes related to safety. The MBTA has consistently monitored the performance of BBB for the entirety of this contract. In fact, the agreement between the MBTA and BBB does incorporate performance monitoring via metrics. The current version of the agreement with BBB also incorporates financial incentives/penalties based on said performance monitoring.
The MBTA and BBB did not agree to service-level agreements based on [request for proposal (RFP)] 41-17. Specifically, 41-17 stipulates the following: “We believe that indicators, method of measurement and penalties/incentives should be structured” and “[s]ome combination of the following four types of metrics could be used to propose service levels and the penalties/incentives associated with those metrics[.]” BBB and the MBTA did agree to track and report monthly Service Performance Metrics through the initial and amended contract periods reviewed in the Draft Report. Over the course of the contract period the metrics have changed upon agreement between BBB and the MBTA. BBB did not respond to the RFP proposing penalties/incentives as described in RFP 41-17, as such the MBTA and BBB did not incorporate financial incentives into the agreement but did track performance of the agreed up on service performance metrics. Financial incentives along with service level metrics have been incorporated into the current agreement with BBB. The MBTA also issues monthly performance reports that are issued and quarterly business reports that are submitted to BBB for review and corrective action where necessary, as well as meeting with BBB quarterly to review the [service-level agreement] and any other contracts matters.
Auditor’s Reply
We strongly disagree with the MBTA’s above claim that they “consistently monitored the performance of BBB for the entirety of this contract.” In fact, the MBTA’s vendor management group informed us that during our audit, these service‑level agreement penalties were not carried out during the initial contract. The group also stated that although they report on elevator checks and missed shifts in 2021, they did not report on station checks or courtesy taps.
Overall, given the enormous amount of public funds expended under this contract, as well as the impact of this contract on the safety of MBTA stations across Massachusetts communities, detailed performance metrics should have been clearly documented and communicated both in the initial contract procurement and in all subsequent contracts. Vague and undocumented performance metrics may have contributed to the safety issues at several MBTA stations in recent years and to the inefficient use of taxpayer and rider dollars.
The MBTA indicates that it has incorporated financial sanctions and service-level metrics into its most recent contract with BBB (which was executed on October 1, 2022 during the audit period). We note that this is five years after the initial contract, which, as an outsourcing, should have been accompanied by clear performance metrics to allow for both appropriate vendor management and an evaluation of the propriety of continued outsourcing of this service in future years. The fact that this was not happening between July 31, 2017 and September 30, 2022 negatively impacted the MBTA’s ability to manage its contractor and likely prevented the MBTA from conducting proper evaluations of contractor work performance upon contract extension or re-procurement in 2018, 2019, 2020, 2021, and twice in 2022. We will be following up on this during our post-audit review process and strongly encourage MBTA management to incorporate our recommendations.
| Date published: | July 7, 2024 |
|---|