Overview
The MBTA did not maintain an accurate Bid Comparison to Cost Estimate Database of information on all DBB projects, nor did it require contracted design engineers to perform a root cause analysis on each bid that varied from the engineer’s estimate by more than 10%. For all of its DBB projects, the MBTA maintains a database of information used to compare the engineers’ estimates to project bids. If a bid varied from the engineer’s estimate by more than 10%, the MBTA did not ensure that its design engineer for the project analyzed it to determine why it was substantially different from the agency’s estimates (when that was the case) or ensure that the design engineer documented its conclusions in a root cause analysis.
The MBTA Did Not Ensure Information in Its Bid Comparison to Cost Estimate Database Was Accurate.
We found several problems with the information in the MBTA’s Bid Comparison to Cost Estimate Database:
- For four contracts, the MBTA used an incorrect engineer’s estimate, resulting in an incorrect calculation of the variance between the estimate and the bids.
- For one contract, the MBTA used the wrong bid amount, resulting in an incorrect variance calculation.
- Two contracts were listed twice.
Without maintaining accurate information in the database, the MBTA is at risk of not capturing the appropriate information needed to calculate variances between project estimates and bids and could be analyzing erroneous data.
Authoritative Guidance
MBTA management is responsible for ensuring that the information in this database is complete and accurate so that the agency can use it to effectively administer the bid analysis process for construction contracts.
Reasons for Noncompliance
The MBTA did not provide a reason for data entry errors in the Bid Comparison to Cost Estimate Database. However, we found that management had not established effective monitoring controls to ensure that the data entered into the database were accurate.
MBTA Did Not Ensure Root Cause Analysis on Bids That Varied From Estimate by Over 10% Were Performed
For the 29 DBB contracts, there were 108 bids received that varied from the engineers’ estimates by more than 10%. The MBTA did not appear to have required the project’s contracted design engineer to perform and document a root cause analysis for 93 of the bids. Without requiring design engineers to perform a root cause analysis on each bid that varies from the engineer’s estimate by more than 10%, the MBTA is not taking all possible measures to ensure that the bids submitted by the low bidders for projects are reasonable (e.g., that a contractor did not submit an artificially low bid to win the contract, with the intent of increasing its compensation through contract change orders) or to determine whether the information used by its contracted design engineers in developing project cost estimates is complete and accurate, which could affect project budgeting.
Authoritative Guidance
Section 8 of the MBTA Project Manager’s Manual Standard Operating Procedure, released January 27, 2014, states,
Contract Administration [a department within the Capital Delivery Department] maintains a database comparing the engineer estimates to actual bids. In the event a bid is received that varies from the engineers estimate by more than 10%, the engineer shall include in the bid analysis a Root Cause Analysis explaining the reasons for the variance.
MBTA management is responsible for ensuring that all the requirements of its bidding procedure are followed.
Reasons for Noncompliance
MBTA management explained that a root cause analysis was only performed for the lowest bid that varied from the engineer’s estimate by more than 10% and stated that there was no value in performing them on each bid that varied from the engineer’s estimate by more than 10%.
Recommendation
MBTA management should establish effective monitoring controls to ensure that information is accurately recorded in the Bid Comparison to Cost Estimate Database and that a root cause analysis is performed for each bid received that varies from the engineer’s estimate by more than 10%.
Auditee’s Response
In a response to Finding 4a dated June 18, 2018, the MBTA stated,
The auditors identified four contracts (A90CN08, C72CN01, and S19CN01 & W46CN03) and indicated the MBTA used incorrect [engineer’s estimate] data. The MBTA investigated this and confirmed that the engineer’ estimate was incorrectly entered for two of the engineer’s estimates. The correct engineer’s estimate are as follows:
- A90CN08—$4,539,149.00 (Entered Correctly)
- C72CN01—$1,990,000.00—(The correct estimate is $1,199,00.00)
- S19CN01—$11,163,297—(Entered correctly)
- W46CN03—$1,980,531.00—(The correct estimate is $1,908,531.00)
The report referenced in the DRAFT audit report does not reflect the comparisons of bids report that details the engineer’s estimate compared to the low bidder as well as other bidders. The “Comparison of Bids” report is the official report that is utilized for official bid results that is published on the MBTA’s web site. The engineer’s estimate does sometimes change from the time it is advertised through the bid phase due to questions and clarifications that may result in changes to quantities within the bidding documents. However, as identified by the auditor, these estimates were incorrectly entered and the MBTA will take necessary steps to ensure this does not happen in the future. After bid openings, data entry will be reviewed and signed off by a Senior Manager.
In response to bullet number 2 (For one contract, the MBTA used the wrong bid amount, resulting in an incorrect variance calculation) and bullet number 3 (Two contracts were listed twice), the MBTA offers the following response:
For Contract No. A90CN06 the MBTA did enter the incorrect bid amount. The correct bid amount should be $13,570,000.00. Additionally, Contract No. R40CN01 was entered twice. The reason that this contract number was entered into the database twice was because the MBTA canceled the original R40CN01 bid after bid opening due to the need to revise the scope of the contract. Contract No. R40CN01 was rebid months later using the same contract number but with “Re-Bid” added to the contract title. MBTA Contract No. B73CN01 was the other contract that was listed twice. This contract should have only been listed once.
Going forward the MBTA will provide more senior management review and controls to ensure that all engineer’s estimates and bid values are accurately recorded and the process for review will be outlined in the Contract Administration [Department’s] Standard Operating Procedure (SOP).
In a subsequent response, dated July 26, 2018, the MBTA stated, “Senior Managers within the Department are currently reviewing the MBTA cost estimate database on all contracts to ensure that Engineer’s Estimates and bid values are accurately recorded.”
In a response to Finding 4b dated June 18, 2018, the MBTA stated,
The intent of Section 8 (Bidding) of the MBTA Project Manager’s Manual Standard Operating Procedure is to perform a root cause analysis in the event the low bid received varies from the engineer’s estimate by more than 10%. A root cause analysis is performed on the low bid if the low bid is not within 10% of the engineer’s estimate and typically the other bids are not analyzed. The MBTA has not required a root cause analysis on bids other than the low bid.
As a result of discussions during the audit process, the MBTA does agree that there is value in performing a root cause analysis on all bids if the low bid varies from the engineer’s estimate by more than 10% to explain the variances. If the low bid falls within 10% of the engineer’s estimate then the MBTA proposes that the Engineer of Record (EOR) perform a root cause analysis on the low bid only. All appropriate documents and manuals will be updated accordingly.
In a subsequent response, dated July 26, 2018, the MBTA stated,
The MBTA has also initiated steps to effectively administer its construction contract bid analysis, as well as ensuring that a root cause analysis is performed on bids varying by more than 10% of Engineer’s Estimate.
Additionally, the MBTA has required its Engineer of Record (EOR), to perform a root cause analysis on the low bid only if the low bid varies by more than 10% of the Engineer’s Estimate. The EOR has included review of market trends and recent bid comparisons on similar scope projects in their analysis. All steps referenced above have been incorporated into MBTA Standard Operating Procedures and appropriate manuals accordingly.
Auditor’s Reply
As noted above, OSA found that the MBTA did not ensure that the information in its Bid Comparison to Cost Estimate Database was accurate. The Bid Comparison to Cost Estimate Database was provided to OSA by MBTA officials during the audit and was represented by MBTA management as the official accurate record of this information. Therefore, we used it in conducting our audit testing in this area. The Comparison of Bids report referred to in the MBTA’s response was never provided to OSA or discussed with us during our audit fieldwork; therefore, we were unaware of its existence.
In its response, the MBTA asserts that two of the four engineers’ estimates that OSA reported as incorrect in the MBTA’s Bid Comparison to Cost Estimate Database (those for Contracts A90CN08 and S19CN01) were recorded correctly. However, when OSA attempted to reconcile the two engineers’ estimates in that database to the amounts in three other MBTA databases (the Contract Bidder Info, Winning Bids Database; the Contract Bidder Info All Bids Database; and the Engineer Estimate and Actual Cost Variance Database), the amounts in those three databases matched one another but did not match those in the Bid Comparison to Cost Estimate Database. This supports our assertion that the estimates for these projects in the Bid Comparison to Cost Estimate Database were inaccurate.
Based on its response, the process the MBTA is following in reviewing its bids is unclear and does not appear to follow its Standard Operating Procedure, which requires a root cause analysis to be performed on bids that vary from the engineer’s estimate by more than 10%. Specifically, in its June response to our draft report, the MBTA agreed that when the low bid varies by more than 10% of the engineer’s estimate, there is value in reviewing all bids; however, when the low bid is within 10% of the engineer’s estimate, the MBTA proposed to analyze only the low bid. In contrast, in its July response, the MBTA stated that it performs a root cause analysis on bids that vary from the engineer’s estimates by more than 10%, while also stating that it analyzes low bids only if they vary from the estimates by more than 10%. OSA recommends that the MBTA follow its Standard Operating Procedure by performing a root cause analysis of any bid that varies from the engineer’s estimate by more than 10%, regardless of the low bid amount. The analysis will provide the MBTA with the necessary information to ensure that it receives the most accurate pricing.
| Date published: | September 28, 2018 |
|---|