Audit of the Massachusetts Water Resources Authority Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Massachusetts Water Resources Authority.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Massachusetts Water Resources Authority (MWRA) for the period July 1, 2019 through June 30, 2021.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
Did MWRA implement a risk and resilience assessment (RRA) for its administrative computer network and supervisory control and data acquisition (SCADA) system in the areas of authorized employee access that support the monitoring of drinking water and wastewater, as required by Section 2013 of the America’s Water Infrastructure Act (AWIA)?	No, see Findings 1, 2, 3, and 4
Did MWRA implement an RRA as it pertains to the use, storage, delivery, or handling of the six chemicals used to treat water at the John J. Carroll Water Treatment Plant and William A. Brutsch Water Treatment Facility, as required by Section 2013 of the AWIA?	Yes
Did MWRA implement an emergency response plan (ERP) as it pertains to the resilience of physical security at the John J. Carroll Water Treatment Plant, Deer Island Treatment Plant, and Wachusett Reservoir, as required by Section 2013 of the AWIA?	Yes
Did MWRA have physical security measures in place to prevent unauthorized access to its water supply facilities and the Deer Island Treatment Plant?	Yes

 

To achieve our objectives, we gained an understanding of MWRA’s internal control environment related to the objectives by reviewing applicable policies and procedures, as well as by interviewing MWRA’s management and staff members. We evaluated the design and tested the effectiveness of MWRA’s process of annually updating standard operating procedure(s) for the use, storage, delivery, or handling of the six chemicals to treat the water.

RRA—Network Access

To verify that MWRA implemented an RRA for its administrative computer network and its SCADA system, as required by Section 2013 of the AWIA, we performed the following procedures.

We obtained a Microsoft Excel list of the names of all 1,364 active and terminated users provided by MWRA’s Human Resources Department. We stratified the list of active and terminated users into 1,342 administrative computer network users and 22 SCADA system users. We selected all 22 SCADA system users and selected a random, nonstatistical sample of 80 out of the 1,342 administrative computer network users.

• To determine whether users received authorization for access, we reviewed the Network Shared Permission Requests to verify each user’s name and supervisor permission.
• To determine whether the Management Information System Department granted multifactor authentication to users, we compared the access permission granted for multifactor authentication to actual remote access usage.
• To confirm that employees and contractors completed cybersecurity awareness trainings, we reviewed each user’s training completion dates in their account for two years of cybersecurity awareness training.
• In addition, we selected all nine terminated contractors and selected a random, nonstatistical sample of 35 out of 143 MWRA terminated employees with user access for a sample of 44 users. We compared the date of the end of their employment to the date of revoked access for each user and looked for timely deactivation of user access.

See Finding 1 for an issue we identified related to MWRA’s information security program. See Findings 2, 3, and 4 for issues we identified related to MWRA’s administrative computer network.

RRA—Chemical Use, Storage, Delivery, or Handling

We determined whether MWRA implemented an RRA as it pertains to the use, storage, delivery, or handling of the six identified chemicals used at the John J. Carroll Water Treatment Plant and William A. Brutsch Water Treatment Facility, as required by Section 2013 of the AWIA. To do this, we selected a random, nonstatistical sample of 95 out of 1,580 chemical deliveries from MWRA’s Microsoft Excel list of all chemical deliveries.

For our testing, we compared the information for each chemical delivery in our sample to the corresponding vendor invoice. Information included the delivery date, the quantity of the chemical, the amount paid, and a signature by an MWRA employee with signatory authority, which were on a bill of lading,⁶ a certificate of analysis, an MWRA form (specific to the chemical), and the scale weight ticket included with each delivery. We also compared the volume calculation on the electronic scale ticket to the invoice volume and confirmed that the delivery documentation was accurate. We also reviewed the MWRA form and verified that it had the required MWRA receiver’s signature at the line titled “Documentation Review and Delivery Hook-up.”

We also selected a random, nonstatistical sample of 8 out of 24 months of the audit period and reviewed each month’s Chemical Addition Report for each of the six chemicals, verified that each chemical was used daily, and verified that the report had the appropriate MWRA employees’ signatures.

We noted no exceptions in our testing; therefore, we conclude that MWRA implemented an RRA as it pertains to the use, storage, delivery, or handling of the six chemicals used to treat water at the John J. Carroll Water Treatment Plant and William A. Brutsch Water Treatment Facility, as required by Section 2013 of the AWIA.

Emergency Response Plan

To determine whether MWRA implemented an ERP as it pertains to the resilience of physical security at the John J. Carroll Water Treatment Plant, Deer Island Treatment Plant, and Wachusett Reservoir, as required by Section 2013 of the AWIA, we selected a judgmental, nonstatistical sample of 20 physical security incidents⁷ out of the 72 incidents that occurred during the audit period at any one of these three locations. We gathered information from cameras as well as badge and motion detector activity at various locations for each incident by date. We confirmed that MWRA had the physical security equipment (camera, badge, and motion detector) data related to the incidents.

We analyzed the total number of transactions (e.g., badge scans or activity captured by motion detectors) for each of the 20 incidents in the physical security equipment data by location and calculated the average number of transactions. We reviewed the description of the incidents for initiation of the ERP steps and for any unauthorized access to MWRA property.

We performed site visits and observed physical security equipment at the John J. Carroll Water Treatment Plant and Deer Island Treatment Plant. We also reviewed evidence to support whether 24-hour security and deterrence signage methods were in place at multiple locations through interviews and direct observation.

We noted no exceptions in our testing; therefore, we conclude that MWRA implemented an ERP as it pertains to the resilience of physical security at the John J. Carroll Water Treatment Plant, Deer Island Treatment Plant, and Wachusett Reservoir, as required by Section 2013 of the AWIA.

Physical Security Measures

To determine whether MWRA had physical security measures in place to prevent unauthorized access to its water and wastewater facilities, we selected a random, nonstatistical sample of 35 incidents from a population of 203 incidents (from all of MWRA’s facilities) during our audit period. We reviewed the details of the 35 incidents to determine whether the security of the water quality was directly affected or whether outside agencies, such as the state and local police, were called for assistance. We divided the 35 incident reports into three categories: significant, borderline, and insignificant, based on the type of incident. For the sample of 35 incidents, we determined whether MWRA’s Security Department’s responses to the incidents were in agreement with the defined protocols established in MWRA’s “Security Guard After Hours Alarm and Event Handling.” Based on MWRA’s descriptions, we believed that six incidents appeared to be of a more significant nature than the other incidents. We assessed whether the contractor and/or MWRA employee took actions in accordance with MWRA’s standard operating procedures. We inquired with MWRA, discussed its responses to these six incidents, and verified that the contractor and/or MWRA staff member followed the steps outlined within its ERP, if necessary.

We noted no exceptions in our testing; therefore, we conclude that MWRA had physical security measures in place to prevent unauthorized access to its water supply facilities and the Deer Island Treatment Plant.

When nonstatistical sampling methods were used, we could not project the results of our testing to the population.

Data Reliability Assessment

To determine the reliability of the Microsoft Excel list of names of active and terminated employees from MWRA’s Human Resources Department, we reconciled the list to MWRA’s payroll report list. We checked the Microsoft Excel list of names for hidden rows and columns and/or formulas. We performed electronic tests for duplicate identification numbers and names within our audit period.

To determine the reliability of the Microsoft Excel list of all chemical deliveries that was exported from MWRA’s procurement database, we filtered the deliveries for the six chemicals. We selected a random, nonstatistical sample of 20 out of 1,580 deliveries from the Microsoft Excel list of all chemical deliveries and traced each delivery to a bill of lading, certificate of analysis, and scale weight ticket. We also randomly selected source documents (bills of lading, certificates of analysis, and scale weight tickets) for 20 deliveries from MWRA files and traced the delivery information from those files to the Microsoft Excel list of all chemical deliveries. We also analyzed the chemical delivery list for hidden rows, columns, or formulas. We used Audit Command Language to check for duplicate data in our audit period and check that there were no large gaps in data files.

To determine the reliability of the badge, camera, and motion detector data obtained from the contractor who provided the security data system / database, we reconciled the data pulled on a specific date to the total of all transactions that we received. We also inspected the data (representing badges, cameras, and motion detectors) for hidden rows, columns, or formulas and imported the data into the Audit Command Language data analytics system. We tested for duplicates and dates outside the audit period.

To determine the reliability of the list of incidents tracked by MWRA’s director of security during the audit period, we selected a random, nonstatistical sample of 10 out of 203 incidents from the list and traced them to Security Incident Reports. We randomly selected a nonstatistical sample of 10 Security Incident Reports and traced the incident identification number and description of the incident from the reports to the list of incidents. We performed other electronic tests, including checking for hidden rows, columns, or formulas; checking that data was in our audit period; testing for duplicates; and testing for large gaps.

Based on the results of our data reliability assessment, we determined that the information obtained for our audit period was sufficiently reliable for the purposes of the audit.

6.    A bill of lading is a list of the items (in this case, chemicals) in a shipment.

7.    MWRA told us that a physical security incident is an event that violates MWRA policy or a law and/or compromises the safety of an MWRA employee, contractor, visitor, or MWRA property—for example, vandalism, theft, or accidental damage to MWRA property.

Date published:	August 25, 2023