Audit of the Merit Rating Board Objectives, Scope, and Methodology

An explanation of what the audit of the board examined and how it was conducted.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor (OSA) has conducted a performance audit of the Merit Rating Board (MRB) for the period July 1, 2014 through June 30, 2016.

Information technology general controls (ITGCs) are a subset of internal controls within a performance audit that are applied to every information technology (IT) system an organization relies on and to the IT staff members who administer those systems. They provide management and stakeholders with assurance regarding the reliability of data and information systems. ITGCs are meant to ensure the confidentiality, integrity, and availability of systems, programs, data files, and computer operations in an organization.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
1. Has MRB designed and effectively implemented certain ITGCs to support its mission-critical and essential application systems?
a. Has MRB designed and effectively implemented logical access security controls, such as background checks for new users; approval processes for new, transferred, and terminated user accounts; and password security, to support its mission-critical application systems?	No; see Findings 1a and 1b
b. Has MRB designed and effectively implemented change-management controls to support its mission-critical and essential application systems?	Yes
c. Has MRB designed and effectively implemented personally identifiable information (PII) controls, such as new-user Acknowledgement Forms and new-employee training, to support its mission-critical application systems, and has it established confidentiality agreements with contractors, vendors, and other third parties?	No; see Findings 2a and 2b
d. Has MRB designed and effectively implemented a business continuity plan (BCP) and disaster recovery (DR) controls, such as established policies and procedures and annual testing, to support its mission-critical and essential application systems?	No; see Findings 3a, 3b, and 3c

We conducted this performance audit using criteria from the Massachusetts Department of Transportation’s (MassDOT’s) information security program (ISP), which documents how MassDOT agencies should keep data secure and reduce the risk of unauthorized disclosure. If MassDOT’s ISP was deficient in a certain area, we relied on industry standards established by (1) the Information Systems Audit and Control Association in its document Control Objectives for Information and Related Technologies (COBIT) 4.2, (2) the National Institute of Standards and Technology, and (3) the Massachusetts Office of Information Technology (MassIT). Although MRB is not required to follow these industry standards, we believe they represent IT-industry best practices for ITGCs. For example, the purpose of COBIT is to provide management and business-process owners with an IT governance model that helps them deliver value from IT and understand and manage the risks associated with IT. According to the Information Systems Audit and Control Association’s website,

COBIT helps bridge the gaps amongst business requirements, control needs and technical issues. It is a control model to meet the needs of IT governance and ensure the integrity of information and information systems.

We gained an understanding of the internal controls we deemed significant to our audit objectives through interviews and observations, and we evaluated the design of MRB’s logical access security, change management, and PII controls, as well as whether it had a BCP and DR plan.

To achieve our objectives, we performed the following audit procedures:

• We assessed logical access security controls, such as background checks for new users; approval processes for new, transferred, and terminated user accounts; and password security during the audit period. Specifically, we performed the following procedures:
  • We requested and verified documentation of the Criminal Offender Record Information checks for all four new employees who were hired during the audit period to ensure that these employees had been properly screened before gaining access to the Automated License and Registration System (ALARS).
  • We reviewed all four new users’ access approvals by verifying their managers’ signatures and dates to ensure that their access to ALARS had been properly approved.
  • We observed the MassDOT Human Resources (HR) Compensation Management System database administrator building a Structured Query Language query to obtain the MRB employee list. The database administrator sorted the data, exported the data to Excel, and saved the data on a USB drive provided by OSA.
  • We also noted that no employees transferred during the audit period, so we did not request documentation to determine whether the access rights of transferred employees had been adjusted properly to reflect their new roles.
  • We compared the employee termination list provided by HR to an ALARS user-account termination list to determine whether the accounts of all nine employees who were terminated during the audit period had been removed upon termination.
  • We interviewed the IT security officer to determine whether user accounts and account access rights to ALARS were reviewed periodically. We reviewed the accounts of all four new users and all nine terminated employees to verify their access status.
  • We selected a nonstatistical judgmental sample of emails to determine whether user accounts and account access rights to ALARS were reviewed periodically. Since the IT security officer sends quarterly emails to MRB management, we selected emails from six out of eight quarters for review. Because our sampling was nonstatistical, we did not project the results of our audit tests to the total populations in the areas we reviewed.
  • We requested password parameters for ALARS and observed the testing of those parameters to ensure that the requirements set by MassDOT were followed. To achieve this, we observed while the ALARS security officer tested the password parameters.
• We assessed MRB’s controls to determine whether change management (monitoring, documenting, and approving modifications to the IT system) was administered adequately. Specifically, we performed the following procedures:
  • We interviewed the MRB IT consultant and the customer-engagement manager for the MassDOT Information Technology Division (MassDOT IT) to obtain an understanding of how the testing and production environments functioned. We also observed the MRB IT consultant logging into the testing and production servers to verify that production and testing environments were separate.
  • We reviewed the MassDOT IT flowchart for the project proposal process and the change-management policy to gain an understanding of the steps and requirements that MRB follows at different phases of IT project change management.
  • We reviewed 100% of the management signoffs of two change tickets (open issues discovered during testing of the two File Transfer Protocol [FTP] server replacement projects) that occurred during our audit period.
• We assessed MRB’s controls to determine whether protected information was properly safeguarded. The controls included new-user Acknowledgement Forms; new-employee training; and confidentiality agreements with contractors, vendors, and other third parties. Specifically, we performed the following procedures:
  • We interviewed MassDOT’s HR managers to obtain an understanding of MRB’s onboarding process for new employees and to verify that all four new employees signed Acknowledgement Forms before gaining access to MRB’s protected information.
  • We reviewed training documentation provided by MassDOT’s HR manager and interim training manager to determine whether all four new employees had completed security training before working with protected information.
  • We reviewed confidentiality agreements for all four new employees to verify that they had all signed the agreements before working with protected information.
  • We interviewed managers from MassDOT and MRB to determine whether documentation regarding data classification and data inventory existed.
  • We observed the MRB IT consultant logging into the server administrative account and obtained screenshots of internal FTP server security controls, such as login identification and password parameters.
  • We interviewed MRB management and requested the MassDOT Shared Vendor Assessment document and the Service Organization Controls Reports to determine the security controls of the service provider’s² FTP server.
• We assessed MRB’s controls to determine whether BCP and DR processes, such as established policies and procedures and annual testing, were in place and properly managed to support its mission-critical and essential application systems. Specifically, we performed the following procedures:
  • We interviewed MassDOT IT managers and MRB IT personnel and obtained documentation related to the 2014 MassIT DR test to determine whether the DR test was performed annually. This documentation consisted of a backup and recovery test schedule for June 2014 created by MassIT for ALARS and various other state systems.
  • We assessed the reliability of MRB data in ALARS and the HR system. Specifically, we reviewed existing information and interviewed knowledgeable staff members about the data. In addition, we performed validity and integrity tests on all data, including (1) testing for missing data, (2) scanning for duplicate records, (3) testing for values outside a designated range, and (4) looking for dates outside specific time periods. Based on our analysis, we determined that the data were sufficiently reliable for the purposes of this audit.

2. Microsoft is the service provider that operates the FTP server, which insurance companies use to transmit data and information to MRB.

 

Date published:	February 1, 2018