Audit of the Massachusetts Department of Transportation Aeronautics Division Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Massachusetts Department of Transportation Aeronautics Division

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Massachusetts Department of Transportation’s (MassDOT’s) Aeronautics Division for the period July 1, 2019 through June 30, 2021.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
1.     For the contract between the Aeronautics Division and Aurigo Software Technologies, Inc., did the division establish a review process and assign a project manager to ensure that changes made to the software development project under contract were reviewed and approved in accordance with the executed contract’s “Statement of Work”?	Yes
Has the Aeronautics Division established information technology (IT) policies and procedures for the acceptable use of its IT resources and for cybersecurity awareness training in accordance with MassDOT’s “Acceptable Use of Information Technology (IT) Resources” policy and Sections 6.2.3 and 6.2.4 of the Executive Office of Technology Services and Security’s (EOTSS’s) Information Security Risk Management Standard IS.010?	No; see Finding 2
Has the Aeronautics Division established a business continuity plan (BCP) in accordance with EOTSS’s Business Continuity and Disaster Recovery Standard IS.005?	No; see Finding 1
Did the Aeronautics Division register aircraft and record the registration revenue generated during the audit period in accordance with the MassDOT Aeronautics Division Finance Department Policies and Procedures Manual?	Yes
Has the Aeronautics Division updated its internal control plan (ICP) in accordance with the Office of the Comptroller of the Commonwealth’s (CTR’s) September 30, 2020 memorandum “[2019 Coronavirus, or] COVID-19 Pandemic Response Internal Controls Guidance”?	No; see Finding 3
Does the Aeronautics Division Drone Program operate in accordance with Section 48 of Title 14 of the Code of Federal Regulations (CFR), 14 CFR 107.61, 14 CFR 107.73, and the “MassDOT Aeronautics Drone Team Inventory Standard Operating Procedures”?	Yes, but see Other Matters

 

To achieve our objectives, we gained an understanding of the Aeronautics Division’s internal controls related to the objectives by reviewing applicable policies and procedures, as well as conducting inquiries with division staff members and management. We evaluated the design, and tested the operating effectiveness, of internal controls used by the division to register drone pilots and aircraft.

To obtain sufficient, appropriate audit evidence to address our audit objectives, we conducted the following audit procedures:

• To determine whether the Aeronautics Division had established a process to review its Aurigo Software Technologies, Inc. contract, we obtained and reviewed the contract to verify that a project manager had been assigned to the project and an implementation team had reviewed and authorized changes before they were implemented.
• To determine whether the Aeronautics Division had established IT policies and procedures, we obtained and reviewed MassDOT’s “Acceptable Use of Information Technology (IT) Resources” policy and EOTSS’s Information Security Risk Management Standard IS.010. We requested signed “Acceptable Use of Information Technology (IT) Resources” policies for all personnel.
• To determine whether Aeronautics Division employees completed cybersecurity awareness training, we obtained and compared a list of all active personnel during the audit period along with their hire dates and requested all cybersecurity awareness training certificates for these personnel that were issued during the audit period.
• To determine whether the Aeronautics Division had established a BCP, we requested the plan from division management.
• To determine whether the Aeronautics Division registered aircraft and recorded registration revenue generated during the audit period in accordance with the MassDOT Aeronautics Division Finance Department Policies and Procedures Manual, we reviewed all the notification emails that the division sent to airport managers notifying them that their based aircraft lists³ were due.
• To determine whether the Aeronautics Division established and implemented policies and procedures for the registration of aircraft and the collection of aircraft registration fees during the audit period, we obtained and reviewed the MassDOT Aeronautics Division Finance Department Policies and Procedures Manual.
• We obtained all aircraft registration and registration revenue data for the audit period. The data included the aircraft registration years, Federal Aviation Administration (FAA) registration numbers, Aeronautics Division registration numbers, aircraft serial numbers, Aeronautics Division registration dates, and Aeronautics Division registration fee amounts. We then selected a random, statistical sample of 50 aircraft registrations from a population of 3,946, with a 90% confidence level, 5% tolerable error rate, and 0% expected error rate. To ensure that the division deposited the correct amounts in its operating bank account, we verified the amounts on the bank deposit detail statements, the check totals, the check numbers, and the totals from the bank’s deposited check summaries and matched them all to the aircraft registration and registration revenue data. The division uses CTR’s Cash Transfer Input form to track the transfer of money to the Commonwealth, which is meant to occur after fees are deposited; we matched the totals on this form to the deposit summary in Aurigo Masterworks, a software application that the Aeronautics Division implemented in 2017 and used to maintain aircraft registration and revenue information. (The Aeronautics Division decommissioned Aurigo Masterworks in April 2021 upon the partial implementation of its Salesforce application.⁴) Finally, we inspected the Cash Transfer Input form for supervisor signoff to ensure that the division created a separation of duties.
• To determine whether the Aeronautics Division updated its ICP in accordance with CTR’s September 30, 2020 memorandum “COVID-19 Pandemic Response Internal Controls Guidance,” we requested the ICP. We obtained and reviewed the division’s latest ICP draft, dated July 2016.
• To determine whether the Aeronautics Division Drone Program operated in accordance with 14 CFR 107.61 and 107.73, we reviewed the requirements and found that all drone pilots must be registered with FAA. We obtained and reviewed a list of all eight of the division’s licensed pilots and their FAA airman registry certificate numbers. We verified that all eight pilots were registered with FAA on FAA’s airman registry website.
• We requested and reviewed the Aeronautics Division’s drone inventory list and conducted an onsite inventory reconciliation at the division’s headquarters. There were 21 drones on the list. We reviewed the list for the following: drone nickname, manufacturer, model, serial number, FAA drone registration number, registration issuance date, registration expiration date, and registration status (“active” or “inactive” in the FAA database). We then matched the serial numbers and FAA registration numbers to the drones and verified that 14 drones were on site. We also verified that the 14 drones externally displayed their FAA registration numbers.
• Through photographic evidence, we verified that the seven drones that were not at the Aeronautics Division’s headquarters displayed their FAA registration numbers externally; what the seven drones’ FAA registration numbers were; and what their serial numbers were.

Data Reliability Assessment

To gain an understanding of Aurigo Masterworks, we interviewed division personnel. To assess the reliability of the data provided to us from the application, we verified that the data did not contain blank fields, duplicates, or dates outside the audit period and that it did contain necessary data fields. We traced a random sample of 20 hardcopy aircraft registrations to the Aurigo Masterworks application to verify the accuracy and completeness of the application.

Drone data for this audit were in the form of spreadsheets. The Aeronautics Division provided a spreadsheet of drone pilots with their FAA airman registry certificate numbers and registry certificate issue dates. To assess the reliability of the data in this spreadsheet, we verified that the data did not contain blank records or duplicates. We interviewed knowledgeable personnel about employee access to the spreadsheet. We selected a sample of five pilots and matched their FAA airman registry certificate numbers from the spreadsheet to the FAA airman registry website.

To assess the reliability of the drone inventory spreadsheet, we verified that the data on the spreadsheet did not contain blank records or duplicates. We traced a sample of five drone records from the drone inventory spreadsheet to the physical drones. We traced the FAA drone registration numbers and serial numbers on the exteriors of the five drones to the drone inventory spreadsheet to verify them.

Based on the procedures described above, we determined that the Aurigo Masterworks, pilot, and drone data were sufficiently reliable for the purposes of our audit.

3.    This is a list of aircraft registered at any of the 35 public-use airports over which the Aeronautics Division has jurisdiction.

4.    Salesforce is a cloud-based customer relationship management software program that the Aeronautics Division uses to track aircraft registrations and process revenue from aircraft registration fees.

Date published:	June 30, 2022