Audit of the Massachusetts Maritime Academy Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Massachusetts Maritime Academy

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of the Massachusetts Maritime Academy (MMA) for the period July 1, 2017 through December 31, 2019. Because crime statistics related to the Jeanne Clery Disclosure of Campus Security Policy and Campus Crime Statistics Act (Clery Act) are reported by calendar year, we expanded our audit period to January 1, 2017 through December 31, 2019 for Objectives 1 and 2 below.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer, the conclusion we reached regarding each objective, and where each objective is discussed in the audit findings.

Objective	Conclusion
Does MMA record all crimes committed within its geography in a daily crime log in accordance with the Clery Act (Section 1092[f] of Title 20 of the United States Code [USC]) and related requirements (Section 668.46 of Title 34 of the Code of Federal Regulations [CFR] and the 2016 edition of the United States Department of Education’s [US ED’s] Handbook for Campus Safety and Security Reporting)?	No; see Finding 1
Does MMA report Clery Act crime statistics and certain required statements about policies in its annual security reports (ASRs) in accordance with the Clery Act (20 USC 1092[f]) and related requirements (34 CFR 668.46 and the 2016 edition of US ED’s Handbook for Campus Safety and Security Reporting)?	No; see Findings 2 and 4
Does MMA provide education programs to newly enrolled students and newly hired employees in accordance with the Clery Act (20 USC 1092[f]) and related requirements (20 USC 1011i, 34 CFR 668.46[j][1], and No. 202 of Volume 79 of the Federal Register)?	No; see Finding 3

 

To achieve our objectives, we gained an understanding of MMA’s internal control environment related to the objectives by conducting interviews and observing processes practiced by MMA officials who were responsible for administering Clery Act–related activities. We also reviewed MMA’s policies and procedures, Regimental Manual, internal control plan, strategic plan, and voyage plan.

We performed the following procedures to obtain sufficient, appropriate audit evidence to address the audit objectives.

Campus Crime Tracking

MMA maintains three separate sources for tracking crime activity within its geography: TriTech, Colleague disciplinary action records, and Class I student offense files.⁵ Because Clery Act crimes are required to be reported by calendar year, we expanded our audit period as previously mentioned to include all of calendar year 2017 for this objective. During our audit, we noted that the three sources that MMA used to record crime information did not specifically identify the Clery Act crime categories for crimes, incidents resulting in disciplinary actions, and student offenses. To determine whether a reported crime, disciplinary action, or student offense that occurred during our audit period fell within a Clery Act crime category, we reviewed all of the information in each of these three sources and compared it to the definitions of incidents that are subject to Clery Act reporting as detailed in the following authoritative sources: the Summary Reporting System (SRS) User Manual,⁶ the National Incident-Based Reporting System (NIBRS) User Manual,⁷ the Hate Crime Data Collection Guidelines and Training Manual,⁸ and the hierarchy rule.⁹ For each Clery Act crime we identified, we documented the location within MMA’s geography and the calendar year of occurrence. To determine whether MMA recorded all Clery Act crimes in a daily crime log that was available for public inspection, we performed procedures in the following three sources:

1. TriTech—We generated a TriTech report of all 2,901 incidents recorded by Public Safety Office staff members in MMA’s daily crime log during calendar years 2017, 2018, and 2019. We noted that each incident recorded in the daily crime log was classified into 1 of 58 categories. We compared each of the 58 categories to Clery Act crime categories to identify which categories might include Clery Act crimes. We identified five categories—(1) larceny/forgery/fraud, (2) burglary / breaking and entering, (3) disturbance, (4) alcohol violations on school property, and (5) sex offenses—that might fall within Clery Act crime categories. We then compared all 20 incident records that were in those categories to Clery Act crime definitions to identify which incidents were in Clery Act crime categories. We found 6 such incidents.
2. Colleague disciplinary action records—We generated a report of all 292 disciplinary action records from Colleague for calendar years 2017, 2018, and 2019. We compared each of the 292 records to Clery Act crime definitions to identify which incidents were in Clery Act crime categories. We found 112 such incidents.
3. Class I student offense files—We identified a total population of 26 Class I student offense files maintained in MMA’s Student Services Office for calendar years 2017, 2018, and 2019. We compared the contents of all 26 files to Clery Act crime definitions to identify which offenses were in Clery Act crime categories. We found 15 such offenses.

After identifying this total population of 133 incidents in these three sources that fell within Clery Act crime categories and occurred within MMA’s geography during calendar years 2017, 2018, and 2019, we classified and totaled incidents by Clery Act crime category, location within MMA geography, and calendar year. We then compared our results with the information MMA maintained in its daily crime log and reported in its ASRs, described below.

Clery Act ASRs

To determine whether MMA’s ASRs contained accurate and complete Clery Act crime statistics and required statements about policies, we compared the information regarding the 133 Clery Act crimes we identified that occurred during calendar years 2017, 2018, and 2019 to the Clery Act crime statistics MMA reported in its 2020 ASR for the same three-year period. As previously mentioned, we expanded our audit period for this objective to include all of calendar year 2017. We also compared certain disclosures MMA made in its 2018 and 2019 ASRs (e.g., statements regarding its Clery Act–related education/training programs, campus security policies and procedures, and collaboration with state and local police agencies related to campus security) to the ASR disclosures required by the Clery Act for each year to determine whether MMA made the required disclosures.

Student and Employee Clery Act Education

To determine whether MMA provided the primary crime prevention and awareness training required by the Clery Act, and alcohol abuse awareness training required by its practices, to newly enrolled students and newly hired employees, we reviewed a list of 5,198 student training records and 502 employee training records from EVERFI, MMA’s online training software. We compared the list from EVERFI with an MMA-provided, Colleague-generated list of the 1,508 students who enrolled in MMA during our audit period and a list of the 45 employees MMA hired during the audit period according to the Human Resources Compensation Management System (HR/CMS), the Commonwealth’s payroll system, to verify that MMA had assigned the training to these students and employees. We also compared the student enrollment dates and employee hire dates to the EVERFI course assignment dates to determine whether MMA assigned the courses within 30 days of enrollment or hire. Finally, we compared the EVERFI course completion dates to the course assignment dates to determine whether students and employees completed the courses within 30 days of the assignment dates.

Data Reliability Assessment

We reviewed certain general information system controls, including security management, access controls, configuration management, segregation of duties, and contingency planning, over EVERFI to determine the reliability of the data therein. We conducted interviews with MMA officials who were knowledgeable about the data. We extracted 9,460 rows of data from EVERFI and identified 5,198 student training records and 502 employee training records for our audit period. We looked for blank records and duplicate records. We also looked for assignment dates and completion dates that were outside the audit period.

We reviewed certain general information system controls, including security management, access controls, configuration management, segregation of duties, and contingency planning, over TriTech to determine the reliability of the data therein. We conducted interviews with MMA officials who were knowledgeable about the data. We extracted rows of daily crime log data from TriTech, totaling 10,707, 8,501, and 9,199 rows for calendar years 2017, 2018, and 2019, respectively. We combined these rows of daily crime log data and identified 2,901 reported incidents in the TriTech daily crime log that occurred during calendar years 2017 through 2019. We also looked for blank records and record dates that fell outside these calendar years.

We extracted 21,363 rows of data related to disciplinary action records from Colleague and identified 292 student disciplinary action records for calendar years 2017, 2018, and 2019. For these 292 records, we verified that the date of offense was during these calendar years and that the following data fields were populated: student identification number (ID), student name, demerit code (an MMA code indicating the type of offense), demerit code description, and demerit narrative (MMA’s description of the offense).

We extracted 1,551 rows of student enrollment data from Colleague and identified 1,508 new students enrolled during the audit period. We selected a random sample of 20 of the 1,508 new students and traced each student’s ID; name; birthdate; start of term; and, if applicable, withdrawal code (an MMA code indicating the type of withdrawal), withdrawal reason, and withdrawal date to student documents retained by MMA’s registrar in Image Silo, a document retention system.

We observed the MMA human resources director accessing HR/CMS and generating a list of the 45 employees hired during the audit period. We verified that all data fields for employee ID, name, title, and hire date were populated; hire dates were within the audit period; and no duplicate records existed. In addition, we selected a random sample of 20 employees and traced each employee ID, name, title, and hire date to hardcopy employee personnel files retained in MMA’s Human Resources Department.

Based on the results of our data reliability assessments, we determined that the information obtained for our audit period was sufficiently reliable for the purpose of our audit objectives.

5.    These are hardcopy files that contain documentation of disciplinary actions related to Class I student offenses.

6.    This manual is from the Federal Bureau of Investigation’s Uniform Crime Reporting Program and defines Clery Act crimes listed in 34 CFR 668.46.

7.    This manual is from the Federal Bureau of Investigation’s Uniform Crime Reporting Program and defines Clery Act crimes listed in 34 CFR 668.46.

8.    This manual is from the Federal Bureau of Investigation’s Uniform Crime Reporting Program and defines Clery Act crimes listed in 34 CFR 668.46.

9.    This rule is defined in 34 CFR 668.46(a) as “a requirement in the [Federal Bureau of Investigation’s Uniform Crime Reporting Program] that, for purposes of reporting crimes . . . when more than one criminal offense was committed during a single incident, only the most serious offense be counted.”

Date published:	March 3, 2022