Audit of the Salem State University Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Salem State University.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of Salem State University (SSU) for the period January 1, 2020 through December 31, 2022.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
Did SSU include all required policies, procedures, and statements in its annual security report (ASR) in accordance with the Jeanne Clery Disclosure of Campus Security Policy and Campus Crime Statistics Act (Clery Act) (Section 668.46[b–h] of Title 34 of the Code of Federal Regulations [CFR])?	Yes
Did SSU record all crimes within its Clery geography in a daily crime log and accurately report these crimes to the US Department of Education (US DOE) and in its ASR in accordance with the Clery Act (34 CFR 668.46[c][1] and [f][1])?	No; see Finding 1
Did SSU have a process in place to ensure that it identified campus security authorities (CSAs) and that these employees completed training on their responsibilities as CSAs in accordance with the Clery Act (34 CFR 668.46[a])?	No; see Finding 2

To accomplish our audit objectives, we gained an understanding of the aspects of SSU’s internal control environment relevant to our objectives by reviewing applicable policies and procedures and by interviewing SSU officials.

To obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures described below.

ASR-Required Policies, Procedures, and Statements

To determine whether SSU included all required policies, procedures, and statements in its ASR in accordance with the Clery Act (34 CFR 668.46[b–h]), we inspected SSU’s ASRs for calendar years 2021, 2022, and 2023. These ASRs included Clery Act–required policies, procedures, and statements for the audit period, January 1, 2020 through December 31, 2022.

We noted no significant exceptions in our testing. Therefore, we concluded that SSU included all required policies, procedures, and statements in its ASRs.

ASR Clery Act Crime Statistics

To determine whether SSU recorded all crimes within its Clery geography in a daily crime log and accurately reported these crimes to US DOE and in its ASR in accordance with the Clery Act (34 CFR 668.46[c][1] and [f][1]), we took the actions described below.

We inspected SSU’s 2023 ASR and electronic submission to US DOE, which included Clery Act crime statistics for calendar years 2020 through 2022 (the audit period). We compared the Clery Act crime statistics published in SSU’s 2023 ASR to those that SSU submitted to US DOE to ensure that they matched.

To ensure that all cases from the daily crime log that must be reported under the Clery Act were included in the Clery Offense Report and reported in SSU’s 2023 ASR, we took the following actions:

• We obtained a list of all 1,534 cases from the daily crime log for the audit period from the SSU Police Department (SSUPD).
• We inspected this list to identify the total number of cases that fell within each of the four categories of Clery Act crimes (as described in the “Daily Crime Log” section of this report).
• We compared the total number of cases we identified as Clery Act crimes to the total number of Clery Act crimes SSU included in its Clery Offense Report and the 2023 ASR.
• We verified that all Clery Act crimes that SPD provided to SSUPD were included in SSU’s 2023 ASR.
• We followed up with SSUPD employees to ask about any variances we identified (i.e., crimes reported in SSU’s ASR but not on the daily crime log in accordance with Section 98F of Chapter 41 of the General Laws,⁷ crimes that are only reportable if they are hate crimes, and crimes that did not occur on or within SSU’s Clery geography).

To ensure that all incidents from SSU’s disciplinary action record management system that must be reported under the Clery Act were included in the standardized Clery Report, the daily crime log, and SSU’s 2023 ASR, we took the following actions:

• We obtained a list of all 2,784 incidents from the disciplinary action record management system for the audit period from SSU’s associate dean of students.
• We inspected this list to identify the total number of incidents that fell within each of the four categories of Clery Act crimes (as described in the “Daily Crime Log” section of this report).
• We compared the total number of incidents we identified as Clery Act crimes to the total number of Clery Act crimes SSU included in its Clery Report, daily crime log, and its 2023 ASR.
• We followed up with employees from SSU’s Resident Life and Student Life Departments to ask about any variances we identified (i.e., Clery Act crimes that were reported in SSU’s ASR but not in SSU’s disciplinary action record management system, and vice versa).

See Finding 1 for information on the results of this testing.

CSAs

To determine whether SSU had a process in place to ensure that it identified CSAs and that these employees completed training on their responsibilities as CSAs in accordance with the Clery Act (34 CFR 668.46[a]), we took the actions described below.

We interviewed SSUPD employees to determine how SSU identifies CSAs and trains them on their responsibilities. SSUPD provided us the Microsoft PowerPoint presentation it uses to train CSAs on the Clery Act and the sign-in sheets used for these trainings during the audit period. We reviewed the sign-in sheets to determine which CSAs took the trainings.

Because SSUPD did not follow the CSA identification process during the audit period, SSUPD could not provide us with a list of CSAs who were active during the audit period. However, SSUPD provided us with a list of identified CSAs from 2015 (the most current list as of the time of our audit), which we used to identify the job titles of employees who should have been identified as CSAs during the audit period. To ensure that appropriate job titles were consistently identified as CSAs, we compared all of SSU’s CSA training sign-in sheets for the audit period to the following:

• SSU’s list of identified CSAs in 2015;
• the Clery Act definition of a CSA;
• SSU’s definition of CSAs published in its 2021 ASR;
• SSU’s definition of CSAs published in its 2022 ASR; and
• SSU’s definition of CSAs published in its 2023 ASR.

Additionally, while comparing these lists, we inspected all of SSU’s CSA training sign-in sheets for the audit period to determine whether CSA trainings were provided to all CSAs identified in the items in the bulleted list above.

See Finding 2 for information on the results of this testing.

Data Reliability Assessment

Daily Crime Log Data

To determine the reliability of the daily crime log data maintained in SSUPD’s case management system, we interviewed SSUPD employees who were knowledgeable about the daily crime log data and SSU’s chief information security officer. We tested certain general information system controls (including security management, access controls, configuration management, and contingency planning for SSUPD’s case management system) to determine the reliability of the data. SSUPD provided us with the daily crime log data for the audit period, which consisted of 1,534 cases. We inspected the daily crime log data for duplicate case numbers to determine whether a case number appeared more than once. Additionally, we inspected the data for gaps in the sequential case numbers to determine whether cases were missing or deleted from the dataset. We conducted a trend analysis by reviewing the case dates within the audit period to confirm our prediction that fewer cases occurred during the months of May through August, when fewer students were on campus. We also followed up with SSUPD employees to understand the reason for the gaps in the sequential case numbers and missing dates that we found while analyzing the daily crime log data.

To determine the accuracy of the daily crime log data from the 1,534 cases that occurred during the audit period, we judgmentally⁸ selected a sample of 25 cases. For each case in our sample, we listened to the recording of the initial call made to SSUPD and traced the date, time, location, and type of alleged incident to the daily crime log data and verified that the information from the recording matched the daily crime log data.

Incident Data

To determine the reliability of the incident data in SSU’s disciplinary action records management system maintained by the Resident Life and Student Life Departments, we interviewed SSU employees who were knowledgeable about the data. We tested certain general information system controls (including security management, access controls, configuration management, and contingency planning for this system) to determine the reliability of the data.

SSU’s associate dean of students provided us with a list of all incidents entered into the disciplinary action records management system for the audit period, which consisted of 2,784 incidents (1,883 incidents reported through SSU’s website and 901 incidents manually entered into the disciplinary action records management system by SSU professional employees). We inspected the list of 2,784 incidents for duplicate incident numbers to determine whether any incident numbers appeared more than once. Additionally, we inspected the list of 2,784 incidents for gaps in the sequential incident numbers to determine whether any incident numbers were missing or deleted from the dataset. We followed up with SSU employees to understand the reason for the gaps in the sequential incident numbers that we found while analyzing this list.

In addition, SSU’s associate dean of students provided us with a list of 1,883 incident reports made through SSU’s website; from this list, we selected a random sample of 20 incidents. For each incident in our sample, we traced the date, time, location, and type of report to the list of 2,784 incidents from the disciplinary action records management system to verify that the incident reports made through SSU’s website matched the records in the disciplinary action records management system.

Based on the results of the data reliability assessment procedures described above, we determined that the information we obtained was sufficiently reliable for the purposes of our audit.

7.    Section 98F of Chapter 41 of the General Laws prohibits the release of certain daily crime log “information concerning responses to reports of domestic violence, rape or sexual assault” from being a public record.

8.   Auditors use judgmental sampling to select items for audit testing when a population is very small, the population items are not similar enough, or there are specific items in the population that the auditors want to review. Auditors use their knowledge and judgment to select the most appropriate sample. For example, an auditor might select items from areas of high risk. The results of testing using judgmental sampling cannot be used to make conclusions or projections about entire populations; however, they can be used to identify specific issues, risks, or weaknesses.

Date published:	November 4, 2024