Overview
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Department of Fire Services (DFS) for the period July 1, 2021 through December 31, 2022.
We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.
|
Objective |
Conclusion |
|
Partially; see Finding 1 |
|
No; see Findings 2, 3, and 4 |
To achieve our audit objectives, we gained an understanding of DFS’s internal control environment related to the objectives by reviewing applicable policies and procedures and by interviewing DFS staff members and management.
We performed the following procedures to obtain sufficient, appropriate audit evidence to address the audit objectives.
Web Accessibility
To determine whether DFS’s website meets EOTSS’s Enterprise Information Technology Accessibility Policy and WCAG 2.1 for user accessibility, keyboard accessibility, navigation accessibility, language, error identification, and color accessibility, we tested a random, nonstatistical sample of 60 out of a total of 972 DFS webpages in the audit population. We performed the following procedures.
User Accessibility
- We determined whether the webpage could be viewed in both portrait and landscape modes.
- We determined whether, when zoomed in to 200%, content on the webpage was undamaged and remained readable.
- We determined whether, when zoomed in to 400%, content on the webpage was undamaged and in a single column.
Keyboard Accessibility
- We determined whether all elements of the webpage could be navigated using only a keyboard.
- We determined whether any elements on the webpage prevented a user from moving to a different element when using only a keyboard to navigate the webpage.
Navigation Accessibility
- We determined whether there was a search function to help users locate content.
- We determined whether related hyperlinks allowed users to navigate to the intended webpages.
Language
- We determined whether words that appeared on the webpage matched the language to which the webpage was set.
- We determined whether proper names were identified in PDF files included on the webpage to avoid improper translation or pronunciation errors from screen readers.
Error Identification
- We determined whether there was text explaining why an error occurred when a user input information into an entry field.
- We determined whether there were examples given to assist the user in correcting mistakes (for example, a warning when entering a letter in a field meant for numbers).
Color Accessibility
- We determined whether there was at least a 3:1 contrast in color and additional visual cues to distinguish hyperlinks, which WCAG recommends for users with colorblindness or other visual impairments.
See Finding 1 for an issue we identified with hyperlinks on DFS’s website.
IT Governance
To determine whether DFS established IT governance policies and procedures over the following areas, we performed the following procedures.
Business Continuity and Disaster Recovery
To determine whether DFS’s business continuity plan met the requirements of Section 6.1.1.4 of EOTSS’s Business Continuity and Disaster Recovery Standard IS.005, we interviewed knowledgeable DFS employees and inspected DFS’s business continuity plan to ensure that it addressed the following: critical business processes, DFS’s manual and automated processes, minimum operating requirements to resume critical functions, the designation of a business continuity lead, clearly defined and communicated roles and responsibilities, assigned points of contact, and annual updates.
To determine whether DFS’s disaster recovery plan met the requirements of Section 6.2.1 of EOTSS’s Business Continuity and Disaster Recovery Standard IS.005, we interviewed knowledgeable DFS employees and inspected DFS’s disaster recovery plan to ensure that it addressed the following:
- developing and maintaining processes for disaster recovery,
- identifying relevant stakeholders,
- conducting damage assessments of impacted IT infrastructure and applications,
- establishing procedures that allow employees facility access to restore data in an emergency,
- recovering critical agency services,
- implementing interim means for performing critical business processes at or above minimum service levels, and
- restoring service at the original site of impact without interruption.
See Finding 2 for an issue we identified with DFS’s business continuity plan.
Information Security Incident Response Plan and Procedures
To determine whether DFS’s information security incident response plan and procedures met the requirements of Sections 6.5.1 and 6.5.2 of EOTSS’s Information Security Incident Management Standard IS.009, we interviewed knowledgeable DFS employees and requested DFS’s information security incident response plan and procedures. We learned that DFS relied on the Executive Office of Public Safety and Security for an information security incident response plan and procedures, so we inspected the Executive Office of Public Safety and Security’s information security incident response plan and procedures to determine whether they met the requirements of the aforementioned EOTSS policy.
See Finding 3 for an issue we identified with DFS’s information security incident response plan and procedures.
Cybersecurity Awareness Training
To determine whether DFS’s cybersecurity awareness training met the requirements of Sections 6.2.3 and 6.2.4 of EOTSS’s Information Security Risk Management Standard IS.010, we performed the following procedures:
- We selected a random sample of 5 from a population of 10 newly hired employees and inspected their cybersecurity awareness training certificates of completion to determine whether they completed the new hire cybersecurity awareness training within 30 days of orientation.
- We selected a random sample of 20 out of a population of 93 employees who had been employed by DFS for more than one year and inspected their cybersecurity awareness training certificates of completion to determine whether they completed the annual refresher cybersecurity awareness training.
See Finding 4 for an issue we identified with DFS’s cybersecurity awareness training.
We used nonstatistical sampling methods for testing and therefore did not project the results of our testing to any population.
Data Reliability Assessment
Web Accessibility Testing
To determine the reliability of the site map spreadsheet that we received from DFS management, we interviewed knowledgeable DFS employees and checked that variable formats (e.g., dates, unique identifiers, and abbreviations) were accurate. Additionally, we ensured that none of the following issues affected the spreadsheet: abbreviation of data fields, missing data (e.g., hidden rows or columns, blank cells, and absent records), and duplicate records. We also ensured that all values in the data set corresponded with expected values.
We selected a random sample of 20 uniform resource locators (URLs)5 from the DFS site map and traced them to the corresponding webpage on DFS’s website, checking that each URL and page title matched the information on the DFS website. We also selected a random sample of 20 URLs from DFS’s website and traced the URL and page title to the site map to ensure that there was a complete and accurate population of URLs on the site map.
IT Governance Testing
To determine the reliability of the employee list we received from DFS management, we checked that variable formats (e.g., dates, unique identifiers, and abbreviations) were accurate. Additionally, we ensured that none of the following issues affected the list: abbreviation of data fields, missing data (e.g., hidden rows or columns, blank cells, and absent records), and duplicate records. We also ensured that all values in the data set corresponded with expected values.
We selected a random sample of 20 employees from the employee list and traced their names to CTHRU, the Commonwealth’s statewide payroll open records system, to verify the list’s accuracy. We also selected a random sample of 20 employees from CTHRU and traced their names back to the employee list to ensure that we received a complete and accurate employee list from DFS.
Based on the results of the data reliability assessment procedures described above, we determined that the site map and employee list were sufficiently reliable for the purposes of our audit.
| Date published: | March 20, 2024 |
|---|