Audit of the Massachusetts Teachers’ Retirement System Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Massachusetts Teachers’ Retirement System.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Massachusetts Teachers’ Retirement System (MTRS) for the period July 1, 2023 through June 30, 2024.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our finding and conclusion based on our audit objective.

Below is our audit objective, indicating the question we intended our audit to answer, the conclusion we reached regarding our objective, and where our objective is discussed in the audit finding.

Objective	Conclusion
Did MTRS’s website (mtrs.state.ma.us) comply with the Executive Office of Technology Services and Security’s Enterprise Information Technology Accessibility Policy and the Web Content Accessibility Guidelines (WCAG) 2.1 for user accessibility, keyboard accessibility, navigation accessibility, language accessibility, error identification, and color accessibility?	No; see Finding 1

To accomplish our audit objective, we gained an understanding of the MTRS internal control environment relevant to our objective by reviewing applicable policies and procedures and by interviewing MTRS staff members and management. In addition, to obtain sufficient, appropriate evidence to address our audit objective, we performed the procedures described below.

Web Accessibility

To determine whether MTRS’s website adhered to WCAG 2.1, for user accessibility, keyboard accessibility, navigation accessibility, language accessibility, error identification, and color accessibility during the audit period, we performed the following procedures for a random, nonstatistical⁹ sample of 50 webpages from a population of 511 MTRS webpages:

User Accessibility

• We determined whether content on each webpage could be viewed in both portrait and landscape modes.
• We determined whether content on each webpage was undamaged and remained readable when zoomed in to both 200% and 400%.

Keyboard Accessibility

• We determined whether all elements¹⁰ on each webpage could be navigated using only keyboard commands.
• We determined whether any elements on each webpage prevented a user from moving to a different element when using only keyboard commands to navigate the webpage in question.
• We determined whether the first focusable control¹¹ on each webpage was a hyperlink that would redirect users to the main content of the webpage.

Navigation Accessibility

• We determined whether each webpage contained a title that was relevant to the webpage’s content.
• We determined whether there was a search function present to help users locate content across the whole website.
• We determined whether hyperlinks correctly navigated to the intended webpages.
• We determined whether headings within webpages related to the content of the section below the header.

Language Accessibility

• We determined whether any video content found on each webpage had all important sounds and dialogue captioned.
• We determined whether the words that appeared on each webpage matched the language attribute¹² to which the webpage in question was set.
• We determined whether any webpage sections that contained language differing from that to which the webpage was set contained their own specified language attribute.

Error Identification

• We determined whether mandatory form fields alerted users if they left these fields blank.
• We determined, for form fields that required a limited set of input values, whether users were alerted if invalid values were entered into these types of fields.
• We determined whether there were labels for any elements that required user input. We also determined whether these labels were programmed correctly.
• We determined whether examples were presented to assist users in correcting mistakes (for example, a warning when entering a letter in a field meant for numbers).

Color Accessibility

• We determined whether there was at least a 3:1 contrast in color and additional visual cues to distinguish hyperlinks, which WCAG recommends for users with colorblindness or other visual impairments.

We used nonstatistical sampling methods for testing and therefore did not project the results of our testing to any corresponding populations.

For our objective, we found certain issues during our testing of the accessibility of MTRS’s website. See Finding 1 for more information.

Data Reliability Assessment

To determine the reliability of the URL list that we received from MTRS management, we interviewed knowledgeable MTRS staff members and checked that certain variable formats (e.g., dates, unique identifiers, and abbreviations) were accurate. Additionally, we ensured that none of the following issues affected the URL list: abbreviation of data fields, missing data (e.g., hidden rows or columns, blank cells, or absent records), and duplicate records. We also ensured that all values in the dataset corresponded with expected values.

We selected a random sample of 20 URLs from the URL list and traced each to the corresponding webpages on MTRS’s website, checking that each URL and webpage title from the URL list matched the information on the MTRS website. We also selected a random sample of 20 URLs from MTRS’s website and traced the URL and webpage title to the URL list to ensure that there was a complete and accurate population of URLs on the URL list.

Based on the results of the data reliability procedures described above, we determined that the URL list was sufficiently reliable for the purposes of our audit.

9.    Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items are not similar enough, or there are specific items in the population that the auditors want to review.

10.   An element is a part of a webpage that contains data, text, or an image.

11.   The first focusable control is the first element a user will be brought to on a webpage when navigating with a keyboard. If the first focusable control also redirects users to the main content of a webpage, then it is known as a bypass block or a skip link.

12.    A language attribute (also known as a language tag) identifies the native language of the content on the webpage or PDF (e.g., a webpage in English should have an EN language attribute). The language attribute is listed in the webpage’s or PDF’s properties. This, among other things, is used to help screen readers use the correct pronunciation for words.

Date published:	October 16, 2025