Audit of the Executive Office of the Trial Court Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Executive Office of the Trial Court

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor (OSA) has conducted a performance audit of certain activities of the Executive Office of the Trial Court (EOTC) for the period July 1, 2023 through June 30, 2024.

We conducted this performance audit in accordance with generally accepted government auditing standards,¹² except for the following areas:¹³

• Paragraph 8.39 of Chapter 8 (which pertains to determining whether internal controls are significant to the audit objectives);
• Paragraphs 8.59–8.67 of Chapter 8 (which pertains to information systems controls considerations);
• Paragraphs 8.72–8.76 of Chapter 8 (which pertains to fraud inquiries);
• Paragraphs 8.77 and 8.78 of Chapter 8 (which pertains to identifying sources of evidence and the amount and types of required evidence);
• Paragraph 8.80 of Chapter 8 (which pertains to determining whether other auditors have conducted, or are conducting, audits that could be relevant to the current audit objectives);
• Paragraph 8.90 of Chapter 8 (which pertains to obtaining sufficient, appropriate evidence to provide a reasonable basis for addressing the audit objectives and supporting their findings and conclusions); and
• Paragraph 9.50 of Chapter 9 (which pertains to obtaining and reporting the views of responsible officials from the auditee concerning the findings, conclusions, and recommendations in the audit report, as well as any planned corrective actions).

We believe that, except for the information outlined in the “Scope Limitations” section, the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is our audit objective, indicating the question we intended our audit to answer; the conclusion we reached regarding our objective; and, if applicable, where our objective is discussed in the audit findings.

Objective	Conclusion
Did EOTC’s website adhere to the World Wide Web Consortium’s (W3C) Web Content Accessibility Guidelines (WCAG) 2.1 for user accessibility, keyboard accessibility, navigation accessibility, language accessibility, error identification, and color accessibility?	No; see Finding 1

Scope Limitations

Paragraph 9.12 of the US Government Accountability Office’s Government Auditing Standards states, “Auditors should . . . report any significant constraints imposed on the audit approach by information limitations or scope impairments.” While performing the audit, we encountered the scope limitations described below.

In response to OSA’s letter of intent to conduct a performance audit (see Appendix A), EOTC’s general counsel, on behalf of EOTC’s chief justice and the court administrator, formally declined participation via written correspondence (see Appendix B). The general counsel asserted that the audit would exceed OSA’s statutory authority under Section 12 of Chapter 11 of the General Laws.

The general counsel further contended that WCAG 2.1, established by W3C, are not government regulations and, therefore, are not enforceable against EOTC. The general counsel maintained that OSA does not have authority to require compliance with, or to audit against, WCAG 2.1. Instead, EOTC stated that it follows the requirements of the US Department of Justice’s Final Rule, Section 35 of Title 28 of the Code of Federal Regulations (Civil Rights Division Docket No. 144, 89 Final Rule 31320),¹⁴ which uses Level AA of WCAG 2.1 as the technical standard for accessibility under Title II of the Americans with Disabilities Act. The deadline established by the US Department of Justice to ensure that websites are compliant with Levels A and AA of WCAG 2.1 is April 24, 2026.

Additionally, while the Enterprise Information Technology Accessibility Policy, issued by the Executive Office of Technology Services and Security (EOTSS) in 2021, requires agencies using EOTSS services, such as Mass.gov, to comply with Levels A and AA of WCAG 2.1, EOTC’s general counsel argued that this policy does not create a legal obligation for EOTC to follow WCAG 2.1 and reaffirmed that OSA does not have enforcement authority in this regard.

Consequently, EOTC declined to allow us to conduct our audit, even though OSA made efforts to resolve the jurisdictional issue by holding meetings with EOTC representatives to clarify this issue. As a result, we were denied access to essential information required to accomplish our audit objective.

As a result of EOTC’s denial to grant us access to essential information required to accomplish our audit objective, we independently compiled a list of URLs¹⁵ from EOTC’s Mass.gov website¹⁶ in order to test our objective. When selecting URLs, we relied on our professional judgement¹⁷ and the webpage tagging system within Mass.gov. The list of URLs obtained through this approach may not be a complete population of EOTC’s webpages.

Web Accessibility

To determine whether EOTC’s website adhered to W3C’s WCAG 2.1 for user accessibility, keyboard accessibility, navigation accessibility, language accessibility, error identification, and color accessibility, we took the following actions. First, we performed accessibility testing procedures on a random, nonstatistical¹⁸ sample of 60 EOTC webpages out of a population of 290. We performed the procedures described below on the sampled webpages.

User Accessibility

• We determined whether content on each website could be viewed in both portrait and landscape modes.
• We determined whether content on each webpage was undamaged and remained readable when zoomed in to both 200% and 400%.

Keyboard Accessibility

• We determined whether all elements¹⁹ of each webpage could be navigated using only keyboard commands.
• We determined whether any elements on each webpage prevented a user from moving to a different element when using only keyboard commands to navigate the webpage in question.
• We determined whether the first focusable control²⁰ on each webpage was a hyperlink that would redirect users to the main content of the webpage.

Navigation Accessibility

• We determined whether each webpage contained a title that was relevant to the webpage’s content.
• We determined whether there was a search function present to help users locate content.
• We determined whether hyperlinks correctly navigated to the intended webpages.
• We determined whether headings within webpages related to the content of the header’s section.

Language Accessibility

• We determined whether any video content found within each webpage had all important sounds and dialogue captioned.
• We determined whether the words that appeared on each webpage matched the language attribute²¹ to which the webpage in question was set.
• We determined whether any webpage sections that contained language differing from that to which the webpage was set contained their own specified language attribute.

Error Identification

• We determined whether mandatory form fields alerted users if they left these fields blank.
• We determined, for form fields that required a limited set of input values, whether users were alerted if invalid values were entered into these types of fields.
• We determined whether there were labels for any elements that required user input.
  • We also determined whether these labels were programmed correctly.
• We determined whether examples were presented to assist users in correcting mistakes (for example, a warning when entering a letter in a field meant for numbers).

Color Accessibility

• We determined whether there was at least a 3:1 contrast in color and additional visual cues to distinguish hyperlinks, which WCAG recommends for users with colorblindness or other visual impairments.

We used nonstatistical sampling methods for testing and therefore did not project the results of our testing to any corresponding populations.

For this objective, we found certain issues during our testing; see Finding 1 for more information.

EOTC’s denial to grant us access to essential information materially affected our ability to carry out necessary procedures. Therefore, in accordance with generally accepted government auditing standards, this report discloses a scope limitation, which may affect the completeness and reliability of the audit results.

12.    Generally accepted government auditing standards is an auditing term used to refer to the standards issued by the US Government Accountability Office, which are listed in its Government Auditing Standards. (The version referenced in this report is the April 2021 technical update, GAO-21-368G.) These standards provide a framework for conducting high-quality audits of government programs and entities. They outline the ethical principles, professional qualifications, and performance requirements that auditors must follow when conducting financial and performance audits—as well as other types of engagements—to ensure accountability and transparency.

13.    We were unable to satisfy the requirements of the listed generally accepted government auditing standards because the auditee refused to participate in the audit.

14.    You can find the text of this rule here.

15.    A URL uniquely identifies an internet resource, such as a website.

16.   Webpages within Mass.gov can be tagged with the names of Commonwealth entities, thus indicating which Commonwealth entity authored the webpage. In the case of EOTC, a user would see something like “Offered by Executive Office of the Trial Court” underneath the website header and above the webpage heading. See the “Accessibility Features of a Website” image in this report for an example of this.

17. Professional judgement, in this situation, involved auditors reviewing the content of the sampled webpages, as well as the path to navigate to said webpage. For example, the webpage “Trial Court Administrative Order 22-3: Elimination of Monthly Probation Fees Imposed Pursuant to [Section 87A of Chapter 276 of the General Laws]” (see https://www.mass.gov/doc/trial-court-administrative-order-22-3-elimination-0f-monthly-probation-fees-imposed-pursuant-to-gl-c-276-ss-87a) is a webpage that auditors could navigate to using only hyperlinks within webpages tagged with “Offered by Executive Office of the Trial Court.”

18.    Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items are not similar enough, or there are specific items in the population that the auditors want to review.

19.   An element is a part of a webpage that contains data, text, or an image.

20.   The first focusable control is the first element a user will be brought to on a webpage when navigating with a keyboard. If first focusable control also redirects users to the main content of a webpage, then it is known as a bypass block or a skip link.

21.    A language attribute (also known as a language tag) identifies the native language of the content on the webpage or PDF (e.g., a webpage in English should have an EN language attribute). The language attribute is listed in the webpage’s or PDF’s properties. This, among other things, is used to help screen readers use the correct pronunciation for words.

Date published:	August 6, 2025