Vendor Digital Accessibility Testing Obligations

Where a third-party is contracted to build and deliver digital products or services to an executive department, office or agency of the Commonwealth, the vendor must perform digital accessibility testing of Deliverables to meet its contractual obligations. This page provides additional information about a vendor’s accessibility testing obligations in accordance with the Commonwealth’s Vendor Accessibility Contract language.

Table of Contents

Vendor Digital Accessibility Testing Obligations

The vendor shall perform digital accessibility testing on its digital products as part of its development and quality assurance processes and prior to delivery to the Commonwealth. Digital accessibility testing includes Automated Accessibility Testing and Manual Accessibility Testing.

Automated Accessibility Testing

Automated Accessibility Testing refers to accessibility testing using a scanning tool that examines code and flags violations of the Web Content Accessibility Guidelines (WCAG) success criteria. Automated Accessibility Testing is limited in its ability to identify Accessibility Violations. Vendors must perform Automated Accessibility Testing of their digital products against WCAG version 2.1 level A and AA or later and remediate any violations prior to delivery to the Commonwealth. Additionally, automated testing should occur as part of the vendor’s quality assurance processes during the development lifecycle. The vendor shall deliver Automated Accessibility Testing results upon request by the Commonwealth.  

Manual Accessibility Testing

Manual Accessibility Testing refers to testing performed by a human tester using manual methods including the use of assistive technology. At a minimum, Manual Accessibility Testing must occur within the environments specified in the Digital Accessibility Testing Matrix defined in Table 1. Due to the limitations of Automated Accessibility Testing, vendors must perform Manual Accessibility Testing of their digital products during development, customization, or configuration and prior to delivery to the Commonwealth. The vendor shall deliver Manual Accessibility Testing results upon request by the Commonwealth. 

Examples of Manual Accessibility Testing methods include but are not limited to the following:

  • keyboard-only operability and navigation
  • screen reader operability and compatibility
  • use of color
  • color contrast
  • focus management
  • reflow and responsiveness
  • text resize
  • manual code inspection
  • consistent navigation and identification
  • error identification and error suggestion for user input

Digital Accessibility Testing Matrix

At a minimum, vendors must perform Manual Accessibility Testing in the environments specified in the Digital Accessibility Testing Matrix as defined in Table 1, including testing performed to support claims of compliance with WCAG 2.1 A and AA.

Table 1: Digital Accessibility Testing Matrix

PlatformOperating SystemBrowserScreen Reader TestingColor Contrast TestingText ResizeReflow and Responsive-nessKeyboard-Only Testing
DesktopWindows (latest version)Chrome or EdgeNVDA and JAWS (latest version)Standard and High Contrast Themes200%400%Keyboard operability and navigation
DesktopMac OS (latest version)Safari or ChromeVoiceOverStandard and Dark Mode200%400%Keyboard operability and navigation
MobileiOS (latest version)SafariVoiceOverStandard and Dark ModeNot applicableNot applicableKeyboard operability and navigation
MobileAndroid (latest version)ChromeTalkBackStandard and Dark ModeNot applicableNot applicableKeyboard operability and navigation

Access to Test Digital Products

In addition to testing its own digital products for accessibility, the vendor shall make its digital products available for accessibility testing by the Commonwealth upon request. Vendor shall cooperate and provide the Commonwealth, or any third-party digital accessibility service provider procured by the Commonwealth, access to test its digital products for accessibility at no additional cost to the Commonwealth.  This may include but is not limited to providing the Commonwealth with login credentials to test a fully operational product in production, or access to test the digital products in a sandbox environment.

Validation of Remediation and Compliance

Written Validation refers to the documented confirmation that an Accessibility Violation has been successfully remediated. This validation is based on testing against criteria such as the Web Content Accessibility Guidelines (WCAG) and includes the technique implemented to remediate the violation and the testing method used to validate conformance. An Accessibility Conformance Report is insufficient as a stand-alone measure of compliance with WCAG 2.1 A and AA.  Thus, vendors shall provide additional Written Validation of successful remediation or a certification of compliance upon request by the Commonwealth.  

[DRAFT note]

This page type doesn't offer a feature to add resources related to the subject of the page.  To add resource pages, they need to be associated with a specific section.

Related

Help Us Improve Mass.gov  with your feedback

Please do not include personal or contact information.
Feedback