Vendor Digital Accessibility Testing Obligations
The vendor shall perform digital accessibility testing on its digital products as part of its development and quality assurance processes and prior to delivery to the Commonwealth. Digital accessibility testing includes Automated Accessibility Testing and Manual Accessibility Testing.
Automated Accessibility Testing
Automated Accessibility Testing refers to accessibility testing using a scanning tool that examines code and flags violations of the Web Content Accessibility Guidelines (WCAG) success criteria. Automated Accessibility Testing is limited in its ability to identify Accessibility Violations. Vendors must perform Automated Accessibility Testing of their digital products against WCAG version 2.1 level A and AA or later and remediate any violations prior to delivery to the Commonwealth. Additionally, automated testing should occur as part of the vendor’s quality assurance processes during the development lifecycle. The vendor shall deliver Automated Accessibility Testing results upon request by the Commonwealth.
Manual Accessibility Testing
Manual Accessibility Testing refers to testing performed by a human tester using manual methods including the use of assistive technology. At a minimum, Manual Accessibility Testing must occur within the environments specified in the Digital Accessibility Testing Matrix defined in Table 1. Due to the limitations of Automated Accessibility Testing, vendors must perform Manual Accessibility Testing of their digital products during development, customization, or configuration and prior to delivery to the Commonwealth. The vendor shall deliver Manual Accessibility Testing results upon request by the Commonwealth.
Examples of Manual Accessibility Testing methods include but are not limited to the following:
- Keyboard-only operability and navigation
- Screen reader operability and compatibility
- Use of color
- Color contrast
- Focus management
- Reflow and responsiveness
- Text resize
- Manual code inspection
- Consistent navigation and identification
- Error identification and error suggestion for user input
Digital Accessibility Testing Matrix
At a minimum, vendors must perform Manual Accessibility Testing in the environments specified in the Digital Accessibility Testing Matrix as defined in Table 1, including testing performed to support claims of compliance with WCAG 2.1 A and AA.
| Platform | Operating System | Browser | Screen Reader Testing | Color Contrast Testing | Text Resize | Reflow and Responsive-ness | Keyboard-Only Testing |
|---|---|---|---|---|---|---|---|
| Desktop | Windows (latest version) | Chrome or Edge | NVDA and JAWS (latest version) | Standard and High Contrast Themes | 200% | 400% | Keyboard operability and navigation |
| Desktop | Mac OS (latest version) | Safari or Chrome | VoiceOver | Standard and Dark Mode | 200% | 400% | Keyboard operability and navigation |
| Mobile | iOS (latest version) | Safari | VoiceOver | Standard and Dark Mode | Not applicable | Not applicable | Keyboard operability and navigation |
| Mobile | Android (latest version) | Chrome | TalkBack | Standard and Dark Mode | Not applicable | Not applicable | Keyboard operability and navigation |
Access to Test Digital Products
In addition to testing its own digital products for accessibility, the vendor shall make its digital products available for accessibility testing by the Commonwealth upon request. Vendor shall cooperate and provide the Commonwealth, or any third-party digital accessibility service provider procured by the Commonwealth, access to test its digital products for accessibility at no additional cost to the Commonwealth. This may include but is not limited to providing the Commonwealth with login credentials to test a fully operational product in production, or access to test the digital products in a sandbox environment.
Validation of Remediation and Compliance
Written Validation refers to the documented confirmation that an Accessibility Violation has been successfully remediated. This validation is based on testing against criteria such as the Web Content Accessibility Guidelines (WCAG) and includes the technique implemented to remediate the violation and the testing method used to validate conformance. An Accessibility Conformance Report (ACR) is insufficient as a stand-alone measure of compliance with WCAG 2.1 A and AA. Thus, vendors shall provide additional Written Validation of successful remediation or a certification of compliance upon request by the Commonwealth.