What is an Accessibility Conformance Report?
An Accessibility Conformance Report (ACR)” is a document that details how well a product or service meets an established accessibility standard such as the Web Content Accessibility Guidelines (WCAG). A completed, VPAT® is referred to as an Accessibility Conformance Report (ACR). While the abbreviations ACR and VPAT® are often used interchangeably, an ACR is a completed report while a VPAT® is a blank template. Note that the existence of an ACR is not a guarantee of conformance but rather a measurement of how well a digital product or service meets accessibility standards.
How to Use this Checklist
The following checks can help procurement teams determine the validity and reliability of a vendor’s ACR. Each check is phrased as a yes/no question; a “Yes” response is positive and supports the validity of the report, while a “No” response may raise concerns or indicate that the ACR does not meet the Commonwealth’s ACR requirements as defined in the Vendor Digital Accessibility Contract Language.
It is recommended that procurement teams discuss their findings with their respective Secretariat Accessibility IT Officer (SAIO) for additional insights.
Accessibility Conformance Report Checklist
Indicate a “Yes” or “No” response to the following questions. Use the guidance provided to perform each check.
Report Date
- Is the ACR no more than 12 months old at the time of submission?
Request the most current version of the ACR from the Vendor. On the first page of the report, check the “Report Date” field. If the date is more than a year old, indicate “No” for this check.
Product and Version
- Is the ACR specific to the product or version proposed for use?
On the first page of the report, check the “Name of Product” and “Product Description” fields. If the report does not measure the product or version proposed for use, indicate “No” for this check.
Report Creator
- Was the ACR completed by knowledgeable personnel with digital accessibility expertise or a reputable third-party digital accessibility firm?
Check responses to the Vendor Digital Accessibility Questionnaire if one was completed.. An external digital accessibility firm will typically include their company name and logo on the report. Consult with your SAIO on the reputation of any external digital accessibility firm indicated on the report. If completed by the vendor’s internal personnel, reach out to the person listed in the “Contact Information” field and ask for the name of the person who created the report, their title, and their level of digital accessibility expertise.
Accessibility Standards
- Does the report measure conformance with WCAG 2.1 or 2.2 levels A and AA?
In the Accessibility Standards table, check if “Yes” is indicated adjacent to WCAG 2.1 levels A and AA or WCAG 2.2 levels A and AA. Note that AAA success criteria are not required. If only WCAG 2.0 success criteria or section 508 standards were tested, indicate “No” for this check.
Evaluation Method
- Did the evaluation method include both Automated and Manual Accessibility Testing?
- Did testing occur in the environments specified in the Digital Accessibility testing Matrix?
In the “Evaluation Method Used” field, check for the use of assistive technologies such as Non-Visual Desktop Access (NVDA), Job Access with Speech (JAWS), keyboard-only navigation, color contrast testing, and browser and operating system settings such as zoom and high contrast. Check if the browsers and operating systems are indicated and consistent with the Digital Accessibility Testing Matrix. If the Evaluation Methods Used field is left blank or only includes an automated scanning tool, indicate “No” for this check.
Scope of Testing
- Does the report clearly identify the scope of testing such as the specific pages or screens, functions, or tasks that were tested?
- If the digital product or service has both an admin interface and an end user interface, does the report indicate which interface was tested?
An ACR is typically created based on testing of a representative sample of pages or screens, or specific functions or tasks. It is important to know what pages or functions were tested to determine if the report is relevant to its intended use. Also, if a product has more than one interface, the vendor should provide a separate report for each interface. Check if the scope of testing was included in the “Evaluation Methods Used” field. If no scope is defined, indicate “No” for this check.
Conformance Level Terms
- Do the terms used in the conformance level column include Supports, Partially Supports, Does Not Support, Not Applicable, or Not Evaluated?
In the conformance level column of the tables, check that the terms used are consistent with those defined in the “Terms” section of the report. If there are any instances where other terms were used such as “Pass” or “Fail”, indicate “No” for this check.
Remarks and Explanations
- For any success criteria where the conformance level is “Partially Supports” or “Does Not Support”, are detailed explanations provided including the location of the violation?
- For any success criteria where the conformance level is “Supports”, are examples or other remarks provided to justify this claim?
Check if the Remarks and Explanations column includes detailed notes for each of the success criteria that has a conformance level of “Partially Supports” or “Does Not Support.” If there are any instances where this field is left blank or where the remarks are insufficient in explaining the violation, indicate “No” for this check. Similarly, if several success criteria are indicated as “Supports” with no explanation or examples, indicate “No” for this check.
Next Steps
Prior to procurement or contract renewal, consult with your respective SAIO to determine a Confidence Levelin the vendor’s ability to build and deliver accessible digital products and services based on the following information:
- ACR Review Checklist
- Responses to the Vendor Digital Accessibility Questionnaire
- Any Vendor Digital Accessibility Roadmap for known Accessibility Violations
- Demos of the product’s accessibility and compatibility with assistive technology.
- Any accessibility testing results provided by the vendor and/or conducted by the Commonwealth
If you reviewed an ACR for a digital product already in use, consult with your SAIO to determine next steps as needed.
Downloads
-
Open PDF file, 151.09 KB, Accessibility Conformance Report Review Checklist (English, PDF 151.09 KB)