0

I've used tools like UsableNet to scan for WCAG 2.1 Level AA (and tools like Siteimprove to check for things like reading age levels for text), but now I'm looking to go a step further and scan for issues as part of WCAG 2.1 Level AAA. However, I haven't found any tools/services that do that.

I did come across this: https://tetralogical.com/blog/2021/12/09/testing-wcag-level-aaa/

And it says:

Most tools available for automated or manual testing only cover Level A and Level AA criteria.

My one bit of hope is that is says "most tools" and not "all tools".

Is there a tool/service that will scan for and surface the level AAA issues (either on a specific webpage or an entire website)?

Nicholas Westby
  • 1,109
  • 13
  • 32
  • 1
    Perhaps check https://www.w3.org/WAI/ER/tools/ to see if any cover AAA. – slugolicious Jul 11 '23 at 23:10
  • @slugolicious You have the best username! And if you post this as an answer (which I found via your suggestion) I will mark it as the accepted answer: https://support.equally.ai/en/articles/6826718-how-does-your-solution-work (they mention support for AAA on that page). – Nicholas Westby Jul 14 '23 at 03:58

2 Answers2

2

The W3C has a list of accessibility tools on https://www.w3.org/WAI/ER/tools/. There are currently 166 tools listed, so plenty to pick from. They have some filters but one of them is not by WCAG level so you can't view just AAA tools. But doing a simple text search of "AAA" on the page shows a few.

Update, based on comment:

Unfortunately, the tool you picked looks like an "overlay", which purports to fix your website with "one line of code" (their marketing line, https://www.equally.ai/interface). There is 0% chance that a tool can fix all accessibility issues. Fixing accessibility issues requires a human being. Finding accessibility issues requires a human being.

Scanning tools help find accessibility issues, although all results from a scanning tool must be scrutinized for false positives. Do not assume scanning tool results of infallible, even if the tool comes from a reputable company.

A lot of WCAG is very subjective and it's impossible to build a tool that can make subjective decisions.

For example, WCAG 2.4.6 Headings and Labels

"Headings and labels describe topic or purpose."

Can a tool decide if a heading (<h1>, <h2>, etc) or a label (<label>, aria-label) describes the element sufficiently? Absolutely not. There's no way a tool can analyze the content of a section and then decide if the heading accurately describes that section.

So if a tool cannot detect if a heading or label is descriptive or not, how can a tool fix those issues with "one line of code"?

What about an easier guideline? WCAG 1.1.1 Non-text Content

"All non-text content that is presented to the user has a text alternative that serves the equivalent purpose, except for the situations listed below..."

Most scanning tools use a simple interpretation of that guideline and say that <img> elements must have an alt attribute. The HTML spec regarding the alt text is a bit more complicated.

However, let's say the tool uses the simple interpretation. Can a tool scan for all <img> elements and see if the alt attributes exists? Absolutely. Can it warn you that it's missing? Absolutely, although it should be a warning and not an error since as mentioned in the spec, there are cases where you don't need alt.

Can a tool decide if an image is decorative or informative? That is, can the value of alt be an empty string, "", or have a null value, or should it describe the image? No way.

So if a tool can't figure that out, how can a tool fix your images? What is it going to inject as a description of the image?

Sorry for the long diatribe here. Scanning tools can be quite helpful in finding accessibility issues but scanning tools are very limited in what they can find. Good scanning tools can find maybe 20% of your accessibility issues. What about the other potential 80% of issues that might exist? That requires a human being - one well-versed in testing for accessibility.

So if a scanning tool has very limited success, do you think a tool can exist to actually fix accessibility issues?

Feel free to use the tool you chose to scan for issues so that you can fix your code. But do not rely on a tool to magically fix issues for you, especially when they charge a monthly subscription to do so.

Here are some other articles regarding "overlay" tools, which again are tools that purport to automatically fix issues for you:

slugolicious
  • 15,824
  • 2
  • 29
  • 43
  • Thanks again! Note that some of the tools mention AAA, but have huge caveats (e.g., one is just a manual checklist and one just focuses on color contrast), but equally.ai stood out as very promising. This page mentions that they "scan and identify components that need adjustment to meet WCAG AAA standards" (paraphrased): https://support.equally.ai/en/articles/6826718-how-does-your-solution-work – Nicholas Westby Jul 14 '23 at 21:09
  • New comment based on your edit: I haven't used the equally.ai tool yet, so you may be correct on many of the items you've listed. However, I didn't get the impression it was an overlay tool. It could be the one line of code is to inject a script that tracks real user interactions rather than a bulk scan of pages. I also disagree with several of your premises (e.g., ChatGPT would do an imperfect but impressive job of grading headers with regard to the content they describe). In any event, I'll just have to try the tool and report back if it works or not. Thanks for the food for thought! – Nicholas Westby Jul 20 '23 at 04:48
  • Was your chatgpt comment just a conjecture or have you actually tried to have it analyze your headings? – slugolicious Jul 20 '23 at 16:21
  • Here is a test I just created that demonstrates my ChatGPT comment (which was based on my experience with ChatGPT): https://drive.google.com/drive/folders/1WlJmYumL6n6VfcKkCHUSYC2ZaINKKCyW?usp=sharing – Nicholas Westby Jul 20 '23 at 18:18
  • To clarify, what my ChatGPT test demonstrates is that ChatGPT did a very good job of scoring headers. I extracted the text and headers from a real article. I then replaced one of the headers with something that made sense, but not in that context. It scored that header the lowest. It also scored a header low because it apparently had not content (this was because the only content was an image and I didn't include the text describing that image). So it seems to work really well. – Nicholas Westby Jul 20 '23 at 18:19
  • Perhaps you can work with a scanning tool company (or start your own) and get chatgpt integrated into the results. – slugolicious Jul 21 '23 at 02:15
  • No doubt I could, but it's not something I'm interested in looking into at the moment. I'm sure somebody will get that working eventually (if they haven't already). For now, I'll try the tools I have available to me (Equally.AI is still my best option, though I still need to vet it). – Nicholas Westby Jul 21 '23 at 05:52
0

I can warmly recommend a combination of tools, since all of them focus on different aspects, or are best used in different situations.

I'm mostly using these:

  1. https://wave.webaim.org/: It's free, very accurate and has a good interface, making it very user-friendly for as well developers as content creators and editors.
  2. https://www.totalvalidator.com/index.html: A very good advanced tool, that can perform also batch checking & site crawling.
  3. On macOS use VoiceOver for testing screen reader experience.
  4. For checking the code, https://pa11y.org/ can be - with some minor effort - nicely integrated into a build pipeline for automated checking
  5. Lighthouse in the browser
  6. sometimes also Axe in the browser as Chrome add-on.
cwillinx
  • 537
  • 2
  • 13
  • Are these tools useful for AAA? I have used some of them and I seem to recall that they only go to AA. – Nicholas Westby Jul 19 '23 at 18:59
  • In Total Commander PRo I'm using WCAG 2.1 AAA, you can select that, in Pally you can specify the Standard to use, and also list exceptions in a config file. For the Wave Tool, I'm not sure. Voice Over is about the experience, not a standard. In the end, you will need to check against the rules in an automated way, but also want to test the UX with keyboard navigation and voice over manually. And you will need to go through all the WCAG rules one by one. Also, professional WCAG audits are donethis way. None of the tools is perfect and a single solution for all the aspects. – cwillinx Jul 21 '23 at 10:09
  • 1
    While those are all great tools, the original ask was for a tool that can do AAA scanning. None of those tools except totalvalidator has an option to choose AAA, but the doc says that AAA is only used with htmlcs (code sniffer). I'm not sure what that means but am guessing you have to use the code sniffer tool to get any AAA scans. Also, voiceover is just a screen reader and not a scanning tool. – slugolicious Jul 21 '23 at 18:55