The Pokédex of Accessibility

You wanna be the very best in software testing, like no one ever was?

Testing for access barriers is part of a comprehensive quality assurance process. But where does one start? While the WCAG are as extensive as the Pokédex, what exactly is the assistive tech equivalent of Bulbasaur? Where do you start to catch ‘em all?

Catch ‘Em All: Hunting Accessibility Bugs Like a Champion

Accessibility is a key part of quality assurance, but navigating the world of accessibility testing tools can feel as overwhelming as choosing your first Pokémon. So, where do you start?

Professor Oak: “You have the ability to become a great accessibility specialist someday!”

The WCAG are your Pokédex

How many Pokemon are there? Depends on the edition, but let’s go with the 151 from the 1st generation. How many success criteria are there? 93 in the 2.2 version. Just like Pokemon have their evolutions, success criteria have levels: A, Double A, and Triple A.

You can think of them as the same thing: Each Pokémon starts out as level A, this is their base form. You can’t battle with an egg (unless you’re Misty), and your website needs to pass at least level A to have basic functionality present. The better your website usability gets, the more high level success criteria you can pass. Like a training montage!

Professor Oak: “Erm, what was this success criteria now?”

Not all success criteria are applicable to all websites or apps, but certain criteria are always required, such as:

• 1.1.1 Non-text Content

• 1.3.3 Sensory Characteristics

• 1.4.1 Use of Color

• 2.1.1 Keyboard Operable

• 2.1.2 No Keyboard Trap

• 2.3.1 Three Flashes or Below Threshold

• 2.4.1 Bypass Blocks

• 2.4.2 Page Titled

• 2.4.3 Focus Order

• 2.4.4 Link Purpose (In Context)

• 2.5.3 Label in Name

• 3.1.1 Language of Page

• 3.2.1 On Focus

• 3.2.6 Consistent Help (from WCAG 2.2 aka the 4th gen)

• 4.1.2 Name, Role, Value

These are the very very basic requirements for your website, the very first Pokémon you catch on your adventure. Others will also be applicable, depending on the type of content you add, but these guys will be your MVP (minimal viable Pokémon) team to challenge the first gym leader to get your Level A compliance score.

While some success criteria build on each other, others are stand alone criteria. Like Ditto, Lapras, and your favorite legendary, they are not part of an evolutionary line, but they could be Level A, AA or AAA in their default form.

So while we can agree that Charmander is cute, ultimately we all want Charizard. In the same way, we shouldn’t be content with simply passing AA. If you want to become an Accessibility Champion, keep evolving and keep striving for AAA.

Pick your starter!

Now that we’ve collected our WCAG Pokédex and encouraging words from Professor Oak, it’s time to pick a starter and get started with accessibility testing!

Automated Testing

Automated testing has come a long way, but it’s not yet good enough for the champion’s league. So to become an Accessibility Champion, you need to learn how to pick the right starter testing tools, then with a few manual testing tips and tricks, plus a training montage, you will be catching accessibility bugs and barriers, like no one ever has! Let’s go meet some starters:

Pikachu = Google Lighthouse

I know Pikachu is as iconic as it can get, all the while Google Lighthouse has a bad reputation due to lack of accuracy. But hear me out: Lighthouse isn’t a bad tool. It can teach you a lot but has its limits, and you won’t be able to challenge the Accessibility Champions League with it alone.

“pika pika”

It suffers from the same problems all automated testing tools have: They can only check what they can check, meaning that they may, for example, state that alt-text is used correctly, even if the string value looks something like “asdfgThisIsGibbrishTM_randomStringOfNumbers”. Yes, alt text is technically present, but with content like that, the screen reader might as well be freestyling.

But the errors it throws up are correct. You will have a straightforward list of what to fix, which makes for an easy starting point.

Web accessibility is a long road to master, including knowledge of disabilities, assistive tech hardware & software, legal frameworks like the ADA or EAA, stretching all the way to human rights, but every now and then you will still get stuck on a bridge with niche ARIA labels and the question of what they are compatible with blocking your way.

Your player character is trying to progress, but “A Pokémon is sprawled out in deep and comfortable slumber. “ It’s Snorlax.

Bulbasaur = WAVE

WAVE is a web accessibility evaluation tool by WebAIM:

Rather than providing a complex technical report, WAVE shows your original web content with embedded icons and indicators that reveal the accessibility information within your page. WAVE cannot tell you if your web content is accessible – only a human can determine true accessibility – but WAVE can help you evaluate the accessibility of your web content.

It is a solid choice when you are starting out, but (like the plant starter in each generation) it also has its weaknesses. For example: Many landing pages use hero images at the top. WAVE doesn’t recognize if the color contrast between the background image and the text in an overlay on top of it is sufficient, all it sees is a textbox with a transparent background, so it will assume that

What I like most about it is that it lets you reference the relevant part of the source code for each issue. To become an Accessibility Champion, you have to learn how to handle accessibility in code, too. With WAVE, it’s very much a bite-sized introduction to it.

Both tools will point you toward learning resources for the particular errors they recognize. This is what makes them the perfect starter Pokémon: They are easy to use and show you where to go for more training.

Manual Testing

There will come a point on your path to becoming an A11y Master when you will need to add manual testing to your team. Of course, getting proficient in testing with multiple assistive tech types would be like tapping into the power of Arceus, but you don't need to face legendary battles straight away to evolve your testing skills:

splish splash, these Magikarps aren’t very effective.

Magikarp’s splash = 1 tab

A lot of assistive tech simply simulates inputs through keystrokes. This is not limited to form inputs, keystrokes or combinations are also used to navigate a webpage. Switch buttons work the same. No, not the on/off toggle kind, I’m talking about adaptive switches. Much like disabilities, they come in a variety of shapes and sizes. They are especially important for people with mobility, flexibility, and body structure disabilities.

Functional keyboard navigation is also important for screen readers, as they mostly utilize the tab key to jump from one focusable element to the next. That makes the tab key a readily accessible tool to cover a lot of ground with. You can check which elements receive focus, if the focus follows the website structure, if one can interact with elements as expected, if the focus is visible…

Like Magikarp's splash, the tab key doesn't do much on its own, but once you understand how keyboard navigation works, it's a strong addition to your testing lineup. And just like that your keyboard evolves into your very own Gyarados.

Eevee = Screen Readers

The normal type Eevee is not only cute but very versatile, just like the wide range of screen reader software available!

Screen readers also include the often forgotten braille display. Yes, it counts as a screen reader too! The only difference being that it doesn’t output in speech but translates the written text to braille code instead.

Eeveelutions & different screen readers in different browsers

Just like an Eevee can evolve into different types, different screen reader and browser combinations will give you different outputs. But they all operate on the same foundation, so make sure landmarks and headings are in order, and your images have alt attributes filled with suitable text alternatives, so your Eevees can read every move of their opponent in battle!

Screen readers in different browsers are like the Eeveelutions: similar but different.

To build your ultimate screen reader team, check the web aim screen reader user survey to find the most popular screen reader and browser pairings.

Stay vigilant: The screen reader survey is not the ultimate representation of which combinations are the most used. The survey is sent out annually by WebAIM but not every person who relies on it is actively keeping up with web accessibility. Especially persons with lower digital literacy who just started out using a screen reader out of necessity are unlikely to bother filling out a survey about their screen reader use. So don’t forget to test with in-build tools such as Talk Back on Android smartphones as well.

Start your adventure

With a strong team like that you are well prepared to venture out into the world wide web and catch all accessibility bugs you encounter. Accessibility issues around the world are waiting for you!

closing words from Professor Oak: “Accessibility issues around the world are waiting for you.”

Your next steps on the Victory Road

• Accessibility First about Accessibility Testing

• Preventing Color Contrast Errors with Effective Simulation Testing