Using Assistive Tech is like Coding Guitar Hero with a Guitar
No Patrick, a guitar is not an assistive tech device, but some "accessible" websites feel like trying to beat Dark Souls.
I didn’t know what to write about this week… That is, until one of my favorite YouTubers to watch while procrastinating uploaded and built a copy of Guitar Hero with an actual guitar. Did it take ages compared to coding with a mouse and keyboard? Absolutely, and that’s what we’re going to talk about today!
Coding Guitar Hero with an actual Guitar
I have been wondering how to explain the vast difference between “compliance” and “inclusive”. Because there is a raging difference between being able to use something with assistive tech and having an accessible user experience that doesn't feel like a boss battle in Dark Souls. Turns out, Guitar Hero is a great, tangible example.
Let’s look at what CodeBullet did:
He used a Python program that accesses the microphone to tell which note he’s playing, and then assigned all mouse movements and keyboard inputs to a note.
But to no one’s surprise, a guitar doesn’t produce enough notes to tie one of each action needed to one and only one note. His workaround: splitting them into 4 categories (numbers, letters, action keys, mouse movements) and switching the guitar between these 4 modes.
This is How Assistive Tech Works
Not all, of course, but in the digital accessibility context, most devices simply simulate keyboard input.
Screen Reader and Mouse Games
Notably, text-to-audio screen readers are output only, but if we’re talking braille screen readers, they can do both: input and output.
However, screen readers don’t bother with mouse movements. Not because of technical limitations, but because it would make no sense. The mouse is a purely visual indicator of your focus on the screen. If you don’t rely on the screen, you don’t need to bother with the mouse!
But just because they don’t mess with the mouse doesn’t mean they aren’t compatible. Try it out yourself: Download the free screen reader NVDA, open it up, wiggle your mouse, and freeze in horror. Mouse movements are too fast and chaotic to derive meaningful audio output from. The universally preferred way to use them is in combination with keyboard navigation, so you can start reading from the top left corner to the bottom right (in Latin-based language formats).
Braille Keyboards
Let’s briefly touch on Braille keyboards as well since they complement screen readers beautifully. An example of this is the Define Keyboard I reported on before. The really cool thing about it is that it can write Unicode characters as well, meaning: You could code with it. You could code Guitar Hero with it (and probably easier than with a guitar).
If you want to learn more about Braille input, here you go:
Different Input Modes
The FLipMouse developed by my friends at the AsTeRICS Foundation also comes to mind, because it uses the same “mode” concept: You can tie any input to minimal joystick movement, as well as sip-and-puff controls, and have the same action do different things in different modes.
You could probably make Guitar Hero with it, too.
The True Trials of Web Accessibility
You can do a lot of things - like coding Guitar Hero from scratch with a guitar - but they won’t necessarily be easy or pleasant. The problem is that Level AA web accessibility compliance only looks at what’s possible, in theory.
This means web accessibility is most often evaluated by a skilled software tester. This makes a real-world use case not a question of “can” but of endurance. Imagine you go to a fitness class and instead of an average fitness level, the workout is based on what the 5-time Fittest Man on Earth can do. Yes, obviously (and by that I mean surprisingly) it is possible for a human to do this, but are you that human? Am I that human? Can anyone who does not train professionally all day, every day really reasonably do this?
The true trial of web accessibility is making access not suck.