Research to Challenge Assumptions
When reading industry articles about VR, there is a pervasive theory that audio is essential to VR immersion. Does this mean immersion is at odds with inclusion?
Identify and explore an existing design challenge.
Understand the intersection of virtual reality and accessibility
Professor Tad Hirsch, PhD
Oct. 2016 – Dec. 2016
How does VR audio impact gamer immersion and inclusion?
I became interested in this question after reading through several VR industry articles about immersion and how it was being defined. It was clear there is a pervasive theory that audio is essential to VR immersion. I was fascinated - does this mean immersion is at odds with including people with limited or no hearing?
I would collect data, come to an understanding through analysis and synthesis, then begin to form design recommendations. Those recommendations would change the more data I collected, thus the loop presented.
Holly Hirzel, a game and VR developer who worked on HoloLens for 7 years
Chris Hegstrom, a sound designer with expertise in games, VR, and UX.
I spent more than 13 hours studying 15 different VR titles on the HTC Vive & Oculus Rift.
"Full" immersion is achieved by designing for the mobile, hearing person.
VR audio achieves immersion in 3 main ways - by simulating realistic hearing, by simulating localization, and by relating what you hear to what you see. This means VR relies on movement, hearing, and seeing to "fully" experience the technology.
How Audio in VR Helps Achieve Immersion
VR audio carries a large information load.
VR audio conveys a large amount of information, namely game information, plot and dialogue, emotion, and literal sounds.
Types of Audio in VR Games (Based on the IEZA framework)
Audio tied to game activity excludes deaf players.
Sounds related to effectively playing the game have a higher potential for excluding players with deafness or hearing loss.
Industry attitudes don't align with accessibility.
The VR industry has competing priorities due to hardware challenges, which affects the attitudes and priorities of VR content companies, developers, and designers.
- Games should implement subtitles at launch.
- Follow traditional subtitle design standards: high contrast, readable font, appropriate size, etc.
- Attach dialogue subtitles to the speaker and object-oriented subtitles to the object.
- If the subtitle contains vital information, make sure the player reads the info before continuing the subtitle sequence.
- Designers of subtitles should keep in mind industry standards will evolve.
Design for Immersive Accessibility
- Interaction designers should consider unique interactions such as combining "Power Gestures" and ASL signing for quick access to accessibility features like subtitles.
- Power Gestures for accessibility should be unique and not impede gameplay.
- As an example, you could design a quick access to subtitles by using the basic gesture for “sign” in ASL and combining that with pressing the bumper button on both controllers.
- Maintain flexibility with Power Gesture designs - VR handset hardware is changing rapidly. Vive recently demo’ed a prototype of a new handset in October.
Design for the Future of Haptic
- Audio designers should couple information-essential audio with sub-audible haptic feedback.
- Ensure important audio and/or visual cues such as UI indicators have sub-audible low frequencies attached to them.
- Frequencies for haptic cues should range between 5 and 40 hz, below what the human ear can detect. This ensures that only the player experiences the cue and not anyone else in proximity.
- Localized vibrations have not yet been realized in haptic devices like SubPac, so visual cues for locating important audio are still important.
Adjust Attitudes and Advocate
Responsibility for accessible VR lies with all parties, but content distributors like Steam have the most power for impact.
After reviewing the attitudes towards accessibility in VR, I believe that the responsibility to cultivate inclusive VR experiences lies with all parties. Users like you and me can advocate for accessibility features in VR. Developers and designers can work collaboratively to overcome difficult accessibility challenges instead of working in a silo or not considering accessibility at all. Companies that sponsor game content can make accessibility a product requirement or company priority. Lastly, and most significant in their ability to affect all the other parties, the content distributors such as Steam could implement minimum accessibility requirements for hosted games.
Want more details about my findings and recommendations? Check out my process book.
Experience prototyping of subtitles and power gesture using low-fidelity simulation methods (paper/cardboard), followed by an evaluation of the proposed design. Pending results, I will either revise my design recommendations or move to the next step.
Higher-fidelity prototypes in Unity, followed by evaluation.