Research to Challenge Assumptions
When reading industry articles about VR, there is a pervasive theory that audio is essential to VR immersion. Does this mean immersion is at odds with inclusion?
Identify and explore an existing design challenge.
Understand the intersection of virtual reality and accessibility
Professor Tad Hirsch, PhD
Oct. 2016 – Dec. 2016
How does VR audio impact gamer immersion and inclusion?
Why this problem?
- In my past work experience as a web developer, accessibility was not integrated into the design process.
- Since VR is still emerging as a consumer-facing industry, there are no federal standards. Instead attitudes will guide the industry.
- There are design opportunities that can push innovation if accessibility is given the same weight as other design considerations.
- The time to push design innovation that includes everyone is now, while VR is young and impressionable.
I would collect data, come to an understanding through analysis and synthesis, then begin to form design recommendations. Those recommendations would change the more data I collected, thus the loop presented.
I believe this model could be used by others in the field, because the VR landscape changes constantly. As the data changes, so should our design standards.
1. Specialist Interviews
Interview with Holly Hirzel, a game and VR developer who worked on HoloLens for 7 years
Interview with Chris Hegstrom, a sound designer with expertise in games, VR, and UX.
2. Participant Observation
- 13.5 hours over 4 weeks using the HTC Vive & Oculus Rift studying 15 different VR titles.
3. Literature Review
For the participant observation, I had to overcome the self-consciousness of being watched using VR as well as the awkwardness of the gear.
Observations in VR are time & attention consuming.
For longer games, I tried recording myself describing the audio I was hearing, but that required a level of multitasking beyond my capabilities. I settled on taking notes immediately after removing the headset.
"Full" immersion is achieved by designing for the mobile, hearing person. VR audio achieves immersion in 3 main ways - by simulating realistic hearing, by simulating localization, and by relating what you hear to what you see. This means VR relies on movement, hearing, and seeing to "fully" experience the technology.
VR audio is more than information, it's experience. VR audio conveys a large amount of information, namely game information, plot and dialogue, emotion, and literal sounds.
Audio tied to game activity excludes deaf players. Sounds related to effectively playing the game have a higher potential for excluding players with deafness or hearing loss.
Industry attitudes don't align with accessibility. The VR industry has competing priorities due to hardware challenges, which affects the attitudes and priorities of VR content companies, developers, and designers.
How Audio in VR Helps Achieve Immersion
Types of Audio in VR Games (Based on the IEZA framework)
- Games should implement subtitles at launch.
- Follow traditional subtitle design standards: high contrast, readable font, appropriate size, etc.
- Attach dialogue subtitles to the speaker and object-oriented subtitles to the object.
- If the subtitle contains vital information, make sure the player reads the info before continuing the subtitle sequence.
- Designers of subtitles should keep in mind industry standards will evolve.
Design for the Future of Haptic
- Audio designers should couple information-essential audio with sub-audible haptic feedback.
- Ensure important audio and/or visual cues such as UI indicators have sub-audible low frequencies attached to them.
- Frequencies for haptic cues should range between 5 and 40 hz, below what the human ear can detect. This ensures that only the player experiences the cue and not anyone else in proximity.
- Localized vibrations have not yet been realized in haptic devices like SubPac, so visual cues for locating important audio are still important.
Design for Immersive Accessibility
- Interaction designers should consider unique interactions such as combining "Power Gestures" and ASL signing for quick access to accessibility features like subtitles.
- Power Gestures for accessibility should be unique and not impede gameplay.
- As an example, you could design a quick access to subtitles by using the basic gesture for “sign” in ASL and combining that with pressing the bumper button on both controllers.
- Maintain flexibility with Power Gesture designs - VR handset hardware is changing rapidly. Vive recently demo’ed a prototype of a new handset in October.
Adjust Attitudes and Advocate
Responsibility for accessible VR lies with all parties, but content distributors like Steam have the most power for impact.
After reviewing the attitudes towards accessibility in VR, I believe that the responsibility to cultivate inclusive VR experiences lies with all parties. Users like you and me can advocate for accessibility features in VR. Developers and designers can work collaboratively to overcome difficult accessibility challenges instead of working in a silo or not considering accessibility at all. Companies that sponsor game content can make accessibility a product requirement or company priority. Lastly, and most significant in their ability to affect all the other parties, the content distributors such as Steam could implement minimum accessibility requirements for hosted games.
Want more details about my findings and recommendations? Check out my process book.
Experience prototyping of subtitles and power gesture using low-fidelity simulation methods (paper/cardboard), followed by an evaluation of the proposed design. Pending results, I will either revise my design recommendations or move to the next step.
Higher-fidelity prototypes in Unity, followed by evaluation.