VR IMMERSION & INCLUSION
When starting this project, I knew immediately I wanted to conduct research related to accessibility.
I was familiar with web accessibility and WCAG 2.0 compliance from my previous job as a web design federal contractor. In my organization, accessibility was considered an afterthought in the development cycle - no more than a checklist to accommodate red tape. As a user-centered designer, this felt like the wrong way to approach accessible design and I wanted to learn more.
While this was noble, it was also safe. Web accessibility has been researched and studied for decades. My professor encouraged me to think outside the web and consider accessibility in emerging technologies. At the time, I had just moved to Seattle and found myself in the midst of what felt like VR-mania. I turned my attention to virtual reality.
Since VR is still emerging as a consumer-facing industry, there are no federal standards. Instead, we are in a Wild West-like land of lawlessness…Well, at least when it comes to VR and accessibility. So whose responsibility is it to push for accessibility in emergent technology? What is being done now to address accessibility in VR?
As I looked deeper into these questions, I noticed a pattern forming in the VR media landscape. Repeatedly, designers and engineers talk about creating the most immersive experience possible, the lynchpin being immersive audio.
So if audio is essential to VR immersion, does this mean VR is at odds with inclusion for people with hearing impairments?
How does VR audio impact gamer immersion and inclusion?
Identify and explore a design challenge in an socially-relevant emergent problem space.
Understand the intersection of virtual reality and accessibility
Professor Tad Hirsch, PhD
Oct. 2016 – Dec. 2016
How does VR audio impact gamer immersion and inclusion?
I became interested in this question after reading through several VR industry articles about immersion and how it was being defined. It was clear there is a pervasive theory that audio is essential to VR immersion. I was fascinated - does this mean immersion is at odds with including people with limited or no hearing?
I would collect data, come to an understanding through analysis and synthesis, then begin to form design recommendations. Those recommendations would change the more data I collected, thus the loop presented.
Locations: CoMotion MakerSpace and Holly Hirzel’s home
Time spent in VR: 13.5 hours over 4 weeks
Hardware used: HTC Vive & Oculus Rift
Since I had never tried VR before, I wanted to understand the experience of VR for myself. Before trying VR for the first time at UW’s CoMotion Makerspace, I saw others trying out the HTC Vive. Seeing them thrash and dance around the invisible box created by the system’s sensors reminded me of watching beta fish in a glass bowl. The experience is voyeuristic. I had to overcome initial self-consciousness of being watched as well as the awkwardness of the gear in order to get to the business of collecting data.
Observations in VR is time & attention consuming. Even smaller, shorter games such as those in the Lab required about 30 minutes of time to experience the game and record my observations. For longer games, I tried recording myself describing the audio I was hearing, but that required a level of multitasking beyond my capabilities. I settled on taking notes immediately after removing the headset. In the future, I would like to record in-game audio and video. I believe this would help increase the accuracy of my reflections.
Game and VR developer
Worked on HoloLens for 7 years
Sound designer with expertise in games, VR, and UX
Worked on audio design for several HBO 360 videos
"Full" immersion is achieved by designing for the mobile, hearing person.
VR audio achieves immersion in 3 main ways - by simulating realistic hearing, by simulating localization, and by relating what you hear to what you see. This means VR relies on movement, hearing, and seeing to "fully" experience the technology.
How Audio in VR Helps Achieve Immersion
2. VR audio carries a large information load.
VR audio conveys a large amount of information, namely game information, plot and dialogue, emotion, and literal sounds.
Types of Audio in VR Games (Based on the IEZA framework)
3. Audio tied to game activity excludes deaf players.
Sounds related to effectively playing the game have a higher potential for excluding players with deafness or hearing loss.
4. Industry attitudes don't align with accessibility.
The VR industry has competing priorities due to hardware challenges, which affects the attitudes and priorities of VR content companies, developers, and designers.
Games should implement subtitles at launch.
Follow traditional subtitle design standards: high contrast, readable font, appropriate size, etc.
Attach dialogue subtitles to the speaker and object-oriented subtitles to the object.
If the subtitle contains vital information, make sure the player reads the info before continuing the subtitle sequence.
Designers of subtitles should keep in mind industry standards will evolve.
Design for Immersive Accessibility
Interaction designers should consider unique interactions such as combining "Power Gestures" and ASL signing for quick access to accessibility features like subtitles.
Power Gestures for accessibility should be unique and not impede gameplay.
As an example, you could design a quick access to subtitles by using the basic gesture for “sign” in ASL and combining that with pressing the bumper button on both controllers.
Maintain flexibility with Power Gesture designs - VR handset hardware is changing rapidly. Vive recently demo’ed a prototype of a new handset in October.
Design for the Future of Haptic
Audio designers should couple information-essential audio with sub-audible haptic feedback.
Ensure important audio and/or visual cues such as UI indicators have sub-audible low frequencies attached to them.
Frequencies for haptic cues should range between 5 and 40 hz, below what the human ear can detect. This ensures that only the player experiences the cue and not anyone else in proximity.
Localized vibrations have not yet been realized in haptic devices like SubPac, so visual cues for locating important audio are still important.
Adjust Attitudes and Advocate
Responsibility for accessible VR lies with all parties, but content distributors like Steam have the most power for impact.
After reviewing the attitudes towards accessibility in VR, I believe that the responsibility to cultivate inclusive VR experiences lies with all parties. Users like you and me can advocate for accessibility features in VR. Developers and designers can work collaboratively to overcome difficult accessibility challenges instead of working in a silo or not considering accessibility at all. Companies that sponsor game content can make accessibility a product requirement or company priority. Lastly, and most significant in their ability to affect all the other parties, the content distributors such as Steam could implement minimum accessibility requirements for hosted games.
Want more details about my findings and recommendations? Check out my process book.
Experience prototyping of subtitles and power gesture using low-fidelity simulation methods (paper/cardboard), followed by an evaluation of the proposed design. Pending results, I will either revise my design recommendations or move to the next step.
Higher-fidelity prototypes in Unity, followed by evaluation.