Supporting Sound Accessibility by Exploring Sound Augmentations in Virtual Reality
Authors: 
Xinyun Cao, Dhruv JainAuthors Info & Claims
Article No.: 122, Pages 1 - 5
Abstract
To increase VR sound accessibility for deaf and hard of hearing users, previous work has substituted sounds with visual or haptic feedback. However, many DHH people (e.g., those with partial hearing) can also benefit from modifying audio (e.g., changing volume based on priorities) instead of fully substituting it with another modality. In this demo paper, we present a toolkit that allows modifying sounds in VR to support DHH people. We designed and implemented 18 VR sound modification tools spanning four categories, including prioritizing sounds, modifying sound parameters, providing spatial assistance, and adding additional sounds. We present five demo scenarios with tools incorporated, covering common VR use cases.
References
[1]
Mina Aghsoleimani, Hamid Jalilvand, Mohammad Ebrahim Mahdavi, Ahmad Reza Nazeri, and Mohammad Kamali. 2018. The Acceptable Noise Level Benefit From Directionality for Listeners With Severe Hearing Loss. Clin Exp Otorhinolaryngol 11, 3 (March 2018), 166–173. https://doi.org/10.21053/ceo.2017.01375
[2]
Ben Bayliss. 2022. Fortnite Accessibility — Menu Deep Dive. Can I Play That? Retrieved February 18, 2024 from https://caniplaythat.com/2022/04/26/fortnite-accessibility-menu-deep-dive/
[3]
Adelbert W. Bronkhorst. 2000. The Cocktail Party Phenomenon: A Review of Research on Speech Intelligibility in Multiple-Talker Conditions. Acta Acustica united with Acustica 86, 1 (January 2000), 117–128.
[4]
Anna Cavender and Richard E. Ladner. 2008. Hearing Impairments. In Web Accessibility: A Foundation for Research, Simon Harper and Yeliz Yesilada (eds.). Springer, London, 25–35. https://doi.org/10.1007/978-1-84800-050-6_3
[5]
Jazmin Collins, Crescentia Jung, Yeonju Jang, Danielle Montour, Andrea Stevenson Won, and Shiri Azenkot. 2023. “The Guide Has Your Back”: Exploring How Sighted Guides Can Enhance Accessibility in Social Virtual Reality for Blind and Low Vision People. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23), October 22, 2023. Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3597638.3608386
[6]
Gilles Courtois, Vincent Grimaldi, Hervé Lissek, Philippe Estoppey, and Eleftheria Georganti. 2019. Perception of Auditory Distance in Normal-Hearing and Moderate-to-Profound Hearing-Impaired Listeners. Trends in Hearing 23, (January 2019), 2331216519887615. https://doi.org/10.1177/2331216519887615
[7]
Coty Craven. 2019. Deaf Game Review - The Elder Scrolls V: Skyrim. Can I Play That? Retrieved April 21, 2024 from https://caniplaythat.com/2019/03/27/deaf-game-review-the-elder-scrolls-v-skyrim/
[8]
David Fourney. 2012. Can Computer Representations of Music Enhance Enjoyment for Individuals Who Are Hard of Hearing? In Computers Helping People with Special Needs, 2012. Springer, Berlin, Heidelberg, 535–542. https://doi.org/10.1007/978-3-642-31522-0_80
[9]
David Gnewikow, Todd Ricketts, Gene W. Bratt, and Laura C. Mutchler. 2009. Real-world benefit from directional microphone hearing aids. JRRD 46, 5 (2009), 603. https://doi.org/10.1682/JRRD.2007.03.0052
[10]
Byung In Han, Ho Won Lee, Tae You Kim, Jun Seong Lim, and Kyoung Sik Shin. 2009. Tinnitus: Characteristics, Causes, Mechanisms, and Treatments. Journal of Clinical Neurology 5, 1 (March 2009), 11–19. https://doi.org/10.3988/jcn.2009.5.1.11
[11]
Trevor Hogan, Uta Hinrichs, and Eva Hornecker. 2017. The Visual and Beyond: Characterizing Experiences with Auditory, Haptic and Visual Data Representations. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS ’17), June 10, 2017. Association for Computing Machinery, New York, NY, USA, 797–809. https://doi.org/10.1145/3064663.3064702
[12]
Lisa L. Hunter, Robert H. Margolis, Joni R. Rykken, Chap T. Le, Kathleen A. Daly, and G. Scott Giebink. 1996. High Frequency Hearing Loss Associated with Otitis Media. Ear and Hearing 17, 1 (February 1996), 1.
[13]
Dhruv Jain, Sasa Junuzovic, Eyal Ofek, Mike Sinclair, John Porter, Chris Yoon, Swetha Machanavajhala, and Meredith Ringel Morris. 2021. A Taxonomy of Sounds in Virtual Reality. In Proceedings of the 2021 ACM Designing Interactive Systems Conference (DIS ’21), June 28, 2021. Association for Computing Machinery, New York, NY, USA, 160–170. https://doi.org/10.1145/3461778.3462106
[14]
Dhruv Jain, Sasa Junuzovic, Eyal Ofek, Mike Sinclair, John R. Porter, Chris Yoon, Swetha Machanavajhala, and Meredith Ringel Morris. 2021. Towards Sound Accessibility in Virtual Reality. In Proceedings of the 2021 International Conference on Multimodal Interaction (ICMI ’21), October 18, 2021. Association for Computing Machinery, New York, NY, USA, 80–91. https://doi.org/10.1145/3462244.3479946
[15]
JimSeaman42. 2023. Creating and Adding Custom Sounds. Retrieved March 21, 2024 from https://learn.microsoft.com/en-us/minecraft/creator/documents/addcustomsounds?view=minecraft-bedrock-stable
[16]
Sushant Kafle, Peter Yeung, and Matt Huenerfauth. 2019. Evaluating the Benefit of Highlighting Key Words in Captions for People who are Deaf or Hard of Hearing. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’19), October 24, 2019. Association for Computing Machinery, New York, NY, USA, 43–55. https://doi.org/10.1145/3308561.3353781
[17]
Regina Kaplan-Rakowski, Deborah Cockerham, and Richard E. Ferdig. The impact of sound and immersive experience on learners when using virtual reality and tablet: A mixed-method study. British Journal of Educational Technology n/a, n/a . https://doi.org/10.1111/bjet.13417
[18]
Marlies Knipper, Pim van Dijk, Holger Schulze, Birgit Mazurek, Patrick Krauss, Verena Scheper, Athanasia Warnecke, Winfried Schlee, Kerstin Schwabe, Wibke Singer, Christoph Braun, Paul H. Delano, Andreas J. Fallgatter, Ann-Christine Ehlis, Grant D. Searchfield, Matthias H. J. Munk, David M. Baguley, and Lukas Rüttiger. 2020. The Neural Bases of Tinnitus: Lessons from Deafness and Cochlear Implants. J. Neurosci. 40, 38 (September 2020), 7190–7202. https://doi.org/10.1523/JNEUROSCI.1314-19.2020
[19]
Andrew J. Kolarik, Brian C. J. Moore, Pavel Zahorik, Silvia Cirstea, and Shahina Pardhan. 2016. Auditory distance perception in humans: a review of cues, development, neuronal bases, and effects of sensory loss. Atten Percept Psychophys 78, 2 (February 2016), 373–395. https://doi.org/10.3758/s13414-015-1015-1
[20]
Daniel P. Kumpik and Andrew J. King. 2019. A review of the effects of unilateral hearing loss on spatial hearing. Hearing Research 372, (February 2019), 17–28. https://doi.org/10.1016/j.heares.2018.08.003
[21]
Ziming Li, Shannon Connell, Wendy Dannels, and Roshan Peiris. 2022. SoundVizVR: Sound Indicators for Accessible Sounds in Virtual Reality for Deaf or Hard-of-Hearing Users. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’22), October 22, 2022. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3517428.3544817
[22]
Ziming Li, Kristen Shinohara, and Roshan L Peiris. 2023. Exploring the Use of the SoundVizVR Plugin with Game Developers in the Development of Sound-Accessible Virtual Reality Games. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA ’23), April 19, 2023. Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3544549.3585750
[23]
Vardit Lichtenstein and David R. Stapells. 1996. Frequency-specific identification of hearing loss using transient-evoked otoacoustic emissions to clicks and tones. Hearing Research 98, 1 (September 1996), 125–136. https://doi.org/10.1016/0378-5955(96)00084-6
[24]
Christian Lorenzi, Gaëtan Gilbert, Héloïse Carn, Stéphane Garnier, and Brian C. J. Moore. 2006. Speech perception problems of the hearing impaired reflect inability to use temporal fine structure. Proceedings of the National Academy of Sciences 103, 49 (December 2006), 18866–18869. https://doi.org/10.1073/pnas.0607364103
[25]
Mohammadreza Mirzaei, Peter Kán, and Hannes Kaufmann. 2020. EarVR: Using Ear Haptics in Virtual Reality for Deaf and Hard-of-Hearing People. IEEE Transactions on Visualization and Computer Graphics 26, 5 (May 2020), 2084–2093. https://doi.org/10.1109/TVCG.2020.2973441
[26]
Brian C. J. Moore. 2016. Effects of Age and Hearing Loss on the Processing of Auditory Temporal Fine Structure. In Physiology, Psychoacoustics and Cognition in Normal and Impaired Hearing, 2016. Springer International Publishing, Cham, 1–8. https://doi.org/10.1007/978-3-319-25474-6_1
[27]
Suranga Chandima Nanayakkara, Lonce Wyse, S. H. Ong, and Elizabeth A. Taylor. 2013. Enhancing Musical Experience for the Hearing-Impaired Using Visual and Haptic Displays. Human–Computer Interaction 28, 2 (March 2013), 115–160. https://doi.org/10.1080/07370024.2012.697006
[28]
Benjamin Petry, Thavishi Illandara, Don Samitha Elvitigala, and Suranga Nanayakkara. 2018. Supporting Rhythm Activities of Deaf Children using Music-Sensory-Substitution Systems. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI ’18), April 21, 2018. Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3173574.3174060
[29]
Michael Richardson, Jan Thar, James Alvarez, Jan Borchers, Jamie Ward, and Giles Hamilton-Fletcher. 2019. How Much Spatial Information Is Lost in the Sensory Substitution Process? Comparing Visual, Tactile, and Auditory Approaches. Perception 48, 11 (November 2019), 1079–1103. https://doi.org/10.1177/0301006619873194
[30]
Todd Andrew Ricketts. 2001. Directional Hearing Aids. Trends in Amplification 5, 4 (December 2001), 139–176. https://doi.org/10.1177/108471380100500401
[31]
Richard J. Stubbs and Quentin Summerfield. 2005. Separation of simultaneous voices. The Journal of the Acoustical Society of America 81, S1 (August 2005), S60. https://doi.org/10.1121/1.2024313
[32]
Ler digital studio. Beat Saber - VR rhythm game. Retrieved March 7, 2024 from https://beatsaber.com/
[33]
Stephen A Wilcox, Kerryn Saunders, Amelia H Osborn, Angela Arnold, Julia Wunderlich, Therese Kelly, Veronica Collins, Leah J Wilcox, RJ McKinlay Gardner, Maria Kamarinos, Barbara Cone-Wesson, Robert Williamson, and Hans-Henrik M. Dahl. 2000. High frequency hearing loss correlated with mutations in the GJB2 gene. Hum Genet 106, 4 (April 2000), 399–405. https://doi.org/10.1007/s004390000273
[34]
Matthew B. Winn and Katherine H. Teece. 2020. Slower Speaking Rate Reduces Listening Effort Among Listeners With Cochlear Implants. Ear Hear 42, 3 (September 2020), 584–595. https://doi.org/10.1097/AUD.0000000000000958
[35]
Alina Zajadacz. 2015. Evolution of models of disability as a basis for further policy changes in accessible tourism. Journal of Tourism Futures 1, 3 (January 2015), 189–202. https://doi.org/10.1108/JTF-04-2015-0015
[36]
Yuhang Zhao, Ed Cutrell, Christian Holz, Meredith Ringel Morris, Eyal Ofek, and Andy Wilson. 2019. SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision. In CHI 2019, May 2019. ACM. Retrieved from https://www.microsoft.com/en-us/research/publication/seeingvr-a-set-of-tools-to-make-virtual-reality-more-accessible-to-people-with-low-vision-2/
[37]
2018. The Persistence - Accessibility Features | PS VR. Retrieved March 6, 2024 from https://www.youtube.com/watch?v=vdenKQ3S5Iw
[38]
2021. How to use Live Listen — Apple Support. Retrieved March 21, 2024 from https://www.youtube.com/watch?v=ClBCyTs5X6Y
[39]
The Last of Us Part II - Accessibility. PlayStation. Retrieved January 28, 2024 from https://www.playstation.com/en-us/games/the-last-of-us-part-ii/accessibility/
[40]
Use Live Listen with Made for iPhone hearing aids. Apple Support. Retrieved March 21, 2024 from https://support.apple.com/en-us/111777
[41]
TS4 Sound Tool. Sims 4 Modding Wiki. Retrieved March 21, 2024 from https://sims-4-modding.fandom.com/wiki/TS4_Sound_Tool
[42]
Hearing Loss and Tinnitus - ClinicalKey. Retrieved March 21, 2024 from https://www.clinicalkey.com/#!/content/playContent/1-s2.0-S0025712521000717?returnurl=null&referrer=null
Index Terms
- Supporting Sound Accessibility by Exploring Sound Augmentations in Virtual Reality
Index terms have been assigned to the content through auto-classification.
Recommendations
Towards Sound Accessibility in Virtual Reality
ICMI '21: Proceedings of the 2021 International Conference on Multimodal InteractionVirtual reality (VR) leverages sight, hearing, and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, however, information conveyed through sound may not be accessible. While prior work has explored making every day ...
SoundModVR: Sound Modifications in Virtual Reality for Sound Accessibility
UIST Adjunct '24: Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and TechnologyPrevious VR sound accessibility work have substituted sounds with visual or haptic output to increase VR accessibility for deaf and hard of hearing (DHH) people. However, deafness occurs on a spectrum, and many DHH people (e.g., those with partial ...
A Study on Realistic Audio Sound Generation according to User's Movement in Virtual Reality System
ICIIT '19: Proceedings of the 2019 4th International Conference on Intelligent Information TechnologyGenerally, a virtual reality (VR) system cannot provide the realistic sound formed by the multi-channel audio signals to a user with the stereo headphone environment. In addition, the VR system has a gap between the visual scene and the sound because it ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2024
1475 pages
ISBN:9798400706776
DOI:10.1145/3663548
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 October 2024
Check for updates
Qualifiers
- Poster
- Research
- Refereed limited
Conference
ASSETS '24
Sponsor:
ASSETS '24: The 26th International ACM SIGACCESS Conference on Computers and Accessibility
October 27 - 30, 2024
NL, St. John's, Canada
Acceptance Rates
Overall Acceptance Rate 436 of 1,556 submissions, 28%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 45Total Downloads
- Downloads (Last 12 months)45
- Downloads (Last 6 weeks)14
Reflects downloads up to 04 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format