Towards inclusive streaming: Building multimodal music experiences for the deaf and hard of hearing

TB McHugh, A Saha, D Bar-El, M Worsley… - Extended Abstracts of …, 2021 - dl.acm.org
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing …, 2021dl.acm.org
As online streaming becomes a primary method for music consumption, the various
modalities that many people with hearing loss rely on for their enjoyment need to be
supported. While visual or tactile representations can be used to experience music in a live
event or from a recording, DRM anti-piracy encryption restricts access to audio data needed
to create these multimodal experiences for music streaming. We introduce BufferBeats, a
toolkit for building multimodal music streaming experiences. To explore the flexibility of the …
As online streaming becomes a primary method for music consumption, the various modalities that many people with hearing loss rely on for their enjoyment need to be supported. While visual or tactile representations can be used to experience music in a live event or from a recording, DRM anti-piracy encryption restricts access to audio data needed to create these multimodal experiences for music streaming. We introduce BufferBeats, a toolkit for building multimodal music streaming experiences. To explore the flexibility of the toolkit and to exhibit its potential use cases, we introduce and reflect upon building a collection of technical demonstrations that bring previous and new multimodal music experiences to streaming. Grounding our work in critical theories on design, making, and disability, as well as experiences from a small group of community partners, we argue that support for multimodal music streaming experiences will not only be more inclusive to the deaf and hard of hearing, but it will also empower researchers and hobbyist makers to use streaming as a platform to build creative new representations of music.
ACM Digital Library