Enhancing collaborative signing songwriting experience of the d/Deaf individuals
Authors: 
Youjin Choi, ChungHa Lee, Songmin Chung, Eunhye Cho, Suhyeon Yoo, Jin-Hyuk HongAuthors Info & Claims
References
[1]
Oliver Alonzo, Matthew Seita, Abraham Glasser, Matt Huenerfauth, Automatic text simplification tools for deaf and hard of hearing adults: benefits of lexical simplification and providing users with autonomy, in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-13, 2020,.
[2]
Andrey Anikin, Niklas Johansson, Implicit associations between individual properties of color and sound, Attention, Perception, & Psychophysics 81 (2019) (2019) 764–777,.
[3]
Allan W Atlas, IJ Belmont's" Color-Music Expressions"(Pt 1): vaughan Williams on Canvas, Music in Art (2011) (2011) 327–345.
[4]
Ben Bahan, Face-to-face tradition in the American Deaf Community, Signing the body poetic (2006) (2006) 21–50.
[5]
Felicity Baker, Tony Wigram, David Stott, Katrina McFerran, Therapeutic songwriting in music therapy: part I: who are the therapists, who are the clients, and why is songwriting used?, Nord. J. Music. Ther. 17 (2008) (2008) 105–123,. 2.
[6]
Brydie-Leigh Bartleet, Naomi Sunderland, Gavin Carfoot, Enhancing intercultural engagement through service learning and music making with Indigenous communities in Australia, Res. Stud. Music. Educ. 38 (2016) (2016) 173–191,. 2.
[7]
John H Beck, Encyclopedia of Percussion, Routledge, 2013, p. 2013.
[8]
Beth S Benedict, Marilyn Sass-Lehrer, Deaf and hearing partnerships: ethical and communication considerations, Am. Ann. Deaf 152 (2007) (2007) 275–282. 3.
[9]
Mairead Berrill, Towards the Inclusion of Group Music Making in Irish second-Level Classrooms, Dublin City University, 2014.
[10]
David M Birnbaum, Marcelo M Wanderley, A systematic approach to musical vibrotactile feedback, In ICMC (2007).
[11]
John Blacking, The value of music in human experience, Yearbook Int. Folk Music Council 1 (1969) (1969) 33–71,.
[12]
Joanna Borowiec, Anita Hökelmann and Wiesław Osiński. 2019. The level of self-esteem of deaf children: can participating in dance lessons with vibrational headphones improve it? The Arts in Psychotherapy. 64 (2019), 34–38. https://doi.org/10.1016/j.aip.2019.03.004.
[13]
Carmen Branje, Deborah I Fels, Playing vibrotactile music: a comparison between the Vibrochord and a piano keyboard, Int J Hum Comput Stud 72 (2014) (2014) 431–439,. 4.
[14]
Virginia Braun, Victoria Clarke, Using thematic analysis in psychology, Qual. Res. Psychol. 3 (2006) (2006) 77–101,. 2.
[15]
Noah Buchholz, Seeing music? An inquiry into the place of music in Deaf culture, J. Am. Sign Languages Literatures (2018) (2018).
[16]
Gervase R Bushe, Graeme Coetzer, Appreciative inquiry as a team-development intervention: a controlled experiment, J. Appl. Behav. Sci. 31 (1995) (1995) 13–30,. 1.
[17]
Janine Butler, Stacy Bick, Sound, captions, action: voices in video composition projects, Comput. Compos. 62 (2021) (2021),.
[18]
Tom Cabaniss. 2010. A Songwriting Workshop Handbook, 2010.
[19]
Warren N Churchill, Deaf and hard-of-hearing musicians: crafting a narrative strategy, Res. Stud. Music. Educ. 37 (2015) (2015) 21–36,. 1.
[20]
Matthew Clauhs, Songwriting with iconic notation in a music technology classroom, Music Educators Journal 107 (2021) (2021) 22–30,. 3.
[21]
Andrea Creech, Keven Larouche, Mariane Generale, David Fortier, Creativity, music, and quality of later life: a systematic review, Psychol. Music. 51 (2023) (2023) 1080–1100,. 4.
[22]
Jody H Cripps, Ethnomusicology & signed music: a breakthrough, J. Am. Sign Languages Literatures (2018) (2018).
[23]
Anh. Dao Nam, Visualization of Musical Emotions by Colors of Images, Soft Comput. Biomed. Applic. Related Topics (2021) (2021) 169–182.
[24]
Genevieve A Dingle, Leah S Sharman, Zoe Bauer, Emma Beckman, Mary Broughton, Emma Bunzli, Robert Davidson, Grace Draper, Sheranne Fairley, Callyn Farrell, How do music activities affect health and well-being? A scoping review of studies examining psychosocial mechanisms, Front. Psychol. 12 (2021) (2021),.
[25]
Chris Donahue, Ian Simon, Sander Dieleman, Piano genie, in: Proceedings of the 24th International Conference on Intelligent User Interfaces, 2019, pp. 160–164,.
[26]
Robyn Dowlen, John Keady, Christine Milligan, Caroline Swarbrick, Nick Ponsillo, Lucy Geddes, Bob Riley, In the moment with music: an exploration of the embodied and sensory experiences of people living with dementia during improvised music-making, Ageing & Society 42 (2022) (2022) 2642–2664,. 11.
[27]
Morwaread M Farbood, Egon Pasztor, Kevin Jennings, Hyperscore: a graphical sketchpad for novice composers, IEEe Comput. Graph. Appl. 24 (2004) (2004) 50–54,. 1.
[28]
David Felce, Jonathan Perry, Quality of life: the scope, Quality of Life for People with Disabilities: Models, Research and Practice (1997) (1997) 56.
[29]
Irune Fernández-Prieto, Jordi Navarra, Ferran Pons, How big is this sound? Crossmodal association between pitch and size in infants, Infant Behavior and Development 38 (2015) (2015) 77–81,.
[30]
Kate A Gee, Vanessa Hawes, Nicholas Alexander Cox, Blue notes: using songwriting to improve student mental health and wellbeing. A pilot randomised controlled trial, Front. Psychol. 10 (2019) (2019) 423,.
[31]
Jon Gillick, Adam Roberts, Jesse Engel, Douglas Eck, David Bamman, Learning to groove with inverse sequence transformations, in: International Conference on Machine Learning. PMLR, 2019, pp. 2269–2279.
[32]
Jan Gugenheimer, Katrin Plaumann, Florian Schaub, Patrizia Di Campli San Vito, Saskia Duck, Melanie Rabus, Enrico Rukzio, The Impact of Assistive Technology On Communication Quality Between Deaf and Hearing Individuals, Association for Computing Machinery, City, 2017.
[33]
John Habron, Felicity Butterly, Imogen Gordon, Annette Roebuck, Being well, being musical: music composition as a resource and occupation for older people, Br. J. Occupation. Ther. 76 (2013) (2013) 308–316,. 7.
[34]
Gaëtan Hadjeres, François Pachet, Frank Nielsen, Deepbach: a steerable model for bach chorales generation, in: International conference on machine learning. PMLR, 2017, pp. 1362–1371.
[35]
Marion Hersh, Deafblind people, communication, independence, and isolation, J. Deaf Stud. Deaf Educ. 18 (2013) (2013) 446–463,. 4.
[36]
Jessica A Holmes, Expert listening beyond the limits of hearing: music and deafness, J. Am. Musicol. Soc. 70 (2017) (2017) 171–220,. 1.
[37]
Cheng-Zhi Anna Huang, Curtis Hawthorne, Adam Roberts, Monica Dinculescu, James Wexler, Leon Hong and Jacob Howcroft. 2019. The bach doodle: approachable music composition with machine learning at scale. arXiv preprintarXiv:1907.06637. (2019). https://doi.org/10.48550/arXiv.1907.06637.
[38]
Kevin Huang, Thad Starner, Ellen Do, Gil Weinberg, Daniel Kohlsdorf, Claas Ahlrichs, Ruediger Leibrandt, Mobile music touch: mobile tactile stimulation for passive learning, in: Proceedings of the SIGCHI conference on human factors in computing systems, 2010, pp. 791–800,.
[39]
Erkki Huovinen, Theories of creativity in music: students' theory appraisal and argumentation, Front. Psychol. 12 (2021) (2021),.
[40]
Junko Ichino, Aura Pon, Ehud Sharlin, David Eagle, Sheelagh Carpendale, Vuzik: the effect of large gesture interaction on children's creative musical expression, in: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design, 2014, pp. 240–249,.
[41]
IME-Project. 2020. Introspection Music Experience (IME) Labs, Programme and Sessions, 2020.
[42]
Sami Kahn, Allan Feldman and Michele L. Cooke. 2014. Signs of Autonomy: facilitating Independence and Inquiry in Deaf Science Classrooms.
[43]
Meghana Wadnerkar Kamble, Christa Lam-Cassettari, Deborah M. James, Communication Skills and Communicative Autonomy of Prelinguistic Deaf and Hard-of-Hearing Children: application of a Video Feedback Intervention, Front. Psychol. 11 (2020),. (2020-August-18 2020).
[44]
Jeeeun Kim, Swamy Ananthanarayan, Tom Yeh, Seen music: ambient music data visualization for children with hearing impairments, in: Proceedings of the 14th International Conference on Interaction Design and Children, 2015, pp. 426–429,.
[45]
Yeon Soo Kim, Hyeonjeong Im, Sunok Lee, Haena Cho, Sangsu Lee, We Speak Visually”: user-Generated Icons for Better Video-Mediated Mixed-Group Communications Between Deaf and Hearing Participants, in: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, pp. 1–16,.
[46]
Raja S. Kushalnagar, Walter S. Lasecki, Jeffrey P. Bigham, A Readability Evaluation of Real-Time Crowd Captions in the Classroom, Association for Computing Machinery, City, 2012.
[47]
Elizabeth Liddiard, Dawn Rose, A pilot study of an intervention for children using music listening and music making to explore core emotions and support wellbeing, Open. J. Soc. Sci. 9 (2021) (2021) 329–364,. 3.
[48]
Tony Lin. 2022. Multi-modal music interfaces for Deaf and Hard of Hearing to explore creativity.
[49]
Yin Ling, Interactive visual music education software for children, in: Proceedings of the 20th International Academic Mindtrek Conference, 2016, pp. 453–456,.
[50]
Ismael Lizakoski. 2022. Deaf diversity in a team: how a deaf person and a hearing person can pair up. Retrieved from https://www.thoughtworks.com/insights/blog/diversity-equity-and-inclusion/deaf-diversity-in-a-team–how-a-deaf-person-and-a-hearing-person.
[51]
Ryan Louie, Andy Coenen, Cheng Zhi Huang, Michael Terry, Carrie J Cai, Novice-AI music co-creation via AI-steering tools for deep generative models, in: Proceedings of the 2020 CHI conference on human factors in computing systems. 1-13, 2020,.
[52]
Rachel Lowy, Chung Eun Lee, Gregory D Abowd, Jennifer G Kim, Building Causal Agency in Autistic Students through Iterative Reflection in Collaborative Transition Planning, in: Proceedings of the ACM on Human-Computer Interaction. 7, CSCW2, 2023, pp. 1–26,. (2023).
[53]
C.A. MACE, A Projected Art of Light, Nature Publishing Group, UK London, City, 1927.
[54]
T. Machover, D. Ellsey, Inventing Instruments That Unlock New Music, TED, City, 2008.
[55]
Kelly Mack, Maitraye Das, Dhruv Jain, Danielle Bragg, John Tang, Andrew Begel, Erin Beneteau, Josh Urban Davis, Abraham Glasser, Joon Sung Park, Mixed Abilities and Varied Experiences: a group autoethnography of a virtual summer internship, in: Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility, 2021, pp. 1–13,.
[56]
Kelly Mack, Emma McDonnell, Dhruv Jain, Lucy Lu Wang, Jon E. Froehlich, Leah Findlater, What do we mean by “accessibility research”? A literature survey of accessibility papers in CHI and ASSETS from 1994 to 2019, in: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-18, 2021,.
[57]
Anabel Maler, Songs for hands: analyzing interactions of sign language and music, Music theory online 19 (2013) (2013) 1.
[58]
Anabel Maler, Musical expression among deaf and hearing song signers, The Oxford handbook of music and disability studies (2015) (2015) 73–91.
[59]
Nora McDonald, Sarita Schoenebeck, Andrea Forte, Reliability and inter-rater reliability in qualitative research: norms and guidelines for CSCW and HCI practice, in: Proceedings of the ACM on human-computer interaction. 3, CSCW, 2019, pp. 1–23,. (2019).
[60]
Suranga Chandima Nanayakkara, Elizabeth Taylor, Lonce Wyse, S.H. Ong, Towards building an experiential music visualizer, in: 2007 6th International Conference on Information, Communications & Signal Processing, IEEE, 2007, pp. 1–5.
[61]
Keita Ohshiro, Mark Cartwright, How people who are deaf, Deaf, and hard of hearing use technology in creative sound activities, in: Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility. 1-4, 2022,.
[62]
Keita Ohshiro, Mark Cartwright, Audio engineering by people who are deaf and hard of hearing: balancing confidence and limitations, in: 2024 CHI Conference on Human Factors in Computing Sytems, CHI 2024, Association for Computing Machinery, 2024, p. 35.
[63]
Mike Oliver, The social model of disability: thirty years on, Disabil. Soc. 28 (2013) (2013) 1024–1026,. 7.
[64]
Daniel V Oppenheim, Andrew Phillips, Revital Hollander, How Music Technology Will Impact Personal Creative Expression, Digitized, Emerald Publishing Limited, 2019, pp. 41–71,.
[65]
Carol A Padden, Tom L Humphries, Deaf in America: Voices from a Culture, Harvard University Press, 1988, p. 1988.
[66]
Yi-Hao Peng, Ming-Wei Hsi, Paul Taele, Ting-Yu Lin, Po-En Lai, Leon Hsu, Tzu-chuan Chen, Te-Yen Wu, Yu-An Chen, Hsien-Hui Tang, Speechbubbles: enhancing captioning experiences for deaf and hard-of-hearing people in group conversations, in: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 2018, pp. 1–10,.
[67]
Joana Moredo Pereira. Deaf On stage: The cultural Impact of Performing. University College London.
[68]
Benjamin Petry, Jochen Huber, Suranga Nanayakkara, Scaffolding the Music Listening and Music Making Experience for the Deaf, Assistive Augmentation (2018) (2018) 23–48,.
[69]
Benjamin Petry, Thavishi Illandara, Don Samitha Elvitigala, Suranga Nanayakkara, Supporting rhythm activities of deaf children using music-sensory-substitution systems, in: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1-10, 2018,.
[70]
Benjamin Petry, Thavishi Illandara, Suranga Nanayakkara, MuSS-bits: sensor-display blocks for deaf people to explore musical sounds, in: Proceedings of the 28th Australian Conference on Computer-Human Interaction, 2016, pp. 72–80,.
[71]
Anne Marie Piper, James D Hollan, Supporting medical conversations between deaf and hearing individuals with tabletop displays, in: Proceedings of the 2008 ACM conference on Computer supported cooperative work, 2008, pp. 147–156,.
[72]
Sarah Lisette Platte, The Maestro myth: Exploring the Impact of Conducting Gestures On the Musician's Body and the Sounding Result, Massachusetts Institute of Technology, 2016.
[73]
Nicholas Reynolds, Musical composition and creativity in an ICT-enriched learning environment: a case study, in: Proceedings of the 3.1 and 3.3 working groups conference on International federation for information processing: ICT and the teacher of the future-Volume 23, 2003, pp. 101–103.
[74]
Masatsugu Sakajiri, Shigeki Miyoshi, Junji Onishi, Tsukasa Ono, Tohru Ifukube, Tactile pitch feedback system for deafblind or hearing impaired persons singing accuracy of hearing persons under conditions of added noise, in: 2014 IEEE Symposium on Computational Intelligence in Robotic Rehabilitation and Assistive Technologies (CIR2AT), IEEE, 2014, pp. 31–35,.
[75]
Hiromu Sato, Yuya Chiba, Kenji Moriya, Masahiro Nakagawa, Relationship between creative inspiration, self-evaluated satisfaction and brain activity through simplified music composition, in: The Proceedings of The 9th IIAE International Conference on Industrial Application Engineering 2020. The Institute of Industrial Applications Engineers, 2021,.
[76]
Susanne Scheel, Music visualization—The interplay of color and sound. Simplicity—The art of complexity, Ars Electronica (2006) (2006) 273–289.
[77]
Sense.org.uk. 2023. Tips for music-making with children with sensory impairments and complex disabilities. Retrieved from https://www.sense.org.uk/information-and-advice/for-professionals/sense-arts-and-wellbeing-for-professionals/tips-for-music-making-with-children-with-sensory-impairments-and-complex-disabilities/.
[78]
Sungyong Shin, Changmok Oh, Hyungcheol Shin, Tactile tone system: a wearable device to assist accuracy of vocal pitch in cochlear implant users, in: Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility, 2020, pp. 1–3,.
[79]
Andrew Sixsmith, Grant Gibson, Music and the wellbeing of people with dementia, Ageing Soc. 27 (2007) (2007) 127–145,. 1.
[80]
Ene Alicia Søderberg, Rasmus Emil Odgaard, Sarah Bitsch, Oliver Høeg-Jensen, Nikolaj Schildt Christensen, Søren Dahl Poulsen, Steven Gelineck, Music Aid: towards a collaborative experience for deaf and hearing people in creating music, New Interfaces for Musical Expression, 2016.
[81]
Katta Spiel, Christopher Frauenberger, Os Keyes, Geraldine Fitzpatrick, Agency of autistic children in technology research—A critical literature review, ACM Transact. Comput. Human Interaction (TOCHI) 26 (2019) (2019) 1–40,. 6.
[82]
Robyn Taylor, Pierre Boulanger, Daniel Torres, Real-time music visualization using responsive imagery, in: 8th International Conference on Virtual Reality. Citeseer, 2006, pp. 26–30.
[83]
Michael Thaut, Rhythm, music, and the brain: Scientific foundations and Clinical Applications, Taylor & Francis, 2013, p. 2013.
[84]
Pauline Tranchant, Martha M Shiell, Marcello Giordano, Alexis Nadeau, Isabelle Peretz, Robert J Zatorre, Feeling the beat: bouncing synchronization to vibrotactile music in hearing and early deaf people, Front. Neurosci. 11 (2017) (2017) 507,.
[85]
Caroline Waddington-Jones, Andrew King, Pamela Burnard, Exploring wellbeing and creativity through collaborative composition as part of Hull 2017 City of Culture, Front. Psychol. 10 (2019) (2019) 548,.
[86]
Emily Q Wang, Anne Marie Piper, Accessibility in action: co-located collaboration among deaf and hearing professionals, in: Proceedings of the ACM on Human-Computer Interaction. 2, CSCW, 2018, pp. 1–25,. (2018).
[87]
Hanqin Wang, Alexei Sourin, Visual signatures for music mood and timbre, Vis. Comput. (2024) (2024) 1–13.
[88]
Jamie Ward, Brett Huckstep, Elias Tsakanikos, Sound-colour synaesthesia: to what extent does it use cross-modal mechanisms common to us all?, Cortex 42 (2006) 264–280,. 22006.
[89]
Michael L Wehmeyer, Beyond self-determination: causal agency theory, J. Dev. Phys. Disabil. 16 (2004) (2004) 337–359,.
[90]
Michael L Wehmeyer, Karrie A Shogren, Todd D Little, Shane J Lopez, Development of Self-Determination Through the Life-Course, Springer, 2017, p. 2017.
[91]
Donna Weston, Caroline Lenette, Performing freedom: the role of music-making in creating a community in asylum seeker detention centres, Int. J. Commun. Music 9 (2016) (2016) 121–134,. 2.
[92]
Ju-Lee A Wolsey, Kim Misener Dunn, Scott W Gentzke, Hannah A Joharchi, M Diane Clark, CSEDL Team, Deaf/hearing research partnerships, Am. Ann. Deaf 161 (2017) (2017) 571–582. 5.
[93]
James Wright, Daniel V Oppenheim, David Jameson, Don Pazel, R. Fuhrer, CyberBand: a" hands-on" music composition program, In ICMC (1997).
[94]
Yongmeng Wu, Nick Bryan-Kinns, Musicking with an interactive musical system: the effects of task motivation and user interface mode on non-musicians’ creative engagement, Int J Hum Comput Stud 122 (2019) (2019) 61–77,.
[95]
H.-J. Yang, Y.-L. Lay, Y.-C. Liou, W.-Y. Tsao, C.-K. Lin, Development and evaluation of computer-aided music-learning system for the hearing impaired, J. Comput. Assist. Learn. 23 (2007) (2007) 466–476,. 6.
[96]
Lining Yao, Yan Shi, Hengfeng Chi, Xiaoyu Ji, Fangtian Ying, Music-touch shoes: vibrotactile interface for hearing impaired dancers, in: Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction, 2010, pp. 275–276,.
[97]
Suhyeon Yoo, Georgianna Lin, Hyeon Jeong Byeon, Amy S Hwang, Khai Nhut Truong, Understanding tensions in music accessibility through song signing for and with d/Deaf and Non-d/Deaf persons, in: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-18, 2023,.
[98]
Yinsheng Zhou, Khe Chai Sim, Patsy Tan, Ye Wang, MOGAT: mobile games with auditory training for children with cochlear implants, in: Proceedings of the 20th ACM international conference on Multimedia, 2012, pp. 429–438,.
Index Terms
- Enhancing collaborative signing songwriting experience of the d/Deaf individuals
Index terms have been assigned to the content through auto-classification.
Recommendations
Understanding tensions in music accessibility through song signing for and with d/Deaf and Non-d/Deaf persons
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsSong signing is a method practiced by people who are d/Deaf and non-d/Deaf individuals to visually represent music and make music accessible through sign language and body movements. Although there is growing interest in song signing, there is a lack of ...
ChordRipple: Recommending Chords to Help Novice Composers Go Beyond the Ordinary
IUI '16: Proceedings of the 21st International Conference on Intelligent User InterfacesNovice composers often find it difficult to go beyond common chord progressions. To make it easier for composers to experiment with radical chord choices, we built a creativity support tool, ChordRipple, which makes chord recommendations that aim to be ...
An enhanced musical experience for the deaf: design and evaluation of a music display and a haptic chair
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsMusic is a multi-dimensional experience informed by much more than hearing alone, and is thus accessible to people of all hearing abilities. In this paper we describe a prototype system designed to enrich the experience of music for the deaf by ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Elsevier Ltd.
Publisher
Academic Press, Inc.
United States
Publication History
Published: 01 January 2025
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 24 Jan 2025