[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3678299.3678304acmotherconferencesArticle/Chapter ViewAbstractPublication PagesamConference Proceedingsconference-collections
research-article
Open access

Making Movement Sonification Usable in Clinical Gait Rehabilitation: A User-Centered Study

Published: 18 September 2024 Publication History

Abstract

Sound-assisted movement rehabilitation is one of the most interesting and socially relevant applications of sonic interaction research, and existing research shows the potential of movement sonification to enhance patient motivation and movement re-learning. However, this technology is not routinely adopted in practice due to fragmented evidence of clinical effectiveness, feasibility, and usability. A key but little-explored challenge is that of designing and developing sonification systems that can readily be set up, configured, and applied by physiotherapists in a manner compatible with clinical practices and tailorable to individual patient needs and capacities, as well as training goals. In this work, we carried out an iterative user-centered design process to develop a sonification system for clinical gait rehabilitation of hemiparetic patients. This yielded a fully functional system that integrated wireless wearable inertial sensors with a laptop-based software interface for physiotherapists to configure and adjust the feedback on the fly. The feedback was ecologically based (naturalistic wading sounds) and highly individualizable through patient-specific adjustments. The system was evaluated in a real-life clinical feasibility study involving four physiotherapists and seven hemiparetic patients (4M,3F, mean age 56.14 ± 15.45). Overall, the physiotherapists found the system easy to set up and seamless to use during training, although they expressed a need for more system portability, a set of additional feedback configuration functions, and extra training / practice with using the more advanced feedback adjustment controls. While future work should address these findings and systematically explore clinical effectiveness in terms of motor learning outcomes, we believe that this work can serve to guide the design of clinically usable movement sonification systems targeting a breadth of rehabilitative applications.

References

[1]
2021. An Update on Android’s Audio Latency. https://android-developers.googleblog.com/2021/03/an-update-on-androids-audio-latency.html
[2]
Birol Balaban and Fatih Tok. 2014. Gait disturbances in patients with stroke. Physical Medicine and Rehabilitation 6, 7 (2014), 635–642.
[3]
Alessio Balsini, Tommaso Cucinotta, Luca Abeni, Joel Fernandes, Phil Burk, Patrick Bellasi, and Morten Rasmussen. 2019. Energy-efficient low-latency audio on android. Journal of Systems and Software 152 (2019), 182–195.
[4]
Frédéric Bevilacqua, Eric O Boyer, Jules Françoise, Olivier Houix, Patrick Susini, Agnès Roby-Brami, and Sylvain Hanneton. 2016. Sensori-motor learning with movement sonification: perspectives from recent interdisciplinary studies. Frontiers in neuroscience 10 (2016), 195146.
[5]
Frédéric Bevilacqua, Iseline Peyre, Maël Segalen, Pascale Pradat-Diehl, Veronique Marchand-Pauvert, and Agnès Roby-Brami. 2018. Exploring different movement sonification strategies for rehabilitation in clinical settings. In Proceedings of the 5th International Conference on Movement and Computing. 1–6.
[6]
Nienke Bleijenberg, Janneke M de Man-van Ginkel, Jaap CA Trappenburg, Roelof GA Ettema, Carolien G Sino, Noor Heim, Thóra B Hafsteindóttir, David A Richards, and Marieke J Schuurmans. 2018. Increasing value and reducing waste by optimizing the development of complex interventions: Enriching the development phase of the Medical Research Council (MRC) Framework. International journal of nursing studies 79 (2018), 86–93.
[7]
Deborah J Bowen, Matthew Kreuter, Bonnie Spring, Ludmila Cofta-Woerpel, Laura Linnan, Diane Weiner, Suzanne Bakken, Cecilia Patrick Kaplan, Linda Squiers, Cecilia Fabrizio, 2009. How we design feasibility studies. American journal of preventive medicine 36, 5 (2009), 452–457.
[8]
Thomas Bowman, Elisa Gervasoni, Chiara Arienti, Stefano Giuseppe Lazzarini, Stefano Negrini, Simona Crea, Davide Cattaneo, and Maria Chiara Carrozza. 2021. Wearable Devices for Biofeedback Rehabilitation: A Systematic Review and Meta-Analysis to Design Application Rules and Estimate the Effectiveness on Balance and Gait Outcomes in Neurological Diseases. Sensors 21, 10 (2021), 3444.
[9]
Louise Brennan, Enrique Dorronzoro Zubiete, and Brian Caulfield. 2019. Feedback Design in Targeted Exercise Digital Biofeedback Systems for Home Rehabilitation: A Scoping Review. Sensors 20, 1 (2019), 181.
[10]
Roberto Bresin, Anna de Witt, Stefano Papetti, Marco Civolani, Federico Fontana, 2010. Expressive sonification of footstep sounds. Proceedings of ISon 2010 (2010), 3rd.
[11]
Lorenzo Chiari, Marco Dozza, Angelo Cappello, Fay B Horak, Velio Macellari, and Daniele Giansanti. 2005. Audio-biofeedback for balance improvement: an accelerometry-based system. IEEE transactions on biomedical engineering 52, 12 (2005), 2108–2111.
[12]
Peter Craig, Paul Dieppe, Sally Macintyre, Susan Michie, Irwin Nazareth, and Mark Petticrew. 2008. Developing and evaluating complex interventions: the new Medical Research Council guidance. Bmj 337 (2008).
[13]
Anabel Immoos Dailly, Roland Sigrist, Yeongmi Kim, Peter Wolf, Hendrik Erckens, Joachim Cerny, Andreas Luft, Roger Gassert, and James Sulzer. 2012. Can simple error sonification in combination with music help improve accuracy in upper limb movements?. In 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob). IEEE, 1423–1427.
[14]
Fred D Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly (1989), 319–340.
[15]
Marco Dozza, Fay Horak, and Lorenzo Chiari. 2007. Auditory Biofeedback Substitutes for Loss of Sensory Information in Maintaining Stance. Experimental Brain Research. Experimentelle Hirnforschung. Expérimentation Cérébrale 178 (04 2007), 37–48. https://doi.org/10.1007/s00221-006-0709-y
[16]
David A Eddins and David M Green. 1995. Temporal Integration and Temporal Resolution. Hearing 1028 (1995), 1022–1029.
[17]
Alfred O Effenberg. 2005. Movement Sonification: Effects on Perception and Action. IEEE Multimedia 12, 2 (2005), 53–59.
[18]
John F Dyer, Paul Stapleton, and Matthew WM Rodger. 2015. Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design. The Open Psychology Journal 8, 1 (2015).
[19]
Xin Feng. 2007. Upper-extremity performance assessment using an interactive, personalized computer-assisted neurorehabilitation motivating framework. (2007).
[20]
Anette Forsberg, Ylva Nilsagård, and Katrin Boström. 2015. Perceptions of using videogames in rehabilitation: a dual perspective of people with multiple sclerosis and physiotherapists. Disability and rehabilitation 37, 4 (2015), 338–344.
[21]
Karmen Franinovic and Stefania Serafin. 2013. Sonic interaction design. Mit Press.
[22]
Emma Frid, Roberto Bresin, Paolo Alborno, and Ludvig Elblaus. 2016. Interactive sonification of spontaneous movement of children—cross-modal mapping and the perception of body movement qualities through sound. Frontiers in neuroscience 10 (2016), 212612.
[23]
Dominik Fuchs, Martin Knauer, Marion Egger, and Petra Friedrich. 2020. Audio feedback for device-supported balance training: parameter mapping and influencing factors. In Acoustics, Vol. 2. MDPI, 650–665.
[24]
Oonagh Giggins, Ulrik McCarthy Persson, and Brian Caulfield. 2013. Biofeedback in Rehabilitation. Journal of Neuroengineering and Rehabilitation 10 (06 2013), 60. https://doi.org/10.1186/1743-0003-10-60
[25]
Alba Gomez-Andres, Jennifer Grau-Sánchez, Esther Duarte, Antoni Rodriguez-Fornells, and Ana Tajadura-Jiménez. 2020. Enriching footsteps sounds in gait rehabilitation in chronic stroke patients: a pilot study. Annals of the New York Academy of Sciences 1467, 1 (2020), 48–59.
[26]
Katharina Gordt, Thomas Gerhardy, Bijan Najafi, and Michael Schwenk. 2017. Effects of wearable sensor-based balance and gait training on balance, gait, and functional performance in healthy and patient populations: a systematic review and meta-analysis of randomized controlled trials. Gerontology 64, 1 (2017), 74–89.
[27]
Jessica A Grahn. 2012. See what I hear? Beat perception in auditory and visual rhythms. Experimental brain research 220, 1 (2012), 51–61.
[28]
Mark A Guadagnoli and Timothy D Lee. 2004. Challenge point: a framework for conceptualizing the effects of various practice conditions in motor learning. Journal of motor behavior 36, 2 (2004), 212–224.
[29]
Joao Guerra, Lee Smith, Domenico Vicinanza, Brendon Stubbs, Nicola Veronese, and G Williams. 2020. The Use of Sonification for Physiotherapy in Human Movement Tasks: A Scoping Review. Science & Sports 35, 3 (2020), 119–129.
[30]
Caitlin Hamilton, Meryl Lovarini, Annie McCluskey, Tarcisio Folly de Campos, and Leanne Hassett. 2019. Experiences of therapists using feedback-based technology to improve physical function in rehabilitation settings: a qualitative systematic review. Disability and Rehabilitation 41, 15 (2019), 1739–1750.
[31]
Caitlin Hamilton, Meryl Lovarini, Maayken van den Berg, Annie McCluskey, and Leanne Hassett. 2022. Usability of affordable feedback-based technologies to improve mobility and physical activity in rehabilitation: a mixed methods study. Disability and Rehabilitation 44, 15 (2022), 4029–4038.
[32]
Matevž Hribernik, Anton Umek, Sašo Tomažič, and Anton Kos. 2022. Review of real-time biomechanical feedback systems in sport and rehabilitation. Sensors 22, 8 (2022), 3006.
[33]
Monique WM Jaspers, Thiemo Steen, Cor Van Den Bos, and Maud Geenen. 2004. The think aloud method: a guide to user interface design. International journal of medical informatics 73, 11-12 (2004), 781–795.
[34]
Gustav Johansson, Mattias Adevåg, and Jacob Milton. 2023. Bluetooth audio codecs in a real-time interactive context.
[35]
Prithvi Kantan, Erika G Spaich, and Sofia Dahl. 2022. An Embodied Sonification Model for Sit-to-Stand Transfers. Frontiers in psychology 13 (2022).
[36]
Prithvi Kantan, Erika G Spaich, and Sofia Dahl. 2022. A technical framework for musical biofeedback in stroke rehabilitation. IEEE Transactions on Human-Machine Systems 52, 2 (2022), 220–231.
[37]
Prithvi Ravi Kantan. 2023. Orchestrating Motion: Real-Time Sonification Tools and Methods to Support Movement Rehabilitation. (2023).
[38]
Prithvi Ravi Kantan, Sofia Dahl, Helle Rovsing Jørgensen, Chetali Khadye, and Erika G Spaich. 2023. Designing Ecological Auditory Feedback on Lower Limb Kinematics for Hemiparetic Gait Training. Sensors 23, 8 (2023), 3964.
[39]
Prithvi Ravi Kantan, Sofia Dahl, Stefania Serafin, and Erika G. Spaich. 2021. Sonifying Gait Kinematics Using the Sound of Wading: A Study on Ecological Movement Representations. In The 28th International Conference on Auditory Display (ICAD 2023).
[40]
Anton Kos and Anton Umek. 2018. Biomechanical Biofeedback. Springer. https://doi.org/10.1007/978-3-319-91349-0_2
[41]
Anton Kos and Anton Umek. 2018. Biomechanical Biofeedback Systems and Applications. Springer.
[42]
Jules JM Kraaijkamp, Eléonore F van Dam van Isselt, Anke Persoon, Anke Versluis, Niels H Chavannes, and Wilco P Achterberg. 2021. eHealth in geriatric rehabilitation: systematic review of effectiveness, feasibility, and usability. Journal of medical Internet research 23, 8 (2021), e24015.
[43]
Amir Lahav, Elliot Saltzman, and Gottfried Schlaug. 2007. Action Representation of Sound: Audiomotor Recognition Network while Listening to Newly Acquired Actions. Journal of Neuroscience 27, 2 (2007), 308–314.
[44]
Peter Langhorne, Julie Bernhardt, and Gert Kwakkel. 2011. Stroke rehabilitation. The Lancet 377, 9778 (2011), 1693–1702.
[45]
Micheline Lesaffre. 2018. Investigating Embodied Music Cognition for Health and Well-Being. Springer Handbook of Systematic Musicology (2018), 779–791.
[46]
Mindy F Levin. 2020. What is the potential of virtual reality for post-stroke sensorimotor rehabilitation?Expert review of neurotherapeutics 20, 3 (2020), 195–197.
[47]
Mindy F Levin and Marika Demers. 2021. Motor learning in neurological rehabilitation. Disability and rehabilitation 43, 24 (2021), 3445–3453.
[48]
James R Lewis. 2018. The system usability scale: past, present, and future. International Journal of Human–Computer Interaction 34, 7 (2018), 577–590.
[49]
Hans Lindetorp. 2024. Learning Body Movement Sonification in an Hour. In Web Audio Conference. Zenodo.
[50]
Dagmar Linnhoff, Shahab Alizadeh, Nina Schaffert, and Klaus Mattes. 2020. Use of Acoustic Feedback to Change Gait Patterns: Implementation and Transfer to Motor Learning Theory—-A Scoping Review. Journal of Motor Learning and Development 1, aop (2020), 1–21.
[51]
Dagmar Linnhoff, Roy Ploigt, and Klaus Mattes. 2022. Sofigait—A Wireless Inertial Sensor-Based Gait Sonification System. Sensors 22, 22 (2022), 8782.
[52]
Christina Ma, Duo Wong, Gilbert W.K. Lam, Anson Wan, and Winson Lee. 2016. Balance Improvement Effects of Biofeedback Systems with State-of-the-Art Wearable Sensors: A Systematic Review. Sensors 16 (03 2016), 434. https://doi.org/10.3390/s16040434
[53]
Pieter-Jan Maes, Jeska Buhmann, and Marc Leman. 2016. 3MO: A Model for Music-Based Biofeedback. Frontiers in Neuroscience 1 (12 2016).
[54]
Pieter-Jan Maes, Marc Leman, Caroline Palmer, and Marcelo Wanderley. 2014. Action-based Effects on Music Perception. Frontiers in Psychology 4 (2014), 1008.
[55]
Mauro Murgia, Ilaria Santoro, Giorgia Tamburini, Valter Prpic, Fabrizio Sors, Alessandra Galmonte, and Tiziano Agostini. 2016. Ecological Sounds Affect Breath Duration More Than Artificial Sounds. Psychological Research 80, 1 (2016), 76–81.
[56]
John G. Neuhoff. 2019. Is Sonification Doomed to Fail?. In Proceedings of the 25th International Conference on Auditory Display (ICAD 2019). Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, 327–330. https://doi.org/10.21785/icad2019.069
[57]
Joseph W Newbold, Nadia Bianchi-Berthouze, Nicolas E Gold, Ana Tajadura-Jiménez, and Amanda CdC Williams. 2016. Musically informed sonification for chronic pain rehabilitation: Facilitating progress & avoiding over-doing. In Proceedings of the 2016 chi conference on human factors in computing systems. 5698–5703.
[58]
Hamza Niaz and Muhammad Hanif. 2010. Investigating the possible role and usefulness of video capture virtual reality in motor impairment rehabilitation.
[59]
Thomas H Nown, Priti Upadhyay, Andrew Kerr, Ivan Andonovic, Christos Tachtatzis, and Madeleine A Grealy. 2022. A Mapping Review of Real-Time Movement Sonification Systems for Movement Rehabilitation. IEEE Reviews in Biomedical Engineering (2022).
[60]
Suzanne R O’Brien, Ying Xue, Gail Ingersoll, and Adam Kelly. 2013. Shorter length of stay is associated with worse functional outcomes for medicare beneficiaries with stroke. Physical therapy 93, 12 (2013), 1592–1602.
[61]
Gaëtan Parseihian, Sølvi Ystad, Mitsuko Aramaki, and Richard Kronland-Martinet. 2015. The process of sonification design for guidance tasks. J Mob Med 9, 2 (2015).
[62]
Juliane Pietschmann, F Geu Flores, and Thomas Jöllenbeck. 2019. Gait training in orthopedic rehabilitation after joint replacement-back to normal gait with sonification?International Journal of computer science in Sport 18, 2 (2019), 34–48.
[63]
Camille E Proulx, Manouchka T Louis Jean, Johanne Higgins, Dany H Gagnon, and Numa Dancause. 2022. Somesthetic, visual, and auditory feedback and their interactions applied to upper limb neurorehabilitation technology: a narrative review to facilitate contextualization of knowledge. Frontiers in Rehabilitation Sciences 3 (2022), 789479.
[64]
Julia Reh, Gerd Schmitz, Tong-Hun Hwang, and Alfred O Effenberg. 2022. Loudness affects motion: asymmetric volume of auditory feedback results in asymmetric gait in healthy young adults. BMC Musculoskeletal Disorders 23, 1 (2022), 586.
[65]
Matthew WM Rodger, William R Young, and Cathy M Craig. 2013. Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson’s Disease. IEEE Transactions on Neural Systems and Rehabilitation Engineering 22, 3 (2013), 543–548.
[66]
Ralph L Sacco, Scott E Kasner, Joseph P Broderick, Louis R Caplan, JJ Connors, Antonio Culebras, Mitchell SV Elkind, Mary G George, Allen D Hamdan, Randall T Higashida, 2013. An updated definition of stroke for the 21st century: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 44, 7 (2013), 2064–2089.
[67]
Ludwig Schmid, Andrea Glässel, Corina Schuster-Amft, 2016. Therapists’ perspective on virtual reality training in patients after stroke: a qualitative study reporting focus group results from three hospitals. Stroke research and treatment 2016 (2016).
[68]
Gerd Schmitz, Daniela Kroeger, and Alfred O Effenberg. 2014. A mobile sonification system for stroke rehabilitation. In The 20th International Conference on Auditory Display. 1–7.
[69]
Daniel S Scholz, Liming Wu, Jonas Pirzer, Johann Schneider, Jens D Rollnik, Michael Großbach, and Eckart O Altenmüller. 2014. Sonification as a possible stroke rehabilitation strategy. Frontiers in neuroscience 8 (2014), 332.
[70]
Roland Sigrist, Georg Rauter, Robert Riener, and Peter Wolf. 2012. Augmented Visual, Auditory, Haptic, and Multimodal Feedback in Motor Learning: A Review. Psychonomic Bulletin & Review 20 (11 2012). https://doi.org/10.3758/s13423-012-0333-8
[71]
Jacob Spencer, Steven L Wolf, and Trisha M Kesar. 2021. Biofeedback for post-stroke gait retraining: a review of current evidence and future research directions in the context of emerging technologies. Frontiers in Neurology 12 (2021), 419.
[72]
Rosalyn Stanton, Louise Ada, Catherine M Dean, and Elisabeth Preston. 2017. Biofeedback improves performance in lower limb activities more than usual therapy in people following stroke: a systematic review. Journal of physiotherapy 63, 1 (2017), 11–16.
[73]
Jeremiah J Tate and Clare E Milner. 2010. Real-time kinematic, temporospatial, and kinetic biofeedback during gait retraining in patients: a systematic review. Physical therapy 90, 8 (2010), 1123–1134.
[74]
James Philip Thomas and Maggie Shiffrar. 2010. I Can See You Better if I Can Hear You Coming: Action-Consistent Sounds Facilitate the Visual Detection of Human Gait. Journal of Vision 10, 12 (2010), 14–14.
[75]
Andrés Villa Torres, Viktoria Kluckner, and Karmen Franinovic. 2013. Development of a sonification method to enhance gait rehabilitation. Proceedings of the ISon (2013), 37–43.
[76]
Anton Umek, Sašo Tomažič, and Anton Kos. 2015. Wearable training system with real-time biofeedback and gesture user interface. Personal and Ubiquitous Computing 19 (2015), 989–998.
[77]
Rudi Villing, Victor Lazzarini, Dawid Czesak, Sean O’Leary, and Joseph Timoney. 2015. Approaches for constant audio latency on Android. (2015).
[78]
Katharina Vogt, David Pirrò, Ingo Kobenz, Robert Höldrich, and Gerhard Eckel. 2009. PhysioSonic-Evaluated movement sonification as auditory feedback in physiotherapy. In International Symposium on Computer Music Modeling and Retrieval. Springer, 103–120.
[79]
Derick T Wade. 2005. Describing rehabilitation interventions. Clinical rehabilitation 19, 8 (2005), 811–818.
[80]
Rosalie H Wang, Lisa K Kenyon, Katherine S McGilton, William C Miller, Nina Hovanec, Jennifer Boger, Pooja Viswanathan, Julie M Robillard, and Stephen M Czarnuch. 2021. The time is now: a FASTER approach to generate research evidence for technology-based interventions in the field of disability and rehabilitation. Archives of Physical Medicine and Rehabilitation 102, 9 (2021), 1848–1859.
[81]
Gert Westerman and Eduardo Reck Miranda. 2002. Modelling the Development of Mirror Neurons for Auditory-Motor Integration. Journal of New Music Research 31, 4 (2002), 367–375.
[82]
Viktoria Zander, Christine Gustafsson, Sara Landerdahl Stridsberg, and Johan Borg. 2023. Implementation of welfare technology: a systematic review of barriers and facilitators. Disability and Rehabilitation: Assistive Technology 18, 6 (2023), 913–928.
[83]
Agnes Zijlstra, Martina Mancini, Lorenzo Chiari, and Wiebren Zijlstra. 2010. Biofeedback for training balance and mobility tasks in older populations: a systematic review. Journal of neuroengineering and rehabilitation 7, 1 (2010), 1–15.

Index Terms

  1. Making Movement Sonification Usable in Clinical Gait Rehabilitation: A User-Centered Study
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Please enable JavaScript to view thecomments powered by Disqus.

            Information & Contributors

            Information

            Published In

            cover image ACM Other conferences
            AM '24: Proceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures
            September 2024
            565 pages
            ISBN:9798400709685
            DOI:10.1145/3678299
            This work is licensed under a Creative Commons Attribution International 4.0 License.

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            Published: 18 September 2024

            Check for updates

            Author Tags

            1. Biofeedback
            2. Ecological Feedback
            3. Gait
            4. Movement Sonification
            5. Rehabilitation Technology
            6. User-Centered Design

            Qualifiers

            • Research-article
            • Research
            • Refereed limited

            Funding Sources

            • BETA.HEALTH (The Danish National Innovation Platform for Future Healthcare
            • NordForsk?s Nordic University Hub, Nordic Sound, and Music Computing Network NordicSMC

            Conference

            AM '24

            Acceptance Rates

            Overall Acceptance Rate 177 of 275 submissions, 64%

            Contributors

            Other Metrics

            Bibliometrics & Citations

            Bibliometrics

            Article Metrics

            • 0
              Total Citations
            • 113
              Total Downloads
            • Downloads (Last 12 months)113
            • Downloads (Last 6 weeks)36
            Reflects downloads up to 15 Jan 2025

            Other Metrics

            Citations

            View Options

            View options

            PDF

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader

            HTML Format

            View this article in HTML Format.

            HTML Format

            Login options

            Media

            Figures

            Other

            Tables

            Share

            Share

            Share this Publication link

            Share on social media