Sensorimotor Synchronization in Blind Musicians: Does Lack of Vision Influencenon-verbal Musical Communication?
Pages 31 - 36
Abstract
Musical communication can be considered as a form of social interaction and it requires a certain degree of inter-individual cortical synchronization. Scientific literature has shown that visual experience is necessary for social interaction and visual impairment negatively influences nonverbal communication both in children and adults. In this study, we present a pilot multimodal dataset to investigate the role that visual experience plays in the context of musical interaction. We selected a mixed interaction scenario (i.e., sighted and blind performers) to investigate non-verbal communication patterns. In particular, we recorded motion tracking data to analyze the interaction between two musicians in three different contexts with the same cellist as soloist accompanied by a (1) blind pianist, (2) sighted pianist, and (3) sighted but blindfolded pianist. Recordings comprised upper-body motion capture and audio data. We also investigated human perception of changes in non-verbal behaviors, by means of an online annotation questionnaire. Preliminary results are presented.
References
[1]
Jean-Julien Aucouturier and Clément Canonne. 2017. Musical friends and foes: The social cognition of affiliation and control in improvised interactions. Cognition 161(2017), 94–108.
[2]
Claudio Babiloni, Fabrizio Vecchio, Francesco Infarinato, Paola Buffo, Nicola Marzano, Danilo Spada, Simone Rossi, Ivo Bruni, Paolo M Rossini, and Daniela Perani. 2011. Simultaneous recording of electroencephalographic data in musicians playing in ensemble. cortex 47, 9 (2011), 1082–1090.
[3]
Leonardo Badino, Alessandro D’Ausilio, Donald Glowinski, Antonio Camurri, and Luciano Fadiga. 2014. Sensorimotor communication in professional quartets. Neuropsychologia 55(2014), 98–104.
[4]
Laura Bishop, Carlos Cancino-Chacón, and Werner Goebl. 2019. Moving to communicate, moving to interact: Patterns of body motion in musical duo performance. Music Perception: An Interdisciplinary Journal 37, 1 (2019), 1–25.
[5]
Laura Bishop and Werner Goebl. 2015. When they listen and when they watch: Pianists’ use of nonverbal audio and visual cues during duet performance. Musicae Scientiae 19, 1 (2015), 84–110.
[6]
Antonio Camurri, Paolo Coletta, Alberto Massari, Barbara Mazzarino, Massimiliano Peri, Matteo Ricchetti, Andrea Ricci, and Gualtiero Volpe. 2004. Toward real-time multimodal processing: EyesWeb 4.0. In Proceedings of the AISB 2004 Convention: Motion, Emotion and Cognition. 22–26.
[7]
Lee J Cronbach. 1951. Coefficient alpha and the internal structure of tests. psychometrika 16, 3 (1951), 297–334.
[8]
Alessandro D’Ausilio, Leonardo Badino, Yi Li, Sera Tokay, Laila Craighero, Rosario Canto, Yiannis Aloimonos, and Luciano Fadiga. 2012. Leadership in orchestra emerges from the causal relationships of movement kinematics. PLoS one 7, 5 (2012), e35757.
[9]
Thomas Dolk, Roman Liepelt, Wolfgang Prinz, and Katja Fiehler. 2013. Visual experience determines the use of external reference frames in joint action control. PLoS One 8, 3 (2013), e59008.
[10]
Alessandro D’Ausilio, Giacomo Novembre, Luciano Fadiga, and Peter E Keller. 2015. What can music tell us about social interaction?Trends in cognitive sciences 19, 3 (2015), 111–114.
[11]
Joseph L Fleiss and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement 33, 3 (1973), 613–619.
[12]
Melissa J Frame. 2000. The relationship between visual impairment and gestures. Journal of Visual Impairment & Blindness 94, 3 (2000), 155–171.
[13]
Donald Glowinski, Fabrizio Bracco, Carlo Chiorri, and Didier Grandjean. 2016. Music ensemble as a resilient system. Managing the unexpected through group interaction. Frontiers in Psychology 7 (2016), 1548.
[14]
Donald Glowinski, Maurizio Mancini, Roddy Cowie, Antonio Camurri, Carlo Chiorri, and Cian Doherty. 2013. The movements made by performers in a skilled quartet: a distinctive pattern, and the function that it serves. Frontiers in psychology 4 (2013), 841.
[15]
Jana M Iverson and Susan Goldin-Meadow. 2001. The resilience of gesture in talk: Gesture in blind speakers and listeners. Developmental Science 4, 4 (2001), 416–422.
[16]
Satoshi Kawase. 2014. Gazing behavior and coordination during piano duo performance. Attention, Perception, & Psychophysics 76, 2 (2014), 527–540.
[17]
J Richard Landis and Gary G Koch. 1977. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics (1977), 363–374.
[18]
Jacques Launay. 2015. Musical sounds, motor resonance, and detectable agency. Empirical musicology review: EMR 10, 1-2 (2015), 30.
[19]
Janeen D Loehr, Dimitrios Kourtis, Cordula Vesper, Natalie Sebanz, and Günther Knoblich. 2013. Monitoring individual and joint action outcomes in duet music performance. Journal of cognitive neuroscience 25, 7 (2013), 1049–1061.
[20]
Nikki Moran, Lauren V Hadley, Maria Bader, and Peter E Keller. 2015. Perception of ‘back-channeling’nonverbal feedback in musical duo improvisation. PLoS One 10, 6 (2015), e0130070.
[21]
Johanna Sänger, Viktor Müller, and Ulman Lindenberger. 2012. Intra-and interbrain synchronization and network properties when playing guitar in duets. Frontiers in human neuroscience 6 (2012), 312.
[22]
Johanna Sänger, Viktor Müller, and Ulman Lindenberger. 2013. Directionality in hyperbrain networks discriminates between leaders and followers in guitar duets. Frontiers in human neuroscience 7 (2013), 234.
[23]
Erica Volta, Radoslaw Niewiadomski, Temitayo Olugbade, Carla Gilio, Elena Cocchi, Nadia Berthouze, Monica Gori, and Gualtiero Volpe. 2019. Analysis of cognitive states during bodily exploration of mathematical concepts in visually impaired children. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 1–7.
[24]
Alan M Wing, Satoshi Endo, Adrian Bradbury, and Dirk Vorberg. 2014. Optimal feedback correction in string quartet synchronization. Journal of The Royal Society Interface 11, 93 (2014), 20131125.
Index Terms
- Sensorimotor Synchronization in Blind Musicians: Does Lack of Vision Influencenon-verbal Musical Communication?
Index terms have been assigned to the content through auto-classification.
Recommendations
Empowering Blind Musicians to Compose and Notate Music with SoundCells
ASSETS '22: Proceedings of the 24th International ACM SIGACCESS Conference on Computers and AccessibilityCommercial technologies for notating music pose usage barriers to blind and visually impaired (BVI) musicians because they use graphic user interfaces and only produce visual, print scores. However, more research to date has studied how to make existing ...
Reflexive loopers for solo musical improvisation
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsLoop pedals are real-time samplers that playback audio played previously by a musician. Such pedals are routinely used for music practice or outdoor "busking". However, loop pedals always playback the same material, which can make performances ...
Synchronization and fluctuation of rhythm in musical cooperative performance
HCII'11: Proceedings of the 14th international conference on Human-computer interaction: design and development approaches - Volume Part IA live musical performance gives us better impression than recorded music heard from a portable music player. From player's point of view, live performance also gives better impression than playing music with metronome or recorded music. Thus, the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2021
418 pages
ISBN:9781450384711
DOI:10.1145/3461615
Copyright © 2021 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 December 2021
Check for updates
Author Tags
Qualifiers
- Abstract
- Research
- Refereed limited
Funding Sources
- University of Genoa
Conference
ICMI '21
Sponsor:
ICMI '21: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
October 18 - 22, 2021
QC, Montreal, Canada
Acceptance Rates
Overall Acceptance Rate 453 of 1,080 submissions, 42%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 68Total Downloads
- Downloads (Last 12 months)10
- Downloads (Last 6 weeks)2
Reflects downloads up to 12 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format