More Web Proxy on the site http://driver.im/

short-paper

Playing music with the eyes through an isomorphic interface

Authors:

Nicola Davanzo,

Piercarlo Dondi,

Marco PortaAuthors Info & Claims

COGAIN '18: Proceedings of the Workshop on Communication by Gaze Interaction

Article No.: 5, Pages 1 - 5

https://doi.org/10.1145/3206343.3206350

Published: 15 June 2018 Publication History

Abstract

Playing music with the eyes is a challenging task. In this paper, we propose a virtual digital musical instrument, usable by both motor-impaired and able-bodied people, controlled through an eye tracker and a "switch". Musically speaking, the layout of the graphical interface is isomorphic, since the harmonic relations between notes have the same geometrical shape regardless of the key signature of the music piece. Four main design principles guided our choices, namely: (1) Minimization of eye movements, especially in case of large note intervals; (2) Use of a grid layout where "nodes" (keys) are connected each other through segments (employed as guides for the gaze); (3) No need for smoothing filters or time thresholds; and (4) Strategic use of color to facilitate gaze shifts. Preliminary tests, also involving another eye-controlled musical instrument, have shown that the developed system allows "correct" execution of music pieces even when characterized by complex melodies.

References

[1]

S. Bailey, A. Scott, H. Wright, I. Symonds, and K. Ng. 2010. Eye.Breathe.Music: creating music through minimal movement. In Proceedings of the 2010 International Conference on Electronic Visualisation and the Arts (EVA'10), 254--258.

Digital Library

[2]

S. Ceurstemont. 2016. Virtual instruments let you play music using only your eyes. New Scientist, 16 March 2016 (https://www.newscientist.com/article/2081150-virtual-instruments-let-you-play-music-using-only-your-eyes/).

[3]

S. Dredge. 2015. Disabled boy learns to play piano with his eyes using virtual-reality headset. The Guardian, 14 January 2015 (https://www.theguardian.com/technology/2015/jan/14/disabled-play-piano-eyes-virtual-reality-headset).

[4]

P. M. Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 1954 47(6): 381--391.

[5]

K. Holmqvist, M. Nyström, and F. MulveyI. 2012. Eye tracker data quality: What it is and how to measure it. In Proceedings of the 2012 ACM Symposium of Eye Tracking Research and Applications (ETRA 2012), 45--52

Digital Library

[6]

A. Hornof. 2014. The Prospects for Eye-Controlled Musical Performance. In Proceedings of the 2014 Conference on New Interfaces for Musical Expression (NIME'14), 461--466.

[7]

A. Hornof, T. Rogers, J. Stolet, and T. Halverson. 2008. Bringing to Life the Musical Properties of the Eyes. University of Oregon Department of Computer and Information Science, Technical Report 08-05.

[8]

A. Hornof and L. Sato. 2004. EyeMusic: Making Music with the Eyes. In Proceedings of the 2004 Conference on New Interfaces for Musical Expression (NIME'04), 185--188.

Digital Library

[9]

R. J. K. Jacob. 1995. Eye tracking in advanced interface design. In Virtual environments and advanced interface design, Oxford University Press, 258--288.

Digital Library

[10]

J. Kim, G. Schiemer, and T. Narushima. 2007. Oculog: Playing with Eye Movements. In Proceedings of the 2007 Conference on New Interfaces for Musical Expression (NIME'07), 50--55.

Digital Library

[11]

C. I. Lou, D. Migotina, J. P. Rodrigues, J. Semedo, F. Wan, P. U. Mak, P. I. Mak, M. I. Vai, F. Melicio, J. G. Pereira, and A. Rosa. 2012. Object Recognition Test in Peripheral Vision: A Study on the Influence of Object Color, Pattern and Shape. In Proceedings of the 2012 International Conference on Brain Informatics (BI'12), 18--26.

Digital Library

[12]

S. Maupin, D. Gerhard, and B. Park. 2011. Isomorphic Tessellations for Musical Keyboards. In Proceedings of the 8th Sound and Music Computing Conference (SMC'11), 471--478.

[13]

C. H. Morimoto, A. Diaz-Tula, J. A. T. Leyva, and C. E. L. Elmadjian. 2015. EyeJam: A Gaze-Controlled Musical Interface. In Proceedings of the 2015 Brazilian Symposium on Human Factors in Computing Systems (IHC'15), 360--368.

Digital Library

[14]

P. Q. Pfordresher and C. Palmer. 2002. Effects of delayed auditory feedback on timing of music performance. Psychological Research, 2002; 16: 71--79.

[15]

Tobii Dynavox. Eye Play Music. Software downloadable from http://www2.tobiidynavox.com/product/eye-play-music/.

[16]

Z. Vamvakousis and R. Ramirez. 2016. The EyeHarp: A Gaze-Controlled Digital Musical Instrument. Front. Pychol. 2016; 7: 906.

Cited By

Dondi PSapuppo SPorta M(2024)LeyenesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103204184:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103204
Dondi PPorta M(2023)Gaze-Based Human–Computer Interaction for Museums and Exhibitions: Technologies, Applications and Future PerspectivesElectronics10.3390/electronics1214306412:14(3064)Online publication date: 13-Jul-2023
https://doi.org/10.3390/electronics12143064
A. F. Camporez HM. de Freitas YA. L. Silva JL. Costalonga LR. O. Rocha H(2023)Music Segmentation and Similarity Estimation Applied to a Gaze-Controlled Musical InterfaceRevista Vórtex10.33871/23179937.2023.11.1.706811:1(1-25)Online publication date: 2-May-2023
https://doi.org/10.33871/23179937.2023.11.1.7068
Show More Cited By

Index Terms

Playing music with the eyes through an isomorphic interface

Recommendations

Playing music: an installation based on Xenakis' musical games
AVI '06: Proceedings of the working conference on Advanced visual interfaces

Iannis Xenakis' works Duel and Strategie are two music games: sounds play the role of moves in a match where the players are the two orchestra conductors. They decide which part of the score is to be played in answer to the opposite conductor's choice, ...
GazeSync: Eye Movement Transfer Using an Optical Eye Tracker and Monochrome Liquid Crystal Displays
IUI '22 Companion: Companion Proceedings of the 27th International Conference on Intelligent User Interfaces

Can we see the world through the eyes of somebody else? We present an early work to transfer eye gaze from one person to another. Imagine you can follow the eye gaze of an instructor while explaining a complex work step, or you can experience a painting ...
Just blink your eyes: a head-free gaze tracking system
CHI EA '03: CHI '03 Extended Abstracts on Human Factors in Computing Systems

We propose a head-free, easy-setup gaze tracking system designed for a gaze-based Human-Computer Interaction. Our system enables the user to interact with the computer soon after catching the user's eye blinks. The user can move his/her head freely ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

COGAIN '18: Proceedings of the Workshop on Communication by Gaze Interaction

June 2018

69 pages

ISBN:9781450357906

DOI:10.1145/3206343

General Chairs:
Carlos Morimoto
University of São Paulo, São Paulo, SP, Brazil
,
Thies Pfeiffer
Bielefeld University, CITEC, Bielefeld, NRW, Germany

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ETRA '18

Sponsor:

ETRA '18: 2018 Symposium on Eye Tracking Research and Applications

June 15, 2018

Warsaw, Poland

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
186
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dondi PSapuppo SPorta M(2024)LeyenesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103204184:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103204
Dondi PPorta M(2023)Gaze-Based Human–Computer Interaction for Museums and Exhibitions: Technologies, Applications and Future PerspectivesElectronics10.3390/electronics1214306412:14(3064)Online publication date: 13-Jul-2023
https://doi.org/10.3390/electronics12143064
A. F. Camporez HM. de Freitas YA. L. Silva JL. Costalonga LR. O. Rocha H(2023)Music Segmentation and Similarity Estimation Applied to a Gaze-Controlled Musical InterfaceRevista Vórtex10.33871/23179937.2023.11.1.706811:1(1-25)Online publication date: 2-May-2023
https://doi.org/10.33871/23179937.2023.11.1.7068
Davanzo NValente LLudovico LAvanzini F(2023)Kiroll: A Gaze-Based Instrument for Quadriplegic Musicians Based on the Context-Switching ParadigmProceedings of the 18th International Audio Mostly Conference10.1145/3616195.3616225(59-62)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3616195.3616225
Ivanyi BTjemsland TTsalidis de Zabala CToth LDyrholm MNaylor SParadiso ALamb DChudge JAdjorlu ASerafin S(2023)DuoRhythmo: Design and remote user experience evaluation (UXE) of a collaborative accessible digital musical interface (CADMI) for people with ALS (PALS)Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581285(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581285
Davanzo NAvanzini FLudovico LMoreno DMoreira ATymoshchuk OAzevedo JMarques C(2023)A Case Study on Netychords: Crafting Accessible Digital Musical Instrument Interaction for a Special Needs ScenarioComputer-Human Interaction Research and Applications10.1007/978-3-031-49425-3_22(353-372)Online publication date: 23-Dec-2023
https://doi.org/10.1007/978-3-031-49425-3_22
Davanzo NAvanzini F(2021)Resin: a Vocal Tract Resonances and Head Based Accessible Digital Musical InstrumentProceedings of the 16th International Audio Mostly Conference10.1145/3478384.3478403(280-283)Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1145/3478384.3478403
Di Maio MDondi PLombardi LPorta M(2021)Hybrid Manual and Gaze-Based Interaction With a Robotic Arm2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA )10.1109/ETFA45728.2021.9613371(1-4)Online publication date: 7-Sep-2021
https://doi.org/10.1109/ETFA45728.2021.9613371
Dondi PPorta MDonvito AVolpe G(2021)A gaze-based interactive system to explore artwork imageryJournal on Multimodal User Interfaces10.1007/s12193-021-00373-z16:1(55-67)Online publication date: 21-May-2021
https://doi.org/10.1007/s12193-021-00373-z
Davanzo NAvanzini F(2020)Hands-Free Accessible Digital Musical Instruments: Conceptual Framework, Challenges, and PerspectivesIEEE Access10.1109/ACCESS.2020.30199788(163975-163995)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3019978
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents