More Web Proxy on the site http://driver.im/

extended-abstract

Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction

Authors:

Richard Allmendinger,

Ricardo ClimentAuthors Info & Claims

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 425, Pages 1 - 8

https://doi.org/10.1145/3544549.3582738

Published: 19 April 2023 Publication History

Abstract

Within the last few years, wearable sensor technologies have allowed us to access novel biometrics that give us the ability to connect musical gesture to computing systems. Doing this affords us to study how we perform musically and understand the process at data level. However, biometric information is complex and cannot be directly mapped to digital systems. In this work, we study how guitar performance techniques can be captured/analysed towards developing an AI which can provide real-time feedback to guitar students. We do this by performing musical exercises on the guitar whilst acquiring and processing biometric (plus audiovisual) information during their performance. Our results show: there are notable differences within biometrics when playing a guitar scale in two different ways (legato and staccato) and this outcome can be used to motivate our intention to build an AI guitar tutor.

Supplementary Material

MP4 File (3544549.3582738-talk-video.mp4)

Pre-recorded Video Presentation

Download
28.52 MB

References

[1]

ABRSM. 2019. Guitar Grade 1. https://tinyurl.com/3euz48xt [Online; accessed 8th of November, 2022].

[2]

Zainal Arief, Indra Adji Sulistijono, and Roby Awal Ardiansyah. 2015. Comparison of Five Time Series EMG Features Extractions Using Myo Armband. In 2015 International Electronics Symposium. 11–14.

[3]

Robert S. Armiger and R. Jacob Vogelstein. 2008. Air-Guitar Hero: A real-time video game interface for training and evaluation of dexterous upper-extremity neuroprosthetic control algorithms. In 2008 IEEE Biomedical Circuits and Systems Conference. 121–124.

[4]

Gobinath Aroganam, Nadarajah Manivannan, and David Harrison. 2019. Review on Wearable Technology Sensors Used in Consumer Sport Applications. Sensors 19, 9 (2019).

[5]

Jamie Bullock and Balandino Di Donato. 2016. Approaches to Visualising the Spatial Position of ‘Sound-objects’. In Proceedings of the Conference on Electronic Visualisation and the Arts. 15–22.

Digital Library

[6]

Eleanor Busby. 2018. Poorer children in UK are being priced out of learning to play musical instruments, report warns. https://tinyurl.com/mpvpm7uz [Online; accessed 22nd of December, 2021].

[7]

ChrisRhodesMusic. 2022. Chris Rhodes - Deep Blue AIr (2020). https://tinyurl.com/5n7wky6x [Online; accessed 8th of December, 2022 ].

[8]

ChrisRhodesMusic. 2022. Chris Rhodes - Membrana Neopermeable (2022). https://tinyurl.com/y97vh2sz [Online; accessed 8th of December, 2022 ].

[9]

David Dalmazzo and Rafael Ramírez. 2019. Bowing Gestures Classification in Violin Performance: A Machine Learning Approach. Frontiers in Psychology 10 (2019), 344.

[10]

David Dalmazzo, George Waddell, and Rafael Ramírez. 2021. Applying Deep Learning Techniques to Estimate Patterns of Musical Gesture. Frontiers in Psychology 11 (2021).

[11]

Balandino Di Donato, Jamie Bullock, and Atau Tanaka. 2018. Myo Mapper: a Myo armband to OSC mapper. In Proceedings of the International Conference on New Interfaces for Musical Expression. 138–143.

[12]

Balandino Di Donato, James Dooley, Jason Hockman, and Simon Hall. 2017. MyoSpat: A hand-gesture controlled system for sound and light projections manipulation. In Proceedings of the 2017 International Computer Music Conference.

[13]

Çağrı Erdem, Charles Martin, Qichao Lan, Jim Tørresen, Julian Fuhrer, and Alexander Jensenius Refsum. 2020. Towards Playing in the ‘Air’: Modeling Motion–Sound Energy Relationships in Electric Guitar Performance Using Deep Neural Networks. In Proceedings of the 17th Sound and Music Computing Conference. 177–184.

[14]

Rob Hamilton. 2015. Carillon. https://tinyurl.com/4j5e9hwp [Online; accessed 7th of December, 2022].

[15]

Keep.It.Human. 2020. Duel of Strings (5k - 360 video): Karin Kellqvist vs. Ricardo Climent- Full concert (ambisonics). https://tinyurl.com/4beakder [Online; accessed 7th of December, 2022].

[16]

Danijel Kucak, Vedran Juricic, and Goran Dambic. 2018. Machine Learning in Education - a Survey of Current Research Trends. In Proceedings of the 29th DAAAM International Symposium. 0406–0410.

[17]

Trinity College London. 2019. Acoustic Guitar. https://tinyurl.com/2p9645u2 [Online; accessed 8th of November, 2022].

[18]

Kevin Ma. 2020. Artificial Intelligence Aided Training in Ping Pong Sport Education. In 2020 Second International Conference on Transdisciplinary AI (TransAI). 43–49.

[19]

Kristian Nymoen, Mari Romarheim Haugen, and Alexander Refsum Jensenius. 2015. MuMYO — Evaluating and Exploring the MYO Armband for Musical Interaction. In Proceedings of the International Conference on New Interfaces for Musical Expression. 215–218.

Digital Library

[20]

Royal Philharmonic Orchestra. 2018. A New Era for Orchestral Music. Technical Report. UK.

[21]

Sungmee Park and S. Jayaraman. 2003. Enhancing the quality of life through wearable technology. IEEE Engineering in Medicine and Biology Magazine 22, 3 (2003), 41–48.

[22]

Dipesh S. Patle, Davide Manca, Salman Nazir, and Swapnil Sharma. 2019. Operator training simulators in virtual reality environment for process operators: a review. Virtual Reality 23, 3 (2019), 293–311.

Digital Library

[23]

Jack Purcher. 2021. Apple wins a Major Apple Watch Patent that supports Face ID and Sports Performance Analysis via in-band Sensors. https://tinyurl.com/2p83puce [Online; accessed 5th of November 2022].

[24]

Jack Purcher. 2021. The U.S. Patent Office has Published Facebook’s Patent behind their Future AR Glasses Wristband Input Device. https://tinyurl.com/c9zhhcy9 [Online; accessed 5th of November 2022].

[25]

Chris Rhodes, Richard Allmendinger, and Ricardo Climent. 2019. New Interfaces for Classifying Performance Gestures in Music. In Intelligent Data Engineering and Automated Learning – IDEAL 2019(Lecture Notes in Computer Science). 31–42.

[26]

Chris Rhodes, Richard Allmendinger, and Ricardo Climent. 2020. New Interfaces and Approaches to Machine Learning When Classifying Gestures within Music. Entropy 22, 12 (2020), 1384.

[27]

Chris Rhodes, Richard Allmendinger, and Ricardo Climent. 2022. Classifying Biometric Data for Musical Interaction Within Virtual Reality. In Artificial Intelligence in Music, Sound, Art and Design(Lecture Notes in Computer Science). 385–400.

[28]

Rockschool. 2019. Acoustic Guitar. https://www.rslawards.com/rockschool/graded-exams/acoustic-guitar/grade-1/ [Online; accessed 8th of November, 2022].

[29]

Ruben C. Rodriguez. 2019. Rocksmith All Nite: Technology-Mediated Guitar Learning in an Online Affinity Space. Ph.D. Dissertation. The University of Texas at San Antonio, United States – Texas.

[30]

John Edel Tamani, Jan Christian Blaise Cruz, Joshua Raphaelle Cruzada, Jolene Valenzuela, Gray Chan, and Jordan Aiko Deja. 2018. Building Guitar Strum Models for an Interactive Air Guitar Prototype. In Proceedings of the 4th International Conference on Human-Computer Interaction and User Experience in Indonesia. 18–22.

Digital Library

[31]

Atau Tanaka. 2019. Sarah Nicolls performs Suspensions, by Atau Tanaka. https://tinyurl.com/y9ezkr8d [Online; accessed 5th of November, 2022].

[32]

George Tzanetakis and Perry Cook. 2002. Musical genre classification of audio signals. IEEE Transactions on Speech and Audio Processing 10, 5(2002), 293–302.

[33]

Maarten Walraven-Freeling. 2021. The pandemic has changed music education for good. These are the opportunities. https://tinyurl.com/9xmpcxx6 [Online; accessed 5th of November, 2022].

[34]

Alexander Winkler-Schwartz, Vincent Bissonnette, Nykan Mirchi, Nirros Ponnudurai, Recai Yilmaz, Nicole Ledwos, Samaneh Siyar, Hamed Azarnoush, Bekir Karlik, and Rolando F. Del Maestro. 2019. Artificial Intelligence in Medical Education: Best Practices Using Machine Learning to Assess Surgical Expertise in Virtual Reality Simulation. Journal of Surgical Education 76, 6 (2019), 1681–1690.

Cited By

Amm VChandran KEngeln LMcGinity M(2024)Mixed reality strategies for piano educationFrontiers in Virtual Reality10.3389/frvir.2024.13971545Online publication date: 5-Nov-2024
https://doi.org/10.3389/frvir.2024.1397154
Sparrow LGalwey CLoveridge BGlasser SOsborne MKelly R(2024)Heart and Soul: The Ethics of Biometric Capture in Immersive Artistic PerformanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642309(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642309
Al-Sawda JMattar COueida S(2024)The Automated Guitarist: A Programmed-Based Musical RobotProceedings of the Future Technologies Conference (FTC) 2024, Volume 410.1007/978-3-031-73128-0_3(32-44)Online publication date: 6-Nov-2024
https://doi.org/10.1007/978-3-031-73128-0_3
Show More Cited By

Index Terms

Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input

Recommendations

guitAR: supporting guitar learning through mobile projection
CHI EA '11: CHI '11 Extended Abstracts on Human Factors in Computing Systems

The guitar is one of the most widespread instruments amongst autodidacts, but even though a huge amount of learning material exists, it is still hard to learn especially without a guitar teacher. In this paper we propose an Augmented Reality application ...
Guitar man: (an implementation of a rhythm game cooperative musical performance system with actual musical instruments)
SIGGRAPH Asia '08: ACM SIGGRAPH ASIA 2008 educators programme

This study proposes a method for learning musical instruments using computer graphics. The conventional musical instrument learning systems have been considered as simple game controllers or simple score processors. This study produced an education ...
Automatic Classification of Guitar Playing Modes
Sound, Music, and Motion
Abstract
When they improvise, musicians typically alternate between several playing modes on their instruments. Guitarists in particular, alternate between modes such as octave playing, mixed chords and bass, chord comping, solo melodies, walking bass, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

3914 pages

ISBN:9781450394222

DOI:10.1145/3544549

Editors:
Albrecht Schmidt
LMU Munich, Germany
,
Kaisa Väänänen
Tampere University, Finland
,
Tesh Goyal
Google Research, USA
,
Per Ola Kristensson
University of Cambridge, UK
,
Anicia Peters
University of Namibia, Namibia

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Check for updates

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
275
Total Downloads

Downloads (Last 12 months)115
Downloads (Last 6 weeks)7

Reflects downloads up to 23 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Amm VChandran KEngeln LMcGinity M(2024)Mixed reality strategies for piano educationFrontiers in Virtual Reality10.3389/frvir.2024.13971545Online publication date: 5-Nov-2024
https://doi.org/10.3389/frvir.2024.1397154
Sparrow LGalwey CLoveridge BGlasser SOsborne MKelly R(2024)Heart and Soul: The Ethics of Biometric Capture in Immersive Artistic PerformanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642309(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642309
Al-Sawda JMattar COueida S(2024)The Automated Guitarist: A Programmed-Based Musical RobotProceedings of the Future Technologies Conference (FTC) 2024, Volume 410.1007/978-3-031-73128-0_3(32-44)Online publication date: 6-Nov-2024
https://doi.org/10.1007/978-3-031-73128-0_3
Fender AWitzig DMöbus MHolz C(2023)PressurePick: Muscle Tension Estimation for Guitar Players Using Unobtrusive Pressure SensingProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606742(1-11)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606742

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents