Representing the Timbre of Traditional Musical Instruments Based On Contemporary Instrumental Samples Using DDSP
Article No.: 31, Pages 1 - 3
Abstract
This project explores the potential of Differentiable Digital Signal Processing (DDSP) to represent and synthesize the timbre of five different notes of the Korean traditional musical instrument, Geomungo, using digital instrumental samples of the bass guitar, which has a similar mechanism to produce the sound. To evaluate the feasibility and quality of the digital recreation process, we compared hand-played Geomungo audio samples with digitally recreated audio samples using DDSP. The MFCC, spectral contrast, chroma features, and raw signal comparison, were used for assessment. Our findings show the possibility of applying DDSP to represent and synthesize the nuances of pitch and dynamics for expressive aspects of Geomungo’s five different notes effectively. We also propose three audio features that can be used to evaluate the results quantitatively under the context of neural sound synthesis.
References
[1]
Xi-zheng Cao, Hui-li Meng, and Jiu-cheng Xu. 2009. Timbre model of software musical instrument based on sine interpolation. In 2009 International Conference on Image Analysis and Signal Processing. 358–361. https://doi.org/10.1109/IASP.2009.5054598 ISSN: 2156-0129.
[2]
Hyeshin Chu, Joohee Kim, Seongouk Kim, Hongkyu Lim, Hyunwook Lee, Seungmin Jin, Jongeun Lee, Taehwan Kim, and Sungahn Ko. 2022. An Empirical Study on How People Perceive AI-generated Music. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 304–314.
[3]
Jeremiah D. Deng, Christian Simmermacher, and Stephen Cranefield. 2008. A Study on Feature Analysis for Musical Instrument Classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 38, 2 (April 2008), 429–438. https://doi.org/10.1109/TSMCB.2007.913394 Conference Name: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).
[4]
Jesse Engel, Lamtharn (Hanoi) Hantrakul, Chenjie Gu, and Adam Roberts. 2020. DDSP: Differentiable Digital Signal Processing. In International Conference on Learning Representations. https://openreview.net/forum?id=B1x1ma4tDr
[5]
Center for Arts and Technologies Seoul National University. 2014. Gugak VSTi. http://en.catsnu.com/Main/Main.aspx
[6]
Francesco Ganis, Erik Frej Knudesn, Søren VK Lyster, Robin Otterbein, David Südholt, and Cumhur Erkut. 2021. Real-time timbre transfer and sound synthesis using ddsp. arXiv preprint arXiv:2103.07220 (2021).
[7]
Yubiry Gonzalez and Ronado C. Prati. 2023. Similarity of Musical Timbres Using FFT-Acoustic Descriptor Analysis and Machine Learning. MDPI (Apr 2023). https://doi.org/10.1007/978-3-319-63450-0
[8]
Simmon Holland (Ed.). 2013. Music and Human-Computer Interaction. Vol. 295. Springer-Verlag, New York, NY, 7–8. https://doi.org/10.1007/978-1-4471-2990-5
[9]
Apple Inc. 2023. Garageband for MacOS. https://www.apple.com/mac/garageband/
[10]
Apple Inc. 2023. Logic. https://www.apple.com/logic-pro/
[11]
Mads Græsbøll Christensen Jesper Højvang Jensen and Søren Holdt Jensen. 2007. A framework for analysis of music similarity measures. In 2007 15th European Signal Processing Conference.
[12]
Dan Nin Jian, Lie Lu, Hong Jian Zhang, Jian-Hua Tao, and Lian Hong Cai. 2002. Music type classification by spectral contrast feature. In Proceedings. IEEE International Conference on Multimedia and Expo. https://doi.org/10.1109/ICME.2002.1035731
[13]
Matti Karjalainen and Vesa VŠlimŠki. [n. d.]. Towards High-Quality Sound Synthesis of the Guitar and String Instruments. ([n. d.]).
[14]
Megan L. Lavengood. 2017. A New Approach to the Analysis of Timbre. Ph. D. Dissertation. The City University of New York.
[15]
Sebastian Ewert Meinard Muller and Sebastian Kreuzer. 2009. Making chroma features more robust to timbre changes. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing. https://doi.org/10.1009/978-1-4244-2354-5
[16]
Monica S. Nagawade and Varsha R. Ratnaparkhe. 2017. Musical Instrument Identification using MFCC. In 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT).
[17]
Denis Smalley. 1994. Defining timbre — Refining timbre. Contemporary Music Review 10, 2 (Jan. 1994), 35–48. https://doi.org/10.1080/07494469400640281 Publisher: Routledge _eprint: https://doi.org/10.1080/07494469400640281.
[18]
Mark D. Plumbley Tuomas Virtanen and Dan Ellis. 2017. Computational Analysis of Sound Scenes and Events. Vol. 417. Springer-Verlag, New York, NY. https://doi.org/10.1007/978-3-319-63450-0
Index Terms
- Representing the Timbre of Traditional Musical Instruments Based On Contemporary Instrumental Samples Using DDSP
Recommendations
An Effective Feature Calculation For Analysis & Classification of Indian Musical Instruments Using Timbre Measurement
ICCCT '15: Proceedings of the Sixth International Conference on Computer and Communication Technology 2015Musical instrument recognition is significant field in the research of computer music which is related to the modelling of sounds. Analysing & synthesing the structure of musical note is of importance both for modelling music signals and their automatic ...
MITT: Musical Instrument Timbre Transfer Based on the Multichannel Attention-Guided Mechanism
Intelligent Computing Theories and ApplicationAbstractResearch on neural style transfer and domain translation has clearly demonstrated the ability of deep learning algorithms to manipulate images based on their artistic style. The idea of image translation has been applied to the task of music-style ...
The influence of polyphony on the dynamical modelling of musical timbre
This letter addresses the problem of pattern recognition of polyphonic musical timbre. Frame-level dynamics of audio features are particularly difficult to model, although they have been identified as crucial perceptive dimensions of timbre perception. ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2023
424 pages
ISBN:9798400700965
DOI:10.1145/3586182
Copyright © 2023 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 October 2023
Check for updates
Author Tags
Qualifiers
- Poster
- Research
- Refereed limited
Conference
UIST '23: The 36th Annual ACM Symposium on User Interface Software and Technology
October 29 - November 1, 2023
CA, San Francisco, USA
Acceptance Rates
Overall Acceptance Rate 355 of 1,733 submissions, 20%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 56Total Downloads
- Downloads (Last 12 months)35
- Downloads (Last 6 weeks)4
Reflects downloads up to 12 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format