Timbre Tools: Ethnographic Perspectives on Timbre and Sonic Cultures in Hackathon Designs
Authors: 
Charalampos Saitis, Bleiz M Del Sette, Jordan Shier, Haokun Tian, 
Shuoyang Zheng, Sophie Skach, 
Courtney N. Reed, 
Corey FordAuthors Info & Claims
Pages 229 - 244
Abstract
Timbre is a nuanced yet abstractly defined concept. Its inherently subjective qualities make it challenging to design and work with. In this paper, we propose to explore the conceptualisation and negotiation of timbre within the creative practice of timbre tool makers. To this end, we hosted a hackathon event and performed an ethnographic study to explore how participants engaged with the notion of timbre and how their conception of timbre was shaped through social interactions and technological encounters. We present individual descriptions of each team’s design process and reflect on our data to identify commonalities in the ways that timbre is understood and informed by sound technologies and their surrounding sonic cultures, e.g., by relating concepts of timbre to metaphors. We further current understanding by offering novel interdisciplinary and multimodal insights into understandings of timbre.
References
[1]
Saul Albert, Claude Heath, Sophie Skach, Matthew Tobias Harris, Madeline Miller, and Patrick GT Healey. 2019. Drawing as transcription: how do graphical techniques inform interaction analysis?Social Interaction Video-Based Studies of Human Sociality (2019).
[2]
Jack Armitage, Victor Shepardson, and Thor Magnusson. 2024. Tölvera: Composing With Basal Agencies. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Utrecht, NL.
[3]
Jean-Julien Aucouturier and Emmanuel Bigand. 2012. Mel Cepstrum & Ann Ova: The Difficult Dialog Between MIR and Music Cognition. In Proc. International Society for Music Information Retrieval (ISMIR) Conference. 397–402.
[4]
Eliot Bates. 2021. The interface and instrumentality of Eurorack modular synthesis. In Rethinking Music through Science and Technology Studies, Antoine Hennion and Christophe Levaux (Eds.). Routledge, 170–188.
[5]
Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health 11, 4 (2019), 589–597. https://doi.org/10.1080/2159676X.2019.1628806
[6]
Ragnhild Brøvig and Anne Danielsen. 2016. Autotuned Voices: Alienation and “Brokenhearted Androids”. In Digital Signatures: The Impact of Digitization on Popular Music Sound, Ragnhild Brøvig and Anne Danielsen (Eds.). MIT Press, Chapter 7, 117–132.
[7]
Marcelo Caetano, Charalampos Saitis, and Kai Siedenburg. 2019. Audio Content Descriptors of Timbre. Springer International Publishing, Cham, 297–333. https://doi.org/10.1007/978-3-030-14832-4_11
[8]
Antoine Caillon and Philippe Esling. 2021. RAVE: A Variational Autoencoder for Fast and High-Quality Neural Audio Synthesis. arxiv:2111.05011 [cs, eess] http://arxiv.org/abs/2111.05011
[9]
Clive Cazeaux. 2002. Metaphor and the Categorization of the Senses. Metaphor and Symbol 17, 1 (Jan. 2002), 3–26. https://doi.org/10.1207/s15327868ms1701_2
[10]
Alan Chamberlain, Andy Crabtree, Tom Rodden, Matt Jones, and Yvonne Rogers. 2012. Research in the Wild: Understanding ’in the wild’ Approaches to Design and Development. In Proceedings of the Designing Interactive Systems Conference (Newcastle Upon Tyne, United Kingdom) (DIS ’12). Association for Computing Machinery, New York, NY, USA, 795–796. https://doi.org/10.1145/2317956.2318078
[11]
Nuno N. Correia and Atau Tanaka. 2015. Prototyping Audiovisual Performance Tools: A Hackathon Approach. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME), Edgar Berdahl and Jesse Allison (Eds.). Louisiana State University, Baton Rouge, Louisiana, USA, 319–321. https://doi.org/10.5281/zenodo.1179044
[12]
Cycling ’74. 2024. Max [software]. https://cycling74.com/products/max
[13]
Thomas Deacon, Nick Bryan-Kinns, Patrick G.T. Healey, and Mathieu Barthet. 2019. Shaping Sounds: The Role of Gesture in Collaborative Spatial Music Composition. In Proceedings of the 2019 Conference on Creativity and Cognition (San Diego, CA, USA) (C&C ’19). Association for Computing Machinery, New York, NY, USA, 121–132. https://doi.org/10.1145/3325480.3325493
[14]
Emily I. Dolan. 2013. The Orchestral Revolution: Haydn and the Technologies of Timbre. Cambridge University Press.
[15]
Emily I. Dolan and Thomas Patteson. 2018. Ethereal Timbres. In The Oxford Handbook of Timbre, Emily I. Dolan and Alexander Rehding (Eds.). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190637224.013.27
[16]
Emily I. Dolan and Alexander Rehding (Eds.). 2021. The Oxford Handbook of Timbre. Oxford University Press.
[17]
Andrew Dubber. 2016. You Have 24 Hours to Invent the Future of Music: Music Hacks, Playful Research and Creative Innovation. In Business Innovation and Disruption in the Music Industry. Edward Elgar Publishing, 211–228.
[18]
Nina Sun Eidsheim. 2019. The race of sound: Listening, timbre, and vocality in African American music. Duke University Press.
[19]
Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. 2023. CLAP Learning Audio Concepts from Natural Language Supervision. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1–5. https://doi.org/10.1109/ICASSP49357.2023.10095889
[20]
Cornelia Fales. 2018. Hearing Timbre: Perceptual Learning among Early Bay Area Ravers. In The Relentless Pursuit of Tone: Timbre in Popular Music. Oxford University Press. https://doi.org/10.1093/oso/9780199985227.003.0002
[21]
Jeanette Falk Olesen and Kim Halskov. 2020. 10 Years of Research With and On Hackathons. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, Netherlands) (DIS ’20). Association for Computing Machinery, New York, NY, USA, 1073–1088. https://doi.org/10.1145/3357236.3395543
[22]
Stefano Fasciani. 2020. Interactive Computation of Timbre Spaces for Sound Synthesis Control. (2020), 69–78. https://doi.org/10.25370/array.v20152528
[23]
Robert Fink, Melinda Latour, and Zachary Wallmark (Eds.). 2018. The relentless pursuit of tone: Timbre in popular music. Oxford University Press.
[24]
Meagan Flus and Ada Hurst. 2021. Design at hackathons: new opportunities for design research. Design Science 7 (2021), e4.
[25]
Corey Ford, Ashley Noel-Hirst, Sara Cardinale, Jackson Loth, Pedro Sarmento, Elizabeth Wilson, Lewis Wolstanholme, Kyle Worrall, and Nick Bryan-Kinns. 2024. Reflection Across AI-based Music Composition. In Proceedings of the 2024 ACM Conference on Creativity and Cognition (C&C ’24) (Chicago, IL, USA). ACM, New York, NY, USA, 15 pages. https://doi.org/10.1145/3635636.3656185
[26]
Arthur W Frank. 2010. Letting stories breathe: A socio-narratology. University of Chicago Press.
[27]
Kiev Gama, George Valença, Pedro Alessio, Rafael Formiga, André Neves, and Nycolas Lacerda. 2023. The developers’ design thinking toolbox in hackathons: a study on the recurring design methods in software development marathons. International Journal of Human–Computer Interaction 39, 12 (2023), 2269–2291.
[28]
Dana Gooley. 2013. Jazz piano pedaling and the production of timbral difference. Keyboard Perspectives 6 (2013), 101––126.
[29]
Paul D Greene and Thomas Porcello (Eds.). 2005. Wired for sound: Engineering and technologies in sonic cultures. Wesleyan University Press.
[30]
John Michael Grey. 1975. An Exploration of Musical Timbre. Ph. D. Dissertation. Stanford University.
[31]
James Hodge, Sarah Foley, Dan Lambton-Howard, Laura Booi, Kyle Montague, Sandra Coulter, David Kirk, and Kellie Morrissey. 2023. Exploring Participants’ Representations and Shifting Sensitivities in a Hackathon for Dementia. ACM Trans. Comput.-Hum. Interact. 30, 3, Article 46 (jun 2023), 35 pages. https://doi.org/10.1145/3571814
[32]
Cheng-Zhi Anna Huang, Hendrik Vincent Koops, Ed Newton-Rex, Monica Dinculescu, and Carrie J. Cai. 2020. AI Song Contest: Human-AI Co-Creation in Songwriting. In Proc. International Society for Music Information Retrieval (ISMIR) Conference. Montréal, Canada. https://arxiv.org/pdf/2010.05388.pdf
[33]
Don Ihde. 2021. A Finnish turn: digital and synthesiser musical instruments. Journal of New Music Research 50, 2 (2021), 165–174.
[34]
Alexander Refsum Jensenius. 2022. Sound actions: Conceptualizing musical instruments. MIT Press.
[35]
Pirkko Juntunen, Inkeri Ruokonen, and Heikki Ruismäki. 2014. The impact of playback in learning musicianship skills and musical communication. The changing face of music and art education : Education Interdisciplinary Journal for Music and Art Pedagogy - CFMAE journal 6, 1 (2014), 49–68.
[36]
Joakim Karlsen and Anders Sundnes Løvlie. 2017. ‘You can dance your prototype if you like’: independent filmmakers adapting the hackathon. Digital Creativity 28, 3 (2017), 224–239. https://doi.org/10.1080/14626268.2017.1351992
[37]
Mark Katz. 2010. Capturing Sound: How Technology has Changed Music. University of California Press, Berkeley and Los Angeles, CA.
[38]
Adam Kendon. 1990. Conducting interaction: Patterns of behavior in focused encounters. Vol. 7. CUP Archive.
[39]
George Lakoff and Mark Johnson. 2008. Metaphors we live by. University of Chicago Press.
[40]
George Lakoff, Mark Johnson, and John F Sowa. 1999. Review of Philosophy in the Flesh: The embodied mind and its challenge to Western thought. Computational Linguistics 25, 4 (1999), 631–634.
[41]
Megan Lavengood. 2019. What makes it sound’80s? The Yamaha DX7 electric piano sound. Journal of Popular Music Studies 31, 3 (2019), 73–94.
[42]
Tilmann Lindberg, Christoph Meinel, and Ralf Wagner. 2011. Design thinking: A fruitful concept for IT development?Design thinking: Understand–improve–apply (2011), 3–18.
[43]
Thomas J Lodato and Carl DiSalvo. 2016. Issue-oriented hackathons as material participation. New Media & Society 18, 4 (2016), 539–557. https://doi.org/10.1177/1461444816629467
[44]
Thor Magnusson. 2019. Sonic Writing Technologies of Material, Symbolic, and Signal Inscriptions. Bloomsbury Academic, New York, NY.
[45]
Juan P Martinez Avila, Vasiliki Tsaknaki, Pavel Karpashevich, Charles Windlin, Niklas Valenti, Kristina Höök, Andrew McPherson, and Steve Benford. 2020. Soma Design for NIME. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME), Romain Michon and Franziska Schroeder (Eds.). Birmingham City University, Birmingham, UK, 489–494. https://doi.org/10.5281/zenodo.4813491
[46]
Stephen McAdams. 2019. The Perceptual Representation of Timbre. In Timbre: Acoustics, Perception, and Cognition. Springer International Publishing, 23–57.
[47]
S. McAdams, Suzanne Winsberg, Sophie Donnadieu, Geert De Soete, and Jochen Krimphoff. 1995. Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes. Psychological Research 58, 3 (1995), 177–192.
[48]
Andrew McPherson and Victor Zappi. 2015. An environment for submillisecond-latency audio and sensor processing on BeagleBone Black. In Audio Engineering Society Convention 138. Audio Engineering Society.
[49]
Albert Mehrabian. 1969. Significance of posture and position in the communication of attitude and status relationships.Psychological Bulletin 71, 5 (1969), 359.
[50]
Alessandro B. Melchiorre and Markus Schedl. 2020. Personality Correlates of Music Audio Preferences for Modelling Music Listeners. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (Genoa, Italy) (UMAP ’20). Association for Computing Machinery, New York, NY, USA, 313–317. https://doi.org/10.1145/3340631.3394874
[51]
David R. Millen. 2000. Rapid Ethnography: Time Deepening Strategies for HCI Field Research. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (New York City, New York, USA) (DIS ’00). Association for Computing Machinery, New York, NY, USA, 280–286. https://doi.org/10.1145/347642.347763
[52]
Landon Morrison. 2022. Hearing Timbre at the Crossroads. Kalfou 9, 2 (Fall 2022), 328–336. https://www.proquest.com/scholarly-journals/hearing-timbre-at-crossroads/docview/2763578225/se-2
[53]
Chris Nash and Alan Blackwell. 2012. Liveness and Flow in Notation Use. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). University of Michigan, Ann Arbor, Michigan. https://doi.org/10.5281/zenodo.1180547
[54]
Alexander Nolte, Ei Pa Pa Pe-Than, Abasi-amefon Obot Affia, Chalalai Chaihirunkarn, Anna Filippova, Arun Kalyanasundaram, Maria Angelica Medina Angarita, Erik Trainer, and James D Herbsleb. 2020. How to organize a hackathon–A planning kit. arXiv preprint arXiv:2008.08025 (2020).
[55]
Charlotte Nordmoen and Andrew P. McPherson. 2022. Making space for material entanglements: A diffractive analysis of woodwork and the practice of making an interactive system. In Proceedings of the 2022 ACM Designing Interactive Systems Conference (Virtual Event, Australia) (DIS ’22). Association for Computing Machinery, New York, NY, USA, 415–423. https://doi.org/10.1145/3532106.3533572
[56]
Jeanette Falk Olesen, Nicolai Brodersen Hansen, and Kim Halskov. 2018. Four factors informing design judgement at a hackathon. In Proceedings of the 30th Australian Conference on Computer-Human Interaction (Melbourne, Australia) (OzCHI ’18). Association for Computing Machinery, New York, NY, USA, 473–483. https://doi.org/10.1145/3292147.3292155
[57]
OpenAI. 2022. ChatGPT: Optimizing Language Models for Dialogue. https://openai.com/blog/chatgpt/ (Online).
[58]
Ei Pa Pa Pe-Than, Alexander Nolte, Anna Filippova, Christian Bird, Steve Scallen, and James Herbsleb. 2022. Corporate hackathons, how and why? A multiple case study of motivation, projects proposal and selection, goal setting, coordination, and outcomes. Human–Computer Interaction 37, 4 (2022), 281–313.
[59]
Jon Pigott and Aidan Taylor. 2024. Sonic cultures of making: DIY sound and electronics since 1981. Sound Studies (2024), 1–19. https://doi.org/10.1080/20551940.2024.2304439
[60]
Trevor Pinch and Frank Trocco. 2004. Analog Days: The Invention and Impact of the Moog Synthesizer. Harvard University Press.
[61]
Jenny Preece, Yvonne Rogers, Helen Sharp, David Benyon, Simon Holland, and Tom Carey. 1994. Human-computer interaction. Addison-Wesley Longman Ltd.
[62]
Catherine Provenzano. 2018. Auto-Tune, Labor, and the Pop-Music Voice. In The relentless pursuit of tone: Timbre in popular music, Robert Fink, Melinda Latour, and Zachary Wallmark (Eds.). Oxford University Press, Chapter 7, 159–181.
[63]
Courtney N. Reed, Paul Strohmeier, and Andrew P. McPherson. 2023. Negotiating Experience and Communicating Information Through Abstract Metaphor. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 185, 16 pages. https://doi.org/10.1145/3544548.3580700
[64]
Charalampos Saitis, Claudia Fritz, Gary P. Scavone, Catherine Guastavino, and Danièle Dubois. 2017. Perceptual evaluation of violins: A psycholinguistic analysis of preference verbal descriptions by experienced musicians. Journal of the Acoustic Society of America 141, 4 (2017), 2746–2757.
[65]
Charalampos Saitis, Hanna Järveläinen, and Claudia Fritz. 2018. The Role of Haptic Cues in Musical Instrument Quality Perception. In Musical Haptics, Stefano Papetti and Charalampos Saitis (Eds.). Springer, Cham, 73–93.
[66]
Charalampos Saitis and Stefan Weinzierl. 2019. The Semantics of Timbre. In Timbre: Acoustics, Perception, and Cognition, Kai Siedenburg, Charalampos Saitis, Stephen McAdams, Arthur N. Popper, and Richard R. Fay (Eds.). Springer, Cham, 119–149.
[67]
Jeffrey Sanchez-Burks, Sally Blount, and Caroline A. Bartel. 2006. Fluidity and Performance in Intercultural Workplace Interactions: The Role of Behavioral Mirroring and Relational Attunement. Ross School of Business Paper 1039 (Sept. 2006), 56. https://doi.org/10.2139/ssrn.928584
[68]
Elizabeth B.-N. Sanders and Pieter Jan Stappers. 2008. Co-creation and the new landscapes of design. CoDesign 4, 1 (2008), 5–18.
[69]
Cleo Schulten, Alexander Nolte, Daniel Spikol, and Irene-Angelica Chounta. 2022. How do participants collaborate during an online hackathon? An empirical, quantitative study of communication traces. Frontiers in Computer Science 4 (2022). https://doi.org/10.3389/fcomp.2022.983164
[70]
Jordie Shier, Charalampos Saitis, Andrew Robertson, and Andrew McPherson. 2024. Real-time timbre remapping with differentiable DSP. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Utrecht, NL.
[71]
Kai Siedenburg, Ichiro Fujinaga, and Stephen McAdams. 2016. A comparison of approaches to timbre descriptors in music information retrieval and music psychology. Journal of New Music Research 45, 1 (2016), 27–41.
[72]
Kai Siedenburg and Stephen McAdams. 2017. Four distinctions for the auditory “wastebasket” of timbre. Frontiers in Psychology 8 (2017), 272043.
[73]
Kai Siedenburg, Charalampos Saitis, Stephen McAdams, Arthur N. Popper, and Richard R. Fay (Eds.). 2019. Timbre: Acoustics, Perception, and Cognition. Springer, Cham.
[74]
Joe Snape and Georgina Born. 2022. Max, music software and the mutual mediation of aesthetics and digital technologies. In Music and Digital Media: A Planetary Anthropology, Georgina Born (Ed.). UCL Press, London, 220–266. http://www.jstor.org/stable/j.ctv2pzbkcg.11
[75]
Makis Solomos. 2020. From Music to Sound: The Emergence of Sound in 20th-and 21st-Century Music. Routledge, Abingdon.
[76]
Andrew Sparkes and Brett Smith. 2008. Narrative Constructionist Inquiry. In Handbook of Constructionist Research, James Holstein and Jaber Gubrium (Eds.). Guilford Publications, London, 295–314.
[77]
Miriam Sturdee, Makayla Lewis, Angelika Strohmayer, Katta Spiel, Nantia Koulidou, Sarah Fdili Alaoui, and Josh Urban Davis. 2021. A Plurality of Practices: Artistic Narratives in HCI Research. In Proceedings of the 13th Conference on Creativity and Cognition (Virtual Event, Italy) (C&C ’21). Association for Computing Machinery, New York, NY, USA, Article 35, 14 pages. https://doi.org/10.1145/3450741.3466771
[78]
Bob L. Sturm, Oded Ben-Tal, Úna Monaghan, Nick Collins, Dorien Herremans, Elaine Chew, Gaëtan Hadjeres, Emmanuel Deruty, and François Pachet. 2019. Machine Learning Research That Matters for Music Creation: A Case Study. Journal of New Music Research 48, 1 (2019), 36–55. https://doi.org/10.1080/09298215.2018.1515233
[79]
Paul Théberge. 1997. Any Sound You Can Imagine: Making Music/Consuming Technology. Wesleyan University Press, Hanover, NH.
[80]
Daniel Villegas Vélez. 2018. The Matter of Timbre: Listening, Genealogy, Sound. In The Oxford Handbook of Timbre, Emily I. Dolan and Alexander Rehding (Eds.). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190637224.013.20
[81]
Ron Wakkary, Doenja Oogjes, Nazmus Sakib, and Armi Behzad. 2023. Turner Boxes and Bees: From Ambivalence to Diffraction. In Proceedings of the 2023 ACM Designing Interactive Systems Conference (Pittsburgh, PA, USA) (DIS ’23). Association for Computing Machinery, New York, NY, USA, 790–807. https://doi.org/10.1145/3563657.3596081
[82]
Zachary Wallmark. 2022. Nothing but Noise: Timbre and Musical Meaning at the Edge. Oxford University Press.
[83]
Bodo Winter. 2019. Sensory Linguistics: Language, perception and metaphor. John Benjamins.
[84]
Asterios Zacharakis, Konstantinos Pastiadis, and Joshua D. Reiss. 2014. An Interlanguage Study of Musical Timbre Semantic Dimensions and Their Acoustic Correlates. Music Perception 31 (2014), 339–358.
[85]
Victor Zappi and Andrew McPherson. 2018. Hackable Instruments: Supporting Appropriation and Modification in Digital Musical Interaction. Frontiers in ICT 5 (2018). https://doi.org/10.3389/fict.2018.00026
Index Terms
- Timbre Tools: Ethnographic Perspectives on Timbre and Sonic Cultures in Hackathon Designs
Index terms have been assigned to the content through auto-classification.
Recommendations
An Effective Feature Calculation For Analysis & Classification of Indian Musical Instruments Using Timbre Measurement
ICCCT '15: Proceedings of the Sixth International Conference on Computer and Communication Technology 2015Musical instrument recognition is significant field in the research of computer music which is related to the modelling of sounds. Analysing & synthesing the structure of musical note is of importance both for modelling music signals and their automatic ...
Representing the Timbre of Traditional Musical Instruments Based On Contemporary Instrumental Samples Using DDSP
UIST '23 Adjunct: Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and TechnologyThis project explores the potential of Differentiable Digital Signal Processing (DDSP) to represent and synthesize the timbre of five different notes of the Korean traditional musical instrument, Geomungo, using digital instrumental samples of the bass ...
Exploring Perceptual Based Timbre Feature for Singer Identification
Computer Music Modeling and Retrieval. Sense of SoundsTimbre can be defined as feature of an auditory stimulus that allows us to distinguish the sounds which have the same pitch and loudness. In this paper, we explore timbre based perceptual feature for singer identification. We start with a vocal ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2024
565 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 September 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
AM '24
Acceptance Rates
Overall Acceptance Rate 177 of 275 submissions, 64%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 162Total Downloads
- Downloads (Last 12 months)162
- Downloads (Last 6 weeks)53
Reflects downloads up to 01 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in