More Web Proxy on the site http://driver.im/

research-article

Sonification of Surface Tapping Changes Behavior, Surface Perception, and Emotion

Authors:

Ana Tajadura-Jimenez,

Nadia Bianchi-Berthouze,

Enrico Furfaro,

Frederic BevilacquaAuthors Info & Claims

IEEE MultiMedia, Volume 22, Issue 1

Pages 48 - 57

https://doi.org/10.1109/MMUL.2015.14

Published: 01 January 2015 Publication History

Abstract

The audio feedback resulting from object interaction provides information about the material of the surface and about one's own motor behavior. With the current developments in interactive sonification, it's now possible to digitally change this audio feedback, making the use of interactive sonification a compelling approach to shape tactile surface interactions. Here, the authors present a prototype for a sonic interactive surface, capable of delivering surface tapping sounds in real time when triggered by users' taps on a real surface or on an imagined "virtual" surface. In this system, the delivered audio feedback can be varied so that the tapping sounds correspond to different applied strengths during tapping. The authors also propose a multidimensional measurement approach to evaluate user experiences of multimodal interactive systems. They evaluated their system by looking at the effect of the altered tapping sounds on emotional action-related responses, the users' interactions with the surface, and perceived surface hardness. Results show the influence of the sonification of tapping at all levels: emotional, behavioral, and perceptual. These results have implications on the design of interactive sonification displays and tangible auditory interfaces aiming to change perceived and subsequent motor behavior as well as perceived material properties.

References

[1]

R.L. Klatzky, and S.J. Lederman, “Multisensory Texture Perception,” Multisensory Object Perception in the Primate Brain, J. Kaiser and M.J. Naumer, eds., Springer, 2010, pp. 211–230.

[2]

Y. Gao, N. Bianchi-Berthouze, and H. Meng, “What Does Touch Tell Us about Emotions in Touchscreen-Based Gameplay?” ACM Trans. Computer-Human Interaction, vol. 19, no. 4, 2012, article no. 31.

[3]

N. Bianchi-Berthouze, “Understanding the Role of Body Movement in Player Engagement,” Human Computer Interaction, vol. 28, no. 1, 2012, pp. 42–75.

[4]

J. Peck and J. Wiggins, “It Just Feels Good: Customers' Affective Response to Touch and Its Influence on Persuasion,” J. Marketing, vol. 70, no. 4, 2006, pp. 56–69.

[5]

M.S.H. Aung etal., “Getting Rid of Pain-Related Behaviour to Improve Social and Self Perception: A Technology-Based Perspective,” Proc. IEEE Int'l Workshop Image and Audio Analysis for Multimedia Interactive Services (WIAMIS), 2013, pp. 1–4.

[6]

T. Hermann, A. Hunt, and J.G. Neuhoff, The Sonification Handbook, Logos Publishing House, 2011.

[7]

D. Rocchesso, and F. Fontana, eds., The Sounding Object, Mondo Estremo, 2003.

[8]

D. Merrill, and H. Raffle, “The Sound of Touch,” Proc. ACM Conf. Human Factors in Computing Systems, 2007, pp. 2807–2812.

[9]

M.R. McGee, P.D. Gray, and S.A. Brewster, “Mixed Feelings: Multimodal Perception of Virtual Roughness,” Proc. Eurohaptics, 2002, pp. 47–52.

[10]

U. Castiello etal., “When Ears Drive Hands: The Influence of Contact Sound on Reaching to Grasp,” PLoS One, 2010; www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0012240.

[11]

N. Schaffert, K. Mattes, and A.O. Effenberg, “Listen to the Boat Motion: Acoustic Information for Elite Rowers,” Proc. Interaction Sonification Workshop (ISon), 2010; http://interactive-sonification.org/ISon2010/proceedings/papers/SchaffertMatteEffenberg_ISon2010.pdf.

[12]

A. Tajadura-Jiménez etal., “Action Sounds Recalibrate Perceived Tactile Distance,” Current Biology, vol. 22, no. 13, 2012, pp. R516–R517.

[13]

F. Menzer etal., “Feeling in Control of Your Footsteps: Conscious Gait Monitoring and the Auditory Consequences of Footsteps,” Cognitive Neuroscience, vol. 1, no. 3, 2010, pp. 184–192.

[14]

R. Bresin etal., “Expressive Sonification of Footstep Sounds,” Proc. Interaction Sonification Workshop (ISon), 2010; http://interactive-sonification.org/ISon2010/proceedings/papers/BresindeWittPapettiCivolaniFontana_ISon2010.pdf.

[15]

W. Boucsein, Electrodermal Activity, Plenum Press, 1992.

[16]

J. Sauro and J.S. Dumas, “Comparison of Three One-Question, Post-Task Usability Questionnaires,” Proc. ACM Conf. Human Factors in Computing Systems, 2009, pp. 1599–1608.

[17]

M. Bradley and P.J. Lang, “Measuring Emotion: The Self-Assessment Manikin and the Semantic Differential,” J. Behavior Therapy and Experimental Psychiatry, vol. 25, 1994, pp. 49–59.

[18]

R. Tuennermann, J. Hammerschmidt, and T. Hermann, “Blended Sonification: Sonification for Casual Information Interaction,” Proc. Int'l Conf. Auditory Display, 2013; https://smartech.gatech.edu/handle/1853/51656.

Cited By

Singh AHrobat MGui SBianchi-Berthouze NLey-Flores JBevilacqua FDuran JSegura ETajadura-JimÉnez A(2024)Pushed by Sound: Effects of Sound and Movement Direction on Body Perception, Experience Quality, and Exercise SupportACM Transactions on Computer-Human Interaction10.1145/364861631:4(1-36)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3648616
Ley-Flores JVidal LSegura ESingh ABevilacqua FCuadrado FDurán JValdiviezo-Hernández OSánchez-Martin MTajadura-Jiménez A(2024)Co-Designing Sensory Feedback for Wearables to Support Physical Activity through Body SensationsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36434998:1(1-31)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643499
Turmo Vidal LTajadura-Jiménez AVega-Cebrián JLey-Flores JDíaz-Durán JMárquez Segura ECiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)Body Transformation: An Experiential Quality of Sensory Feedback Wearables for Altering Body PerceptionProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633373(1-19)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3623509.3633373
Show More Cited By

Index Terms

Sonification of Surface Tapping Changes Behavior, Surface Perception, and Emotion
1. Hardware
  1. Communication hardware, interfaces and storage
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices
    2. Interaction paradigms

Index terms have been assigned to the content through auto-classification.

Recommendations

Surface interactions for interactive sonification
CMMR/ICAD'09: Proceedings of the 6th international conference on Auditory Display

This paper presents novel interaction modes for Model-Based Sonification (MBS) via interactive surfaces. We first discuss possible interactions for MBS on a multi-touch surface. This is followed by a description of the Data Sonogram Sonification and the ...
Cross-Surface: Workshop on Interacting with Multi-Device Ecologies in the Wild
ITS '15: Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces

In this workshop, we will review and discuss opportunities, technical challenges and problems with cross-device interactions in interactive multi-surface and multi-device ecologies. We aim to bring together researchers and practitioners currently ...
Surface Parameterization Using Riemann Surface Structure
ICCV '05: Proceedings of the Tenth IEEE International Conference on Computer Vision - Volume 2

We propose a general method that parameterizes general surfaces with complex (possible branching) topology using Riemann surface structure. Rather than evolve the surface geometry to a plane or sphere, we instead use the fact that all orientable ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE MultiMedia

IEEE MultiMedia Volume 22, Issue 1

Jan.-Mar. 2015

93 pages

ISSN:1070-986X

Issue’s Table of Contents

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 January 2015

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Singh AHrobat MGui SBianchi-Berthouze NLey-Flores JBevilacqua FDuran JSegura ETajadura-JimÉnez A(2024)Pushed by Sound: Effects of Sound and Movement Direction on Body Perception, Experience Quality, and Exercise SupportACM Transactions on Computer-Human Interaction10.1145/364861631:4(1-36)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3648616
Ley-Flores JVidal LSegura ESingh ABevilacqua FCuadrado FDurán JValdiviezo-Hernández OSánchez-Martin MTajadura-Jiménez A(2024)Co-Designing Sensory Feedback for Wearables to Support Physical Activity through Body SensationsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36434998:1(1-31)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643499
Turmo Vidal LTajadura-Jiménez AVega-Cebrián JLey-Flores JDíaz-Durán JMárquez Segura ECiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)Body Transformation: An Experiential Quality of Sensory Feedback Wearables for Altering Body PerceptionProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633373(1-19)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3623509.3633373
Montano-Murillo RPittera DFrier WGeorgiou OObrist MCornelio P(2024)It Sounds Cool: Exploring Sonification of Mid-Air Haptic Textures Exploration on Texture Judgments, Body Perception, and Motor BehaviourIEEE Transactions on Haptics10.1109/TOH.2023.332049217:2(237-248)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TOH.2023.3320492
Kantan PDahl SSerafin SSpaich E(2024)Sonifying gait kinematics: generating water wading sounds through a digital Foley approachPersonal and Ubiquitous Computing10.1007/s00779-024-01829-128:5(823-843)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s00779-024-01829-1
Tajadura-Jimenez ALey-Flores JValdiviezo OSingh ASanchez-Martin MDiaz Duran JMárquez Segura E(2022)Exploring the Design Space for Body Transformation Wearables to Support Physical Activity through Sensitizing and BodystormingProceedings of the 8th International Conference on Movement and Computing10.1145/3537972.3538001(1-9)Online publication date: 22-Jun-2022
https://dl.acm.org/doi/10.1145/3537972.3538001
Liu WMagalhaes MMackay WBeaudouin-Lafon MBevilacqua F(2022)Motor Variability in Complex Gesture Learning: Effects of Movement Sonification and Musical BackgroundACM Transactions on Applied Perception10.1145/348296719:1(1-21)Online publication date: 6-Jan-2022
https://dl.acm.org/doi/10.1145/3482967
Toffa OMignotte M(2022)Dataset and semantic based-approach for image sonificationMultimedia Tools and Applications10.1007/s11042-022-12914-z82:1(1505-1518)Online publication date: 3-May-2022
https://dl.acm.org/doi/10.1007/s11042-022-12914-z
Liu WDementyev ASchwarz DFlety EMackay WBeaudouin-Lafon MBevilacqua FKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)SonicHoop: Using Interactive Sonification to Support Aerial Hoop PracticesProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445539(1-16)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445539
Tajadura-Jiménez ANewbold JZhang LRick PBianchi-Berthouze NBrewster SFitzpatrick GCox AKostakos V(2019)As Light as You Aspire to BeProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300888(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300888
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents