More Web Proxy on the site http://driver.im/

research-article

Influence of Electric Taste, Smell, Color, and Thermal Sensory Modalities on the Liking and Mediated Emotions of Virtual Flavor Perception

Authors:

Nimesha Ranasinghe,

Meetha Nesam James,

Michael Gecawicz,

Jonathan Bland,

David SmithAuthors Info & Claims

ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction

Pages 296 - 304

https://doi.org/10.1145/3382507.3418862

Published: 22 October 2020 Publication History

Abstract

Little is known about the influence of various sensory modalities such as taste, smell, color, and thermal, towards perceiving simulated flavor sensations, let alone their influence on people's emotions and liking. Although flavor sensations are essential in our daily experiences and closely associated with our memories and emotions, the concept of flavor and the emotions caused by different sensory modalities are not thoroughly integrated into Virtual and Augmented Reality technologies. Hence, this paper presents 1) an interactive technology to simulate different flavor sensations by overlaying taste (via electrical stimulation on the tongue), smell (via micro air pumps), color (via RGB Lights), and thermal (via Peltier elements) sensations on plain water, and 2) a set of experiments to investigate a) the influence of different sensory modalities on the perception and liking of virtual flavors and b) varying emotions mediated through virtual flavor sensations. Our findings reveal that the participants perceived and liked various stimuli configurations and mostly associated them with positive emotions while highlighting important avenues for future research.

Supplementary Material

ZIP File (icmi1218aux.zip)

Elicited emotions from virtual beverage glass and virtual coffee mug

Download
542.51 KB

MP4 File (3382507.3418862.mp4)

Presentation Video.

Download
11.81 MB

References

[1]

G Ares and SR Jaeger. 2015. Check-all-that-apply (CATA) questions with consumers in practice: experimental considerations and impact on outcome. In Rapid sensory profiling techniques. Elsevier, 227--245.

[2]

Malika Auvray and Charles Spence. 2008. The multisensory perception of flavor. Consciousness and cognition 17, 3 (2008), 1016--1031.

[3]

Jessica Bennett, Geoffrey Greene, and Donna Schwartz-Barcott. 2013. Perceptions of emotional eating behavior. A qualitative study of college students. Appetite 60 (2013), 187--192.

[4]

Natnicha Bhumiratana, Mona Wolf, IV Chambers, Koushik Adhikari, et al. 2019. Coffee Drinking and Emotions: Are There Key Sensory Drivers for Emotions? Beverages 5, 2 (2019), 27.

[5]

S Bonnans and AC Noble. 1993. Effect of sweetener type and of sweetener and acid levels on temporal perception of sweetness, sourness and fruitiness. Chemical senses 18, 3 (1993), 273--283.

[6]

Margaret M Bradley and Peter J Lang. 1994. Measuring emotion: the selfassessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry 25, 1 (1994), 49--59.

[7]

Fabiana M Carvalho, Valentina Moksunova, and Charles Spence. 2020. Cup texture influences taste and tactile judgments in the evaluation of specialty coffee. Food Quality and Preference 81 (2020), 103841.

[8]

Fabiana M Carvalho and Charles Spence. 2018. The shape of the cup influences aroma, taste, and hedonic judgements of specialty coffee. Food quality and preference 68 (2018), 315--321.

[9]

Mabel M Chan and Catherine Kane-Martinelli. 1997. The effect of color on perceived flavor intensity and acceptance of foods by young adults and elderly adults. Journal of the Academy of Nutrition and Dietetics 97, 6 (1997), 657.

[10]

The Right Cup. accessed August 30, 2020. https://www.therightcup.com/

[11]

Jan De Houwer, Sarah Thomas, and Frank Baeyens. 2001. Association learning of likes and dislikes: A review of 25 years of research on human evaluative conditioning. Psychological bulletin 127, 6 (2001), 853.

[12]

Jeannine Delwiche. 2004. The impact of perceptual interactions on perceived flavor. Food Quality and preference 15, 2 (2004), 137--146.

[13]

Lauren Dooley, Young-seung Lee, and Jean-François Meullenet. 2010. The application of check-all-that-apply (CATA) consumer profiling to preference mapping of vanilla ice cream and its comparison to classical external preference mapping. Food quality and preference 21, 4 (2010), 394--401.

[14]

Robin Fondberg, Johan N Lundström, Maria Blöchl, Mats J Olsson, and Janina Seubert. 2018. Multisensory flavor perception: The relationship between congruency, pleasantness, and odor referral to the mouth. Appetite 125 (2018), 244--252.

[15]

Tom Gayler, Corina Sas, and Vaiva Kalnikaite. 2019. Taste Your Emotions: An Exploration of the Relationship Between Taste and Emotional Experience for HCI. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS '19). ACM, New York, NY, USA, 1279--1291. https://doi. org/10.1145/3322276.3322336

Digital Library

[16]

Daniel Harley, Alexander Verni, Mackenzie Willis, Ashley Ng, Lucas Bozzo, and Ali Mazalek. 2018. Sensory vr: Smelling, touching, and eating virtual reality. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction. 386--397.

Digital Library

[17]

Alisa Kanjanakorn and Jeehyun Lee. 2017. Examining emotions and comparing the EsSense Profile® and the Coffee Drinking Experience in coffee drinkers in the natural environment. Food quality and preference 56 (2017), 69--79.

[18]

Silvia C King and Herbert L Meiselman. 2010. Development of a method to measure consumer emotions associated with foods. Food Quality and Preference 21, 2 (2010), 168--177.

[19]

Silvia C King, Herbert L Meiselman, and B Thomas Carr. 2010. Measuring emotions associated with foods in consumer testing. Food Quality and Preference 21, 8 (2010), 1114--1116.

[20]

Silvia C King, Herbert L Meiselman, and B Thomas Carr. 2013. Measuring emotions associated with foods: Important elements of questionnaire and test design. Food Quality and Preference 28, 1 (2013), 8--16.

[21]

I Lackovic and Zoran Stare. 2007. Low-frequency dielectric properties of the oral mucosa. In 13th International Conference on Electrical Bioimpedance and the 8th Conference on Electrical Impedance Tomography. Springer, 154--157.

[22]

Harry T Lawless and Hildegarde Heymann. 2010. Sensory evaluation of food: principles and practices. Springer Science & Business Media.

[23]

Sarah Lefebvre and Marissa Orlowski. 2019. Can, cup, or bottle? The influence of beverage vessel on taste and willingness to pay. International Journal of Hospitality Management 76 (2019), 194--205.

[24]

Juyun Lim and Tomomi Fujimaru. 2010. Evaluation of the labeled hedonic scale under different experimental conditions. Food quality and preference 21, 5 (2010), 521--530.

[25]

Michael Macht. 2008. How emotions affect eating: a five-way model. Appetite 50, 1 (2008), 1--11.

[26]

Michael Macht and Gwenda Simons. 2011. Emotional eating. In Emotion regulation and well-being. Springer, 281--295.

[27]

María Isabel Miranda. 2012. Taste and odor recognition memory: the emotional flavor of life. 23, 5--6 (2012), 481--499.

[28]

Sirous Mobini, Lucy C Chambers, and Martin R Yeomans. 2007. Effects of hunger state on flavour pleasantness conditioning at home: flavour--nutrient learning vs. flavour--flavour learning. Appetite 48, 1 (2007), 20--28.

[29]

Claire Murphy and William S Cain. 1980. Taste and olfaction: independence vs interaction. Physiology & Behavior 24, 3 (1980), 601--605.

[30]

Hiromi Nakamura and Homei Miyashita. 2011. Augmented gustation using electricity. In Proceedings of the 2nd augmented human international conference. ACM, 34.

Digital Library

[31]

Hiromi Nakamura and Homei Miyashita. 2012. Development and evaluation of interactive system for synchronizing electric taste and visual content. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 517--520.

Digital Library

[32]

Takuji Narumi, Munehiko Sato, Tomohiro Tanikawa, and Michitaka Hirose. 2010. Evaluating cross-sensory perception of superimposing virtual color onto real drink: toward realization of pseudo-gustatory displays. In Proceedings of the 1st Augmented Human International Conference. ACM, 18.

Digital Library

[33]

Michael A Nestrud, Herbert L Meiselman, Silvia C King, Larry L Lesher, and Armand V Cardello. 2016. Development of EsSense25, a shorter version of the EsSense Profile®. Food Quality and Preference 48 (2016), 107--117.

[34]

M Ng, C Chaya, and J Hort. 2013. Beyond liking: Comparing the measurement of emotional response using EsSense Profile and consumer defined check-all-thatapply methodologies. Food Quality and Preference 28, 1 (2013), 193--205.

[35]

Corinna Noel and Robin Dando. 2015. The effect of emotional state on taste perception. Appetite 95 (2015), 89--95.

[36]

Maija Paakki, Mari Sandell, and Anu Hopia. 2016. Consumer?s reactions to natural, atypically colored foods: An investigation using blue potatoes. Journal of sensory studies 31, 1 (2016), 78--89.

[37]

John Prescott. 2015. Multisensory processes in flavour perception and their influence on food choice. Current Opinion in Food Science 3 (2015), 47--52.

[38]

Nimesha Ranasinghe, Adrian Cheok, Ryohei Nakatsu, and Ellen Yi-Luen Do. 2013. Simulating the sensation of taste for immersive experiences. In Proceedings of the 2013 ACM international workshop on Immersive media experiences. 29--34.

Digital Library

[39]

Nimesha Ranasinghe and Ellen Yi-Luen Do. 2017. Digital lollipop: Studying electrical stimulation on the human tongue to simulate taste sensations. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 13, 1 (2017), 5.

Digital Library

[40]

Nimesha Ranasinghe, Ryohei Nakatsu, Hideaki Nii, and Ponnampalam Gopalakrishnakone. 2012. Tongue mounted interface for digitally actuating the sense of taste. In 2012 16th International Symposium on Wearable Computers. IEEE, 80--87.

Digital Library

[41]

Nimesha Ranasinghe, Thi Ngoc Tram Nguyen, Yan Liangkun, Lien-Ya Lin, David Tolley, and Ellen Yi-Luen Do. 2017. Vocktail: A virtual cocktail for pairing digital taste, smell, and color sensations. In Proceedings of the 25th ACM international conference on Multimedia. ACM, 1139--1147.

Digital Library

[42]

Nimesha Ranasinghe, David Tolley, Thi Ngoc Tram Nguyen, Liangkun Yan, Barry Chew, and Ellen Yi-Luen Do. 2019. Augmented Flavours: Modulation of Flavour Experiences Through Electric Taste Augmentation. Food Research International 117 (2019), 60--68.

[43]

Vera Sacharin, Katja Schlegel, and Klaus R Scherer. 2012. Geneva emotion wheel rating study. (2012).

[44]

Hendrik NJ Schifferstein. 2001. Effects of product beliefs on product perception and liking. In Food, people and society. Springer, 73--96.

[45]

Dana M Small and Marilyn K Jones-Gotman. 2001. Neural substrates of taste/smell interaction and flavor in the human brain. Chemical Senses 26, 8 (2001), 1034.

[46]

Charles Spence. 2011. Mouth-watering: the influence of environmental and cognitive factors on salivation and gustatory/flavor perception. Journal of Texture Studies 42, 2 (2011), 157--171.

[47]

Charles Spence. 2017. Multisensory flavour perception. Flavour: From Food to Perception (2017), 373--394.

[48]

Charles Spence and Fabiana M Carvalho. 2019. Assessing the influence of the coffee cup on the multisensory tasting experience. Food quality and preference (2019).

[49]

Hannelize van Zyl. 2016. Emotion in Beverages. In Emotion Measurement. Elsevier, 473--499.

[50]

Mark T Wallace. 2015. Multisensory perception: the building of flavor representations. Current Biology 25, 20 (2015), R986--R988.

Cited By

Pham BVu NPhan PLong HLong TPham V(2024)Pectinase‐Driven Optimization of Pectin Hydrolysis for Enhanced Clarity, Anthocyanin Retention, and Consumer Appeal in Red Dragon Fruit Mint Flavored BeverageJournal of Food Processing and Preservation10.1155/jfpp/92734642024:1Online publication date: 20-Nov-2024
https://doi.org/10.1155/jfpp/9273464
Mayumi DNakamura YMatsuda YMisaki SYasumoto K(2024)Aromug: A Mug-Type Olfactory Interface to Enhance the Sweetness Perception of BeveragesIEEE Access10.1109/ACCESS.2024.340139212(78366-78378)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3401392
Fei XWang YLi YLou ZYan YTian YChen Q(2024) OdorCarousel : A Design Tool for Customizing Smell-Enhanced Virtual Experiences International Journal of Human–Computer Interaction10.1080/10447318.2024.2314818(1-16)Online publication date: 20-Feb-2024
https://doi.org/10.1080/10447318.2024.2314818
Show More Cited By

Index Terms

Influence of Electric Taste, Smell, Color, and Thermal Sensory Modalities on the Liking and Mediated Emotions of Virtual Flavor Perception
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
  2. Interaction design
    1. Systems and tools for interaction design

Recommendations

Human perception of a conversational virtual human: an empirical study on the effect of emotion and culture

Virtual reality applications with virtual humans, such as virtual reality exposure therapy, health coaches and negotiation simulators, are developed for different contexts and usually for users from different countries. The emphasis on a virtual human's ...
Effects of Tactile Perception on Emotion and Immersion to Film Viewing in a Virtual Environment
VRST '19: Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology

In this research, we compared three tactile conditions (No vs. Soft vs. Spiky) in both positive and negative scenes to explore whether tactile perception could influence emotional responses and immersive experience in a virtual environment (VE). The ...
Is virtual reality emotionally arousing? Investigating five emotion inducing virtual park scenarios

Following the idea of using virtual environments (VEs) as mood induction procedures (MIPs), this study set out to examine whether five different virtual park scenarios would each elicit a specific affective state (i.e., joy, sadness, boredom, anger and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction

October 2020

920 pages

ISBN:9781450375818

DOI:10.1145/3382507

General Chairs:
Khiet Truong
University of Twente, the Netherlands
,
Dirk Heylen
University of Twente, the Netherlands
,
Mary Czerwinski
Microsoft Research, USA
,
Program Chairs:
Nadia Berthouze
University College London, United Kingdom
,
Mohamed Chetouani
Sorbonne University, France
,
Mikio Nakano
C4A Research Institute, Japan

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICMI '20

Sponsor:

SIGCHI

ICMI '20: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

October 25 - 29, 2020

Virtual Event, Netherlands

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
401
Total Downloads

Downloads (Last 12 months)82
Downloads (Last 6 weeks)15

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pham BVu NPhan PLong HLong TPham V(2024)Pectinase‐Driven Optimization of Pectin Hydrolysis for Enhanced Clarity, Anthocyanin Retention, and Consumer Appeal in Red Dragon Fruit Mint Flavored BeverageJournal of Food Processing and Preservation10.1155/jfpp/92734642024:1Online publication date: 20-Nov-2024
https://doi.org/10.1155/jfpp/9273464
Mayumi DNakamura YMatsuda YMisaki SYasumoto K(2024)Aromug: A Mug-Type Olfactory Interface to Enhance the Sweetness Perception of BeveragesIEEE Access10.1109/ACCESS.2024.340139212(78366-78378)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3401392
Fei XWang YLi YLou ZYan YTian YChen Q(2024) OdorCarousel : A Design Tool for Customizing Smell-Enhanced Virtual Experiences International Journal of Human–Computer Interaction10.1080/10447318.2024.2314818(1-16)Online publication date: 20-Feb-2024
https://doi.org/10.1080/10447318.2024.2314818
Mayumi DNakamura YMatsuda YAicher AYasumoto KMinker W(2023)Design of a Spoken Dialogue System to Provide Food Knowledge to Users while EatingProceedings of the 13th International Conference on the Internet of Things10.1145/3627050.3630728(154-157)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1145/3627050.3630728
Mayumi DNakamura YMatsuda YMisaki SYasumoto K(2023)Kaolid: a Lid-type Olfactory Interface to Present Retronasal Smell towards Beverage Flavor AugmentationProceedings of the 13th International Conference on the Internet of Things10.1145/3627050.3627056(1-8)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1145/3627050.3627056
Bhavadharini BMonica VAnbarasan RMahendran R(2023)Virtual, augmented, and mixed reality as a versatile tool in food consumer behavior evaluation: Recent advances in aroma, taste, and texture incorporationComprehensive Reviews in Food Science and Food Safety10.1111/1541-4337.1324822:6(4925-4956)Online publication date: 13-Oct-2023
https://doi.org/10.1111/1541-4337.13248
Weidner FMaier JBroll W(2023)Eating, Smelling, and Seeing: Investigating Multisensory Integration and (In)congruent Stimuli while Eating in VRIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324709929:5(2423-2433)Online publication date: May-2023
https://doi.org/10.1109/TVCG.2023.3247099
Mayumi DNakamura YMatsuda YUeda KMisaki SYasumoto K(2023)Kaolid: Lid-type Olfactory Interface to Improve Taste of Beverages with Ortho-Retronasal Smell2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops56833.2023.10150412(303-305)Online publication date: 13-Mar-2023
https://doi.org/10.1109/PerComWorkshops56833.2023.10150412
Abilkaiyrkyzy AElhagry ALaamarti FSaddik A(2023)Metaverse Key Requirements and Platforms SurveyIEEE Access10.1109/ACCESS.2023.332584411(117765-117787)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3325844
Ranasinghe N(2023)Virtual taste: digital simulation of taste sensations via electric, thermal, and hybrid stimulationsMultimedia Tools and Applications10.1007/s11042-023-17690-y83:19(56517-56548)Online publication date: 12-Dec-2023
https://doi.org/10.1007/s11042-023-17690-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents