More Web Proxy on the site http://driver.im/

research-article

Reading with the Tongue: Individual Differences Affect the Perception of Ambiguous Stimuli with the BrainPort

Authors:

Mike L. Richardson,

Tayfun Lloyd-Esenkaya,

Michael J. ProulxAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 10

https://doi.org/10.1145/3313831.3376184

Published: 23 April 2020 Publication History

Abstract

There is an increasing interest in non-visual interfaces for HCI to take advantage of the information processing capability of the other sensory modalities. The BrainPort is a vision-to-tactile sensory substitution device that conveys information through electro-stimulation on the tongue. As the tongue is a horizontal surface, it makes for an interesting platform to study the brain's representation of space. But which way is up on the tongue? We provided participants with perceptually ambiguous stimuli and measured how often different perspectives were adopted; furthermore, whether camera orientation and gender had an effect. Additionally, we examined whether personality (trait extraversion and openness) could predict the perspective taken. We found that self-centered perspectives were predominantly adopted, and that trait openness may predict perspective. This research demonstrates how individual differences can affect the usability of sensory substitution devices, and highlights the need for flexible and customisable interfaces.

References

[1]

Abboud, S., Hanassy, S., Levy-Tzedek, S., Maidenbaum, S. and Amedi, A. 2014. EyeMusic: Introducing a "visual" colorful experience for the blind using auditory sensory substitution. Restorative Neurology and Neuroscience. 32, 2 (Jan. 2014), 247--257.

[2]

Abdullah, M., Kim, M., Hassan, W., Kuroda, Y. and Jeon, S. 2017. HapticDrone: An Encountered-Type Kinesthetic Haptic Interface with Controllable Force Feedback: Initial Example for 1D Haptic Feedbac. Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology - UIST '17 (New York, New York, USA, 2017),115--117.

Digital Library

[3]

Amedi, A., Stern, W.M., Camprodon, J.A., Bermpohl, F., Merabet, L., Rotman, S., Hemond, C., Meijer, P. and Pascual-Leone, A. 2007. Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nature Neuroscience. 10, 6 (Jun. 2007), 687--689.

[4]

Arnold, G., Spence, C. and Auvray, M. 2017. A unity of the self or a multiplicity of locations? How the graphesthesia task sheds light on the role of spatial perspectives in bodily self-consciousness. Consciousness and Cognition. 56, (Nov. 2017), 100-- 114. 12.

[5]

Arnold, G., Spence, C. and Auvray, M. 2016. Taking someone else's spatial perspective: Natural stance or effortful decentring? Cognition. 148, (Mar. 2016), 27--33. 2.006.

[6]

Bach-y-rita, P. 2004. Tactile Sensory Substitution Studies. Annals of the New York Academy of Sciences. 1013, 1 (2004), 83--91. 196/annals.1305.006.

[7]

Bach-y-Rita, P. and Kercel, S. 2003. Sensory substitution and the human--machine interface. Trends in Cognitive Sciences. 7, 12 (Dec. 2003), 541--546.

[8]

Bertossa, F., Besa, M., Ferrari, R. and Ferri, F. 2008. Point Zero: A Phenomenological Inquiry into the Seat of Consciousness. Perceptual and Motor Skills. 107, 2 (Oct. 2008), 323--335.

[9]

Birk, M. and Mandryk, R.L. 2013. Control your game-self. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '13 (New York, New York, USA, 2013), 685.

Digital Library

[10]

Van Boven, R.W. and Johnson, K.O. 1994. The limit of tactile spatial resolution in humans: grating orientation discrimination at the lip, tongue, and finger. Neurology. 44, 12 (Dec. 1994), 2361--6.

[11]

Brown, D., Macpherson, T. and Ward, J. 2011. Seeing with Sound? Exploring Different Characteristics of a Visual-to-Auditory Sensory Substitution Device. Perception. 40, 9 (Sep. 2011), 1120--1135.

[12]

Carton, A. and Dunne, L.E. 2013. Tactile distance feedback for firefighters. Proceedings of the 4th Augmented Human International Conference on AH '13 (New York, New York, USA, 2013), 58--64.

Digital Library

[13]

Chebat, D.-R., Rainville, C., Kupers, R. and Ptito, M. 2007. Tactile--'visual' acuity of the tongue in early blind individuals. NeuroReport. 18, 18 (Dec. 2007), 1901--1904. 3.

[14]

Danilov, Y. and Tyler, M. 2005. BrainPort: An Alternative Input to the Brain. Journal of Integrative Neuroscience. 04, 04 (Dec. 2005), 537--550.

[15]

Dublon, G. and Paradiso, J.A. 2012. Tongueduino. (2012), 1453.

Digital Library

[16]

Eysenck, H.J. 1961. Personality and Social Attitudes. The Journal of Social Psychology. 53, 2 (Apr. 1961), 243--248. 22.

[17]

Ferrè, E.R., Lopez, C. and Haggard, P. 2014. Anchoring the Self to the Body. Psychological Science. 25, 11 (Nov. 2014), 2106--2108.

[18]

Field, A. 2013. Discovering Statistics using IBM SPSS Statistics. Sage.

Digital Library

[19]

Funk, M. 2018. Human-drone interaction. Interactions. 25, 3 (Apr. 2018), 78--81.

Digital Library

[20]

Grant, P., Spencer, L., Arnoldussen, A., Hogle, R., Nau, A., Szlyk, J., Nussdorf, J., Fletcher, D.C., Gordon, K. and Seiple, W. 2016. The Functional Performance of the BrainPort V100 Device in Persons who Are Profoundly Blind. Journal of Visual Impairment & Blindness. 110, 2 (Mar. 2016), 77--88. .

[21]

Hoy, M.B. 2018. Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants. Medical Reference Services Quarterly. 37, 1 (Jan. 2018), 81--88. 91.

[22]

John, O.P., Donahue, E.M. and Kentle, R.L. 1991. The big five inventory-versions 4a and 54. Berkeley, CA: University of California, Berkeley, Institute of Personality.

[23]

McCrae, R.R. 1996. Social consequences of experiential openness. Psychological Bulletin. 120, 3 (1996), 323--337.

[24]

Nau, A., Bach, M. and Fisher, C. 2013. Clinical Tests of Ultra-Low Vision Used to Evaluate Rudimentary Visual Perceptions Enabled by the BrainPort Vision Device. Translational Vision Science & Technology. 2, 3 (Jun. 2013), 1.

[25]

Parsons, L.M. and Shimojo, S. 1987. Perceived spatial organization of cutaneous patterns on surfaces of the human body in various positions. Journal of Experimental Psychology: Human Perception and Performance. 13, 3 (1987), 488--504.

[26]

Richardson, M., Thar, J., Borchers, J., Ward, J. and Hamilton-Fletcher, G. 2019. How much spatial information is lost in the sensory substitution process? Comparing visual, tactile, and auditory approaches. Perception. (2019).

[27]

Sampaio, E., Maris, S. and Bach-y-Rita, P. 2001. Brain plasticity: 'visual' acuity of blind persons via the tongue. Brain Research. 908, 2 (Jul. 2001), 204-- 207.

[28]

Schober, M.F. 1993. Spatial perspective-taking in conversation. Cognition. 47, 1 (Jan. 1993), 1--24.

[29]

Sekiyama, K. 1991. Importance of head axes in perception of cutaneous patterns drawn on vertical body surfaces. Perception & Psychophysics. 49, 5 (Sep. 1991), 481--492.

[30]

Shelton, A.L., Clements-Stephens, A.M., Lam, W.Y., Pak, D.M. and Murray, A.J. 2012. Should social savvy equal good spatial skills? The interaction of social skills with spatial perspective taking. Journal of Experimental Psychology: General. 141, 2 (2012), 199--205.

[31]

Stronks, H.C., Mitchell, E.B., Nau, A.C. and Barnes, N. 2016. Visual task performance in the blind with the BrainPort V100 Vision Aid. Expert Review of Medical Devices. 13, 10 (Oct. 2016), 919--931. 87.

[32]

Tarampi, M.R., Heydari, N. and Hegarty, M. 2016. A Tale of Two Types of Perspective Taking. Psychological Science. 27, 11 (Nov. 2016), 1507-- 1516.

[33]

Ward, J. and Meijer, P. 2010. Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness and Cognition. 19, 1 (Mar. 2010), 492--500.

Cited By

Benhamida LLarabi SMetatla O(2024)Hapstick-Figure: Investigating the Design of a Haptic Representation of Human Gestures from Theater Performances for Blind and Visually-Impaired PeopleProceedings of the ACM on Human-Computer Interaction10.1145/36981528:ISS(649-673)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698152
Steer CSauvé KJain ALawal OProulx MJicol CAlexander J(2024)Squishy, Yet Satisfying: Exploring Deformable Shapes' Cross-Modal Correspondences with Colours and EmotionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641952(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641952
Barros De Amorim ISantos Dos Santos SMoreira Miranda Da Silva IRios Da Hora Rodrigues KPereira Mota M(2023)Gender Nuances in Human-Computer Interaction ResearchProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638077(1-12)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638077
Show More Cited By

Index Terms

Reading with the Tongue: Individual Differences Affect the Perception of Ambiguous Stimuli with the BrainPort
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. Laboratory experiments

Recommendations

Reading Fluent Speech from Talking Faces: Typical Brain Networks and Individual Differences

Listeners are able to extract important linguistic information by viewing the talker's face—a process known as ''speechreading.'' Previous studies of speechreading present small closed sets of simple words and their results indicate that visual speech ...
fMRI Data Predict Individual Differences of Behavioral Effects of Nicotine: A Partial Least Square Analysis

Reorienting of visuospatial attention can be investigated by comparing reaction times to validly and invalidly cued targets (“validity effect”). The cholinergic agonist nicotine reduces the validity effect and neural activity in the posterior parietal ...
Individual Differences in the Effects of Perceived Controllability on Pain Perception: Critical Role of the Prefrontal Cortex

The degree to which perceived controllability alters the way a stressor is experienced varies greatly among individuals. We used functional magnetic resonance imaging to examine the neural activation associated with individual differences in the impact ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
700
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)8

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Benhamida LLarabi SMetatla O(2024)Hapstick-Figure: Investigating the Design of a Haptic Representation of Human Gestures from Theater Performances for Blind and Visually-Impaired PeopleProceedings of the ACM on Human-Computer Interaction10.1145/36981528:ISS(649-673)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698152
Steer CSauvé KJain ALawal OProulx MJicol CAlexander J(2024)Squishy, Yet Satisfying: Exploring Deformable Shapes' Cross-Modal Correspondences with Colours and EmotionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641952(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641952
Barros De Amorim ISantos Dos Santos SMoreira Miranda Da Silva IRios Da Hora Rodrigues KPereira Mota M(2023)Gender Nuances in Human-Computer Interaction ResearchProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638077(1-12)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638077
Mukashev DRanasinghe NNittala A(2023)TactTongue: Prototyping ElectroTactile Stimulations on the TongueProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606829(1-14)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606829
Franzke LGartmann JBachmann DTöpper TFraninović K(2023)TOFI: Designing Intraoral Computer Interfaces for Gamified Myofunctional TherapyExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3573848(1-8)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3573848
Best JChapleau MRabinovici G(2023)Posterior cortical atrophy: clinical, neuroimaging, and neuropathological featuresExpert Review of Neurotherapeutics10.1080/14737175.2023.219088523:3(227-236)Online publication date: 15-Mar-2023
https://doi.org/10.1080/14737175.2023.2190885
Lin YLi YGe SXu JLi LXu S(2022)Three-Dimensional Encoding Approach for Wearable Tactile Communication DevicesSensors10.3390/s2224956822:24(9568)Online publication date: 7-Dec-2022
https://doi.org/10.3390/s22249568
Hu XSong AWei ZZeng H(2022)StereoPilot: A Wearable Target Location System for Blind and Visually Impaired Using Spatial Audio RenderingIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2022.318266130(1621-1630)Online publication date: 2022
https://doi.org/10.1109/TNSRE.2022.3182661
Bleau MParé SDjerourou IChebat DKupers RPtito M(2021)Blindness and the Reliability of Downwards Sensors to Avoid Obstacles: A Study with the EyeCaneSensors10.3390/s2108270021:8(2700)Online publication date: 12-Apr-2021
https://doi.org/10.3390/s21082700
Hamilton-Fletcher GAlvarez JObrist MWard J(2021)SoundSight: a mobile sensory substitution device that sonifies colour, distance, and temperatureJournal on Multimodal User Interfaces10.1007/s12193-021-00376-w16:1(107-123)Online publication date: 2-Jul-2021
https://doi.org/10.1007/s12193-021-00376-w
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents