More Web Proxy on the site http://driver.im/

short-paper

Open access

Using Deep Learning to Classify Saccade Direction from Brain Activity

Authors:

Martyna Beata Plomecka,

Roger Wattenhofer,

Nicolas LangerAuthors Info & Claims

ETRA '21 Short Papers: ACM Symposium on Eye Tracking Research and Applications

Article No.: 28, Pages 1 - 6

https://doi.org/10.1145/3448018.3458014

Published: 25 May 2021 Publication History

All formats PDF

Abstract

We present first insights into our project that aims to develop an Electroencephalography (EEG) based Eye-Tracker. Our approach is tested and validated on a large dataset of simultaneously recorded EEG and infrared video-based Eye-Tracking, serving as ground truth. We compared several state-of-the-art neural network architectures for time series classification: InceptionTime, EEGNet, and investigated other architectures such as convolutional neural networks (CNN) with Xception modules and Pyramidal CNN. We prepared and tested these architectures with our rich dataset and obtained a remarkable accuracy of the left/right saccades direction classification (94.8 %) for the InceptionTime network, after hyperparameter tuning.

References

[1]

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, 2016. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16). 265–283.

Digital Library

[2]

Chrystalina Antoniades, Ulrich Ettinger, Bertrand Gaymard, Iain Gilchrist, Arni Kristjánsson, Christopher Kennard, R John Leigh, Imran Noorani, Pierre Pouget, Nikolaos Smyrnis, 2013. An internationally standardised antisaccade protocol. Vision research 84(2013), 1–5.

[3]

Andreas Bulling, Jamie A Ward, Hans Gellersen, and Gerhard Tröster. 2010. Eye movement analysis for activity recognition using electrooculography. IEEE transactions on pattern analysis and machine intelligence 33, 4(2010), 741–753.

Digital Library

[4]

François Chollet. 2017. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1251–1258.

[5]

Michael X Cohen, Christian E Elger, and Charan Ranganath. 2007. Reward expectation modulates feedback-related negativity and EEG spectra. Neuroimage 35, 2 (2007), 968–978.

[6]

Benedikt V Ehinger and Olaf Dimigen. 2019. Unfold: An integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis. PeerJ 7(2019), e7838.

[7]

Hassan Ismail Fawaz, Benjamin Lucas, Germain Forestier, Charlotte Pelletier, Daniel F Schmidt, Jonathan Weber, Geoffrey I Webb, Lhassane Idoumghar, Pierre-Alain Muller, and François Petitjean. 2020a. Inceptiontime: Finding alexnet for time series classification. Data Mining and Knowledge Discovery 34, 6 (2020), 1936–1962.

Digital Library

[8]

Hassan Ismail Fawaz, Benjamin Lucas, Germain Forestier, Charlotte Pelletier, Daniel F Schmidt, Jonathan Weber, Geoffrey I Webb, Lhassane Idoumghar, Pierre-Alain Muller, and François Petitjean. 2020b. Inceptiontime: Finding alexnet for time series classification. Data Mining and Knowledge Discovery 34, 6 (2020), 1936–1962.

Digital Library

[9]

Elizabeth R Grant and Michael J Spivey. 2003. Eye movements and problem solving: Guiding attention guides thought. Psychological Science 14, 5 (2003), 462–466.

[10]

Nikolay V Kapralov, Jaroslav V Ekimovskii, and Vyacheslav V Potekhin. 2019. EEG-Based Brain-Computer Interface for Control of Assistive Devices. In International Conference Cyber-Physical Systems and Control. Springer, 536–543.

[11]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60, 6 (2017), 84–90.

Digital Library

[12]

S LaConte, C Glielmi, K Heberlein, and X Hu. 2007. Verifying visual fixation to improve fMRI with predictive eye estimation regression (PEER). In Proc. Intl. Soc. Magn. Reson. Med, Vol. 3438.

[13]

Vernon J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent J Lance. 2018. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. Journal of neural engineering 15, 5 (2018), 056013.

[14]

José del R Millán, Rüdiger Rupp, Gernot Mueller-Putz, Roderick Murray-Smith, Claudio Giugliemma, Michael Tangermann, Carmen Vidaurre, Febo Cincotti, Andrea Kubler, Robert Leeb, 2010. Combining brain–computer interfaces and assistive technologies: state-of-the-art and challenges. Frontiers in neuroscience 4 (2010), 161.

[15]

Andreas Pedroni, Amirreza Bahreini, and Nicolas Langer. 2019. Automagic: Standardized preprocessing of big EEG data. Neuroimage 200(2019), 460–473.

[16]

Martyna Beata Płomecka, Zofia Barańczuk-Turska, Christian Pfeiffer, and Nicolas Langer. 2020. Aging Effects and Test–Retest Reliability of Inhibitory Control for Saccadic Eye Movements. Eneuro 7, 5 (2020).

[17]

Yannick Roy, Hubert Banville, Isabela Albuquerque, Alexandre Gramfort, Tiago H Falk, and Jocelyn Faubert. 2019a. Deep learning-based electroencephalography analysis: a systematic review. Journal of neural engineering 16, 5 (2019), 051001.

[18]

Yannick Roy, Hubert Banville, Isabela Albuquerque, Alexandre Gramfort, Tiago H Falk, and Jocelyn Faubert. 2019b. Deep learning-based electroencephalography analysis: a systematic review. Journal of neural engineering 16, 5 (2019), 051001.

[19]

Jake Son, Lei Ai, Ryan Lim, Ting Xu, Stanley Colcombe, Alexandre Rosa Franco, Jessica Cloud, Stephen LaConte, Jonathan Lisinski, Arno Klein, 2020. Evaluating fMRI-Based Estimation of Eye Gaze during Naturalistic Viewing. Cerebral Cortex 30, 3 (2020), 1171–1184.

[20]

Amirali Vahid, Moritz Mückschel, Sebastian Stober, Ann-Kathrin Stock, and Christian Beste. 2020. Applying deep learning to single-trial EEG data provides evidence for complementary theories on action control. Communications biology 3, 1 (2020), 1–11.

Cited By

Chandel SBhattacharya RNayak MPathak A(2024)Integrating Eye Gaze Estimation with the Internet of Medical Things (IoMT) for Individualized and Efficient Healthcare2024 2nd World Conference on Communication & Computing (WCONF)10.1109/WCONF61366.2024.10692177(1-6)Online publication date: 12-Jul-2024
https://doi.org/10.1109/WCONF61366.2024.10692177
Bature ZAbdullahi SYeamkuan SChiracharit WChamnongthai K(2023)Boosted Gaze Gesture Recognition Using Underlying Head Orientation SequenceIEEE Access10.1109/ACCESS.2023.327028511(43675-43689)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3270285

Index Terms

Using Deep Learning to Classify Saccade Direction from Brain Activity

Index terms have been assigned to the content through auto-classification.

Recommendations

Determining Confused Brain Activity from EEG Sensor Signals
NSysS '21: Proceedings of the 8th International Conference on Networking, Systems and Security

Human brain, being the most remarkable logic processing unit, sometimes fails to act properly. Often the task at hand becomes too cumbersome for the brain to perceive, which is known as confusion in simple terms. In some certain critical moments, it is ...
A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals
Abstract
In the Motor Imagery (MI)-based Brain Computer Interface (BCI), users' intention is converted into a control signal through processing a specific pattern in brain signals reflecting motor characteristics. There are such restrictions as ...
Highlights
- Data Augmentation by the cropping method to increase the number of trials.
- Time-...
Normalized deep learning algorithms based information aggregation functions to classify motor imagery EEG signal
Abstract
Recently, the discipline of Brain-Computer-Interface (BCI) has attracted attention to exploiting Electroencephalograph (EEG) mental activities such as Motor Imagery (MI). Neurons in the human brain are activated during these MI tasks and generate ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '21 Short Papers: ACM Symposium on Eye Tracking Research and Applications

May 2021

232 pages

ISBN:9781450383455

DOI:10.1145/3448018

Editors:
Andreas Bulling
University of Stuttgart, Germany
,
Anke Huckauf
Ulm University, Germany
,
Hans Gellersen
Aarhus University, Denmark
,
Daniel Weiskopf
University of Stuttgart, Germany
,
Mihai Bace
ETH Zürich, Switzerland
,
Teresa Hirzle
Ulm University, Germany
,
Florian Alt
Bundeswehr University Munich, Germany
,
Thies Pfeiffer
Hochschule Emden/Leer, Germany
,
Roman Bednarik
University of Eastern Finland, Finland
,
Krzysztof Krejtz
SWPS University of Social Sciences and Humanities, Poland
,
Tanja Blascheck
University of Stuttgart, Germany
,
Michael Burch
University of Applied Sciences, Chur, Switzerland
,
Peter Kiefer
ETH Zurich, Switzerland
,
Michael Dodd
University of Nebraska-Lincoln, USA
,
Bonita Sharif
University of Nebraska-Lincoln, USA

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

ETRA '21

Sponsor:

SIGGRAPH

ETRA '21: 2021 Symposium on Eye Tracking Research and Applications

May 25 - 27, 2021

Virtual Event, Germany

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
988
Total Downloads

Downloads (Last 12 months)189
Downloads (Last 6 weeks)11

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chandel SBhattacharya RNayak MPathak A(2024)Integrating Eye Gaze Estimation with the Internet of Medical Things (IoMT) for Individualized and Efficient Healthcare2024 2nd World Conference on Communication & Computing (WCONF)10.1109/WCONF61366.2024.10692177(1-6)Online publication date: 12-Jul-2024
https://doi.org/10.1109/WCONF61366.2024.10692177
Bature ZAbdullahi SYeamkuan SChiracharit WChamnongthai K(2023)Boosted Gaze Gesture Recognition Using Underlying Head Orientation SequenceIEEE Access10.1109/ACCESS.2023.327028511(43675-43689)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3270285

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents