A Novel Bci Paradigm Combining Visual Imagery and Emotion: A Pilot Study
Authors: 
Long Chen, Chenghu Tang, Changyun Li, Wanping Qiu, Zhongpeng Wang, Xin Zhang, Dong MingAuthors Info & Claims
Pages 143 - 150
Abstract
Visual imagery (VI) has become a research hotspot with the development of brain-computer Interface (BCI). But it's very difficult to decode the electroencephalogram (EEG) during the VI task because of the lack of effective classification features. However, the classification of emotion-based EEG now can achieve a very high accuracy. We proposed a new BCI paradigm in which emotions would be integrated into VI to improve the performance. To verify the feasibility of our paradigm, we designed a two-session EEG acquisition experiment. We designed three kinds of emotions and three kinds of semantic correspondence i.e., negative animal, positive human and neutral thing. Sixteen subjects were asked to observe and imagine three pictures selected by subjects in visual observation (VO) and VI session, respectively. Then, we used power spectral density (PSD), common spatial pattern (CSP) and Riemann tangent space three features with SVM to analysis the EEG during three statements. The classification results show that the Riemann features obtain the highest average accuracies which are 84.36% and 67.36% for VO and VI, respectively. All the three types of features can exceed chance level on all the subjects. The results prove that our VI integrating emotion is a promising paradigm.
References
[1]
María A. Cervera, Surjo R. Soekadar, Junichi Ushiba, José del R. Millán, Meigen Liu, Niels Birbaumer, and Gangadhar Garipelli. 2018. Brain-computer interfaces for post-stroke motor rehabilitation: a meta-analysis. Ann. Clin. Transl. Neurol. 5, 5 (May 2018), 651–663. https://doi.org/10.1002/acn3.544
[2]
F Lotte, L Bougrain, A Cichocki, M Clerc, M Congedo, A Rakotomamonjy, and F Yger. 2018. A review of classification algorithms for EEG-based brain–computer interfaces: a 10-year update. J. Neural Eng. 15, 3 (June 2018), 031005. https://doi.org/10.1088/1741-2552/aab2f2
[3]
B. J. Edelman, J. Meng, D. Suma, C. Zurn, E. Nagarajan, B. S. Baxter, C. C. Cline, and B. He. 2019. Noninvasive neuroimaging enhances continuous neural tracking for robotic device control. Sci. Robot. 4, 31 (June 2019), eaaw6844. https://doi.org/10.1126/scirobotics.aaw6844
[4]
Reza Abiri, Soheil Borhani, Eric W Sellers, Yang Jiang, and Xiaopeng Zhao. 2019. A comprehensive review of EEG-based brain–computer interface paradigms. J. Neural Eng. 16, 1 (February 2019), 011001. https://doi.org/10.1088/1741-2552/aaf12e
[5]
Minpeng Xu, Jin Han, Yijun Wang, Tzyy-Ping Jung, and Dong Ming. 2020. Implementing Over 100 Command Codes for a High-Speed Hybrid Brain-Computer Interface Using Concurrent P300 and SSVEP Features. IEEE Trans. Biomed. Eng. 67, 11 (November 2020), 3073–3082. https://doi.org/10.1109/TBME.2020.2975614
[6]
Xiaolin Xiao, Minpeng Xu, Jing Jin, Yijun Wang, Tzyy-Ping Jung, and Dong Ming. 2020. Discriminative Canonical Pattern Matching for Single-Trial Classification of ERP Components. IEEE Trans. Biomed. Eng. 67, 8 (August 2020), 2266–2275. https://doi.org/10.1109/TBME.2019.2958641
[7]
Jing Jin, Ruocheng Xiao, Ian Daly, Yangyang Miao, Xingyu Wang, and Andrzej Cichocki. 2020. Internal Feature Selection Method of CSP Based on L1-Norm and Dempster-Shafer Theory. IEEE Trans. Neural Netw. Learn. Syst. (August 2020), 1–12. https://doi.org/10.1109/TNNLS.2020.3015505
[8]
Markus Knauff, Jan Kassubek, Thomas Mulack, and Mark W. Greenlee. 2000. Cortical activation evoked by visual mental imagery as measured by fMRI: NeuroReport 11, 18 (December 2000), 3957–3962. https://doi.org/10.1097/00001756-200012180-00011
[9]
Nataliya Kosmyna, Jussi T. Lindgren, and Anatole Lécuyer. 2018. Attending to Visual Stimuli versus Performing Visual Imagery as a Control Strategy for EEG-based Brain-Computer Interfaces. Sci. Rep. 8, 1 (December 2018), 13222. https://doi.org/10.1038/s41598-018-31472-9
[10]
Milan Rybář, Riccardo Poli, and Ian Daly. 2021. Decoding of semantic categories of imagined concepts of animals and tools in fNIRS. J. Neural Eng. 18, 4 (August 2021), 046035. https://doi.org/10.1088/1741-2552/abf2e5
[11]
Bertrand Thirion, Edouard Duchesnay, Edward Hubbard, Jessica Dubois, Jean-Baptiste Poline, Denis Lebihan, and Stanislas Dehaene. 2006. Inverse retinotopy: Inferring the visual content of images from brain activation patterns. NeuroImage 33, 4 (December 2006), 1104–1116. https://doi.org/10.1016/j.neuroimage.2006.06.062
[12]
Max A van den Boom, Mariska J Vansteensel, Melissa I Koppeschaar, Matthijs A H Raemaekers, and Nick F Ramsey. 2019. Towards an intuitive communication-BCI: decoding visually imagined characters from the early visual cortex using high-field fMRI. Biomed. Phys. Eng. Express 5, 5 (August 2019), 055001. https://doi.org/10.1088/2057-1976/ab302c
[13]
Seo-Hyun Lee, Minji Lee, and Seong-Whan Lee. 2020. Neural Decoding of Imagined Speech and Visual Imagery as Intuitive Paradigms for BCI Communication. IEEE Trans. Neural Syst. Rehabil. Eng. (December 2020), 1–1. https://doi.org/10.1109/TNSRE.2020.3040289
[14]
Wei-Long Zheng, Jia-Yi Zhu, and Bao-Liang Lu. 2019. Identifying Stable Patterns over Time for Emotion Recognition from EEG. IEEE Trans. Affect. Comput. 10, 3 (July 2019), 417–429. https://doi.org/10.1109/TAFFC.2017.2712143
[15]
Huiping Jiang, Zequn Wang, Rui Jiao, and Shan Jiang. 2020. Picture-induced EEG Signal Classification Based on CVC Emotion Recognition System. Comput. Mater. Contin. 65, 2 (2020), 1453–1465. https://doi.org/10.32604/cmc.2020.011793
[16]
Merve Dogruyol Basar, Adil Deniz Duru, and Aydin Akan. 2020. Emotional state detection based on common spatial patterns of EEG. Signal Image Video Process. 14, 3 (April 2020), 473–481. https://doi.org/10.1007/s11760-019-01580-8
[17]
Aurelien Appriou, Andrzej Cichocki, and Fabien Lotte. 2020. Modern Machine-Learning Algorithms: For Classifying Cognitive and Affective States From Electroencephalography Signals. IEEE Syst. Man Cybern. Mag. 6, 3 (July 2020), 29–38. https://doi.org/10.1109/MSMC.2020.2968638
[18]
Mengmeng Yan, Zhao Lv, Wenhui Sun, and Ning Bi. 2020. An improved common spatial pattern combined with channel-selection strategy for electroencephalography-based emotion recognition. Med. Eng. Phys. 83, (September 2020), 130–141. https://doi.org/10.1016/j.medengphy.2020.05.006
[19]
Cédric Simar, Ana-Maria Cebolla, Gaëlle Chartier, Mathieu Petieau, Gianluca Bontempi, Alain Berthoz, and Guy Cheron. 2020. Hyperscanning EEG and Classification Based on Riemannian Geometry for Festive and Violent Mental State Discrimination. Front. Neurosci. 14, (December 2020), 588357. https://doi.org/10.3389/fnins.2020.588357
[20]
Yixin Wang, Shuang Qiu, Xuelin Ma, and Huiguang He. 2021. A prototype-based SPD matrix network for domain adaptation EEG emotion recognition. Pattern Recognit. 110, (February 2021), 107626. https://doi.org/10.1016/j.patcog.2020.107626
[21]
Ren Li, Jared S. Johansen, Hamad Ahmed, Thomas V. Ilyevsky, Ronnie B. Wilbur, Hari M. Bharadwaj, and Jeffrey Mark Siskind. 2021. The Perils and Pitfalls of Block Design for EEG Classification Experiments. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1 (January 2021), 316–333. https://doi.org/10.1109/TPAMI.2020.2973153
[22]
Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International Affective Picture System (IAPS): Affective ratings of pictures and instruction manual (Rep. No. A-8). Gainesville, FL: University of Florida.
[23]
A. Barachant, S. Bonnet, M. Congedo, and C. Jutten. 2012. Multiclass Brain–Computer Interface Classification by Riemannian Geometry. IEEE Trans. Biomed. Eng. 59, 4 (April 2012), 920–928. https://doi.org/10.1109/TBME.2011.2172210
[24]
Alexandre Barachant, Stéphane Bonnet, Marco Congedo, and Christian Jutten. 2013. Classification of covariance matrices using a Riemannian-based kernel for BCI applications. Neurocomputing 112, (July 2013), 172–178. https://doi.org/10.1016/j.neucom.2012.12.039
[25]
Xinjiang Lu and Yunxu Bai. 2020. Robust Least-Squares Support Vector Machine Using Probabilistic Inference. IEEE Trans. Cybern. (October 2020), 1–9. https://doi.org/10.1109/TCYB.2020.3026680
[26]
Rasmus Bro and Age K. Smilde. 2014. Principal component analysis. Anal Methods 6, 9 (2014), 2812–2831. https://doi.org/10.1039/C3AY41907J
[27]
A. D. Bigirimana, N. Siddique, and D. Coyle. 2020. Emotion-Inducing Imagery Versus Motor Imagery for a Brain-Computer Interface. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 4 (April 2020), 850–859. https://doi.org/10.1109/TNSRE.2020.2978951
[28]
Daniela Iacoviello, Andrea Petracca, Matteo Spezialetti, and Giuseppe Placidi. 2015. A real-time classification algorithm for EEG-based BCI driven by self-induced emotions. Comput. Methods Programs Biomed. 122, 3 (December 2015), 293–303. https://doi.org/10.1016/j.cmpb.2015.08.011
Recommendations
Brain-computer interface research at Katholieke Universiteit Leuven
ISABEL '11: Proceedings of the 4th International Symposium on Applied Sciences in Biomedical and Communication TechnologiesWe present an overview of our Brain-computer interface (BCI) research, invasive as well as non-invasive, during the past four years. The invasive BCIs are based on local field-and action potentials recorded with microelectrode arrays implanted in the ...
Quantification of event related brain patterns for the motor imagery tasks using inter-trial variance technique
AbstractQuantification of event-related (de) synchronization (ERD/ERS) patterns is a challenging task in the field of motor imagery (MI)-based brain–computer interface (BCI). Accurately determining the optimal time and frequency band for localizing the ...
A hybrid brain-computer interface for smart home control
HCII'11: Proceedings of the 14th international conference on Human-computer interaction: interaction techniques and environments - Volume Part IIBrain-computer interfaces (BCI) provide a new communication channel between the human brain and a computer without using any muscle activities. Applications of BCI systems comprise communication, restoration of movements or environmental control. Within ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2021
216 pages
ISBN:9781450385077
DOI:10.1145/3502871
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 March 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICBBE '21
ICBBE '21: 2021 8th International Conference on Biomedical and Bioinformatics Engineering
November 12 - 15, 2021
Kyoto, Japan
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 99Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)1
Reflects downloads up to 09 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format