Combining a Brain-Machine Interface and an Electrooculography Interface to perform pick and place tasks with a robotic arm
Authors: 
Enrique Hortal, Eduardo Iáñez, Andrés Úbeda, Carlos Perez-Vidal, José M. AzorínAuthors Info & Claims
Pages 181 - 188
Abstract
This paper presents a multimodal Human-Machine Interface system that combines an Electrooculography Interface and a Brain-Machine Interface. This multimodal interface has been used to control a robotic arm to perform pick and place tasks in a three dimensional environment. Five volunteers were asked to pick two boxes and place them in different positions. The results prove the feasibility of the system in the performance of pick and place tasks. By using the multimodal interface, all the volunteers (even naive users) were able to successfully move two objects within a satisfactory period of time with the help of the robotic arm. A multimodal HMI to perform pick and place tasks with a robotic arm is presented.The EOG interface is applied to control planar movements and to operate the gripper.The BMI is used to control the height of the gripper through two mental tasks.The system had been tested by five healthy subjects.The results prove the feasibility of the system in the performance of these tasks.
References
[1]
M.A.L. Nicolelis, Actions from thoughts, Nature, 409 (2001) 403-407.
[2]
I. Iturrate, J. Antelis, A. Kubler, J. Minguez, A noninvasive brain-actuated wheelchair based on a p300 neurophysiological protocol and automated navigation, IEEE Trans. Robot., 25 (2009) 614-627.
[3]
P.F. Diez, S.M.T. Muller, V.A. Mut, E. Laciar, E. Avila, T.F. Bastos-Filho, M. Sarcinelli-Filho, Commanding a robotic wheelchair with a high-frequency steady-state visual evoked potential based brain-computer interface, Med. Eng. Phys., 35 (2013) 1155-1164.
[4]
D. Pérez-Marcos, J.A. Buitrago, F.D.G. Velásquez, Writing through a robot: A proof of concept for a brain-machine interface, Med. Eng. Phys., 33 (2011) 1314-1317.
[5]
H.J. Hwang, J.H. Lim, Y.J. Jung, H. Choi, S.W. Lee, C.H. Im, Development of an ssvep-based {BCI} spelling system adopting a qwerty-style {LED} keyboard, J. Neurosci. Methods, 208 (2012) 59-65.
[6]
E. Hortal, A. Úbeda, E. Iánez, J.M. Azorín, Control of a 2 DoF robot using a brain-machine interface, Comput. Methods Programs Biomed., 116 (2014) 169-176.
[7]
D.J. Mcfarland, J.R. Wolpaw, Chapter 4-brain-computer interfaces for the operation of robotic and prosthetic devices, in: Advances in Computers. Vol. 79, Elsevier, 2010, pp. 169-187.
[8]
C.C. Postelnicu, F. Girbacia, D. Talaba, Eog-based visual navigation interface development, Expert Syst. Appl., 39 (2012) 10857-10866.
[9]
R. Berta, F. Bellotti, A. De Gloria, D. Pranantha, C. Schatten, Electroencephalogram and physiological signal analysis for assessing flow in games, IEEE Trans. Comput. Intell. AI Games, 5 (2013) 164-175.
[10]
R. Barea, L. Boquete, M. Mazo, E. López, Wheelchair guidance strategies using eog, J. Intell. Robot. Syst., 34 (2002) 279-299.
[11]
K. Tanaka, K. Matsunaga, H. Wang, Electroencephalogram-based control of an electric wheelchair, IEEE Trans. Robot., 21 (2005) 762-766.
[12]
A. Úbeda, E. Iánez, J. Azorín, Wireless and portable eog-based interface for assisting disabled people, IEEE/ASME Trans. Mechatronics, 16 (2011) 870-873.
[13]
Y. Chae, J. Jeong, S. Jo, Toward brain-actuated humanoid robots: Asynchronous direct control using an eeg-based bci, IEEE Trans. Robot., 28 (2012) 1131-1144.
[14]
A. Frisoli, C. Loconsole, D. Leonardis, F. Banno, M. Barsotti, C. Chisari, M. Bergamasco, A new gaze-bci-driven control of an upper limb exoskeleton for rehabilitation in real-world tasks, IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., 42 (2012) 1169-1179.
[15]
E.C. Lee, J.C. Woo, J.H. Kim, M. Whang, K.R. Park, A brain-computer interface method combined with eye tracking for 3d interaction, J. Neurosci. Methods, 190 (2010) 289-298.
[16]
D.P. McMullen, G. Hotson, K.D. Katyal, B.A. Wester, M.S. Fifer, T.G. McGee, A. Harris, M.S. Johannes, R.J. Vogelstein, A.D. Ravitz, W.S. Anderson, N.V. Thakor, N.E. Crone, Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial eeg, eye tracking, and computer vision to control a robotic upper limb prosthetic, IEEE Trans. Neural Syst. Rehabil. Eng., 22 (2014) 784-796.
[17]
M.H. Lee, S. Fazli, J. Mehnert, S.W. Lee, Hybrid brain-computer interface based on EEG and NIRS modalities, in: 2014 International Winter Workshop on Brain-Computer Interface, BCI, 2014, pp. 1-2. http://dx.doi.org/10.1109/iww-BCI.2014.6782577.
[18]
K. Takahashi, T. Kashiyama, Remarks on multimodal nonverbal interface and its application to controlling mobile robot, in: Mechatronics and Automation, 2005 IEEE International Conference, Vol. 2, 2005, pp. 999-1004. http://dx.doi.org/10.1109/ICMA.2005.1626688.
[19]
C.C. Postelnicu, D. Talaba, M.I. Toma, Controlling a robotic arm by brainwaves and eye movement, in: IFIP Advances in Information and Communication Technology, vol. 349, Springer, Berlin, Heidelberg, 2011, pp. 157-164.
[20]
F. Babiloni, C. Del Gratta, F. Carducci, C. Babiloni, G.M. Roberti, V. Pizzella, P.M. Rossini, G. Romani, A. Urbano, Combined high resolution eeg and meg data for linear inverse estimate of human event-related cortical activity, in: Engineering in Medicine and Biology Society, 1998. Proceedings of the 20th Annual International Conference of the IEEE, Vol. 4, 1998, pp. 2151-2154. http://dx.doi.org/10.1109/IEMBS.1998.747035.
[21]
B.H. Kim, M. Kim, S. Jo, Quadcopter flight control using a low-cost hybrid interface with eeg-based classification and eye tracking, Comput. Biol. Med., 51 (2014) 82-92.
[22]
E. Hortal, A. Úbeda, E. Iánez, D. Planelles, J. Azorín, Online classification of two mental tasks using a SVM-based BCI system, in: 2013 6th International IEEE/EMBS Conference on Neural Engineering, NER, 2013, pp. 1307-1310. http://dx.doi.org/10.1109/NER.2013.6696181.
[23]
C.W. Hsu, C.C. Chang, C.J. Lin, A practical guide to support vector classification, online (Updated April 2012). URL: http://www.csie.ntu.edu.tw/cjlin/libsvm/ ¿Accessed 11.10.12.
[24]
F. Flórez, J.M. Azorín, E. Iánez, A. Úbeda, E. Fernández, Development of a low-cost svm-based spontaneous brain-computer interface, in: IJCCI (NCTA), SciTePress, 2011, pp. 415-421.
[25]
E. Iánez, J.M. Azorín, C. Perez-Vidal, Using eye movement to control a computer: A design for a lightweight electro-oculogram electrode array and computer interface, PLoS One, 8 (2013) e67099.
[26]
C. Guger, A. Schlögl, C. Neuper, D. Walterspacher, T. Strein, G. Pfurtscheller, Rapid prototyping of an EEG-based brain-computer interface (BCI), IEEE Trans. Rehabil. Eng., 9 (2001) 49-58.
- Combining a Brain-Machine Interface and an Electrooculography Interface to perform pick and place tasks with a robotic arm
Recommendations
Towards Hybrid Multimodal Brain Computer Interface for Robotic Arm Command
Augmented CognitionAbstractA hybrid brain-computer interface (BCI) is a system that combines multiple biopotentials or different types of devices with a typical BCI system to enhance the interaction paradigms and various functional parameters. In this paper we present the ...
Brain-Computer Interface using Directional Auditory Perception
AHs '23: Proceedings of the Augmented Humans International Conference 2023We investigate the potential of brain-computer interface (BCI) using electroencephalogram (EEG) induced by listening (or recalling) auditory stimuli of different directions. In the initial attempt, we apply a time series classification model based on ...
An electrocorticographic decoder for arm movement for brain–machine interface using an echo state network and Gaussian readout
AbstractBrain–machine interface (BMI) studies typically use an electrocorticogram (ECoG) to record neural signals from the surface of the cortex because of the high spatial and temporal resolution and high signal-to-noise levels of the data ...
Highlights- Electrocorticography (ECoG) decoder developed for brain–machine interface (BCI) using echo state networks and Gaussian readouts.
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © Elsevier B.V.
Publisher
North-Holland Publishing Co.
Netherlands
Publication History
Published: 01 October 2015
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Dec 2024
Other Metrics
Citations
Cited By
View allView Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in