abstract

Integrating human and computer vision with EEG toward the control of a prosthetic arm

Authors:

Eugene Lavely,

Geoffrey Meltzner,

Rick ThompsonAuthors Info & Claims

HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

Pages 179 - 180

https://doi.org/10.1145/2157689.2157744

Published: 05 March 2012 Publication History

Get Access

Abstract

We are undertaking the development of a brain computer interface (BCI) [1] for control of an upper limb prosthetic. Our approach exploits electrical neural activity data for motor intent estimation, and eye gaze direction for target selection. These data streams are augmented by computer vision (CV) for 3D scene reconstruction, and are integrated with a hierarchical controller to achieve semi-autonomous control. User interfaces for the effective control of the many degrees of freedom (DOF) of advanced prosthetic arms are not yet available [2]. Ideally the combined arm and interface technology provides the user with reliable and dexterous capability for reaching, grasping and fine-scale manipulation. Technologies that improve arm embodiment i.e., the impression by the amputee subject that the arm is a natural part of their body-concept presents an important and difficult challenge to the human-robot interaction research community. Such embodiment is clearly predicated on cross-disciplinary advances, including accurate intent estimation and an and an algorithmic basis for natural arm control.

References

[1]

R. P. N. Rao and R. Scherer, Statistical Pattern Recognition and Machine Learning in Brain-Computer Interfaces, in Statistical Signal Processing for Neuroscience and Neurotechnology, K. Oweiss, Ed., New York: Academic Press, 433 pages, 2010.

Google Scholar

[2]

R. F. Weir and J. W. Sensinger, Design of artificial arms and hands for prosthetic applications, in Standard Handbook of Biomedical Engineering & Design, Vol. 2, M. Kutz, Ed. New York: McGraw-Hill, 2009.

Google Scholar

[3]

S. Thrun, Toward a framework for human-robot interaction, Hum.-Comput. Interact., vol. 19, 2004.

Digital Library

Google Scholar

[4]

P. D. Marasco, K. Kim, J. E. Colgate, M. A. Peshkin and T. A. Kuike, Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees, Brain, vol. 134, pp. 747--758, 2011.

Crossref

Google Scholar

[5]

A. Blank, A. M. Okamura and K. J. Kuchenbecker, Identifying the Role of Proprioception in Upper-Limb Prosthesis Control: Studies on Targeted Motion, ACM Trans. on Applied Perception, vol. 7, pp. 1--23, 2010.

Digital Library

Google Scholar

[6]

S. Gallagher, How the Body Shapes the Mind, New York: Oxford University Press, 224 pages, 2006.

Google Scholar

[7]

J. H. Hochberg and J. P. Donoghue, Sensors for Brain-Computer Interfaces, IEEE Eng. in Medicine and Biology Magazine, vol. 25, pp. 32--38, 2006.

Crossref

Google Scholar

[8]

Y. M. Chi, T. P. Jung and G. Cauwenberghs, Dry-Contact and Noncontact Biopotential Electrodes: Methodological Review, IEEE Reviews in Biomedical Engineering, vol. 3, pp. 106--119, 2010.

Google Scholar

[9]

L. Srinivasan, U. T. Eden, S. K. Mitter and E. N. Brown, General-purpose filter design for neural prosthetic devices, J. Neurophysiol. vol. 98, pp. 2456--75, 2007.

Crossref

Google Scholar

[10]

L. Ding, Y. Ni, J. Sweeney and B. He, Sparse cortical current density imaging in motor potentials induced by finger movement, J. Neural Eng., vol. 8, 2011.

Crossref

Google Scholar

[11]

C. J. Long, P. L. Purdon, S. Temereanca, N. U. Desai, M. S. Hämäl#228;inen and E. N. Brown, State-space solutions to the dynamic magnetoencephalography inverse problem using high performance computing, Ann. Appl. Stat., vol. 5, pp. 1207--1228, 2011.

Crossref

Google Scholar

[12]

R. A. Newcombe, S. Lovegrove and A. J. Davison, DTAM: Dense tracking and mapping in real-time, in Proc. of the Int. Conf. on Computer Vision, 2011.

Digital Library

Google Scholar

[13]

E. Todorov, W. Li and X. Pan, From task parameters to motor synergies: A hierarchical framework for approximately-optimal feedback control of redundant manipulators, J. of Robotic Systems, vol. 22, pp. 691--710, 2005.

Digital Library

Google Scholar

Cited By

View all

Toedtheide AFortunić EKühn JJensen EHaddadin S(2024)A transhumeral prosthesis with an artificial neuromuscular system: Sim2real-guided design, modeling, and controlThe International Journal of Robotics Research10.1177/0278364923121871943:7(942-980)Online publication date: 20-Feb-2024
https://doi.org/10.1177/02783649231218719
Gholizadeh HamlAbadi KLaamarti FEl Saddik A(2024)Meta-Review on Brain-Computer Interface (BCI) in the MetaverseACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369610920:12(1-42)Online publication date: 14-Sep-2024
https://dl.acm.org/doi/10.1145/3696109
Martin HDonaw JKelly RJung YKim J(2014)A novel approach of prosthetic arm control using computer vision, biosignals, and motion capture2014 IEEE Symposium on Computational Intelligence in Robotic Rehabilitation and Assistive Technologies (CIR2AT)10.1109/CIRAT.2014.7009737(26-30)Online publication date: Dec-2014
https://doi.org/10.1109/CIRAT.2014.7009737

Index Terms

Integrating human and computer vision with EEG toward the control of a prosthetic arm
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
      1. Robotic components

Recommendations

Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance
Abstract
Control of a robotic arm using a brain–computer interface (BCI) for reach and grasp activities is one of the most fascinating applications for some severely disabled people, which is especially challenging for the non-invasive BCIs ...
Neural Control using EEG as a BCI Technique for Low Cost Prosthetic Arms
IJCCI 2014: Proceedings of the International Joint Conference on Computational Intelligence - Volume 3

There have been significant advancements in brain computer interface (BCI) techniques using EEG-like methods. EEG can serve as non-invasive BMI technique, to control devices like wheelchairs, cursors and robotic arm. In this paper, we discuss the use of ...
Utilizing human vision and computer vision to direct a robot in a semi-structured environment via task-level commands
IROS '95: Proceedings of the International Conference on Intelligent Robots and Systems-Volume 1 - Volume 1

A novel approach to directing a highly autonomous robot operating in a semi-structured environment is presented. In this approach, the human operator assists the robot in perceiving unexpected situations in the environment through simple point-and-click ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

March 2012

518 pages

ISBN:9781450310635

DOI:10.1145/2157689

General Chairs:
Holly Yanco
University of Massachusetts Lowell, USA
,
Aaron Steinfeld
Carnegie Mellon University, USA
,
Program Chairs:
Vanessa Evers
University of Amsterdam, The Netherlands
,
Odest Chadwicke Jenkins
Brown University, USA

In-Cooperation

IEEE-RAS: Robotics and Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 March 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Abstract

Conference

HRI'12

Sponsor:

HRI'12: International Conference on Human-Robot Interaction

March 5 - 8, 2012

Massachusetts, Boston, USA

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
389
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)4

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Toedtheide AFortunić EKühn JJensen EHaddadin S(2024)A transhumeral prosthesis with an artificial neuromuscular system: Sim2real-guided design, modeling, and controlThe International Journal of Robotics Research10.1177/0278364923121871943:7(942-980)Online publication date: 20-Feb-2024
https://doi.org/10.1177/02783649231218719
Gholizadeh HamlAbadi KLaamarti FEl Saddik A(2024)Meta-Review on Brain-Computer Interface (BCI) in the MetaverseACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369610920:12(1-42)Online publication date: 14-Sep-2024
https://dl.acm.org/doi/10.1145/3696109
Martin HDonaw JKelly RJung YKim J(2014)A novel approach of prosthetic arm control using computer vision, biosignals, and motion capture2014 IEEE Symposium on Computational Intelligence in Robotic Rehabilitation and Assistive Technologies (CIR2AT)10.1109/CIRAT.2014.7009737(26-30)Online publication date: Dec-2014
https://doi.org/10.1109/CIRAT.2014.7009737

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance

Neural Control using EEG as a BCI Technique for Low Cost Prosthetic Arms

Utilizing human vision and computer vision to direct a robot in a semi-structured environment via task-level commands