[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Gesture recognition based on sEMG using multi-attention mechanism for remote control

  • S.I.: IoT-based Health Monitoring System
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Remote controlling using surface electromyography (sEMG) plays a more and more important role in a human–robot interface, such as controlling prosthesis devices, and exoskeleton. Different gestures are controlled by the cooperation of muscle groups, and sEMG represent the energy of the activated muscle fibers. With the limit of the low performance of wearable device, this article proposed a remote hand gesture recognized system based on deep learning framework of multi-attention mechanism convolutional neural network using sEMG energy to decoding hand gestures with remote server host. In the first part, an adaptive channel weighted method is proposed on multi-channel data of sEMG for enhancing the related feature map of sEMG and reducing the feature map low contribution of sEMG. The second part is improving the shortcuts by adding adaptively weighted instead of a simple short concatenation of feature maps. A novel multi-attention deep learning framework with multi-view (MMDL) for hand gestures recognition is proposed in our study, using sEMG. We verify the MMDL framework on myo dataset, myoUp dataset, and ninapro DB5, with the average accuracy 99.27%, 97.86%, and 97.0%, which is improved by 0.46%, 18.88%, 7% compared with prior works. In addition, the framework can classify seven hand gestures with 99.92% accuracy on ours datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Kundu AS, Mazumder O, Lenka PK, Bhaumik S (2018) Hand gesture recognition based omnidirectional wheelchair control using IMU and EMG sensors. J Intell Robot Syst Theory Appl 91:529–541. https://doi.org/10.1007/s10846-017-0725-0

    Article  Google Scholar 

  2. Çoban M, Gelen G (2018) Wireless teleoperation of an industrial robot by using myo arm band. In: 2018 International conference on artificial intelligence and data processing (IDAP), Malatya, IEEE. https://doi.org/10.1109/IDAP.2018.8620789

  3. Petkos K, Koutsoftidis S, Guiho T et al (2019) A high-performance 8 nV/√Hz 8-channel wearable and wireless system for real-time monitoring of bioelectrical signals. J Neuroeng Rehabil 16:1–24. https://doi.org/10.1186/s12984-019-0629-2

    Article  Google Scholar 

  4. Chen X, Zeng Y, Yin Y (2017) Improving the transparency of an exoskeleton knee joint based on the understanding of motor intent using energy kernel method of EMG. IEEE Trans Neural Syst Rehabil Eng 25:577–588. https://doi.org/10.1109/TNSRE.2016.2582321

    Article  Google Scholar 

  5. Pancholi S, Joshi AM (2020) Advanced energy kernel-based feature extraction scheme for improved EMG-PR-based prosthesis control against force variation. IEEE Trans Cybern. https://doi.org/10.1109/tcyb.2020.3016595

    Article  Google Scholar 

  6. Kim JS, Pan SB (2017) A study on EMG-based biometrics. Journal of Internet Services and Information Security https://doi.org/10.22667/JISIS.2017.05.31.019

  7. Gillette JC, Stephenson ML. (2018) EMG analysis of an upper body exoskeleton during automotive assembly. In: 42nd Annual Meeting of the American Society of Biomechanics, Rochester, MN. https://www.researchgate.net/publication/327187565_EMG_analysis_of_an_upper_body_exoskeleton_during_automotive_assembly

  8. Morais GD, Neves LC, Masiero AA, Castro MCF (2016) Application of Myo Armband System to control a robot interface. BIOSIGNALS 4:227–231. https://doi.org/10.5220/0005706302270231

    Article  Google Scholar 

  9. Kim KT, Guan C, Lee SW (2020) A subject-transfer framework based on single-trial EMG analysis using convolutional neural networks. IEEE Trans Neural Syst Rehabil Eng 28:94–103. https://doi.org/10.1109/TNSRE.2019.2946625

    Article  Google Scholar 

  10. Zhai X, Jelfs B, Chan RHM, Tin C (2017) Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network. Front Neurosci 11:1–11. https://doi.org/10.3389/fnins.2017.00379

    Article  Google Scholar 

  11. Meattini R, Benatti S, Scarcia U et al (2018) An sEMG-based human-robot interface for robotic hands using machine learning and synergies. IEEE Trans Compon Packag Manuf Technol 8:1149–1158. https://doi.org/10.1109/TCPMT.2018.2799987

    Article  Google Scholar 

  12. Lv X, Dai C, Chen L et al (2020) A robust real-time detecting and tracking framework for multiple kinds of unmarked object. Sensors (Switzerland) 20:1–17. https://doi.org/10.3390/s20010002

    Article  Google Scholar 

  13. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: International Conference on Machine Learning. pp. 1–6

  14. Szegedy C, Vanhoucke V, Ioffe S, et al. (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), Las Vegas, IEEE. https://doi.org/10.1109/CVPR.2016.308

  15. Huang G, Liu Z, Van Der Maaten L, et al. (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), Honolulu, IEEE. https://doi.org/10.1109/CVPR.2017.243

  16. Atzori M, Gijsberts A, Heynen S, et al (2012) Building the Ninapro database: A resource for the biorobotics community. In: 2012 4th IEEE RAS & EMBS international CONFERENCE ON BIOMEDICAL ROBOTICS AND biomechatronics (BioRob), Rome, IEEE. https://doi.org/10.1109/BioRob.2012.6290287

  17. Wei W, Dai Q, Wong Y et al (2019) Surface-electromyography-based gesture recognition by multi-view deep learning. IEEE Trans Biomed Eng 66:2964–2973. https://doi.org/10.1109/TBME.2019.2899222

    Article  Google Scholar 

  18. Côté-Allard U, Fall CL, Drouin A et al (2019) Deep learning for electromyographic hand gesture signal classification using transfer learning. IEEE Trans Neural Syst Rehabil Eng 4320:1–11

    Google Scholar 

  19. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, IEEE. https://doi.org/10.1109/CVPR.2018.00745

  20. Rodriguez A, Zhang H, Klaminder J et al (2018) ToxTrac: a fast and robust software for tracking organisms. Methods Ecol Evol 9:460–464. https://doi.org/10.1111/2041-210X.12874

    Article  Google Scholar 

  21. Yang W, Yang D, Liu Y, Liu H (2019) Decoding simultaneous multi-DOF wrist movements from raw emg signals using a convolutional neural network. IEEE Trans Human-Mach Syst 49:411–420. https://doi.org/10.1109/THMS.2019.2925191

    Article  Google Scholar 

  22. Tam S, Boukadoum M, Campeau-Lecours A, Gosselin B (2020) A fully embedded adaptive real-time hand gesture classifier leveraging HD-sEMG and deep learning. IEEE Trans Biomed Circuits Syst 14:232–243. https://doi.org/10.1109/TBCAS.2019.2955641

    Article  Google Scholar 

  23. Jia G, Lam H-K, Liao J, Wang R (2020) Classification of electromyographic hand gesture signals using machine learning techniques. Neurocomputing 401:236–248. https://doi.org/10.1016/j.neucom.2020.03.009

    Article  Google Scholar 

  24. Ameri A, Akhaee MA, Scheme E, Englehart K (2020) A deep transfer learning approach to reducing the effect of electrode shift in EMG pattern recognition-based control. IEEE Trans Neural Syst Rehabil Eng 28:370–379. https://doi.org/10.1109/TNSRE.2019.2962189

    Article  Google Scholar 

  25. Geng W, Du Y, Jin W et al (2016) Gesture recognition by instantaneous surface EMG images. Sci Rep 6:1–8. https://doi.org/10.1038/srep36571

    Article  Google Scholar 

  26. Wei W, Wong Y, Du Y et al (2019) A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface. Pattern Recognit Lett 119:131–138. https://doi.org/10.1016/j.patrec.2017.12.005

    Article  Google Scholar 

  27. Rahimian E, Zabihi S, Asif A et al (2021) FS-HGR: Few-shot learning for hand gesture recognition via ElectroMyography. IEEE Trans Neural Syst Rehabil Eng 29:1004–1015. https://doi.org/10.1109/TNSRE.2021.3077413

    Article  Google Scholar 

  28. Lignments A (2020) Enhancing adversarial defense by k-winners-take-all. arXiv Prepr arXiv https://arxiv.org/abs/1905.10510, pp. 1–10.

  29. Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition. pp 510–519

  30. Chen X, Yin Y, Fan Y (2014) EMG oscillator model-based energy kernel method for characterizing muscle intrinsic property under isometric contraction. Chinese Sci Bull 59:1556–1567. https://doi.org/10.1007/s11434-014-0147-3

    Article  Google Scholar 

  31. Vigotsky AD, Halperin I, Lehman GJ et al (2018) Interpreting signal amplitudes in surface electromyography studies in sport and rehabilitation sciences. Front Physiol. https://doi.org/10.3389/fphys.2017.00985

    Article  Google Scholar 

  32. Chen Y, Li J, Xiao H, et al (2017) Dual Path Networks. arXiv Prepr arXiv https://arxiv.org/abs/1707.01629, pp. 1–9

  33. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. http://proceedings.mlr.press/v37/ioffe15.html

  34. Lu Z, Tong KY, Zhang X et al (2019) Myoelectric pattern recognition for controlling a robotic hand: a feasibility study in stroke. IEEE Trans Biomed Eng 66:365–372. https://doi.org/10.1109/TBME.2018.2840848

    Article  Google Scholar 

  35. Tsagkas N, Tsinganos P, Skodras A (2019) On the use of deeper CNNs in hand gesture recognition based on sEMG signals. In: 2019 10th International conference on information, intelligence, systems and applications (IISA) (pp. 1-4). IEEE.https://doi.org/10.1109/IISA.2019.8900709

  36. Pizzolato S, Tagliapietra L, Cognolato M et al (2017) Comparison of six electromyography acquisition setups on hand movement classification tasks. PLoS ONE 12:1–17. https://doi.org/10.1371/journal.pone.0186132

    Article  Google Scholar 

  37. Asif AR, Waris A, Gilani SO et al (2020) Performance evaluation of convolutional neural network for hand gesture recognition using EMG. Sensors (Switzerland) 20:1–11. https://doi.org/10.3390/s20061642

    Article  Google Scholar 

  38. Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874. https://doi.org/10.1016/j.patrec.2005.10.010

    Article  Google Scholar 

  39. Côté-allard U, Fall CL, Drouin A et al (2019) Deep learning for electromyographic hand transfer learning. iEEE Trans Neural Syst Rehabil Eng 27:760–771

    Article  Google Scholar 

  40. Chen L, Fu J, Wu Y et al (2020) Hand gesture recognition using compact CNN via surface electromyography signals. Sensors (Switzerland). https://doi.org/10.3390/s20030672

    Article  Google Scholar 

  41. Josephs D, Drake C, Heroy A, et al (2020) sEMG gesture recognition with a simple model of attention. In: SMU Data Sci. Rev., 3(1). http://proceedings.mlr.press/v136/josephs20a

  42. Atzori M, Gijsberts A, Kuzborskij I et al (2015) Characterization of a benchmark database for myoelectric movement classification. IEEE Trans Neural Syst Rehabil Eng 23:73–83. https://doi.org/10.1109/TNSRE.2014.2328495

    Article  Google Scholar 

  43. Shen S, Gu K, Chen XR et al (2019) Movements classification of multi-channel sEMG based on CNN and stacking ensemble learning. IEEE Access 7:137489–137500. https://doi.org/10.1109/ACCESS.2019.2941977

    Article  Google Scholar 

  44. Hu Y, Wong Y, Dai Q et al (2019) sEMG-based gesture recognition with embedded virtual hand poses and adversarial learning. IEEE Access 7:104108–104120. https://doi.org/10.1109/access.2019.2930005

    Article  Google Scholar 

  45. Chui KT, Fung DCL, Lytras MD, Lam TM (2020) Predicting at-risk university students in a virtual learning environment via a machine learning algorithm. Comput Human Behav. https://doi.org/10.1016/j.chb.2018.06.032

    Article  Google Scholar 

Download references

Funding

The authors would like to thank the esteemed reviewers for their insightful and helpful comments. This work was supported by National Key R&D Program of China (Grant No. 2018YFB1307301), National Natural Science Foundation of China (Grant Nos. 91648207, 61673068), the research fund of PLA of China (BWS17J024).

Author information

Authors and Affiliations

  1. Beijing Advanced Innovation Center for Intelligent Robot and System, Beijing Institute of Technology, Beijing, 100081, China

    Xiaodong Lv, Chuankai Dai, Ye Tian, Luyao Chen, Yiran Lang, Rongyu Tang & Jiping He

  2. Xuanwu Hospital Capital Medical University, Capital Medical University, Beijing, 100081, China

    Haijie Liu

Authors
  1. Xiaodong Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chuankai Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haijie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ye Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Luyao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yiran Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Rongyu Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jiping He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ye Tian, Luyao Chen, Yiran Lang or Rongyu Tang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lv, X., Dai, C., Liu, H. et al. Gesture recognition based on sEMG using multi-attention mechanism for remote control. Neural Comput & Applic 35, 13839–13849 (2023). https://doi.org/10.1007/s00521-021-06729-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06729-6

Keywords

Search

Navigation