[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Textural feature descriptors for a static and dynamic hand gesture recognition system

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Hand gesture recognition has become one of the most important directions in human-computer interaction (HCI) research. Despite recent advances in this area, the development of methods and techniques to correctly recognize gestures is still ongoing. In this paper, a hand gesture and sign recognition system (HGRS/SGRS) based on textural features is implemented using a local binary pattern (LBP), local directional pattern (LDP), local optimal-oriented pattern (LOOP) and local Gabor binary pattern histogram sequence (LGBPHS). In terms of feature extraction, we introduce Modifiedi-LOOP, a modified local texture descriptor for HGRS and SGRS to improve the efficiency of our system. The experiments are carried out on five datasets, for Arabic, American Alphabet Sign languages and dynamic gestures where the proposed Mi-LOOP as well as LGBPHS achieve satisfactory simulation results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data Availability

1. The dataset generated during the current study are available in the [IEEE] repository, [https://ieee-dataport.org/open-access/static-hand-gesture-asl-dataset].

2. The dataset generated during the current study are available in the [ArSL2018 Arabic] repository, [https://data.mendeley.com/datasets/y7pckrw6z2/1].

3. The dataset generated during the current study are available in the [Jochen Triesch] repository, [https://www.idiap.ch/webarchives/sites/www.idiap.ch/resource/gestures].

4. The dataset generated during the current study are available in the [Sebastien Marcel] repository, [https://www.idiap.ch/webarchives/sites/www.idiap.ch/resource/gestures].

5. The dataset (named El Halawani) generated during the current study are available from the corresponding author on reasonable request.

References

  1. Adeyanju I, Bello O, Azeez M (2022) Development of an american sign language recognition system using canny edge and histogram of oriented gradient. Niger J Technol Dev 19(3):195–205

    Google Scholar 

  2. Agab S, Chelali F (2018) Hog and hoof spatio-temporal descriptors for gesture recognition. In: 2018 International Conference on Signal, Image, Vision and their Applications (SIVA), IEEE, pp 1–7

  3. Agab S, Chelali F (2019) Dynamic hand gesture recognition based on textural features. In: 2019 International Conference on Advanced Electrical Engineering (ICAEE), IEEE, pp 1–6

  4. Agab SE, Ferhat R, Chelali FZ (2021) Robust hand gesture recognition based on mwld and ltep descriptors. In: 2021 International conference on artificial intelligence for cyber security systems and privacy (AI-CSP), IEEE, pp 1–5

  5. Al-Jarrah O, Halawani A (2001) Recognition of gestures in arabic sign language using neuro-fuzzy systems. Artif Intell 133(1-2):117–138

    Google Scholar 

  6. Ali MHK, Azman MA, Ismail ZH et al (2012) Real-time hand gestures system for mobile robots control. Procedia Engineering 41:798–804

    Google Scholar 

  7. Aljuhani R, Alfaidi A, Alshehri B, Alwadei H, Aldhahri E, Aljojo N (2022) Arabic sign language recognition using convolutional neural network and mobilenet. Arab J Sci Eng p 1–8

  8. Alshomrani S, Aljoudi L, Arif M (2021) Arabic and american sign languages alphabet recognition by convolutional neural network. Advances in Science and Technology Research Journal 15(4)

  9. Anand D et al (2021) An improved hand gesture recognition system based on optimized msvm and sift feature extraction algorithm. In: Proceedings of international conference on big data, machine learning and their applications, Springer, pp 365–377

  10. Angelopoulou A, Garcia-Rodriguez J, Orts-Escolano S, Kapetanios E, Liang X, Woll B, Psarrou A (2019) Evaluation of different chrominance models in the detection and reconstruction of faces and hands using the growing neural gas network. Pattern Anal Applic 22(4):1667–1685

    MathSciNet  Google Scholar 

  11. Assaleh K, Al-Rousan M (2005) Recognition of arabic sign language alphabet using polynomial classifiers. EURASIP J Adv Signal Process 2005(13):1–10

    Google Scholar 

  12. Bastos IL, Angelo MF, Loula AC (2015) Recognition of static gestures applied to brazilian sign language (libras). In: 2015 28th SIBGRAPI conference on graphics, patterns and images, IEEE, pp 305–312

  13. Bose SR, Kumar VS (2022) In-situ recognition of hand gesture via enhanced xception based single-stage deep convolutional neural network. Exp Syst Appl 193:116–427

    Google Scholar 

  14. Chakraborti T, McCane B, Mills S, Pal U (2018) Loop descriptor: local optimal-oriented pattern. IEEE Signal Process Lett 25(5):635–639

    Google Scholar 

  15. Ċevik N, Ċevik T (2020) A novel high-performance holistic descriptor for face retrieval. Pattern Anal Appl 23(1):371–383

    Google Scholar 

  16. Damaneh MM, Mohanna F, Jafari P (2023) Static hand gesture recognition in sign language based on convolutional neural network with feature extraction method using orb descriptor and gabor filter. Exp Syst Appl 211:118–559

    Google Scholar 

  17. Davies E (1986) Constraints on the design of template masks for edge detection. Pattern Recognit Lett 4(2):111–120

    MathSciNet  Google Scholar 

  18. Dong C, Leu MC, Yin Z (2015) American sign language alphabet recognition using microsoft kinect. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 44–52

  19. Dong J, Xia Z, Zhao Q (2021) Augmented reality assisted assembly training oriented dynamic gesture recognition and prediction. Appl Sci 11(21):9789

    Google Scholar 

  20. Elmezain M, Al-Hamadi A, Appenrodt J, Michaelis B (2008) A hidden markov model-based continuous gesture recognition system for hand motion trajectory. In: 2008 19th international conference on pattern recognition, IEEE, pp 1–4

  21. Elmezain M, Al-Hamadi A, Krell G, El-Etriby S, Michaelis B (2007) Gesture recognition for alphabets from hand motion trajectory using hidden markov models. In: 2007 IEEE international symposium on signal processing and information technology, IEEE, pp 1192–1197

  22. Elouariachi I, Benouini R, Zenkouar K, Zarghili A (2020) Robust hand gesture recognition system based on a new set of quaternion tchebichef moment invariants. Pattern Anal Appl 23(3):1337–1353

    MathSciNet  Google Scholar 

  23. Fagiani M, Principi E, Squartini S, Piazza F (2015) Signer independent isolated italian sign recognition based on hidden markov models. Pattern Anal Applic 18(2):385–402

    MathSciNet  Google Scholar 

  24. Ferhat R, Chelali FZ et al (2020) Static and dynamic hand gesture recognition system using contourlet transform. In: The proceedings of the 3rd international conference on smart city applications, Springer, pp 844–854

  25. Halidou A, You X, Hamidine M, Etoundi RA, Diakite LH et al (2014) Fast pedestrian detection based on region of interest and multi-block local binary pattern descriptors. Comput Electric Eng 40(8):375–389

    Google Scholar 

  26. Haria A, Subramanian A, Asokkumar N, Poddar S, Nayak JS (2017) Hand gesture recognition for human computer interaction. Procedia Comput Sci 115:367–374

    Google Scholar 

  27. Heikkilä M, Pietikäinen M, Schmid C (2006) Description of interest regions with center-symmetric local binary patterns. In: Computer vision, graphics and image processing, Springer, pp 58–69

  28. Huang H, Chong Y, Nie C, Pan S (2019) Hand gesture recognition with skin detection and deep learning method. In: Journal of physics: Conference series, IOP Publishing, vol 1213, p 022001

  29. Ibrahim NB, Selim MM, Zayed HH (2018) An automatic arabic sign language recognition system (arslrs). J King Saud Univ-Comput Inf Sci 30(4):470–477

    Google Scholar 

  30. Jabid T, Kabir MH, Chae O (2010) Robust facial expression recognition based on local directional pattern. ETRI J 32(5):784–794

    Google Scholar 

  31. Jabid T, Kabir MH, Chae O (2010) Local directional pattern (ldp) for face recognition. In: 2010 Digest of technical papers international conference on consumer electronics (ICCE), IEEE, pp 329–330

  32. Jasim M, Hasanuzzaman M (2014) Sign language interpretation using linear discriminant analysis and local binary patterns. In: 2014 International conference on informatics, electronics & vision (ICIEV), IEEE, pp 1–5

  33. Karanwal S, Diwakar M (2021) Neighborhood and center difference-based-lbp for face recognition. Pattern Anal Applic 24(2):741–761

    Google Scholar 

  34. Kaur B, Joshi G (2016) Lower order krawtchouk moment-based feature-set for hand gesture recognition. Advances in Human-Computer Interaction p 2016

  35. Kim J, Yu S, Kim D, Toh KA, Lee S (2017) An adaptive local binary pattern for 3d hand tracking. Pattern Recogn 61:139–152

    Google Scholar 

  36. Kowdiki M, Khaparde A (2021) Automatic hand gesture recognition using hybrid meta-heuristic-based feature selection and classification with dynamic time warping. Comput Sci Rev 39:100320

    Google Scholar 

  37. Lahiani H, Kherallah M, Neji M (2017) Hand gesture recognition system based on local binary pattern approach for mobile devices. In: International conference on intelligent systems design and applications, Springer, pp 180–190

  38. Lahiani H, Neji M (2018) Hand gesture recognition method based on hog-lbp features for mobile devices. Procedia Computer Science 126:254–263

    Google Scholar 

  39. Latif G, Mohammad N, AlKhalaf R, AlKhalaf R, Alghazo J, Khan M (2020) An automatic arabic sign language recognition system based on deep cnn: an assistive system for the deaf and hard of hearing. Int J Comput Digit Syst 9(4):715–724

    Google Scholar 

  40. Li L, Qin S, Lu Z, Zhang D, Xu K, Hu Z (2021) Real-time one-shot learning gesture recognition based on lightweight 3d inception-resnet with separable convolutions. Pattern Anal Applic 24(3):1173–1192

    Google Scholar 

  41. Liu Wf, Li Sj, Wang Yj (2009) Automatic facial expression recognition based on local binary patterns of local areas. In: 2009 WASE International conference on information engineering, IEEE, vol 1, pp 197–200

  42. Maqueda AI, del Blanco CR, Jaureguizar F (2015) Human–computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns. Comput Vis Image Underst 141:126–137

    Google Scholar 

  43. Meena K, Suruliandi A (2011) Local binary patterns and its variants for face recognition. In: 2011 International conference on recent trends in information technology (ICRTIT), IEEE, pp 782–786

  44. Mohanty A, Rambhatla SS, Sahay RR (2017) Deep gesture: static hand gesture recognition using cnn. In: Proceedings of international conference on computer vision and image processing, Springer, pp 449–461

  45. Nasri S, Behrad A, Razzazi F (2015) A novel approach for dynamic hand gesture recognition using contour-based similarity images. Int J Comput Math 92(4):662–685

    Google Scholar 

  46. Palanisamy G, Thangaswamy SS (2022) An efficient hand gesture recognition based on optimal deep embedded hybrid convolutional neural network-long short term memory network model. Concurrency and Computation: Practice and Experience p e7109

  47. Parvathy P, Subramaniam K, Prasanna Venkatesan G, Karthikaikumar P, Varghese J, Jayasankar T (2021) Development of hand gesture recognition system using machine learning. J Ambient Intell Humanized Comput 12 (6):6793–6800

    Google Scholar 

  48. Popov PA, Laganiere R (2022) Long hands gesture recognition system: 2 step gesture recognition with machine learning and geometric shape analysis. Multimed Tools Appl pp 1–32

  49. Ren Z, Yuan J, Zhang Z (2011) Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: Proceedings of the 19th ACM international conference on Multimedia, pp 1093–1096

  50. Ruichek Y, Chetverikov D, Tarawneh AS et al (2022) Local ternary pattern based multi-directional guided mixed mask (mdgmm-ltp) for texture and material classification. Expert Systems with Applications p 117646

  51. Sadeddine K, Djeradi R, Chelali FZ, Djeradi A (2018) Recognition of static hand gesture. In: 2018 6th international conference on multimedia computing and systems (ICMCS), IEEE, pp 1–6

  52. Sahoo JP, Ari S, Ghosh DK (2018) Hand gesture recognition using dwt and f-ratio based feature descriptor. IET Image Process 12(10):1780–1787

    Google Scholar 

  53. Shanableh T, Assaleh K, Al-Rousan M (2007) Spatio-temporal feature-extraction techniques for isolated gesture recognition in arabic sign language. IEEE Trans Systems, Man, Cybern Part B (Cybernetics) 37(3):641–650

    Google Scholar 

  54. Shanmuganathan V, Yesudhas HR, Khan MS, Khari M, Gandomi AH (2020) R-cnn and wavelet feature extraction for hand gesture recognition with emg signals. Neural Comput Appl 32(21):16723–16736

    Google Scholar 

  55. Shanthakumar VA, Peng C, Hansberger J, Cao L, Meacham S, Blakely V (2020) Design and evaluation of a hand gesture recognition approach for real-time interactions. Multimed Tools Appl 79(25):17707–17730

    Google Scholar 

  56. Sharma S, Singh S (2021) Vision-based hand gesture recognition using deep learning for the interpretation of sign language. Exp Syst Appl 182:115–657

    Google Scholar 

  57. Sheena C, Narayanan N (2021) Static gesture classification and recognition using hog feature parameters and k-nn and svm-based machine learning algorithms. In: Machine vision and augmented intelligence—theory and applications, Springer, pp 157–166

  58. Singh S, Jain A, Kumar D (2012) Recognizing and interpreting sign language gesture for human robot interaction. International Journal of Computer Applications 52(11)

  59. Song T, Zhao H, Liu Z, Liu H, Hu Y, Sun D (2021) Intelligent human hand gesture recognition by local–global fusing quality-aware features. Futur Gener Comput Syst 115:298–303

    Google Scholar 

  60. Starner T, Pentland A (1997) Real-time american sign language recognition from video using hidden markov models. In: Motion-based recognition, Springer, pp 227–243

  61. Stergiopoulou E, Papamarkos N (2009) Hand gesture recognition using a neural network shape fitting technique. Eng Appl Artif Intell 22(8):1141–1158

    Google Scholar 

  62. Subash Kumar T, Nagarajan V (2019) Local curve pattern for content-based image retrieval. Pattern Anal Applic 22(3):1233–1242

    MathSciNet  Google Scholar 

  63. Tang J, Cheng H, Zhao Y, Guo H (2018) Structured dynamic time warping for continuous hand trajectory gesture recognition. Pattern Recogn 80:21–31

    Google Scholar 

  64. Tang H, Liu H, Xiao W, Sebe N (2019) Fast and robust dynamic hand gesture recognition via key frames extraction and feature fusion. Neurocomputing 331:424–433

    Google Scholar 

  65. Tharwat G, Ahmed AM, Bouallegue B (2021) Arabic sign language recognition system for alphabets using machine learning techniques. Journal of Electrical and Computer Engineering p 2021

  66. Torre V, Poggio TA (1986) On edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (2) pp 147–163

  67. Triesch J, Von Der Malsburg C (1996) Robust classification of hand postures against complex backgrounds. In: Proceedings of the second international conference on automatic face and gesture recognition, IEEE, pp 170–175

  68. Tyagi A, Bansal S (2022) Hybrid fist_cnn approach for feature extraction for vision-based indian sign language recognition. Int Arab J Inf Technol 19 (3):403–411

    Google Scholar 

  69. Wang L, He DC (1990) Texture classification using texture spectrum. Pattern Recognit 23(8):905–910

    Google Scholar 

  70. Wu XY (2020) A hand gesture recognition algorithm based on dc-cnn. Multimed Tools Appl 79(13):9193–9205

    Google Scholar 

  71. Yang MH, Ahuja N, Tabb M (2002) Extraction of 2d motion trajectories and its application to hand gesture recognition. IEEE Trans Pattern Anal Mach Intell 24(8):1061–1074

    Google Scholar 

  72. Yang L, Chen J, Zhu W (2020) Dynamic hand gesture recognition based on a leap motion controller and two-layer bidirectional recurrent neural network. Sensors 20(7):2106

    Google Scholar 

  73. Zhang W, Shan S, Gao W, Chen X, Zhang H (2005) Local gabor binary pattern histogram sequence (lgbphs): a novel non-statistical model for face representation and recognition. In: 10th IEEE international conference on computer vision (ICCV’05) vol 1, IEEE, vol 1, pp 786–791

  74. Zheng J, Feng Z, Xu C, Hu J, Ge W (2017) Fusing shape and spatio-temporal features for depth-based dynamic hand gesture recognition. Multimed Tools Appl 76(20):20525–20544

    Google Scholar 

  75. Zhong F, Zhang J (2013) Face recognition with enhanced local directional patterns. Neurocomputing 119:375–384

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Speech Communication and Signal Processing Laboratory, Faculty of Electrical Engineering University of Sciences and Technology Houari Boumediene (USTHB), El-Alia, Bab-Ezzouar, Algiers, 16111, Algeria

    Roumiassa Ferhat & Fatma Zohra Chelali

Authors
  1. Roumiassa Ferhat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fatma Zohra Chelali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roumiassa Ferhat.

Ethics declarations

Conflict of Interests

Authors declares that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ferhat, R., Chelali, F.Z. Textural feature descriptors for a static and dynamic hand gesture recognition system. Multimed Tools Appl 83, 8165–8187 (2024). https://doi.org/10.1007/s11042-023-15410-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15410-0

Keywords

Search

Navigation