[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Trajectory-based recognition of in-air handwritten Assamese words using a hybrid classifier network

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

In-air handwriting is a rapidly emerging human–machine interactive paradigm that helps users to write and communicate naturally and intuitively in free space. In this paper, we develop a hybrid one-dimensional convolutional recurrent attention framework model for in-air handwritten Assamese word recognition (IAHAWR) which associates an encoder and a decoder framework for efficiently recognizing air-written words. The encoder is an assimilation of 1D convolutional neural network and bidirectional gated recurrent unit neural network for input trajectory feature sequence learning, while the decoder is an attention-based gated recurrent unit for predicting the target words. In contrast to conventional pen-based handwriting, in-air handwriting is intricate in the sense that the handwriting is finished in a single continuous stroke giving rise to many irrelevant motions called ligatures in between adjacent character strokes. So, we have imbibed a salient stroke extraction and a critical point detection scheme into the proposed system, which helps in removal of insignificant ligatures thus enhancing the recognition performance. Further, air-writing trajectories contain intermittent jitters and suffer wide variations in writing patterns due to unrestricted writing in free space. So, we incorporate a multistage word normalization methodology which generalizes the air-written patterns and aids in efficient recognition. We have assessed the performance of our proposed system on an air-written Assamese word dataset as well as some air-written Latin words. Experimental evaluation connotes that our proposed IAHAWR system can effectively procure characteristic information from air-writing sequences and provides comparable recognition accuracy and computational performance with that of other state-of-the-art recognition frameworks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Gan, J., Wang, W.: In-air handwritten English word recognition using attention recurrent translator. Neural Comput. Appl. 31, 1–18 (2017)

    Google Scholar 

  2. Choudhury, A., Sarma, K.K.: A CNN-LSTM based ensemble framework for in-air handwritten Assamese character recognition. Multimedia Tools Appl. 80(28), 35649–35684 (2021)

    Article  Google Scholar 

  3. Chen, M., AlRegib, G., Juang, B.H.: Air-writing recognition—part I: modeling and recognition of characters, words, and connecting motions. IEEE Trans. Hum. Mach. Syst. 46(3), 403–413 (2015)

    Article  Google Scholar 

  4. Ren, H., Wang, W., Lu, K., Zhou, J., Yuan, Q.: An end-to-end recognizer for in-air handwritten Chinese characters based on a new recurrent neural networks. In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), 10–14 July 2017, Hong Kong, China, pp. 841–846 (2017)

  5. Kumar, P., Saini, R., Roy, P.P., Pal, U.: A lexicon-free approach for 3D handwriting recognition using classifier combination. Pattern Recognit. Lett. 103, 1–7 (2018)

    Article  Google Scholar 

  6. Gan, J., Wang, W., Lu, K.: A unified CNN-RNN approach for in-air handwritten English word recognition. In: Proceedings of IEEE International Conference on Multimedia and Expo (ICME), 23–27 July 2018, San Diego, CA, USA, pp. 1–6 (2018)

  7. Amma, C., Georgi, M., Schultz, T.: Airwriting: hands-free mobile text input by spotting and continuous recognition of 3D-space handwriting with inertial sensors. In: Proceedings of IEEE 16th International Symposium on Wearable Computers, pp. 52–59 (2012)

  8. Chen, M., AlRegib, G., Juang, B.H.: Air-writing recognition-part II: detection and recognition of writing activity in continuous stream of motion data. IEEE Trans. Hum. Mach. Syst. 46(3), 436–444 (2016)

    Article  Google Scholar 

  9. Kumar, P., Saini, R., Roy, P.P., Dogra, D.P.: 3D text segmentation and recognition using leap motion. Multimedia Tools Appl. 76(15), 16491–16510 (2017)

    Article  Google Scholar 

  10. Vikram, S., Li, L., Russell, S.: Handwriting and gestures in the air, recognizing on the fly. In: Proceedings of the CHI, vol. 13, pp. 1179–1184 (2013)

  11. Chiang, C.C., Wang, R.H., Chen, B.R.: Recognizing arbitrarily connected and superimposed handwritten numerals in intangible writing interfaces. Pattern Recognit. 61, 15–28 (2017)

    Article  Google Scholar 

  12. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2008)

    Article  Google Scholar 

  13. Choudhury, A., Sarma, K.K.: A vision-based framework for spotting and segmentation of gesture-based Assamese characters written in the air. J. Inf. Technol. Res. JITR 14(1), 70–91 (2021)

    Article  Google Scholar 

  14. Luong, M. T., Pham, H., Manning, C. D.: Effective approaches to attention-based neural machine translation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 1412–1421 (2015)

  15. Yang, C., Ku, B., Han, D.K., Ko, H.: Alpha-numeric hand gesture recognition based on fusion of spatial feature modelling and temporal feature modelling. Electron. Lett. 52(20), 1679–1681 (2016)

    Article  Google Scholar 

  16. Roy, P., Ghosh, S., Pal, U.: A CNN based framework for unistroke numeral recognition in air-writing. In: Proceedings of 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), 05–08 August 2018, Niagara Falls, NY, USA, pp. 404–409 (2018)

  17. Rahman, A., Roy, P., Pal, U.: Continuous motion numeral recognition using RNN architecture in air-writing environment. In: Proceedings of Asian Conference on Pattern Recognition, pp. 76–90. Springer (2019)

  18. Gan, J., Wang, W., Lu, K.: A new perspective: recognizing online handwritten Chinese characters via 1-dimensional CNN. Inf. Sci. 478, 375–390 (2019)

    Article  Google Scholar 

  19. Agarwal, C., Dogra, D. P., Saini, R., Roy, P. P.: Segmentation and recognition of text written in 3d using leap motion interface. In: Proceedings of IEEE IAPR Asian Conference on Pattern Recognition (ACPR), 03–06 November 2015, Kuala Lumpur, Malaysia, pp. 539–543 (2015)

  20. Wang, Z.R., Du, J., Wang, W.C., Zhai, J.F., Hu, J.S.: A comprehensive study of hybrid neural network hidden Markov model for offline handwritten Chinese text recognition. Int. J. Doc. Anal. Recognit. IJDAR 21(4), 241–251 (2018)

    Article  Google Scholar 

  21. Wang, Z.R., Du, J., Wang, J.M.: Writer-aware CNN for parsimonious HMM-based offline handwritten Chinese text recognition. Pattern Recognit. 100, 107102 (2020)

    Article  Google Scholar 

  22. Xie, Z., Sun, Z., Jin, L., Feng, Z., Zhang, S.: Fully convolutional recurrent network for handwritten Chinese text recognition. In: Proceedings of IEEE 23rd International Conference on Pattern Recognition (ICPR), pp. 4011–4016 (2016)

  23. Ahmed, S.B., Naz, S., Razzak, M.I., Rashid, S.F., Afzal, M.Z., Breuel, T.M.: Evaluation of cursive and non-cursive scripts using recurrent neural networks. Neural Comput. Appl. 27(3), 603–613 (2016)

    Article  Google Scholar 

  24. Kumar, P., Saini, R., Roy, P.P., Dogra, D.P.: Study of text segmentation and recognition using leap motion sensor. IEEE Sens. J. 17(5), 1293–1301 (2017)

    Article  Google Scholar 

  25. Phung, S.L., Bouzerdoum, A., Chai, D.: Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 148–154 (2005)

    Article  Google Scholar 

  26. Choudhury, A., Talukdar, A.K., Sarma, K.K., Bhuyan, M.K.: An adaptive thresholding-based movement epenthesis detection technique using hybrid feature set for continuous fingerspelling recognition. SN Comput. Sci. 2(2), 1–21 (2021)

    Article  Google Scholar 

  27. Wilson, J.N., Ritter, G.X.: Handbook of Computer Vision Algorithms in Image Algebra. CRC Press, Boca Raton (2000)

    MATH  Google Scholar 

  28. Gose, E., Johnsonbaugh, R., Jost, S.: Pattern Recognition and Image Analysis. Prentice-Hall, Englewood Cliffs (1996)

    Google Scholar 

  29. Petrick, N., Chan, H.P., Sahiner, B., Helvie, M.A.: Combined adaptive enhancement and region-growing segmentation of breast masses on digitized mammograms. Med. Phys. 26(8), 1642–1654 (1999)

    Article  Google Scholar 

  30. Gallagher, N.B.: Savitzky–Golay Smoothing and Differentiation Filter. https://eigenvector.com/wp-content/uploads/2020/01/SavitzkyGolay.pdf

  31. Zhang, X.Y., Yin, F., Zhang, Y.M., Liu, C.L., Bengio, Y.: Drawing and recognizing Chinese characters with recurrent neural network. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 849–862 (2017)

    Article  Google Scholar 

  32. Liu, Y.K., Žalik, B., Wang, P.J., Podgorelec, D.: Directional difference chain codes with quasi-lossless compression and run-length encoding. Signal Process. Image Commun. 27(9), 973–984 (2012)

    Article  Google Scholar 

  33. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Baltimore, Maryland, pp. 655–665. Association for Computational Linguistics (2014)

  34. Chollet, F.: Deep Learning with Python. Simon and Schuster, New York (2021)

    Google Scholar 

  35. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)

  36. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  37. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Ministry of Electronics and Information Technology (MeitY), Government of India for providing the Visvesvaraya PhD Fellowship Scheme for conducting the research. The authors also wish to thank all the editors and anonymous reviewers for their constructive advice.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Gauhati University, Guwahati, Assam, 781014, India

    Ananya Choudhury & Kandarpa Kumar Sarma

Authors
  1. Ananya Choudhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kandarpa Kumar Sarma
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AC formulated the methodology and algorithms incorporated in the work, designed and implemented the proposed framework, performed experimental evaluations and wrote the manuscript. KKS defined and supervised the research, assessed the framework’s functionality and provided suggestions for revision of the manuscript.

Corresponding author

Correspondence to Ananya Choudhury.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choudhury, A., Sarma, K.K. Trajectory-based recognition of in-air handwritten Assamese words using a hybrid classifier network. IJDAR 26, 375–400 (2023). https://doi.org/10.1007/s10032-022-00426-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-022-00426-3

Keywords

Search

Navigation