[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Log in

Scene word recognition from pieces to whole

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Convolutional neural networks (CNNs) have had great success with regard to the object classification problem. For character classification, we found that training and testing using accurately segmented character regions with CNNs resulted in higher accuracy than when roughly segmented regions were used. Therefore, we expect to extract complete character regions from scene images. Text in natural scene images has an obvious contrast with its attachments. Many methods attempt to extract characters through different segmentation techniques. However, for blurred, occluded, and complex background cases, those methods may result in adjoined or over segmented characters. In this paper, we propose a scene word recognition model that integrates words from small pieces to entire after-cluster-based segmentation. The segmented connected components are classified as four types: background, individual character proposals, adjoined characters, and stroke proposals. Individual character proposals are directly inputted to a CNN that is trained using accurately segmented character images. The sliding window strategy is applied to adjoined character regions. Stroke proposals are considered as fragments of entire characters whose locations are estimated by a stroke spatial distribution system. Then, the estimated characters from adjoined characters and stroke proposals are classified by a CNN that is trained on roughly segmented character images. Finally, a lexicon-driven integration method is performed to obtain the final word recognition results. Compared to other word recognition methods, our method achieves a comparable performance on Street View Text and the ICDAR 2003 and ICDAR 2013 benchmark databases. Moreover, our method can deal with recognizing text images of occlusion and improperly segmented text images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Weinman J J, Butler Z, Knoll D, Feild J. Toward integrated scene text reading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(2): 375–387

    Article  Google Scholar 

  2. Ye Q, Doermann D. Text detection and recognition in imagery: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(7): 1480–1500

    Article  Google Scholar 

  3. Zhu Y Y, Yao C, Bai X. Scene text detection and recognition: Recent advances and future trends. Frontiers of Computer Science, 2016, 10(1): 19–36

    Article  Google Scholar 

  4. Goel V, Mishra A, Alahari K, Jawahar C V. Whole is greater than sum of parts: Recognizing scene text words. In: Proceedings of IEEE International Conference on Document Analysis and Recognition. 2013, 398–402

    Google Scholar 

  5. Jaderberg M, Simonyan K, Vedaldi A, Zisserman A. Reading text in the wild with convolutional neural networks. International Journal of Computer Vision, 2016, 116(1): 1–20

    Article  MathSciNet  Google Scholar 

  6. Wang T, Wu D J, Coates A, Ng A Y. End-to-end text recognition with convolutional neural networks. In: Proceedings of IEEE International Conference on Pattern Recognition. 2012, 3304–3308

    Google Scholar 

  7. Mishra A, Alahari K, Jawahar C V. Top-down and bottom-up cues for scene text recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2687–2694

    Google Scholar 

  8. He P, Huang W, Qiao Y, Loy C C, Tang X. Reading scene text in deep convolutional sequences. In: Proceedings of AAAI Conference on Artificial Intelligence. 2016

    Google Scholar 

  9. Shi B G, Bai X, Yao C. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. 2015, arXiv preprint arXiv:1507.05717

    Google Scholar 

  10. Alsharif O, Pineau J. End-to-end text recognition with hybrid HMM maxout models. 2013, arXiv preprint arXiv:1310.1811

    Google Scholar 

  11. Yao C, Bai X, Shi B, Liu W Y. Strokelets: a learned multi-scale representation for scene text recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2014, 4042–4049

    Google Scholar 

  12. Zitnick C L, Dollár P. Edge boxes: locating object proposals from edges. In: Proceedings of European Conference on Computer Vision. 2014, 391–405

    Google Scholar 

  13. Mancas-Thillou C, Gosselin B. Color text extraction with selective metric-based clustering. Computer Vision and Image Understanding, 2007, 107(1): 97–107

    Article  Google Scholar 

  14. Sarawagi S, Cohen W W. Semi-Markov conditional random fields for information extraction. In: Proceedings of International Conference on Neural Information Processing Systems. 2004, 1185–1192

    Google Scholar 

  15. Wang B, Li X F, Liu F, Hu F Q. Color text image binarization based on binary texture analysis. Pattern Recognition Letters, 2005, 26(11): 1650–1657

    Article  Google Scholar 

  16. Seok J H, Kim J H. Scene text recognition using a Hough forest implicit shape model and semi-Markov conditional random fields. Pattern Recognition, 2015, 48(11): 3584–3599

    Article  Google Scholar 

  17. Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2005, 886–893

    Google Scholar 

  18. McCann S, Lowe D G. Spatially local coding for object recognition. In: Proceedings of Asian Conference on Computer Vision. 2012, 204–217

    Google Scholar 

  19. Neubeck A, Van Gool L. Efficient non-maximum suppression. In: Proceedings of IEEE International Conference on Pattern Recognition. 2006, 850–855

    Google Scholar 

  20. de Campos T E, Babu B R, Varma M. Character Recognition in Natural Images. In: Proceedings of International Conference on Computer Vision Theory and Applications. 2009, 273–280

    Google Scholar 

  21. Lucas S M, Panaretos A, Sosa L, Tang A, Wong S, Young R. ICDAR 2003 robust reading competitions. In: Proceedings of IEEE International Conference on Document Analysis and Recognition. 2003

    Google Scholar 

  22. Wang K, Babenko B, Belongie S. End-to-end scene text recognition. In: Proceedings of International Conference on Computer Vision. 2011, 1457–1464

    Google Scholar 

  23. Wang K, Belongie S. Word spotting in the wild. In: Proceedings of European Conference on Computer Vision. 2010, 591–604

    Google Scholar 

  24. Bai X, Yao C, Liu W Y. Strokelets: a learned multi-scale mid-level representation for scene text recognition. IEEE Transactions on Image Processing, 2016, 25(6): 2789–2802

    Article  MathSciNet  MATH  Google Scholar 

  25. Shi C Z, Wang C H, Xiao B H, Gao S, Hu J L. End-to-end scene text recognition using tree-structured models. Pattern Recognition, 2014, 47(9): 2853–2866

    Article  Google Scholar 

  26. Mishra A, Alahari K, Jawahar C V. Scene text recognition using higher order language priors. In: Proceedings of British Machine Vision Conference. 2012

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant No. 61703316), and in part by the Human Interface Lab of Kyushu University, Japan.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anna Zhu.

Additional information

Anna Zhu received the BS and PhD degrees from Huazhong University of Science and Technology, China in 2011 and 2016, respectively. She was once a research fellow at the Human Interface Laboratory, Kyushu University, Japan. Her research interests include text detection, image processing, pattern recognition, and machine learning.

Seiichi Uchida received the BS, MS, and PhD degrees from Kyushu University, Japan in 1990, 1992, and 1999, respectively. From 1992 to 1996, he was with SECOM Co., Ltd., Japan. Currently, he is a professor at Kyushu University. His research interests include pattern recognition and image processing. Dr. Uchida is a member of IEEE and IPSJ.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, A., Uchida, S. Scene word recognition from pieces to whole. Front. Comput. Sci. 13, 292–301 (2019). https://doi.org/10.1007/s11704-017-6420-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-017-6420-2

Keywords

Navigation