More Web Proxy on the site http://driver.im/

research-article

TL-NER: A Transfer Learning Model for Chinese Named Entity Recognition

Authors:

Zhang ChenAuthors Info & Claims

Information Systems Frontiers, Volume 22, Issue 6

Pages 1291 - 1304

https://doi.org/10.1007/s10796-019-09932-y

Published: 01 December 2020 Publication History

Abstract

Most of the current research on Named Entity Recognition (NER) in the Chinese domain is based on the assumption that annotated data are adequate. However, in many scenarios, the sufficient amount of annotated data required for Chinese NER task is difficult to obtain, resulting in poor performance of machine learning methods. In view of this situation, this paper tries to excavate the information contained in the massive unlabeled raw text data and utilize it to enhance the performance of Chinese NER task. A deep learning model combined with Transfer Learning technique is proposed in this paper. This method can be leveraged in some domains where there is a large amount of unlabeled text data and a small amount of annotated data. The experiment results show that the proposed method performs well on different sized datasets, and this method also avoids errors that occur during the word segmentation process. We also evaluate the effect of transfer learning from different aspects through a series of experiments.

References

[1]

Agrawal A, Lu J, Antol S, Mitchell M, Zitnick CL, Parikh D, and Batra D Vqa: Visual question answering International Journal of Computer Vision 2015 123 1 1-28

[2]

Chakrabarty B and Shkilko A Information transfers and learning in financial markets: Evidence from short selling around insider sales Journal of Banking & Finance 2013 37 5 1560-1572

[3]

Che, W., Wang, M., Manning, C.D., Liu, T. (2013). Named entity recognition with bilingual constraints. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 52–62).

[4]

Chiu JP and Nichols E Named entity recognition with bidirectional lstm-cnns Transactions of the Association for Computational Linguistics 2016 4 357-370

[5]

Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y. (2014). Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1724–1734).

[6]

Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, and Kuksa P Natural language processing (almost) from scratch Journal of Machine Learning Research 2011 12 2493-2537

[7]

Derczynski L, Maynard D, Rizzo G, Erp MV, Gorrell G, Troncy R, Petrak J, and Bontcheva K Analysis of named entity recognition and linking for tweets Information Processing & Management 2015 51 2 32-49

[8]

Dernoncourt F, Lee JY, Uzuner O, and Szolovits P De-identification of patient notes with recurrent neural networks Journal of the American Medical Informatics Association 2017 24 3 596-606

[9]

Dong, C., Zhang, J., Zong, C., Hattori, M., Di, H. (2016). Character-based lstm-crf with radical-level features for chinese named entity recognition. In Natural Language Understanding and Intelligent Applications (pp. 239–250): Springer.

[10]

Dong, X., Chowdhury, S., Qian, L., Guan, Y., Yang, J., Yu, Q. (2017). Transfer bi-directional lstm rnn for named entity recognition in chinese electronic medical records. In 2017 IEEE 19Th international conference on e-health networking, applications and services (Healthcom) (pp. 1–4): IEEE.

[11]

Forney GD The viterbi algorithm Proceedings of the IEEE 1973 61 3 268-278

[12]

Graves, A. (2012). Long short-term memory. In Supervised sequence labelling with recurrent neural networks (pp. 37–45): Springer.

[13]

Guo H, Jiang J, Hu G, and Zhang T Chinese Named Entity Recognition Based on Multilevel Linguistic Features 2004 Berlin Springer

[14]

He, H., & Sun, X. (2017). F-score driven max margin neural network for named entity recognition in chinese social media. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, (Vol. 2 pp. 713–718).

[15]

Huang, Z., Xu, W., Yu, K. (2015). Bidirectional lstm-crf models for sequence tagging. arXiv:abs/150801991.

[16]

Huang, S., Sun, X., Wang, H. (2017). Addressing domain adaptation for chinese word segmentation with global recurrent structure. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), (Vol. 1 pp. 184–193).

[17]

Kalchbrenner, N., Grefenstette, E., Blunsom, P. (2014). A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), (Vol. 1 pp. 655–665).

[18]

Kingma, D.P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv:abs/14126980.

[19]

Kuru, O., Can, O.A., Yuret, D. (2016). Charner: Character-level named entity recognition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (pp. 911–921).

[20]

Lafferty, J.D., Mccallum, A., Pereira, F.C.N. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Eighteenth International Conference on Machine Learning (pp. 282–289).

[21]

Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C. (2016). Neural architectures for named entity recognition. In Proceedings of NAACL-HLT (pp. 260–270).

[22]

Levow, G.A. (2006). The third international chinese language processing bakeoff: Word segmentation and named entity recognition. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing (pp. 108–117).

[23]

Li, H., Hagiwara, M., Li, Q., Ji, H. (2014). Comparison of the impact of word segmentation on name tagging for chinese and japanese. In LREC (pp. 2532–2536).

[24]

Liu, Z., Zhu, C., Zhao, T. (2010). Chinese named entity recognition with a sequence labeling approach: based on characters, or based on words? In Advanced intelligent computing theories and applications. With aspects of artificial intelligence (pp. 634–640): Springer.

[25]

Liu, L., Shang, J., Ren, X., Xu, F.F., Gui, H., Peng, J., Han, J. (2018). Empower sequence labeling with task-aware neural language model. In Proceedings of Thirty-Second AAAI Conference on Artificial Intelligence.

[26]

Lu, Y., Zhang, Y., Ji, D. (2016). Multi-prototype chinese character embedding. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016).

[27]

Luo, G., Huang, X., Lin, C.Y., Nie, Z. (2015). Joint entity recognition and disambiguation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 879–888).

[28]

Ma, X., & Hovy, E. (2016). End-to-end sequence labeling via bi-directional lstm-cnns-crf. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (vol. 1, pp. 1064–1074).

[29]

Mikolov, T., Chen, K., Corrado, G., Dean, J. (2013a). Efficient estimation of word representations in vector space. arXiv:abs/13013781.

[30]

Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J. (2013b). Distributed representations of words and phrases and their compositionality, (Vol. 26.

[31]

Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K. (2014). Recurrent models of visual attention. In Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2 (pp. 2204–2212): MIT Press.

[32]

Mou, L., Meng, Z., Yan, R., Li, G., Xu, Y., Zhang, L., Jin, Z. (2016). How transferable are neural networks in nlp applications? In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 479–489).

[33]

Nadeau D and Sekine S A survey of named entity recognition and classification Lingvisticae Investigationes 2007 30 1 3-26

[34]

Nemeskey DM and Kornai A Emergency vocabulary Information Systems Frontiers 2018 20 5 909-923

[35]

Oquab, M., Bottou, L., Laptev, I., Sivic, J. (2014). Learning and transferring mid-level image representations using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1717–1724).

[36]

Pan SJ and Yang Q A survey on transfer learning IEEE Transactions on Knowledge & Data Engineering 2010 22 10 1345-1359

[37]

Passos, A., Kumar, V., McCallum, A. (2014). Lexicon infused phrase embeddings for named entity resolution. CoNLL-2014, 78.

[38]

Peng, N., & Dredze, M. (2015). Named entity recognition for chinese social media with jointly trained embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp 548–554).

[39]

Peng, N., & Dredze, M. (2016). Improving named entity recognition for chinese social media with word segmentation representation learning. In Meeting of the Association for Computational Linguistics (pp 149–155).

[40]

Qiu, L., & Zhang, Y. (2015). Word segmentation for chinese novels. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (pp. 2440–2446): AAAI Press.

[41]

Rei, M., Crichton, G., Pyysalo, S. (2016). Attending to characters in neural sequence labeling models. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (pp 309–318).

[42]

Rei, M. (2017). Semi-supervised multitask learning for sequence labeling. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (vol. 1, pp. 2121–2130.

[43]

Smith KS, McCreadie R, Macdonald C, and Ounis I Regional sentiment bias in social media reporting during crises Information Systems Frontiers 2018 20 5 1013-1025

[44]

Wang, M., Che, W., Manning, C.D. (2013). Effective bilingual constraints for semi-supervised learning of named entity recognizers. In Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence (pp. 919–925): AAAI Press.

[45]

Wang, D., & Zheng, T.F. (2015). Transfer learning for speech and language processing. In Proceedings of 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) (pp. 1225–1237).

[46]

Weischedel R, Pradhan S, Ramshaw L, Palmer M, Xue N, Marcus M, Taylor A, Greenberg C, Hovy E, Belvin R, et al. Ontonotes release 4.0. LDC2011t03 2011 Philadelphia Linguistic Data Consortium

[47]

Williams RJ and Zipser D A learning algorithm for continually running fully recurrent neural networks Neural computation 1989 1 2 270-280

[48]

Wu, Y., Zhao, J., Xu, B. (2003). Chinese named entity recognition combining a statistical model with human knowledge. In ACL 2003 Workshop on Multilingual and Mixed-Language Named Entity Recognition (pp. 65–72).

[49]

Yang HL and Chao AFY Sentiment analysis for chinese reviews of movies in multi-genre based on morpheme-based features and collocations Information Systems Frontiers 2015 17 6 1335-1352

[50]

Yang, J., Teng, Z., Zhang, M., Zhang, Y. (2016). Combining discrete and neural features for sequence labeling. In International Conference on Intelligent Text Processing and Computational Linguistics (pp. 140–154): Springer.

[51]

Yang, J., Zhang, Y., Dong, F. (2017a). Neural word segmentation with rich pretraining. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (vol. 1, pp. 839–849).

[52]

Yang, Z., Salakhutdinov, R., Cohen, W.W. (2017b). Transfer learning for sequence tagging with hierarchical recurrent networks. arXiv:abs/170306345.

[53]

Zhang, S., Qin, Y., Wen, J., Wang, X. (2006). Word segmentation and named entity recognition for sighan bakeoff3. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing (pp. 158–161).

[54]

Zhou J, Qu W, and Zhang F Chinese named entity recognition via joint identification and categorization Chinese Journal of Electronics 2013 22 2 225-230

[55]

Zhuang FZ, Ping L, Qing HE, and Shi ZZ Survey on transfer learning research Journal of Software 2015 26 26-39

Cited By

Hou WZhao WLiu XGuo W(2024)Knowledge-Enriched Prompt for Low-Resource Named Entity RecognitionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365994823:5(1-15)Online publication date: 10-May-2024
https://dl.acm.org/doi/10.1145/3659948
Lanza-Cruz IBerlanga RAramburu M(2024)Multidimensional Author Profiling for Social Business IntelligenceInformation Systems Frontiers10.1007/s10796-023-10370-026:1(195-215)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s10796-023-10370-0
Meske CBunde E(2023)Design Principles for User Interfaces in AI-Based Decision Support Systems: The Case of Explainable Hate Speech DetectionInformation Systems Frontiers10.1007/s10796-021-10234-525:2(743-773)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s10796-021-10234-5
Show More Cited By

Index Terms

TL-NER: A Transfer Learning Model for Chinese Named Entity Recognition
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
  2. Machine learning
    1. Learning paradigms
    2. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Chinese Named Entity Recognition with CRFs: Two Levels
CIS '08: Proceedings of the 2008 International Conference on Computational Intelligence and Security - Volume 02

Named Entity Recognition (NER) is one of the key techniques in natural language processing tasks such as information extraction, text summarization and so on. Chinese NER is more complicated and difficult than other languages because of its ...
NERA: Named Entity Recognition for Arabic

Name identification has been worked on quite intensively for the past few years, and has been incorporated into several products revolving around natural language processing tasks. Many researchers have attacked the name identification problem in a ...
An Empirical Study of Multi-domain and Multi-task Learning in Chinese Named Entity Recognition
Artificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning
Abstract
Named entity recognition (NER) often suffers from lack of annotation data. Multi-domain and multi-task learning solve this problem in some degree. However, previous multi-domain and multi-task learning are often studied in English. In the other ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Information Systems Frontiers

Information Systems Frontiers Volume 22, Issue 6

Dec 2020

296 pages

ISSN:1387-3326

Issue’s Table of Contents

© Springer Science+Business Media, LLC, part of Springer Nature 2019.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 December 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hou WZhao WLiu XGuo W(2024)Knowledge-Enriched Prompt for Low-Resource Named Entity RecognitionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365994823:5(1-15)Online publication date: 10-May-2024
https://dl.acm.org/doi/10.1145/3659948
Lanza-Cruz IBerlanga RAramburu M(2024)Multidimensional Author Profiling for Social Business IntelligenceInformation Systems Frontiers10.1007/s10796-023-10370-026:1(195-215)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s10796-023-10370-0
Meske CBunde E(2023)Design Principles for User Interfaces in AI-Based Decision Support Systems: The Case of Explainable Hate Speech DetectionInformation Systems Frontiers10.1007/s10796-021-10234-525:2(743-773)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s10796-021-10234-5
Ding JXu WWang AZhao SZhang Q(2023)Joint multi-view character embedding model for named entity recognition of Chinese car reviewsNeural Computing and Applications10.1007/s00521-023-08476-235:20(14947-14962)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1007/s00521-023-08476-2
Zhou MGong K(2022)Chinese Medical Named Entity Recognition Based on Parameter Transfer LearningProceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence10.1145/3579654.3579713(1-6)Online publication date: 23-Dec-2022
https://dl.acm.org/doi/10.1145/3579654.3579713
Nie MCheng LYe HZhang W(2022)Chinese NER with High-Level Features in Specific DomainProceedings of the 2022 14th International Conference on Machine Learning and Computing10.1145/3529836.3529937(146-152)Online publication date: 18-Feb-2022
https://dl.acm.org/doi/10.1145/3529836.3529937
Savitri Jadhav Vandana Inamdar (2022)Convolutional Neural Network and Histogram of Oriented Gradient Based Invariant Handwritten MODI Character RecognitionPattern Recognition and Image Analysis10.1134/S105466182202010932:2(402-418)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1134/S1054661822020109
Ma HDing ZZhou DWang JNiu S(2022)Research on NER Based on Register Migration and Multi-task LearningWireless Algorithms, Systems, and Applications10.1007/978-3-031-19211-1_55(657-666)Online publication date: 24-Nov-2022
https://dl.acm.org/doi/10.1007/978-3-031-19211-1_55
Zhao DZhang PMeng JWu Y(2022)Adversarial Transfer Learning for Named Entity Recognition Based on Multi-Head Attention Mechanism and Feature FusionNatural Language Processing and Chinese Computing10.1007/978-3-031-17120-8_22(272-284)Online publication date: 24-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-17120-8_22

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents