Learning Word Embeddings by Incorporating Latent Meanings of Chinese Characters, Radicals and Sub-characters
Authors: 
Xinyu Su, Shuyu Zhao, Chunxiao Cai, Weigang Zhao, Yingbin Wang, Weiwei Wang, Chongdi DuanAuthors Info & Claims
Pages 69 - 73
Abstract
Different from traditional word embeddings that ignore implicit information within words, Chinese words usually consist of characters, most of which and can be broken down into radicals and components, which also contain a lot of latent meanings. In this paper, we propose a model named LMJWE for fine-grained enhancement of Chinese word vectors by integrating implicit information from characters, radicals and sub-characters. These methods include selective reference of latent meaning and parallel parameter updating. We verify the performance of our model through experiments such as word similarity, syntactic analogy, and data set size. The results show that our proposed model has certain advantages over the state-of-the-art models in most baseline tasks, and has a great improvement on the Chinese analog reasoning dataset.
References
[1]
A. Radford and K. Narasimhan, "Improving Language Understanding by Generative Pre-Training," 2018.
[2]
S. Sun, Q. Sun, K. Zhou, and T. Lv, "Hierarchical Attention Prototypical Networks for Few-Shot Text Classification," Hong Kong, China, November 2019: Association for Computational Linguistics, in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 476-485.
[3]
L. Qin, W. Wang, Q. Chen, and W. Che, "CLIPText: A New Paradigm for Zero-shot Text Classification," Toronto, Canada, July 2023: Association for Computational Linguistics, in Findings of the Association for Computational Linguistics: ACL 2023, pp. 1077-1088.
[4]
D. Zhang, R. Ye, T. Ko, M. Wang, and Y. Zhou, "DUB: Discrete Unit Back-translation for Speech Translation," ArXiv, vol. abs/2305.11411, 2023.
[5]
N. Kambhatla, L. Born, and A. Sarkar, "CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation," Dublin, Ireland, May 2022: Association for Computational Linguistics, in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 201-218.
[6]
W. Zheng, J. Yu, R. Xia, and S. Wang, "A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations," Toronto, Canada, July 2023: Association for Computational Linguistics, in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15445-15459.
[7]
T. Shi and S.-L. Huang, "MultiEMO: An Attention-Based Correlation-Aware Multimodal Fusion Framework for Emotion Recognition in Conversations," Toronto, Canada, July 2023: Association for Computational Linguistics, in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14752-14766.
[8]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, "Distributed Representations of Words and Phrases and their Compositionality," in Neural Information Processing Systems, 2013.
[9]
T. Mikolov, K. Chen, G. S. Corrado, and J. Dean, "Efficient Estimation of Word Representations in Vector Space," in International Conference on Learning Representations, 2013.
[10]
Y. Goldberg and O. Levy, "word2vec Explained: deriving Mikolov 's negative-sampling word-embedding method," ArXiv, vol. abs/1402.3722, 2014.
[11]
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," Minneapolis, Minnesota, June 2019: Association for Computational Linguistics, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186.
[12]
Y. Li, W. Li, F. Sun, and S. Li, "Component-Enhanced Chinese Character Embeddings," Lisbon, Portugal, September 2015: Association for Computational Linguistics, in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 829-834.
[13]
X. Chen, L. Xu, Z. Liu, M. Sun, and H. Luan, "Joint Learning of Character and Word Embeddings," in International Joint Conference on Artificial Intelligence, 2015.
[14]
J. Xu, J. Liu, L. Zhang, Z. Li, and H. Chen, "Improve Chinese Word Embeddings by Exploiting Internal Structure," San Diego, California, June 2016: Association for Computational Linguistics, in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1041-1050. [Online]. Available: https://aclanthology.org/N16-1119
[15]
R. Yin, Q. Wang, P. Li, R. Li, and B. Wang, "Multi-Granularity Chinese Word Embedding," Austin, Texas, November 2016: Association for Computational Linguistics, in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 981-986.
[16]
S. Cao, W. Lu, J. Zhou, and X. Li, "cw2vec: Learning Chinese Word Embeddings with Stroke n-gram Information," in AAAI Conference on Artificial Intelligence, 2018.
[17]
J. Yu, X. Jian, H. Xin, and Y. Song, "Joint Embeddings of Chinese Words, Characters, and Fine-grained Subcharacter Components," Copenhagen, Denmark, September 2017: Association for Computational Linguistics, in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 286-291.
[18]
T.-R. Su and H.-Y. Lee, "Learning Chinese Word Representations From Glyphs Of Characters," Copenhagen, Denmark, September 2017: Association for Computational Linguistics, in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 264-273.
Recommendations
Learning Chinese Word Embeddings By Discovering Inherent Semantic Relevance in Sub-characters
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge ManagementLearning Chinese word embeddings is important in many tasks of Chinese language information processing, such as entity linking, entity extraction, and knowledge graph. A Chinese word consists of Chinese characters, which can be decomposed into sub-...
Learning Chinese Word Embeddings from Stroke, Structure and Pinyin of Characters
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge ManagementChinese word embeddings have recently attracted much attention in natural language processing (NLP). Existing researches learn Chinese word embeddings based on characters, radicals, components and stroke n-gram. Besides abovementioned features, Chinese ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2024
668 pages
ISBN:9798400716751
DOI:10.1145/3670105
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 July 2024
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CNIOT 2024
CNIOT 2024: 2024 5th International Conference on Computing, Networks and Internet of Things
May 24 - 26, 2024
Tokyo, Japan
Acceptance Rates
Overall Acceptance Rate 39 of 82 submissions, 48%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 5Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)1
Reflects downloads up to 06 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format