Automatic Construction of Chinese Typo-Pairs Based on Web Corpus
Pages 159 - 164
Abstract
With the development of big data, the amount of text data is growing bigger and bigger in which errors are also more and more. The traditional human-correction cannot meet the actual demand. It is a trend for automatic text proofing by using computer data processing. Chinese text errors can be divided into two categories: non-word error and real-word error. One or more character in a Chinese word replaced by other character will result in the word does not belong to the Chinese dictionary, which we call "non-word error". The word segmentation is firstly performed on the corpus in Chinese NLP, and non-word error will be divided into several disperse strings, which bring Chinese text proofreading several problems, because there are single-character words and multi-characters words in Chinese dictionary. In this paper, an approach is proposed to construct Chinese typo-pairs from Web corpus, which can be used in Chinese text automatic proofreading efficiently. Firstly, the method adds similar words into a candidate set using fuzzy matching algorithm, and then validates the similar words in the candidate set using statistical models, and finally constructs the typo-pairs.
References
[1]
KukichK.Techniques for automatically correcting words in text[J].ACM Computing Surveys, 1992,24(4):377–439.
[2]
ATWELL E, ELLIOTT S. Dealing with ill-formed English text[J]. The Computational Analysis of English: A Corpus-Based Approach, 1987: 120–138.[17] GALE W A, CHURCH .
[3]
CHURCH K W, GALE WA. Probability scoring for spelling correction[J]. Statistics and Computing, 1991,1(2):93–103.
[4]
BERLINSKY-SCHINE A. 2004. Context-based detection of real word typographical errors using markovmodels[R]. Technical report, Cornell University, Ithaca, NY.
[5]
Zheng Fang, Mou Xiaolong, Xu Mingxing, Wu Jian, Song Zhanjiang, Wu Wenhu. Studies and implementation of the techniques for Chinese dictation machines. Journal of Software, 1999,04.
[6]
Luo Weihua, Luo Zhengsheng, Gong Xiaojin. Research on the Chinese text automatic proofreading technology. Journal of Computer Research and Development. 2004.41(1).
[7]
Yu Yu, Yao Tianshun. A Mixed Chinese Text Proofreading Method[J] . Journal of Chinese Information Processing,1998,12(2):31-36. (in Chinese)
[8]
Wang Yun.The Principles and Methods of Computer Chinese Text Proofreading[J].Teaching and Technology,2003,16(2):41-44. (in Chinese)
[9]
Sun Cai, Luo Zhensheng. Research on the Correction of Chinese Text Proofreading Words[A]. Proceedings of the Fourth Conference on Computational Linguistics (Language Engineering)[C]. Beijing: Tsinghua University Press, 1997, 319 -324. (in Chinese)
[10]
Guo Zhili Algorithm for modifying suggestions in Chinese proofreading system [A]. Proceedings of the 4th Conference on Computational Linguistics (Language Engineering) [C]. Beijing: Tsinghua University Press, 1997.325-330. (in Chinese)
[11]
Li Jianhua, Wang Xiaolong, Wang Pingping.The Research of Multi-Feature Chinese Text Proofreading Algorithms. [J].Computer Engineering & Science. 2001,(3):93. (in Chinese)
[12]
Zhang Yangsen and so on. Automatic Errors Detecting of Chinese Texts based on the Bi-neighborship. Journal of Chinese Information Processing. 2001, 15 (3). (in Chinese)
[13]
Yi Rongxiang,He Kekang. Proofreading Chinese Manuscript with Computer [J]. Computer Research & Development, 1997(05):28-32. (in Chinese)
[14]
Wu Yan, Li Xiukun, Liu Ting Research on and implementation of Chinese Text proof-reading system[J].Journal of Harbin Institute of Technology.2001(2):60-64,2001.(in Chinese)
[15]
Zhang Yangsen, Cao Yuanda, Yu Shiwen. A Hybrid Model of Combining Rule-Based and Statistics-Based Approaches for Automatic Detecting Errors in Chinese Text [J]. Journal of Chinese Information Processing, 2006, 4:000. (in Chinese)
[16]
Shi Desheng, Wang Liangzhi, Chen Zhida A Statistics based Approach for Automatic Detecting Errors in Chinese Text [J]. Computer and Communications, 1992, 8: 19-26.(in Chinese)
[17]
Zhang Lei, Zhou Ming, Huang Changning Pan Haihua. Automatic detecting/correction errors in Chinese text by anapproximateword-matching algorithm. Proceedings of the 3rd world congress on Intelligent Control and Automation.2000, pp.
[18]
Jinshan Ma, Yu Zhang, Ting Liu, Sheng Li. Detecting Chinese text errors based on trigram and dependency parsing. Intelligence Science, 2005, 23(6):723–728.
[19]
Lin Wu, Yangsen Zhang. Reasoning model of multi-level Chinese text error-detecting based on knowledge bases. Computer Engineering, 2012༌Volume 38, No 20:21-25.
[20]
Liu L, Cao C. A seed-based method for generating Chinese confusion sets[J]. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 2016, 16(1): 5.
Recommendations
A Seed-Based Method for Generating Chinese Confusion Sets
TALLIP Notes and Regular PapersIn natural language, people often misuse a word (called a “confused word”) in place of other words (called “confusing words”). In misspelling corrections, many approaches to finding and correcting misspelling errors are based on a simple notion called a ...
Revision for recognizing Chinese handwritten sentences based on lexical, syntactical and corpus rules
ROCLING '11: ROCLING 2011 Poster PapersRecognition of off-line handwritten Chinese character had been an important problem. Because of the variation and vagueness derived from different users' handwritings, it was hard to recognize handwriting characters via statistical features obtained ...
Corpus-based analysis of the co-occurrence of Chinese antonym pairs
ADMA'10: Proceedings of the 6th international conference on Advanced data mining and applications - Volume Part IIChinese antonym pairs pattern differently than antonyms in English. The opposites in a Chinese antonym pair often co-occur in a sentence. In a phenomenon unique to Chinese, most antonym pairs provide a good basis for the formation of four-character ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2019
326 pages
ISBN:9781450369886
DOI:10.1145/3358695
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 October 2019
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
WI '19
WI '19: IEEE/WIC/ACM International Conference on Web Intelligence
October 14 - 17, 2019
Thessaloniki, Greece
Acceptance Rates
Overall Acceptance Rate 118 of 178 submissions, 66%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 66Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 11 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format