Multitask Pointer Network for Korean Dependency Parsing
Article No.: 24, Pages 1 - 10
Abstract
Dependency parsing is a fundamental problem in natural language processing. We introduce a novel dependency-parsing framework called head-pointing--based dependency parsing. In this framework, we cast the Korean dependency parsing problem as a statistical head-pointing and arc-labeling problem. To address this problem, a novel neural network called the multitask pointer network is devised for a neural sequential head-pointing and type-labeling architecture. Our approach does not require any handcrafted features or language-specific rules to parse dependency. Furthermore, it achieves state-of-the-art performance for Korean dependency parsing.
References
[1]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations.
[2]
Xavier Carreras. 2007. Experiments with a higher-order projective dependency parser. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL’07). 957--961.
[3]
Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 740--750.
[4]
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, et al. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14).
[5]
Jinho D. Choi and Martha Palmer. 2011. Statistical dependency parsing in Korean: From corpus generation to automatic parsing. In Proceedings of the 2nd Workshop on Statistical Parsing of Morphologically Rich Languages. 1--11.
[6]
Jan Chorowski, Michał Zapotoczny, and Paweł Rychlikowski. 2016. Read, tag, and parse all at once, or fully-neural dependency parsing. arXiv:1609.03441.
[7]
Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12, 2493--2537.
[8]
Michael A. Covington. 2001. A fundamental algorithm for dependency parsing. In Proceedings of the 39th Annual ACM Southeast Conference. 95--102.
[9]
Aron Culotta and Jeffrey Sorensen. 2004. Dependency tree kernels for relation extraction. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. 423.
[10]
Yuan Ding and Martha Palmer. 2005. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 541--548.
[11]
Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 334--343.
[12]
Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of the 16th Conference on Computational Linguistics, Vol. 1. 340--345.
[13]
Yoav Goldberg and Joakim Nivre. 2013. Training deterministic parsers with non-deterministic oracles. Transactions of the Association for Computational Linguistics 1, 403--414.
[14]
James Henderson. 2003. Inducing history representations for broad coverage statistical parsing. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, Vol. 1. 24--31.
[15]
James Henderson, Paola Merlo, Ivan Titov, and Gabriele Musillo. 2013. Multilingual joint parsing of syntactic and semantic dependencies with a latent variable model. Computational Linguistics 39, 4 (2013), 949--998.
[16]
James Henderson and Ivan Titov. 2010. Incremental sigmoid belief networks for grammar learning. Journal of Machine Learning Research 11, 3541--3570.
[17]
Terry Koo and Michael Collins. 2010. Efficient third-order dependency parsers. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. 1--11.
[18]
Changki Lee, Junseok Kim, and Jeonghee Kim. 2014. Korean dependency parsing using deep learning. In Proceedings of the Annual Conference on Human and Cognitive Language Technology.
[19]
Changki Lee, Junseok Kim, Jeonghee Kim, and Hyunki Kim. 2013. Joint models for Korean word spacing and POS tagging using structural SVM. Journal of KIISE: Software and Applications 40, 12 (2013), 826--832.
[20]
Jianri Li and Jong-Hyeok Lee. 2015. Korean transition-based dependency parsing with recurrent neural network. KIISE Transactions on Computing Practices 21, 8 (2015), 567--571.
[21]
Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1412--1421.
[22]
Xuezhe Ma and Hai Zhao. 2012. Fourth-order dependency parsing. In Proceedingsof the 25th International Conference on Computational Linguistics (Posters). 785--796.
[23]
Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 91--98.
[24]
Ryan T. McDonald and Fernando C. N. Pereira. 2006. Online learning of approximate dependency parsing algorithms. In Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics (EACL’06). 81--88.
[25]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781.
[26]
Jin-Woo Min and Seung-Hoon Na. 2016. Korean dependency parsing using sequence to sequence model. In Proceedings of Korean Institute of Information Scientists and Engineers (KISSE’16). 491--493.
[27]
Jin-Woo Min and Seung-Hoon Na. 2017. SyntaxNet models using transition based recurrent unit for Korean dependency parsing. In Proceedings of Korean Institute of Information Scientists and Engineers (KISSE’17). 602--604.
[28]
Seung-Hoon Na, Kangil Kim, and Young-Kil Kim. 2016. Stack LSTMs for transition-based Korean dependency parsing. In Proceedings of the Korea Computer Congress (KCC’16). 732--734.
[29]
Seung-Hoon Na, Jong-Hoon Shin, and Kangil Kim. 2016. Improving stack LSTMs by combining syllables and morphemes for Korean dependency parsing. In Proceedings of the Annual Conference on Human and Cognitive Language Technology.
[30]
Joakim Nivre. 2003. An efficient algorithm for projective dependency parsing. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT’03).
[31]
Joakim Nivre and Mario Scholz. 2004. Deterministic dependency parsing of English text. In Proceedings of the 20th International Conference on Computational Linguistics. 64.
[32]
Kenji Sagae. 2009. Analysis of discourse structure with syntactic dependencies and data-driven shift-reduce parsing. In Proceedings of the 11th International Conference on Parsing Technologies. 81--84.
[33]
Yusuke Shinyama, Satoshi Sekine, and Kiyoshi Sudo. 2002. Automatic paraphrase acquisition from news articles. In Proceedings of the 2nd International Conference on Human Language Technology Research. 313--318.
[34]
Richard Socher, John Bauer, Christopher D. Manning, and Andrew Y. Ng. 2013. Parsing with compositional vector grammars. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL’13), Vol. 1. 455--465.
[35]
Richard Socher, Cliff C. Lin, Chris Manning, and Andrew Y. Ng. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML’11). 129--136.
[36]
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS’14). 3104--3112.
[37]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, et al. 2017. Attention is all you need. arXiv:1706.03762.
[38]
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’15). 2692--2700.
[39]
Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’15). 2773--2781.
[40]
Wenhui Wang and Baobao Chang. 2016. Graph-based dependency parsing with bidirectional LSTM. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16).
[41]
Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT’03), Vol. 3. 195--206.
[42]
J. T. Yoon and K.-S. Choi. 1999. Study on Kaist Corpus. Technical Report CS-TR-99-139. Department of Computer Science, KAIST.
[43]
Xingxing Zhang, Jianpeng Cheng, and Mirella Lapata. 2016. Dependency parsing as head selection. arXiv:1606.01280.
[44]
Yue Zhang and Stephen Clark. 2008. A tale of two parsers: Investigating and combining graph-based and transition-based dependency parsing using beam-search. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 562--571.
Index Terms
- Multitask Pointer Network for Korean Dependency Parsing
Recommendations
Multitask Pointer Network for multi-representational parsing
AbstractDependency and constituent trees are widely used by many artificial intelligence applications for representing the syntactic structure of human languages. Typically, these structures are separately produced by either dependency or ...
Highlights- Dependency and constituent trees are used by many artificial intelligence applications.
Dependency parsing with bottom-up Hierarchical Pointer Networks
AbstractDependency parsing is a crucial step towards deep language understanding and, therefore, widely demanded by numerous Natural Language Processing applications. In particular, left-to-right and top-down transition-based algorithms that ...
Highlights- The left-to-right transition-based algorithm is a state-of-the-art dependency parser.
Korean Dependency Parsing with Proper Noun Encodin
ICIIT '19: Proceedings of the 2019 4th International Conference on Intelligent Information TechnologyDependency parsing is the process of analyzing the linguistic relationship of words that make up a sentence. In natural language processing using deep learning, previously registered words are represented in a continuous vector space. However, if proper ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2019
386 pages
Copyright © 2019 ACM.
© 2019 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 February 2019
Accepted: 01 September 2018
Revised: 01 March 2018
Received: 01 June 2017
Published in TALLIP Volume 18, Issue 3
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed
Funding Sources
- National Research Foundation of Korea (NRF)
- Korea Electric Power Corporation
- Ministry of Science and ICT
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 272Total Downloads
- Downloads (Last 12 months)6
- Downloads (Last 6 weeks)1
Reflects downloads up to 15 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format