More Web Proxy on the site http://driver.im/

research-article

Free access

Unsupervised multilingual grammar induction

Authors:

Benjamin Snyder,

Regina BarzilayAuthors Info & Claims

ACL '09: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - Volume 1

Pages 73 - 81

Published: 02 August 2009 Publication History

Abstract

We investigate the task of unsupervised constituency parsing from bilingual parallel corpora. Our goal is to use bilingual cues to learn improved parsing models for each language and to evaluate these models on held-out monolingual test data. We formulate a generative Bayesian model which seeks to explain the observed parallel data through a combination of bilingual and monolingual parameters. To this end, we adapt a formalism known as unordered tree alignment to our probabilistic setting. Using this formalism, our model loosely binds parallel trees while allowing language-specific syntactic structure. We perform inference under this model using Markov Chain Monte Carlo and dynamic programming. Applying this model to three parallel corpora (Korean-English, Urdu-English, and Chinese-English) we find substantial performance gains over the CCM model, a strong monolingual baseline. On average, across a variety of testing scenarios, our model achieves an 8.8 absolute gain in F-measure.

References

[1]

Ann Bies, Martha Palmer, Justin Mott, and Colin Warner. 2007. English Chinese translation treebank v 1.0. LDC2007T02.

[2]

Phil Blunsom, Trevor Cohn, and Miles Osborne. 2008. Bayesian synchronous grammar induction. In Proceedings of NIPS.

[3]

David Burkett and Dan Klein. 2008. Two languages are better than one (for syntactic parsing). In Proceedings of EMNLP, pages 877--886.

Digital Library

[4]

Eugene Charniak and Glen Carroll. 1992. Two experiments on learning probabilistic dependency grammars from corpora. In Proceedings of the AAAI Workshop on Statistically-Based NLP Techniques, pages 1--13.

[5]

David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of the ACL, pages 263--270.

Digital Library

[6]

Shay B. Cohen and Noah A. Smith. 2009. Shared logistic normal distributions for soft parameter tying in unsupervised grammar induction. In Proceedings of the NAACL/HLT.

Digital Library

[7]

Jason Eisner. 2003. Learning non-isomorphic tree mappings for machine translation. In The Companion Volume to the Proceedings of the ACL, pages 205--208.

Digital Library

[8]

Andrew Gelman, John B. Carlin, Hal S. Stern, and Donald B. Rubin. 2004. Bayesian data analysis. Chapman and Hall/CRC.

[9]

Dmitriy Genzel. 2005. Inducing a multilingual dictionary from a parallel multitext in related languages. In Proceedings of EMNLP/HLT, pages 875--882.

Digital Library

[10]

C. Han, N. R. Han, E. S. Ko, H. Yi, and M. Palmer. 2002. Penn Korean Treebank: Development and evaluation. In Proc. Pacific Asian Conf. Language and Comp.

[11]

W. K. Hastings. 1970. Monte carlo sampling methods using Markov chains and their applications. Biometrika, 57:97--109.

[12]

R. Hwa, P. Resnik, A. Weinberg, C. Cabezas, and O. Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel texts. Journal of Natural Language Engineering, 11(3):311--325.

Digital Library

[13]

T. Jiang, L. Wang, and K. Zhang. 1995. Alignment of trees -- an alternative to tree edit. Theoretical Computer Science, 143(1):137--148.

Digital Library

[14]

M. Johnson, T. Griffiths, and S. Goldwater. 2007. Bayesian inference for PCFGs via Markov chain Monte Carlo. In Proceedings of the NAACL/HLT, pages 139--146.

[15]

Dan Klein and Christopher D. Manning. 2002. A generative constituent-context model for improved grammar induction. In Proceedings of the ACL, pages 128--135.

Digital Library

[16]

D. Klein. 2005. The Unsupervised Learning of Natural Language Structure. Ph.D. thesis, Stanford University.

Digital Library

[17]

Jonas Kuhn. 2004. Experiments in parallel-text based grammar induction. In Proceedings of the ACL, pages 470--477.

Digital Library

[18]

I. Dan Melamed. 2003. Multitext grammars and synchronous parsers. In Proceedings of the NAACL/HLT, pages 79--86.

Digital Library

[19]

Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19--51.

Digital Library

[20]

Yoav Seginer. 2007. Fast unsupervised incremental parsing. In Proceedings of the ACL, pages 384--391.

[21]

David A. Smith and Noah A. Smith. 2004. Bilingual parsing with factored estimation: Using English to parse Korean. In Proceeding of EMNLP, pages 49--56.

[22]

Benjamin Snyder and Regina Barzilay. 2008. Unsupervised multilingual learning for morphological segmentation. In Proceedings of the ACL/HLT, pages 737--745.

Digital Library

[23]

Benjamin Snyder, Tahira Naseem, Jacob Eisenstein, and Regina Barzilay. 2008. Unsupervised multilingual learning for POS tagging. In Proceedings of EMNLP, pages 1041--1050.

Digital Library

[24]

Benjamin Snyder, Tahira Naseem, Jacob Eisenstein, and Regina Barzilay. 2009. Adding more languages improves unsupervised multilingual part-of-speech tagging: A Bayesian non-parametric approach. In Proceedings of the NAACL/HLT.

Digital Library

[25]

Andreas Stolcke and Stephen M. Omohundro. 1994. Inducing probabilistic grammars by Bayesian model merging. In Proceedings of ICGI, pages 106--118.

Digital Library

[26]

Dekai Wu and Hongsing Wong. 1998. Machine translation with a stochastic grammatical channel. In Proceedings of the ACL/COLING, pages 1408--1415.

Digital Library

[27]

Dekai Wu. 1997. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 23(3):377--403.

Digital Library

[28]

Chenhai Xi and Rebecca Hwa. 2005. A backoff model for bootstrapping resources for non-english languages. In Proceedings of EMNLP, pages 851--858.

Digital Library

[29]

Hao Zhang and Daniel Gildea. 2005. Stochastic lexicalized inversion transduction grammar for alignment. In Proceedings of the ACL, pages 475--482.

Digital Library

Cited By

Kim YSnyder BTsujii JHenderson JPasca M(2012)Universal grapheme-to-phoneme prediction over Latin alphabetsProceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning10.5555/2390948.2390990(332-343)Online publication date: 12-Jul-2012
https://dl.acm.org/doi/10.5555/2390948.2390990
Titov IKlementiev ALi HLin COsborne M(2012)Crosslingual induction of semantic rolesProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 110.5555/2390524.2390615(647-656)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390524.2390615
Naseem TBarzilay RGloberson ALi HLin COsborne M(2012)Selective sharing for multilingual dependency parsingProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 110.5555/2390524.2390613(629-637)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390524.2390613
Show More Cited By

Index Terms

Unsupervised multilingual grammar induction
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
  2. Machine learning
2. Theory of computation
  1. Formal languages and automata theory
    1. Formalisms
      1. Rewrite systems
    2. Grammars and context-free languages

Recommendations

Identifying patterns for unsupervised grammar induction
CoNLL '10: Proceedings of the Fourteenth Conference on Computational Natural Language Learning

This paper describes a new method for unsupervised grammar induction based on the automatic extraction of certain patterns in the texts. Our starting hypothesis is that there exist some classes of words that function as separators, marking the beginning ...
Unsupervised Grammar Induction of Clinical Report Sublanguage
ICMLA '11: Proceedings of the 2011 10th International Conference on Machine Learning and Applications and Workshops - Volume 02

Clinical reports are written using a subset of natural language while employing many domain-specific terms, such a language is also known as a sub language for a scientific or a technical domain. In this paper, we present a method which automatically ...
Learning common grammar from multilingual corpus
ACLShort '10: Proceedings of the ACL 2010 Conference Short Papers

We propose a corpus-based probabilistic framework to extract hidden common syntax across languages from non-parallel multilingual corpora in an unsupervised fashion. For this purpose, we assume a generative model for multilingual corpora, where each ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image DL Hosted proceedings

ACL '09: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - Volume 1

August 2009

572 pages

ISBN:9781932432459

General Chair:
Keh-Yih Su
Behavior Design Corp., Taiwan

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 02 August 2009

Qualifiers

Research-article

Acceptance Rates

Overall Acceptance Rate 85 of 443 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
389
Total Downloads

Downloads (Last 12 months)55
Downloads (Last 6 weeks)3

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kim YSnyder BTsujii JHenderson JPasca M(2012)Universal grapheme-to-phoneme prediction over Latin alphabetsProceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning10.5555/2390948.2390990(332-343)Online publication date: 12-Jul-2012
https://dl.acm.org/doi/10.5555/2390948.2390990
Titov IKlementiev ALi HLin COsborne M(2012)Crosslingual induction of semantic rolesProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 110.5555/2390524.2390615(647-656)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390524.2390615
Naseem TBarzilay RGloberson ALi HLin COsborne M(2012)Selective sharing for multilingual dependency parsingProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 110.5555/2390524.2390613(629-637)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390524.2390613
Jiang WLiu QLü YMerlo PBarzilay RJohnson M(2011)Relaxed cross-lingual projection of constituent syntaxProceedings of the Conference on Empirical Methods in Natural Language Processing10.5555/2145432.2145558(1192-1201)Online publication date: 27-Jul-2011
https://dl.acm.org/doi/10.5555/2145432.2145558
Kim YGraça JSnyder BMerlo PBarzilay RJohnson M(2011)Universal morphological analysis using structured nearest neighbor predictionProceedings of the Conference on Empirical Methods in Natural Language Processing10.5555/2145432.2145470(322-332)Online publication date: 27-Jul-2011
https://dl.acm.org/doi/10.5555/2145432.2145470
DeNero JUszkoreit JMerlo PBarzilay RJohnson M(2011)Inducing sentence structure from parallel corpora for reorderingProceedings of the Conference on Empirical Methods in Natural Language Processing10.5555/2145432.2145455(193-203)Online publication date: 27-Jul-2011
https://dl.acm.org/doi/10.5555/2145432.2145455
Cohen SDas DSmith NMerlo PBarzilay RJohnson M(2011)Unsupervised structure prediction with non-parallel multilingual guidanceProceedings of the Conference on Empirical Methods in Natural Language Processing10.5555/2145432.2145439(50-61)Online publication date: 27-Jul-2011
https://dl.acm.org/doi/10.5555/2145432.2145439
Bergsma SYarowsky DChurch KLin D(2011)Using large monolingual and bilingual corpora to improve coordination disambiguationProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 110.5555/2002472.2002639(1346-1355)Online publication date: 19-Jun-2011
https://dl.acm.org/doi/10.5555/2002472.2002639
Das DPetrov SLin D(2011)Unsupervised part-of-speech tagging with bilingual graph-based projectionsProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 110.5555/2002472.2002549(600-609)Online publication date: 19-Jun-2011
https://dl.acm.org/doi/10.5555/2002472.2002549
Snyder BBarzilay R(2010)Climbing the tower of babelProceedings of the 27th International Conference on International Conference on Machine Learning10.5555/3104322.3104328(29-36)Online publication date: 21-Jun-2010
https://dl.acm.org/doi/10.5555/3104322.3104328
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents