More Web Proxy on the site http://driver.im/

research-article

Low-Resource Machine Transliteration Using Recurrent Neural Networks

Authors:

Dien DinhAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), Volume 18, Issue 2

Article No.: 13, Pages 1 - 14

https://doi.org/10.1145/3265752

Published: 16 January 2019 Publication History

Abstract

Grapheme-to-phoneme models are key components in automatic speech recognition and text-to-speech systems. With low-resource language pairs that do not have available and well-developed pronunciation lexicons, grapheme-to-phoneme models are particularly useful. These models are based on initial alignments between grapheme source and phoneme target sequences. Inspired by sequence-to-sequence recurrent neural network--based translation methods, the current research presents an approach that applies an alignment representation for input sequences and pretrained source and target embeddings to overcome the transliteration problem for a low-resource languages pair. Evaluation and experiments involving French and Vietnamese showed that with only a small bilingual pronunciation dictionary available for training the transliteration models, promising results were obtained with a large increase in BLEU scores and a reduction in Translation Error Rate (TER) and Phoneme Error Rate (PER). Moreover, we compared our proposed neural network--based transliteration approach with a statistical one.

References

[1]

Rami Al-Rfou, Guillaume Alain, Amjad Almahairi, Christof Angermueller, Dzmitry Bahdanau, Nicolas Ballas, Frédéric Bastien, Justin Bayer, Anatoly Belikov, Alexander Belopolsky, et al. 2016. Theano: A Python framework for fast computation of mathematical expressions. arXiv preprint arXiv:1605.02688 472 (2016), 473.

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).

[3]

Maria João Barros and Christian Weiss. 2006. Maximum entropy motivated grapheme-to-phoneme, stress and syllable boundary prediction for Portuguese text-to-speech. IV Jornadas en Tecnologías del Habla. Zaragoza, Spain (2006), 177--182.

[4]

Maximilian Bisani and Hermann Ney. 2008. Joint-sequence models for grapheme-to-phoneme conversion. Speech Communication 50, 5 (2008), 434--451.

Digital Library

[5]

Nam X. Cao, Nhut M. Pham, and Quan H. Vu. 2010. Comparative analysis of transliteration techniques based on statistical machine translation and joint-sequence model. In Proceedings of the 2010 Symposium on Information and Communication Technology. Association for Computing Machinery, 59--63.

Digital Library

[6]

Stanley F. Chen et al. 2003. Conditional and joint models for grapheme-to-phoneme conversion. In Proceedings of the Interspeech Geneva. 2033--2036.

[7]

Jonathan H. Clark, Chris Dyer, Alon Lavie, and Noah A. Smith. 2011. Better hypothesis testing for statistical machine translation: Controlling for optimizer instability. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers-Volume 2. Association for Computational Linguistics, 176--181.

Digital Library

[8]

Sabine Deligne, Francois Yvon, and Frédéric Bimbot. 1995. Variable-length sequence matching for phonetic transcription using joint multigrams. In Proceedings of the 4th European Conference on Speech Communication and Technology. 2243--2246.

[9]

Xiangyu Duan, Rafael E. Banchs, Min Zhang, Haizhou Li, and A. Kumaran. 2016. Report of NEWS 2016 machine transliteration shared task. ACL 2016 (2016), 58--72.

[10]

Andrew Finch, Lemao Liu, Xiaolin Wang, and Eiichiro Sumita. 2016. Target-bidirectional neural models for machine transliteration. ACL 2016 (2016), 78--82.

[11]

Andrew Finch and Eiichiro Sumita. 2010. Transliteration using a phrase-based statistical machine translation system to re-score the output of a joint multigram model. In Proceedings of the 2010 Named Entities Workshop. Association for Computational Linguistics, 48--52.

Digital Library

[12]

Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, and Kyunghyun Cho. 2016. Zero-resource translation with multi-lingual neural machine translation. arXiv preprint arXiv:1606.04164 (2016).

[13]

Qin Gao and Stephan Vogel. 2008. Parallel implementations of word alignment tool. In Software Engineering, Testing, and Quality Assurance for Natural Language Processing. Association for Computational Linguistics, 49--57.

Digital Library

[14]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (Nov. 1997), 1735--1780.

Digital Library

[15]

Sebastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2014. On using very large target vocabulary for neural machine translation. arXiv preprint arXiv:1412.2007 (2014).

[16]

Sittichai Jiampojamarn, Grzegorz Kondrak, and Tarek Sherif. 2007. Applying many-to-many alignments and hidden Markov models to letter-to-phoneme conversion. In HLT-NAACL, Vol. 7. 372--379.

[17]

Sarvnaz Karimi, Falk Scholer, and Andrew Turpin. 2011. Machine transliteration survey. ACM Computing Surveys (CSUR) 43, 3 (2011), 17.

Digital Library

[18]

Alexandre Klementiev and Dan Roth. 2006. Weakly supervised named entity transliteration and discovery from multilingual comparable corpora. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 817--824.

Digital Library

[19]

Kevin Knight and Jonathan Graehl. 1998. Machine transliteration. Computational Linguistics 24, 4 (1998), 599--612.

Digital Library

[20]

Philipp Koehn. 2017. Neural machine translation. arXiv preprint arXiv:1709.07809 (2017).

[21]

Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions. Association for Computational Linguistics, 177--180.

Digital Library

[22]

A. Kumaran, Mitesh M. Khapra, and Haizhou Li. 2010. Report of NEWS 2010 transliteration mining shared task. In Proceedings of the 2010 Named Entities Workshop. Association for Computational Linguistics, 21--28.

Digital Library

[23]

Antoine Laurent, Paul Deléglise, Sylvain Meignier, and France Spécinov-Trélazé. 2009. Grapheme to phoneme conversion using an SMT system. In Proceedings of INTERSPEECH, ISCA. 708--711.

[24]

Ngoc Tan Le and Fatiha Sadat. 2017. A neural network transliteration model in low resource settings. In Proceedings of the 16th International Conference of Machine Translation Summit. September 18-22, 2017, Nagoya, Japan, Volume 1. Research Track, 337--345.

[25]

Minh-Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, and Wojciech Zaremba. 2014. Addressing the rare word problem in neural machine translation. arXiv preprint arXiv:1410.8206 (2014).

[26]

Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In hlt-Naacl, Vol. 13. 746--751.

[27]

Hoang Gia Ngo, Nancy F. Chen, Binh Minh Nguyen, Bin Ma, and Haizhou Li. 2015. Phonology-augmented statistical transliteration for low-resource languages. In Proceedings of Interspeech. 3670--3674.

[28]

Garrett Nicolai, Bradley Hauer, Mohammad Salameh, Adam St Arnaud, Ying Xu, Lei Yao, and Grzegorz Kondrak. 2015. Multiple system combination for transliteration. In Proceedings of NEWS 2015, the 5th Named Entities Workshop. 72--79.

[29]

Jong-Hoon Oh, Key-Sun Choi, and Hitoshi Isahara. 2006. A machine transliteration model based on correspondence between graphemes and phonemes. ACM Transactions on Asian Language Information Processing (TALIP) 5, 3 (2006), 185--208.

Digital Library

[30]

Robert Östling and Jörg Tiedemann. 2017. Neural machine translation for low-resource languages. arXiv preprint arXiv:1708.05729 (2017).

[31]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 311--318.

Digital Library

[32]

Álvaro Peris. 2017. NMT-Keras. https://github.com/lvapeab/nmt-keras. GitHub repository.

[33]

Hoang Phe. 1997. Vietnamese dictionary. Vietnam Lexicography Centre, Da Nang Publishing House (1997).

[34]

Kanishka Rao, Fuchun Peng, Haşim Sak, and Françoise Beaufays. 2015. Grapheme-to-phoneme conversion using long short-term memory recurrent neural networks. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’15). IEEE, 4225--4229.

[35]

Mihaela Rosca and Thomas Breuel. 2016. Sequence-to-sequence neural network models for transliteration. arXiv preprint arXiv:1610.09565 (2016).

[36]

Hassan Sajjad, Helmut Schmid, Alexander Fraser, and Hinrich Schütze. 2017. Statistical models for unsupervised, semi-supervised and supervised transliteration mining. Computational Linguistics 43, 2 (2017), 349--375.

Digital Library

[37]

Yan Shao and Joakim Nivre. 2016. Applying neural networks to English-Chinese named EntityTransliteration. In Proceedings of the 6th Named Entity Workshop, Joint with 54th ACL, Berlin. 73--77.

[38]

Matthew G. Snover, Nitin Madnani, Bonnie Dorr, and Richard Schwartz. 2009. TER-Plus: Paraphrase, semantic, and alignment enhancements to translation edit rate. Machine Translation 23, 2--3 (2009), 117--127.

Digital Library

[39]

Andreas Stolcke et al. 2002. SRILM-an extensible language modeling toolkit. In Interspeech, Vol. 2002. 2002.

[40]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the Advances in Neural Information Processing Systems. 3104--3112.

Digital Library

[41]

Ye Kyaw Thu, Win Pa Pa, Yoshinori Sagisaka, and Naoto Iwahashi. 2016. Comparison of grapheme--to--phoneme conversion methods on a Myanmar pronunciation dictionary. In Proceedings of the 6th Workshop on South and Southeast Asian Natural Language Processing (2016). 11--22.

[42]

Phuoc Tran, Dien Dinh, and Hien T. Nguyen. 2016. A character level based and word level based approach for Chinese-Vietnamese machine translation. Computational Intelligence and Neuroscience 2016, Article 9821608 (2016), 1--11.

Digital Library

[43]

Raghavendra Udupa, K. Saravanan, A. Kumaran, and Jagadeesh Jagarlamudi. 2009. Mint: A method for effective and scalable mining of named entity transliterations from large comparable corpora. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 799--807.

Digital Library

[44]

Sonjia Waxmonsky and Sravana Reddy. 2012. G2P conversion of proper names using word origin information. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 367--371.

Digital Library

[45]

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016).

[46]

Kaisheng Yao and Geoffrey Zweig. 2015. Sequence-to-sequence neural net models for grapheme-to-phoneme conversion. arXiv preprint arXiv:1506.00196 (2015).

[47]

Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. arXiv preprint arXiv:1604.02201 (2016).

Cited By

Nath BSarkar SMukhopadhyay SRoy A(2024)Improving neural machine translation by integrating transliteration for low-resource English–Assamese languageNatural Language Processing10.1017/nlp.2024.20(1-22)Online publication date: 27-May-2024
https://doi.org/10.1017/nlp.2024.20
Sato S(2023)Translating the List of Participants in the 2020 Tokyo Olympic Games into Japanese2020 東京オリンピック参加者名簿の翻訳Journal of Natural Language Processing10.5715/jnlp.30.74830:2(748-772)Online publication date: 2023
https://doi.org/10.5715/jnlp.30.748
Liu HDay MWang C(2023)Speech-to-speech Low-resource Translation2023 IEEE 24th International Conference on Information Reuse and Integration for Data Science (IRI)10.1109/IRI58017.2023.00023(91-95)Online publication date: Aug-2023
https://doi.org/10.1109/IRI58017.2023.00023
Show More Cited By

Index Terms

Low-Resource Machine Transliteration Using Recurrent Neural Networks
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Machine translation

Recommendations

Machine transliteration survey

Machine transliteration is the process of automatically transforming the script of a word from a source language to a target language, while preserving pronunciation. The development of algorithms specifically for machine transliteration began over a ...
Enhancing recurrent neural network-based language models by word tokenization

Different approaches have been used to estimate language models from a given corpus. Recently, researchers have used different neural network architectures to estimate the language models from a given corpus using unsupervised learning neural networks ...
MorphoGen: Full Inflection Generation Using Recurrent Neural Networks
Computational Linguistics and Intelligent Text Processing
Abstract
Sub-word level alternations during inflection (apophonies) are an common linguistic phenomenon present in morphologically-rich languages, like Romanian. Inflection learning, or predicting the inflection class of a partially regular or fully ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 18, Issue 2

June 2019

208 pages

ISSN:2375-4699

EISSN:2375-4702

DOI:10.1145/3300146

Editor:
Nianwen Xue
Brandeis University, Waltham, USA

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 January 2019

Accepted: 01 August 2018

Revised: 01 May 2018

Received: 01 February 2018

Published in TALLIP Volume 18, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
411
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nath BSarkar SMukhopadhyay SRoy A(2024)Improving neural machine translation by integrating transliteration for low-resource English–Assamese languageNatural Language Processing10.1017/nlp.2024.20(1-22)Online publication date: 27-May-2024
https://doi.org/10.1017/nlp.2024.20
Sato S(2023)Translating the List of Participants in the 2020 Tokyo Olympic Games into Japanese2020 東京オリンピック参加者名簿の翻訳Journal of Natural Language Processing10.5715/jnlp.30.74830:2(748-772)Online publication date: 2023
https://doi.org/10.5715/jnlp.30.748
Liu HDay MWang C(2023)Speech-to-speech Low-resource Translation2023 IEEE 24th International Conference on Information Reuse and Integration for Data Science (IRI)10.1109/IRI58017.2023.00023(91-95)Online publication date: Aug-2023
https://doi.org/10.1109/IRI58017.2023.00023
Nath BSarkar SC. Debnath N(2023)A Study of Word Embedding Models for Machine Translation of North Eastern LanguagesComputational Intelligence in Communications and Business Analytics10.1007/978-3-031-48879-5_26(343-359)Online publication date: 30-Nov-2023
https://doi.org/10.1007/978-3-031-48879-5_26
Mahajan SRani R(2022)Word Level Script Identification Using Convolutional Neural Network Enhancement for Scenic ImagesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/350669921:4(1-29)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.1145/3506699
Shekhar SSharma DAgarwal DPathak Y(2022)Artificial Immune Systems-Based Classification Model for Code-Mixed Social Media DataIRBM10.1016/j.irbm.2020.07.00443:2(120-129)Online publication date: Apr-2022
https://doi.org/10.1016/j.irbm.2020.07.004
Laitonjam LSingh S(2022)A Hybrid Machine Transliteration Model Based on Multi-source Encoder–Decoder Framework: English to ManipuriSN Computer Science10.1007/s42979-021-01005-93:2Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1007/s42979-021-01005-9
Kumar SKumar SPati J(2022)A Review on Transliterated Text Retrieval for Indian LanguagesProceedings of International Conference on Computational Intelligence10.1007/978-981-19-2126-1_10(137-146)Online publication date: 4-Oct-2022
https://doi.org/10.1007/978-981-19-2126-1_10
Shekhar S(2021)Artificial Intelligence based Temporal Material Identification for Improving Qulaity of Service in CommunicationIOP Conference Series: Materials Science and Engineering10.1088/1757-899X/1116/1/0121251116:1(012125)Online publication date: 1-Apr-2021
https://doi.org/10.1088/1757-899X/1116/1/012125
Matos Veliz CDe Clercq OHoste V(2021)Is neural always better? SMT versus NMT for Dutch text normalizationExpert Systems with Applications10.1016/j.eswa.2020.114500170(114500)Online publication date: May-2021
https://doi.org/10.1016/j.eswa.2020.114500
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents