More Web Proxy on the site http://driver.im/

research-article

Public Access

Cross-lingual Text Classification via Model Translation with Limited Dictionaries

Authors:

Andrew HsiAuthors Info & Claims

CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

Pages 95 - 104

https://doi.org/10.1145/2983323.2983732

Published: 24 October 2016 Publication History

Abstract

Cross-lingual text classification (CLTC) refers to the task of classifying documents in different languages into the same taxonomy of categories. An open challenge in CLTC is to classify documents for the languages where labeled training data are not available. Existing approaches rely on the availability of either high-quality machine translation of documents (to the languages where massively training data are available), or rich bilingual dictionaries for effective translation of trained classification models (to the languages where labeled training data are lacking). This paper studies the CLTC challenge under the assumption that neither condition is met. That is, we focus on the problem of translating classification models with highly incomplete bilingual dictionaries. Specifically, we propose two new approaches that combines unsupervised word embedding in different languages, supervised mapping of embedded words across languages, and probabilistic translation of classification models. The approaches show significant performance improvement in CLTC on a benchmark corpus of Reuters news stories (RCV1/RCV2) in English, Spanish, German, French and Chinese and an internal dataset in Uzbek, compared to representative baseline methods using conventional bilingual dictionaries or highly incomplete ones.

References

[1]

M. Amini, N. Usunier, and C. Goutte. Learning from multiple partially observed views-an application to multilingual text categorization. In Advances in neural information processing systems, pages 28--36, 2009.

Digital Library

[2]

N. Bel, C. H. Koster, and M. Villegas. Cross-lingual text categorization. Research and Advanced Technology for Digital Libraries, pages 126--139, 2003.

[3]

M. Faruqui and C. Dyer. Improving vector space word representations using multilingual correlation. In Conference of the European Chapter of the Association for Computational Linguistics, page 462--471. Association for Computational Linguistics, 2014.

[4]

M. Gardner, K. Huang, E. Papalexakis, X. Fu, P. Talukdar, C. Faloutsos, N. Sidiropoulos, and T. Mitchell. Translation invariant word embeddings. In Conference on Empirical Methods in Natural Language Processing, page 1084--1088, 2015.

[5]

S. Gouws, Y. Bengio, and G. Corrado. Bilbowa: Fast bilingual distributed representations without word alignments. In Proceedings of The 32nd International Conference on Machine Learning, page 748--756, 2015.

[6]

J. Guo, W. Che, D. Yarowsky, H. Wang, and T. Liu. Cross-lingual dependency parsing based on distributed representations. In Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, page 1234--1244, 2015.

[7]

J. Guo, W. Che, D. Yarowsky, H. Wang, and T. Liu. A representation learning framework for multi-source transfer parsing. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI), 2016.

Digital Library

[8]

Y. Guo and M. Xiao. Cross language text classification via subspace co-regularized multi-view learning. arXiv preprint arXiv:1206.6481, 2012.

[9]

Y. Guo and M. Xiao. Transductive representation learning for cross-lingual text classification. In Data Mining (ICDM), 2012 IEEE 12th International Conference on, pages 888--893. IEEE, 2012.

Digital Library

[10]

A. Klementiev, I. Titov, and B. Bhattarai. Inducing crosslingual distributed representations of words. In COLING, pages 1459--1474, 2012.

[11]

T. Kočiskỳ, K. M. Hermann, and P. Blunsom. Learning bilingual word representations by marginalizing alignments. arXiv preprint arXiv:1405.0947, 2014.

[12]

D. D. Lewis, Y. Yang, T. G. Rose, and F. Li. Rcv1: A new benchmark collection for text categorization research. The Journal of Machine Learning Research, 5:361--397, 2004.

Digital Library

[13]

X. Ling, G.-R. Xue, W. Dai, Y. Jiang, Q. Yang, and Y. Yu. Can chinese web pages be classified with english data source? In Proceedings of the 17th international conference on World Wide Web, pages 969--978. ACM, 2008.

Digital Library

[14]

H. Liu and Y. Yang. Bipartite edge prediction via transductive learning over product graphs. In Proceedings of The 32nd International Conference on Machine Learning, pages 1880--1888, 2015.

[15]

A. McCallum, K. Nigam, et al. A comparison of event models for naive bayes text classification. In AAAI-98 workshop on learning for text categorization, volume 752, pages 41--48, 1998.

[16]

T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.

[17]

T. Mikolov, Q. V. Le, and I. Sutskever. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168, 2013.

[18]

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111--3119, 2013.

Digital Library

[19]

T. Mikolov, W.-t. Yih, and G. Zweig. Linguistic regularities in continuous space word representations. In HLT-NAACL, pages 746--751, 2013.

[20]

S. C. A. P, S. Lauly, H. Larochelle, M. M. Khapra, B. Ravindran, V. C. Raykar, and A. Saha. An autoencoder approach to learning bilingual word representations. In Advances in Neural Information Processing Systems, pages 1853--1861, 2014.

Digital Library

[21]

L. Rigutini, M. Maggini, and B. Liu. An em based training algorithm for cross-language text categorization. In Web Intelligence, 2005. Proceedings. The 2005 IEEE/WIC/ACM International Conference on, pages 529--535. IEEE, 2005.

Digital Library

[22]

L. Shi, R. Mihalcea, and M. Tian. Cross language text classification by model translation and semi-supervised learning. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1057--1067. Association for Computational Linguistics, 2010.

Digital Library

[23]

A. Søgaard, Željko Agić, H. M. Alonso, B. Plank, B. Bohnet, and A. Johannsen. Inverted indexing for cross-lingual nlp. In Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, page 1713--1722, 2015.

[24]

S. Upadhyay, M. Faruqui, C. Dyer, and D. Roth. Cross-lingual models of word embeddings: An empirical comparison. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, page 1661--1670, 2016.

[25]

I. Vulic and M.-F. Moens. Bilingual word embeddings from non-parallel document-aligned data applied to bilingual lexicon induction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL 2015), page 719--725. ACL, 2015.

[26]

X. Wan. Co-training for cross-lingual sentiment classification. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 235--243. Association for Computational Linguistics, 2009.

Digital Library

[27]

W. Y. Zou, R. Socher, D. M. Cer, and C. D. Manning. Bilingual word embeddings for phrase-based machine translation. In EMNLP, pages 1393--1398, 2013.

Cited By

Vo T(2022)An Integrated Topic Modelling and Graph Neural Network for Improving Cross-lingual Text ClassificationACM Transactions on Asian and Low-Resource Language Information Processing10.1145/353026022:1(1-18)Online publication date: 25-Nov-2022
https://dl.acm.org/doi/10.1145/3530260
Li ZPetegrosso RSmith SSterling DKarypis GKuang R(2022)Scalable Label Propagation for Multi-Relational Learning on the Tensor Product of GraphsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.306398534:12(5964-5978)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TKDE.2021.3063985
Barbosa AFerreira MFerreira Mello RDueire Lins RGasevic DScheffel MDowell NJoksimovic SSiemens G(2021)The impact of automatic text translation on classification of online discussions for social and cognitive presencesLAK21: 11th International Learning Analytics and Knowledge Conference10.1145/3448139.3448147(77-87)Online publication date: 12-Apr-2021
https://dl.acm.org/doi/10.1145/3448139.3448147
Show More Cited By

Index Terms

Cross-lingual Text Classification via Model Translation with Limited Dictionaries
1. General and reference
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Clustering and classification

Recommendations

Low-Resource Text Classification via Cross-Lingual Language Model Fine-Tuning
Chinese Computational Linguistics
Abstract
Text classification tends to be difficult when data are inadequate considering the amount of manually labeled text corpora. For low-resource agglutinative languages including Uyghur, Kazakh, and Kyrgyz (UKK languages), in which words are ...
Generalized Funnelling: Ensemble Learning and Heterogeneous Document Embeddings for Cross-Lingual Text Classification
Funnelling (Fun) is a recently proposed method for cross-lingual text classification (CLTC) based on a two-tier learning ensemble for heterogeneous transfer learning (HTL). In this ensemble method, 1st-tier classifiers, each working on a different and ...
SWAT: Cross-lingual lexical substitution using local context matching, bilingual dictionaries and machine translation
SemEval '10: Proceedings of the 5th International Workshop on Semantic Evaluation

We present two systems that select the most appropriate Spanish substitutes for a marked word in an English test sentence. These systems were official entries to the SemEval-2010 Cross-Lingual Lexical Substitution task. The first system, Swat-E, finds ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

October 2016

2566 pages

ISBN:9781450340731

DOI:10.1145/2983323

General Chairs:
Snehasis Mukhopadhyay
Indiana University Purdue University Indianapolis, USA
,
ChengXiang Zhai
University of Illinois at Urbana-Champaign, USA
,
Program Chairs:
Elisa Bertino
Purdue University
,
Fabio Crestani
University of Lugano
,
Javed Mostafa
University of North Carolina
,
Jie Tang
Tsinghua University
,
Luo Si
Alibaba Group Inc & Purdue University
,
Xiaofang Zhou
University of Queensland
,
Yi Chang
Yahoo Research
,
Yunyao Li
IBM Research - Almaden
,
Parikshit Sondhi
WalmartLabs

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CIKM'16

Sponsor:

CIKM'16: ACM Conference on Information and Knowledge Management

October 24 - 28, 2016

Indiana, Indianapolis, USA

Acceptance Rates

CIKM '16 Paper Acceptance Rate 160 of 701 submissions, 23%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
827
Total Downloads

Downloads (Last 12 months)112
Downloads (Last 6 weeks)11

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vo T(2022)An Integrated Topic Modelling and Graph Neural Network for Improving Cross-lingual Text ClassificationACM Transactions on Asian and Low-Resource Language Information Processing10.1145/353026022:1(1-18)Online publication date: 25-Nov-2022
https://dl.acm.org/doi/10.1145/3530260
Li ZPetegrosso RSmith SSterling DKarypis GKuang R(2022)Scalable Label Propagation for Multi-Relational Learning on the Tensor Product of GraphsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.306398534:12(5964-5978)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TKDE.2021.3063985
Barbosa AFerreira MFerreira Mello RDueire Lins RGasevic DScheffel MDowell NJoksimovic SSiemens G(2021)The impact of automatic text translation on classification of online discussions for social and cognitive presencesLAK21: 11th International Learning Analytics and Knowledge Conference10.1145/3448139.3448147(77-87)Online publication date: 12-Apr-2021
https://dl.acm.org/doi/10.1145/3448139.3448147
Lang XLin YQiu D(2021)Dropout Graph Product for Improved Relationship Discovery Across Multiple Heterogeneous GraphsIEEE Access10.1109/ACCESS.2021.30871859(106340-106351)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3087185
Gupta PRaman S(2021)Translate2Classify: Machine Translation for E-Commerce Product Categorization in Comparison with Machine Learning & Deep Learning ClassificationProceedings of Data Analytics and Management10.1007/978-981-16-6285-0_60(769-788)Online publication date: 22-Nov-2021
https://doi.org/10.1007/978-981-16-6285-0_60
Singla KNarayanan S(2020)Multitask Learning for Darpa Lorelei’s Situation Frame Extraction TaskICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP40776.2020.9054710(8149-8153)Online publication date: May-2020
https://doi.org/10.1109/ICASSP40776.2020.9054710
Rabbimov IMporas ISimaki VKobilov S(2020)Investigating the Effect of Emoji in Opinion Classification of Uzbek Movie Review CommentsSpeech and Computer10.1007/978-3-030-60276-5_42(435-445)Online publication date: 29-Sep-2020
https://doi.org/10.1007/978-3-030-60276-5_42
Littell PTian TXu RSheikh ZMortensen DLevin LTyers FHayashi HHorwood GSloto STagtow EBlack AYang YMitamura THovy E(2018)The ARIEL-CMU situation frame detection pipeline for LoReHLT16Machine Translation10.1007/s10590-017-9205-332:1-2(105-126)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.1007/s10590-017-9205-3

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents