More Web Proxy on the site http://driver.im/

short-paper

Improving Multilabel Text Classification with Stacking and Recurrent Neural Networks

Authors:

Rodrigo Mansueli,

Marcos Aurélio Domingues,

Valéria Delisandra FeltrimAuthors Info & Claims

WebMedia '22: Proceedings of the Brazilian Symposium on Multimedia and the Web

Pages 117 - 122

https://doi.org/10.1145/3539637.3557000

Published: 07 November 2022 Publication History

Abstract

Multilabel text classification can be defined as a mapping function that categorizes a text in natural language into one or more labels defined by the scope of a problem. In this work we propose an architecture of stacked classifiers for multilabel text classification. The proposed models use an LSTM recurrent neural network in the first stage of the stack and different multilabel classifiers in the second stage. We evaluated our proposal in two datasets well-known in the literature (TMDB and EUR-LEX Subject Matters), and the results showed that the proposed stack consistently outperforms the baselines.

References

[1]

Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik. 1992. A Training Algorithm for Optimal Margin Classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory (Pittsburgh, Pennsylvania, USA) (COLT ’92). Association for Computing Machinery, New York, NY, USA, 144–152. https://doi.org/10.1145/130385.130401

Digital Library

[2]

Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1724–1734. https://doi.org/10.3115/v1/D14-1179

[3]

Corinna Cortes and Vladimir Vapnik. 1995. Support-Vector Networks. Machine Learning 20, 3 (1995), 273–297. https://doi.org/10.1007/BF00994018

[4]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186. https://doi.org/10.18653/v1/N19-1423

[5]

Katti Faceli, Ana Carolina Lorena, João Gama, and André C. P. L. F de Carvalho. 2019. Inteligência Artificial - Uma Abordagem de Aprendizado de Máquina (3 ed.). Grupo Gen - LTC.

[6]

Francisco Herrera, Francisco Charte, Antonio J. Rivera, and María J. del Jesus. 2016. Multilabel Classification. Springer International Publishing.

[7]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

Digital Library

[8]

Anwesha Law and Ashish Ghosh. 2019. Multi-label classification using a cascade of stacked autoencoder and extreme learning machines. Neurocomputing 358(2019), 222–234. https://doi.org/10.1016/j.neucom.2019.05.051

Digital Library

[9]

Rémi Lebret and Ronan Collobert. 2014. Word Embeddings through Hellinger PCA. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Gothenburg, Sweden, 482–490. https://doi.org/10.3115/v1/E14-1051

[10]

Eneldo Loza Mencía and Johannes Fürnkranz. 2010. Efficient Multilabel Classification Algorithms for Large-Scale Problems in the Legal Domain. In Semantic Processing of Legal Texts: Where the Language of Law Meets the Law of Language. Springer Berlin Heidelberg, Berlin, Heidelberg, 192–215. https://doi.org/10.1007/978-3-642-12837-0_11

[11]

Eneldo Loza Mencía and Frederik Janssen. 2016. Learning rules for multi-label classification: a stacking and a separate-and-conquer approach. Machine Learning 105, 1 (2016), 77–126. https://doi.org/10.1007/s10994-016-5552-1

Digital Library

[12]

Rafael B. Mangolin, Rodolfo M. Pereira, Alceu S. Britto, Carlos N. Silla, Valéria D. Feltrim, Diego Bertolini, and Yandre M. G. Costa. 2022. A Multimodal Approach for Multi-Label Movie Genre Classification. Multimedia Tools Appl. 81, 14 (2022), 19071–19096. https://doi.org/10.1007/s11042-020-10086-2

Digital Library

[13]

Gonçalo Marques, Marcos Aurélio Domingues, Thibault Langlois, and Fabien Gouyon. 2011. Three Current Issues In Music Autotagging. In Proceedings of the 12th International Society for Music Information Retrieval Conference, ISMIR 2011, Miami, Florida, USA, 2011. 795–800. http://ismir2011.ismir.net/papers/OS10-1.pdf

[14]

Thomas M. Mitchell. 1997. Machine Learning (1ed.). McGraw-Hill, Inc., USA.

[15]

Elena Montañes, Robin Senge, Jose Barranquero, José Ramón Quevedo, Juan José del Coz, and Eyke Hüllermeier. 2014. Dependent binary relevance models for multi-label classification. Pattern Recognition 47, 3 (2014), 1494–1508. https://doi.org/10.1016/j.patcog.2013.09.029

Digital Library

[16]

Rodrigo Mansueli Nunes. 2021. Explorando stacking na classificação automática de textos multirrótulos. Master’s thesis. Universidade Estadual de Maringá. https://sucupira.capes.gov.br/sucupira/public/consultas/coleta/trabalhoConclusao/viewTrabalhoConclusao.jsf?popup=true&id_trabalho=11010061

[17]

H. Peng, J. Li, S. Wang, L. Wang, Q. Gong, R. Yang, B. Li, P. Yu, and L. He. 2019. Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification. IEEE Transactions on Knowledge and Data Engineering (2019), 1–1. https://doi.org/10.1109/TKDE.2019.2959991

[18]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1532–1543. https://doi.org/10.3115/v1/D14-1162

[19]

Giuseppe Portolese, Marcos Aurélio Domingues, and Valéria Delisandra Feltrim. 2019. Exploring Textual Features for Multi-label Classification of Portuguese Film Synopses. In Progress in Artificial Intelligence, 19th EPIA Conference on Artificial Intelligence, EPIA 2019, Vila Real, Portugal, 2019, Proceedings, Part II(Lecture Notes in Computer Science, Vol. 11805). Springer, 669–681. https://doi.org/10.1007/978-3-030-30244-3_55

Digital Library

[20]

Giuseppe Portolese and Valéria Feltrin. 2018. On the Use of Synopsis-based Features for Film Genre Classification. In Anais do XV Encontro Nacional de Inteligência Artificial e Computacional (São Paulo). SBC, Porto Alegre, RS, Brasil, 892–902. https://doi.org/10.5753/eniac.2018.4476

[21]

David Martin Powers. 2011. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies 2 (2011), 37–63.

[22]

J. Ross Quinlan. 1986. Induction of Decision Trees. Machine Learning 1, 1 (1986), 81–106. https://doi.org/10.1023/A:1022643204877

[23]

Raul Rojas. 1996. Neural networks : a systematic introduction. Springer-Verlag, Berlin New York.

[24]

Fabrizio Sebastiani. 2002. Machine Learning in Automated Text Categorization. ACM Comput. Surv. 34, 1 (2002), 1–47. https://doi.org/10.1145/505282.505283

Digital Library

[25]

Muhammad Atif Tahir, Josef Kittler, and Ahmed Bouridane. 2016. Multi-label classification using stacked spectral kernel discriminant analysis. Neurocomputing 171(2016), 127–137. https://doi.org/10.1016/j.neucom.2015.06.023

Digital Library

[26]

Pang-Ning Tan, Michael S. Steinbach, and Vipin Kumar. 2005. Introduction to Data Mining. Addison-Wesley.

Digital Library

[27]

G. Tsoumakas, I. Katakis, and I. Vlahavas. 2011. Random k-Labelsets for Multilabel Classification. IEEE Transactions on Knowledge and Data Engineering 23, 7(2011), 1079–1089. https://doi.org/10.1109/TKDE.2010.164

Digital Library

[28]

Ran Wang, Robert Ridley, Xiao Su, Weiguang Qu, and Xinyu Dai. 2021. A novel reasoning mechanism for multi-label text classification. Information Processing & Management 58, 2 (2021), 102441. https://doi.org/10.1016/j.ipm.2020.102441

[29]

Ian H. Witten, Eibe Frank, Mark A. Hall, and Christopher J. Pal. 2016. Data Mining, Fourth Edition: Practical Machine Learning Tools and Techniques (4th ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

[30]

Yuelong Xia, Ke Chen, and Yun Yang. 2021. Multi-label classification with weighted classifier selection and stacked ensemble. Information Sciences 557(2021), 421–442. https://doi.org/10.1016/j.ins.2020.06.017

Cited By

Aras AAlikaşifoğlu TKoç A(2024)Graph Receptive Transformer Encoder for Text ClassificationIEEE Transactions on Signal and Information Processing over Networks10.1109/TSIPN.2024.338036210(347-359)Online publication date: 2024
https://doi.org/10.1109/TSIPN.2024.3380362
Liu YXu FZhao YMa ZWang TZhang STian Y(2024)Hierarchical multi-instance multi-label learning for Chinese patent text classificationConnection Science10.1080/09540091.2023.229581836:1Online publication date: 3-Jan-2024
https://doi.org/10.1080/09540091.2023.2295818

Index Terms

Improving Multilabel Text Classification with Stacking and Recurrent Neural Networks
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification

Recommendations

Convolutional Recurrent Neural Networks for Text Classification

Recurrent neural network (RNN) and convolutional neural network (CNN) are two prevailing architectures used in text classification. Traditional approaches combine the strengths of these two networks by straightly streamlining them or linking features ...
Randomized neural networks for multilabel classification
Abstract
Multilabel classification is a supervised learning problem in which input instances belong to multiple output labels. In this paper, we propose noniterative randomization-based neural networks for multilabel classification. These ...
Highlights
- Randomized neural network based approaches ML-RVFL, ML-KRVFL, ML-BLS, and ML-FBLS are proposed for multilabel classification.
Graph Neural Networks-Based Multilabel Classification of Citation Network
Intelligent Information and Database Systems
Abstract
There is an increasing number of applications where data can be represented as graphs. Besides, it is well-known that artificial intelligence approaches have become a very active and promising research field, mostly due to deep learning ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WebMedia '22: Proceedings of the Brazilian Symposium on Multimedia and the Web

November 2022

389 pages

ISBN:9781450394093

DOI:10.1145/3539637

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

WebMedia '22

WebMedia '22: Brazilian Symposium on Multimedia and Web

November 7 - 11, 2022

Curitiba, Brazil

Acceptance Rates

Overall Acceptance Rate 270 of 873 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
65
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Aras AAlikaşifoğlu TKoç A(2024)Graph Receptive Transformer Encoder for Text ClassificationIEEE Transactions on Signal and Information Processing over Networks10.1109/TSIPN.2024.338036210(347-359)Online publication date: 2024
https://doi.org/10.1109/TSIPN.2024.3380362
Liu YXu FZhao YMa ZWang TZhang STian Y(2024)Hierarchical multi-instance multi-label learning for Chinese patent text classificationConnection Science10.1080/09540091.2023.229581836:1Online publication date: 3-Jan-2024
https://doi.org/10.1080/09540091.2023.2295818

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents