Abstract
Text classification has become relevant in recent years because of its usefulness in supporting different text mining solutions. Neural networks for this purpose have benefited from the creation of word embedding for learning semantics among words in a corpus. However, artificial neural network training by conventional methods present several theoretical and computational limitations. In this work, we develop a hybrid training method that combines gradient-based methods and Covariance Matrix Adaptation Evolution Strategy to train Convolutional Neural Network for the text classification task. For this, the training process is divided into two stages taking advantage of the speed of the gradient-based methods for learning the parameters of the convolutional filters and the application of the Covariance Matrix Adaptation Evolution Strategy for learning the weights of the fully connected layer. Our proposal was evaluated using a Spanish dataset for text classification, taken off the EcuRed Cuban Encyclopedia, divided into five classes. The proposed method increases the accuracy significantly of the convolutional network applied to the text classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
Dataset available at: https://github.com/ogtoledano/ecured_five_tags.
- 5.
Source code at: https://github.com/ogtoledano/Text_Cat_Based_EDA.
References
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195 (2001)
Hartmann, J., Huppertz, J., Schamp, C., Heitmann, M.: Comparing automated text classification methods. Int. J. Res. Mark. 36(1), 20–38 (2019)
Indolia, S., Kumar, A., Mishra, S.P., Asopa, P.: Conceptual understanding of convolutional neural network- a deep learning approach. Proc. Comput. Sci. 132, 679–688 (2018)
Kiefer, J., Wolfowitz, J.: Stochastic estimation of the maximum of a regression function. Ann. Math. Stat. 23(3), 462–466 (1952)
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–15 (2015)
Kowsari, K., Meimandi, K.J., Heidarysafa, M., Mendu, S., Barnes, L., Brown, D.: Text classification algorithms: a survey (2019)
Larrañaga, P., Lozano, J.A.: Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation, vol. 2. Springer, Heidelberg (2002). https://doi.org/10.1007/978-1-4615-1539-5
Le, L.T., Nguyen, H., Dou, J., Zhou, J.: A comparative study of PSO-ANN, GA-ANN, ICA-ANN, and ABC-ANN in estimating the heating load of buildings’ energy efficiency for smart city planning. Appl. Sci. (Switzerland) 9(13), 2630 (2019)
Liang, Y., et al.: Fusion of heterogeneous attention mechanisms in multi-view convolutional neural network for text classification. Inf. Sci. 548, 295–312 (2021)
Luan, Y., Lin, S.: Research on text classification based on CNN and LSTM. In: Proceedings of 2019 IEEE International Conference on Artificial Intelligence and Computer Applications, ICAICA 2019, pp. 352–355 (2019)
Madera, J., Dorronsoro, B.: Estimation of distribution algorithms. In: Alba, E., Martí, R. (eds.) Metaheuristic Procedures for Training Neural Networks, 1 edn., pp. 87–108. Springer, Boston (2006). https://doi.org/10.1007/0-387-33416-5_5. ISBN: 978-0-387-33415-8
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 1–9 (2013)
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., Gao, J.: Deep learning based text classification: a comprehensive review (2020)
Nair, V., Mohapatra, S.K., Malhotra, R.: A machine learning algorithm for product classification based on unstructured text description. Int. J. Eng. Res. Technol. 7(06), 404–407 (2018)
Ojha, V.K., Abraham, A., Snásel, V.: Metaheuristic design of feedforward neural networks : a review of two decades of research. Eng. Appl. Artif. Intell. 60, 97–116 (2017)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics (2014)
Raunak, V., Metze, F.: Effective dimensionality reduction for word embeddings. In: Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), , Florence, pp. 235–243. Association for Computational Linguistics (2019)
Reddy, T., Williams, R., Breazeal, C.: Text classification for AI education. In: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, SIGCSE 2021, pp. 1381. Association for Computing Machinery, New York (2021)
Rojas-Delgado, J., Trujillo-Rasúa, R., Bello, R.: A continuation approach for training Artificial Neural Networks with meta-heuristics. Pattern Recogn. Lett. 125, 373–380 (2019)
Shen, H.: Towards a mathematical understanding of the difficulty in learning with feedforward neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 811–820 (2018)
Suyan, W., Entong, S., Binyang, L., Jiangrui, W.: TextCNN-based text classification for E-government. In: International Conference on Information Science and Control Engineering, ICISCE 2019, pp. 929–934 (2019)
Wang, R., Li, Z., Cao, J., Chen, T., Wang, L.: Convolutional recurrent neural networks for text classification. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2019-July, no. 2018, pp. 1–6 (2019)
Wu, H., Liu, Y., Wang, J.: Review of text classification methods on deep learning. Comput. Mater. Continua 63(3), 1309–1321 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Toledano-López, O.G., Madera, J., González, H., Cuevas, A.S. (2021). Covariance Matrix Adaptation Evolution Strategy for Convolutional Neural Network in Text Classification. In: Hernández Heredia, Y., Milián Núñez, V., Ruiz Shulcloper, J. (eds) Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2021. Lecture Notes in Computer Science(), vol 13055. Springer, Cham. https://doi.org/10.1007/978-3-030-89691-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-89691-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89690-4
Online ISBN: 978-3-030-89691-1
eBook Packages: Computer ScienceComputer Science (R0)