Proceedings of the 18th Conference on Computer Science and Intelligence Systems

DOI: http://dx.doi.org/10.15439/2023F3561

Citation: Proceedings of the 18th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 35, pages 281–284 (2023)

Full text

Abstract. The exponential growth of data generated by the Moroccan commercial court system, coupled with the manual archiving of legal documents, has led to increasingly complex information access. As data classification becomes imperative, researchers are exploring automatic language processing tech- niques and refining text classification methods. In this study, we propose a BERT-CLSTM model for the classification of Moroccan commercial court verdicts. By adding a Convolutional Long Short-Term Memory Network to the task-specific layers of BERT, our model can get information on important fragments in the text. In addition, we input the representation along with the output of the BERT into the transformer encoder to take advantage of the self-attention mechanism and finally get the representation of the whole text through the transformer. The proposed model outperformed the compared baselines and achieved good results by getting an F-measure value of 93.61%

References

1. C. Blake, “Text Mining,” Annual review of information science and technology, vol. 45, 2011, pp. 121–155.
2. X. Zhang, J. Zhao, Y. LeCun, “Character-level convolutional networks for text classification,” Advances in Neural Information Processing Systems, 2015.
3. A. Conneau, H. Schwenk, L. Barrault, Y. LeCun, “Very deep convolutional networks for text classification,” Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, vol. 1, 2017, pp. 1107–1116.
4. Y. Kim, “Convolutional neural networks for sentence classification,” Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), vol. 1, 2014, pp. 1746–1751.
5. K. Sheng Tai, R. Socher, C.D. Manning, “Improved semantic representations from tree-structured long short-term memory networks,” Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, vol. 1, 2015, pp. 1556–1566.
6. M. Huang, Q. Qian, X. Zhu, “Encoding syntactic knowledge in neural networks for sentiment classification,” ACM Transactions on Information Systems, vol. 35, 2017, pp. 1–27.
7. H. El Rifai, L. Al Qadi, A. Elnagar, “Arabic text classification: the need for multi-labeling systems,” Neural Computing and Applications, vol. 34, 2022, pp. 1135–1159.
8. M. AL-Qurishi, S. AlQaseemi, R. Soussi, “AraLegal-BERT: A pretrained language model for Arabic Legal text,” Proceedings of the Natural Legal Language Processing Workshop 2022, 2022, pp. 338–344.
9. F. El-Alami, A. El Mahdaouy, S.O. El Alaoui, N. En-Nahnahi, “A deep autoencoder-based representation for Arabic text categorization,” Journal of Information and Communication Technology, vol. 3, 2020, pp. 381–398.
10. W.H.G. Gwad, I.M.I. Ismael, Y. Gultepe, “Twitter Sentiment Analysis Classification in the Arabic Language using Long Short-Term Memory Neural Networks,” International Journal of Engineering and Advanced Technology, vol. 9, 2020, pp. 235–239.
11. M. Alhawarat, A.O. Aseeri, “A Superior Arabic Text Categorization Deep Model (SATCDM),” IEEE Access, vol. 8, 2020, pp. 24653–24661.
12. M. Galal, M.M. Madbouly, A. El-Zoghby, “Classifying Arabic text using deep learning,” Journal of Theoretical and Applied Information Technology, vol. 97, 2019, pp. 3412–3422.
13. S. Boukil, M. Biniz, F. El Adnani, L. Cherrat, A.E. El Moutaouakkil, “Arabic Text Classification Using Deep Learning Technics,” International journal of grid and distributed computing, vol. 11, 2018, pp. 103–114.
14. R. Al-khurayji and A. Sameh, “An Effective Arabic Text Classification Approach Based on Kernel Naive Bayes Classifier,” International Journal of Artificial Intelligence and Applications, vol. 8, 2017, pp. 01–10.