Abstract
Argument mining (AM) is defined as the task of automatically identifying and extracting argumentative components (e.g. premises, claims, etc.) and detecting the existing relations among them (i.e., support, attack, no relations). Deep learning models enable us to analyze arguments more efficiently than traditional methods and extract their semantics. This paper presents comparative studies between a few deep learning-based models in argument mining. The work concentrates on argument classification. The research was done on a wide spectrum of datasets (Args.me, UKP, US2016). The main novelty of this paper is the ensemble model which is based on BERT architecture and ChatGPT-4 as fine tuning model. The presented results show that BERT+ChatGPT-4 outperforms the rest of the models including other Transformer-based and LSTM-based models. The observed improvement is, in most cases, greater than 10%. The presented analysis can provide crucial insights into how the models for argument classification should be further improved. Additionally, it can help develop a prompt-based algorithm to eliminate argument classification errors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Schaefer, R., Stede, M.: GerCCT: an annotated corpus for mining arguments in German tweets on climate change. In: Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 6121–6130. European Language Resources Association, Marseille (2022)
Lawrence, J., Reed, C.: Argument mining: a survey. Comput. Linguist. 45(4), 765–818 (2019)
Lippi, M., Torroni, P.: Argumentation mining: state of the art and emerging trends. ACM Trans. Internet Technol. 16(2), 1–25 (2016). https://doi.org/10.1145/2850417. Article 10
Park, J., Cardie, C.: A corpus of eRulemaking user comments for measuring evaluability of arguments. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC (2018)
Moens, M. F., Boiy, E., Palau, R. M., Reed, C.: Automatic detection of arguments in legal texts. In: Proceedings of the 11th International Conference on Artificial Intelligence and Law, Stanford, CA, pp. 225–230 (2007)
Levy, R., Bilu, Y., Hershcovich, D., Aharoni, E., Slonim, N.: Context dependent claim detection. In: Proceedings of the 25th International Conference on Computational Linguistics, Dublin, pp. 1489–1500 (2014)
Stab, C., Gurevych, I.: Annotating argument components and relations in persuasive essays. In: Proceedings of the 25th International Conference on Computational Linguistics, Dublin, pp. 1501–1510 (2014)
Dusmanu, M., Cabrio, E., Villata, S.: Argument mining on Twitter: arguments, facts and sources. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, pp. 2317–2322 (2017)
Srivastava, P., Bhatnagar, P., Goel, A.: Argument mining using BERT and self-attention based embeddings. In: 2022 4th International Conference on Advances in Computing, Communication Control and Networking (ICAC3N) (2022). https://doi.org/10.48550/arXiv.2302.13906
Niculae, V., Park, J., Cardie, C.: Argument mining with structured SVMs and RNNs. arXiv preprint arXiv:1704.06869 (2017)
Galassi, A., Lippi, M., Torroni, P.: Argumentative link prediction using residual networks and multi-objective learning. In: Proceedings of the 5th Workshop on Argument Mining, pp. 1–10 (2018)
Morio, G., Ozaki, H., Morishita, T., Koreeda, Y., Yanai, K.: Towards better non-tree argument mining: Proposition-level biaffine parsing with task-specific parameterization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3259–3266 (2020)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186 (2019)
Reimers, N., Schiller, B., Beck, T., Daxenberger, J., Stab, C., Gurevych, I.: Classification and clustering of arguments with contextualized word embeddings. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 567–578 (2019)
Chakrabarty, T., Hidey, C., Muresan, S., McKeown, K., Hwang, A.: AMPERSAND: argument mining for PERSuAsive oNline discussions. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 2933–2943 (2019)
Chen, T.: BERT argues: how attention informs argument mining (2021). Honors Theses. 1589. https://scholarship.richmond.edu/honors-theses/1589
Ruiz-Dolz, R., Heras, S., Alemany, J., García-Fornes, A.: Transformer-based models for automatic identification of argument relations: a cross-domain evaluation. arXiv (2020). https://doi.org/10.48550/arXiv.2011.13187
Ajjour, Y., Wachsmuth, H., Kiesel, J., Potthast, M., Hagen, M., Stein, B.: Data acquisition for argument search: the args.me corpus. In: Benzmüller, C., Stuckenschmidt, H. (eds.) KI 2019. LNCS (LNAI), vol. 11793, pp. 48–59. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30179-8_4
Bar-Haim, R., Bhattacharya, I., Dinuzzo, F., Saha, A., Slonim, N.: Stance classification of context-dependent claims. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pp. 251–261 (2017)
Boltuzic, F., Snajder, J.: Back up your stance: recognizing arguments in online discussions. In: Proceedings of the First Workshop on Argumentation Mining, pp. 49–58 (2014)
Cabrio, E., Villata, S.: Node: a benchmark of natural language arguments. In: Computational Models of Argument - Proceedings of COMMA 2014, Atholl Palace Hotel, Scottish Highlands, UK, 9–12 September 2014, pp. 449–450 (2014)
Carstens, L., Toni, F.: Using argumentation to improve classification in natural language problems. ACM Trans. Internet Technol. 17(3), 30:1–30:23 (2017)
Stab, C., Miller, T., Schiller, B., Rai, P., Gurevych, I.: Cross-topic argument mining from heterogeneous sources. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3664–3674. Association for Computational Linguistics, Brussels (2018)
Visser, J., Lawrence, J., Wagemans, J., Reed, C.: An annotated corpus of argument schemes in US election debates. In: Proceedings of the 9th Conference of the International Society for the Study of Argumentation (ISSA), pp. 1101–1111 (2018)
Budzynska, K., Reed, C.: Speech acts of argumentation: inference anchors and peripheral cues in dialogue. In: Proceedings of the 10th AAAI Conference on Computational Models of Natural Argument (AAAIWS 2011-10), pp. 3–10. AAAI Press (2011)
Carletta, J.: Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist. 22(2), 249–254 (1996)
Li, K., et al.: An empirical study of transformer-based neural language model adaptation. In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, pp. 7934–7938 (2020). https://doi.org/10.1109/ICASSP40776.2020.9053399
Yao, S., et al.: Tree of thoughts: deliberate problem solving with large language models. ArXiv (2023). https://arxiv.org/abs/2305.10601
Akiki, C., Potthast, M.: Exploring argument retrieval with transformers. CLEF (2020). https://ceur-ws.org/Vol-2696
Acknowledgements
This research was funded by Narodowe Centrum Nauki (National Science Centre, Republic of Poland), the research grant “Argument Mining: Public Debate Models and Algorithmically-Assisted Argument Extraction” (“Argument mining: Modele debat publicznych i wspomagana algorytmicznie ekstrakcja argumentów”).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pietron, M., Olszowski, R., Gomułka, J. (2024). Efficient Argument Classification with Compact Language Models and ChatGPT-4 Refinements. In: Nguyen, N.T., et al. Computational Collective Intelligence. ICCCI 2024. Lecture Notes in Computer Science(), vol 14810. Springer, Cham. https://doi.org/10.1007/978-3-031-70816-9_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-70816-9_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70815-2
Online ISBN: 978-3-031-70816-9
eBook Packages: Computer ScienceComputer Science (R0)