BERT-Based Transformers for Early Detection of Mental Health Illnesses
Authors: 
Rodrigo Martínez-Castaño, Amal Htait, 
Leif Azzopardi, 
Yashar MoshfeghiAuthors Info & Claims
Pages 189 - 200
Abstract
This paper briefly describes our research groups’ efforts in tackling Task 1 (Early Detection of Signs of Self-Harm), and Task 2 (Measuring the Severity of the Signs of Depression) from the CLEF eRisk Track. Core to how we approached these problems was the use of BERT-based classifiers which were trained specifically for each task. Our results on both tasks indicate that this approach delivers high performance across a series of measures, particularly for Task 1, where our submissions obtained the best performance for precision, F1, latency-weighted F1 and ERDE at 5 and 50. This work suggests that BERT-based classifiers, when trained appropriately, can accurately infer which social media users are at risk of self-harming, with precision up to 91.3% for Task 1. Given these promising results, it will be interesting to further refine the training regime, classifier and early detection scoring mechanism, as well as apply the same approach to other related tasks (e.g., anorexia, depression, suicide).
References
[1]
Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., Blackburn, J.: The pushshift reddit dataset. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 14, pp. 830–839 (2020)
[2]
Burdisso SG, Errecalde M, and Montes-y Gómez M A text classification framework for simple and effective early depression detection over social media streams Expert Syst. Appl. 2019 133 182-197
[3]
Burdisso, S.G., Errecalde, M., Montes-y Gómez, M.: UNSL at eRisk 2019: a unified approach for anorexia, self-harm and depression detection in social media. In: CLEF (Working Notes) (2019)
[4]
Chancellor S and De Choudhury M Methods in predictive techniques for mental health status on social media: a critical review NPJ Digit. Med. 2020 3 1 1-11
[5]
Cohan, A., Young, S., Goharian, N.: Triaging mental health forum posts. In: Proceedings of the Third Workshop on Computational Linguistics and Clinical Psychology, pp. 143–147 (2016)
[6]
Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8440–8451 (2020)
[7]
Devlin, J., Chang, M.W.: Open sourcing BERT: state-of-the-art pre-training for natural language processing (2018). http://aiweb.techfak.uni-bielefeld.de/content/bworld-robot-control-software/. Accessed 22 Apr 2021
[8]
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
[9]
Gao Z, Feng A, Song X, and Wu X Target-dependent sentiment classification with BERT IEEE Access 2019 7 154290-154299
[10]
Lample, G., Conneau, A.: Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291 (2019)
[11]
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
[12]
Losada DE, Crestani F, et al. Fuhr N et al. A test collection for research on depression and language use Experimental IR Meets Multilinguality, Multimodality, and Interaction 2016 Cham Springer 28-39
[13]
Losada, D.E., Crestani, F., Parapar, J.: CLEF 2017 eRisk overview: early risk prediction on the internet: experimental foundations. In: CEUR Workshop Proceedings, vol. 1866 (2017)
[14]
Losada, D.E., Crestani, F., Parapar, J.: Overview of eRisk 2018: early risk prediction on the internet (extended lab overview). In: CEUR Workshop Proceedings, vol. 2125 (2018)
[15]
Losada DE, Crestani F, Parapar J, et al. Crestani F et al. Overview of eRisk 2019 early risk prediction on the internet Experimental IR Meets Multilinguality, Multimodality, and Interaction 2019 Cham Springer 340-357
[16]
Losada DE, Crestani F, Parapar J, et al. Arampatzis A et al. Overview of eRisk 2020: early risk prediction on the internet Experimental IR Meets Multilinguality, Multimodality, and Interaction 2020 Cham Springer 272-287
[17]
Martínez-Castaño, R., Htait, A., Azzopardi, L., Moshfeghi, Y.: Early risk detection of self-harm and depression severity using BERT-based transformers: iLab at CLEF eRisk 2020. Early Risk Prediction on the Internet (2020)
[18]
Nikolov, A., Radivchev, V.: Nikolov-Radivchev at SemEval-2019 task 6: offensive tweet classification with BERT and ensembles. In: Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 691–695 (2019)
[19]
Obeid JS et al. Identifying and predicting intentional self-harm in electronic health record clinical notes: deep learning approach JMIR Med. Inform. 2020 8 7 e17784
[20]
Parikh, P., et al.: Multi-label categorization of accounts of sexism using a neural framework. In: EMNLP/IJCNLP (1) (2019)
[21]
Peters, M., et al.: Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long Papers), pp. 2227–2237 (2018)
[22]
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018). https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf. Accessed 22 Apr 2021
[23]
Sadeque, F., Xu, D., Bethard, S.: Measuring the latency of depression detection in social media. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 495–503 (2018)
[24]
Skaik R and Inkpen D Using social media for mental health surveillance: a review ACM Comput. Surv. (CSUR) 2020 53 6 1-31
[25]
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
[26]
Yates, A., Cohan, A., Goharian, N.: Depression and self-harm risk assessment in online forums. CoRR abs/1709.01848 (2017)
Index Terms
- BERT-Based Transformers for Early Detection of Mental Health Illnesses
Index terms have been assigned to the content through auto-classification.
Recommendations
Sensitive Self-disclosures, Responses, and Social Support on Instagram: The Case of #Depression
CSCW '17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social ComputingPeople can benefit from disclosing negative emotions or stigmatized facets of their identities, and psychologists have noted that imagery can be an effective medium for expressing difficult emotions. Social network sites like Instagram offer ...
Towards Suicide Prevention: Early Detection of Depression on Social Media
Internet ScienceAbstractThe statistics presented by the World Health Organization inform that 90% of the suicides can be attributed to mental illnesses in high-income countries. Besides, previous studies concluded that people with mental illnesses tend to reveal their ...
Role of social media in tackling challenges in mental health
SAM '13: Proceedings of the 2nd international workshop on Socially-aware multimediaMental illness is a serious and widespread health challenge in our society today. Tens of millions of people each year suffer from depression and only a fraction receives adequate treatment. This position paper highlights some recent attempts examining ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sep 2021
486 pages
ISBN:978-3-030-85250-4
DOI:10.1007/978-3-030-85251-1
© Springer Nature Switzerland AG 2021.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 21 September 2021
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Jan 2025