Abstract
The significant rise in suicides is a major cause of concern in public health domain. Depression plays a major role in increasing suicide ideation among the individuals. Although most of the suicides can be avoided with prompt intercession and early diagnosis, it has been a serious challenge to detect the at-risk individuals. Our current work focuses on learning three closely related tasks, viz. depression detection, sentiment citation, and to investigate their impact in analysing the mental state of the victims. We extend the existing standard emotion annotated corpus of suicide notes in English, CEASE, with additional 2539 sentences collected from 120 new notes. We annotate the consolidated corpus with appropriate depression labels and multi-label emotion classes. We further leverage weak supervision to annotate the corpus with sentiment labels. We propose a deep multitask framework that features a knowledge module that uses SenticNet’s IsaCore and AffectiveSpace vector-spaces to infuse external knowledge specific features into the learning process. The system models emotion recognition (the primary task), depression detection and sentiment classification (the secondary tasks) simultaneously. Experiments show that our proposed multitask system obtains the highest cross-validation MR of 56.47 %. Evaluation results show that all our multitask models perform better than their single-task variants indicating that the secondary tasks (depression detection and sentiment classification) improve the performance of the primary task (emotion recognition) when all tasks are learned jointly.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Resource available at: http://www.iitp.ac.in/~ai-nlp-ml/resources.html
Resource to be made available at: https://www.iitp.ac.in/~ai-nlp-ml/resources.html
Emotion labels: forgiveness, happiness_peacefulness, love, pride, hopefulness, thankfulness, blame, anger, fear, abuse, sorrow, hopelessness, guilt, information, instructions
abuse, anger, blame, fear, forgiveness, guilt, happiness_peacefulness, hopefulness, hopelessness, love, pride, sorrow, thankfulness, information and instructions
mood and suicidal thought tracking, safety plan development, the recommendation of activities to deter suicidal thoughts, information and education, access to support networks, and access to emergency counselling
’suicide notes’, ’suicide note \(+ <\mathrm {YEAR}>\)’, ’social media \(+\) suicide note’, etc.
List of some sources of collected notes:
https://www.denverpost.com/2007/12/07/mall-gunmans-suicide-note-i-just-snapped/
https://www.huffpost.com/entry/bill-zeller-dead-princeto_n_805689
https://twitter.com/harrisongolden/status/880816422145970177
https://www.mirror.co.uk/sport/football/news/gary-speeds-widow-speaks-suicide-13256742
http://www.homelandnewsng.com/other-news/4518-18-year-old-girl-kills-self-over-man
https://www.writechoice.co.in/writechoice/geetika-sharma-suicide-note/
Full list of countries: Australia, Canada, China, England, India, Ireland, Japan, Pakistan, Philippines, South Korea, USA and Zimbabwe
a high-level neural networks API: https://keras.io/
not necessarily same 100 words are considered for all 3 spaces, as vectors are fetched as on availability in the respective vector spaces
using loss_weights parameter of keras compile function
We perform Student’s t test for assessing the statistical significance
References
Abdul-Mageed M, Ungar. Emonet L. Fine-grained emotion detection with gated recurrent neural networks. In Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: Long papers) 2017. p. 718–728.
Akhtar MS, Ekbal A, Cambria E. How intense are you? predicting intensities of emotions and sentiments using stacked ensemble. IEEE Comput Intell Mag. 2020;15(1):64–75.
Akhtar MS, Ghosal D, Ekbal A, Bhattacharyya P, Kurohashi S. A multi-task ensemble framework for emotion, sentiment and intensity prediction. arXiv preprint 2018. arXiv:1808.01216
Bălan O, Moise G, Petrescu L, Moldoveanu A, Leordeanu M, Moldoveanu F. Emotion classification based on biophysical signals and machine learning techniques. Symmetry. 2020;12(1):21.
Batbaatar E, Li M, Ryu KH. Semantic-emotion neural network for emotion recognition from text. IEEE Access. 2019;7:111866–78.
Benton A, Mitchell M, Hovy D. Multi-task learning for mental health using social media text. arXiv preprint 2017. arXiv:1712.03538
Bucci W, Freedman N. The language of depression. Bull Men Clin. 1981;45(4):334.
Buechel S, Hahn U. Word emotion induction for multiple languages as a deep multi-task learning problem. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) 2018. p. 1907–1918.
Cambria E. Affective computing and sentiment analysis. IEEE Intell Syst. 2016;31(2):102–7.
Cambria E, Fu J, Bisio F, Poria S. Affectivespace 2: Enabling affective intuition for concept-level sentiment analysis. In B. Bonet and S. Koenig, editors, Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. AAAI Press, Austin, Texas, USA 2015. p. 508–514.
Cambria E, Li Y, Xing FZ, Poria S, Kwok K. Senticnet 6: Ensemble application of symbolic and subsymbolic ai for sentiment analysis. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management 2020. p. 105–114.
Cambria E, Song Y, Wang H, Howard N. Semantic multidimensional scaling for open-domain sentiment analysis. IEEE Intell Syst. 2012;29(2):44–51.
Canetto SS, Lester D. The epidemiology of women’s suicidal behavior. Springer Publishing Co, 1995.
Capstick A. Recognition of emotional disturbance and the prevention of suicide. British Medical Journal. 1960;1(5180):1179.
Caruana R. Multitask learning. Machine learning. 1997;28(1):41–75.
Chen SY, Hsu CC, Kuo CC, Ku LW, et al. Emotionlines: An emotion corpus of multi-party conversations. arXiv preprint 2018. arXiv:1802.08379
Cho K, Van Merriënboer B, Bahdanau D, Bengio Y. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint 2014. arXiv:1409.1259
Chollet F, et al. Keras: Deep learning library for theano and tensorflow. https://keras.io/k 7(8):T1, 2015.
Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37–46.
Dai J, He K, Sun J. Instance-aware semantic segmentation via multi-task network cascades. In Proceedings of the IEEE Conf Comput Vis Pattern Recognit 2016. p. 3150–3158.
Delgado-Gomez D, Blasco-Fontecilla H, Alegria AA, Legido-Gil T, Artes-Rodriguez A, Baca-Garcia E. Improving the accuracy of suicide attempter classification. Artif Intell Med. 2011;52(3):165–8.
Desmet B, Hoste V. Emotion detection in suicide notes. Expert Systems with Applications. 2013;40(16):6351–8.
Edelman AM, Renshaw SL. Genuine versus simulated suicide notes: an issue revisited through discourse analysis. Suicide and Life-Threatening Behavior. 1982;12(2):103–13.
Ekman P. An argument for basic emotions. Cognit Emot. 1992;6(3–4):169–200.
Ekman P, Sorenson ER, Friesen WV. Pan-cultural elements in facial displays of emotion. Science. 1969;164(3875):86–88.
Faundez-Zanuy M, Hussain A, Mekyska J, Sesa-Nogueras E, Monte-Moreno E, Esposito A, Chetouani M, Garre-Olmo J, Abel A, Smekal Z, et al. Biometric applications related to human beings: there is life beyond security. Cognit Comput. 2013;5(1):136–51.
Gawęda Ł, Pionke R, Krężołek M, Frydecka D, Nelson B, Cechnicki A. The interplay between childhood trauma, cognitive biases, psychotic-like experiences and depression and their additive impact on predicting lifetime suicidal behavior in young adults. Psychol Med. 2020;50(1):116–24.
Ghosh S, Ekbal A, Bhattacharyya P. Cease, a corpus of emotion annotated suicide notes in english. In N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, and S. Piperidis, editors, Proceedings of The 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France,European Language Resources Association, 2020. p. 1618–1626.
Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics 2011. p. 315–323.
GOTTSCHALK L, GLESER G. An analysis of the verbal content of suicide notes. Br J Med Psychol. 33:195204, 1960.
Ho T, Yip PS, Chiu C, Halliday P. Suicide notes: what do they tell us? Acta Psychiatrica Scandinavica. 1998;98(6):467–73.
Holub M. Review: Natural language processing for online applications: Text retrieval, extraction and categorization, by peter jackson and isabelle moulinier. Prague Bull. Math. Linguistics. 2005;83:83–4.
Hussain A, Cambria E. Semi-supervised learning for big social data analysis. Neurocomputing. 2018;275:1662–1673.
Ingram RE, Miranda J, Segal ZV. Cognitive vulnerability to depression. Guilford Press; 1998.
Ji S, Li X, Huang Z, Cambria E. Suicidal ideation and mental disorder detection with attentive relation networks. arXiv preprint 2020. arXiv:2004.07601
Ji S, Long G, Pan S, Zhu T, Jiang J, Wang S. Detecting suicidal ideation with data protection in online communities. In International Conference on Database Systems for Advanced Applications. Springer, 2019. p. 225–229.
Ji S, Pan S, Li X, Cambria E, Long G, Huang Z. Suicidal ideation detection: A review of machine learning methods and applications. arXiv preprint 2019. arXiv:1910.12611
Ji S, Yu CP, Fung S, Pan S, Long G. Supervised learning for suicidal ideation detection in online user content. Complex. 2018;6157249(1–6157249):10.
Joiner TE, Pettit JW, Walker RL, Voelz ZR, Cruz J, Rudd MD, Lester D. Perceived burdensomeness and suicidality: Two studies on the suicide notes of those attempting and those completing suicide. J Soc Clin Psychol. 2002;21(5):531–45.
Joo J, Hwang S, Gallo J. Death ideation and suicidal ideation in a community sample who do not meet criteria for major depression. Crisis. 2016;37(2):161–5.
Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint 2014. arXiv:1412.6980
Klinger R, et al. An analysis of annotated corpora for emotion classification in text. In Proceedings of the 27th International Conference on Computational Linguistics 2018. p. 2104–2119.
Kovačević A, Dehghan A, Keane JA, Nenadic G. Topic categorisation of statements in suicide notes with integrated rules and machine learning. Biomedical informatics insights, 5:BII–S8978, 2012.
Krippendorff K. Content Analysis: An Introduction to Its Methodology. SAGE Publications, 2013.
Lewis V, Williams K, Koko C, Woolmore J, Jones C, Powell T. Disability, depression and suicide ideation in people with multiple sclerosis. J Affect Disord. 2017;208:662–9.
Liu G, Wang C, Peng K, Huang H, Li Y, Cheng W. Socinf: Membership inference attacks on social media health data with machine learning. IEEE Transactions on Computational Social Systems. 2019;6(5):907–21.
Lopez-Castroman J, Moulahi B, Azé J, Bringay S, Deninotti J, Guillaume S, Baca-García E. Mining social networks to improve suicide prevention: A scoping review. J Neurosci Res. 2019;98:616–25.
Ma Y, Peng H, Cambria E. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In S. A. McIlraith and K. Q. Weinberger, editors, Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, AAAI Press, 2018. p. 5876–5883.
Majumder N, Poria S, Peng H, Chhaya N, Cambria E, Gelbukh A. Sentiment and sarcasm classification with multitask learning. IEEE Intell Syst. 2019;34(3):38–433.
Martinengo L, Van Galen L, Lum E, Kowalski M, Subramaniam M, Car J. Suicide prevention and depression apps suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines. BMC medicine. 2019;17(1):1–12.
O’Connor RC, Nock MK. The psychology of suicidal behaviour. The Lancet Psychiatry. 2014;1(1):73–85.
O’dea B, Wan S, Batterham PJ, Calear AL, Paris C, Christensen H. Detecting suicidality on twitter. Internet Interventions. 2015;2(2):183–188.
Osgood CE, Walker EG. Motivation and language behavior: a content analysis of suicide notes. J Abnorm Soc Psychol. 1959;59(1):58.
Park S, Kim J, Jeon J, Park H, Oh A. Toward dimensional emotion detection from categorical emotion annotations. arXiv preprint 2019. arXiv:1808.01216
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, et al. Scikit-learn: Machine learning in python. J Mach Learn Res, 201112(Oct):2825–2830, .
Pennington J, Socher R, Manning C. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) 2014. p. 1532–1543.
Pestian J, Nasrallah H, Matykiewicz P, Bennett A, Leenaars A. Suicide note classification using natural language processing: A content analysis. Biomedical informatics insights, 3:BII–S4706, 2010.
Pestian JP, Matykiewicz P, Grupp-Phelan J. Using natural language processing to classify suicide notes. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing. Association for Computational Linguistics, 2008. p. 96–97.
Pestian JP, Matykiewicz P, Linn-Gust M, South B, Uzuner O, Wiebe J, Cohen KB, Hurdle J, Brew. C Sentiment analysis of suicide notes: A shared task. Biomedical informatics insights 2012. 5:BII–S9042.
Pestian J, Matykiewicz P. Classification of suicide notes using natural language processing. Proceedings of ACL Bio NLP 2008, 967.
Plutchik R. The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. Am Sci. 2001;89(4):344–50.
Qureshi SA, Saha S, Hasanuzzaman M, Dias G. Multitask representation learning for multimodal estimation of depression level. IEEE Intell Syst. 2019;34(5):45–52.
Ren F, Kang X, Quan C. Examining accumulated emotional traits in suicide blogs with an emotion topic model. IEEE J Biomed Health Inform. 2015;20(5):1384–96.
Roberts K, Harabagiu SM. statistical and similarity methods for classifying emotion in suicide notes. Biomedical informatics insights 2012. 5:BII–S8958.
Russell JA. A circumplex model of affect. J Pers Soc Psychol. 1980;39(6):1161.
Russell JA, Mehrabian A. Evidence for a three-factor theory of emotions. Journal of research in Personality. 1977;11(3):273–94.
Sander L, Gerhardinger K, Bailey E, Robinson J, Lin J, Cuijpers P, Mühlmann C. Suicide risk management in research on internet-based interventions for depression: A synthesis of the current state and recommendations for future research. J Affect Disord. 2020;263:676–83.
Shapero JJ. The language of suicide notes. PhD thesis, University of Birmingham, 2011.
Shing HC, Nair S, Zirikly A, Friedenberg M, Daumé H III, Resnik P. Expert, crowdsourced, and machine assessment of suicide risk via online postings. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic 2018. p. 25–36.
Shneidman ES. Suicide notes reconsidered. Psychiatry. 1973;36(4):379–94.
Shneidman ES, Farberow NL. Clues to suicide. Public health reports. 1956;71(2):109.
Shneidman ES, Farberow NL. A socio-psychological investigation of suicide. In Perspectives in personality research. Springer, 1960. p. 270–293.
Simon GE, Coleman KJ, Rossom RC, Beck A, Oliver M, Johnson E, Whiteside U, Operskalski B, Penfold RB, Shortreed SM, et al. Risk of suicide attempt and suicide death following completion of the patient health questionnaire depression module in community practice. J Clin Psychol. 2016;77(2):221.
Spitzer RL, Cohen J, Fleiss JL, Endicott J. Quantification of agreement in psychiatric diagnosis: A new approach. Archives of General Psychiatry. 1967;17(1):83–7.
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(1):1929–58.
Stewart SL, Celebre A, Hirdes JP, Poss JW. Risk of suicide and self-harm in kids: The development of an algorithm to identify high-risk individuals within the childrens mental health system. Child Psychiatry Hum Dev 2020. p. 1–12.
Susanto Y, Livingstone AG, Ng BC, Cambria E. The hourglass model revisited. IEEE Intell Syst. 2020;35(5):96–102.
Tam DN. Computation in emotional processing: quantitative confirmation of proportionality hypothesis for angry unhappy emotional intensity to perceived loss. Cogn Comput. 2011;3(2):394–415.
Teismann T, Forkmann T, Brailovskaia J, Siegmann P, Glaesmer H, Margraf J. Positive mental health moderates the association between depression and suicide ideation: A longitudinal study. Int J Clin Health Psychol. 2018;18(1):1–7.
Tuckman J, Kleiner RJ, Lavell M. Emotional content of suicide notes. American Journal of Psychiatry. 1959;116(1):59–63.
Venek V, Scherer S, Morency L-P, Pestian J, et al. Adolescent suicidal risk assessment in clinician-patient interaction. IEEE Trans Affect Comput. 2017;8(2):204–15.
Vioulès MJ, Moulahi B, Azé J, Bringay S. Detection of suicide-related posts in twitter data streams. IBM J Res Dev. 2018;62(1):7–1.
Wagner F. Suicide notes. Danish Medical Journal. 1960;7:62–4.
Wang Z, Ho SB, Cambria E. A review of emotion sensing: Categorization models and algorithms. Multimedia Tools and Applications 2020. p. 1–30.
Wicentowski R, Sydes MR. Emotion detection in suicide notes using maximum entropy classification. Biomedical informatics insights 2012. 5:BII–S8972.
Xia Y, Cambria E, Hussain A, Zhao H. Word polarity disambiguation using bayesian model and opinion-level features. Cogn Comput. 2015;7(3):369–80.
Yang H, Willis A, De Roeck A, Nuseibeh B. A hybrid model for automatic emotion recognition in suicide notes. Biomedical informatics insights 2012. 5:BII–S8948.
Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E. Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies 2016. p. 1480–1489.
Yue L, Chen W, Li X, Zuo W, Yin M. A survey of sentiment analysis in social media. Knowl Inf Syst 2018. p. 1–47.
Zhou D, Zhang X, Zhou Y, Zhao Q, Geng X. Emotion distribution learning from texts. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing 2016. p. 638–647.
Acknowledgements
The authors gratefully acknowledge the support from the project titled ’Development of CDAC Digital Forensic Centre with AI based Knowledge Support Tools’, supported by Ministry of Electronics and Information Technology (MeitY), Government of India and Government of Bihar (project number P-264). The authors would also like to thank Suman Sekhar and Saroj Jha for their valuable efforts in labelling the sentences. Asif Ekbal acknowledges the Young Faculty Research Fellowship (YFRF), supported by Visvesvaraya PhD scheme for Electronics and IT, Ministry of Electronics and Information Technology (MeitY), Government of India, being implemented by Digital India Corporation (formerly Media Lab Asia.
Funding
This study was funded as part of the project titled ’Development of CDAC Digital Forensic Centre with AI based Knowledge Support Tools’ supported by Ministry of Electronics and Information Technology (MeitY), Government of India and Government of Bihar (project number P-264).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Ethical Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Rights and permissions
About this article
Cite this article
Ghosh, S., Ekbal, A. & Bhattacharyya, P. A Multitask Framework to Detect Depression, Sentiment and Multi-label Emotion from Suicide Notes. Cogn Comput 14, 110–129 (2022). https://doi.org/10.1007/s12559-021-09828-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-021-09828-7