More Web Proxy on the site http://driver.im/

poster

Audiovisual behavior descriptors for depression assessment

Authors:

Stefan Scherer,

Louis-Philippe MorencyAuthors Info & Claims

ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interaction

Pages 135 - 140

https://doi.org/10.1145/2522848.2522886

Published: 09 December 2013 Publication History

Abstract

We investigate audiovisual indicators, in particular measures of reduced emotional expressivity and psycho-motor retardation, for depression within semi-structured virtual human interviews. Based on a standard self-assessment depression scale we investigate the statistical discriminative strength of the audiovisual features on a depression/no-depression basis. Within subject-independent unimodal and multimodal classification experiments we find that early feature-level fusion yields promising results and confirms the statistical findings. We further correlate the behavior descriptors with the assessed depression severity and find considerable correlation. Lastly, a joint multimodal factor analysis reveals two prominent factors within the data that show both statistical discriminative power as well as strong linear correlation with the depression severity score. These preliminary results based on a standard factor analysis are promising and motivate us to investigate this approach further in the future, while incorporating additional modalities.

References

[1]

P. Alku, T. Bäckström, and E. Vilkman. Glottal wave analysis with pitch synchronous iterative adaptive inverse filtering. Speech Communication, 11(2--3):109--118, 1992.

Digital Library

[2]

P. Alku, T. B\"ackström, and E. Vilkman. Normalized amplitude quotient for parameterization of the glottal flow. Journal of the Acoustical Society of America, 112(2):701--710, 2002.

[3]

T. Baltrusaitis, P. Robinson, and L. Morency. 3d constrained local model for rigid and non-rigid facial tracking. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2610--2617, 2012.

Digital Library

[4]

L. Batrinca, G. Stratou, L.-P. Morency, and S. Scherer. Cicero - towards a multimodal virtual audience platform for public speaking training. In Proceedings of Intelligent Virtual Agents (IVA) 2013, pages 116--128. Springer, 2013.

[5]

J. S. Buyukdura, S. M. McClintock, and P. E. Croarkin. Psychomotor retardation in depression: biological underpinnings, measurement, and treatment. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 35(2):395--409, 2011.

[6]

L. M. Bylsam, B. H. Morris, and J. Rottenberg. A meta-analysis of emotional reactivity in major depressive disorder. Clinical Psychology Review, 28:676--691, 2008.

[7]

J. F. Cohn, T. S. Kruez, I. Matthews, Y. Ying, M. H. Nguyen, M. T. Padilla, F. Zhou, and F. De la Torre. Detecting depression from facial actions and vocal prosody. In 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, pages 1--7, 2009.

[8]

N. Cummins, J. R. Epps, M. J. Breakspear, and R. Goecke. An investigation of depressed speech detection: Features and normalization. In Proceedings of Interspeech 2011. ISCA, 2011.

[9]

C. d'Alessandro and N. Sturmel. Glottal closure instant and voice source analysis using time-scale lines of maximum amplitude. Sadhana, 36(5):601--622, 2011.

[10]

J. K. Darby, N. Simmons, and P. A. Berger. Speech and voice parameters of depression: a pilot study. Journal of Communication Disorders, 17(2):75--85, 1984.

[11]

H. Ellgring. Nonverbal communication in depression. Cambridge University Press, Cambridge, 1989.

[12]

M. Elliott, M. A. Clements, J. W. Peifer, and L. Weisser. Critical analysis of the impact of glottal features in the classification of clinical depression in speech. IEEE Transactions on Biomedical Engineering, 55(1):96--107, 2008.

[13]

L. A. Fairbanks, M. T. McGuire, and C. J. Harris. Nonverbal interaction of patients and therapists during psychiatric interviews. Journal of Abnormal Psychology, 91(2):109--119, 1982.

[14]

A. J. Flint, S. E. Black, I. Campbell-Taylor, G. F. G. Gailey, and C. Levinton. Abnormal speech articulation, psychomotor retardation, and subcortical dysfunction in major depression. Journal of Psychiatric Research, 27(3):309--319, 1993.

[15]

T. Hacki. Klassifizierung von glottisdysfunktionen mit hilfe der elektroglottographie. Folia Phoniatrica, pages 43--48, 1989.

[16]

J. A. Hall, J. A. Harrigan, and R. Rosenthal. Nonverbal behavior in clinician-patient interaction. Applied and Preventive Psychology, 4(1):21--37, 1995.

[17]

L. V. Hedges. Distribution theory for glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2):107--128, 1981.

[18]

J. Kane, S. Scherer, M. Aylett, L.-P. Morency, and C. Gobl. Speaker and language independent voice quality classification applied to unlabelled corpora of expressive speech. In Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 7982--7986. IEEE, 2013.

[19]

J. Kane, S. Scherer, L.-P. Morency, and C. Gobl. A comparative study of glottal open quotient estimation techniques. In to appear in Proceedings of Interspeech 2013. ISCA, 2013.

[20]

K. Kroenke, R. L. Spitzer, and J. B. W. Williams. The phq-9. Journal of General Internal Medicine, 16(9):606--613, 2001.

[21]

C. Kublbeck and A. Ernst. Face detection and tracking in video sequences using the modifiedcensus transformation. Image and Vision Computing, 24(6):564 -- 572, 2006.

Digital Library

[22]

G. Littlewort, J. Whitehill, T. Wu, I. Fasel, M. Frank, J. Movellan, and M. Bartlett. The computer expression recognition toolbox (cert). In Automatic Face Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, pages 298--305, 2011.

[23]

L.-P. Morency, J. Whitehill, and J. Movellan. Generalized adaptive view-based appearance model: Integrated framework for monocular head pose estimation. In 8th IEEE International Conference on Automatic Face Gesture Recognition (FG08), pages 1--8, sept. 2008.

[24]

J. E. Perez and R. E. Riggio. Nonverbal social skills and psychopathology, pages 17--44. Nonverbal behavior in clinical settings. Oxford University Press, 2003.

[25]

J. T. M. Schelde. Major depression: Behavioral markers of depression and recovery. The Journal of Nervous and Mental Disease, 186(3):133--140, 1998.

[26]

S. Scherer, J. Kane, C. Gobl, and F. Schwenker. Investigating fuzzy-input fuzzy-output support vector machines for robust voice quality classification. Computer Speech and Language, 27(1):263--287, 2013.

Digital Library

[27]

S. Scherer, J. P. Pestian, and L.-P. Morency. Investigating the speech characteristics of suicidal adolescents. In Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 709--713. IEEE, 2013.

[28]

S. Scherer, G. Stratou, J. Gratch, and L.-P. Morency. Investigating voice quality as a speaker-independent indicator of depression and ptsd. In Proceedings of Interspeech 2013. ISCA, 2013.

[29]

S. Scherer, G. Stratou, M. Mahmoud, J. Boberg, J. Gratch, A. Rizzo, and L.-P. Morency. Automatic behavior descriptors for psychological disorder analysis. In Proceedings of IEEE Conference on Automatic Face and Gesture Recognition. IEEE, 2013.

[30]

M. Thiebaux, S. Marsella, A. N. Marshall, and M. Kallmann. Smartbody: behavior realization for embodied conversational agents. In Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1, AAMAS '08, pages 151--158. International Foundation for Autonomous Agents and Multiagent Systems, 2008.

Digital Library

Cited By

Yan KMiao SJin XMu YZheng HTian YWang PYu QHu D(2024)TCEDN: A Lightweight Time-Context Enhanced Depression Detection NetworkLife10.3390/life1410131314:10(1313)Online publication date: 16-Oct-2024
https://doi.org/10.3390/life14101313
Li MWang YYang CLu ZChen J(2024)Automatic Diagnosis of Depression Based on Facial Expression Information and Deep Convolutional Neural NetworkIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.339324711:5(5728-5739)Online publication date: Oct-2024
https://doi.org/10.1109/TCSS.2024.3393247
Hinduja SDarzi AErtugrul IProvenza NGadot RStorch ESheth SGoodman WCohn J(2024)Multimodal Prediction of Obsessive-Compulsive Disorder and Comorbid Depression Severity and Energy Delivered by Deep Brain ElectrodesIEEE Transactions on Affective Computing10.1109/TAFFC.2024.339511715:4(2025-2041)Online publication date: Oct-2024
https://doi.org/10.1109/TAFFC.2024.3395117
Show More Cited By

Index Terms

Audiovisual behavior descriptors for depression assessment

Recommendations

A Multimodal Context-based Approach for Distress Assessment
ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

The increasing prevalence of psychological distress disorders, such as depression and post-traumatic stress, necessitates a serious effort to create new tools and technologies to help with their diagnosis and treatment. In recent years, new ...
Dyadic Behavior Analysis in Depression Severity Assessment Interviews
ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

Previous literature suggests that depression impacts vocal timing of both participants and clinical interviewers but is mixed with respect to acoustic features. To investigate further, 57 middle-aged adults (men and women) with Major Depression Disorder ...
Facebook use, envy, and depression among college students

A survey of 736 college students found that Facebook use can trigger feelings of envy.Feelings of envy were found to predict depression symptoms.The effect of surveillance use of Facebook on depression is mediated by feelings of envy.Surveillance use of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interaction

December 2013

630 pages

ISBN:9781450321297

DOI:10.1145/2522848

General Chairs:
Julien Epps
The University of New South Wales, Australia
,
Fang Chen
National ICT Australia, Australia
,
Sharon Oviatt
Incaa Designs, USA
,
Kenji Mase
Nagoya University, Japan
,
Program Chairs:
Andrew Sears
Rochester Institute of Technology, USA
,
Kristiina Jokinen
University of Helsinki, Finland
,
Björn Schuller
Technische Universität München, Germany

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 December 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

ICMI '13

Sponsor:

SIGCHI

ICMI '13: 2013 International Conference on Multimodal Interaction

December 9 - 13, 2013

Sydney, Australia

Acceptance Rates

ICMI '13 Paper Acceptance Rate 49 of 133 submissions, 37%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

47
Total Citations
View Citations
481
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)4

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yan KMiao SJin XMu YZheng HTian YWang PYu QHu D(2024)TCEDN: A Lightweight Time-Context Enhanced Depression Detection NetworkLife10.3390/life1410131314:10(1313)Online publication date: 16-Oct-2024
https://doi.org/10.3390/life14101313
Li MWang YYang CLu ZChen J(2024)Automatic Diagnosis of Depression Based on Facial Expression Information and Deep Convolutional Neural NetworkIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.339324711:5(5728-5739)Online publication date: Oct-2024
https://doi.org/10.1109/TCSS.2024.3393247
Hinduja SDarzi AErtugrul IProvenza NGadot RStorch ESheth SGoodman WCohn J(2024)Multimodal Prediction of Obsessive-Compulsive Disorder and Comorbid Depression Severity and Energy Delivered by Deep Brain ElectrodesIEEE Transactions on Affective Computing10.1109/TAFFC.2024.339511715:4(2025-2041)Online publication date: Oct-2024
https://doi.org/10.1109/TAFFC.2024.3395117
Zhang SZhang XZhao XFang JNiu MZhao ZYu JTian Q(2024)MTDAN: A Lightweight Multi-Scale Temporal Difference Attention Networks for Automated Video Depression DetectionIEEE Transactions on Affective Computing10.1109/TAFFC.2023.331226315:3(1078-1089)Online publication date: Jul-2024
https://doi.org/10.1109/TAFFC.2023.3312263
Chen JHu YLai QWang WChen JLiu HSrivastava GBashir AHu X(2024)IIFDD: Intra and inter-modal fusion for depression detection with multi-modal information from Internet of Medical ThingsInformation Fusion10.1016/j.inffus.2023.102017102(102017)Online publication date: Feb-2024
https://doi.org/10.1016/j.inffus.2023.102017
Guan ZZhang XHuang WLi KChen DLi WSun JChen LMao YSun HTang XCao LLi Y(2024)A Method for Detecting Depression in Adolescence Based on an Affective Brain-Computer Interface and Resting-State Electroencephalogram SignalsNeuroscience Bulletin10.1007/s12264-024-01319-7Online publication date: 20-Nov-2024
https://doi.org/10.1007/s12264-024-01319-7
Takano TMizuguchi DOmiya YHiguchi MNakamura MShinohara SMitsuyoshi SSaito TYoshino AToda HTokuno S(2023)Estimating Depressive Symptom Class from VoiceInternational Journal of Environmental Research and Public Health10.3390/ijerph2005396520:5(3965)Online publication date: 23-Feb-2023
https://doi.org/10.3390/ijerph20053965
Li YLiu ZZhou LYuan XShangguan ZHu XHu B(2023)A facial depression recognition method based on hybrid multi-head cross attention networkFrontiers in Neuroscience10.3389/fnins.2023.118843417Online publication date: 24-May-2023
https://doi.org/10.3389/fnins.2023.1188434
Fisher HReiss PAtias DMalka MShahar BShamay-Tsoory SZilcha-Mano S(2023)Facing Emotions: Between- and Within-Sessions Changes in Facial Expression During Psychological Treatment for DepressionClinical Psychological Science10.1177/2167702623119579312:5(840-854)Online publication date: 25-Sep-2023
https://doi.org/10.1177/21677026231195793
Mathur LMataric MMorency L(2023)Expanding the Role of Affective Phenomena in Multimodal Interaction ResearchProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614171(253-260)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3614171
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents