More Web Proxy on the site http://driver.im/

research-article

A Novel deep neural network-based emotion analysis system for automatic detection of mild cognitive impairment in the elderly

Authors:

Wenju ZhouAuthors Info & Claims

Volume 468, Issue C

Pages 306 - 316

https://doi.org/10.1016/j.neucom.2021.10.038

Published: 11 January 2022 Publication History

Abstract

A significant number of people are suffering from cognitive impairment all over the world. Early detection of cognitive impairment is of great importance to both patients and caregivers. However, existing approaches have their shortages, such as time consumption and financial expenses involved in clinics and the neuroimaging stage. It has been found that patients with cognitive impairment show abnormal emotion patterns. In this paper, we present a novel deep neural network-based system to detect the cognitive impairment through the analysis of the evolution of facial emotions while participants are watching designed video stimuli. In our proposed system, a novel facial expression recognition algorithm is developed using layers from MobileNet and Support Vector Machine (SVM), which showed satisfactory performance in 3 datasets. To verify the proposed system in detecting cognitive impairment, 61 elderly people including patients with cognitive impairment and healthy people as a control group have been invited to participate in the experiments and a dataset was built accordingly. With this dataset, the proposed system has successfully achieved the detection accuracy of 73.3%.

References

[1]

R.C. Petersen, Mild cognitive impairment as a diagnostic entity, J. Intern. Med. 256 (3) (2004) 183–194,.

[2]

K. Wild, D. Howieson, F. Webbe, A. Seelye, J. Kaye, Status of computerized cognitive testing in aging: A systematic review, Alzheimer’s Dementia. 4 (6) (2008) 428–437,.

[3]

L. Mosconi, M. Brys, L. Glodzik-Sobanska, S. De Santi, H. Rusinek, M.J. de Leon, Early detection of Alzheimer’s disease using neuroimaging, Exp. Gerontol. 42 (1-2) (2007) 129–138,.

[4]

N. Zeng, H. Li, Y. Peng, A new deep belief network-based multi-task learning for diagnosis of Alzheimer’s disease, Neural Comput. Appl. (2021),.

Digital Library

[5]

N. Zeng, H. Qiu, Z. Wang, W. Liu, H. Zhang, Y. Li, A new switching-delayed-PSO-based optimized SVM algorithm for diagnosis of Alzheimer’s disease, Neurocomputing. 320 (2018) 195–202,.

[6]

K.-H. Chen, S.J. Lwi, A.Y. Hua, C.M. Haase, B.L. Miller, R.W. Levenson, Increased subjective experience of non-target emotions in patients with frontotemporal dementia and Alzheimer’s disease, Curr. Opin. Behav. Sci. 15 (2017) 77–84,.

[7]

M.C. Smith, Facial expression in mild dementia of the Alzheimer type, Behav. Neurol. 8 (1995) 149–156.

[8]

K.W. Burton, A.W. Kaszniak, Emotional experience and facial expression in Alzheimer’s disease, Aging Neuropsychol Cognit. 13 (3-4) (2006) 636–651,.

[9]

J.D. Henry, P.G. Rendell, A. Scicluna, M. Jackson, L.H. Phillips, Emotion experience, expression, and regulation in Alzheimer’s Disease, Psychol. Aging 24 (1) (2009) 252–257,.

[10]

N. Zeng, Z. Wang, H. Zhang, K.-E. Kim, Y. Li, X. Liu, An improved particle filter with a novel hybrid proposal distribution for quantitative analysis of gold immunochromatographic strips, IEEE Trans. Nanotechnol. 18 (2019) 819–829,.

Digital Library

[11]

F. Gao, T. Huang, J. Wang, J. Sun, A. Hussain, E. Yang, Dual-branch deep convolution neural network for polarimetric SAR image classification, Appl. Sci. (Switzerland). 7 (5) (2017) 447,.

[12]

F. Gao, F. Ma, J. Wang, J. Sun, E. Yang, H. Zhou, Visual saliency modeling for river detection in high-resolution SAR imagery, IEEE Access 6 (2018) 1000–1014,.

[13]

D.K. Jain, P. Shamsolmoali, P. Sehdev, Extended deep neural network for facial emotion recognition, Pattern Recogn. Lett. 120 (2019) 69–74,.

Digital Library

[14]

N. Zeng, H. Zhang, B. Song, W. Liu, Y. Li, A.M. Dobaie, Facial expression recognition via learning deep sparse autoencoders, Neurocomputing. 273 (2018) 643–649,.

[15]

J. Dong, H. Zheng, L. Lian, Dynamic Facial Expression Recognition Based on Convolutional Neural Networks with Dense Connections, in: Proceedings - International Conference on Pattern Recognition, Institute of Electrical and Electronics Engineers Inc., 2018: pp. 3433–3438. https://doi.org/10.1109/ICPR.2018.8545596.

[16]

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L.-C. Chen, MobileNetV2: Inverted Residuals and Linear Bottlenecks, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2018, pp. 4510–4520. 10.1109/CVPR.2018.00474.

[17]

Mobilenetv2 - MathWorks, (n.d.). https://ww2.mathworks.cn/help/deeplearning/ref/mobilenetv2.html (accessed July 25, 2021).

[18]

P. McAllister, H. Zheng, R. Bond, A. Moorhead, Combining deep residual neural network features with supervised machine learning algorithms to classify diverse food image datasets, Comput. Biol. Med. 95 (2018) 217–233,.

Digital Library

[19]

Feature Extraction Using AlexNet - MATLAB & Simulink - MathWorks United Kingdom, (n.d.). https://uk.mathworks.com/help/deeplearning/examples/feature-extraction-using-alexnet.html;jsessionid=a9dd0dd508fd2b96854cd6f11d5c (accessed January 23, 2019).

[20]

P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, IEEE Comput. Soc, n.d.: pp. I-511-I–518. https://doi.org/10.1109/CVPR.2001.990517.

[21]

Detect objects using the Viola-Jones algorithm - MATLAB - MathWorks United Kingdom, (n.d.). https://uk.mathworks.com/help/vision/ref/vision.cascadeobjectdetector-system-object.html (accessed January 29, 2019).

[22]

R. Collobert, J. Weston, A unified architecture for natural language processing, in: Proceedings of the 25th International Conference on Machine Learning - ICML ’08, ACM Press, New York, New York, USA, 2008: pp. 160–167. https://doi.org/10.1145/1390156.1390177.

[23]

A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, Adv. Neural Inform. Process. Syst. (2012) 1097–1105.

Digital Library

[24]

Z. Fei, E. Yang, D.-U. Li, S. Butler, W. Ijomah, X. Li, H. Zhou, Deep convolution network based emotion analysis towards mental health care, Neurocomputing. 388 (2020) 212–227,.

Digital Library

[25]

Pretrained AlexNet convolutional neural network - MATLAB alexnet - MathWorks United Kingdom, (n.d.). https://uk.mathworks.com/help/deeplearning/ref/alexnet.html;jsessionid=c4669357f290858d36ed1bcbf8cf (accessed September 21, 2018).

[26]

Q. Ye, N. Ye, T. Yin, Fast orthogonal linear discriminant analysis with application to image classification, Neurocomputing. 158 (2015) 216–224,.

Digital Library

[27]

Compute convolutional neural network layer activations - MATLAB activations - MathWorks United Kingdom, (n.d.). https://uk.mathworks.com/help/deeplearning/ref/activations.html (accessed January 30, 2019).

[28]

D. Lundqvist, A. Flykt, A. Ohman, The Karolinska directed emotional faces (KDEF), CD ROM from department of clinical neuroscience, Psychol. Sect. Karolinska Institutet. (1998) 91–630,.

[29]

A. Mollahosseini, B. Hasani, M.H. Mahoor, AffectNet: A database for facial expression, valence, and arousal computing in the wild, IEEE Trans. Affective Comput. 10 (1) (2019) 18–31,.

Digital Library

[30]

K. Rujirakul, C. So-In, Histogram equalized deep PCA with ELM classification for expressive face recognition, in: 2018International Workshop on Advanced Image Technology (IWAIT), IEEE, 2018, pp. 1–4. 10.1109/IWAIT.2018.8369725.

[31]

P. Lucey, J.F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, I. Matthews, The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression, in: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010, 2010. https://doi.org/10.1109/CVPRW.2010.5543262.

[32]

T. Jabid, Md.H. Kabir, O. Chae, Robust facial expression recognition based on local directional pattern, ETRI J. 32 (2010) 784–794,.

Cited By

Sadigov RYıldırım EKocaçınar BPatlar Akbulut FCatal C(2024)Deep learning-based user experience evaluation in distance learningCluster Computing10.1007/s10586-022-03918-327:1(443-455)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s10586-022-03918-3
Zhang BFei ZZhou WFei M(2023)A Systematic Review of Artificial Intelligence Techniques for Early Detection of Mild Cognitive Impairment in the ElderlyProceedings of the 4th International Conference on Artificial Intelligence and Computer Engineering10.1145/3652628.3652808(1096-1101)Online publication date: 17-Nov-2023
https://dl.acm.org/doi/10.1145/3652628.3652808
Feng WFei ZZhou WFei M(2023)Multiple Attention Network for Facial Expression RecognitionPRICAI 2023: Trends in Artificial Intelligence10.1007/978-981-99-7025-4_12(141-152)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1007/978-981-99-7025-4_12
Show More Cited By

Index Terms

A Novel deep neural network-based emotion analysis system for automatic detection of mild cognitive impairment in the elderly

Index terms have been assigned to the content through auto-classification.

Recommendations

Opportunistic sensing and detection of mild cognitive impairment
PETRA '14: Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments

Dementia is a growing healthcare problem, and it has become necessary to find a way to reduce the prevalence of this disease. Mild cognitive impairment is a risk factor for dementia, and being able to detect the onset of mild cognitive impairment gives ...
Automatic Diagnosis of Mild Cognitive Impairment Using Siamese Neural Networks
Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications
Abstract
The use of Artificial Intelligence techniques as an aid tool in the medical field is a current and undeniable challenge. In this context, similarity detection methods and Convolutional Neural Networks used in computer vision tasks for feature ...
On the analysis of speech and disfluencies for automatic detection of Mild Cognitive Impairment
Abstract
Alzheimer’s disease is characterized by a progressive and irreversible cognitive deterioration. In a previous stage, the so-called Mild Cognitive Impairment or cognitive loss appears. Nevertheless, this previous stage does not seem sufficiently ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 468, Issue C

Jan 2022

511 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 11 January 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sadigov RYıldırım EKocaçınar BPatlar Akbulut FCatal C(2024)Deep learning-based user experience evaluation in distance learningCluster Computing10.1007/s10586-022-03918-327:1(443-455)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s10586-022-03918-3
Zhang BFei ZZhou WFei M(2023)A Systematic Review of Artificial Intelligence Techniques for Early Detection of Mild Cognitive Impairment in the ElderlyProceedings of the 4th International Conference on Artificial Intelligence and Computer Engineering10.1145/3652628.3652808(1096-1101)Online publication date: 17-Nov-2023
https://dl.acm.org/doi/10.1145/3652628.3652808
Feng WFei ZZhou WFei M(2023)Multiple Attention Network for Facial Expression RecognitionPRICAI 2023: Trends in Artificial Intelligence10.1007/978-981-99-7025-4_12(141-152)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1007/978-981-99-7025-4_12
Gorbunova YKiselev G(2023)Development of a Mechanism for Recognizing the Emotional State Based on the Unconscious Movements of the SubjectInteractive Collaborative Robotics10.1007/978-3-031-43111-1_8(81-92)Online publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-43111-1_8
Dantas Ado Nascimento M(2022)Face emotions: improving emotional skills in individuals with autismMultimedia Tools and Applications10.1007/s11042-022-12810-681:18(25947-25969)Online publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1007/s11042-022-12810-6

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents