More Web Proxy on the site http://driver.im/

short-paper

A Movement in Multiple Time Neural Network for Automatic Detection of Pain Behaviour

Authors:

Temitayo Olugbade,

Amanda C de C Williams,

Nadia Bianchi-BerthouzeAuthors Info & Claims

ICMI '20 Companion: Companion Publication of the 2020 International Conference on Multimodal Interaction

Pages 442 - 445

https://doi.org/10.1145/3395035.3425969

Published: 27 December 2020 Publication History

Abstract

The use of multiple clocks has been a favoured approach to modelling the multiple timescales of sequential data. Previous work based on clocks and multi-timescale studies in general have not clearly accounted for multidimensionality of data such that each dimension has its own timescale(s). Focusing on body movement data which has independent yet coordinating degrees of freedom, we propose a Movement in Multiple Time (MiMT) neural network. Our MiMT models multiple timescales by learning different levels of movement interpretation (i.e. labels) and further allows for separate timescales across movements dimensions. We obtain 0.75 and 0.58 average F1 scores respectively for binary frame-level and three-class window-level classification of pain behaviour based on the MiMT. Findings in ablation studies suggest that these two elements of the MiMT are valuable to modelling multiple timescales of multidimensional sequential data.

References

[1]

Min Aung, Sebastian Kaltwang, Bernardino Romera-Paredes, Brais Martinez, Aneesha Singh, Matteo Cella, et al. 2016. The automatic detection of chronic pain-related expression: requirements, challenges and the multimodal EmoPain dataset. IEEE Transactions on Affective Computing, Vol. 7, 4 (2016), 435--451.

Digital Library

[2]

Junyoung Chung, Sungjin Ahn, and Yoshua Bengio. 2019. Hierarchical multiscale recurrent neural networks. In Proceedings of ICML. 1--13.

[3]

Lá szló A. Jeni, Jeffrey F. Cohn, and Fernando De La Torre. 2013. Facing imbalanced data - Recommendations for the use of performance metrics. In Proceedings of Conference on Affective Computing and Intelligent Interaction. IEEE, 245--251.

Digital Library

[4]

Francis Keefe and Andrew Block. 1982. Development of an observation method for assessing pain behavior in chronic low back pain patients. Behavior Therapy, Vol. 13, 4 (1982), 363--375.

[5]

KM Kerr, JA White, DA Barr, and RAB Mollan. 1994. Standardization and definitions of the sit-stand-sit movement cycle. Gait and Posture, Vol. 2, 3 (1994), 182--190.

[6]

Jan Koutní k, Klaus Greff, Faustino Gomez, and Jü rgen Schmidhuber. 2014. A clockwork RNN. In International Conference on Machine Learning. 1863--1871.

Digital Library

[7]

Qianli Ma, Enhuan Chen, Zhenxi Lin, Jiangyue Yan, Zhiwen Yu, and Wing Ng. 2019. Convolutional Multitimescale Echo State Network. IEEE Trans Cybern (2019), 1--13.

[8]

Katherine Metcalf and David Leake. 2019. Unsupervised hierarchical temporal abstraction by simultaneously learning expectations and representations. In Proceedings of International Joint Conference on Artificial Intelligence. 3144--3150.

Digital Library

[9]

Temitayo Olugbade, Joseph Newbold, Rose Johnson, Erica Volta, Paolo Alborno, Radoslaw Niewiadomski, et al. 2020. Automatic Detection of Reflective Thinking in Mathematical Problem Solving based on Unconstrained Bodily Exploration. IEEE Transactions on Affective Computing (2020).

[10]

Temitayo A Olugbade, Aneesha Singh, Nadia Bianchi-Berthouze, Nicolai Marquardt, Min SH Aung, and Amanda C De C Williams. 2019. How can affect be detected and represented in technological support for physical rehabilitation? ACM Transactions on Computer-Human Interaction, Vol. 26, 1 (2019), 1--29.

Digital Library

[11]

Martin Peniak, Davide Marocco, Jun Tani, Yuichi Yamashita, Kerstin Ficher, and Angelo Cangelosi. 2011. Multiple Time Scales Recurrent Neural Network for Complex Action Acquisition. Frontiers in Computational Neuroscience, Vol. 5 (2011).

[12]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uskoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need Ashish. In Proceedings of NIPS. 5998--6008.

Digital Library

[13]

Chongyang Wang, Temitayo Olugbade, Akhil Mathur, Amanda C De C. Williams, Nicholas Lane, and Nadia Bianchi-Berthouze. 2019 a. Recurrent network based automatic detection of chronic pain protective behavior using mocap and semg data. In Proceedings of International Symposium on Wearable Computers. 225--230.

Digital Library

[14]

Chongyang Wang, Min Peng, Temitayo Olugbade, Nicholas Lane, Amanda C de C Williams, and Nadia Bianchi-Berthouze. 2019 b. Learning temporal and bodily attention in protective movement behavior detection. In Proceedings of Affective Computing and Intelligent Interaction Workshops and Demos. 324--330.

[15]

Yuichi Yamashita and Jun Tani. 2008. Emergence of functional hierarchy in a multiple timescale neural network model: A humanoid robot experiment. PLoS Computational Biology, Vol. 4 (2008).

[16]

Jeffrey Zacks and Khena Swallow. 2007. Event segmentation. Current Directions in Psychological Science, Vol. 16, 2 (2007), 80--84.

Cited By

Olugbade TWilliams AGold NBianchi-Berthouze N(2024)Movement Representation Learning for Pain Level ClassificationIEEE Transactions on Affective Computing10.1109/TAFFC.2023.333452215:3(1303-1314)Online publication date: Jul-2024
https://doi.org/10.1109/TAFFC.2023.3334522
Dehshibi MOlugbade TDiaz-de-Maria FBianchi-Berthouze NTajadura-Jiménez A(2023)Pain Level and Pain-Related Behaviour Classification Using GRU-Based Sparsely-Connected RNNsIEEE Journal of Selected Topics in Signal Processing10.1109/JSTSP.2023.326235817:3(677-688)Online publication date: May-2023
https://doi.org/10.1109/JSTSP.2023.3262358

Index Terms

A Movement in Multiple Time Neural Network for Automatic Detection of Pain Behaviour
1. Theory of computation

Index terms have been assigned to the content through auto-classification.

Recommendations

Multiple timescale mixed bursting dynamics in a respiratory neuron model

Experimental results in rodent medullary slices containing the pre-Bötzinger complex (pre-BötC) have identified multiple bursting mechanisms based on persistent sodium current (INaP) and intracellular Ca2+. The classic two-timescale approach to the ...
Circadian Rhythm and Pain: Mathematical Model based on Multiagent Simulation
Abstract
The circadian rhythm is responsible for the daily variations in metabolism, and circadian rhythm disorders have direct implications for many diseases, such as obesity and mental disorders. The regulation of sleep time is the most common example of ...
Neural network models for the gaze shift system in the superior colliculus and cerebellum

We investigate the role that the superior colliculus (SC) and the cerebellum might play in generating gaze shifts. The discharge of cells in the intermediate layers of the SC is tightly linked to the occurrence of saccades. Many studies have ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '20 Companion: Companion Publication of the 2020 International Conference on Multimodal Interaction

October 2020

548 pages

ISBN:9781450380027

DOI:10.1145/3395035

General Chairs:
Khiet Truong
University of Twente, the Netherlands
,
Dirk Heylen
University of Twente, the Netherlands
,
Mary Czerwinski
Microsoft Research, USA
,
Program Chairs:
Nadia Berthouze
University College London, United Kingdom
,
Mohamed Chetouani
Sorbonne University, France
,
Mikio Nakano
C4A Research Institute, Japan

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 December 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Horizon 2020

Conference

ICMI '20

Sponsor:

SIGCHI

ICMI '20: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

October 25 - 29, 2020

Virtual Event, Netherlands

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
85
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Olugbade TWilliams AGold NBianchi-Berthouze N(2024)Movement Representation Learning for Pain Level ClassificationIEEE Transactions on Affective Computing10.1109/TAFFC.2023.333452215:3(1303-1314)Online publication date: Jul-2024
https://doi.org/10.1109/TAFFC.2023.3334522
Dehshibi MOlugbade TDiaz-de-Maria FBianchi-Berthouze NTajadura-Jiménez A(2023)Pain Level and Pain-Related Behaviour Classification Using GRU-Based Sparsely-Connected RNNsIEEE Journal of Selected Topics in Signal Processing10.1109/JSTSP.2023.326235817:3(677-688)Online publication date: May-2023
https://doi.org/10.1109/JSTSP.2023.3262358

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents