More Web Proxy on the site http://driver.im/

short-paper

Multi-attention deep recurrent neural network for nursing action evaluation using wearable sensor

Authors:

Jun OtaAuthors Info & Claims

IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

Pages 546 - 550

https://doi.org/10.1145/3377325.3377530

Published: 17 March 2020 Publication History

Abstract

A nursing action evaluation system that can assess the performance of students practicing patient handling related nursing skills becomes an urgent need for solving the nursing educator shortage problem. Such an evaluation system should be designed with less hand-crafted procedures for its scalability. Additionally, realizing high accuracy of nursing action recognition, especially fine-grained action recognition remains a problem. This reflects in the recognition of the correct and incorrect methods when students perform a nursing action, and low accuracy of that would mislead the nursing students. We propose a multi-attention deep recurrent neural network (MA-DRNN) model for nursing action recognition by directly processing the raw acceleration and rotational speed signals from wearable sensors. Data samples of target nursing actions in a nursing skill called patient transfer were collected to train and compare the models. The experiment results show that the proposed model can reach approximately 96% recognition accuracy for four target fine-grained nursing action classes helped by the attention mechanism on time and layer domains, which outperforms the state-of-the-art models of wearable sensor-based HAR.

References

[1]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).

[2]

Yoshua Bengio, Patrice Simard, Paolo Frasconi, et al. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks 5, 2 (1994), 157--166.

[3]

Ryan Burchfield and S Venkatesan. 2010. A framework for golf training using low-cost inertial sensors. In 2010 International Conference on Body Sensor Networks. IEEE, 267--272.

Digital Library

[4]

Nandini Chakravorti, Tanya Le Sage, Siân E Slawson, Paul P Conway, and Andrew A West. 2013. Design and implementation of an integrated performance monitoring tool for swimming to extract stroke information at real time. IEEE transactions on human-machine systems 43, 2 (2013), 199--213.

[5]

Jacky CP Chan, Howard Leung, Jeff KT Tang, and Taku Komura. 2010. A virtual reality dance training system using motion capture technology. IEEE Transactions on Learning Technologies 4, 2 (2010), 187--195.

Digital Library

[6]

Guilhem Chéron, Ivan Laptev, and Cordelia Schmid. 2015. P-cnn: Pose-based cnn features for action recognition. In Proceedings of the IEEE international conference on computer vision. 3218--3226.

Digital Library

[7]

Li Deng. 2014. A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Transactions on Signal and Information Processing 3 (2014).

[8]

Shinichi Furuya and Hiroshi Kinoshita. 2008. Organization of the upper limb movement for piano key-depression differs between expert pianists and novice players. Experimental Brain Research 185, 4 (2008), 581--593.

[9]

A Garg, BD Owen, and B Carlson. 1992. An ergonomic evaluation of nursing assistants' job in a nursing home. Ergonomics 35, 9 (1992), 979--995.

[10]

Hristijan Gjoreski, Jani Bizjak, Martin Gjoreski, and Matjaž Gams. 2016. Comparing deep and classical machine learning methods for human activity recognition using wrist accelerometer. In Proceedings of the IJCAI 2016 Workshop on Deep Learning for Artificial Intelligence, New York, NY, USA, Vol. 10.

[11]

Patricia A Grady. 2011. Advancing the health of our aging population: A lead role for nursing science. Nursing outlook 59, 4 (2011), 207.

[12]

Alex Graves, Marcus Liwicki, Santiago Fernández, Roman Bertolami, Horst Bunke, and Jürgen Schmidhuber. 2008. A novel connectionist system for unconstrained handwriting recognition. IEEE transactions on pattern analysis and machine intelligence 31, 5 (2008), 855--868.

[13]

Lisa M Haddad and Tammy J Toney-Butler. 2019. Nursing shortage. In StatPearls [Internet]. StatPearls Publishing.

[14]

Zhifeng Huang, Ayanori Nagata, Masako Kanai-Pak, Jukai Maeda, Yasuko Kitajima, Mitsuhiro Nakamura, Kyoko Aida, Noriaki Kuwahara, Taiki Ogata, and Jun Ota. 2014. Automatic evaluation of trainee nurses' patient transfer skills using multiple kinect sensors. IEICE TRANSACTIONS on Information and Systems 97, 1 (2014), 107--118.

[15]

Wenchao Jiang and Zhaozheng Yin. 2015. Human activity recognition using wearable sensors by deep convolutional neural networks. In Proceedings of the 23rd ACM international conference on Multimedia. Acm, 1307--1310.

Digital Library

[16]

Hirokatsu Kataoka, Yoshimitsu Aoki, Yutaka Satoh, Shoko Oikawa, and Yasuhiro Matsui. 2015. Fine-grained walking activity recognition via driving recorder dataset. In 2015 IEEE 18th International Conference on Intelligent Transportation Systems. IEEE, 620--625.

Digital Library

[17]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[18]

Chingszu Lin, Masako Kanai-Pak, Jukai Maeda, Yasuko Kitajima, Mitsuhiro Nakamura, Noriaki Kuwahara, Taiki Ogata, and Jun Ota. 2018. Translational Acceleration, Rotational Speed, and Joint Angle of Patients Related to Correct/Incorrect Methods of Transfer Skills by Nurses. Sensors 18, 9 (2018), 2975.

[19]

Zhuohua Lin, Munenori Uemura, Massimiliano Zecca, Salvatore Sessa, Hiroyuki Ishii, Luca Bartolomeo, Kazuko Itoh, Morimasa Tomikawa, Makoto Hashizume, and Atsuo Takanishi. 2011. Waseda Bioinstrumentation system WB-3 as a wearable tool for objective laparoscopic skill evaluation. In 2011 IEEE International Conference on Robotics and Automation. IEEE, 5737--5742.

[20]

M Marć, A Bartosiewicz, J Burzyńska, Z Chmiel, and P Januszewicz. 2019. A nursing shortage-a prospect of global and local policies. International nursing review 66, 1 (2019), 9--16.

[21]

Abdulmajid Murad and Jae-Young Pyun. 2017. Deep recurrent neural networks for human activity recognition. Sensors 17, 11 (2017), 2556.

[22]

Deena A Nardi and Charlene C Gyurko. 2013. The global nursing faculty shortage: Status and solutions for change. Journal of Nursing Scholarship 45, 3 (2013), 317--326.

[23]

Francisco Ordóñez and Daniel Roggen. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.

[24]

Lutz Prechelt. 1998. Early stopping-but when? In Neural Networks: Tricks of the trade. Springer, 55--69.

Digital Library

[25]

Soumitry J Ray and Jochen Teizer. 2012. Real-time construction worker posture analysis for ergonomics training. Advanced Engineering Informatics 26, 2 (2012), 439--455.

Digital Library

[26]

Marcus Rohrbach, Sikandar Amin, Mykhaylo Andriluka, and Bernt Schiele. 2012. A database for fine grained activity detection of cooking activities. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1194--1201.

[27]

Bharat Singh, Tim K Marks, Michael Jones, Oncel Tuzel, and Ming Shao. 2016. A multi-stream bi-directional recurrent neural network for fine-grained action detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1961--1970.

[28]

Chen Sun, Sanketh Shetty, Rahul Sukthankar, and Ram Nevatia. 2015. Temporal localization of fine-grained actions in videos by domain transfer from web images. In Proceedings of the 23rd ACM international conference on Multimedia. ACM, 371--380.

Digital Library

[29]

Dapeng Tao, Yonggang Wen, and Richang Hong. 2016. Multicolumn bidirectional long short-term memory for mobile devices-based human activity recognition. IEEE Internet of Things Journal 3, 6 (2016), 1124--1134.

[30]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.

[31]

Martin Wollmer, Florian Eyben, Joseph Keshet, Alex Graves, Bjorn Schuller, and Gerhard Rigoll. 2009. Robust discriminative keyword spotting for emotionally colored spontaneous speech using bidirectional LSTM networks. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 3949--3952.

Digital Library

[32]

Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, and Shonali Krishnaswamy. 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In Twenty-Fourth International Joint Conference on Artificial Intelligence.

Digital Library

[33]

Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies. 1480--1489.

[34]

Yu Zhao, Rennong Yang, Guillaume Chevalier, Ximeng Xu, and Zhenxing Zhang. 2018. Deep residual bidir-LSTM for human activity recognition using wearable sensors. Mathematical Problems in Engineering 2018 (2018).

[35]

Yang Zhou, Bingbing Ni, Richang Hong, Meng Wang, and Qi Tian. 2015. Interaction part mining: A mid-level approach for fine-grained action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3323--3331.

Cited By

Jnah MJnah A(2024)The Infiltration of Artificial Intelligence Into Higher EducationNeonatal Network10.1891/NN-2024-000643:3(133-138)Online publication date: 30-May-2024
https://doi.org/10.1891/NN-2024-0006
Gao CChen YJiang XHu LZhao ZZhang Y(2023)Bi-STAN: bilinear spatial-temporal attention network for wearable human activity recognitionInternational Journal of Machine Learning and Cybernetics10.1007/s13042-023-01781-114:7(2545-2561)Online publication date: 2-Feb-2023
https://doi.org/10.1007/s13042-023-01781-1
Baskaran PHeard JAdams J(2022)Emergency Clinical Procedure Detection via Wearable SensorsProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132266153966:1(2239-2243)Online publication date: 27-Oct-2022
https://doi.org/10.1177/1071181322661539
Show More Cited By

Index Terms

Multi-attention deep recurrent neural network for nursing action evaluation using wearable sensor
1. Computing methodologies
  1. Machine learning
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Recurrent neural network for facial landmark detection

Facial landmark detection is an important issue in many computer vision applications about faces. It is very challenging as human faces in wild conditions often present large variations in shape due to different poses, occlusions or expressions. Deep ...
Nursing Care Quality in Nursing Homes: Cross-Sectional Versus Longitudinal Analysis

This empirical study of the relationship between nursing care adequacy and nursing care quality demonstrates that a positive relationship exists between the process and outcome dimensions of quality of nursing care. The results from the analysis of ...
The impact of extraneous features on the performance of recurrent neural network models in clinical tasks
Graphical abstract

Display Omitted
Highlights
- Use RNNs and EMR to predict in-ICU mortality, ICU re-admission, and ICU-free days.
Abstract
Electronic Medical Records (EMR) are a rich source of patient information, including measurements reflecting physiologic signs and administered therapies. Identifying which variables or features are useful in predicting clinical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

March 2020

607 pages

ISBN:9781450371186

DOI:10.1145/3377325

General Chairs:
Fabio Paternò,
Nuria Oliver,
Program Chairs:
Cristina Conati,
Lucio Davide Spano,
Nava Tintarev

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

IUI '20

Sponsor:

IUI '20: 25th International Conference on Intelligent User Interfaces

March 17 - 20, 2020

Cagliari, Italy

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
257
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)2

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jnah MJnah A(2024)The Infiltration of Artificial Intelligence Into Higher EducationNeonatal Network10.1891/NN-2024-000643:3(133-138)Online publication date: 30-May-2024
https://doi.org/10.1891/NN-2024-0006
Gao CChen YJiang XHu LZhao ZZhang Y(2023)Bi-STAN: bilinear spatial-temporal attention network for wearable human activity recognitionInternational Journal of Machine Learning and Cybernetics10.1007/s13042-023-01781-114:7(2545-2561)Online publication date: 2-Feb-2023
https://doi.org/10.1007/s13042-023-01781-1
Baskaran PHeard JAdams J(2022)Emergency Clinical Procedure Detection via Wearable SensorsProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132266153966:1(2239-2243)Online publication date: 27-Oct-2022
https://doi.org/10.1177/1071181322661539
Zhong ZLin CKanai-Pak MMaeda JKitajima YNakamura MKuwahara NOgata TOta J(2021)Multistream Temporal Convolutional Network for Correct/Incorrect Patient Transfer Action Detection Using Body Sensor NetworkIEEE Internet of Things Journal10.1109/JIOT.2021.30754778:23(17000-17013)Online publication date: 1-Dec-2021
https://doi.org/10.1109/JIOT.2021.3075477

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents