More Web Proxy on the site http://driver.im/

research-article

A sequence-to-sequence model-based deep learning approach for recognizing activity of daily living for senior care

Authors:

Randall BrownAuthors Info & Claims

Volume 84, Issue C

Pages 148 - 158

https://doi.org/10.1016/j.jbi.2018.07.006

Published: 01 August 2018 Publication History

Graphical abstract

Display Omitted

Highlights

•

We proposed an activity state representation for arbitrary sensor combinations.

•

We developed a Seq2Seq model-based activity recognition framework.

•

The framework provides an end-to-end recognition from raw data to activities.

•

Our method out-performed benchmark methods on two publicly available datasets.

•

The model shows potential for real-world smart home monitoring.

Abstract

Ensuring the health and safety of independent-living senior citizens is a growing societal concern. Researchers have developed sensor based systems to monitor senior citizens' Activity of Daily Living (ADL), a set of daily activities that can indicate their self-caring ability. However, most ADL monitoring systems are designed for one specific sensor modality, resulting in less generalizable models that is not flexible to account variations in real-life monitoring settings. Current classic machine learning and deep learning methods do not provide a generalizable solution to recognize complex ADLs for different sensor settings. This study proposes a novel Sequence-to-Sequence model based deep-learning framework to recognize complex ADLs leveraging an activity state representation. The proposed activity state representation integrated motion and environment sensor data without labor-intense feature engineering. We evaluated our proposed framework against several state-of-the-art machine learning and deep learning benchmarks. Overall, our approach outperformed baselines in most performance metrics, accurately recognized complex ADLs from different types of sensor input. This framework can generalize to different sensor settings and provide a viable approach to understand senior citizen's daily activity patterns with smart home health monitoring systems.

References

[1]

Census Bureau, The Nation’s Older Population is Still Growing, Census Bur. Reports, 2017. <https://www.census.gov/newsroom/press-releases/2017/cb17-100.htm> (accessed October 27, 2017).

[2]

Eurostat, Population structure and ageing, 2017. <http://ec.europa.eu/eurostat/statistics-explained/index.php/Population_structure_and_ageing> (accessed October 27, 2017).

[3]

Administration on Aging, A profile of older American: 2013, 2013. <http://www.aoa.acl.gov/Aging_Statistics/Profile/2013/docs/2013_Profile.pdf> (accessed November 20, 2015).

[4]

C.M. Williams, Healthy aging and assessing older adults, in: J.E. South-Paul, S.C. Matheny, E.L. Lewis (Eds.), Curr. Diagnosis Treat. Fam. Med., third ed., The McGraw-Hill Companies, New York, NY, 2011.

[5]

S.E. Hardy, Consideration of function & functional decline, in: B.A. Williams, A. Chang, C. Ahalt, H. Chen, R. Conant, C.S. Landefeld, C. Ritchie, M. Yukawa (Eds.), Curr. Diagnosis Treat. Geriatr., second ed., McGraw-Hill, New York, NY, 2014, pp. 3–4.

[6]

S. Katz, Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living, J. Am. Geriatr. Soc. 31 (1983) 721–727,.

[7]

S.S. Roley, J.V. DeLany, C.J. Barrows, S. Brownrigg, D. Honaker, D.I. Sava, V. Talley, K. Voelkerding, D.A. Amini, E. Smith, P. Toto, S. King, D. Lieberman, Occupational therapy practice framework: domain & process, Am. J. Occup. Ther. 62 (2008) 625.

[8]

D. Foti, J.S. Koketsu, Activities of daily living, in: Pedretti’s Occup. Ther. Pract. Ski. Phys. Dysfunct., 2013, pp. 157–232.

[9]

M.M.E. de Werd, D. Boelen, M.G.M. Olde Rikkert, R.P.C. Kessels, Development and evaluation of a clinical manual on errorless learning in people with dementia, Brain Impair. 16 (2015) 52–63,.

[10]

K.L. Votruba, C. Persad, B. Giordani, Patient mood and instrumental activities of daily living in Alzheimer disease, J. Geriatr. Psychiatry Neurol. 28 (2015) 203–209,.

[11]

C.M. Giebel, C. Sutcliffe, D. Challis, Activities of daily living and quality of life across different stages of dementia: a UK study, Aging Ment. Health. 19 (2015) 63–71,.

[12]

Federal Interagency Forum on Aging-Related Statistics, Older Americans 2016: key indicators of well-being, Federal Interagency Forum on Aging Related Statistics, 2016. <https://agingstats.gov/docs/LatestReport/Older-Americans-2016-Key-Indicators-of-WellBeing.pdf>.

[13]

K. Jekel, M. Damian, C. Wattmo, L. Hausner, R. Bullock, P.J. Connelly, B. Dubois, M. Eriksdotter, M. Ewers, E. Graessel, M.G. Kramberger, E. Law, P. Mecocci, J.L. Molinuevo, L. Nygård, M.G. Olde-Rikkert, J.-M. Orgogozo, F. Pasquier, K. Peres, E. Salmon, S.A. Sikkes, T. Sobow, R. Spiegel, M. Tsolaki, B. Winblad, L. Frölich, Mild cognitive impairment and deficits in instrumental activities of daily living: a systematic review, Alzheimers. Res. Ther. 7 (2015) 17,.

[14]

T.G. Fong, L.J. Gleason, B. Wong, D. Habtemariam, R.N. Jones, E.M. Schmitt, S.E. de Rooij, J.S. Saczynski, A.L. Gross, J.F. Bean, C.J. Brown, D.M. Fick, A.L. Gruber-Baldini, M. O’Connor, P.A. Tabloski, E.R. Marcantonio, S.K. Inouye, Cognitive and physical demands of activities of daily living in older adults: validation of expert panel ratings, PM&R 7 (2015) 727–735,.

[15]

I. Singh, A. Varanasi, K. Williamson, Assessment and management of dementia in the general hospital setting, Rev. Clin. Gerontol. 24 (2014) 205–218,.

[16]

J. Chung, M. Ozkaynak, G. Demiris, Examining daily activity routines of older adults using workflow, J. Biomed. Inform. 71 (2017) 82–90,.

[17]

J. Bravo, D. Cook, G. Riva, Ambient intelligence for health environments, J. Biomed. Inform. 64 (2016) 207–210,.

[18]

B.M.C. Silva, J.J.P.C. Rodrigues, I. de la Torre Díez, M. López-Coronado, K. Saleem, Mobile-health: a review of current state in 2015, J. Biomed. Inform. 56 (2015) 265–272,.

Digital Library

[19]

D. Roggen, A. Calatroni, M. Rossi, T. Holleczek, K. Forster, G. Troster, P. Lukowicz, D. Bannach, G. Pirkl, A. Ferscha, J. Doppler, C. Holzmann, M. Kurz, G. Holl, R. Chavarriaga, H. Sagha, H. Bayati, M. Creatura, J.del.R. Millan, Collecting complex activity datasets in highly rich networked sensor environments, in: Proc. 7th Int. Conf. Networked Sens. Syst., IEEE, 2010, pp. 233–240,.

[20]

I.A. Emi, J.A. Stankovic, SARRIMA: smart ADL recognizer and resident identifier in multi-resident accommodations, in: Proc. Conf. Wirel. Heal. – WH ’15, ACM Press, New York, New York, USA, 2015, pp. 1–8,.

Digital Library

[21]

N.C. Krishnan, D.J. Cook, Activity recognition on streaming sensor data, Pervasive Mob. Comput. 10 (2014) 138–154,.

Digital Library

[22]

K. Safi, S. Mohammed, F. Attal, M. Khalil, Y. Amirat, Recognition of different daily living activities using Hidden Markov Model regression, in: 3rd Middle East Conf. Biomed. Eng., IEEE, 2016, pp. 16–19,.

[23]

J. Gong, J. Lach, J.A. Stankovic, K.M. Rose, I.A. Emi, J.P. Specht, E. Hoque, D. Fan, S.R. Dandu, R.F. Dickerson, Y. Perkhounkova, Home wireless sensing system for monitoring nighttime agitation and incontinence in patients with Alzheimer’s disease, in: Proc. Conf. Wirel. Heal. - WH ’15, ACM Press, New York, New York, USA, 2015, pp. 1–8,.

Digital Library

[24]

C. Flores-Vazquez, J. Aranda, Human activity recognition from object interaction in domestic scenarios, in: 2016 IEEE Ecuador Tech. Chapters Meet., IEEE, 2016, pp. 1–6,.

[25]

C. Debes, A. Merentitis, S. Sukhanov, M. Niessen, N. Frangiadakis, A. Bauer, Monitoring activities of daily living in smart homes: understanding human behavior, IEEE Signal Process. Mag. 33 (2016) 81–94,.

[26]

F. Ordóñez, D. Roggen, Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition, Sensors 16 (2016) 115,.

[27]

H. Cao, M.N. Nguyen, C. Phua, S. Krishnaswamy, X. Li, An integrated framework for human activity classification, in: Proc. 2012 ACM Conf. Ubiquitous Comput., ACM Press, New York, New York, USA, 2012, pp. 331–340,.

Digital Library

[28]

L. Wang, T. Gu, X. Tao, J. Lu, Sensor-based human activity recognition in a multi-user scenario, in: Eur. Conf. Ambient Intell., Springer, Berlin, Heidelberg, 2009, pp. 78–87,.

Digital Library

[29]

J.-L. Reyes-Ortiz, L. Oneto, A. Samà, X. Parra, D. Anguita, Transition-aware human activity recognition using smartphones, Neurocomputing 171 (2016) 754–767,.

Digital Library

[30]

Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (2015) 436–444,.

[31]

N.Y. Hammerla, S. Halloran, T. Ploetz, Deep, convolutional, and recurrent models for human activity recognition using wearables, in: Proc. Twenty-Fifth Int. Jt. Conf. Artif. Intell., 2016, pp. 1533–1540. .

[32]

Y. Chen, Y. Xue, A deep learning approach to human activity recognition based on single accelerometer, in: IEEE Int. Conf. Syst. Man, Cybern., IEEE, 2015, pp. 1488–1492,.

Digital Library

[33]

M.A. Alsheikh, A. Selim, D. Niyato, L. Doyle, S. Lin, H.-P. Tan, Deep activity recognition models with triaxial accelerometers, in: AAAI Work. Artif. Intell. Appl. to Assist. Technol. Smart Environ., 2016.

[34]

J.B. Yang, M.N. Nguyen, P.P. San, X.L. Li, K. Shonali, Deep convolutional neural networks on multichannel time series for human activity recognition, in: Proc. 24th Int. Conf. Artif. Intell., 2015, pp. 3995–4001.

[35]

M. Zeng, L.T. Nguyen, B. Yu, O.J. Mengshoel, J. Zhu, P. Wu, J. Zhang, Convolutional neural networks for human activity recognition using mobile sensors, in: Proc. 6th Int. Conf. Mob. Comput. Appl. Serv., ICST, 2014, pp. 197–205. http://doi./10.4108/icst.mobicase.2014.257786.

[36]

I. Goodfellow, Y. Bengio, A. Courville, Y. Bengio, Deep Learning, MIT Press, Cambridge, 2016.

Digital Library

[37]

Z.C. Lipton, J. Berkowitz, C. Elkan, A critical review of recurrent neural networks for sequence learning, 2015. .

[38]

S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput. 9 (1997) 1735–1780,.

Digital Library

[39]

K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, Learning phrase representations using RNN Encoder-Decoder for statistical machine translation, in: 2014 Conf. Empir. Methods Nat. Lang. Process., 2014, pp. 1724–1734. .

[40]

I. Sutskever, O. Vinyals, Q. V Le, Sequence to sequence learning with neural networks, in: Adv. Neural Inf. Process. Syst., 2014, pp. 3104–3112.

[41]

J.W. Lockhart, G.M. Weiss, Limitations with activity recognition methodology & data sets, in: Proc. 2014 ACM Int. Jt. Conf. Pervasive Ubiquitous Comput., ACM Press, New York, New York, USA, 2014, pp. 747–756,.

Digital Library

[42]

M.T. Pourazad, Z.K. Mousavi, G. Thomas, Heart sound cancellation from lung sound recordings using adaptive threshold and 2D interpolation in time-frequency domain, in: Proc. 25th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., IEEE, 2003, pp. 2586–2589,.

[43]

A. Wickramasinghe, D.C. Ranasinghe, Recognising activities in real time using body worn passive sensors with sparse data streams: to interpolate or not to interpolate?, in: Proc. 12th EAI Int. Conf. Mob. Ubiquitous Syst. Comput. Netw. Serv., ICST, 2015,.

Digital Library

[44]

A. Mishra, D.P. Agrawal, Continuous health condition monitoring by 24x7 sensing and transmission of physiological data over 5-G cellular channels, Int. Conf. Comput. Netw. Commun IEEE 2015 (2015) 584–590,.

[45]

C. Olah, Understanding LSTM Networks, 2015. <http://colah.github.io/posts/2015-08-Understanding-LSTMs/>.

[46]

D.P. Kingma, J.L. Ba, Adam: a Method for Stochastic Optimization, Int. Conf. Learn. Represent. 2015 (2015) 1–15. http://doi.acm.org.ezproxy.lib.ucf.edu/10.1145/1830483.1830503.

Digital Library

[47]

P.J. Werbos, Backpropagation through time: what it does and how to do it, Proc. IEEE. 78 (1990) 1550–1560,.

[48]

T. Plötz, N.Y. Hammerla, P. Olivier, Feature learning for activity recognition in ubiquitous computing, in: Proc. 22nd Int. Jt. Conf. Artif. Intell., AAAI Press, 2011, pp. 1729–1734,.

[49]

G. Singla, D.J. Cook, M. Schmitter-Edgecombe, Tracking activities in complex settings using smart environment technologies, Int. J. Biosci. Psychiatr. Technol. IJBSPT 1 (2009) 25–35. http://www.ncbi.nlm.nih.gov/pubmed/20019890.

[50]

L. Atallah, B. Lo, R. King, G.-Z. Yang, Sensor positioning for activity recognition using wearable accelerometers, IEEE Trans. Biomed. Circ. Syst. 5 (2011) 320–329,.

[51]

S.D. Chowdhury, U. Bhattacharya, S.K. Parui, Online handwriting recognition using Levenshtein distance metric, in: 12th Int. Conf. Doc. Anal. Recognit, IEEE, 2013, pp. 79–83,.

Digital Library

[52]

V.I. Levenshtein, Binary codes capable of correcting deletions, insertions, and reversals, Sov. Phys. Dokl. 10 (1966) 707–710.

[53]

F. Chollet, Keras, 2015. <https://github.com/keras-team/keras>.

[54]

L. Sergei, HMMlearn, 2016. <https://github.com/hmmlearn/hmmlearn>.

[55]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al., Scikit-learn: machine learning in Python, J. Mach. Learn. Res. 12 (2011) 2825–2830.

Cited By

Galvão YCastro LFerreira JNeto FFagundes RFernandes B(2024)Anomaly Detection in Smart Houses for HealthcareSN Computer Science10.1007/s42979-023-02480-y5:1Online publication date: 2-Jan-2024
https://dl.acm.org/doi/10.1007/s42979-023-02480-y
Zhao QLi GCai JZhou MFeng L(2023)A Tutorial on Internet of Behaviors: Concept, Architecture, Technology, Applications, and ChallengesIEEE Communications Surveys & Tutorials10.1109/COMST.2023.324699325:2(1227-1260)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/COMST.2023.3246993
Miled MMessaoud MBouzid A(2023)Lip reading of words with lip segmentation and deep learningMultimedia Tools and Applications10.1007/s11042-022-13321-082:1(551-571)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s11042-022-13321-0
Show More Cited By

Index Terms

A sequence-to-sequence model-based deep learning approach for recognizing activity of daily living for senior care
1. Theory of computation

Index terms have been assigned to the content through auto-classification.

Recommendations

SARRIMA: smart ADL recognizer and resident identifier in multi-resident accommodations
WH '15: Proceedings of the conference on Wireless Health

Systems for measuring Activities of Daily Livings (ADL) play a significant role in home health-care. The ability of performing ADLs successfully is used as an important factor in deciding treatments and services for patients and elderly citizens. ...
A multi-sensor deep learning approach for complex daily living activity recognition
DigiBiom '22: Proceedings of the 2022 Workshop on Emerging Devices for Digital Biomarkers

With the ever increasing elderly population, human activity trackers can help monitor the daily physical activities performed by the elderly in order to contribute towards improvements in independent living and quality of life.

Little activity ...
Pre-trained non-intrusive load monitoring model for recognizing activity of daily living
Abstract
Non-intrusive load monitoring (NILM) is a technology that analyzes total electricity consumption data to determine whether a specific type of residential appliance is operated. However, despite many previous studies conducted in this field, NILM ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Journal of Biomedical Informatics

Journal of Biomedical Informatics Volume 84, Issue C

Aug 2018

203 pages

ISSN:1532-0464

Issue’s Table of Contents

Elsevier Inc.

Publisher

Elsevier Science

San Diego, CA, United States

Publication History

Published: 01 August 2018

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Galvão YCastro LFerreira JNeto FFagundes RFernandes B(2024)Anomaly Detection in Smart Houses for HealthcareSN Computer Science10.1007/s42979-023-02480-y5:1Online publication date: 2-Jan-2024
https://dl.acm.org/doi/10.1007/s42979-023-02480-y
Zhao QLi GCai JZhou MFeng L(2023)A Tutorial on Internet of Behaviors: Concept, Architecture, Technology, Applications, and ChallengesIEEE Communications Surveys & Tutorials10.1109/COMST.2023.324699325:2(1227-1260)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/COMST.2023.3246993
Miled MMessaoud MBouzid A(2023)Lip reading of words with lip segmentation and deep learningMultimedia Tools and Applications10.1007/s11042-022-13321-082:1(551-571)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s11042-022-13321-0
Srinivasan KCurrim FRam S(2022)A Human-in-the-Loop Segmented Mixed-Effects Modeling Method for Analyzing Wearables DataACM Transactions on Management Information Systems10.1145/356427614:2(1-17)Online publication date: 24-Sep-2022
https://dl.acm.org/doi/10.1145/3564276
Yuan MWei SZhao JSun M(2022)A Systematic Survey on Human Behavior Recognition MethodsSN Computer Science10.1007/s42979-021-00932-x3:1Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s42979-021-00932-x
Manocha ASingh R(2022)A Novel Edge Analytics Assisted Motor Movement Recognition Framework Using Multi-Stage Convo-GRU ModelMobile Networks and Applications10.1007/s11036-019-01321-827:2(657-676)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s11036-019-01321-8

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents