[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content
Springer Nature Link
Log in

HHAR-net: Hierarchical Human Activity Recognition using Neural Networks

  • Conference paper
  • First Online:
Intelligent Human Computer Interaction (IHCI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12615))

Included in the following conference series:

Abstract

Activity recognition using built-in sensors in smart and wearable devices provides great opportunities to understand and detect human behavior in the wild and gives a more holistic view of individuals’ health and well being. Numerous computational methods have been applied to sensor streams to recognize different daily activities. However, most methods are unable to capture different layers of activities concealed in human behavior. Also, the performance of the models starts to decrease with increasing the number of activities. This research aims at building a hierarchical classification with Neural Networks to recognize human activities based on different levels of abstraction. We evaluate our model on the Extrasensory dataset; a dataset collected in the wild and containing data from smartphones and smartwatches. We use a two-level hierarchy with a total of six mutually exclusive labels namely, “lying down”, “sitting”, “standing in place”, “walking”, “running”, and “bicycling” divided into “stationary” and “non-stationary”. The results show that our model can recognize low-level activities (stationary/non-stationary) with 95.8% accuracy and overall accuracy of 92.8% over six labels. This is 3% above our best performing baseline (HHAR-net is shared as an open source tool at https://github.com/mehrdadfazli/HHAR-Net).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 35.99
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 44.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Avci, A., Bosch, S., Marin-Perianu, M., Marin-Perianu, R., Havinga, P.: Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: a survey, pp. 167–176 (2010)

    Google Scholar 

  2. Bardram, J.E., Doryab, A., Jensen, R.M., Lange, P.M., Nielsen, K.L., Petersen, S.T.: Phase recognition during surgical procedures using embedded and body-worn sensors. In: 2011 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp. 45–53. IEEE (2011)

    Google Scholar 

  3. Bayat, A., Pomplun, M., Tran, D.A.: A study on human activity recognition using accelerometer data from smartphones. Procedia Comput. Sci. 34, 450–457 (2014). https://doi.org/10.1016/j.procs.2014.07.009

  4. Bulling, A., Blanke, U., Schiele, B.: A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv. 46(3) (2014). https://doi.org/10.1145/2499621

  5. Chen, L., Hoey, J., Nugent, C.D., Cook, D.J., Yu, Z.: Sensor-based activity recognition (2012). https://doi.org/10.1109/TSMCC.2012.2198883

  6. Chikhaoui, B., Wang, S., Pigot, H.: A frequent pattern mining approach for ADLs recognition in smart environments. In: 2011 IEEE International Conference on Advanced Information Networking and Applications, pp. 248–255. IEEE (2011)

    Google Scholar 

  7. Chollet, F., et al.: Keras: deep learning library for theano and tensorflow (2015). https://keras.io/

  8. Direkoǧlu, C., O’Connor, N.E.: Team activity recognition in sports. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 69–83. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33786-4_6

    Chapter  Google Scholar 

  9. Doryab, A., Togelius, J.: Activity recognition in collaborative environments. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2012)

    Google Scholar 

  10. Dumais, S., Chen, H.: Hierarchical classification of web content. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 256–263 (2000)

    Google Scholar 

  11. Duong, T.V., Bui, H.H., Phung, D.Q., Venkatesh, S.: Activity recognition and abnormality detection with the switching hidden semi-Markov model. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 838–845. IEEE (2005)

    Google Scholar 

  12. Gu, T., Wang, L., Wu, Z., Tao, X., Lu, J.: A pattern mining approach to sensor-based human activity recognition. IEEE Trans. Knowl. Data Eng. 23(9), 1359–1372 (2010)

    Article  Google Scholar 

  13. Gu, T., Wu, Z., Wang, L., Tao, X., Lu, J.: Mining emerging patterns for recognizing activities of multiple users in pervasive computing. In: 2009 6th Annual International Mobile and Ubiquitous Systems: Networking & Services, MobiQuitous, pp. 1–10. IEEE (2009)

    Google Scholar 

  14. Hong, Y.J., Kim, I.J., Ahn, S.C., Kim, H.G.: Activity recognition using wearable sensors for elder care. In: 2008 Second International Conference on Future Generation Communication and Networking, vol. 2, pp. 302–305. IEEE (2008)

    Google Scholar 

  15. Jalal, A., Kamal, S., Kim, D.: A depth video sensor-based life-logging human activity recognition system for elderly care in smart indoor environments. Sensors 14(7), 11735–11759 (2014)

    Article  Google Scholar 

  16. Jatoba, L.C., Grossmann, U., Kunze, C., Ottenbacher, J., Stork, W.: Context-aware mobile health monitoring: evaluation of different pattern recognition methods for classification of physical activity. In: 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 5250–5253. IEEE (2008)

    Google Scholar 

  17. Khan, A.M., Lee, Y., Lee, S.Y., Kim, T.: Human activity recognition via an accelerometer-enabled-smartphone using kernel discriminant analysis. In: 2010 5th International Conference on Future Information Technology, pp. 1–6, May 2010. https://doi.org/10.1109/FUTURETECH.2010.5482729

  18. Kowsari, K., Brown, D.E., Heidarysafa, M., Meimandi, K.J., Gerber, M.S., Barnes, L.E.: Hdltex: hierarchical deep learning for text classification. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 364–371. IEEE (2017)

    Google Scholar 

  19. Kowsari, K., Heidarysafa, M., Brown, D.E., Meimandi, K.J., Barnes, L.E.: RMDL: random multimodel deep learning for classification. In: Proceedings of the 2nd International Conference on Information System and Data Mining, pp. 19–28 (2018)

    Google Scholar 

  20. Kowsari, K., et al.: HMIC: hierarchical medical image classification, a deep learning approach. Information 11(6), 318 (2020)

    Article  Google Scholar 

  21. Lane, N., et al.: Bewell: a smartphone application to monitor, model and promote wellbeing. In: Proceedings of the 5th International ICST Conference on Pervasive Computing Technologies for Healthcare (2011). https://doi.org/10.4108/icst.pervasivehealth.2011.246161

  22. Lara, Ó.D., Labrador, M.A.: A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutorials 15(3), 1192–1209 (2013). https://doi.org/10.1109/SURV.2012.110112.00192

    Article  Google Scholar 

  23. Maurer, U., Smailagic, A., Siewiorek, D.P., Deisher, M.: Activity recognition and monitoring using multiple sensors on different body positions. Technical report. CARNEGIE-MELLON UNIV PITTSBURGH PA SCHOOL OF COMPUTER SCIENCE (2006)

    Google Scholar 

  24. McCall, C., Reddy, K.K., Shah, M.: Macro-class selection for hierarchical k-NN classification of inertial sensor data. In: PECCS, pp. 106–114 (2012)

    Google Scholar 

  25. Murad, A., Pyun, J.Y.: Deep recurrent neural networks for human activity recognition. Sensors 17(11), 2556 (2017)

    Article  Google Scholar 

  26. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 807–814 (2010)

    Google Scholar 

  27. Nazerfard, E., Das, B., Holder, L.B., Cook, D.J.: Conditional random fields for activity recognition in smart environments. In: Proceedings of the 1st ACM International Health Informatics Symposium, pp. 282–286. ACM (2010)

    Google Scholar 

  28. Nguyen, N.T., Phung, D.Q., Venkatesh, S., Bui, H.: Learning and detecting activities from movement trajectories using the hierarchical hidden Markov model. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 2, pp. 955–960. IEEE (2005)

    Google Scholar 

  29. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  30. Ronao, C.A., Cho, S.B.: Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst. Appl. 59, 235–244 (2016). https://doi.org/10.1016/j.eswa.2016.04.032

  31. Silla, C.N., Freitas, A.A.: A survey of hierarchical classification across different application domains (2011). https://doi.org/10.1007/s10618-010-0175-9

  32. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  33. Su, X., Tong, H., Ji, P.: Activity recognition with smartphone sensors. Tsinghua Sci. Technol. 19(3), 235–249 (2014). https://doi.org/10.1109/TST.2014.6838194

    Article  Google Scholar 

  34. Vail, D.L., Veloso, M.M., Lafferty, J.D.: Conditional random fields for activity recognition. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, p. 235. ACM (2007)

    Google Scholar 

  35. Vaizman, Y., Ellis, K., Lanckriet, G.: Recognizing detailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Comput. 16(4), 62–74 (2017). https://doi.org/10.1109/MPRV.2017.3971131

    Article  Google Scholar 

  36. Wang, A., Chen, G., Yang, J., Zhao, S., Chang, C.: A comparative study on human activity recognition using inertial sensors in a smartphone. IEEE Sens. J. 16(11), 4566–4578 (2016). https://doi.org/10.1109/JSEN.2016.2545708

    Article  Google Scholar 

  37. Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L.: Deep learning for sensor-based activity recognition: a survey. Pattern Recogn. Lett. 119, 3–11 (2019). https://doi.org/10.1016/j.patrec.2018.02.010

  38. Zeng, M., et al.: Convolutional neural networks for human activity recognition using mobile sensors. In: 6th International Conference on Mobile Computing, Applications and Services, pp. 197–205, November 2014. https://doi.org/10.4108/icst.mobicase.2014.257786

Download references

Author information

Authors and Affiliations

  1. University of Virginia, Charlottesville, VA, 22903, USA

    Mehrdad Fazli, Kamran Kowsari, Erfaneh Gharavi, Laura Barnes & Afsaneh Doryab

  2. University of California, Los Angeles, CA, 90095, USA

    Kamran Kowsari

Authors
  1. Mehrdad Fazli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamran Kowsari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erfaneh Gharavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Laura Barnes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Afsaneh Doryab
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehrdad Fazli .

Editor information

Editors and Affiliations

  1. Woosong University, Daejeon, Korea (Republic of)

    Madhusudan Singh

  2. Dongseo University, Busan, Korea (Republic of)

    Dae-Ki Kang

  3. Keimyung University, Daegu, Korea (Republic of)

    Jong-Ha Lee

  4. Indian Institute of Information Technoloy, Allahabad, India

    Uma Shanker Tiwary

  5. Hankuk University of Foreign Studies, Yongin, Korea (Republic of)

    Dhananjay Singh

  6. Pukyong National University, Busan, Korea (Republic of)

    Wan-Young Chung

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fazli, M., Kowsari, K., Gharavi, E., Barnes, L., Doryab, A. (2021). HHAR-net: Hierarchical Human Activity Recognition using Neural Networks. In: Singh, M., Kang, DK., Lee, JH., Tiwary, U.S., Singh, D., Chung, WY. (eds) Intelligent Human Computer Interaction. IHCI 2020. Lecture Notes in Computer Science(), vol 12615. Springer, Cham. https://doi.org/10.1007/978-3-030-68449-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68449-5_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68448-8

  • Online ISBN: 978-3-030-68449-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation