[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

User identification using deep learning and human activity mobile sensor data

  • Regular contribution
  • Published:
International Journal of Information Security Aims and scope Submit manuscript

Abstract

The ownership of user actions in computer and mobile applications is an important concern, especially when using shared devices. User identification using physical biometric authentication methods permits the actual user to access the device. However, in cases where different users may access a shared device during the same active session, the person who owns the active session will be accountable for any actions performed by the other users. Thus, user identification using behavioral characteristics has come into the picture. Human activity recognition from mobile sensor data is gaining more interest with the advent of mobile devices and the emergence of the Internet of Things, where different applications such as elderly health monitoring, athletic evaluation, and context-aware behavior are being developed. In this paper, we show how human activity data can be utilized to identify the actual device user. We build deep learning models that are capable of identifying the users of mobile and wearable devices based on their body movements and daily activities. We use the Long Short-Term Memory classifier for building the user identification model based on time-series data from mobile motion sensors. The model targets the users that were involved in the training process. We tested our approach on two publicly available human activity datasets that contain daily activities and fall states data from accelerometer and gyroscope mobile sensors. The results show that the models are capable of identifying the actual mobile device users from their motion data with an accuracy of up to 90%. Further, the results show that the model from the accelerometer data outperforms the one from gyroscope data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability statement

This research uses the UCI-HAR [22] and the UniMiB SHAR [23] public datasets.

Notes

  1. https://colab.research.google.com/

References

  1. Elmaghraby, A.S., Losavio, M.M.: Cyber security challenges in smart cities: safety, security and privacy. J. Adv. Res. 5(4), 491–497 (2014)

    Article  Google Scholar 

  2. Malatras, A., Geneiatakis, D., Vakalis, I.: On the efficiency of user identification: a system-based approach. Int. J. Inf. Secur. 16(6), 653–671 (2017)

    Article  Google Scholar 

  3. Jain, A.K., Ross, A., Prabhakar, S.: An introduction to biometric recognition. IEEE Trans. Circuits Syst. Video Technol. 14(1), 4–20 (2004)

    Article  Google Scholar 

  4. Khan, S.H., Akbar, M.A., Shahzad, F., Farooq, M., Khan, Z.: Secure biometric template generation for multi-factor authentication. Pattern Recogn. 48(2), 458–472 (2015)

    Article  Google Scholar 

  5. Parkin, S., Patel, T., Lopez-Neira, I., Tanczer, L.: Usability analysis of shared device ecosystem security: informing support for survivors of iot-facilitated tech-abuse. In: Proceedings of the New Security Paradigms Workshop, pp. 1–15 (2019)

  6. Nurse, J.R., Buckley, O., Legg, P.A., Goldsmith, M., Creese, S., Wright, G.R., Whitty, M.: Understanding insider threat: A framework for characterising attacks. In: IEEE Security and Privacy Workshops. IEEE 2014, pp. 214–228 (2014)

  7. Gadaleta, M., Rossi, M.: Idnet: Smartphone-based gait recognition with convolutional neural networks. Pattern Recogn. 74, 25–37 (2018)

    Article  Google Scholar 

  8. Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Cell phone-based biometric identification. In: 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS)

  9. Vrigkas, M., Nikou, C., Kakadiaris, I.A.: A review of human activity recognition methods. Front. Robot. AI 2, 28 (2015)

    Article  Google Scholar 

  10. Chen, Y., Zhu, X., Zheng, W., Lai, J.: Person re-identification by camera correlation aware feature augmentation. IEEE Trans. Pattern Anal. Mach. Intell. 40(2), 392–408 (2018)

    Article  Google Scholar 

  11. Khalil, N., Gnawali, O., Benhaddou, D., Subhlok, J.: Sonicdoor: A person identification system based on modeling of shape, behavior, and walking patterns. ACM Trans. Sens. Networks 14(3–4), 27-1–27-21 (2018)

    Google Scholar 

  12. Zhao, Y., Dong, L., Wang, J., Hu, B., Fu, Y.: Implementing indoor positioning system via zigbee devices. In: 2008 42nd Asilomar Conference on Signals, Systems and Computers, Pacic Grove, CA, 2008, pp. 1867–1871 (2008)

  13. Saevanee, H., Bhatarakosol, P.: User authentication using combination of behavioral biometrics over the touchpad acting like touch screen of mobile device. In: 2008 International Conference on Computer and Electrical Engineering. IEEE, pp. 82–86 (2008)

  14. Eberz, S., Rasmussen, K.B., Lenders, V., Martinovic, I.: Looks like eve: exposing insider threats using eye movement biometrics. ACM Trans. Privacy Secur. 19(1), 1–31 (2016)

    Article  Google Scholar 

  15. Tang, C., Phoha, V.V.: An empirical evaluation of activities and classifiers for user identification on smartphones. In: 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (Btas). IEEE, pp. 1–8 (2016)

  16. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  17. Hnoohom, N., Jitpattanakul, A., You, I., Mekruksavanich, S.: Deep learning approach for complex activity recognition using heterogeneous sensors from wearable device. In: 2021 Research, Invention, and Innovation Congress: Innovation Electricals and Electronics (RI2C), Bangkok, Thailand, September pp. 60–65 (2021)

  18. Chathuramali, K.M., Rodrigo, R.: Faster human activity recognition with svm. In: International Conference on Advances in ICT for Emerging Regions (ICTer2012). IEEE, pp. 197–203 (2012)

  19. Sprager, S., Zazula, D.: A cumulant-based method for gait identification using accelerometer data with principal component analysis and support vector machine. WSEAS Trans. Signal Process. 5(11), 369–378 (2009)

    Google Scholar 

  20. Mekruksavanich, S., Jitpattanakul, A.: Biometric user identification based on human activity recognition using wearable sensors: An experiment using deep learning models. Electronics 10(3), 308 (2021)

    Article  Google Scholar 

  21. Zhang, M.: Gait activity authentication using lstm neural networks with smartphone sensors. In: 2019 15th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN), Shenzhen, China, pp. 456–461 (2019)

  22. Anguita, D., Ghio, A., Oneto, L., Parra Perez, X., Reyes Ortiz, J.L.: A public domain dataset for human activity recognition using smartphones. In: Proceedings of the 21th International European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp. 437–442 (2013)

  23. Micucci, D., Mobilio, M., Napoletano, P.: Unimib shar: A dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7(10), 1101 (2017)

    Article  Google Scholar 

  24. BenAbdelkader, C., Cutler, R., Davis, L.S.: Person identification using automatic height and stride estimation. In: 16th International Conference on Pattern Recognition, ICPR 2002. Quebec, Canada: IEEE Computer Society, pp. 377–380 (2002)

  25. Johnson, A.Y., Bobick, A.F.: A multi-view method for gait recognition using static body parameters. In: Bigün, J., Smeraldi, F. (eds.) Third International Conference on Audio-and Video-Based Biometric Person Authentication, AVBPA 2001, Halmstad, Sweden, pp. 301–311 (2001)

  26. Srinivasan, V., Stankovic, J.A., Whitehouse, K.: Using height sensors for biometric identification in multi-resident homes. In: Floréen, P., Krüger, A., Spasojevic, M. (Eds.) 8th International Conference on Pervasive Computing, Helsinki, Finland, 17-20 May, pp. 337–354 (2010)

  27. Wong, K.B., Zhang, T., Aghajan, H.K.: Extracting patterns of behavior from a network of binary sensors. J. Ambient. Intell. Humaniz. Comput. 6(1), 83–105 (2015)

    Article  Google Scholar 

  28. Mokhtari, G., Bashi, N., Zhang, Q., Nourbakhsh, G.: Non-wearable human identification sensors for smart home environment: a review. Sens. Rev. 38(3), 391–404 (2018)

    Article  Google Scholar 

  29. Kim, H., Kim, I., Kim, J.: Designing the smart foot mat and its applications: as a user identification sensor for smart home scenarios. Adv. Sci. Technol. Lett. 87, 1–5 (2015)

    Google Scholar 

  30. Heo, K.H., Jeong, S., Kang, S.J.: Real-time user identification and behavior prediction based on foot-pad recognition. Sensors 19(13), 2899 (2019)

    Article  Google Scholar 

  31. Crandall, A.S., Cook, D.J.: Behaviometrics for identifying smart home residents. In: Human Aspects in Ambient Intelligence. Atlantis Press, Paris, Vol. 8, pp. 55–71 (2013)

  32. Lesani, F.S., Ghazvini, F.F., Amirkhani, H.: Smart home user identification using bag of events approach. In: 2017 7th International Conference on Computer and Knowledge Engineering (ICCKE). Mashhad, Iran: IEEE, pp. 379–383 (2017)

  33. Zhang, J., Wei, B., Hu, W., Kanhere, S.S.: Wifi-id: Human identification using wifi signal. In: 2016 International Conference on Distributed Computing in Sensor Systems (DCOSS), Washington, DC, USA, pp. 75–82 (2016)

  34. Zeng, Y., Pathak, P.H., Mohapatra, P.: Wiwho: Wifi-based person identification in smart spaces. In: 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), Vienna, Austria, pp. 1–12 (2016)

  35. Ropponen, A., Rimminen, H., Sepponen, R.: Robust system for indoor localisation and identification for the health care environment. Wireless Pers. Commun. 59(1), 57–71 (2011)

    Article  Google Scholar 

  36. Truong, P.H., You, S., Ji, S.-H., Jeong, G.-M.: Wearable system for daily activity recognition using inertial and pressure sensors of a smart band and smart shoes. Int. J. Comput. Commun. Control 14(6), 726–742 (2020)

    Article  Google Scholar 

  37. Bergmann, J., McGregor, A.: Body-worn sensor design: what do patients and clinicians want? Ann. Biomed. Eng. 39(9), 2299–2312 (2011)

    Article  Google Scholar 

  38. López, G., Marín, G., Calderón, M.: Human aspects of ubiquitous computing: a study addressing willingness to use it and privacy issues. J. Ambient. Intell. Humaniz. Comput. 8(4), 497–511 (2017)

    Article  Google Scholar 

  39. Zheng, N., Bai, K., Huang, H., Wang, H.: You are how you touch: User verification on smartphones via tapping behaviors. In: 2014 IEEE 22nd International Conference on Network Protocols. IEEE, Raleigh, NC, pp. 221–232 (2014)

  40. Muaaz, M., Mayrhofer, R.: Accelerometer based gait recognition using adapted gaussian mixture models. In: Proceedings of the 14th International Conference on Advances in Mobile Computing and Multi Media, Singapore, Singapore, pp. 288–291 (2016)

  41. Ramos, F.B.A., Lorayne, A., Costa, A.A.M., de Sousa, R.R., Almeida, H.O., Perkusich, A.: Combining smartphone and smartwatch sensor data in activity recognition approaches: an experimental evaluation. In: Gou, J. (Ed.) The 28th International Conference on Software Engineering and Knowledge Engineering, SEKE 2016, Redwood City, San Francisco Bay, USA, July 1-3 pp. 267–272 (2016)

  42. Cola, G., Avvenuti, M., Musso, F., Vecchio, A.: Gait-based authentication using a wrist-worn device. In: Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, Hiroshima, Japan, November 28–December 1, pp. 208–217 (2016)

  43. Findling, R.D., Muaaz, M., Hintze, D., Mayrhofer, R.: Shakeunlock: Securely transfer authentication states between mobile devices. IEEE Trans. Mob. Comput. 16(4), 1163–1175 (2016)

    Article  Google Scholar 

  44. Lewis, A., Li, Y., Xie, M.: Real time motion-based authentication for smartwatch. In: 2016 IEEE Conference on Communications and Network Security (CNS), Philadelphia, PA, USA, October 17-19, pp. 380–381 (2016)

  45. Liang, G.-C., Xu, X.-Y., Yu, J.-D.: User-authentication on wearable devices based on punch gesture biometrics. In: ITM Web of Conferences, vol. 11. EDP Sciences, (2017)

  46. Griswold-Steiner, I., Matovu, R., Serwadda, A.: Handwriting watcher: A mechanism for smartwatch-driven handwriting authentication. In: 2017 IEEE International Joint Conference on Biometrics (IJCB). IEEE, Denver, 1-4 October, pp. 216–224 (2017)

  47. Kumar, R., Phoha, V.V., Serwadda, A.: Continuous authentication of smartphone users by fusing typing, swiping, and phone movement patterns. In; 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS). IEEE, Niagara Falls, NY, vol 6–9, pp. 1–8 (2016)

  48. Zeng, Y., Pande, A., Zhu, J., Mohapatra, P.: Wearia: Wearable device implicit authentication based on activity information. In: 2017 IEEE 18th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM). Macau, China, Vol 12–15, pp. 1–9 (2017)

  49. Yang, J., Li, Y., Xie, M.: Motionauth: Motion-based authentication for wrist worn smart devices. In: 2015 IEEE International conference on pervasive computing and communication workshops (PerCom Workshops). IEEE, St. Louis, MO, 23-27 March, pp. 550–555 (2015)

  50. Al-Naffakh, N., Clarke, N., Li, F.: Continuous user authentication using smartwatch motion sensor data. In: IFIP International Conference on Trust Management. Springer, Toronto, July 10-13, pp. 15–28 (2018)

  51. Xu, W., Shen, Y., Zhang, Y., Bergmann, N., Hu, W.: Gait-watch: A context-aware authentication system for smart watch based on gait recognition. In: Proceedings of the Second International Conference on Internet-of-Things Design and Implementation, Pittsburgh, PA, USA, April 18–21, pp. 59–70 (2017)

  52. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. ACM SIGKDD Explor. Newsl 11(1), 10–18 (2009)

    Article  Google Scholar 

  53. Amini, S., Noroozi, V., Pande, A., Gupte, S., Yu, P.S., Kanich, C.: Deepauth: A framework for continuous user re-authentication in mobile apps. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, ser. CIKM ’18, Torino, Italy, October, pp. 2027–2035 (2018)

  54. Lee, S.-M., Yoon, S.M., Cho, H.: Human activity recognition from accelerometer data using convolutional neural network. In; 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, South Korea, February pp. 131–134 (2017)

  55. Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters, vol. 119, pp. 3–11, deep Learning for Pattern Recognition (2019)

  56. Weiss, G.M., Yoneda, K., Hayajneh, T.: Smartphone and smartwatch-based biometrics using activities of daily living. IEEE Access 7, 133190–133202 (2019)

    Article  Google Scholar 

  57. Giorgi, G., Saracino, A., Martinelli, F.: Using recurrent neural networks for continuous authentication through gait analysis. Pattern Recogn. Lett. 147, 157–163 (2021)

    Article  Google Scholar 

  58. Sitová, Z., Šeděnka, J., Yang, Q., Peng, G., Zhou, G., Gasti, P., Balagani, K.S.: Hmog: New behavioral biometric features for continuous authentication of smartphone users. IEEE Trans. Inf. Forensics Secur. 11(5), 877–892 (2016)

    Article  Google Scholar 

  59. Abuhamad, M., Abuhmed, T., Mohaisen, D., Nyang, D.: AUToSen: deep-learning-based implicit continuous authentication using smartphone sensors. IEEE Int. Things J. 7(6), 5008–5020 (2020)

    Article  Google Scholar 

  60. Chicco, D.: Ten quick tips for machine learning in computational biology. BioData Mining 10(1), 1–17 (2017)

    Article  Google Scholar 

  61. Mekruksavanich, S., Jantawong, P., Jitpattanakul, A.: Enhancement of sensor-based user identification using data augmentation techniques. In: 2022 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT NCON), Chiang Rai, Thailand, January pp. 333–337 (2022)

  62. Casale, P., Pujol, O., Radeva, P.: Personalization and user verification in wearable systems using biometric walking patterns. Pers. Ubiquit. Comput. 16(5), 563–580 (2012)

    Article  Google Scholar 

  63. Ye, N., Sun, C., Xu, R., Sun, F.: A method of equipment safety certification based on daily cycle activity. In: 2021 4th International Conference on Artificial Intelligence and Big Data (ICAIBD). IEEE, pp. 651–658 (2021)

  64. Taylor, S.J., Letham, B.: Forecasting at scale. Am. Stat. 72(1), 37–45 (2018)

    Article  Google Scholar 

  65. Yao, S., Hu, S., Zhao, Y., Zhang, A., Abdelzaher, T.: Deepsense: A unified deep learning framework for time-series mobile sensing data processing. In: Proceedings of the 26th International Conference on World Wide Web, ser. WWW ’17, Perth, Australia, April, pp. 351–360 (2017)

  66. Deb, D., Ross, A., Jain, A.K., Prakah-Asante, K., Prasad, K.V.: Actions speak louder than (pass)words: Passive authentication of smartphone* users via deep temporal features. In: 2019 International Conference on Biometrics (ICB), Crete, Greece, June, pp. 1–8 (2019)

  67. Chen, L., Zhang, Y., Peng, L.: Metier: a deep multi-task learning based activity and user recognition model using wearable sensors. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 4(1), 1–18 (2020)

    Article  Google Scholar 

  68. Medsker, L.R., Jain, L.: Recurrent neural networks. Design and Applications, vol. 5, (2001)

  69. Sherstinsky, A.: Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network. Physica D 404, 132306 (2020)

    Article  MATH  Google Scholar 

  70. Chen, Y., Zhong, K., Zhang, J., Sun, Q., Zhao, X.: Lstm networks for mobile human activity recognition. In: International conference on artificial intelligence: technologies and applications. Atlantis Press, pp. 50–53 (2016)

  71. Zebin, T., Sperrin, M., Peek, N., Casson, A.J.: Human activity recognition from inertial sensor time-series using batch normalized deep lstm recurrent networks. In: 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, pp. 1–4 (2018)

  72. San-Segundo, R., Lorenzo-Trueba, J., Martínez-González, B., Pardo, J.M.: Segmenting human activities based on hmms using smartphone inertial sensors. Pervasive Mob. Comput. 30, 84–96 (2016)

    Article  Google Scholar 

  73. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, (2014)

  74. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MATH  Google Scholar 

  75. Ramachandran, P., Zoph, B., Le, Q.V.: Searching for activation functions. arXiv preprint arXiv:1710.05941, (2017)

  76. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  77. Huang, W., Zhang, L., Teng, Q., Song, C., He, J.: The convolutional neural networks training with channel-selectivity for human activity recognition based on sensors. IEEE J. Biomed. Health Inform. 25(10), 3834–3843 (2021)

    Article  Google Scholar 

  78. Alawneh, L., Alsarhan, T., Al-Zinati, M., Al-Ayyoub, M., Jararweh, Y., Lu, H.: Enhancing human activity recognition using deep learning and time series augmented data. J. Ambient Intell. Human. Comput. 12, 1–16 (2021)

    Article  Google Scholar 

  79. Alawneh, L., Al-Ayyoub, M., Al-Sharif, Z.A., Shatnawi, A.: Personalized human activity recognition using deep learning and edge-cloud architecture. J. Ambient Intell. Human. Comput. pp. 1–13, (2022)

  80. Han, C., Zhang, L., Tang, Y., Huang, W., Min, F., He, J.: Human activity recognition using wearable sensors by heterogeneous convolutional neural networks. Exp. Syst. Appl. pp. 116764 (2022)

  81. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard M.: Tensorflow: a system for large-scale machine learning. In: 12th \(\{USENIX\}\) symposium on operating systems design and implementation (\(\{OSDI\}\) 16), pp. 265–283 (2016)

  82. Ketkar, N.: Introduction to keras. In: Deep learning with Python. Springer, pp. 97–111 (2017)

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer and Information Technology, Jordan University of Science and Technology, Irbid, Jordan

    Luay Alawneh, Mohammad Al-Zinati & Mahmoud Al-Ayyoub

  2. Department of Information Technology, College of Engineering and Information Technology, Ajman University, Ajman, UAE

    Mahmoud Al-Ayyoub

Authors
  1. Luay Alawneh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Al-Zinati
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahmoud Al-Ayyoub
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luay Alawneh.

Ethics declarations

Conflict of interest

No conflicts of interest.

Research involving human participants and/or animals

This research study uses publicly available data that were gathered from human participants. Thus, the responsibility lies with the authors of the two datasets. Further, we did not add new data in this study.

Informed consent

Not applicable in this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alawneh, L., Al-Zinati, M. & Al-Ayyoub, M. User identification using deep learning and human activity mobile sensor data. Int. J. Inf. Secur. 22, 289–301 (2023). https://doi.org/10.1007/s10207-022-00640-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10207-022-00640-4

Keywords

Search

Navigation