Abstract
Human Posture Recognition is gaining increasing attention in the field of computer vision due to its promising applications in the areas of health care, human-computer interaction, and surveillance systems. This paper presents a novel method for human posture recognition by combining both color and depth images and feeding the resulting information into the vision transformer (ViT) model. We want to take advantage of integrating the Lab-D HOG descriptor [18] into the ViT architecture [8]. First, we compute the multispectral Lab-D edge detector by opting for the maximum eigenvalue of the multiplication of the jacobian matrix by its transpose. Second, we select the multispectral corner points by picking the minimum of the eigenvalues of the multispectral Harris matrix. Third, for each selected corner point, we compute a Lab-D HOG descriptor. Last, we feed the extracted Lab-D HOG descriptors into the transformer encoder/decoder by implementing two different strategies. Results show that we outperform state-of-the-art methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abobakr, A., Nahavandi, D., Iskander, J., Hossny, M., Nahavandi, S., Smets, M.: RGB-D human posture analysis for ergonomie studies using deep convolutional neural network. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2885–2890 (2017)
Amine Elforaici, M.E., Chaaraoui, I., Bouachir, W., Ouakrim, Y., Mezghani, N.: Posture recognition using an RGB-D camera: exploring 3D body modeling and deep learning approaches. In: 2018 IEEE Life Sciences Conference (LSC), pp. 69–72 (2018). https://doi.org/10.1109/LSC.2018.8572079
Ayre-Storie, A., Zhang, L.: Deep learning-based human posture recognition. In: 2021 International Conference on Machine Learning and Cybernetics (ICMLC), pp. 1–6 (2021). https://doi.org/10.1109/ICMLC54886.2021.9737241
Baronti, P., Girolami, M., Mavilia, F., Palumbo, F., Luisetto, G.: On the analysis of human posture for detecting social interactions with wearable devices. In: 2020 IEEE International Conference on Human-Machine Systems (ICHMS), pp. 1–6 (2020). https://doi.org/10.1109/ICHMS49158.2020.9209510
Cai, Y., Wang, X., Kong, X.: 3D human pose estimation from RGB+D images with convolutional neural networks. In: Proceedings of the 2nd International Conference on Biomedical Engineering and Bioinformatics, ICBEB 2018, pp. 64–69. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3278198.3278225
Cao, B., Bi, S., Zheng, J., Yang, D.: Human posture recognition using skeleton and depth information. In: 2018 WRC Symposium on Advanced Robotics and Automation (WRC SARA), pp. 275–280 (2018). https://doi.org/10.1109/WRC-SARA.2018.8584233
Ding, W., Hu, B., Liu, H., Wang, X., Huang, X.: Human posture recognition based on multiple features and rule learning. Int. J. Mach. Learn. Cybern. 11, 2529–2540 (2020). https://doi.org/10.1007/s13042-020-01138-y
Dosovitskiy, A., et al.: An image is worth \(16\times 16\) words: transformers for image recognition at scale (2020). https://doi.org/10.48550/ARXIV.2010.11929. https://arxiv.org/abs/2010.11929
Giannakos, I., Mathe, E., Spyrou, E., Mylonas, P.: A study on the effect of occlusion in human activity recognition, pp. 473–482 (2021). https://doi.org/10.1145/3453892.3461337
Gjoreski, H., Gams, M.: Activity/posture recognition using wearable sensors placed on different body locations. In: Proceedings of Signal and Image Processing and Applications (2011). https://doi.org/10.2316/P.2011.716-067
Iazzi, A., Rziza, M., Thami, R.O.H.: Human posture recognition based on projection histogram and Support Vector Machine. In: 2018 9th International Symposium on Signal, Image, Video and Communications (ISIVC), pp. 329–333 (2018). https://doi.org/10.1109/ISIVC.2018.8709235
Li, X., Sun, M., Fang, X.: An approach for detecting human posture by using depth image. In: 2016 2nd International Conference on Artificial Intelligence and Industrial Engineering (AIIE 2016), pp. 257–261. Atlantis Press (2016)
Li, X., Zhou, Z., Wu, J., Xiong, Y.: Human posture detection method based on wearable devices. J. Healthc. Eng. 2021, 1–8 (2021). https://doi.org/10.1155/2021/8879061
Liu, S., Ostadabbas, S.: A vision-based system for in-bed posture tracking. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 1373–1382 (2017). https://doi.org/10.1109/ICCVW.2017.163
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows (2021). https://doi.org/10.48550/ARXIV.2103.14030. https://arxiv.org/abs/2103.14030
Malmsten, J., Cengiz, H., Lood, D.: Histogram of oriented gradients in a vision transformer (2022)
Mefteh, S., Kaâniche, M.B., Ksantini, R., Bouhoula, A.: A novel multispectral lab-depth based edge detector for color images with occluded objects. In: VISIGRAPP (4: VISAPP), pp. 272–279 (2019)
Mefteh, S., Kaâniche, M.B., Ksantini, R., Bouhoula, A.: A novel multispectral corner detector and a new local descriptor: an application to human posture recognition. Multimed. Tools Appl. 82, 28937–28956 (2023). https://doi.org/10.1007/s11042-023-14788-1
Ni, B., Wang, G., Moulin, P.: RGBD-HuDaAct: a color-depth video database for human daily activity recognition. In: Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K. (eds.) Consumer Depth Cameras for Computer Vision. ACVPR, pp. 193–208. Springer, London (2013). https://doi.org/10.1007/978-1-4471-4640-7_10
Popescu, A.C., Mocanu, I., Cramariuc, B.: PRECIS HAR (2019). https://doi.org/10.21227/mene-ck48
Qi, L., Han, Y.: Human motion posture detection algorithm using deep reinforcement learning. Mob. Inf. Syst. 2021, 1–10 (2021). https://doi.org/10.1155/2021/4023861
Ramanan, D., Sminchisescu, C.: Training deformable models for localization, vol. 1, pp. 206–213 (2006). https://doi.org/10.1109/CVPR.2006.315
Reddy, B.H., Karthikeyan, P.: Classification of fire and smoke images using decision tree algorithm in comparison with logistic regression to measure accuracy, precision, recall, f-score. In: 2022 14th International Conference on Mathematics, Actuarial Science, Computer Science and Statistics (MACS), pp. 1–5. IEEE (2022)
Redmon, J., Divvala, S.K., Girshick, R.B., Farhadi, A.: You only look once: unified, real-time object detection. CoRR abs/1506.02640 (2015). https://arxiv.org/abs/1506.02640
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jegou, H.: Training data-efficient image transformers & distillation through attention. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 10347–10357. PMLR (2021). https://proceedings.mlr.press/v139/touvron21a.html
Wang, W.J., Chang, J.W., Haung, S.F., Wang, R.J.: Human posture recognition based on images captured by the kinect sensor. Int. J. Adv. Robot. Syst. 13(2), 54 (2016). https://doi.org/10.5772/62163
Wu, Q., Xu, G., Zhang, S., Li, Y., Wei, F.: Human 3D pose estimation in a lying position by RGB-D images for medical diagnosis and rehabilitation (2020). https://doi.org/10.1109/EMBC44109.2020.9176407
Wu, Y., et al.: Rethinking classification and localization for object detection, pp. 10183–10192 (2020). https://doi.org/10.1109/CVPR42600.2020.01020
Zhang, J., Wu, C., Wang, Y.: Human fall detection based on body posture spatio-temporal evolution. Sensors 20(3), 946 (2020). https://doi.org/10.3390/s20030946. https://www.mdpi.com/1424-8220/20/3/946
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mefteh, S., Kaâniche, MB., Ksantini, R., Bouhoula, A. (2023). Learning Human Postures Using Lab-Depth HOG Descriptors. In: Nguyen, N.T., et al. Computational Collective Intelligence. ICCCI 2023. Lecture Notes in Computer Science(), vol 14162. Springer, Cham. https://doi.org/10.1007/978-3-031-41456-5_42
Download citation
DOI: https://doi.org/10.1007/978-3-031-41456-5_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-41455-8
Online ISBN: 978-3-031-41456-5
eBook Packages: Computer ScienceComputer Science (R0)