More Web Proxy on the site http://driver.im/

research-article

Hybrid SNN-based Privacy-Preserving Fall Detection using Neuromorphic Sensors

Authors:

Shyam Sunder Prasad,

Naval Kishore Mehta,

Himanshu Kumar,

Abeer Banerjee,

Sanjay SinghAuthors Info & Claims

ICVGIP '23: Proceedings of the Fourteenth Indian Conference on Computer Vision, Graphics and Image Processing

Article No.: 19, Pages 1 - 9

https://doi.org/10.1145/3627631.3627650

Published: 31 January 2024 Publication History

Abstract

Indoor surveillance is crucial for ensuring the safety and security of occupants within the premises. Only those who are ill or elderly tend to spend the most time at home. The use of indoor surveillance to continuously monitor these people’s security could help in the early detection and avoidance of tragic incidents. Ensuring privacy while achieving this task has led to a recent research focus on protecting privacy in human fall detection. This paper attempts to address the issue of privacy-preserving fall detection by employing the Dynamic Vision Sensor (DVS), which captures intensity changes without compromising individuals’ privacy. This paper introduces a novel event-based dataset named “DVSFall”, incorporating diverse daily living activities (ADL) and simulated falls. Captured from multiple viewpoints using DVS cameras, the dataset encompasses twenty-one participants across varying age groups. To evaluate the dataset, we employed Spiking Neural Networks (SNN) designed to replicate neural activity. Furthermore, we explored a hybrid framework, the 3D-CNN & SNN (NeuCube) approach, for fall detection. Our proposed framework achieved an accuracy of 94.59% with SNN and notably improved to 97.84% using the hybrid approach, as measured against the recorded dataset.

References

[1]

2022. Dynamic Vision Sensor. https://inivation.com/products/customsolutions/videos/. Accessed: 2022-04-13.

[2]

2022. SLAYER PyTorch. https://bamsumit.github.io/slayerPytorch/build/html/spikeLoss.html. Accessed: 2022-04-28.

[3]

Abeer Banerjee, Shyam Sunder Prasad, Naval Kishore Mehta, Himanshu Kumar, Sumeet Saurav, and Sanjay Singh. 2022. Gaze Detection Using Encoded Retinomorphic Events. In International Conference on Intelligent Human Computer Interaction. Springer, 442–453.

[4]

Ahmed Nabil Belbachir, Stephan Schraml, and Aneta Nowakowska. 2011. Event-driven stereo vision for fall detection. In CVPR 2011 WORKSHOPS. IEEE, 78–83.

[5]

Tobias Bolten, Regina Pohle-Frohlich, and Klaus D Tonnies. 2021. DVS-OUTLAB: A Neuromorphic Event-Based Long Time Monitoring Dataset for Real-World Outdoor Scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1348–1357.

[6]

Paula Branco, Luís Torgo, and Rita P Ribeiro. 2016. A survey of predictive modeling on imbalanced domains. ACM Computing Surveys (CSUR) 49, 2 (2016), 1–50.

Digital Library

[7]

Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, 2018. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro 38, 1 (2018), 82–99.

[8]

Koldo De Miguel, Alberto Brunete, Miguel Hernando, and Ernesto Gambao. 2017. Home camera-based fall detection system for the elderly. Sensors 17, 12 (2017), 2864.

[9]

Mireille El-Assal, Pierre Tirilly, and Ioan Marius Bilasco. 2021. A Study On the Effects of Pre-processing On Spatio-temporal Action Recognition Using Spiking Neural Networks Trained with STDP. In 2021 International Conference on Content-Based Multimedia Indexing (CBMI). IEEE, 1–6.

[10]

Yaxiang Fan, Martin D Levine, Gongjian Wen, and Shaohua Qiu. 2017. A deep neural network for real-time detection of falling humans in naturally occurring scenes. Neurocomputing 260 (2017), 43–58.

Digital Library

[11]

Steve B Furber, Francesco Galluppi, Steve Temple, and Luis A Plana. 2014. The spinnaker project. Proc. IEEE 102, 5 (2014), 652–665.

[12]

Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, 2020. Event-based vision: A survey. IEEE transactions on pattern analysis and machine intelligence 44, 1 (2020), 154–180.

Digital Library

[13]

Yves M Galvão, Janderson Ferreira, Vinícius A Albuquerque, Pablo Barros, and Bruno JT Fernandes. 2021. A multimodal approach using deep learning for fall detection. Expert Systems with Applications 168 (2021), 114226.

[14]

Samuele Gasparrini, Enea Cippitelli, Susanna Spinsante, and Ennio Gambi. 2014. A depth-based fall detection system using a Kinect® sensor. Sensors 14, 2 (2014), 2756–2775.

[15]

Ronak Gupta, Prashant Anand, Santanu Chaudhury, Brejesh Lall, and Sanjay Singh. 2020. Compressive sensing based privacy for fall detection. In Computer Vision, Pattern Recognition, Image Processing, and Graphics: 7th National Conference, NCVPRIPG 2019, Hubballi, India, December 22–24, 2019, Revised Selected Papers 7. Springer, 429–438.

[16]

Nikola Kasabov, Nathan Matthew Scott, Enmei Tu, Stefan Marks, Neelava Sengupta, Elisa Capecci, Muhaini Othman, Maryam Gholami Doborjeh, Norhanifah Murli, Reggio Hartono, 2016. Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications. Neural Networks 78 (2016), 1–14.

Digital Library

[17]

Nikola K Kasabov. 2014. NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Networks 52 (2014), 62–76.

Digital Library

[18]

Erik Krempel, Pascal Birnstill, and Jürgen Beyerer. 2017. A Privacy-Aware Fall Detection System for Hospitals and Nursing Facilities. European Journal for Security Research 2, 2 (2017), 83–95.

[19]

Hyunwoo Lee, Jooyoung Kim, Dojun Yang, and Joon-Ho Kim. 2017. Embedded real-time fall detection using deep learning for elderly care. arXiv preprint arXiv:1711.11200 (2017).

[20]

Fuyou Liao, Feichi Zhou, and Yang Chai. 2021. Neuromorphic vision sensors: Principle, progress and perspectives. Journal of Semiconductors 42, 1 (2021), 013105.

[21]

Jixin Liu, Rong Tan, Guang Han, Ning Sun, and Sam Kwong. 2020. Privacy-Preserving In-Home Fall Detection Using Visual Shielding Sensing and Private Information-Embedding. IEEE Transactions on Multimedia 23 (2020), 3684–3699.

Digital Library

[22]

Qianhui Liu, Dong Xing, Huajin Tang, De Ma, and Gang Pan. 2021. Event-based Action Recognition Using Motion Information and Spiking Neural Networks. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Z.-H. Zhou, Ed. International Joint Conferences on Artificial Intelligence Organization, Vol. 8. 1743–1749.

[23]

Chao Ma, Atsushi Shimada, Hideaki Uchiyama, Hajime Nagahara, and Rin-ichiro Taniguchi. 2019. Fall detection using optical level anonymous image sensing system. Optics & Laser Technology 110 (2019), 44–61.

[24]

A. Marchisio, G. Pira, M. Martina, G. Masera, and M. Shafique. 2021. DVS-Attacks: Adversarial Attacks on Dynamic Vision Sensors for Spiking Neural Networks. In 2021 International Joint Conference on Neural Networks (IJCNN).

[25]

Paul A Merolla, John V Arthur, Rodrigo Alvarez-Icaza, Andrew S Cassidy, Jun Sawada, Filipp Akopyan, Bryan L Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, 2014. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 6197 (2014), 668–673.

[26]

Shu Miao, Guang Chen, Xiangyu Ning, Yang Zi, Kejia Ren, Zhenshan Bing, and Alois Knoll. 2019. Neuromorphic Vision Datasets for Pedestrian Detection, Action Recognition, and Fall Detection. Frontiers in Neurorobotics 13 (2019). https://doi.org/10.3389/fnbot.2019.00038

[27]

S-G Miaou, Pei-Hsu Sung, and Chia-Yuan Huang. 2006. A customized human fall detection system using omni-camera images and personal information. In 1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare, 2006. D2H2. IEEE, 39–42.

[28]

Dariusz Mrozek, Anna Koczur, and Bożena Małysiak-Mrozek. 2020. Fall detection in older adults with mobile IoT devices and machine learning in the cloud and on the edge. Information Sciences 537 (2020), 132–147.

[29]

Elias Mueggler, Christian Forster, Nathan Baumli, Guillermo Gallego, and Davide Scaramuzza. 2015. Lifetime estimation of events from dynamic vision sensors. In 2015 IEEE international conference on Robotics and Automation (ICRA). IEEE, 4874–4881.

[30]

Balint Petro, Nikola Kasabov, and Rita M Kiss. 2019. Selection and optimization of temporal spike encoding methods for spiking neural networks. IEEE transactions on neural networks and learning systems 31, 2 (2019), 358–370.

[31]

Shyam Sunder Prasad, Naval Kishore Mehta, Abeer Banerjee, Himanshu Kumar, Sumeet Saurav, and Sanjay Singh. 2022. Real-Time Privacy-Preserving Fall Detection using Dynamic Vision Sensors. In 2022 IEEE 19th India Council International Conference (INDICON). IEEE, 1–6.

[32]

Sumeet Saurav, Ravi Saini, and Sanjay Singh. 2022. A dual-stream fused neural network for fall detection in multi-camera and 360° videos. Neural Computing and Applications 34, 2 (2022), 1455–1482.

Digital Library

[33]

Sumeet Saurav, Ravi Saini, and Sanjay Singh. 2022. Vision-based techniques for fall detection in 360° videos using deep learning: Dataset and baseline results. Multimedia Tools and Applications 81, 10 (2022), 14173–14216.

Digital Library

[34]

Wann-Yun Shieh and Ju-Chin Huang. 2012. Falling-incident detection and throughput enhancement in a multi-camera video-surveillance system. Medical engineering & physics 34, 7 (2012), 954–963.

[35]

Sumit B Shrestha and Garrick Orchard. 2018. Slayer: Spike layer error reassignment in time. Advances in neural information processing systems 31 (2018).

[36]

Martino Sorbaro, Qian Liu, Massimo Bortone, and Sadique Sheik. 2020. Optimizing the energy consumption of spiking neural networks for neuromorphic applications. Frontiers in neuroscience (2020), 662.

[37]

Erik E Stone and Marjorie Skubic. 2014. Fall detection in homes of older adults using the Microsoft Kinect. IEEE journal of biomedical and health informatics 19, 1 (2014), 290–301.

[38]

Guangmin Sun and Zhongqi Wang. 2020. Fall detection algorithm for the elderly based on human posture estimation. In 2020 Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC). IEEE, 172–176.

[39]

Clarence Tan, Marko Šarlija, and Nikola Kasabov. 2020. Spiking neural networks: Background, recent development and the NeuCube architecture. Neural Processing Letters 52, 2 (2020), 1675–1701.

Digital Library

[40]

Huachun Tan, Yang Zhou, Yong Zhu, Danya Yao, and Keqiang Li. 2014. A novel curve lane detection based on Improved River Flow and RANSA. In 17th international ieee conference on intelligent transportation systems (itsc). IEEE, 133–138.

[41]

Shigeyuki Tateno, Fanxing Meng, Renzhong Qian, and Yuriko Hachiya. 2020. Privacy-preserved fall detection method with three-dimensional convolutional neural network using low-resolution infrared array sensor. Sensors 20, 20 (2020), 5957.

[42]

Jixiang Wan, Ming Xia, Zunkai Huang, Li Tian, Xiaoying Zheng, Victor Chang, Yongxin Zhu, and Hui Wang. 2021. Event-Based Pedestrian Detection Using Dynamic Vision Sensors. Electronics 10, 8 (2021), 888.

[43]

Bo-Hua Wang, Jie Yu, Kuo Wang, Xuan-Yu Bao, and Ke-Ming Mao. 2020. Fall detection based on dual-channel feature integration. IEEE Access 8 (2020), 103443–103453.

[44]

Xueyi Wang, Joshua Ellul, and George Azzopardi. 2020. Elderly fall detection systems: A literature survey. Frontiers in Robotics and AI 7 (2020), 71.

[45]

Pei-Chen Wu, Chin-Yu Chang, and Chang Hong Lin. 2014. Lane-mark extraction for automobiles under complex conditions. Pattern Recognition 47, 8 (2014), 2756–2767.

Cited By

Cazzato DBono F(2024)An Application-Driven Survey on Event-Based Neuromorphic Computer VisionInformation10.3390/info1508047215:8(472)Online publication date: 9-Aug-2024
https://doi.org/10.3390/info15080472
Kumar HMehta NSaurav SSingh S(2024)Gaze Estimation via Synthetic Event-Driven Neural Networks2024 39th International Conference on Image and Vision Computing New Zealand (IVCNZ)10.1109/IVCNZ64857.2024.10794202(1-6)Online publication date: 4-Dec-2024
https://doi.org/10.1109/IVCNZ64857.2024.10794202
Banerjee AMehta NPrasad SKumar HSaurav SSingh S(2024)Generalized Gaze-Vector Estimation in Low-light with Encoded Event-driven Neural Network2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650655(1-7)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650655

Index Terms

Hybrid SNN-based Privacy-Preserving Fall Detection using Neuromorphic Sensors

Index terms have been assigned to the content through auto-classification.

Recommendations

Elderly Assistance Using Wearable Sensors by Detecting Fall and Recognizing Fall Patterns
UbiComp '18: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers

Falling is a serious threat to the elderly people. One severe fall can cause hazardous problems like bone fracture or may lead to some permanent disability or even death. Thus, it has become the need of the hour to continuously monitor the activities of ...
Privacy preserving automatic fall detection for elderly using RGBD cameras
ICCHP'12: Proceedings of the 13th international conference on Computers Helping People with Special Needs - Volume Part I

In this paper, we propose a new privacy preserving automatic fall detection method to facilitate the independence of older adults living in the community, reduce risks, and enhance the quality of life at home activities of daily living (ADLs) by using ...
A smartphone-based fall detection system

Falls are a major cause of injuries and hospital admissions among elderly people. Thus, the caregiving process and the quality of life of older adults can be improved by adopting systems for the automatic detection of falls. This paper presents a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICVGIP '23: Proceedings of the Fourteenth Indian Conference on Computer Vision, Graphics and Image Processing

December 2023

352 pages

ISBN:9798400716256

DOI:10.1145/3627631

Copyright © 2023 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 January 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICVGIP '23

ICVGIP '23: Indian Conference on Computer Vision, Graphics and Image Processing

December 15 - 17, 2023

Rupnagar, India

Acceptance Rates

Overall Acceptance Rate 95 of 286 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
60
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)10

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cazzato DBono F(2024)An Application-Driven Survey on Event-Based Neuromorphic Computer VisionInformation10.3390/info1508047215:8(472)Online publication date: 9-Aug-2024
https://doi.org/10.3390/info15080472
Kumar HMehta NSaurav SSingh S(2024)Gaze Estimation via Synthetic Event-Driven Neural Networks2024 39th International Conference on Image and Vision Computing New Zealand (IVCNZ)10.1109/IVCNZ64857.2024.10794202(1-6)Online publication date: 4-Dec-2024
https://doi.org/10.1109/IVCNZ64857.2024.10794202
Banerjee AMehta NPrasad SKumar HSaurav SSingh S(2024)Generalized Gaze-Vector Estimation in Low-light with Encoded Event-driven Neural Network2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650655(1-7)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650655

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents