DVSGesture Recognition with Neuromorphic Observation Space Reduction Techniques
Authors: 
Charles Rizzo, Luke Mccombs, Braxton Haynie, Catherine Schuman, James PlankAuthors Info & Claims
Article No.: 10, Pages 1 - 8
Abstract
Event-based cameras and classification datasets pair nicely with neuromorphic computing. Furthermore, it is attractive from a SWaP perspective to have a fully neuromorphic pipeline from event-based camera output to classification instead of having to preprocess the camera data prior to classification. In this work, we examine how two neuromorphic observation space reduction techniques impact classification performance on the DVSGesture dataset. The two techniques can be implemented as spiking neural networks so that no preprocessing of the camera data is required, and instead, only the routing of the events to the proper input neurons is necessary.
References
[1]
Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. 2015. Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE transactions on computer-aided design of integrated circuits and systems 34, 10 (2015), 1537--1557.
[2]
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, Jeff Kusnitz, Michael Debole, Steve Esser, Tobi Delbruck, Myron Flickner, and Dharmendra Modha. 2017. A Low Power, Fully Event-Based Gesture Recognition System. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[3]
Moez Baccouche, Franck Mamalet, Christian Wolf, Christophe Garcia, and Atilla Baskurt. 2011. Sequential deep learning for human action recognition. In Human Behavior Understanding: Second International Workshop, HBU 2011, Amsterdam, The Netherlands, November 16, 2011. Proceedings 2. Springer, 29--39.
[4]
Luis Camunas-Mesa, Antonio Acosta-Jimenez, Teresa Serrano-Gotarredona, and Bernabe Linares-Barranco. 2005. A digital pixel cell for address event representation image convolution processing. In Bioengineered and Bioinspired Systems II, Vol. 5839. SPIE, 160--171.
[5]
Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. 2018. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro 38, 1 (2018), 82--99.
[6]
S Dey, A Mukherjee, G Bzard, and D McLelland. 2019. Human gesture recognition using spiking input on akida neuromorphic platform. Neural Information Processing Systems (NeurIPS) (2019).
[7]
Steve B Furber, Francesco Galluppi, Steve Temple, and Luis A Plana. 2014. The spinnaker project. Proc. IEEE 102, 5 (2014), 652--665.
[8]
Arun M. George, Dighanchal Banerjee, Sounak Dey, Arijit Mukherjee, and P. Balamurali. 2020. A Reservoir-based Convolutional Spiking Neural Network for Gesture Recognition from DVS Input. In 2020 International Joint Conference on Neural Networks (IJCNN). 1--9.
[9]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.
[10]
Herbert Jaeger and Harald Haas. 2004. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. science 304, 5667 (2004), 78--80.
[11]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436--444.
[12]
Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck. 2006. A 128 x 128 120db 30mw asynchronous vision sensor that responds to relative intensity change. In 2006 IEEE International Solid State Circuits Conference-Digest of Technical Papers. IEEE, 2060--2069.
[13]
Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck. 2008. A 128 × 128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE journal of solid-state circuits 43, 2 (2008), 566--576.
[14]
Wolfgang Maass, Thomas Natschläger, and Henry Markram. 2002. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural computation 14, 11 (2002), 2531--2560.
[15]
Henry Markram, Wulfram Gerstner, and Per Jesper Sjöström. 2012. Spike-timing-dependent plasticity: a comprehensive overview. Frontiers in synaptic neuroscience 4 (2012), 2.
[16]
Priyadarshini Panda and Narayan Srinivasa. 2018. Learning to recognize actions from limited training examples using a recurrent spiking neural model. Frontiers in neuroscience 12 (2018), 126.
[17]
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. the Journal of machine Learning research 12 (2011), 2825--2830.
[18]
Charles P Rizzo, Catherine D Schuman, and James S Plank. 2023. Neuromorphic Downsampling of Event-Based Camera Output. In Neuro-Inspired Computational Elements Conference. 26--34.
[19]
Catherine D. Schuman, James S. Plank, Maryam Parsa, Shruti R. Kulkarni, Nicholas Skuda, and J. Parker Mitchell. 2021. A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems. In 2021 International Joint Conference on Neural Networks (IJCNN). 1--10.
[20]
William Severa, Ojas Parekh, Kristofor D. Carlson, Conrad D. James, and James B. Aimone. 2016. Spiking network algorithms for scientific computing. In 2016 IEEE International Conference on Rebooting Computing (ICRC). 1--8.
[21]
Sumit B Shrestha and Garrick Orchard. 2018. Slayer: Spike layer error reassignment in time. Advances in neural information processing systems 31 (2018).
[22]
Karen Simonyan and Andrew Zisserman. 2014. Two-stream convolutional networks for action recognition in videos. Advances in neural information processing systems 27 (2014).
[23]
Nicholas Soures and Dhireesha Kudithipudi. 2019. Deep liquid state machines with neural plasticity for video activity recognition. Frontiers in neuroscience 13 (2019), 686.
[24]
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision. 4489--4497.
[25]
Yannan Xing, Gaetano Di Caterina, and John Soraghan. 2020. A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition. Frontiers in neuroscience 14 (2020), 590164.
Index Terms
- DVSGesture Recognition with Neuromorphic Observation Space Reduction Techniques
Recommendations
Algorithm and Application Impacts of Programmable Plasticity in Spiking Neuromorphic Hardware
ICONS '23: Proceedings of the 2023 International Conference on Neuromorphic SystemsSynaptic plasticity has long been thought to be key to learning in both biological neural networks and artificial neural networks. There are a wide array of plasticity rules in biological neural systems, but neuromorphic computing has mainly focused ...
Effects of Noise on Leaky Integrate-and-Fire Neuron Models for Neuromorphic Computing Applications
Computational Science and Its Applications – ICCSA 2022AbstractArtificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), ...
Introducing Astrocytes on a Neuromorphic Processor: Synchronization, Local Plasticity and Edge of Chaos
NICE '19: Proceedings of the 7th Annual Neuro-inspired Computational Elements WorkshopWhile there is still a lot to learn about astrocytes and their neuromodulatory role in the spatial and temporal integration of neuronal activity, their introduction to neuromorphic hardware is timely, facilitating their computational exploration in ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
August 2023
270 pages
ISBN:9798400701757
DOI:10.1145/3589737
- Conference Chair:
- Catherine Schuman,
- Program Co-chairs:
- Melika Payvand,
- Maryam Parsa
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 August 2023
Check for updates
Author Tags
Qualifiers
- Short-paper
Funding Sources
Conference
ICONS '23
Sponsor:
Acceptance Rates
Overall Acceptance Rate 13 of 22 submissions, 59%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 76Total Downloads
- Downloads (Last 12 months)51
- Downloads (Last 6 weeks)7
Reflects downloads up to 30 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in