Abstract
A biologically inspired event camera being able to produce more than 500 pictures per second [1], has been proposed in recent years. Event cameras can achieve profound efficiency in addressing many drawbacks of traditional cameras, for example redundant data and low frame rate during classification. In this paper, we apply a Celex IV DVS camera to fabricate a four-class hand gesture dataset for the first time. Meanwhile, we propose a real-time workflow for reconstructing Celex event data into intensity images while implementing gesture classification with a proposed LeNet-based [2] network and keyframe detection method. More than 30 fps has been achieved with our proposed workflow on a laptop. Compared to the state-of-art work [3] with an accuracy of 99.3%, our proposed network achieves a competent accuracy of 99.75%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, S., Guo, M.: Live demonstration: CeleX-V: a 1m pixel multi-mode event-based sensor. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Lungu, I.-A., Corradi, F., Delbrück, T.: Live demonstration: convolutional neural network driven by dynamic vision sensor playing roshambo. In: 2017 IEEE International Symposium on Circuits and Systems (ISCAS), p. 1. IEEE (2017)
Gallego, G., et al.: Event-based vision: a survey. arXiv preprint arXiv:1904.08405 (2019)
Huang, J., Guo, M., Chen, S.: A dynamic vision sensor with direct logarithmic output and full-frame picture-on-demand. In: 2017 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–4. IEEE (2017)
Amir, A., et al.: A low power, fully event-based gesture recognition system. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7243–7252 (2017)
Baby, S.A., Vinod, B., Chinni, C., Mitra, K.: Dynamic vision sensors for human activity recognition. In: 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), pp. 316–321. IEEE (2017)
Wang, Q., Zhang, Y., Yuan, J., Lu, Y.: Space-time event clouds for gesture recognition: From RGB cameras to event cameras. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1826–1835. IEEE (2019)
Lee, J.H., et al.: Real-time gesture interface based on event-driven processing from stereo silicon retinas. IEEE Trans. Neural Netw. Learn. Syst. 25(12), 2250–2263 (2014)
Lungu, I.A., Liu, S.-C., Delbruck, T.: Fast event-driven incremental learning of hand symbols. In: 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 25–28. IEEE (2019)
Lungu, I.A., Liu, S.-C., Delbruck, T.: Incremental learning of hand symbols using event-based cameras. IEEE J. Emerg. Sel. Top. Circ. Syst. 9(4), 690–696 (2019)
Zong, X., Xiao, P., Wen, S.: An event camera tracking based on MLS surface fitting algorithm. In: 2018 Chinese Control And Decision Conference (CCDC), pp. 5001–5005. IEEE (2018)
Zhu, Q., Triesch, J., Shi, B.E.: Retinal slip estimation and object tracking with an active event camera. In: 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 59–63 (2020)
Glover, A., Bartolozzi, C.: Robust visual tracking with a freely-moving event camera. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3769–3776. IEEE (2017)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: 2011 International Conference on Computer Vision, pp. 2564–2571. IEEE (2011)
Sivilotti, M.: Wiring considerations in analog VLSI systems with application to field-programmable networks. Ph.D. Thesis, California Institute of Technology, Pasadena, CA (1991)
CelePixel: Celepixel technology. [EB/OL]. https://github.com/CelePixel/CeleX4-OpalKelly/tree/master/Documentation Accessed 15 Apr 2020
Moeys, D.P., et al.: Steering a predator robot using a mixed frame/event-driven convolutional neural network. In: 2016 Second International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), pp. 1–8. IEEE (2016)
Pavlovic, V.I., Sharma, R., Huang, T.S.: Visual interpretation of hand gestures for human-computer interaction: a review. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 677–695 (1997)
Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)
Nagarajan, S., Subashini, T.S.: Static hand gesture recognition for sign language alphabets using edge oriented histogram and multi class SVM. Int. J. Comput. Appl. 82(4), 2013
Cheng, W., Sun, Y., Li, G., Jiang, G., Liu, H.: Jointly network: a network based on cnn and rbm for gesture recognition. Neural Comput. Appl. 31(1), 309–323 (2019)
Ye, C.: Learning of Dense Optical Flow, Motion and Depth, from Sparse Event Cameras. Ph.D. thesis (2019)
Zhu, A.Z., Yuan, L., Chaney, K., Daniilidis, K.: EV-FlowNet: self-supervised optical flow estimation for event-based cameras. arXiv preprint arXiv:1802.06898 (2018)
Bardow, P., Davison, A.J., Leutenegger, S.: Simultaneous optical flow and intensity estimation from an event camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 884–892 (2016)
Brosch, T., Tschechne, S., Neumann, H.: On event-based optical flow detection. Front. Neurosci. 9, 137 (2015)
Ahn, E.Y., Lee, J.H., Mullen, T., Yen, J.: Dynamic vision sensor camera based bare hand gesture recognition. In: 2011 IEEE Symposium On Computational Intelligence For Multimedia, Signal And Vision Processing, pp. 52–59. IEEE (2011)
Lee, J.H., et al.: Real-time motion estimation based on event-based vision sensor. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 204–208. IEEE (2014)
Liu, H., Brandli, C., Li, C., Liu, S.-C., Delbruck, T.: Design of a spatiotemporal correlation filter for event-based sensors. In: 2015 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 722–725. IEEE (2015)
Guo, S., et al.: A noise filter for dynamic vision sensors based on global space and time information. arXiv preprint arXiv:2004.04079 (2020)
Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)
Zhang, Y.: Celexmatlabtoolbox. [EB/OL]. https://github.com/yucicheung/CelexMatlabToolbox Accessed 15 Apr 2020
Viswanathan, D.G.: Features from accelerated segment test (FAST). Homepages. Inf. Ed. Ac. UK (2009)
Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_56
Acknowledgments
This work is founded by National Key R&D Program of China [grant numbers 2018YFB2202603].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, X., Wang, J., Zhang, L., Guo, S., Qu, L., Wang, L. (2020). Real-Time Gesture Classification System Based on Dynamic Vision Sensor. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-63830-6_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63829-0
Online ISBN: 978-3-030-63830-6
eBook Packages: Computer ScienceComputer Science (R0)