Abstract
Computational models of visual processing often use frame-based image acquisition techniques to process a temporally changing stimulus. This approach is unlike biological mechanisms that are spike-based and independent of individual frames. The neuromorphic Dynamic Vision Sensor (DVS) [Lichtsteiner et al., 2008] provides a stream of independent visual events that indicate local illumination changes, resembling spiking neurons at a retinal level. We introduce a new approach for the modelling of cortical mechanisms of motion detection along the dorsal pathway using this type of representation. Our model combines filters with spatio-temporal tunings also found in visual cortex to yield spatio-temporal and direction specificity. We probe our model with recordings of test stimuli, articulated motion and ego-motion. We show how our approach robustly estimates optic flow and also demonstrate how this output can be used for classification purposes.
Chapter PDF
Similar content being viewed by others
References
Adelson, E.H., Bergen, J.H.: The plenoptic function and the elements of early vision. In: Landy, M., Movshon, J.A. (eds.) Computational Models of Visual Processing, pp. 3–20. MIT Press (1991)
Adelson, E.H., Bergen, J.R.: Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. 2(2), 284–299 (1985)
Bayerl, P., Neumann, H.: Disambiguating visual motion through contextual feedback modulation. Neural Computation 16(10), 2041–2066 (2004)
Benosman, R., Clercq, C., Lagorce, X., Ieng, S.H., Bartolozzi, C.: Event-Based visual flow. IEEE Trans. on Neural Networks and Learning Systems 25(2), 407–417 (2014)
Benosman, R., Ieng, S.-H., Clercq, C., Bartolozzi, C., Srinivasan, M.: Asynchronous frameless event–based optical flow. Neural Networks 27, 32–37 (2012)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
De Valois, R., Cottaris, N.P., Mahon, L.E., Elfar, S.D., Wilson, J.A.: Spatial and temporal receptive fields of geniculate and cortical cells and directional selectivity. Vision Research 40, 3685–3702 (2000)
Delbruck, T.: Fun with asynchronous vision sensors and processing. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part I. LNCS, vol. 7583, pp. 506–515. Springer, Heidelberg (2012)
Drazen, D., Lichtsteiner, P., Häflinger, P., Delbrück, T., Jensen, A.: Toward real-time particle tracking using an event-based dynamic vision sensor. Exp. Fluids 51, 1465–1469 (2011)
Escobar, M.J., Kornprobst, P.: Action recognition via bio–inspired features: The richness of center-surround interaction. Computer Vision and Image Understanding 116(5), 593–605 (2012)
Fu, Z., Delbrück, T., Lichtsteiner, P., Culurciello, E.: An address-event fall detector for assisted living applications. IEEE Trans. on Biomedical Circuits and Systems 2(2), 88–96 (2008)
Hassenstein, B., Reichard, W.: Functional structure of a mechanism of perception of optical movement. In: Proc. 1st Intl. Congress on Cybernetics, pp. 797–801 (1956)
Lichtsteiner, P., Posch, C., Delbrück, T.: A 128×128 120dB 15μs Latency Asychronous Temporal Contrast Vision Sensor. IEEE Journal of Solid–State Circuits 43(2), 566–576 (2008)
Liu, S.C., Delbrück, T.: Neuromorphic Sensory Systems. Current Opinion in Neurobiology 20, 288–295 (2010)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, IJCAI, pp. 674–679 (1981)
Rogister, P., Benosman, R., Ieng, S.H., Posch, C.: Asynchronous event–based binocular stereo matching. IEEE Transactions on Neural Networks 22(11), 1723–1734 (2011)
Ungerleider, L.G., Haxby, J.V.: ‘what’ and ‘where’ in the human brain. Current Opinion in Neurobiology 4(2), 157–165 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Tschechne, S., Sailer, R., Neumann, H. (2014). Bio-Inspired Optic Flow from Event-Based Neuromorphic Sensor Input. In: El Gayar, N., Schwenker, F., Suen, C. (eds) Artificial Neural Networks in Pattern Recognition. ANNPR 2014. Lecture Notes in Computer Science(), vol 8774. Springer, Cham. https://doi.org/10.1007/978-3-319-11656-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-11656-3_16
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11655-6
Online ISBN: 978-3-319-11656-3
eBook Packages: Computer ScienceComputer Science (R0)