Unsupervised optical flow estimation with dynamic timing representation for spike camera
Advances in Neural Information Processing Systems, 2024•proceedings.neurips.cc
Efficiently selecting an appropriate spike stream data length to extract precise information is
the key to the spike vision tasks. To address this issue, we propose a dynamic timing
representation for spike streams. Based on multi-layers architecture, it applies dilated
convolutions on temporal dimension to extract features on multi-temporal scales with few
parameters. And we design layer attention to dynamically fuse these features. Moreover, we
propose an unsupervised learning method for optical flow estimation in a spike-based …
the key to the spike vision tasks. To address this issue, we propose a dynamic timing
representation for spike streams. Based on multi-layers architecture, it applies dilated
convolutions on temporal dimension to extract features on multi-temporal scales with few
parameters. And we design layer attention to dynamically fuse these features. Moreover, we
propose an unsupervised learning method for optical flow estimation in a spike-based …
Abstract
Efficiently selecting an appropriate spike stream data length to extract precise information is the key to the spike vision tasks. To address this issue, we propose a dynamic timing representation for spike streams. Based on multi-layers architecture, it applies dilated convolutions on temporal dimension to extract features on multi-temporal scales with few parameters. And we design layer attention to dynamically fuse these features. Moreover, we propose an unsupervised learning method for optical flow estimation in a spike-based manner to break the dependence on labeled data. In addition, to verify the robustness, we also build a spike-based synthetic validation dataset for extreme scenarios in autonomous driving, denoted as SSES dataset. It consists of various corner cases. Experiments show that our method can predict optical flow from spike streams in different high-speed scenes, including real scenes. For instance, our method achieves and error reduction on PHM dataset compared to the best spike-based work, SCFlow, in and respectively, using the same settings as in previous works. The source code and dataset are available at\href {https://github. com/Bosserhead/USFlow}{https://github. com/Bosserhead/USFlow}.
proceedings.neurips.cc