Abstract
RGBD scene flow has attracted increasing attention in the computer vision community with the popularity of depth sensor. To accurately estimate three-dimensional motion of object, a layered scene flow estimation with global non-rigid, local rigid motion assumption is presented in this paper. Firstly, depth image is inpainted based on RGB image due to original depth image contains noises. Secondly, depth image is layered according to K-means clustering algorithm, which can quickly and simply layer the depth image. Thirdly, scene flow is estimated based on the assumption we proposed. Finally, experiments are implemented on RGBD tracking dataset and deformable 3D reconstruction dataset, and the analysis of quantitative indicators, RMS (Root Mean Square error) and AAE (Average Angular Error). The results show that the proposed method can distinguish moving regions from the static background better, and more accurately estimate the motion information of the scene by comparing with the global rigid, local non-rigid assumption.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. In: The 17th IEEE International Conference on Computer Vision, 1999, vol. 2, pp. 722–729. IEEE (1999)
Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: The 11th International Conference on Computer Vision, 2007, pp. 1–7. IEEE (2007)
Vogel, C., Schindler, K., Roth, S.: Piecewise rigid scene flow. In: 2013 IEEE International Conference on Computer Vision, 2013, pp. 1377–1384. IEEE (2013)
Quiroga, J., Devernay, F., Crowley, J.: Scene flow by tracking in intensity and depth data. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2012, pp. 50–57. IEEE (2012)
Hadfield, S., Bowden, R.: Kinecting the dots: particle based scene flow from depth sensors. In: IEEE International Conference on Computer Vision, 2012, pp. 2290–2295. IEEE (2012)
Gottfried, J.-M., Fehr, J., Garbe, C.S.: Computing range flow from multi-modal Kinect Data. In: Bebis, G., et al. (eds.) ISVC 2011. LNCS, vol. 6938, pp. 758–767. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24028-7_70
Xiao, D., Yang, Q., Yang, B., Wei, W.: Monocular scene flow estimation via variational method. Multimedia Tools Appl. 6(8), 10575–10597 (2017)
Sun, D., Sudderth, E., Pfister, H.: Layered RGBD scene flow estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 548–556. IEEE (2015)
Hadfield, S., Bowden, R.: Scene particles: unregularized particle-based scene flow estimation. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 564–576 (2014)
Xiang, X., Zhai, M., Zhang, R., Xu, W., El Saddik, A.: Scene flow estimation based on 3D local rigidity assumption and depth map driven anisotropic smoothness. IEEE Access 6, 30012–30023 (2018)
Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88682-2_56
Schuster, R., Wasenmuller, O., Kuschk, G., Bailer, C., Stricker, D.: SceneFlowFields: dense interpolation of sparse scene flow correspondences. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 2018, pp. 1056–1065. IEEE (2016)
Ren, Z., Sun, D., Kautz, J., Sudderth, E.: Cascaded scene flow prediction using semantic segmentation. In: 2017 International Conference on 3D Vision (3DV), 2017, pp. 225–233. IEEE (2017)
Sun, D., Sudderth, E., Black, M.: Layered image motion with explicit occlusions, temporal consistency, and depth ordering. In: Advances in Neural Information Processing Systems (NIPS), 2010, pp. 2226–2234 (2016)
Sun, D., Wulff, J., Sudderth, E., Pfister, H., Black, M.: A fully-connected layered model of foreground and background flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 2451–2458. IEEE (2013)
Zhang, Y., Dai, J., Zhang, H., Yang, L.: Depth inpainting algorithm of RGB-D camera combined with color image. In: 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), 2018, pp. 1391–1395. IEEE (2018)
Slavcheva, M., Baust, M., Cremers, D., Ilic, S.: KillingFusion: non-rigid 3D reconstruction without correspondences. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5474–5483 (2017)
Yan, Y., et al.: Cognitive fusion of thermal and visible imagery for effective detection and tracking of pedestrians in videos. Cogn. Comput. 10(1), 94–104 (2018)
Ren, J.: Fusion of intensity and inter-component chromatic difference for effective and robust colour edge detection. IET Image Process. 4(4), 294–301 (2010)
Feng, Y., et al.: Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications. IEEE Trans. Broadcast. 57(2), 500–509 (2011)
Ren, J., et al.: High-accuracy sub-pixel motion estimation from noisy images in Fourier domain. IEEE Trans. Image Process. 19(5), 1379–1384 (2009)
Feng, W., Huang, W., Ren, J.: Class imbalance ensemble learning based on margin theory. Appl. Sci. 8(5), 815 (2018)
Sun, G., Ma, P., et al.: A stability constrained adaptive alpha for gravitational search algorithm. Knowl.-Based Syst. 139, 200–213 (2018)
Wang, Z., et al.: A deep-learning based feature hybrid framework for spatiotemporal saliency detection inside videos. Neurocomputing 287, 68–83 (2018)
Han, J., et al.: Background prior-based salient object detection via deep reconstruction residual. IEEE Trans. Circuits Syst. Video Technol. 25(8), 1309–1321 (2014)
Acknowledgment
This work has been supported by the National Natural Science Foundation of China under grant Nos. 6150238, 61501370 and 61703333. Thanks Deqing Sun et al. for providing the code.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, X., Liu, Y., Jin, H., Cai, L., Zheng, J. (2020). Layered RGBD Scene Flow Estimation with Global Non-rigid Local Rigid Assumption. In: Ren, J., et al. Advances in Brain Inspired Cognitive Systems. BICS 2019. Lecture Notes in Computer Science(), vol 11691. Springer, Cham. https://doi.org/10.1007/978-3-030-39431-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-39431-8_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39430-1
Online ISBN: 978-3-030-39431-8
eBook Packages: Computer ScienceComputer Science (R0)