More Web Proxy on the site http://driver.im/

article

Structure guided fusion for depth map inpainting

Authors:

Fu LiAuthors Info & Claims

Pattern Recognition Letters, Volume 34, Issue 1

Pages 70 - 76

https://doi.org/10.1016/j.patrec.2012.06.003

Published: 01 January 2013 Publication History

Abstract

Depth acquisition becomes inexpensive after the revolutionary invention of Kinect. For computer vision applications, depth maps captured by Kinect require additional processing to fill up missing parts. However, conventional inpainting methods for color images cannot be applied directly to depth maps as there are not enough cues to make accurate inference about scene structures. In this paper, we propose a novel fusion based inpainting method to improve depth maps. The proposed fusion strategy integrates conventional inpainting with the recently developed non-local filtering scheme. The good balance between depth and color information guarantees an accurate inpainting result. Experimental results show the mean absolute error of the proposed method is about 20mm, which is comparable to the precision of the Kinect sensor.

References

[1]

Image inpainting. ACM Press, New York, NY, USA.

[2]

Simultaneous structure and texture image inpainting. IEEE Trans. Image Process. v12 i8. 882-889.

[3]

Buades, A., Coll, B., Morel, J., 2005. A non-local algorithm for image denoising. In: IEEE Comp. Soc. Conf. Computer Vision and Pattern Recognition, CVPR, vol. 2. pp. 60-65.

Digital Library

[4]

Nonlocal image and movie denoising. Internat. J. Comput. Vision. v76 i2. 123-139.

[5]

A comprehensive framework for image inpainting. IEEE Trans. Image Process. v19 i10. 2634-2645.

[6]

Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 1019-1043.

[7]

Chiu, W., Blanke, U., Fritz, M., 2011. Improving the kinect by Cross-Modal stereo. In: British Machine Vision Conf. BMVA, pp. 116.1-116.10.

[8]

Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. v13 i9. 1200-1212.

[9]

Multiple View Geometry in Computer Vision. Cambridge Univ Press.

[10]

Iddan, G.J., Yahav, G., 2001. Three-dimensional imaging in the studio and elsewhere. In: Three-Dimensional Image Capture and Applications IV, Vol. 4298 of Proc., SPIE. pp. 48-55.

[11]

Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors. v12 i2. 1437-1454.

[12]

Kim, S., Cho, J., Koschan, A., Abidi, M.A., 2010. Spatial and temporal enhancement of depth images captured by a Time-of-Flight depth sensor. In: 20th Int. Conf. Pattern Recognit., ICPR. pp. 2358-2361.

[13]

Matyunin, S., Vatolin, D., Berdnikov, Y., Smirnov, M., 2011. Temporal filtering for depth maps generated by kinect depth camera. In: 3DTV Conf.: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON). p. 14.

[14]

Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenom. v60 i1-4. 259-268.

[15]

Scharstein, D., Szeliski, R., 2003. High-accuracy stereo depth maps using structured light. In: IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, CVPR, vol. 1, pp. 195-202.

[16]

Principal neighborhood dictionaries for nonlocal means image denoising. IEEE Trans. Image Process. v18 i12. 2649-2660.

[17]

An image inpainting technique based on the fast marching method. J. Graphics Tools. v9 i1. 23-34.

[18]

An improved algorithm for TVL1 optical flow. In: Cremers, D., Rosenhahn, B., Yuille, A., Schmidt, F. (Eds.), Statistical and Geometrical Approaches to Visual Motion Analysis, Springer. pp. 23-45.

[19]

Fast acquisition of dense depth data by a new structured light scheme. Comput. Vision Image Understanding. v98 i3. 398-422.

[20]

A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Machine Intell. v22 i11. 1330-1334.

[21]

Zhu, J., Wang, L., Yang, R., Davis, J., 2008. Fusion of time-of-flight depth and stereo for high accuracy depth maps. In: IEEE Conf. on Computer Vision and Pattern Recognition, CVPR, pp. 1-8.

Cited By

Medhi MRanjan Sahay R(2025)Adversarial learning for unguided single depth map completion of indoor scenesMachine Vision and Applications10.1007/s00138-024-01652-x36:2Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1007/s00138-024-01652-x
Zhang SYu TChoi BOuyang FDing Z(2024)Radiomap Inpainting for Restricted Areas Based on Propagation Priority and Depth MapIEEE Transactions on Wireless Communications10.1109/TWC.2024.336087223:8_Part_2(9330-9344)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TWC.2024.3360872
Castellano RTerreran MGhidoni S(2023)Performance Evaluation of Depth Completion Neural Networks for Various RGB-D Camera Technologies in Indoor ScenariosAIxIA 2023 – Advances in Artificial Intelligence10.1007/978-3-031-47546-7_24(351-364)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1007/978-3-031-47546-7_24
Show More Cited By

Structure guided fusion for depth map inpainting
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Computer graphics

Recommendations

Kinect Depth Map Inpainting using a Multi-Scale Deep Convolutional Neural Network
ICIGP '18: Proceedings of the 2018 International Conference on Image and Graphics Processing

Consumer level RGB-D camera, such as Kinect, is one of the most important devices to get depth data of 3D vision. However, it is quite difficult to get a high-quality depth map which has the same resolution with its corresponding color image and can be ...
Depth map inpainting via sparse distortion model

The depth map captured from a real scene by the Kinect motion sensor is always influenced by noise and other environmental factors. As a result, some depth information is missing from the map. This distortion of the depth map directly deteriorates the ...
Depth map enhancement based on color and depth consistency

Current low-cost depth sensing techniques, such as Microsoft Kinect, still can achieve only limited precision. The resultant depth maps are often found to be noisy, misaligned with the color images, and even contain many large holes. These limitations ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Pattern Recognition Letters

Pattern Recognition Letters Volume 34, Issue 1

January, 2013

107 pages

ISSN:0167-8655

Issue’s Table of Contents

Copyright © Elsevier B.V. © 2012.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 January 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Medhi MRanjan Sahay R(2025)Adversarial learning for unguided single depth map completion of indoor scenesMachine Vision and Applications10.1007/s00138-024-01652-x36:2Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1007/s00138-024-01652-x
Zhang SYu TChoi BOuyang FDing Z(2024)Radiomap Inpainting for Restricted Areas Based on Propagation Priority and Depth MapIEEE Transactions on Wireless Communications10.1109/TWC.2024.336087223:8_Part_2(9330-9344)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TWC.2024.3360872
Castellano RTerreran MGhidoni S(2023)Performance Evaluation of Depth Completion Neural Networks for Various RGB-D Camera Technologies in Indoor ScenariosAIxIA 2023 – Advances in Artificial Intelligence10.1007/978-3-031-47546-7_24(351-364)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1007/978-3-031-47546-7_24
Wu YLi TShen ILin HChuang Y(2021)Multi-resolution shared representative filtering for real-time depth completionProceedings of the Conference on High-Performance Graphics10.2312/hpg.20211280(11-21)Online publication date: 6-Jul-2021
https://dl.acm.org/doi/10.2312/hpg.20211280
Shen VSpann JHarrison C(2021)FarOut Touch: Extending the Range of ad hoc Touch Sensing with Depth CamerasProceedings of the 2021 ACM Symposium on Spatial User Interaction10.1145/3485279.3485281(1-12)Online publication date: 9-Nov-2021
https://dl.acm.org/doi/10.1145/3485279.3485281
Zhang GLiu JTian LChen Y(2021)Reliably detecting humans with RGB-D camera with physical blob detector followed by learning-based filtering2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2016.7472028(2004-2008)Online publication date: 11-Mar-2021
https://dl.acm.org/doi/10.1109/ICASSP.2016.7472028
Li TNguyen NZhang XWang TSheng B(2020)PROMAR: Practical Reference Object-based Multi-user Augmented RealityIEEE INFOCOM 2020 - IEEE Conference on Computer Communications10.1109/INFOCOM41043.2020.9155506(1359-1368)Online publication date: 6-Jul-2020
https://dl.acm.org/doi/10.1109/INFOCOM41043.2020.9155506
Dourado APedrino E(2020)Multi-objective Cartesian Genetic Programming optimization of morphological filters in navigation systems for Visually Impaired PeopleApplied Soft Computing10.1016/j.asoc.2020.10613089:COnline publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1016/j.asoc.2020.106130
Atapour-Abarghouei AAkcay SPayen de La Garanderie GBreckon T(2019)Generative adversarial framework for depth filling via Wasserstein metric, cosine transform and domain transferPattern Recognition10.1016/j.patcog.2019.02.01091:C(232-244)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1016/j.patcog.2019.02.010
Tian YMa YQuan SXu Y(2019)Occlusion and Collision Aware Smartphone AR Using Time-of-Flight CameraAdvances in Visual Computing10.1007/978-3-030-33723-0_12(141-153)Online publication date: 7-Oct-2019
https://dl.acm.org/doi/10.1007/978-3-030-33723-0_12
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents