More Web Proxy on the site http://driver.im/

article

Online loop closure for real-time interactive 3D scanning

Authors:

Luc Van GoolAuthors Info & Claims

Computer Vision and Image Understanding, Volume 115, Issue 5

Pages 635 - 648

https://doi.org/10.1016/j.cviu.2010.11.023

Published: 01 May 2011 Publication History

Abstract

We present a real-time interactive 3D scanning system that allows users to scan complete object geometry by turning the object around in front of a real-time 3D range scanner. The incoming 3D surface patches are registered and integrated into an online 3D point cloud. In contrast to previous systems the online reconstructed 3D model also serves as final result. Registration error accumulation which leads to the well-known loop closure problem is addressed already during the scanning session by distorting the object as rigidly as possible. Scanning errors are removed by explicitly handling outliers based on visibility constraints. Thus, no additional post-processing is required which otherwise might lead to artifacts in the model reconstruction. Both geometry and texture are used for registration which allows for a wide range of objects with different geometric and photometric properties to be scanned. We show the results of our modeling approach on several difficult real-world objects. Qualitative and quantitative results are given for both synthetic and real data demonstrating the importance of online loop closure and outlier handling for model reconstruction. We show that our real-time scanning system has comparable accuracy to offline methods with the additional benefit of immediate feedback and results.

References

[1]

Allen, B., Curless, B. and Popović, Z., The space of human body shapes: reconstruction and parameterization from range scans. ACM Transactions on Graphics. v22. 587-594.

[2]

A. Angeli, S. Doncieux, J.A. Meyer, D. Filliat, Real-time visual loop-closure detection, in: IEEE International Conference on Robotics and Automation (ICRA'08), 2008.

[3]

H. Bay, T. Tuytelaars, L. Van Gool, SURF: Speeded up robust features, in: Europeen Conference on Computer Vision (ECCV '06), 2006.

[4]

Bernardini, F. and Rushmeier, H., The 3D model acquisition pipeline. Computer Graphics Forum. v21. 149-172.

[5]

Besl, P.J. and McKay, N.D., A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence. v14. 239-258.

[6]

F. Blais, M. Picard, G. Godin, Accurate 3D acquisition of freely moving objects, in: 3D Data Processing, Visualization and Transmission (3DPVT '04), 2004.

[7]

Botsch, M. and Sorkine, O., On linear variational surface deformation methods. IEEE Transactions on Visualization and Computer Graphics. v14. 213-230.

[8]

B. Büttgen, T. Oggier, M. Lehmann, Ccd/cmos lock-in pixel for range imaging: Challenges, limitations and state-of-the-art, in: 1st range imaging day, 2005.

[9]

Campbell, R.J. and Flynn, P.J., A survey of free-form object representation and recognition techniques. Computer Vision and Image Understanding. v81. 166-210.

[10]

Chen, Y. and Medioni, G., Object modelling by registration of multiple range images. International Journal of Image and Vision Computing. v10. 145-155.

[11]

N. Cornelis, L. Van Gool, Real-time connectivity constrained depth map computation using programmable graphics hardware, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR '05), 2005.

[12]

B. Curless, M. Levoy, A volumetric method for building complex models from range images, in: SIGGRAPH '96: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, 1996.

Digital Library

[13]

J.D. Deschênes, P. Lambert, P. Hebert, Interactive modeling with automatic online compression, in: 3D Data Processing, Visualization, and Transmission (3DPVT'06), 2006.

[14]

N. Gelfand, S. Rusinkiewicz, L. Ikemoto, M. Levoy, Geometrically stable sampling for the ICP algorithm, in: 3-D Digital Imaging and Modeling (3DIM'03), 2003.

[15]

F.F. Gionis, R.B. Fisher, A. Fitzgibbon, A. Gionis, M. Wright, D. Eggert, A hand-held optical surface scanner for environmental modeling and virtual reality, in: Virtual Reality World, 1996.

[16]

G. Godin, M. Rioux, R. Baribeau, Three-dimensional registration using range and intensity information, in: Videometrics III, 1994.

[17]

Guennebaud, G. and Gross, M., Algebraic point set surfaces. ACM Transactions on Graphics. v26. 23

[18]

M. Habbecke, L. Kobbelt, A surface-growing approach to multi-view stereo reconstruction, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07), 2007.

[19]

C. Harris, M. Stephens, A combined corner and edge detection, in: Proceedings of The Fourth Alvey Vision Conference, 1988.

[20]

Hilton, A. and Illingworth, J., Geometric fusion for a hand-held 3d sensor. Machine Vision Applications. v12. 44-51.

[21]

Hilton, A., Stoddart, A.J., Illingworth, J. and Windeatt, T., Implicit surface-based geometric fusion. Computer Vision and Image Understanding. v69. 273-291.

[22]

Horn, B., Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America. v4. 629-642.

[23]

Huber, D. and Hebert, M., Fully automatic registration of multiple 3D data sets. Image and Vision Computing. v21. 637-650.

[24]

Iddan, G. and Yahav, G., 3d imaging in the studio (and elsewhere). SPIE Proceedings Series. v4298. 48-55.

[25]

T. Jaeggli, T. Koninckx, L. Van Gool, Online 3d acquisition and model integration, in: IEEE International Workshop on Projector-Camera Systems (ICCV '03), 2003.

[26]

S.B. Kang, A.E. Johnson, Registration and integration of textured 3-D data, in: 3-D Digital Imaging and Modeling (3DIM '97), 1997.

[27]

H. Kawasaki, R. Furukawa, R. Sagawa, Y. Yagi, Dynamic scene shape reconstruction using a single structured light pattern, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08), 2008.

[28]

M. Kazhdan, M. Bolitho, H. Hoppe, Poisson surface reconstruction, in: Symposium on Geometry Processing (SGP '06), 2006.

[29]

T. Koninckx, T. Jaeggli, L. Van Gool, Adaptive scanning for online 3D model acquisition, in: Workshop on Real-Time 3D Sensors and Their Use, 2004.

[30]

S. Krishnan, P.Y. Lee, J.B. Moore, S. Venkatasubramanian, Global registration of multiple 3D point sets via optimization-on-a-manifold, in: Symposium on Geometry Processing (SGP'05), 2005.

[31]

R. Lange, P. Seitz, A. Biber, R. Schwarte, Time-of-flight range imaging with a custom solid state image sensor, in: Laser Metrology and Inspection, 1999.

[32]

Max, N., Weights for computing vertex normals from facet normals. Journal of Graphics Tools. v4. 1-6.

[33]

Q. Pan, G. Reitmayr, T. Drummond, ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition, in: British Machine Vision Conference (BMVC '09), 2009.

[34]

K. Pulli, Multiview registration for large data sets, in: 3-D Digital Imaging and Modeling (3DIM '99), 1999.

[35]

Rusinkiewicz, S., Hall-Holt, O. and Levoy, M., Real-time 3d model acquisition. ACM Transactions on Graphics. v21. 438-446.

[36]

S. Rusinkiewicz, M. Levoy, Efficient variants of the ICP algorithm, in: 3-D Digital Imaging and Modeling (3DIM'01), 2001.

[37]

C. Schütz, T. Jost, H. Hügli, Multi-feature matching algorithm for free-form 3d surface registration, in: International Conference on Pattern Recognition (ICPR'98), 1998.

[38]

J.K. Seo, G.C. Sharp, S.W. Lee, Range data registration using photometric features, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR'05), 2005.

[39]

O. Sorkine, M. Alexa, As-rigid-as-possible surface modeling, in: Symposium on Geometry Processing (SGP'07), 2007.

[40]

Stanford Scanning Repository, 2009. <http://graphics.stanford.edu/data/3dscanrep/>.

[41]

Sumner, R.W., Schmid, J. and Pauly, M., Embedded deformation for shape manipulation. ACM Transactions on Graphics. v26. 80

[42]

D. Tubic, P. Hébert, J.D. Deschênes, D. Laurendeau, A unified representation for interactive 3D modeling, in: 3D Data Processing, Visualization and Transmission (3DPVT '04), 2004.

[43]

Weickert, J., Anisotropic Diffusion in Image Processing. 1998. Teubner.

[44]

S. Weik, Registration of 3-d partial surface models using luminance and depth information, in: 3-D Digital Imaging and Modeling (3DIM '97), 1997.

[45]

T. Weise, B. Leibe, L. Van Gool, Fast 3d scanning with automatic motion compensation, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07), 2007.

[46]

T. Weise, B. Leibe, L. Van Gool, Accurate and robust registration for in-hand modeling, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08), 2008.

[47]

T. Weise, T. Wismer, B. Leibe, L. Van Gool, In-hand scanning with online loop closure, in: IEEE International Workshop on 3-D Digital Imaging and Modeling (3DIM '09), 2009.

[48]

Z Corporation, <http://www.zcorp.com>, 2010.

[49]

S. Zhang, P. Huang, High-resolution, real-time 3d shape acquisition, in: IEEE Conference on Computer Vision and Pattern Recognition Workshop, 2004.

Cited By

Chen DTang ZLiu YMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Cyclical Fusion: Accurate 3D Reconstruction via Cyclical MonotonicityProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3547931(3955-3964)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3547931
Zhang HZhou YTian YYong JXu F(2021)Single Depth View Based Real-Time Reconstruction of Hand-Object InteractionsACM Transactions on Graphics10.1145/345134140:3(1-12)Online publication date: 15-Jul-2021
https://dl.acm.org/doi/10.1145/3451341
Schöps TSattler TPollefeys M(2020)SurfelMeshing: Online Surfel-Based Mesh ReconstructionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2019.294704842:10(2494-2507)Online publication date: 2-Sep-2020
https://dl.acm.org/doi/10.1109/TPAMI.2019.2947048
Show More Cited By

Online loop closure for real-time interactive 3D scanning
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
  2. Computer graphics
2. Theory of computation

Recommendations

When Specular Object Meets RGB-D Camera 3D Scanning: Color Image Plus Fragmented Depth Map
ISM '14: Proceedings of the 2014 IEEE International Symposium on Multimedia

3D scanning is an important technology since one can apply the scanning results of objects to numerous 3D applications (e.g. Artwork sculpture preserving, 3D animation and 3D printing). However, 3D laser scanners are too expensive to be adopted in daily ...
Stylized rendering of 3D scanned real world environments
NPAR '04: Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering

This paper presents an interactive non-photorealistic rendering (NPR) system that stylizes and renders outdoor scenes captured by 3D laser scanning. In order to deal with the large size, complexity and inherent incompleteness of data obtained from ...
Proactive 3D scanning of inaccessible parts

The evolution of 3D scanning technologies have revolutionized the way real-world object are digitally acquired. Nowadays, high-definition and high-speed scanners can capture even large scale scenes with very high accuracy. Nevertheless, the acquisition ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computer Vision and Image Understanding

Computer Vision and Image Understanding Volume 115, Issue 5

May, 2011

163 pages

ISSN:1077-3142

Issue’s Table of Contents

Copyright © Elsevier Inc. © 2010.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 May 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen DTang ZLiu YMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Cyclical Fusion: Accurate 3D Reconstruction via Cyclical MonotonicityProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3547931(3955-3964)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3547931
Zhang HZhou YTian YYong JXu F(2021)Single Depth View Based Real-Time Reconstruction of Hand-Object InteractionsACM Transactions on Graphics10.1145/345134140:3(1-12)Online publication date: 15-Jul-2021
https://dl.acm.org/doi/10.1145/3451341
Schöps TSattler TPollefeys M(2020)SurfelMeshing: Online Surfel-Based Mesh ReconstructionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2019.294704842:10(2494-2507)Online publication date: 2-Sep-2020
https://dl.acm.org/doi/10.1109/TPAMI.2019.2947048
Zhang HBo ZYong JXu F(2019)InteractionFusionACM Transactions on Graphics10.1145/3306346.332299838:4(1-11)Online publication date: 12-Jul-2019
https://dl.acm.org/doi/10.1145/3306346.3322998
Wang FHauser K(2019)In-hand Object Scanning via RGB-D Video Segmentation2019 International Conference on Robotics and Automation (ICRA)10.1109/ICRA.2019.8794467(3296-3302)Online publication date: 20-May-2019
https://dl.acm.org/doi/10.1109/ICRA.2019.8794467
Della Corte BBogoslavskyi IStachniss CGrisetti G(2018)A General Framework for Flexible Multi-Cue Photometric Point Cloud Registration2018 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA.2018.8461049(1-8)Online publication date: 21-May-2018
https://dl.acm.org/doi/10.1109/ICRA.2018.8461049
Chaudhary KChen XChan WOkada KInaba M(2017)STAIR3D: Simultaneous tracking and incremental registration for modeling 3D handheld objects2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM)10.1109/AIM.2017.8014016(185-192)Online publication date: 3-Jul-2017
https://dl.acm.org/doi/10.1109/AIM.2017.8014016
Prankl JAldoma ASvejda AVincze M(2015)RGB-D object modelling for object recognition and tracking2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS.2015.7353360(96-103)Online publication date: 28-Sep-2015
https://dl.acm.org/doi/10.1109/IROS.2015.7353360
Stückler JBehnke S(2014)Multi-resolution surfel maps for efficient dense 3D modeling and trackingJournal of Visual Communication and Image Representation10.1016/j.jvcir.2013.02.00825:1(137-147)Online publication date: 1-Jan-2014
https://dl.acm.org/doi/10.1016/j.jvcir.2013.02.008
Li HVouga EGudym ALuo LBarron JGusev G(2013)3D self-portraitsACM Transactions on Graphics10.1145/2508363.250840732:6(1-9)Online publication date: 1-Nov-2013
https://dl.acm.org/doi/10.1145/2508363.2508407
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents