More Web Proxy on the site http://driver.im/

research-article

Deep learning of curvature features for shape completion

Authors:

Marina Hernández-Bautista,

Francisco Javier MeleroAuthors Info & Claims

Volume 115, Issue C

Pages 204 - 215

https://doi.org/10.1016/j.cag.2023.07.007

Published: 01 October 2023 Publication History

Abstract

The paper presents a novel solution to the issue of incomplete regions in 3D meshes obtained through digitization. Traditional methods for estimating the surface of missing geometry and topology often yield unrealistic outcomes for intricate surfaces. To overcome this limitation, the paper proposes a neural network-based approach that generates points in areas where geometric information is lacking. The method employs 2D inpainting techniques on color images obtained from the original mesh parameterization and curvature values. The network used in this approach can reconstruct the curvature image, which then serves as a reference for generating a polygonal surface that closely resembles the predicted one. The paper’s experiments show that the proposed method effectively fills complex holes in 3D surfaces with a high degree of naturalness and detail. This paper improves the previous work in terms of a more in-depth explanation of the different stages of the approach as well as an extended results section with exhaustive experiments.

Graphical abstract

Display Omitted

Highlights

•

We perform 3D surface reconstructions using generative inpainting techniques.

•

2D representation of a 3D surface geometry based on its curvature.

•

Application of a general purpose neural network for inpainting.

•

Our approach does not require dataset nor training time.

•

Results outperform state-of-the-art quality and naturalness of the reconstructions.

References

[1]

Davis J., Marschner S.R., Garr M., Levoy M., Filling holes in complex surfaces using volumetric diffusion, in: Proceedings. First international symposium on 3d data processing visualization and transmission, IEEE, 2002, pp. 428–441.

[2]

Park S., Guo X., Shin H., Qin H., Surface completion for shape and appearance, Vis Comput 22 (2006) 168–180.

[3]

Guo T.Q., Li J.J., Weng J.G., Zhuang Y.T., Filling holes in complex surfaces using oriented voxel diffusion, in: 2006 international conference on machine learning and cybernetics, IEEE, 2006, pp. 4370–4375.

[4]

Liepa P. Filling holes in meshes. In: Proceedings of the 2003 eurographics/ACM SIGGRAPH symposium on geometry processing. 2003, p. 200–5.

[5]

Attene M., A lightweight approach to repairing digitized polygon meshes, Vis Comput 26 (2010) 1393–1406.

Digital Library

[6]

Brunton A., Wuhrer S., Shu C., Bose P., Demaine E.D., Filling holes in triangular meshes by curve unfolding, in: 2009 IEEE international conference on shape modeling and applications, IEEE, 2009, pp. 66–72.

[7]

Yuan W., Khot T., Held D., Mertz C., Hebert M., Pcn: Point completion network, in: 2018 international conference on 3D vision, IEEE, 2018, pp. 728–737.

[8]

Chu L., Pan H., Wang W., Unsupervised shape completion via deep prior in the neural tangent kernel perspective, ACM Trans Graph 40 (3) (2021) 1–17.

[9]

Stutz D, Geiger A. Learning 3D shape completion from laser scan data with weak supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, p. 1955–64.

[10]

Hanocka R., Metzer G., Giryes R., Cohen-Or D., Point2mesh: A self-prior for deformable meshes, 2020, arXiv preprint arXiv:2005.11084.

[11]

Ramesh A., Dhariwal P., Nichol A., Chu C., Chen M., Hierarchical text-conditional image generation with clip latents, 2022, arXiv preprint arXiv:2204.06125.

[12]

Rombach R., Blattmann A., Lorenz D., Esser P., Ommer B., High-resolution image synthesis with latent diffusion models, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2022, URL https://github.com/CompVis/latent-diffusionhttps://arxiv.org/abs/2112.10752.

[13]

Floater M.S., Mean value coordinates, Comput Aided Geom Design 20 (1) (2003) 19–27.

[14]

Guennebaud G, Gross M. Algebraic point set surfaces. In: ACM Siggraph 2007 papers. 2007, p. 23–es.

[15]

Hernandez-Bautista M, Melero FJ. Deep Learning Surface Completion Based on Curvature Features. In: XXXII Spanish computer graphics conference. 2023.

[16]

Kazhdan M., Hoppe H., Screened poisson surface reconstruction, ACM Trans Graph 32 (3) (2013) 1–13.

Digital Library

[17]

Centin M., Signoroni A., RameshCleaner: Conservative fixing of triangular meshes, The Eurographics Association, 2015.

[18]

Botsch M, Kobbelt L. A remeshing approach to multiresolution modeling. In: Proceedings of the 2004 eurographics/ACM SIGGRAPH symposium on geometry processing. 2004, p. 185–92.

[19]

De Floriani L., Puppo E., An on-line algorithm for constrained delaunay triangulation, CVGIP, Graph Models Image Process 54 (4) (1992) 290–300.

[20]

Qiang H., Shusheng Z., Xiaoliang B., Xin Z., Hole filling based on local surface approximation, in: 2010 international conference on computer application and system modeling, vol. 3, IEEE, 2010, pp. V3–242.

[21]

Fang T.P., Piegl L.A., Delaunay triangulation in three dimensions, IEEE Comput Graph Appl 15 (5) (1995) 62–69.

[22]

Harary G., Tal A., Grinspun E., Context-based coherent surface completion, ACM Trans Graph 33 (1) (2014) 1–12.

Digital Library

[23]

Vichitvejpaisal P., Kanongchaiyos P., Surface completion using Laplacian transform, Eng J 18 (1) (2014) 129–144.

[24]

Park JJ, Florence P, Straub J, Newcombe R, Lovegrove S. Deepsdf: Learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019, p. 165–74.

[25]

Atzmon M, Lipman Y. Sal: Sign agnostic learning of shapes from raw data. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020, p. 2565–74.

[26]

Geiger A., Lenz P., Urtasun R., Are we ready for autonomous driving? the kitti vision benchmark suite, in: 2012 IEEE conference on computer vision and pattern recognition, IEEE, 2012, pp. 3354–3361.

[27]

Chang A.X., Funkhouser T., Guibas L., Hanrahan P., Huang Q., Li Z., et al., Shapenet: An information-rich 3d model repository, 2015, arXiv preprint arXiv:1512.03012.

[28]

Wu J, Zhang C, Zhang X, Zhang Z, Freeman WT, Tenenbaum JB. Learning shape priors for single-view 3d completion and reconstruction. In: Proceedings of the European conference on computer vision. 2018, p. 646–62.

[29]

Dai A, Ruizhongtai Qi C, Nießner M. Shape completion using 3D-encoder-predictor cnns and shape synthesis. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, p. 5868–77.

[30]

Wang X., Xu D., Gu F., 3D model inpainting based on 3D deep convolutional generative adversarial network, IEEE Access 8 (2020) 170355–170363.

[31]

Zhang J, Chen X, Cai Z, Pan L, Zhao H, Yi S, et al. Unsupervised 3D shape completion through gan inversion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021, p. 1768–77.

[32]

Williams F, Schneider T, Silva C, Zorin D, Bruna J, Panozzo D. Deep geometric prior for surface reconstruction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019, p. 10130–9.

[33]

Sarmad M, Lee HJ, Kim YM. Rl-gan-net: A reinforcement learning agent controlled gan network for real-time point cloud shape completion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019, p. 5898–907.

[34]

Efros AA, Freeman WT. Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques. 2001, p. 341–6.

[35]

Liang L., Liu C., Xu Y.Q., Guo B., Shum H.Y., Real-time texture synthesis by patch-based sampling, ACM Trans Graph 20 (3) (2001) 127–150.

[36]

Bertalmio M, Sapiro G, Caselles V, Ballester C. Image inpainting. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques. 2000, p. 417–24.

[37]

Maggiordomo A., Cignoni P., Tarini M., Texture inpainting for photogrammetric models, in: Computer graphics forum, Wiley Online Library, 2023.

[38]

Zeng Y, Lin Z, Lu H, Patel VM. Cr-fill: Generative image inpainting with auxiliary contextual reconstruction. In: Proceedings of the IEEE/CVF international conference on computer vision. 2021, p. 14164–73.

[39]

Yu J, Lin Z, Yang J, Shen X, Lu X, Huang TS. Generative image inpainting with contextual attention. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, p. 5505–14.

[40]

Radford A., Kim J.W., Hallacy C., Ramesh A., Goh G., Agarwal S., et al., Learning transferable visual models from natural language supervision, in: International conference on machine learning, PMLR, 2021, pp. 8748–8763.

[41]

Chao I., Pinkall U., Sanan P., Schröder P., A simple geometric model for elastic deformations, ACM Trans Graph 29 (4) (2010) 1–6.

[42]

Berger M., Levine J.A., Nonato L.G., Taubin G., Silva C.T., A benchmark for surface reconstruction, ACM Trans Graph 32 (2) (2013) 1–17.

Cited By

Jorge J(2023)Note computers & graphics issue 115Computers and Graphics10.1016/j.cag.2023.10.018115:C(A1-A3)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.cag.2023.10.018

Index Terms

Deep learning of curvature features for shape completion
1. Computing methodologies
2. Theory of computation

Index terms have been assigned to the content through auto-classification.

Recommendations

Non-rigid Registration with Reliable Distance Field for Dynamic Shape Completion
3DV '14: Proceedings of the 2014 2nd International Conference on 3D Vision - Volume 01

We propose a non-rigid registration method for completion of dynamic shapes with occlusion. Our method is based on the idea that an occluded region in a certain frame should be visible in another frame and that local regions should be moving rigidly ...
Learning Shape Priors for Single-View 3D Completion And Reconstruction
Computer Vision – ECCV 2018
Abstract
The problem of single-view 3D shape completion or reconstruction is challenging, because among the many possible shapes that explain an observation, most are implausible and do not correspond to natural objects. Recent research in the field has ...
Non-rigid Registration with Reliable Distance Field for Dynamic Shape Completion
3DV '14: Proceedings of the 2014 Second International Conference on 3D Vision - Volume 02

We propose a non-rigid registration method for completion of dynamic shapes with occlusion. Our method is based on the idea that an occluded region in a certain frame should be visible in another frame and that local regions should be moving rigidly ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers and Graphics

Computers and Graphics Volume 115, Issue C

Oct 2023

554 pages

ISSN:0097-8493

Issue’s Table of Contents

The Authors.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 October 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jorge J(2023)Note computers & graphics issue 115Computers and Graphics10.1016/j.cag.2023.10.018115:C(A1-A3)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.cag.2023.10.018

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents