[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3472749.3474791acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

SimpModeling: Sketching Implicit Field to Guide Mesh Modeling for 3D Animalmorphic Head Design

Published: 12 October 2021 Publication History

Abstract

Head shapes play an important role in 3D character design. In this work, we propose SimpModeling, a novel sketch-based system for helping users, especially amateur users, easily model 3D animalmorphic heads - a prevalent kind of heads in character design. Although sketching provides an easy way to depict desired shapes, it is challenging to infer dense geometric information from sparse line drawings. Recently, deepnet-based approaches have been taken to address this challenge and try to produce rich geometric details from very few strokes. However, while such methods reduce users’ workload, they would cause less controllability of target shapes. This is mainly due to the uncertainty of the neural prediction. Our system tackles this issue and provides good controllability from three aspects: 1) we separate coarse shape design and geometric detail specification into two stages and respectively provide different sketching means; 2) in coarse shape designing, sketches are used for both shape inference and geometric constraints to determine global geometry, and in geometric detail crafting, sketches are used for carving surface details; 3) in both stages, we use the advanced implicit-based shape inference methods, which have strong ability to handle the domain gap between freehand sketches and synthetic ones used for training. Experimental results confirm the effectiveness of our method and the usability of our interactive system. We also contribute to a dataset of high-quality 3D animal heads, which are manually created by artists.

Supplementary Material

VTT File (p854-talk.vtt)
VTT File (p854-video_figure.vtt)
VTT File (p854-video_preview.vtt)
Supplemental files (p854-supplement_material.zip)
MP4 File (p854-video_preview.mp4)
Video preview and captions
MP4 File (p854-talk.mp4)
Talk video and captions
MP4 File (p854-video_figure.mp4)
Video figure and captions

References

[1]
Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models. In Proceedings of the 21st annual ACM symposium on User interface software and technology. 151–160.
[2]
Adrien Bernhardt, Adeline Pihuit, Marie-Paule Cani, and Loïc Barthe. 2008. Matisse: Painting 2D regions for modeling free-form shapes. In SBM’08-Eurographics Workshop on Sketch-Based Interfaces and Modeling. Eurographics Association, 57–64.
[3]
L Paul Chew. 1989. Constrained delaunay triangulations. Algorithmica 4, 1 (1989), 97–108.
[4]
Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll. 2020. Implicit functions in feature space for 3d shape reconstruction and completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6970–6981.
[5]
Doug DeCarlo, Adam Finkelstein, Szymon Rusinkiewicz, and Anthony Santella. 2003. Suggestive contours for conveying shape. In ACM SIGGRAPH 2003 Papers. 848–855.
[6]
Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A Efros, and Adrien Bousseau. 2018. 3d sketching using multi-view deep volumetric prediction. Proceedings of the ACM on Computer Graphics and Interactive Techniques 1, 1(2018), 1–22.
[7]
Chao Ding and Ligang Liu. 2016. A survey of sketch based modeling systems. Frontiers of Computer Science 10, 6 (2016), 985–999.
[8]
Dong Du, Xiaoguang Han, Hongbo Fu, Feiyang Wu, Yizhou Yu, Shuguang Cui, and Ligang Liu. 2020. SAniHead: Sketching Animal-like 3D Character Heads Using a View-surface Collaborative Mesh Generative Network. IEEE Transactions on Visualization and Computer Graphics (2020).
[9]
Dong Du, Heming Zhu, Yinyu Nie, Xiaoguang Han, Shuguang Cui, Yizhou Yu, and Ligang Liu. 2020. Learning Part Generation and Assembly for Sketching Man-Made Objects. In Computer Graphics Forum. Wiley Online Library.
[10]
Andreas Fabri and Sylvain Pion. 2009. CGAL: The computational geometry algorithms library. In Proceedings of the 17th ACM SIGSPATIAL international conference on advances in geographic information systems. 538–539.
[11]
Yotam Gingold, Takeo Igarashi, and Denis Zorin. 2009. Structured annotations for 2D-to-3D modeling. In ACM SIGGRAPH Asia 2009 papers. 1–9.
[12]
Xiaoguang Han, Chang Gao, and Yizhou Yu. 2017. DeepSketch2Face: a deep learning based sketching system for 3D face and caricature modeling. ACM Transactions on graphics (TOG) 36, 4 (2017), 1–12.
[13]
Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, and Radomir Mech. 2016. Shape synthesis from sketches via procedural models and convolutional networks. IEEE transactions on visualization and computer graphics 23, 8(2016), 2003–2013.
[14]
Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. 1999. Teddy: a sketching interface for 3D freeform design. In Computer graphics proceedings, annual conference series. Association for Computing Machinery SIGGRAPH, 409–416.
[15]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134.
[16]
Pushkar Joshi and Nathan A Carr. 2008. Repoussé: Automatic Inflation of 2D Artwork. In SBM. 49–55.
[17]
Olga A Karpenko and John F Hughes. 2006. SmoothSketch: 3D free-form shapes from complex sketches. In ACM SIGGRAPH 2006 Papers. 589–598.
[18]
Ismail Khalid Kazmi, Lihua You, and Jian Jun Zhang. 2014. A survey of sketch based modeling systems. In 2014 11th International Conference on Computer Graphics, Imaging and Visualization. IEEE, 27–36.
[19]
Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J Mitra. 2020. Sketch2cad: Sequential cad modeling by sketching in context. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–14.
[20]
Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2017. BendSketch: modeling freeform surfaces through 2D sketching. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1–14.
[21]
Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2018. Robust flow-guided neural prediction for sketch-based freeform surface modeling. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–12.
[22]
Zhaoliang Lun, Matheus Gadelha, Evangelos Kalogerakis, Subhransu Maji, and Rui Wang. 2017. 3d shape reconstruction from sketches via multi-view convolutional networks. In 2017 International Conference on 3D Vision (3DV). IEEE, 67–77.
[23]
Andrew Nealen, Takeo Igarashi, Olga Sorkine, and Marc Alexa. 2007. FiberMesh: designing freeform surfaces with 3D curves. In ACM SIGGRAPH 2007 papers. 41–es.
[24]
Gen Nishida, Ignacio Garcia-Dorado, Daniel G Aliaga, Bedrich Benes, and Adrien Bousseau. 2016. Interactive sketching of urban procedural models. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–11.
[25]
Luke Olsen, Faramarz Samavati, and Joaquim Jorge. 2011. Naturasketch: Modeling from images and natural sketches. IEEE Computer Graphics and Applications 31, 6 (2011), 24–34.
[26]
Luke Olsen, Faramarz F Samavati, Mario Costa Sousa, and Joaquim A Jorge. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1 (2009), 85–103.
[27]
Hao Pan, Yang Liu, Alla Sheffer, Nicholas Vining, Chang-Jian Li, and Wenping Wang. 2015. Flow aligned surfacing of curve networks. ACM Transactions on Graphics (TOG) 34, 4 (2015), 1–10.
[28]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, 2019. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703(2019).
[29]
Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. 2019. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE International Conference on Computer Vision. 2304–2314.
[30]
Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. 2020. PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 84–93.
[31]
Ryan Schmidt, Azam Khan, Karan Singh, and Gord Kurtenbach. 2009. Analytic drawing of 3D scaffolds. In ACM SIGGRAPH Asia 2009 papers. 1–10.
[32]
Ryan Schmidt, Brian Wyvill, Mario Costa Sousa, and Joaquim A Jorge. 2007. Shapeshop: Sketch-based solid modeling with blobtrees. In ACM SIGGRAPH 2007 courses. 43–es.
[33]
Cloud Shao, Adrien Bousseau, Alla Sheffer, and Karan Singh. 2012. CrossShade: shading concept sketches using cross-section curves. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1–11.
[34]
Dmitriy Smirnov, Mikhail Bessmeltsev, and Justin Solomon. 2019. Deep sketch-based modeling of man-made shapes. arXiv preprint arXiv:1906.12337(2019).
[35]
Olga Sorkine, Daniel Cohen-Or, Yaron Lipman, Marc Alexa, Christian Rössl, and H-P Seidel. 2004. Laplacian surface editing. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing. 175–184.
[36]
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. 2018. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European Conference on Computer Vision (ECCV). 52–67.
[37]
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018. High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
[38]
Max Wardetzky, Miklós Bergou, David Harmon, Denis Zorin, and Eitan Grinspun. 2007. Discrete quadratic curvature energies. Computer Aided Geometric Design 24, 8-9 (2007), 499–518.
[39]
Jiajun Wu, Yifan Wang, Tianfan Xue, Xingyuan Sun, William T Freeman, and Joshua B Tenenbaum. 2017. MarrNet: 3D shape reconstruction via 2.5 D sketches. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 540–550.
[40]
Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014. True2Form: 3D curve networks from 2D sketches via selective regularization. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–13.
[41]
Youyi Zheng, Hongbo Fu, Oscar Kin-Chung Au, and Chiew-Lan Tai. 2010. Bilateral normal filtering for mesh denoising. IEEE Transactions on Visualization and Computer Graphics 17, 10(2010), 1521–1530.
[42]
Silvia Zuffi, Angjoo Kanazawa, David W Jacobs, and Michael J Black. 2017. 3D menagerie: Modeling the 3D shape and pose of animals. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6365–6373.

Cited By

View all
  • (2024)GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of DetailsACM Transactions on Graphics10.1145/368792143:6(1-12)Online publication date: 19-Dec-2024
  • (2024)Get Your Hands Dirty? A Comparative Study of Tool Usage and Perceptual Engagement in Physical and Digital SculptingProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656188(358-373)Online publication date: 23-Jun-2024
  • (2024)InkBrush: A Sketching Tool for 3D Ink PaintingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642128(1-15)Online publication date: 11-May-2024
  • Show More Cited By

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
UIST '21: The 34th Annual ACM Symposium on User Interface Software and Technology
October 2021
1357 pages
ISBN:9781450386357
DOI:10.1145/3472749
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. 3D modeling interface
  2. datasets
  3. implicit fields
  4. neural networks

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

UIST '21

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25
The 38th Annual ACM Symposium on User Interface Software and Technology
September 28 - October 1, 2025
Busan , Republic of Korea

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)95
  • Downloads (Last 6 weeks)5
Reflects downloads up to 19 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of DetailsACM Transactions on Graphics10.1145/368792143:6(1-12)Online publication date: 19-Dec-2024
  • (2024)Get Your Hands Dirty? A Comparative Study of Tool Usage and Perceptual Engagement in Physical and Digital SculptingProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656188(358-373)Online publication date: 23-Jun-2024
  • (2024)InkBrush: A Sketching Tool for 3D Ink PaintingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642128(1-15)Online publication date: 11-May-2024
  • (2024)DeepTreeSketch: Neural Graph Prediction for Faithful 3D Tree Modeling from SketchesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642125(1-19)Online publication date: 11-May-2024
  • (2024)VRTree: Example‐Based 3D Interactive Tree Modeling in Virtual RealityComputer Graphics Forum10.1111/cgf.1525443:7Online publication date: 7-Nov-2024
  • (2024) SketchMetaFace: A Learning-Based Sketching Interface for High-Fidelity 3D Character Face Modeling IEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329170330:8(5260-5275)Online publication date: Aug-2024
  • (2024)CrossKeys: Text Entry for Virtual Reality Using a Single Controller via Wrist RotationInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2358456(1-13)Online publication date: 5-Jun-2024
  • (2023)Learning Assisted Interactive 3D modelling from 3D sketchesSIGGRAPH Asia 2023 Doctoral Consortium10.1145/3623053.3623370(1-4)Online publication date: 28-Nov-2023
  • (2023)SketchFaceNeRF: Sketch-based Facial Generation and Editing in Neural Radiance FieldsACM Transactions on Graphics10.1145/359210042:4(1-17)Online publication date: 26-Jul-2023
  • (2023)InkBrush: A Flexible and Controllable Authoring Tool for 3D Ink PaintingAdjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586182.3615782(1-3)Online publication date: 29-Oct-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media