More Web Proxy on the site http://driver.im/

research-article

Interactive mechanism modeling from multi-view images

Authors:

Kun ZhouAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 35, Issue 6

Article No.: 236, Pages 1 - 13

https://doi.org/10.1145/2980179.2982425

Published: 05 December 2016 Publication History

Abstract

In this paper, we present an interactive system for mechanism modeling from multi-view images. Its key feature is that the generated 3D mechanism models contain not only geometric shapes but also internal motion structures: they can be directly animated through kinematic simulation. Our system consists of two steps: interactive 3D modeling and stochastic motion parameter estimation. At the 3D modeling step, our system is designed to integrate the sparse 3D points reconstructed from multi-view images and a sketching interface to achieve accurate 3D modeling of a mechanism. To recover the motion parameters, we record a video clip of the mechanism motion and adopt stochastic optimization to recover its motion parameters by edge matching. Experimental results show that our system can achieve the 3D modeling of a range of mechanisms from simple mechanical toys to complex mechanism objects.

Supplementary Material

ZIP File (a236-xu.zip)

Supplemental file.

Download
35.30 MB

References

[1]

Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., and Szeliski, R. 2011. Building rome in a day. Commun. ACM 54, 10 (Oct.), 105--112.

Digital Library

[2]

Ahmadabadian, A. H., Robson, S., Boehm, J., Shortis, M., Wenzel, K., and Fritsch, D. 2013. A comparison of dense matching algorithms for scaled surface reconstruction using stereo camera rigs. ISPRS Journal of Photogrammetry and Remote Sensing 78, 0, 157 -- 167.

[3]

Bächer, M., Bickel, B., James, D. L., and Pfister, H. 2012. Fabricating articulated characters from skinned meshes. ACM Trans. Graph. 31, 4 (July), 47:1--47:9.

Digital Library

[4]

Bey, A., Chaine, R., and Raphael, M. 2011. Reconstruction of consistent 3d cad models from point cloud data using a priori cad models. In Proceedings of ISPRS, vol. 12.

[5]

Canny, J. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 6, 679--698.

Digital Library

[6]

Cao, Y.-P., Ju, T., Fu, Z., and Hu, S.-M. 2014. Interactive image-guided modeling of extruded shapes. Comput. Graph. Forum 33, 7, 101--110.

Digital Library

[7]

Ceylan, D., Li, W., Mitra, N. J., Agrawala, M., and Pauly, M. 2013. Designing and fabricating mechanical automata from mocap sequences. ACM Trans. Graph. 32, 6, 186.

Digital Library

[8]

Chen, X., Kang, S. B., Xu, Y.-Q., Dorsey, J., and Shum, H.-Y. 2008. Sketching reality: Realistic interpretation of architectural designs. ACM Trans. Graph. 27, 2 (May), 11:1--11:15.

Digital Library

[9]

Chen, T., Zhu, Z., Shamir, A., Hu, S.-M., and Cohen-Or, D. 2013. 3-sweep: extracting editable objects from a single photo. ACM Trans. Graph. 32, 6, 195.

Digital Library

[10]

Chin, R. T., and Dyer, C. R. 1986. Model-based recognition in robot vision. ACM Comput. Surv. 18, 1, 67--108.

Digital Library

[11]

Coros, S., Thomaszewski, B., Noris, G., Sueda, S., Forberg, M., Sumner, R. W., Matusik, W., and Bickel, B. 2013. Computational design of mechanical characters. ACM Trans. Graph. 32, 4 (July), 83:1--83:12.

Digital Library

[12]

Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. 31, 4 (July), 44:1--44:10.

Digital Library

[13]

Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM Trans. Graph. 31, 4 (July), 31:1--31:10.

Digital Library

[14]

Furukawa, Y., and Ponce, J. 2007. Accurate, dense, and robust multi-view stereopsis. In CVPR '07, 1--8.

[15]

Gal, R., Sorkine, O., Mitra, N. J., and Cohen-Or, D. 2009. iwires: An analyze-and-edit approach to shape manipulation. ACM Trans. Graph. 28, 3 (July), 33:1--33:10.

Digital Library

[16]

Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, second ed. Cambridge University Press, ISBN: 0521540518.

Digital Library

[17]

Huang, J., and Menq, C.-H. 2002. Automatic cad model reconstruction from multiple point clouds for reverse engineering. J. Comput. Inf. Sci. Eng. 2, 3, 160--170.

[18]

Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: A sketching interface for 3d freeform design. In Proc. of SIGGRAPH'99, 409--416.

Digital Library

[19]

Jacquet, B., Angst, R., and Pollefeys, M. 2013. Articulated and restricted motion subspaces and their signatures. In CVPR, IEEE Computer Society, 1506--1513.

Digital Library

[20]

Jorge, N., and Stephen, W. 2006. Numerical Optimization. Springer-Verlag New York.

[21]

Kalogerakis, E., Chaudhuri, S., Koller, D., and Koltun, V. 2012. A probabilistic model for component-based shape synthesis. ACM Trans. Graph. 31, 4 (July), 55:1--55:11.

Digital Library

[22]

Kraevoy, V., Sheffer, A., Shamir, A., and Cohen-Or, D. 2008. Non-homogeneous resizing of complex models. ACM Trans. Graph. 27, 5 (Dec.), 111:1--111:9.

Digital Library

[23]

Kragic, D., and Christensen, H. I. 2003. Confluence of parameters in model based tracking. In ICRA, IEEE, 3485--3490.

[24]

Li, W., Agrawala, M., Curless, B., and Salesin, D. 2008. Automated generation of interactive 3d exploded view diagrams. ACM Trans. Graph. 27, 3.

Digital Library

[25]

Li, Y., Wu, X., Chrysathou, Y., Sharf, A., Cohen-Or, D., and Mitra, N. J. 2011. Globfit: Consistently fitting primitives by discovering global relations. ACM Trans. Graph. 30, 4 (July), 52:1--52:12.

Digital Library

[26]

Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91--110.

Digital Library

[27]

Mitra, N., Wand, M., Zhang, H. R., Cohen-Or, D., Kim, V., and Huang, Q.-X. 2013. Structure-aware shape processing. In SIGGRAPH Asia 2013 Courses, SA '13, 1:1--1:20.

Digital Library

[28]

Mitra, N. J., Yang, Y.-L., Yan, D.-M., Li, W., and Agrawala, M. 2013. Illustrating how mechanical assemblies work. Commun. ACM 56, 1 (Jan.), 106--114.

Digital Library

[29]

Olsen, L., Samavati, F. F., Sousa, M. C., and Jorge, J. A. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1, 85--103.

Digital Library

[30]

Owada, S., Nielsen, F., Okabe, M., and Igarashi, T. 2004. Volumetric illustration: Designing 3d models with internal textures. ACM Trans. Graph. 23, 3 (Aug.), 322--328.

Digital Library

[31]

Owada, S., Nielsen, F., Nakazawa, K., and Igarashi, T. 2007. A sketching interface for modeling the internal structures of 3d shapes. In ACM SIGGRAPH 2007 Courses, SIGGRAPH '07.

Digital Library

[32]

Schmidt, R., Wyvill, B., Sousa, M. C., and Jorge, J. A. 2007. Shapeshop: Sketch-based solid modeling with blobtrees. In ACM SIGGRAPH 2007 Courses, SIGGRAPH '07.

Digital Library

[33]

Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR'06, vol. 1, 519--528.

Digital Library

[34]

Shao, T., Li, W., Zhou, K., Xu, W., Guo, B., and Mitra, N. J. 2013. Interpreting concept sketches. ACM Trans. Graph. 32, 4, 56:1--56:10.

Digital Library

[35]

Shen, C.-H., Fu, H., Chen, K., and Hu, S.-M. 2012. Structure recovery by part assembly. ACM Trans. Graph. 31, 6 (Nov.), 180:1--180:11.

Digital Library

[36]

Sinha, S. N., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. 2008. Interactive 3d architectural modeling from unordered photo collections. ACM Trans. Graph. 27, 5 (Dec.), 159:1--159:10.

Digital Library

[37]

Slater, N., 2011. Mechanisms and mechanical devices source-book. McGraw-Hill.

[38]

Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Trans. Graph. 25, 3, 835--846.

Digital Library

[39]

Song, L., Wu, W., Guo, J., and Li, X. 2013. Survey on camera calibration technique. In Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2013 5th International Conference on, vol. 2, 389--392.

Digital Library

[40]

Sun, D., Roth, S., and Black, M. J. 2010. Secrets of optical flow estimation and their principles. In IEEE CVPR, 2432--2439.

[41]

Thomaszewski, B., Coros, S., Gauge, D., Megaro, V., Grinspun, E., and Gross, M. 2014. Computational design of linkage-based characters. ACM Trans. Graph. 33, 4 (July), 64:1--64:9.

Digital Library

[42]

Valgaerts, L., Bruhn, A., Mainberger, M., and Weickert, J. 2012. Dense versus sparse approaches for estimating the fundamental matrix. International Journal of Computer Vision 96, 2, 212--234.

Digital Library

[43]

Wang, Y., Xu, K., Li, J., Zhang, H., Shamir, A., Liu, L., Cheng, Z., and Xiong, Y. 2011. Symmetry hierarchy of man-made objects. Computer Graphics Forum 30, 2, 287--296.

[44]

Wikipedia, 2015. Involute. http://en.wikipedia.org/wiki/Involute/.

[45]

Wu, C., 2015. Visualsfm: A visual structure from motion system. http://ccwu.me/vsfm/.

[46]

Xu, W., Wang, J., Yin, K., Zhou, K., van de Panne, M., Chen, F., and Guo, B. 2009. Joint-aware manipulation of deformable models. ACM Trans. Graph. 28, 3.

Digital Library

[47]

Xu, K., Zheng, H., Zhang, H., Cohen-Or, D., Liu, L., and Xiong, Y. 2011. Photo-inspired model-driven 3d object modeling. ACM Trans. Graph. 30, 4 (July), 80:1--80:10.

Digital Library

[48]

Xu, K., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Fit and diverse: set evolution for inspiring 3d shape galleries. ACM Trans. Graph. 31, 4, 57.

Digital Library

[49]

Xu, B., Chang, W., Sheffer, A., Bousseau, A., McCrae, J., and Singh, K. 2014. True2form: 3d curve networks from 2d sketches via selective regularization. ACM Trans. Graph. 33, 4 (July), 131:1--131:13.

Digital Library

[50]

Ycer, K., Wang, O., Sorkine-Hornung, A., and Sorkine-Hornung, O. 2015. Reconstruction of articulated objects from a moving camera. In IEEE ICCV, 823--831.

Digital Library

[51]

Yin, K., Huang, H., Zhang, H., Gong, M., Cohen-Or, D., and Chen, B. 2014. Morfit: Interactive surface reconstruction from incomplete point clouds with curve-driven topology and geometry control. ACM Transactions on Graphics (Special Issue of SIGGRAPH Asia) 33, 6, 202:1--202:13.

Digital Library

[52]

Zhang, Z. 2000. A flexible new technique for camera calibration. IEEE TPAMI 22, 11, 1330--1334.

Digital Library

[53]

Zheng, Y., Fu, H., Cohen-Or, D., Au, O. K.-C., and Tai, C.-L. 2011. Component-wise controllers for structure-preserving shape manipulation. Comput. Graph. Forum 30, 2, 563--572.

[54]

Zheng, Y., Chen, X., Cheng, M.-M., Zhou, K., Hu, S.-M., and Mitra, N. J. 2012. Interactive images: Cuboid proxies for smart image manipulation. ACM Trans. Graph. 31, 4 (July), 99:1--99:11.

Digital Library

[55]

Zhu, L., Xu, W., Snyder, J., Liu, Y., Wang, G., and Guo, B. 2012. Motion-guided mechanical toy modeling. ACM Trans. Graph. 31, 6 (Nov.), 127:1--127:10.

Digital Library

Cited By

Wayo DIrawan SSatyanaga AKim JBin Mohamad Noor MRasouli V(2024)Filter Cake Neural-Objective Data Modeling and Image OptimizationSymmetry10.3390/sym1608107216:8(1072)Online publication date: 19-Aug-2024
https://doi.org/10.3390/sym16081072
Siddique ACignoni PCorsini MBanterle F(2024)Evaluating Image-Based Interactive 3D Modeling ToolsIEEE Access10.1109/ACCESS.2024.343458412(104138-104152)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3434584
Li DLiu YHuang DXu C(2023)A Semisupervised Soft-Sensor of Just-in-Time Learning With Structure Entropy Clustering and Applications for Industrial Processes MonitoringIEEE Transactions on Artificial Intelligence10.1109/TAI.2022.32170284:4(722-733)Online publication date: Aug-2023
https://doi.org/10.1109/TAI.2022.3217028
Show More Cited By

Index Terms

Interactive mechanism modeling from multi-view images
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling
      1. Parametric curve and surface models
      2. Shape analysis

Recommendations

Structured annotations for 2D-to-3D modeling

We present a system for 3D modeling of free-form surfaces from 2D sketches. Our system frees users to create 2D sketches from arbitrary angles using their preferred tool, which may include pencil and paper. A 3D model is created by placing primitives ...
Structured annotations for 2D-to-3D modeling
SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers

We present a system for 3D modeling of free-form surfaces from 2D sketches. Our system frees users to create 2D sketches from arbitrary angles using their preferred tool, which may include pencil and paper. A 3D model is created by placing primitives ...
Geometry-Aware Reference Synthesis for Multi-View Image Super-Resolution
MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Recent multi-view multimedia applications struggle between high-resolution (HR) visual experience and storage or bandwidth constraints. Therefore, this paper proposes a Multi-View Image Super-Resolution (MVISR) task. It aims to increase the resolution ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 35, Issue 6

November 2016

1045 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2980179

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2016

Published in TOG Volume 35, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

49
Total Citations
View Citations
717
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)2

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wayo DIrawan SSatyanaga AKim JBin Mohamad Noor MRasouli V(2024)Filter Cake Neural-Objective Data Modeling and Image OptimizationSymmetry10.3390/sym1608107216:8(1072)Online publication date: 19-Aug-2024
https://doi.org/10.3390/sym16081072
Siddique ACignoni PCorsini MBanterle F(2024)Evaluating Image-Based Interactive 3D Modeling ToolsIEEE Access10.1109/ACCESS.2024.343458412(104138-104152)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3434584
Li DLiu YHuang DXu C(2023)A Semisupervised Soft-Sensor of Just-in-Time Learning With Structure Entropy Clustering and Applications for Industrial Processes MonitoringIEEE Transactions on Artificial Intelligence10.1109/TAI.2022.32170284:4(722-733)Online publication date: Aug-2023
https://doi.org/10.1109/TAI.2022.3217028
Cheng YSun YSong PLiu L(2021)Spatial-temporal motion control via composite cam-follower mechanismsACM Transactions on Graphics10.1145/3478513.348047740:6(1-15)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480477
Roumen TKommana YApel ILempert CBrand MBrendel ESeidel LRambold LGoedecken CCrenzin PHurdelhey BAbdullah MBaudisch PKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Assembler3: 3D Reconstruction of Laser-Cut ModelsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445453(1-11)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445453
Chen MLau M(2021)A Motion‐guided Interface for Modeling 3D Multi‐functional FurnitureComputer Graphics Forum10.1111/cgf.1441640:7(229-240)Online publication date: 27-Nov-2021
https://doi.org/10.1111/cgf.14416
Noeckel JZhao HCurless BSchulz A(2021)Fabrication‐Aware Reverse Engineering for CarpentryComputer Graphics Forum10.1111/cgf.1437540:5(301-314)Online publication date: 23-Aug-2021
https://doi.org/10.1111/cgf.14375
Xiong GFu QFu HZhou BLuo GDeng Z(2021)Motion Planning for Convertible Indoor Scene Layout DesignIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.300568027:12(4413-4424)Online publication date: 1-Dec-2021
https://doi.org/10.1109/TVCG.2020.3005680
Hao JLuo SPan L(2021)Computer-aided intelligent design using deep multi-objective cooperative optimization algorithmFuture Generation Computer Systems10.1016/j.future.2021.05.014124:C(49-53)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1016/j.future.2021.05.014
Hou RRen GGao WLiu L(2021)Research on cyberspace multi-objective security algorithm and decision mechanism of Energy InternetFuture Generation Computer Systems10.1016/j.future.2021.02.007120(119-124)Online publication date: Jul-2021
https://doi.org/10.1016/j.future.2021.02.007
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents