[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Computational Parquetry: Fabricated Style Transfer with Wood Pixels

Published: 04 February 2020 Publication History

Abstract

Parquetry is the art and craft of decorating a surface with a pattern of differently colored veneers of wood, stone, or other materials. Traditionally, the process of designing and making parquetry has been driven by color, using the texture found in real wood only for stylization or as a decorative effect. Here, we introduce a computational pipeline that draws from the rich natural structure of strongly textured real-world veneers as a source of detail to approximate a target image as faithfully as possible using a manageable number of parts. This challenge is closely related to the established problems of patch-based image synthesis and stylization in some ways, but fundamentally different in others. Most importantly, the limited availability of resources (any piece of wood can only be used once) turns the relatively simple problem of finding the right piece for the target location into the combinatorial problem of finding optimal parts while avoiding resource collisions. We introduce an algorithm that efficiently solves an approximation to the problem. It further addresses challenges like gamut mapping, feature characterization, and the search for fabricable cuts. We demonstrate the effectiveness of the system by fabricating a selection of pieces of parquetry from different kinds of unstained wood veneer.

References

[1]
M. Ashikhmin. 2001. Synthesizing natural textures. In Proceedings of the Symposium on Interactive 3D Graphics. ACM, New York, NY, 217--226.
[2]
C. Barnes, D. B. Goldman, E. Shechtman, and Adam Finkelstein. 2011. The PatchMatch randomized matching algorithm for image manipulation. Commun. ACM 54, 11 (2011), 103--110.
[3]
C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3 (2009), 24:1--24:11.
[4]
C. Barnes, E. Shechtman, D. B. Goldman, and A. Finkelstein. 2010. The generalized patchmatch correspondence algorithm. In Proceedings of the European Conference on Computer Vision Conference on Computer Vision: Part III. Springer-Verlag, Berlin, 29--43.
[5]
C. Barnes and F.-L. Zhang. 2017. A survey of the state-of-the-art in patch-based synthesis. Comput. Visual Media 3, 1 (2017), 3--20.
[6]
S. Battiato, G. Di Blasi, G. M. Farinella, and G. Gallo. 2006. A survey of digital mosaic techniques. In Proceedings of the Eurographics Italian Chapter Conference. The Eurographics Association.
[7]
S. Battiato, G. Di Blasi, G. M. Farinella, and G. Gallo. 2007. Digital mosaic frameworks—An overview. Comput. Graph. Forum 26, 4 (2007), 794--812.
[8]
S. Battiato, A. Milone, and G. Puglisi. 2012. Artificial mosaics with irregular tiles based on gradient vector flow. In Proceedings of the European Conference on Computer Vision (ECCV’12). Springer, Berlin, 581--588.
[9]
B. Bickel, P. Cignoni, L. Malomo, and N. Pietroni. 2018. State of the art on stylized fabrication. Comput. Graph. Forum 37, 6 (2018), 325--342.
[10]
G. Di Blasi and M. Petralia. 2005. Fast photomosaic. In Proceedings of the of World Society for Computer Graphics (WSCG’05).
[11]
G. Bradski. 2000. The OpenCV library. Dr. Dobb’s J. Softw. Tools (2000).
[12]
J. Yau C. Close. 1995. Recent Paintings. Pace Wildenstein.
[13]
John Canny. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 6 (1986), 679--698.
[14]
Z. Chen, B. Kim, D. Ito, and H. Wang. 2015. Wetbrush: GPU-based 3D painting simulation at the bristle level. ACM Trans. Graph. 34, 6 (2015), 200:1--200:11.
[15]
S. Dali. 1991. The Salvador Dali Museum Collection. Bulfinch Press, Boston.
[16]
S. Darabi, E. Shechtman, C. Barnes, D. B. Goldman, and P. Sen. 2012. Image melding: Combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31, 4 (2012), 82:1--82:10.
[17]
M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni. 2004. Locality-sensitive hashing scheme based on P-stable distributions. In Proceedings of the 20th Annual Symposium on Computational Geometry. ACM, New York, NY, 253--262.
[18]
O. Deussen, T. Lindemeier, S. Pirk, and M. Tautzenberger. 2012. Feedback-guided stroke placement for a painting machine. In Proceedings of the Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging. Eurographics Association, Goslar, Germany, 25--33.
[19]
G. Di Blasi, G. Gallo, and M. Petralia. 2005. Puzzle image mosaic. In Proceedings of the IASTED International Conference on Visualization, Imaging, and Image Processing (IASTED/VIIP’05).
[20]
A. A. Efros and W. T. Freeman. 2001. Image quilting for texture synthesis and transfer. Proc. SIGGRAPH (2001), 341--346.
[21]
A. A. Efros and T. K. Leung. 1999. Texture synthesis by non-parametric sampling. In Proceedings of the IEEE International Conference on Computer Vision, Vol. 2. 1033--1038.
[22]
G. Elber and G. Wolberg. 2003. Rendering traditional mosaics. Visual Comput. 19, 1 (2003), 67--78.
[23]
Pedro F. Felzenszwalb and Daniel P. Huttenlocher. 2012. Distance transforms of sampled functions. Theory Comput. 8, 1 (2012), 415--428.
[24]
A. Finkelstein and M. Range. 1998. Image mosaics. In Proceedings of the International Conference on Electronic Publishing. Springer-Verlag, London, UK, 11--22.
[25]
L. A. Gatys, A. S. Ecker, and M. Bethge. 2015. Texture synthesis using convolutional neural networks. In Proceedings of the International Conference on Neural Information Processing Systems. 262--270.
[26]
L. A. Gatys, A. S. Ecker, and M. Bethge. 2016. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 2414--2423.
[27]
Y. J. Gi, Y. S. Park, S. H. Seo, and K. H. Yoon. 2006. Mosaic rendering using colored paper. In Proceedings of the International Conference on Virtual Reality, Archaeology and Intelligent Cultural Heritage (VAST’06). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 25--30.
[28]
J. Han, K. Zhou, L.-Y. Wei, M. Gong, H. Bao, X. Zhang, and B. Guo. 2006. Fast example-based surface texture synthesis via discrete optimization. Vis. Comput. 22, 9 (2006), 918--925.
[29]
A. Hausner. 2001. Simulating decorative mosaics. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, NY, 573--580.
[30]
K. He and J. Sun. 2012. Computing nearest-neighbor fields via propagation-assisted KD-Trees. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 111--118.
[31]
A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin. 2001. Image analogies. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, NY, 327--340.
[32]
A. Jackson, D. Day, and S. Jennings. 1996. The Complete Manual of Woodworking. Knopf.
[33]
N. Jetchev, U. Bergmann, and C. Seward. 2017. GANosaic: Mosaic creation with generative texture manifolds. CoRR abs/1712.00269 (2017).
[34]
Y. Jing, Y. Yang, Z. Feng, J. Ye, and M. Song. 2017. Neural style transfer: A review. CoRR abs/1705.04058 (2017).
[35]
D. Kang, S. Seo, S. Ryoo, and K. Yoon. 2011. A parallel framework for fast photomosaics. IEICE Trans. Info. Syst. 94-D, 10 (2011), 2036--2042.
[36]
A. Kaspar, B. Neubert, D. Lischinski, M. Pauly, and J. Kopf. 2015. Self tuning texture optimization. Comput. Graph. Forum 34, 2 (2015), 349--359.
[37]
J. Kim and F. Pellacini. 2002. Jigsaw image mosaics. ACM Trans. Graph. 21, 3 (2002), 657--664.
[38]
J. Kopf, C.-W. Fu, D. Cohen-Or, O. Deussen, D. Lischinski, and T.-T. Wong. 2007. Solid texture synthesis from 2D exemplars. ACM Trans. Graph. 26, 3 (2007).
[39]
V. Kwatra, I. Essa, A. Bobick, and N. Kwatra. 2005. Texture optimization for example-based synthesis. ACM Trans. Graph. 24, 3 (2005), 795--802.
[40]
V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Trans. Graph. 22, 3 (2003), 277--286.
[41]
J. E. Kyprianidis, J. Collomosse, T. Wang, and T. Isenberg. 2013. State of the “Art”: A taxonomy of artistic stylization techniques for images and video. IEEE Trans. Visual. Comput. Graph. 19, 5 (2013), 866--885.
[42]
Thomas Leung and Jitendra Malik. 2001. Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vision 43, 1 (June 2001), 29--44.
[43]
C. Li and M. Wand. 2016a. Combining markov random fields and convolutional neural networks for image synthesis. CoRR abs/1601.04589 (2016).
[44]
C. Li and M. Wand. 2016b. Precomputed real-time texture synthesis with markovian generative adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV’16), Part III. 702--716.
[45]
L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum. 2001. Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20, 3 (2001), 127--150.
[46]
T. Lindemeier, S. Pirk, and O. Deussen. 2013. Image stylization with a painting machine using semantic hints. Comput. Graph. 37, 5 (2013), 293--301.
[47]
Y. Liu, O. Veksler, and O. Juan. 2010. Generating classic mosaics with graph cuts. Comput. Graph. Forum 29, 8 (2010), 2387--2399.
[48]
J. Lu, C. Barnes, S. DiVerdi, and A. Finkelstein. 2013. RealBrush: Painting with examples of physical media. ACM Trans. Graph. 32, 4 (July 2013), 117:1--117:12.
[49]
Sebastian Montabone and Alvaro Soto. 2010. Human detection using a mobile platform and novel features derived from a visual saliency mechanism. Image Vision Comput. 28, 3 (Mar. 2010), 391--402.
[50]
I. Olonetsky and S. Avidan. 2012. TreeCANN - K-d tree coherence approximate nearest neighbor algorithm. In Proceedings of the European Conference on Computer Vision Part IV. Springer-Verlag, Berlin, 602--615.
[51]
R. Paget and I. D. Longstaff. 1998. Texture synthesis via a noncausal nonparametric multiscale Markov random field. IEEE Trans. Image Process. 7, 6 (1998), 925--931.
[52]
A. Panotopoulou, S. Paris, and E. Whiting. 2018. Watercolor woodblock printing with image analysis. Comput. Graph. Forum 37, 2 (2018), 275--286.
[53]
D. Pavić, U. Ceumern, and L. Kobbelt. 2009. GIzMOs: Genuine image mosaics with adaptive tiling. Computer Graph. Forum 28, 8 (2009), 2244--2254.
[54]
J. Portilla and E. P. Simoncelli. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vision 40, 1 (2000), 49--70.
[55]
E. Praun, A. Finkelstein, and H. Hoppe. 2000. Lapped textures. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., New York, NY, 465--470.
[56]
Y. Pritch, E. Kav-Venaki, and S. Peleg. 2009. Shift-map image editing. In Proceedings of the IEEE 12th International Conference on Computer Vision. 151--158.
[57]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. CoRR abs/1505.04597 (2015). arxiv:1505.04597 http://arxiv.org/abs/1505.04597.
[58]
John C. Russ. 2002. Image Processing Handbook (4th ed.). CRC Press, Boca Raton, FL.
[59]
Kaisei Sakurai, Yoshinori Dobashi, Kei Iwasaki, and Tomoyuki Nishita. 2018. Fabricating reflectors for displaying multiple images. ACM Trans. Graph. 37, 4 (July 2018).
[60]
R. Silvers. 1997. Photomosaics. Henry Holt and Co., Inc., New York, NY, USA.
[61]
D. Simakov, Y. Caspi, E. Shechtman, and M. Irani. 2008. Summarizing visual data using bidirectional similarity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.
[62]
R. Szeliski and H.-Y. Shum. 1997. Creating full view panoramic image mosaics and environment maps. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., New York, NY, 251--258.
[63]
Taeg Sang Cho, M. Butman, S. Avidan, and W. T. Freeman. 2008. The patch transform and its applications to image editing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.
[64]
C. Tomasi and R. Manduchi. 1998. Bilateral filtering for gray and color images. In Proceedings of the 6th International Conference on Computer Vision. 839--846.
[65]
X. Tong, J. Zhang, L. Liu, X. Wang, B. Guo, and H.-Y. Shum. 2002. Synthesis of bidirectional texture functions on arbitrary surfaces. ACM Trans. Graph. 21, 3 (July 2002), 665--672.
[66]
P. A. Tresset and F. Fol Leymarie. 2012. Sketches by paul the robot. In Proceedings of the Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging. Eurographics Association, Goslar, Germany, 17--24.
[67]
R. B. Ulrich. 2007. Roman Woodworking. Yale University Press.
[68]
L.-Y. Wei, S. Lefebvre, V. Kwatra, and G. Turk. 2009. State of the art in example-based texture synthesis. In Eurographics 2009: State of the Art Reports. Eurographics Association, 93--117.
[69]
L.-Y. Wei and M. Levoy. 2000. Fast texture synthesis using tree-structured vector quantization. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., New York, NY, 479--488.
[70]
Y. Wexler, E. Shechtman, and M. Irani. 2007. Space-time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29, 3 (2007), 463--476.
[71]
Qi Zhang, Xiaoyong Shen, Li Xu, and Jiaya Jia. 2014. Rolling guidance filter. In Proceedings of the European Conference on Computer Vision (ECCV’14).

Cited By

View all
  • (2024)DARTS: Diffusion Approximated Residual Time Sampling for Time-of-flight Rendering in Homogeneous Scattering MediaACM Transactions on Graphics10.1145/368793043:6(1-14)Online publication date: 19-Dec-2024
  • (2024)Plug-and-Play Algorithms for Dynamic Non-line-of-sight ImagingACM Transactions on Graphics10.1145/3665139Online publication date: 14-May-2024
  • (2024)Shadow Based Non-Line-of-Sight Pedestrian Rushing Detection for Automated DrivingIEEE Transactions on Mobile Computing10.1109/TMC.2024.344790723:12(14754-14767)Online publication date: 1-Dec-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 39, Issue 2
April 2020
136 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/3381407
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 February 2020
Accepted: 01 December 2019
Revised: 01 October 2019
Received: 01 April 2019
Published in TOG Volume 39, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Computational arts
  2. fabrication
  3. style transfer
  4. texture synthesis

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)42
  • Downloads (Last 6 weeks)6
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)DARTS: Diffusion Approximated Residual Time Sampling for Time-of-flight Rendering in Homogeneous Scattering MediaACM Transactions on Graphics10.1145/368793043:6(1-14)Online publication date: 19-Dec-2024
  • (2024)Plug-and-Play Algorithms for Dynamic Non-line-of-sight ImagingACM Transactions on Graphics10.1145/3665139Online publication date: 14-May-2024
  • (2024)Shadow Based Non-Line-of-Sight Pedestrian Rushing Detection for Automated DrivingIEEE Transactions on Mobile Computing10.1109/TMC.2024.344790723:12(14754-14767)Online publication date: 1-Dec-2024
  • (2023)Ghost-Probe: NLOS Pedestrian Rushing Detection with Monocular Camera for Automated DrivingProceedings of the 21st ACM Conference on Embedded Networked Sensor Systems10.1145/3625687.3625791(390-402)Online publication date: 12-Nov-2023
  • (2023)Doppler Time-of-Flight RenderingACM Transactions on Graphics10.1145/361833542:6(1-18)Online publication date: 5-Dec-2023
  • (2023)Computational Design of LEGO® Sketch ArtACM Transactions on Graphics10.1145/361830642:6(1-15)Online publication date: 5-Dec-2023
  • (2023)Self-Calibrating, Fully Differentiable NLOS Inverse RenderingSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618140(1-11)Online publication date: 10-Dec-2023
  • (2023)Neural Volumetric Reconstruction for Coherent Synthetic Aperture SonarACM Transactions on Graphics10.1145/359214142:4(1-20)Online publication date: 26-Jul-2023
  • (2022)Image Fusion and Stylization Processing Based on Multiscale Transformation and Convolutional Neural NetworkComputational Intelligence and Neuroscience10.1155/2022/11811892022Online publication date: 1-Jan-2022
  • (2022)Non-Line-of-Sight Transient RenderingACM SIGGRAPH 2022 Posters10.1145/3532719.3543223(1-2)Online publication date: 27-Jul-2022
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media