More Web Proxy on the site http://driver.im/

research-article

From image parsing to painterly rendering

Authors:

Song-Chun ZhuAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 29, Issue 1

Article No.: 2, Pages 1 - 11

https://doi.org/10.1145/1640443.1640445

Published: 15 December 2009 Publication History

Abstract

We present a semantics-driven approach for stroke-based painterly rendering, based on recent image parsing techniques [Tu et al. 2005; Tu and Zhu 2006] in computer vision. Image parsing integrates segmentation for regions, sketching for curves, and recognition for object categories. In an interactive manner, we decompose an input image into a hierarchy of its constituent components in a parse tree representation with occlusion relations among the nodes in the tree. To paint the image, we build a brush dictionary containing a large set (760) of brush examples of four shape/appearance categories, which are collected from professional artists, then we select appropriate brushes from the dictionary and place them on the canvas guided by the image semantics included in the parse tree, with each image component and layer painted in various styles. During this process, the scene and object categories also determine the color blending and shading strategies for inhomogeneous synthesis of image details. Compared with previous methods, this approach benefits from richer meaningful image semantic information, which leads to better simulation of painting techniques of artists using the high-quality brush dictionary. We have tested our approach on a large number (hundreds) of images and it produced satisfactory painterly effects.

References

[1]

Boykov, Y. and Jolly, M.-P. 2001. Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In Proceedings of the 8th IEEE International Conference on Computer Vision (ICCV'01). Vol. 1. 105--112.

[2]

Chen, H. and Zhu, S.-C. 2006. A generative sketch model for human hair analysis and synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 28, 7, 1025--1040.

Digital Library

[3]

Chu, N. S.-H. and Tai, C.-L. 2005. Moxi: Real-Time ink dispersion in absorbent paper. ACM Trans. Graph. 24, 3, 504--511.

Digital Library

[4]

Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. ACM Trans. Graph. 25, 3, 624--630.

Digital Library

[5]

Comaniciu, D. and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 5, 603--619.

Digital Library

[6]

Curtis, C. J., Anderson, S. E., Seims, J. E., Fleischer, K. W., and Salesin, D. H. 1997. Computer-Generated watercolor. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '97). 421--430.

Digital Library

[7]

Funch, B. S. 1997. The Psychology of Art Appreciation. Museum Tusculanum Press.

[8]

Gooch, A., Gooch, B., Shirley, P., and Cohen, E. 1998. A non-photorealistic lighting model for automatic technical illustration. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'98). 447--452.

Digital Library

[9]

Gooch, B., Coombe, G., and Shirley, P. 2002. Artistic vision: Painterly rendering using computer vision techniques. In Proceedings of the 2nd International Symposium on Non-Photorealistic Animation and Rendering (NPAR'02). 83--90.

Digital Library

[10]

Gooch, B. and Gooch, A. 2001. Non-Photorealistic Rendering. A K Peters, Ltd.

Digital Library

[11]

Gooch, B., Sloan, P.-P. J., Gooch, A., Shirley, P., and Riesenfeld, R. 1999. Interactive technical illustration. In Proceedings of the 1999 Symposium on Interactive 3D Graphics (I3D'99). 31--38.

Digital Library

[12]

Guo, C.-E., Zhu, S.-C., and Wu, Y. N. 2007. Primal sketch: Integrating structure and texture. Comput. Vis. Image Understand. 106, 1, 5--19.

Digital Library

[13]

Haeberli, P. 1990. Paint by numbers: Abstract image representations. In Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'90). 207--214.

Digital Library

[14]

Hertzmann, A. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'98). 453--460.

Digital Library

[15]

Hertzmann, A. 2003. Tutorial: A survey of stroke-based rendering. IEEE Comput. Graph. Appl. 23, 4, 70--81.

Digital Library

[16]

Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'01). 327--340.

Digital Library

[17]

Li, F.-F., Fergus, R., and Torralba, A. 2005. Recognizing and learning object categories. A short course at ICCV 2005.

[18]

Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Trans. Graph. 23, 3, 303--308.

Digital Library

[19]

Litwinowicz, P. 1997. Processing images and video for an impressionist effect. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'97). 407--414.

Digital Library

[20]

Lowe, D. G. 1999. Object recognition from local scale-invariant features. In Proceedings of the International Conference on Computer Vision (ICCV'99), Volume 2. 1150--1157.

Digital Library

[21]

Marr, D. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W. H. Freeman.

Digital Library

[22]

Perona, P. 1998. Orientation diffusions. IEEE Trans. Image Proces. 7, 3, 457--467.

Digital Library

[23]

Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 21, 5, 34--41.

Digital Library

[24]

Sousa, M. C. and Buchanan, J. W. 1999. Computer-Generated graphite pencil rendering of 3d polygonal models. In Proceedings of EuroGraphics'99 Conference. 195--207.

[25]

Strassmann, S. 1986. Hairy brushes. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'86). 225--232.

Digital Library

[26]

Strothotte, T. and Schlechtweg, S. 2002. Non-Photorealistic Computer Graphics: Modeling, Rendering and Animation. Morgan Kaufmann.

Digital Library

[27]

Teece, D. 1998. 3d painting for non-photorealistic rendering. In ACM Conference on Abstracts and Applications (SIGGRAPH'98). 248.

Digital Library

[28]

Tu, Z., Chen, X., Yuille, A. L., and Zhu, S.-C. 2005. Image parsing: Unifying segmentation, detection, and recognition. Int. J. Comput. Vis. 63, 2, 113--140.

Digital Library

[29]

Tu, Z. and Zhu, S.-C. 2006. Parsing images into regions, curves, and curve groups. Int. J. Comput. Vis. 69, 2, 223--249.

Digital Library

[30]

Turk, G. and Banks, D. 1996. Image-Guided streamline placement. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). 453--460.

Digital Library

[31]

Winkenbach, G. and Salesin, D. H. 1994. Computer-Generated pen-and-ink illustration. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'94). 91--100.

Digital Library

[32]

Xu, S., Xu, Y., Kang, S. B., Salesin, D. H., Pan, Y., and Shum, H.-Y. 2006. Animating chinese paintings through stroke-based decomposition. ACM Trans. Graph. 25, 2, 239--267.

Digital Library

[33]

Yao, B., Yang, X., and Zhu, S.-C. 2007. Introduction to a large-scale general purpose ground truth database: Methodology, annotation tool and benchmarks. In Proceedings of the International Conferences on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR '07). 169--183.

Digital Library

Cited By

Hu ZChen YZhao ZLiu JKe BNi BCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Towards Artist-Like Painting Agents with Multi-Granularity Semantic AlignmentProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681245(10191-10199)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681245
Yang FLi XLin HChen L(2024)Cross-domain Image Translation Algorithm based on Self-cross Auto-encoder2024 Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC)10.1109/IPEC61310.2024.00017(35-43)Online publication date: 12-Apr-2024
https://doi.org/10.1109/IPEC61310.2024.00017
Karimov AKopets EShpilevaya TKatser ELeonov SButusov D(2023)Comparing Neural Style Transfer and Gradient-Based Algorithms in Brushstroke Rendering TasksMathematics10.3390/math1110225511:10(2255)Online publication date: 11-May-2023
https://doi.org/10.3390/math11102255
Show More Cited By

Index Terms

From image parsing to painterly rendering
1. Applied computing
  1. Arts and humanities
    1. Fine arts
    2. Performing arts
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Hierarchical representations
  2. Computer graphics
    1. Graphics systems and interfaces

Recommendations

Gloss perception in painterly and cartoon rendering

Depictions with traditional media such as painting and drawing represent scene content in a stylized manner. It is unclear, however, how well stylized images depict scene properties like shape, material, and lighting. In this article, we describe the ...
Real-time painterly rendering for MR applications
GRAPHITE '04: Proceedings of the 2nd international conference on Computer graphics and interactive techniques in Australasia and South East Asia

In this paper we describe a real-time system for AR/MR rendering applications in a painterly style. Impressionistic images are created using a large number of brush strokes, which are organized as 3d particles to achieve frame-to-frame coherence. ...
Real-Time Painterly Rendering of Terrains
ICVGIP '08: Proceedings of the 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing

We present a non-photo realistic, real-time painterly rendering technique for terrains. The painterly appearance and the impression of terrains is created by effectively rendering several brush strokes. The strokes have fixed locations on the surfaces ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 29, Issue 1

December 2009

127 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1640443

Issue’s Table of Contents

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 December 2009

Accepted: 01 July 2009

Revised: 01 May 2009

Received: 01 November 2008

Published in TOG Volume 29, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

138
Total Citations
View Citations
1,935
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)2

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hu ZChen YZhao ZLiu JKe BNi BCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Towards Artist-Like Painting Agents with Multi-Granularity Semantic AlignmentProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681245(10191-10199)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681245
Yang FLi XLin HChen L(2024)Cross-domain Image Translation Algorithm based on Self-cross Auto-encoder2024 Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC)10.1109/IPEC61310.2024.00017(35-43)Online publication date: 12-Apr-2024
https://doi.org/10.1109/IPEC61310.2024.00017
Karimov AKopets EShpilevaya TKatser ELeonov SButusov D(2023)Comparing Neural Style Transfer and Gradient-Based Algorithms in Brushstroke Rendering TasksMathematics10.3390/math1110225511:10(2255)Online publication date: 11-May-2023
https://doi.org/10.3390/math11102255
Ilinkin ISong DKim Y(2023)Stroke-Based Rendering and Planning for Robotic Performance of Artistic Drawing2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS55552.2023.10341808(8062-8068)Online publication date: 1-Oct-2023
https://doi.org/10.1109/IROS55552.2023.10341808
Schaldenbrand PMcCann JOh J(2023)FRIDA: A Collaborative Robot Painter with a Differentiable, Real2Sim2Real Planning Environment2023 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA48891.2023.10160702(11712-11718)Online publication date: 29-May-2023
https://doi.org/10.1109/ICRA48891.2023.10160702
Pan TWang FXie JLiu W(2023)Generating New Paintings by Semantic GuidanceMultiMedia Modeling10.1007/978-3-031-27818-1_49(599-610)Online publication date: 9-Jan-2023
https://dl.acm.org/doi/10.1007/978-3-031-27818-1_49
Cárdenas RHenares KArroba PRisco-Martín JWainer G(2022)The DEVStone Metric: Performance Analysis of DEVS Simulation EnginesACM Transactions on Modeling and Computer Simulation10.1145/354384932:3(1-20)Online publication date: 25-Jul-2022
https://dl.acm.org/doi/10.1145/3543849
Blackman DVigna S(2022)A New Test for Hamming-Weight DependenciesACM Transactions on Modeling and Computer Simulation10.1145/352758232:3(1-13)Online publication date: 25-Jul-2022
https://dl.acm.org/doi/10.1145/3527582
Ungredda JPearce MBranke J(2022)Bayesian Optimisation vs. Input Uncertainty ReductionACM Transactions on Modeling and Computer Simulation10.1145/351038032:3(1-26)Online publication date: 25-Jul-2022
https://dl.acm.org/doi/10.1145/3510380
Tong ZWang XYuan SChen XWang JFang XMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Im2Oil: Stroke-Based Oil Painting Rendering with Linearly Controllable Fineness Via Adaptive SamplingProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3547759(1035-1046)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3547759
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents