More Web Proxy on the site http://driver.im/

research-article

Robust pencil drawing generation via fast Retinex decomposition

Authors:

Yanrong GuoAuthors Info & Claims

Volume 97, Issue C

Pages 67 - 77

https://doi.org/10.1016/j.cag.2021.04.008

Published: 01 June 2021 Publication History

Highlights

•

A new framework for generating pencil drawings based on the Retinex model.

•

A mixed-norm-based Retinex decomposition model with fast solution.

•

A scale-aware pencil line layer generation model.

•

A illumination-aware pencil tone layer generation model.

Graphical abstract

Display Omitted

Abstract

As imaging devices have been rapidly developed, it is very convenient to obtain natural images at all times and places. As a step further, generating the visual effect of artistic paintings from natural images has become an attractive image processing task, such as pencil drawing generation. However, current pencil drawing generation methods mainly concentrate on simulating the pencil-drawing patterns, while neglecting the complex illumination variations of an image. Therefore, various unsatisfying effects can be inevitably introduced, in turn weakening the quality of the generated pencil drawing patterns. To address this problem, we present a novel pencil drawing generation method based on Retinex decomposition. A mixed-norm Retinex image decomposition model is firstly proposed, which decomposes an image into an illumination layer and a reflectance layer. Then, with the obtained layers, the pencil line layer and tone layer are produced based on our simple but effective models, respectively. In experiments, the results show that our method consistently generates high-quality pencil drawing patterns from images with different illumination conditions. For example, as for the images with low light or back light, our method still ensures the visual quality of the obtained pencil drawings.

References

[1]

M. Wang, R. Hong, X.-T. Yuan, S. Yan, T.-S. Chua, Movie2Comics: towards a lively video content presentation, IEEE Trans Multimed 14 (3) (2012) 858–870.

[2]

G. Jing, Y. Hu, Y. Guo, Y. Yu, W. Wang, Content-aware video2comics with manga-style layout, IEEE Trans Multimed 17 (12) (2015) 2122–2133.

[3]

Z. Hu, S. Liu, J. Jiang, R. Hong, M. Wang, S. Yan, PicWords: render a picture by packing keywords, IEEE Trans Multimed 16 (4) (2014) 1156–1164.

[4]

L.A. Gatys, A.S. Ecker, M. Bethge, Image style transfer using convolutional neural networks, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2414–2423.

[5]

C. Lu, L. Xu, J. Jia, Combining sketch and tone for pencil drawing production, Proceedings of the symposium on non-photorealistic animation and rendering, Citeseer, 2012, pp. 65–73.

[6]

Y. Li, C. Fang, A. Hertzmann, E. Shechtman, M.-H. Yang, Im2Pencil: controllable pencil illustration from photographs, Proceedings of the IEEE conference on computer vision and pattern recognition, 2019, pp. 1525–1534.

[7]

M. Hata, M. Toyoura, X. Mao, Automatic pencil drawing generation using saliency map, Proceedings of the ACM symposium on applied perception, 2013.

[8]

S. Hao, Y. Guo, R. Hong, M. Wang, Scale-aware spatially guided mapping, IEEE MultiMed 23 (3) (2016) 34–42.

[9]

X. Cai, B. Song, Image-based pencil drawing synthesized using convolutional neural network feature maps, Mach Vis Appl 29 (3) (2018) 503–512.

[10]

Q. Kong, Y. Sheng, G. Zhang, Hybrid noise for LIC-based pencil hatching simulation, Proceedings of the 2018 IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2018, pp. 1–6.

[11]

J. Qiu, B. Liu, J. He, C. Liu, Y. Li, Parallel fast pencil drawing generation algorithm based on GPU, IEEE Access 7 (2019) 83543–83555.

[12]

D. Yan, Y. Sheng, X. Mao, Pencil drawing video rendering using convolutional networks, Proceedings of the computer graphics forum, 38, Wiley Online Library, 2019, pp. 91–102.

[13]

N. Inoue, D. Ito, N. Xu, J. Yang, B. Price, T. Yamasaki, Learning to trace: expressive line drawing generation from photographs, Proceedings of the computer graphics forum, 38, Wiley Online Library, 2019, pp. 69–80.

[14]

Zheng Q., Li Z., Bargteil A. Learning to shade hand-drawn sketches. arXiv preprint arXiv:2002118122020;.

[15]

B. Cai, X. Xu, K. Guo, K. Jia, B. Hu, D. Tao, A joint intrinsic-extrinsic prior model for Retinex, Proceedings of the IEEE international conference on computer vision, 2017, pp. 4000–4009.

[16]

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, L. Shao, Star: a structure and texture aware Retinex model, IEEE Trans Image Process 29 (2020) 5022–5037.

[17]

M. Li, J. Liu, W. Yang, X. Sun, Z. Guo, Structure-revealing low-light image enhancement via robust Retinex model, IEEE Trans Image Process 27 (6) (2018) 2828–2841.

[18]

X. Ren, W. Yang, W.-H. Cheng, J. Liu, LR3M: robust low-light enhancement via low-rank regularized Retinex model, IEEE Trans Image Process 29 (2020) 5862–5876.

[19]

X. Guo, Y. Li, H. Ling, Lime: Low-light image enhancement via illumination map estimation, IEEE Trans Image Process 26 (2) (2016) 982–993.

Digital Library

[20]

Q. Zhang, G. Yuan, C. Xiao, L. Zhu, W.-S. Zheng, High-quality exposure correction of underexposed photos, Proceedings of the 26th ACM international conference on multimedia, 2018, pp. 582–590.

[21]

Y. Gao, H.-M. Hu, B. Li, Q. Guo, Naturalness preserved nonuniform illumination estimation for image enhancement based on Retinex, IEEE Trans Multimed 20 (2) (2017) 335–344.

[22]

Wei C., Wang W., Yang W., Liu J. Deep Retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808045602018;.

[23]

Y. Zhang, J. Zhang, X. Guo, Kindling the darkness: a practical low-light image enhancer, Proceedings of the 27th ACM international conference on multimedia, 2019, pp. 1632–1640.

[24]

Y. Wang, Y. Cao, Z.-J. Zha, J. Zhang, Z. Xiong, W. Zhang, F. Wu, Progressive Retinex: mutually reinforced illumination-noise perception network for low-light image enhancement, Proceedings of the 27th ACM international conference on multimedia, 2019, pp. 2015–2023.

[25]

R. Kimmel, M. Elad, D. Shaked, R. Keshet, I. Sobel, A variational framework for Retinex, Int J Comput Vis 52 (1) (2003) 7–23.

Digital Library

[26]

M.K. Ng, W. Wang, A total variation model for Retinex, SIAM J Imaging Sci 4 (1) (2011) 345–365.

Digital Library

[27]

X. Fu, D. Zeng, Y. Huang, X.-P. Zhang, X. Ding, A weighted variational model for simultaneous reflectance and illumination estimation, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2782–2790.

[28]

X. Fu, D. Zeng, Y. Huang, X. Ding, X.-P. Zhang, A variational framework for single low light image enhancement using bright channel prior, Proceedings of the 2013 IEEE global conference on signal and information processing, IEEE, 2013, pp. 1085–1088.

[29]

S. Boyd, N. Parikh, E. Chu, Distributed optimization and statistical learning via the alternating direction method of multipliers, Now Publishers Inc, 2011.

Digital Library

[30]

S. Park, B. Moon, S. Ko, S. Yu, J. Paik, Low-light image restoration using bright channel prior-based variational Retinex model, EURASIP J Image Video Process 2017 (1) (2017) 1–11.

[31]

Z. Liang, J. Xu, D. Zhang, Z. Cao, L. Zhang, A hybrid l1-l0 layer decomposition model for tone mapping, Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4758–4766.

[32]

M. Li, Z. Lin, R. Mech, E. Yumer, D. Ramanan, Photo-sketching: Inferring contour drawings from images, Proceedings of the 2019 IEEE winter conference on applications of computer vision (WACV), IEEE, 2019, pp. 1403–1412.

Index Terms

Robust pencil drawing generation via fast Retinex decomposition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation
    2. Rendering

Index terms have been assigned to the content through auto-classification.

Recommendations

Enhancing pencil drawing patterns via using semantic information
Abstract
Pencil drawing is recognized as a typical non-photorealistic visual art form. It is attractive to automatically generate high-quality pencil drawings from real-world photographs. Traditional model-based methods are highlighted in their ...
Architectural Drawing Using Pencil Sketches and AutoCAD
A Novel Method for Pencil Drawing Generation in Non-Photo-Realistic Rendering
PCM '08: Proceedings of the 9th Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing

This paper puts forward a novel method for automatically generating a pencil drawing from a real 2D color image in non-photo-realistic rendering. First, the edge of the color image is detected by Sobel operator. Next, the color image is sharpened by ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers and Graphics

Computers and Graphics Volume 97, Issue C

Jun 2021

296 pages

ISSN:0097-8493

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 June 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents