[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3478512.3488607acmconferencesArticle/Chapter ViewAbstractPublication Pagessiggraph-asiaConference Proceedingsconference-collections
research-article

Comic Image Inpainting via Distance Transform

Published: 14 December 2021 Publication History

Abstract

Inpainting techniques for natural images have progressed significantly. However, if these methods are applied to comic images, the results are not satisfactory because of very noticeable artifacts, especially around line drawings. Line drawings are challenging to inpaint because of their high-frequency components. In this paper, we propose a novel method for inpainting comic images in the distance transform domain. In this method, we first convert a line drawing into a distance image and then inpaint the distance image. By transforming line drawings into distance images, we can eventually reduce the high-frequency components, which leads to improve inpainting performance. We compared the results of our proposed method with those of the conventional methods. The results showed that the proposed method achieved 0.1% lower l1 loss, 0.5dB higher PSNR, and 0.5% higher SSIM than those of the conventional inpainting methods.

Supplementary Material

Supplemental file (SIGGRAPH_technical_comm_supplemental.pdf)
MP4 File (3478512.3488607.mp4)
presentation

References

[1]
Kiyoharu Aizawa, Azuma Fujimoto, Atsushi Otsubo, Toru Ogawa, Yusuke Matsui, Koki Tsubota, and Hikaru Ikuta. 2020. Building a Manga Dataset ”Manga109” with Annotations for Multimedia Applications. IEEE MultiMedia (2020).
[2]
Mohammed Almatrafi, Raymond Baldwin, Kiyoharu Aizawa, and Keigo Hirakawa. 2020. Distance surface for event-based optical flow. IEEE transactions on pattern analysis and machine intelligence (2020).
[3]
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2016. Image style transfer using convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[4]
Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial networks. arXiv preprint arXiv:1406.2661(2014).
[5]
Ryota Hinami, Shonosuke Ishiwatari, Kazuhiko Yasuda, and Yusuke Matsui. 2021. Towards Fully Automated Manga Translation. In Proceedings of the AAAI Conference on Artificial Intelligence.
[6]
Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. 2017. Globally and Locally Consistent Image Completion. ACM Transactions on Graphics (Proc. of SIGGRAPH 2017) (2017).
[7]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In The European Conference on Computer Vision (ECCV).
[8]
Chengze Li, Xueting Liu, and Tien-Tsin Wong. 2017. Deep Extraction of Manga Structural Lines. ACM Transactions on Graphics (SIGGRAPH 2017 issue) (July 2017).
[9]
Jingyuan Li, Ning Wang, Lefei Zhang, Bo Du, and Dacheng Tao. 2020. Recurrent Feature Reasoning for Image Inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[10]
Yusuke Matsui, Kota Ito, Yuji Aramaki, Azuma Fujimoto, Toru Ogawa, Toshihiko Yamasaki, and Kiyoharu Aizawa. 2017. Sketch-based Manga Retrieval using Manga109 Dataset. Multimedia Tools and Applications(2017).
[11]
Rei Narita, Keigo Hirakawa, and Kiyoharu Aizawa. 2019. Optical Flow Based Line Drawing Frame Interpolation Using Distance Transform to Support Inbetweenings. In 2019 IEEE International Conference on Image Processing (ICIP).
[12]
Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Qureshi, and Mehran Ebrahimi. 2019. EdgeConnect: Structure Guided Image Inpainting using Edge Prediction. In The IEEE International Conference on Computer Vision (ICCV) Workshops.
[13]
Azriel Rosenfeld and John L Pfaltz. 1966. Sequential operations in digital picture processing. Journal of the ACM (JACM)(1966).
[14]
Minshan Xie, Menghan Xia, Chengze Li, Xueting Liu, and Tien-Tsin Wong. 2021. Seamless Manga Inpainting with Semantics Awareness. ACM Transactions on Graphics (SIGGRAPH 2021 issue) (2021).
[15]
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. 2018. Generative image inpainting with contextual attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

Cited By

View all
  • (2024)Bridging partial-gated convolution with transformer for smooth-variation image inpaintingMultimedia Tools and Applications10.1007/s11042-024-18590-583:32(78387-78406)Online publication date: 23-Feb-2024
  • (2024)Automated inpainting of Tex Willer comics: adapting manga techniques for western comic transformationThe Visual Computer10.1007/s00371-024-03440-040:7(4585-4592)Online publication date: 28-May-2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
SA '21: SIGGRAPH Asia 2021 Technical Communications
December 2021
88 pages
ISBN:9781450390736
DOI:10.1145/3478512
  • Editors:
  • Shuzo John Shiota,
  • Ayumi Kimura,
  • Wan-Chun Alex Ma
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 December 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. comic
  2. inpainting
  3. neural networks

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SA '21
Sponsor:
SA '21: SIGGRAPH Asia 2021
December 14 - 17, 2021
Tokyo, Japan

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)40
  • Downloads (Last 6 weeks)1
Reflects downloads up to 03 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Bridging partial-gated convolution with transformer for smooth-variation image inpaintingMultimedia Tools and Applications10.1007/s11042-024-18590-583:32(78387-78406)Online publication date: 23-Feb-2024
  • (2024)Automated inpainting of Tex Willer comics: adapting manga techniques for western comic transformationThe Visual Computer10.1007/s00371-024-03440-040:7(4585-4592)Online publication date: 28-May-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media