More Web Proxy on the site http://driver.im/

research-article

Sketch2Photo: internet image montage

Authors:

Ming-Ming Cheng,

Shi-Min HuAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 28, Issue 5

Pages 1 - 10

https://doi.org/10.1145/1618452.1618470

Published: 01 December 2009 Publication History

Abstract

We present a system that composes a realistic picture from a simple freehand sketch annotated with text labels. The composed picture is generated by seamlessly stitching several photographs in agreement with the sketch and text labels; these are found by searching the Internet. Although online image search generates many inappropriate results, our system is able to automatically select suitable photographs to generate a high quality composition, using a filtering scheme to exclude undesirable images. We also provide a novel image blending algorithm to allow seamless image composition. Each blending result is given a numeric score, allowing us to find an optimal combination of discovered images. Experimental results show the method is very successful; we also evaluate our system using the results from two user studies.

Supplementary Material

Supplemental material. (124-chen.zip)

Download
96.59 MB

References

[1]

Belongie, S., Malik, J., and Puzicha, J. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4, 509--522.

Digital Library

[2]

Ben-Haim, N., Babenko, B., and Belongie, S. 2006. Improvingweb-based image search via content based clustering. In Proc. of CVPR Workshop.

Digital Library

[3]

Diakopoulos, N., Essa, I., and Jain, R. 2004. Content based image synthesis. In Proc. of International Conference on Image and Video Retrieval (CIVR).

[4]

Eitz, M., Hildebrand, K., Boubekeur, T., and Alexa, M. 2009. Photosketch: A sketch based image query and compositing system. In SIGGRAPH 2009 Talk Program.

Digital Library

[5]

Farbman, Z., Hoffer, G., Lipman, Y., Cohen-Or, D., and Lischinski, D. 2009. Coordinates for instant image cloning. ACM Transactions on Graphics 28, 3 (Aug.), 67.

Digital Library

[6]

Felzenszwalb, P. F., and Huttenlocher, D. P. 2004. Efficient graph-based image segmentation. Int. J. of Comput. Vision 59, 2, 167--181.

Digital Library

[7]

Fergus, R., Fei-Fei, L., Perona, P., and Zisserman, A. 2005. Learning object categories from google's image search. In Proc. of ICCV.

Digital Library

[8]

Georgescu, B., Shimshoni, I., and Meer, P. 2003. Mean shift based clustering in high dimensions: A texture classification example. In Proc. of ICCV.

Digital Library

[9]

Hays, J. H., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics 26, 3 (July), 4.

Digital Library

[10]

Hou, X., and Zhang, L. 2007. Saliency detection: A spectral residual approach. In Proc. of CVPR.

[11]

Jacobs, C., Finkelstein, A., and Salesin, D. 1995. Fast multiresolution image querying. In SIGGRAPH 1995.

Digital Library

[12]

Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2006. Drag-and-drop pasting. SIGGRAPH 2006.

Digital Library

[13]

Johnson, M., Brostow, G. J., Shotton, J., Arandjelović, O., Kwatra, V., and Cipolla, R. 2006. Semantic photo synthesis. Proc. of Eurographics.

[14]

Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. ACM Transactions on Graphics 26, 3 (Aug.), 3.

Digital Library

[15]

Levin, A., Lischinski, D., and Weiss, Y. 2008. A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2, 228--242.

Digital Library

[16]

Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. SIGGRAPH 2004.

Digital Library

[17]

Liu, T., Sun, J., Zheng, N.-N., Tang, X., and Shum, H.-Y. 2007. Learning to detect a salient object. In Proc. of CVPR.

[18]

Manjunath, B. S., and Ma, W. Y. 1996. Texture features for browsing and retrieval of image data. IEEE Trans. Pattern Anal. Mach. Intell. 18, 8, 837--842.

Digital Library

[19]

Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. SIGGRAPH 2003.

Digital Library

[20]

Rajendran, R., and Chang, S. 2000. Image retrieval with sketches and compositions. In Proc. of International Conference on Multimedia&Expo (ICME).

[21]

Rother, C., Kolmogorov, V., and Blake, A. 2004. "grabcut": interactive foreground extraction using iterated graph cuts. SIGGRAPH2004.

Digital Library

[22]

Saxena, A., Chung, S. H., and Ng, A. Y. 2008. 3-d depth reconstruction from a single still image. Int. J. of Comput. Vision 76, 1, 53--69.

Digital Library

[23]

Smeulders, A., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22, 12, 1349--1380.

Digital Library

[24]

Wang, J., and Cohen, M. 2007. Simultaneous matting and compositing. In Proc. of CVPR, 1--8.

Cited By

Liu FFu HLai YGao L(2024)SketchDream: Sketch-based Text-To-3D Generation and EditingACM Transactions on Graphics10.1145/365812043:4(1-13)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658120
Cheng ZWu LLi XMeng X(2024)SMFS‐GAN: Style‐Guided Multi‐class Freehand Sketch‐to‐Image SynthesisComputer Graphics Forum10.1111/cgf.1519043:6Online publication date: 7-Aug-2024
https://doi.org/10.1111/cgf.15190
Chen JCong RIp HKwong S(2024)KepSalinst: Using Peripheral Points to Delineate Salient InstancesIEEE Transactions on Cybernetics10.1109/TCYB.2023.332616554:6(3392-3405)Online publication date: Jun-2024
https://doi.org/10.1109/TCYB.2023.3326165
Show More Cited By

Index Terms

Sketch2Photo: internet image montage
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        3D imaging
    2. Knowledge representation and reasoning
  2. Computer graphics
2. Theory of computation
  1. Logic

Recommendations

Sketch2Photo: internet image montage
SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers

We present a system that composes a realistic picture from a simple freehand sketch annotated with text labels. The composed picture is generated by seamlessly stitching several photographs in agreement with the sketch and text labels; these are found ...
Sketch2Photo: Synthesizing photo-realistic images from sketches via global contexts
Abstract
Sketch-to-image synthesis aims to generate realistic images that match the input sketches or edge maps exactly. Most known sketch-to-image synthesis methods use various generative adversarial networks (GANs) that are trained with ...
Highlights
- We propose a novel generator for sketch-to-image synthesis, which can capture global context information in the early layers of the network and maintain the ...
An art-directed wrinkle system for CG character clothing and skin

We present a kinematic system for creating art-directed clothing and skin wrinkles on CG characters used in the production of computer-animated feature films. This system employs a curve-based method for generating wrinkles on reference poses, which are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 28, Issue 5

December 2009

646 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1618452

Issue’s Table of Contents

Copyright © 2009 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2009

Published in TOG Volume 28, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Singapore FRC
Ministry of Science and Technology of the People's Republic of China
FDCT, Macau

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

398
Total Citations
View Citations
2,713
Total Downloads

Downloads (Last 12 months)112
Downloads (Last 6 weeks)13

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu FFu HLai YGao L(2024)SketchDream: Sketch-based Text-To-3D Generation and EditingACM Transactions on Graphics10.1145/365812043:4(1-13)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658120
Cheng ZWu LLi XMeng X(2024)SMFS‐GAN: Style‐Guided Multi‐class Freehand Sketch‐to‐Image SynthesisComputer Graphics Forum10.1111/cgf.1519043:6Online publication date: 7-Aug-2024
https://doi.org/10.1111/cgf.15190
Chen JCong RIp HKwong S(2024)KepSalinst: Using Peripheral Points to Delineate Salient InstancesIEEE Transactions on Cybernetics10.1109/TCYB.2023.332616554:6(3392-3405)Online publication date: Jun-2024
https://doi.org/10.1109/TCYB.2023.3326165
Nazarieh FFeng ZAwais MWang WKittler J(2024)A Survey of Cross-Modal Visual Content GenerationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.335160134:8(6814-6832)Online publication date: Aug-2024
https://doi.org/10.1109/TCSVT.2024.3351601
Ye NSun Y(2024)CDM: Text-Driven Image Editing with Composable Diffusion Models2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10649901(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10649901
Hasan MZillanee ARabiul Alam M(2024)Generating Photorealistic Images from Human-Generated Sketches: A GAN-based Synthesis Approach for Enhanced Visual Realism2024 6th International Conference on Electrical Engineering and Information & Communication Technology (ICEEICT)10.1109/ICEEICT62016.2024.10534434(160-165)Online publication date: 2-May-2024
https://doi.org/10.1109/ICEEICT62016.2024.10534434
Bell TLi D(2024)Diffusion Models for Cross-Domain Image-to-Image Translation with Paired and Partially Paired Datasets2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA61799.2024.10722775(1-8)Online publication date: 6-Oct-2024
https://doi.org/10.1109/DSAA61799.2024.10722775
Sharma NTripathi AChakraborty AMishra A(2024)Sketch-guided Image Inpainting with Partial Discrete Diffusion Process2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00609(6024-6034)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00609
Pandey KGuerrero PGadelha MHold-Geoffroy YSingh KMitra N(2024)Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00735(7695-7704)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00735
Hou YZhao YYao K(2024)Federated Learning Backdoor Defense Based on Watermark Integrity2024 10th International Conference on Big Data and Information Analytics (BigDIA)10.1109/BigDIA63733.2024.10808344(288-294)Online publication date: 25-Oct-2024
https://doi.org/10.1109/BigDIA63733.2024.10808344
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents