More Web Proxy on the site http://driver.im/

research-article

Stereoscopizing cel animations

Authors:

Tien-Tsin WongAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 32, Issue 6

Article No.: 223, Pages 1 - 10

https://doi.org/10.1145/2508363.2508396

Published: 01 November 2013 Publication History

Abstract

While hand-drawn cel animation is a world-wide popular form of art and entertainment, introducing stereoscopic effect into it remains difficult and costly, due to the lack of physical clues. In this paper, we propose a method to synthesize convincing stereoscopic cel animations from ordinary 2D inputs, without labor-intensive manual depth assignment nor 3D geometry reconstruction. It is mainly automatic due to the need of producing lengthy animation sequences, but with the option of allowing users to adjust or constrain all intermediate results. The system fits nicely into the existing production flow of cel animation. By utilizing the T-junction cue available in cartoons, we first infer the initial, but not reliable, ordering of regions. One of our major contributions is to resolve the temporal inconsistency of ordering by formulating it as a graph-cut problem. However, the resultant ordering remains insufficient for generating convincing stereoscopic effect, as ordering cannot be directly used for depth assignment due to its discontinuous nature. We further propose to synthesize the depth through an optimization process with the ordering formulated as constraints. This is our second major contribution. The optimized result is the spatiotemporally smooth depth for synthesizing stereoscopic effect. Our method has been evaluated on a wide range of cel animations and convincing stereoscopic effect is obtained in all cases.

Supplementary Material

ZIP File (a223-liu.zip)

Supplemental material.

Download
57.92 MB

References

[1]

Amer, M., Raich, R., and Todorovic, S. 2010. Monocular extraction of 2.1 d sketch. In Proc. of the International Conference on Image Processing.

[2]

Apostoloff, N., and Fitzgibbon, A. 2005. Learning spatiotemporal T-junctions for occlusion detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 2, IEEE, 553--559.

Digital Library

[3]

Ashikhmin, M. 2001. Synthesizing natural textures. In Proceedings of the 2001 symposium on Interactive 3D graphics, ACM, 217--226.

Digital Library

[4]

Assa, J., and Wolf, L. 2007. Diorama construction from a single image. In Computer Graphics Forum, vol. 26, Wiley Online Library, 599--608.

[5]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. Pattern Analysis and Machine Intelligence, IEEE Transactions on 23, 11, 1222--1239.

Digital Library

[6]

Bruce, V., Green, P. R., and Georgeson, M. A. 2003. Visual Perception: Physiology, Psychology, & Ecology. Psychology Press.

[7]

Criminisi, A., Pérez, P., and Toyama, K. 2004. Region filling and object removal by exemplar-based image inpainting. Image Processing, IEEE Transactions on 13, 9, 1200--1212.

Digital Library

[8]

Dimiccoli, M., and Salembier, P. 2009. Exploiting t-junctions for depth segregation in single images. In Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on, IEEE, 1229--1232.

Digital Library

[9]

Dimiccoli, M., and Salembier, P. 2009. Hierarchical region-based representation for segmentation and filtering with depth in single images. In Image Processing (ICIP), 2009 16th IEEE International Conference on, IEEE, 3533--3536.

Digital Library

[10]

Forsyth, D. 2001. Shape from texture and integrability. In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, vol. 2, IEEE, 447--452.

[11]

Gill, P., Murray, W., Saunders, M., and Wright, M. 1984. Procedures for optimization problems with a mixture of bounds and general linear constraints. ACM Transactions on Mathematical Software (TOMS) 10, 3, 282--298.

Digital Library

[12]

Gingold, Y., Igarashi, T., and Zorin, D. 2009. Structured annotations for 2d-to-3d modeling. In ACM Transactions on Graphics (TOG), vol. 28, ACM, 148.

Digital Library

[13]

Goldberg, E. 2009. Medial axis techniques for stereoscopic extraction. In SIGGRAPH 2009: Talks, ACM, 74.

Digital Library

[14]

Guzmán, A. 1968. Decomposition of a visual scene into three dimensional bodies. In Proceedings of the December 9-11, 1968, fall joint computer conference, part I, AFIPS '68 (Fall, part I), 291--304.

Digital Library

[15]

He, K., Sun, J., and Tang, X. 2009. Single image haze removal using dark channel prior. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, IEEE, 1956--1963.

[16]

Horn, B. 1990. Height and gradient from shading. International journal of computer vision 5, 1, 37--75.

Digital Library

[17]

Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: a sketching interface for 3d freeform design. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 409--416.

Digital Library

[18]

Jia, Z., Gallagher, A., Chang, Y., and Chen, T. 2012. A learning-based framework for depth ordering. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, IEEE, 294--301.

Digital Library

[19]

Joshi, P., and Carr, N. 2008. Repoussé: Automatic inflation of 2D artwork. In Eurographics Workshop on Sketch-Based Interfaces and Modeling, The Eurographics Association, 49--55.

Digital Library

[20]

Kahn, A. 1962. Topological sorting of large networks. Communications of the ACM 5, 11, 558--562.

Digital Library

[21]

Kang, S., and Szeliski, R. 2004. Extracting view-dependent depth maps from a collection of images. International Journal of Computer Vision 58, 2, 139--163.

Digital Library

[22]

Karpenko, O., and Hughes, J. 2006. SmoothSketch: 3D freeform shapes from complex sketches. In ACM Transactions on Graphics (TOG), vol. 25, ACM, 589--598.

Digital Library

[23]

Kim, Y., Winnemöller, H., and Lee, S. 2013. Wysiwyg stereo painting. In Proceedings of ACM Symposium on Interactive 3D Graphics and Games 2013.

Digital Library

[24]

Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. 2010. Nonlinear disparity mapping for stereoscopic 3d. ACM Transactions on Graphics (TOG) 29, 4, 75.

Digital Library

[25]

Lee, S., Feng, D., and Gooch, B. 2008. Automatic construction of 3d models from architectural line drawings. In Proceedings of the 2008 symposium on Interactive 3D graphics and games, ACM, 123--130.

Digital Library

[26]

Lipson, H., and Shpitalni, M. 1996. Optimization-based reconstruction of a 3d object from a single freehand line drawing. Computer-Aided Design 28, 8, 651--663.

[27]

Metzger, W. 1936. Gesetze des sehens. W. Kramer.

[28]

Nayar, S., and Nakagawa, Y. 1990. Shape from focus: An effective approach for rough surfaces. In Robotics and Automation, 1990. Proceedings., 1990 IEEE International Conference on, IEEE, 218--225.

[29]

Nealen, A., Igarashi, T., Sorkine, O., and Alexa, M. 2007. Fibermesh: designing freeform surfaces with 3D curves. In ACM Transactions on Graphics (TOG), vol. 26, ACM, 41.

Digital Library

[30]

Noris, G., Hornung, A., Sumner, R. W., Simmons, M., and Gross, M. 2013. Topology-driven vectorization of clean line drawings. ACM Transactions on Graphics (TOG) 32, 1, 4.

Digital Library

[31]

Production I. G., Sanzigen Animation Studio, and Ishimori Productions, 2012. Cyborg 009.

[32]

Production I.G., 2011. Ghost in the shell: Stand alone complex: Solid state society.

[33]

Rademacher, P. 1999. View-dependent geometry. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, 439--446.

Digital Library

[34]

Rivers, A., Igarashi, T., and Durand, F. 2010. 2-5D cartoon models. ACM Transactions on Graphics (TOG) 29, 4, 59.

Digital Library

[35]

Schar, S., Bieri, H., Killer, T., and Jiang, X. 2008. Introducing stereo effects into cel animations. In 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video, 2008, IEEE, 353--356.

[36]

Sun, J., Yuan, L., Jia, J., and Shum, H. 2005. Image completion with structure propagation. ACM Transactions on Graphics (ToG) 24, 3, 861--868.

Digital Library

[37]

Super, B., and Bovik, A. 1995. Shape from texture using local spectral moments. Pattern Analysis and Machine Intelligence, IEEE Transactions on 17, 4, 333--343.

Digital Library

[38]

Sýkora, D., Sedlacek, D., Jinchao, S., Dingliana, J., and Collins, S. 2010. Adding depth to cartoons using sparse depth (in) equalities. In Computer Graphics Forum, vol. 29, Wiley Online Library, 615--623.

[39]

Tokyo Movie Shinsha, 1977. Ie naki ko.

[40]

Varley, P., and Martin, R. 2002. Estimating depth from line drawing. In Proceedings of the seventh ACM symposium on Solid modeling and applications, ACM, 180--191.

Digital Library

[41]

Ventura, J., Diverdi, S., and Höllerer, T. 2009. A sketch-based interface for photo pop-up. In Proceedings of the 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling, ACM, 21--28.

Digital Library

[42]

Wang, O., Lang, M., Frei, M., Hornung, A., Smolic, A., and Gross, M. 2011. Stereobrush: interactive 2d to 3d conversion using discontinuous warps. In Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling, ACM, 47--54.

Digital Library

[43]

Ward, B., Kang, S. B., and Bennett, E. P. 2011. Depth director: A system for adding depth to movies. Computer Graphics and Applications, IEEE 31, 1, 36--48.

Digital Library

[44]

Wu, T., Sun, J., Tang, C., and Shum, H. 2008. Interactive normal reconstruction from a single image. ACM Transactions on Graphics (TOG) 27, 5, 119.

Digital Library

[45]

Zhang, L., Dugas-Phocion, G., Samson, J., and Seitz, S. 2002. Single-view modeling of free-form scenes. The Journal of Visualization and Computer Animation 13, 4, 225--235.

[46]

Zhang, G., Jia, J., Wong, T., and Bao, H. 2008. Recovering consistent video depth maps via bundle optimization. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, IEEE, 1--8.

[47]

Zhang, S., Chen, T., Zhang, Y., Hu, S., and Martin, R. 2009. Vectorizing cartoon animations. Visualization and Computer Graphics, IEEE Transactions on 15, 4, 618--629.

Digital Library

Cited By

Zhang SMa JWu JRitchie DAgrawala M(2023)Editing Motion Graphics Video via Motion Vectorization and TransformationACM Transactions on Graphics10.1145/361831642:6(1-13)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618316
Even MBénard PBarla P(2023)Non‐linear Rough 2D Animation using Transient EmbeddingsComputer Graphics Forum10.1111/cgf.1477142:2(411-425)Online publication date: 23-May-2023
https://doi.org/10.1111/cgf.14771
Xu CQu WXu XLiu X(2023)Multi-Scale Flow-Based Occluding Effect and Content Separation for Cartoon AnimationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317465629:9(4001-4014)Online publication date: 1-Sep-2023
https://doi.org/10.1109/TVCG.2022.3174656
Show More Cited By

Index Terms

Stereoscopizing cel animations
1. Applied computing
  1. Arts and humanities
    1. Fine arts
    2. Performing arts
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding

Recommendations

Shadows for cel animation
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques

We present a semi-automatic method for creating shadow mattes in cel animation. In conventional cel animation, shadows are drawn by hand, in order to provide visual cues about the spatial relationships and forms of characters in the scene. Our system ...
2D shading for cel animation
Expressive '18: Proceedings of the Joint Symposium on Computational Aesthetics and Sketch-Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering

We present a semi-automatic method for creating shades and self-shadows in cel animation. Besides producing attractive images, shades and shadows provide important visual cues about depth, shapes, movement and lighting of the scene. In conventional cel ...
3D Character Model Creation from Cel Animation
CW '04: Proceedings of the 2004 International Conference on Cyberworlds

When creating a cel animation, the animators often use 3D character models to add some effects on the character or to generate intermediate images between the key frames. However, it is a troublesome and time-consuming task to create a 3D model. In this ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 32, Issue 6

November 2013

671 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2508363

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2013

Published in TOG Volume 32, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Research Grants Council, University Grants Committee, Hong Kong
National Natural Science Foundation of China
Chinese University of Hong Kong
Basic Research for 2012 Shenzhen Municipal Science and Technology Programme

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
988
Total Downloads

Downloads (Last 12 months)45
Downloads (Last 6 weeks)8

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang SMa JWu JRitchie DAgrawala M(2023)Editing Motion Graphics Video via Motion Vectorization and TransformationACM Transactions on Graphics10.1145/361831642:6(1-13)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618316
Even MBénard PBarla P(2023)Non‐linear Rough 2D Animation using Transient EmbeddingsComputer Graphics Forum10.1111/cgf.1477142:2(411-425)Online publication date: 23-May-2023
https://doi.org/10.1111/cgf.14771
Xu CQu WXu XLiu X(2023)Multi-Scale Flow-Based Occluding Effect and Content Separation for Cartoon AnimationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317465629:9(4001-4014)Online publication date: 1-Sep-2023
https://doi.org/10.1109/TVCG.2022.3174656
Ashtari ASeo CKang CCha SNoh J(2022)Reference Based Sketch Extraction via Attention MechanismACM Transactions on Graphics10.1145/3550454.355550441:6(1-16)Online publication date: 30-Nov-2022
https://doi.org/10.1145/3550454.3555504
Jiang JSeah HLiew H(2021)Stroke‐Based Drawing and Inbetweening with Boundary StrokesComputer Graphics Forum10.1111/cgf.1443341:1(257-269)Online publication date: 29-Dec-2021
https://doi.org/10.1111/cgf.14433
Siyao LZhao SYu WSun WMetaxas DLoy CLiu Z(2021)Deep Animation Video Interpolation in the Wild2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR46437.2021.00652(6583-6591)Online publication date: Jun-2021
https://doi.org/10.1109/CVPR46437.2021.00652
Xin YWong HLo SLi J(2019)Progressive Full Data Convolutional Neural Networks for Line Extraction from Anime-Style IllustrationsApplied Sciences10.3390/app1001004110:1(41)Online publication date: 19-Dec-2019
https://doi.org/10.3390/app10010041
Zhang YLai YZhang F(2019)Stereoscopic image stitching with rectangular boundariesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-019-01694-735:6-8(823-835)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s00371-019-01694-7
Dvorožňák MNejad SJamriška OJacobson AKavan LSýkora DWyvill BFu H(2018)Seamless reconstruction of part-based high-relief models from hand-drawn imagesProceedings of the Joint Symposium on Computational Aesthetics and Sketch-Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering10.1145/3229147.3229153(1-9)Online publication date: 17-Aug-2018
https://dl.acm.org/doi/10.1145/3229147.3229153
Leimkuhler TKellnhofer PRitschel TMyszkowski KSeidel H(2018)Perceptual Real-Time 2D-to-3D Conversion Using Cue FusionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2017.270361224:6(2037-2050)Online publication date: 1-Jun-2018
https://doi.org/10.1109/TVCG.2017.2703612
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents