[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
article

Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging

Published: 01 August 2004 Publication History

Abstract

We present a non-photorealistic rendering approach to capture and convey shape features of real-world scenes. We use a camera with multiple flashes that are strategically positioned to cast shadows along depth discontinuities in the scene. The projective-geometric relationship of the camera-flash setup is then exploited to detect depth discontinuities and distinguish them from intensity edges due to material discontinuities.We introduce depiction methods that utilize the detected edge features to generate stylized static and animated images. We can highlight the detected features, suppress unnecessary details or combine features from multiple images. The resulting images more clearly convey the 3D structure of the imaged scenes.We take a very different approach to capturing geometric features of a scene than traditional approaches that require reconstructing a 3D model. This results in a method that is both surprisingly simple and computationally efficient. The entire hardware/software setup can conceivably be packaged into a self-contained device no larger than existing digital cameras.

Supplementary Material

MOV File (pps059.mov)

References

[1]
AKERS, D., LOSASSO, F., KLINGER, J., AGRAWALA, M., RICK, J., AND HANRAHAN, P. 2003. Conveying Shape and Features with Image-Based Relighting. In IEEE Visualization.
[2]
AVENUE AMY, 2002. Curious Pictures.
[3]
BIRCHFIELD, S. 1999. Depth and Motion Discontinuities. PhD thesis, Stanford University.
[4]
CHUANG, Y.-Y., GOLDMAN, D. B., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2003. Shadow matting and compositing. ACM Trans. Graph. 22, 3, 494--500.
[5]
COHEN, M. F., COLBURN, A., AND DRUCKER, S. 2003. Image stacks. Tech. Rep. MSR-TR-2003-40, Microsoft Research.
[6]
DAUM, M., AND DUDEK, G. 1998. On 3-D Surface Reconstruction using Shape from Shadows. In CVPR, 461--468.
[7]
DECARLO, D., AND SANTELLA, A. 2002. Stylization and Abstraction of Photographs. In Proc. Siggraph 02, ACM Press.
[8]
DECARLO, D., FINKELSTEIN, A., RUSINKIEWICZ, S., AND SANTELLA, A. 2003. Suggestive contours for conveying shape. ACM Trans. Graph. 22, 3, 848--855.
[9]
DURAND, F. 2002. An Invitation to Discuss Computer Depiction. In Proceedings of NPAR 2002.
[10]
ELDER, J. 1999. Are Edges Incomplete? International Journal of Computer Vision 34, 2/3, 97--122.
[11]
FATTAL, R., LISCHINSKI, D., AND WERMAN, M. 2002. Gradient Domain High Dynamic Range Compression. In Proceedings of SIGGRAPH 2002, ACM SIGGRAPH, 249--256.
[12]
FERIS, R., TURK, M., RASKAR, R., TAN, K., AND OHASHI, G. 2004. Exploiting Depth Discontinuities for Vision-based Fingerspelling Recognition. In IEEE Workshop on Real-time Vision for Human-Computer Interaction (in conjunction with CVPR'04).
[13]
FORSYTH, AND PONCE. 2002. Computer Vision, A Modern Approach.
[14]
GEIGER, D., LADENDORF, B., AND YUILLE, A. L. 1992. Occlusions and Binocular Stereo. In European Conference on Computer Vision, 425--433.
[15]
GOOCH, B., AND GOOCH, A. 2001. Non-Photorealistic Rendering. A K Peters, Ltd., Natick.
[16]
HERTZMANN, A. 1998. Painterly Rendering with Curved Brush Strokes of Multiple Sizes. In ACM SIGGRAPH, 453--460.
[17]
HUERTAS, A., AND NEVATIA, R. 1988. Detecting buildings in aerial images. Computer Vision, Graphics and Image Processing 41, 2, 131--152.
[18]
HUGGINGS, P., CHEN, H., BELHUMEUR, P., AND ZUCKER, S. 2001. Finding Folds: On the Appearance and Identification of Occlusion. In IEEE CVPR, vol. 2, 718--725.
[19]
IRVIN, R., AND MCKEOWN, D. 1989. Methods for exploiting the relationship between buildings and their shadows in aerial imagery. IEEE Transactions on Systems, Man and Cybernetics 19, 6, 1564--1575.
[20]
JOHNSTON, S. F. 2002. Lumo: Illumination for cel animation. In Proceedings of NPAR, ACM Press, 45--52.
[21]
KANG, S. B., SZELISKI, R., AND CHAI, J. 2001. Handling occlusions in dense multi-view stereo. In IEEE CVPR, vol. 1, 102--110.
[22]
KRIEGMAN, D., AND BELHUMEUR, P. 2001. What Shadows Reveal About Object Structure. Journal of the Optical Society of America, 1804--1813.
[23]
LANGER, M., DUDEK, G., AND ZUCKER, S. 1995. Space Occupancy using Multiple Shadow Images. International Conference on Intelligent Robots and Systems, 390--396.
[24]
LIN, C., AND NEVATIA, R. 1998. Building detection and description from a single intensity image. Computer Vision and Image Understanding: CVIU 72, 2, 101--121.
[25]
PAPADEMETRIS, X., AND BELHUMEUR, P. N. 1996. Estimation of motion boundary location and optical flow using dynamic programming. In Proc. Int. Conf. on Image Processing.
[26]
PRESS, W. H., TEUKOLSKY, S., VETTERLING, W. T., AND FLANNERY, B. P. 1992. Numerical Recipes in C: The Art of Scientific Computing. Pearson Education.
[27]
RASKAR, R., ILIE, A., AND YU, J. 2004. Image Fusion for Context Enhancement and Video Surrealism. In Proceedings of NPAR.
[28]
RAVIV, D., PAO, Y., AND LOPARO, K. A. 1989. Reconstruction of Three-dimensional Surfaces from Two-dimensional Binary Images. In IEEE Transactions on Robotics and Automation, vol. 5(5), 701--710.
[29]
SAITO, T., AND TAKAHASHI, T. 1990. Comprehensible Rendering of 3-D Shapes. In ACM SIGGRAPH, 197--206.
[30]
SATO, I., SATO, Y., AND IKEUCHI, K. 2001. Stability issues in recovering illumination distribution from brightness in shadows. IEEE Conf. on CVPR, 400--407.
[31]
SAVARESE, S., RUSHMEIER, H., BERNARDINI, F., AND PERONA, P. 2001. Shadow Carving. In ICCV.
[32]
SCHARSTEIN, D., AND SZELISKI, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In International Journal of Computer Vision, vol. 47(1), 7--42.
[33]
SHIRAI, Y., AND TSUJI, S. 1972. Extraction of the Line Drawing of 3-Dimensional Objects by Sequential Illumination from Several Directions. Pattern Recognition 4, 4, 345--351.
[34]
STROTHOTTE, T., AND SCHLECHTWEG, S. 2002. NonPhotorealistic Computer Graphics: Modeling, Rendering and Animation. Morgan Kaufmann, San Francisco.
[35]
TAN, P., LIN, S., QUAN, L., AND SHUM, H.-Y. 2003. Highlight Removal by Illumination-Constrained Inpainting. In Ninth IEEE International Conference on Computer Vision.
[36]
TAN, K., KOBLER, J., DIETZ, P., FERIS, R., AND RASKAR, R. 2004. Shape-Enhanced Surgical Visualizations and Medical Illustrations with Multi-Flash Imaging. In MERL TR/38.
[37]
TOYAMA, K., KRUMM, J., BRUMITT, B., AND MEYERS, B. 1999. Wallflower: Principles and Practice of Background Maintenance. In ICCV, 255--261.
[38]
WAKING LIFE, 2001. Waking Life, the movie.
[39]
WEISS, Y. 2001. Deriving intrinsic images from image sequences. In Proceedings of ICCV, vol. 2, 68--75.
[40]
YANG, D. K.-M. 1996. Shape from Darkness Under Error. PhD thesis, Columbia University.

Cited By

View all
  • (2024)Image Segmentation from Shadow-Hints using Minimum Spanning TreesACM SIGGRAPH 2024 Posters10.1145/3641234.3671025(1-2)Online publication date: 25-Jul-2024
  • (2024)Shape from Shading for Robotic Manipulation2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00820(8374-8383)Online publication date: 3-Jan-2024
  • (2024)SFF-RTI: an active multi-light approach to shape from focusThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02902-140:3(2067-2079)Online publication date: 1-Mar-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 23, Issue 3
August 2004
684 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/1015706
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2004
Published in TOG Volume 23, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. depth edges
  2. image enhancement
  3. non-photorealistic rendering

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)64
  • Downloads (Last 6 weeks)6
Reflects downloads up to 12 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Image Segmentation from Shadow-Hints using Minimum Spanning TreesACM SIGGRAPH 2024 Posters10.1145/3641234.3671025(1-2)Online publication date: 25-Jul-2024
  • (2024)Shape from Shading for Robotic Manipulation2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00820(8374-8383)Online publication date: 3-Jan-2024
  • (2024)SFF-RTI: an active multi-light approach to shape from focusThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02902-140:3(2067-2079)Online publication date: 1-Mar-2024
  • (2024)Flexible and Transportable Illumination Control System for Machine Vision ApplicationsNew Technologies, Development and Application VII10.1007/978-3-031-66271-3_7(58-65)Online publication date: 28-Jul-2024
  • (2023)Light field measurement of specular surfaces by multi-polarization and hybrid modulated illuminationApplied Optics10.1364/AO.49931962:30(8060)Online publication date: 16-Oct-2023
  • (2023)ShadowPatch: Shadow Based Segmentation for Reliable Depth Discontinuities in Photometric StereoComputer Graphics Forum10.1111/cgf.1470741:7(635-646)Online publication date: 20-Mar-2023
  • (2023)Adaptive Information Fusion Network for Arbitrary Style TransferIECON 2023- 49th Annual Conference of the IEEE Industrial Electronics Society10.1109/IECON51785.2023.10312640(1-7)Online publication date: 16-Oct-2023
  • (2023)Image neural style transferComputers and Electrical Engineering10.1016/j.compeleceng.2023.108723108:COnline publication date: 1-May-2023
  • (2022)Cartoon-Style Image Rendering Transfer Based on Neural NetworksComputational Intelligence and Neuroscience10.1155/2022/29583382022Online publication date: 1-Jan-2022
  • (2022)Seeing Far in the Dark with Patterned FlashComputer Vision – ECCV 202210.1007/978-3-031-20068-7_41(709-727)Online publication date: 23-Oct-2022
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media