More Web Proxy on the site http://driver.im/

article

Multi-aperture photography

Authors:

Wojciech Matusik,

Frédo DurandAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 26, Issue 3

Pages 68 - es

https://doi.org/10.1145/1276377.1276462

Published: 29 July 2007 Publication History

Abstract

The emergent field of computational photography is proving that, by coupling generalized imaging optics with software processing, the quality and flexibility of imaging systems can be increased. In this paper, we capture and manipulate multiple images of a scene taken with different aperture settings (f-numbers). We design and implement a prototype optical system and associated algorithms to capture four images of the scene in a single exposure, each taken with a different aperture setting. Our system can be used with commercially available DSLR cameras and photographic lenses without modification to either. We leverage the fact that defocus blur is a function of scene depth and f/# to estimate a depth map. We demonstrate several applications of our multi-aperture camera, such as post-exposure editing of the depth of field, including extrapolation beyond the physical limits of the lens, synthetic refocusing, and depth-guided deconvolution.

Supplementary Material

JPG File (pps068.jpg)

Download
17.31 KB

MP4 File (pps068.mp4)

Download
33.39 MB

References

[1]

Adelson, E. H., and Wang, J. Y. A. 1992. Single lens stereo with a plenoptic camera. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 2, 99--106.

Digital Library

[2]

Aggarwal, M., and Ahuja, N. 2004. Split aperture imaging for high dynamic range. Int. J. Comp. Vision 58, 1 (June), 7--17.

Digital Library

[3]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 11.

Digital Library

[4]

Chaudhuri, S., and Rajagopalan, A. 1999. Depth From Defocus: A Real Aperture Imaging Approach. Springer Verlag.

[5]

Farid, H., and Simoncelli, E. 1996. A differential optical range camera. In Optical Society of America, Annual Meeting.

[6]

Georgiev, T., Zheng, K. C., Curless, B., Salesin, D., Nayar, S., and Intwala, C. 2006. Spatio-angular resolution tradeoffs in integral photography. In Proceedings of Eurographics Symposium on Rendering (2006), 263--272.

[7]

Harvey, R. P., 1998. Optical beam splitter and electronic high speed camera incorporating such a beam splitter. US 5734507, United States Patent.

[8]

Hasinoff, S., and Kutulakos, K. 2006. Confocal stereo. In European Conference on Computer Vision (2006), 620--634.

Digital Library

[9]

Hecht, E. 2002. Optics. Addison-Wesley, Reading, MA.

[10]

Hiura, S., and Matsuyama, T. 1998. Depth measurement by the multi-focus camera. In IEEE Computer Vision and Pattern Recognition (1998), 953--961.

Digital Library

[11]

Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 297--306.

Digital Library

[12]

Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Transactions on Graphics 23, 3 (Aug.), 689--694.

Digital Library

[13]

McGuire, M., Matusik, W., Pfister, H., Hughes, J. F., and Durand, F. 2005. Defocus video matting. ACM Transactions on Graphics 24, 3 (Aug.), 567--576.

Digital Library

[14]

McGuire, M., Matusik, W., Chen, B., Hughes, J. F., Pfister, H., and Nayar, S. 2007. Optical splitting trees for high-precision monocular imaging. IEEE Computer Graphics and Applications (2007) (March).

Digital Library

[15]

Naemura, T., Yoshida, T., and Harashima, H. 2001. 3-D computer graphics based on integral photography. Opt. Expr. 8.

[16]

Narasimhan, S., and Nayar, S. 2005. Enhancing resolution along multiple imaging dimensions using assorted pixels. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 4 (Apr), 518--530.

Digital Library

[17]

Ng, R. 2005. Fourier slice photography. ACM Transactions on Graphics 24, 3 (Aug.), 735--744.

Digital Library

[18]

Okano, F., Arai, J., Hoshino, H., and Yuyama, I. 1999. Three-dimensional video system based on integral photography. Optical Engineering 38 (June), 1072--1077.

[19]

Pentland, A. P. 1987. A new sense for depth of field. IEEE Transactions on Pattern Analysis and Machine Intelligence 9, 4, 523--531.

Digital Library

[20]

Ray, S. 1988. Applied photographic optics. Focal Press.

[21]

Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M. F., and Rother, C. 2006. A comparative study of energy minimization methods for markov random fields. In European Conference on Computer Vision (2006), 16--29.

Digital Library

[22]

Watanabe, M., Nayar, S., and Noguchi, M. 1996. Real-time computation of depth from defocus. In Proceedings of The International Society for Optical Engineering (SPIE), vol. 2599, 14--25.

Cited By

Zia AZhou JGao Y(2021)Exploring Chromatic Aberration and Defocus Blur for Relative Depth Estimation From Monocular Hyperspectral ImageIEEE Transactions on Image Processing10.1109/TIP.2021.307168230(4357-4370)Online publication date: 2021
https://doi.org/10.1109/TIP.2021.3071682
Agrawal A(2021)Multi-focus ImagesComputer Vision10.1007/978-3-030-63416-2_445(849-851)Online publication date: 13-Oct-2021
https://doi.org/10.1007/978-3-030-63416-2_445
Ishida HKagawa KKomuro TZhang BSeo MTakasawa TYasutomi KKawahito S(2018)Multi-Aperture-Based Probabilistic Noise Reduction of Random Telegraph Signal Noise and Photon Shot Noise in Semi-Photon-Counting Complementary-Metal-Oxide-Semiconductor Image SensorSensors10.3390/s1804097718:4(977)Online publication date: 26-Mar-2018
https://doi.org/10.3390/s18040977
Show More Cited By

Index Terms

Multi-aperture photography
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning

Recommendations

Programmable aperture photography: multiplexed light field acquisition

In this paper, we present a system including a novel component called programmable aperture and two associated post-processing algorithms for high-quality light field acquisition. The shape of the programmable aperture can be adjusted and used to ...
Multi-aperture photography
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

The emergent field of computational photography is proving that, by coupling generalized imaging optics with software processing, the quality and flexibility of imaging systems can be increased. In this paper, we capture and manipulate multiple images ...
Beyond Perspective Dual Photography With Illumination Masks
Scene appearance from the point of view of a light source is called a reciprocal or dual view. Since there exists a large diversity in illumination, these virtual views may be nonperspective and multiviewpoint in nature. In this paper, we demonstrate the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 26, Issue 3

July 2007

976 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1276377

Issue’s Table of Contents

Copyright © 2007 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 July 2007

Published in TOG Volume 26, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
1,859
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zia AZhou JGao Y(2021)Exploring Chromatic Aberration and Defocus Blur for Relative Depth Estimation From Monocular Hyperspectral ImageIEEE Transactions on Image Processing10.1109/TIP.2021.307168230(4357-4370)Online publication date: 2021
https://doi.org/10.1109/TIP.2021.3071682
Agrawal A(2021)Multi-focus ImagesComputer Vision10.1007/978-3-030-63416-2_445(849-851)Online publication date: 13-Oct-2021
https://doi.org/10.1007/978-3-030-63416-2_445
Ishida HKagawa KKomuro TZhang BSeo MTakasawa TYasutomi KKawahito S(2018)Multi-Aperture-Based Probabilistic Noise Reduction of Random Telegraph Signal Noise and Photon Shot Noise in Semi-Photon-Counting Complementary-Metal-Oxide-Semiconductor Image SensorSensors10.3390/s1804097718:4(977)Online publication date: 26-Mar-2018
https://doi.org/10.3390/s18040977
Liu HZhong JWaller L(2017)Multiplexed phase-space imaging for 3D fluorescence microscopyOptics Express10.1364/OE.25.01498625:13(14986)Online publication date: 21-Jun-2017
https://doi.org/10.1364/OE.25.014986
Kanojia GRaman S(2017)Postcapture Focusing Using Regression ForestIEEE Signal Processing Letters10.1109/LSP.2017.269062124:6(751-755)Online publication date: Jun-2017
https://doi.org/10.1109/LSP.2017.2690621
Park HKim YLee YYun WLim JKang DKhan MKhan APark JChoi WHwang YKyung C(2017)Depth Estimation Using Single Camera with Dual AperturesSmart Sensors at the IoT Frontier10.1007/978-3-319-55345-0_7(167-189)Online publication date: 31-May-2017
https://doi.org/10.1007/978-3-319-55345-0_7
Mochizuki FKagawa KOkihara SSeo MZhang BTakasawa TYasutomi KKawahito S(2016)Single-event transient imaging with an ultra-high-speed temporally compressive multi-aperture CMOS image sensorOptics Express10.1364/OE.24.00415524:4(4155)Online publication date: 19-Feb-2016
https://doi.org/10.1364/OE.24.004155
Chang JKauvar IHu XWetzstein G(2016)Variable Aperture Light Field Photography: Overcoming the Diffraction-Limited Spatio-Angular Resolution Tradeoff2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR.2016.406(3737-3745)Online publication date: Jun-2016
https://doi.org/10.1109/CVPR.2016.406
Agrawal A(2016)Multi-focus ImagesComputer Vision10.1007/978-0-387-31439-6_445(511-513)Online publication date: 5-Feb-2016
https://doi.org/10.1007/978-0-387-31439-6_445
Chih-Tsung Shen Hung-Hsun Liu Ming-Hsuan Yang Yi-Ping Hung Soo-Chang Pei (2015)Viewing-Distance Aware Super-Resolution for High-Definition DisplayIEEE Transactions on Image Processing10.1109/TIP.2014.237563924:1(403-418)Online publication date: Jan-2015
https://doi.org/10.1109/TIP.2014.2375639
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents