[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Compressive light field photography using overcomplete dictionaries and optimized projections

Published: 21 July 2013 Publication History

Abstract

Light field photography has gained a significant research interest in the last two decades; today, commercial light field cameras are widely available. Nevertheless, most existing acquisition approaches either multiplex a low-resolution light field into a single 2D sensor image or require multiple photographs to be taken for acquiring a high-resolution light field. We propose a compressive light field camera architecture that allows for higher-resolution light fields to be recovered than previously possible from a single image. The proposed architecture comprises three key components: light field atoms as a sparse representation of natural light fields, an optical design that allows for capturing optimized 2D light field projections, and robust sparse reconstruction methods to recover a 4D light field from a single coded 2D projection. In addition, we demonstrate a variety of other applications for light field atoms and sparse coding, including 4D light field compression and denoising.

Supplementary Material

ZIP File (a46-marwah.zip)
Supplemental material.
MP4 File (tp171.mp4)

References

[1]
Adelson, E., and Wang, J. 1992. Single Lens Stereo with a Plenoptic Camera. IEEE Trans. PAMI 14, 2, 99--106.
[2]
Ashok, A., and Neifeld, M. A. 2010. Compressive Light Field Imaging. In Proc. SPIE 7690, 76900Q.
[3]
Babacan, S., Ansorge, R., Luessi, M., Mataran, P., Molina, R., and Katsaggelos, A. 2012. Compressive Light Field Sensing. IEEE Trans. Im. Proc. 21, 12, 4746--4757.
[4]
Bishop, T., Zanetti, S., and Favaro, P. 2009. Light-Field Superresolution. In Proc. ICCP, 1--9.
[5]
Candès, E., and Wakin, M. B. 2008. An Introduction to Compressive Sampling. IEEE Signal Processing 25, 2, 21--30.
[6]
Candès, E. J., Eldar, Y. C., Needell, D., and Randall, P. 2011. Compressed Sensing with Coherent and Redundant Dictionaries. Appl. and Comp. Harmonic Analysis 31, 1, 59--73.
[7]
Chen, S. S., Donoho, D. L., Michael, and Saunders, A. 1998. Atomic Decomposition by Basis Pursuit. SIAM J. on Scientific Computing 20, 33--61.
[8]
Donoho, D. 2006. Compressed Sensing. IEEE Trans. Inform. Theory 52, 4, 1289--1306.
[9]
Duarte-Carvajalino, J., and Sapiro, G. 2009. Learning to sense sparse signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization. IEEE Trans. Im. Proc. 18, 7, 1395--1408.
[10]
Elad, M., and Aharon, M. 2006. Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries. IEEE Trans. Im. Proc. 15, 12, 3736--3745.
[11]
Feigin, M., Feldman, D., and Sochen, N. A. 2012. From High Definition Image to Low Space Optimization. In Scale Space and Var. Methods in Comp. Vision, vol. 6667, 459--470.
[12]
Fergus, R., Torralba, A., and Freeman, W. T. 2006. Random Lens Imaging. Tech. Rep. TR-2006-058, MIT.
[13]
Georgiev, T., and Lumsdaine, A. 2006. Spatio-angular Resolution Tradeoffs in Integral Photography. Proc. EGSR, 263--272.
[14]
Gortler, S., Grzeszczuk, R., Szelinski, R., and Cohen, M. 1996. The Lumigraph. In Proc. ACM SIGGRAPH, 43--54.
[15]
Heide, F., Wetzstein, G., Raskar, R., and Heidrich, W. 2013. Adaptive Image Synthesis for Compressive Displays. ACM Trans. Graph. (SIGGRAPH) 32, 4, 1--11.
[16]
Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., and Nayar, S. K. 2011. Video from a Single Coded Exposure Photograph using a Learned Over-Complete Dictionary. In Proc. IEEE ICCV.
[17]
Ives, H., 1903. Parallax Stereogram and Process of Making Same. US patent 725,567.
[18]
Kamal, M., Golbabaee, M., and Vandergheynst, P. 2012. Light Field Compressive Sensing in Camera Arrays. In Proc. ICASSP, 5413--5416.
[19]
Lanman, D., Raskar, R., Agrawal, A., and Taubin, G. 2008. Shield Fields: Modeling and Capturing 3D Occluders. ACM Trans. Graph. (SIGGRAPH Asia) 27, 5, 131.
[20]
Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization Fields: Dynamic Light Field Display using Multi-Layer LCDs. ACM Trans. Graph. (SIGGRAPH Asia) 30, 1--9.
[21]
Levin, A., Freeman, W. T., and Durand, F. 2008. Understanding Camera Trade-Offs through a Bayesian Analysis of Light Field Projections. In Proc. ECCV, 88--101.
[22]
Levin, A., Hasinoff, S. W., Green, P., Durand, F., and Freeman, W. T. 2009. 4D Frequency Analysis of Computational Cameras for Depth of Field Extension. ACM Trans. Graph. (SIGGRAPH) 28, 3, 97.
[23]
Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In Proc. ACM SIGGRAPH, 31--42.
[24]
Liang, C.-K., Lin, T.-H., Wong, B.-Y., Liu, C., and Chen, H. H. 2008. Programmable Aperture Photography: Multiplexed Light Field Acquisition. ACM Trans. Graph. (SIGGRAPH) 27, 3, 1--10.
[25]
Lippmann, G. 1908. La Photographie Intégrale. Academie des Sciences 146, 446--451.
[26]
Lumsdaine, A., and Georgiev, T. 2009. The Focused Plenoptic Camera. In Proc. ICCP, 1--8.
[27]
Mairal, J., Bach, F., Ponce, G., and Sapiro, G. 2009. Online Dictionary Learning For Sparse Coding. In International Conference on Machine Learning.
[28]
Marcia, R. F., and Willett, R. M. 2008. Compressive coded aperture video reconstruction. In EUSIPCO.
[29]
Natarajan, B. K. 1995. Sparse Approximate Solutions to Linear Systems. SIAM J. Computing 24, 227--234.
[30]
Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Tech. rep., Stanford University.
[31]
Ng, R. 2005. Fourier Slice Photography. ACM Trans. Graph. (SIGGRAPH) 24, 3, 735--744.
[32]
Park, J. Y., and Wakin, M. B. 2012. A geometric approach to multi-view compressive imaging. EURASIP Journal on Advances in Signal Processing 37.
[33]
Peers, P., Mahajan, D. K., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., and Debevec, P. 2009. Compressive Light Transport Sensing. ACM Trans. Graph. 28, 3.
[34]
Perwass, C., and Wietzke, L. 2012. Single Lens 3D-Camera with Extended Depth-of-Field. In Proc. SPIE 8291, 29--36.
[35]
Reddy, D., Veeraraghavan, A., and Chellappa, R. 2011. P2C2: Programmable Pixel Compressive Camera for High Speed Imaging. In Proc. IEEE CVPR, 329--336.
[36]
Sen, P., and Darabi, S. 2009. Compressive Dual Photography. Computer Graphics Forum 28, 609--618.
[37]
Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocussing. ACM Trans. Graph. (SIGGRAPH) 26, 3, 69.
[38]
Wakin, M. B., Laska, J. N., Duarte, M. F., Baron, D., Sarvotham, S., Takhar, D., Kelly, K. F., and Baraniuk, R. G. 2006. Compressive imaging for video representation and coding. In Picture Coding Symposium.
[39]
Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic Image Synthesis for Attenuation-based Light Field and High Dynamic Range Displays. ACM Trans. Graph. (SIGGRAPH).
[40]
Wetzstein, G., Ihrke, I., and Heidrich, W. 2012. On Plenoptic Multiplexing and Reconstruction. IJCV, 1--16.
[41]
Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 1--11.
[42]
Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High Performance Imaging using Large Camera Arrays. ACM Trans. Graph. (SIGGRAPH) 24, 3, 765--776.
[43]
Xu, Z., and Lam, E. Y. 2012. A High-resolution Lightfield Camera with Dual-mask Design. In Proc. SPIE 8500, 85000U.
[44]
Yang, A., Ganesh, A., Sastry, S., and Ma, Y. 2010. Fast L1-Minimization Algorithms and An Application in Robust Face Recognition: A Review. Tech. rep., UC Berkeley.
[45]
Yang, J., Wang, Z., Lin, Z., Cohen, S., and Huang, T. 2012. Coupled Dictionary Training for Image Super-Resolution. IEEE Trans. Im. Proc. 21, 8, 3467--3478.

Cited By

View all
  • (2024)基于梯度下降深度均衡模型的动态光场重建(特邀)Laser & Optoelectronics Progress10.3788/LOP24140061:16(1611006)Online publication date: 2024
  • (2024)光场表征及其分辨率提升技术:文献综述及最新进展(特邀)Infrared and Laser Engineering10.3788/IRLA2024034753:9(20240347)Online publication date: 2024
  • (2024)A Light-Field Video Dataset of Scenes with Moving Objects Captured with a Plenoptic Video CameraElectronics10.3390/electronics1311222313:11(2223)Online publication date: 6-Jun-2024
  • Show More Cited By

Index Terms

  1. Compressive light field photography using overcomplete dictionaries and optimized projections

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 32, Issue 4
    July 2013
    1215 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/2461912
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 July 2013
    Published in TOG Volume 32, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. compressive sensing
    2. computational photography

    Qualifiers

    • Research-article

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)101
    • Downloads (Last 6 weeks)17
    Reflects downloads up to 13 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)基于梯度下降深度均衡模型的动态光场重建(特邀)Laser & Optoelectronics Progress10.3788/LOP24140061:16(1611006)Online publication date: 2024
    • (2024)光场表征及其分辨率提升技术:文献综述及最新进展(特邀)Infrared and Laser Engineering10.3788/IRLA2024034753:9(20240347)Online publication date: 2024
    • (2024)A Light-Field Video Dataset of Scenes with Moving Objects Captured with a Plenoptic Video CameraElectronics10.3390/electronics1311222313:11(2223)Online publication date: 6-Jun-2024
    • (2024)[Paper] Compressive Acquisition of Light Field Video Using Aperture-Exposure-Coded CameraITE Transactions on Media Technology and Applications10.3169/mta.12.2212:1(22-35)Online publication date: 2024
    • (2024)Noise sensitivity analysis of focal scanning light field imagingOptics Express10.1364/OE.53947432:21(37542)Online publication date: 2-Oct-2024
    • (2024)Lensless light-field imaging using LMIOptics Express10.1364/OE.53902132:22(38112)Online publication date: 7-Oct-2024
    • (2024)Light-field photography using differential high-speed aperture codingApplied Optics10.1364/AO.52033863:11(2939)Online publication date: 4-Apr-2024
    • (2024)Astronomical adaptive optics: a reviewPhotoniX10.1186/s43074-024-00118-75:1Online publication date: 1-May-2024
    • (2024)Learning-based light field imaging: an overviewJournal on Image and Video Processing10.1186/s13640-024-00628-12024:1Online publication date: 30-May-2024
    • (2024)Light Field Display Point RenderingProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36513007:1(1-18)Online publication date: 13-May-2024
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media