[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain

Published: 29 December 2014 Publication History

Abstract

Sparsity in the Fourier domain is an important property that enables the dense reconstruction of signals, such as 4D light fields, from a small set of samples. The sparsity of natural spectra is often derived from continuous arguments, but reconstruction algorithms typically work in the discrete Fourier domain. These algorithms usually assume that sparsity derived from continuous principles will hold under discrete sampling. This article makes the critical observation that sparsity is much greater in the continuous Fourier spectrum than in the discrete spectrum. This difference is caused by a windowing effect. When we sample a signal over a finite window, we convolve its spectrum by an infinite sinc, which destroys much of the sparsity that was in the continuous domain. Based on this observation, we propose an approach to reconstruction that optimizes for sparsity in the continuous Fourier spectrum. We describe the theory behind our approach and discuss how it can be used to reduce sampling requirements and improve reconstruction quality. Finally, we demonstrate the power of our approach by showing how it can be applied to the task of recovering non-Lambertian light fields from a small number of 1D viewpoint trajectories.

Supplementary Material

shi (shi.zip)
Supplemental movie and image files for, Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain
MP4 File (a12.mp4)

References

[1]
E. H. Adelson and J. Y. A. Wang. 1992. Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 99--106.
[2]
T. E. Bishop, S. Zanetti, and P. Favaro. 2009. Light field superesolution. In Proceedings of the IEEE International Conference on Computational Photography (ICCP'09).
[3]
C. Buehler, M. Bosse, L. Mcmillan, S. Gortler, and M. Cohen. 2001. Unstructured lumigraph rendering. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'01). ACM Press, New York, 425--432.
[4]
E. Candes, J. Romberg, and T. Tao. 2006a. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 2, 489--509.
[5]
E. Candes, J. Romberg, and T. Tao. 2006b. Stable signal recovery incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59, 1207--1223.
[6]
A. Davis, M. Levoy, and F. Durand. 2012. Unstructured light fields. Comput. Graph. Forum 31, 2.1, 305--314.
[7]
D. Donoho. 2006. Compressed sensing. IEEE Trans. Inf. Theory 52, 4, 1289--1306.
[8]
F. Durand, N. Holzschuch, C. Soler, E. Chan, and F. X. Sillion. 2005. A frequency analysis of light transport. ACM Trans. Graph. 24, 3, 1115--1126.
[9]
K. Egan, Y.-T. Tseng, N. Holzschuch, F. Durand, and R. Ramamoorthi. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. ACM Trans. Graph. 28, 3, 93:1--93:13.
[10]
T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala. 2006. Spatio-angular resolution tradeoff in integral photography. In Proceedings of the Eurographics Symposium on Rendering (EGSR'06). 263--272.
[11]
B. Ghazi, H. Hassanieh, P. Indyk, D. Katabi, E. Price, and L. Shi. 2013. Sample-optimal average-case sparse fourier transform in two dimensions. http://people.csail.mit.edu/lixin/files/ALLER13_0252_FI.pdf.
[12]
A. Gilbert, M. Muthukrishnan, and M. Strauss. 2005. Improved time bounds for near-optimal sparse fourier representations. Proc. SPIE 5914.
[13]
J. W. Goodman. 1996. Introduction to Fourier Optics. McGraw-Hill.
[14]
S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen. 1996. The lumigraph. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). ACM Press, New York, 43--54.
[15]
H. Hassanieh, P. Indyk, D. Katabi, and E. Price. 2012a. Nearly optimal sparse fourier transform. In Proceedings of the 44th Annual ACM Symposium on Theory of Computing (STOC'12). 563--578.
[16]
H. Hassanieh, P. Indyk, D. Katabi, and E. Price. 2012b. Simple and practical algorithm for sparse fft. In Proceedings of the 23rd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA'12). 1183--1194.
[17]
P. Heckbert. 1989. Fundamentals of texture mapping and image warping. https://www.cs.cmu.edu/∼ph/texfund/texfund.pdf.
[18]
A. Levin and F. Durand. 2010. Linear view synthesis using a dimensionality gap light field prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10). 1831--1838.
[19]
A. Levin, R. Fergus, F. Durand, and W. T. Freeman. 2007. Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26, 3.
[20]
A. Levin, W. T. Freeman, and F. Durand. 2008a. Understanding camera trade-offs through a bayesian analysis of light field projections. http://people.csail.mit.edu/alevin/papers/lightfields-Levin-Freeman-Durand-ECCV08.pdf.
[21]
A. Levin, P. Sand, T. S. Cho, F. Durand, and W. T. Freeman. 2008b. Motion-invariant photography. ACM Trans. Graph. 27, 3.
[22]
A. Levin, S. W. Hasinoff, P. Green, F. Durand, and W. T. Freeman. 2009. 4d frequency analysis of computational cameras for depth of field extension. ACM Trans. Graph. 28, 3.
[23]
M. Levoy and P. Hanrahan. 1996. Light field rendering. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). ACM Press, New York, 31--42.
[24]
K. Marwah, G. Wetzstein, Y. Bando, and R. Raskar. 2013. Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Trans. Graph. 32, 4, 1--11.
[25]
D. P. Mitchell. 1991. Spectrally optimal sampling for distributed ray tracing. ACM Comput. Graph. 25, 4, 157--164.
[26]
R. Ng. 2005. Fourier slice photography. ACM Trans. Graph. 24, 735--744.
[27]
R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan. 2005. Light field photography with a hand-held plenoptic camera. https://graphics.stanford.edu/papers/lfcamera/lfcamera-150dpi.pdf.
[28]
C. Soler and F. X. Sillion. 1998. Fast calculation of soft shadow textures using convolution. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'98). 321--332.
[29]
Stanford. 2008. Stanford light field archive. http://lightfield.stanford.edu/.
[30]
A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin. 2007a. Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Trans. Graph. 26, 3, 69.
[31]
B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy. 2005. High performance imaging using large camera arrays. ACM Trans. Graph. 24, 3, 765--776.
[32]
Z. Zhang and M. Levoy. 2009. Wigner distributions and how they relate to the light field. In Proceedings of the IEEE International Conference on Computational Photography (ICCP'09). 1--10.

Cited By

View all
  • (2024)光场表征及其分辨率提升技术:文献综述及最新进展(特邀)Infrared and Laser Engineering10.3788/IRLA2024034753:9(20240347)Online publication date: 2024
  • (2024)LF2MV: Learning an Editable Meta-View Towards Light Field RepresentationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.322077330:3(1672-1684)Online publication date: Mar-2024
  • (2024)Sheared Epipolar Focus Spectrum for Dense Light Field ReconstructionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.3337516(1-15)Online publication date: 2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 34, Issue 1
November 2014
153 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/2702692
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 December 2014
Accepted: 01 May 2014
Received: 01 February 2014
Published in TOG Volume 34, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Fourier transform
  2. Light fields
  3. computational photography
  4. continuous spectrum
  5. sparse FFT

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)113
  • Downloads (Last 6 weeks)24
Reflects downloads up to 13 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)光场表征及其分辨率提升技术:文献综述及最新进展(特邀)Infrared and Laser Engineering10.3788/IRLA2024034753:9(20240347)Online publication date: 2024
  • (2024)LF2MV: Learning an Editable Meta-View Towards Light Field RepresentationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.322077330:3(1672-1684)Online publication date: Mar-2024
  • (2024)Sheared Epipolar Focus Spectrum for Dense Light Field ReconstructionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.3337516(1-15)Online publication date: 2024
  • (2024)Light Field View Synthesis Using Deformable Convolutional Neural Networks2024 Picture Coding Symposium (PCS)10.1109/PCS60826.2024.10566360(1-5)Online publication date: 12-Jun-2024
  • (2024)Incremental 3D Reconstruction through a Hybrid Explicit-and-Implicit Representation2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610868(15121-15127)Online publication date: 13-May-2024
  • (2024)Depth Guided Fast Rendering Neural Radiance Fields2024 3rd Conference on Fully Actuated System Theory and Applications (FASTA)10.1109/FASTA61401.2024.10595382(860-867)Online publication date: 10-May-2024
  • (2024)Light Field Reconstruction With Dual Features Extraction and Macro-Pixel UpsamplingIEEE Access10.1109/ACCESS.2024.344659212(121624-121634)Online publication date: 2024
  • (2024)An unmanned ground vehicle phenotyping-based method to generate three-dimensional multispectral point clouds for deciphering spatial heterogeneity in plant traitsMolecular Plant10.1016/j.molp.2024.09.00417:10(1624-1638)Online publication date: Oct-2024
  • (2024)Clear-Plenoxels: Floaters free radiance fields without neural networksKnowledge-Based Systems10.1016/j.knosys.2024.112096299(112096)Online publication date: Sep-2024
  • (2024)Adaptive pixel aggregation for joint spatial and angular super-resolution of light field imagesInformation Fusion10.1016/j.inffus.2023.102183104:COnline publication date: 1-Apr-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media