Aperture-Aware Lens Design
Article No.: 117, Pages 1 - 10
Abstract
Optics designers use simulation tools to assist them in designing lenses for various applications. Commercial tools rely on finite differencing and sampling methods to perform gradient-based optimization of lens design objectives. Recently, differentiable rendering techniques have enabled more efficient gradient calculation of these objectives. However, these techniques are unable to optimize for light throughput, often an important metric for many applications.
We develop a method for calculating the gradients of optical systems with respect to both focus and light throughput. We formulate lens performance as an integral loss over a dynamic domain, which allows for the use of differentiable rendering techniques to calculate the required gradients. We also develop a ray tracer specifically designed for refractive lenses and derive formulas for calculating gradients that simultaneously optimize for focus and light throughput. Explicitly optimizing for light throughput produces lenses that outperform traditional optimized lenses that tend to prioritize for only focus. To evaluate our lens designs, we simulate various applications where our lenses:
Supplemental Material
MP4 File - presentation
presentation
- Download
- 327.29 MB
References
[1]
Simon Baker and Shree K Nayar. 1999. A theory of single-viewpoint catadioptric image formation. International journal of computer vision 35 (1999), 175–196.
[2]
Sai Praveen Bangaru, Michael Gharbi, Fujun Luan, Tzu-Mao Li, Kalyan Sunkavalli, Milos Hasan, Sai Bi, Zexiang Xu, Gilbert Bernstein, and Fredo Durand. 2022. Differentiable Rendering of Neural SDFs through Reparameterization. In SIGGRAPH Asia 2022 Conference Papers(SA ’22). Article 5, 9 pages. https://doi.org/10.1145/3550469.3555397
[3]
Sai Praveen Bangaru, Tzu-Mao Li, and Frédo Durand. 2020. Unbiased warped-area sampling for differentiable rendering. ACM Trans. Graph. 39, 6, Article 245 (nov 2020), 18 pages. https://doi.org/10.1145/3414685.3417833
[4]
Max Born and Emil Wolf. 2013. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light. Elsevier.
[5]
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. 2018. JAX: composable transformations of Python+NumPy programs. http://github.com/google/jax
[6]
Guangyan Cai, Kai Yan, Zhao Dong, Ioannis Gkioulekas, and Shuang Zhao. 2022. Physics-Based Inverse Rendering using Combined Implicit and Explicit Geometries. Computer Graphics Forum 41, 4 (2022).
[7]
Praneeth Chakravarthula, Jipeng Sun, Xiao Li, Chenyang Lei, Gene Chou, Mario Bijelic, Johannes Froesch, Arka Majumdar, and Felix Heide. 2023. Thin On-Sensor Nanophotonic Array Cameras. ACM Trans. Graph. 42, 6, Article 249 (dec 2023), 18 pages. https://doi.org/10.1145/3618398
[8]
Guy Chavent. 1974. Identification of Functional Parameters in Partial Differential Equations. Joint Automatic Control Conference.
[9]
Min Chen and James Arvo. 2000. Theory and application of specular path perturbation. ACM Trans. Graph. 19, 4 (oct 2000), 246–278. https://doi.org/10.1145/380666.380670
[10]
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. 2018. Neural Ordinary Differential Equations. NeurIPS (2018).
[11]
Roger N. Clark. 2023. ClarkVision.com. https://clarkvision.com/articles/digital.sensor.performance.summary/index.html Accessed: 2024-13-1.
[12]
Blender Online Community. 2023. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam. http://www.blender.org
[13]
Geoffroi Côté, Jean-François Lalonde, and Simon Thibault. 2021. Deep learning-enabled framework for automatic lens design starting point generation. Opt. Express 29, 3 (Feb 2021), 3841–3854. https://doi.org/10.1364/OE.401590
[14]
Arthur Cox. 1974. Photographic optics: a modern approach to the technique of definition. Focal Press.
[15]
Gerwin Damberg, James Gregson, and Wolfgang Heidrich. 2016. High Brightness HDR Projection Using Dynamic Freeform Lensing. ACM Trans. Graph. 35, 3, Article 24 (may 2016), 11 pages. https://doi.org/10.1145/2857051
[16]
Gerwin Damberg and Wolfgang Heidrich. 2015. Efficient freeform lens optimization for computational caustic displays. Opt. Express 23, 8 (Apr 2015), 10224–10232. https://doi.org/10.1364/OE.23.010224
[17]
Jonathan P Gardner, John C Mather, Mark Clampin, Rene Doyon, Matthew A Greenhouse, Heidi B Hammel, John B Hutchings, Peter Jakobsen, Simon J Lilly, Knox S Long, 2006. The james webb space telescope. Space Science Reviews 123 (2006), 485–606.
[18]
Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz Bächer, Bernhard Thomaszewski, and Stelian Coros. 2020. ADD: analytically differentiable dynamics for multi-body systems with frictional contact. ACM Trans. Graph. 39, 6, Article 190 (nov 2020), 15 pages. https://doi.org/10.1145/3414685.3417766
[19]
Andreas Griewank and Andrea Walther. 2008. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. SIAM review.
[20]
Samuel W Hasinoff, Frédo Durand, and William T Freeman. 2010. Noise-optimal capture for high dynamic range photography. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 553–560.
[21]
Michael Hinze, René Pinnau, Michael Ulbrich, and Stefan Ulbrich. 2008. Optimization with PDE constraints. Springer.
[22]
Matthias B. Hullin, Johannes Hanika, and Wolfgang Heidrich. 2012. Polynomial Optics: A Construction Kit for Efficient Ray-Tracing of Lens Systems. Computer Graphics Forum (Proceedings of EGSR 2012) 31, 4 (July 2012).
[23]
Wenzel Jakob and Steve Marschner. 2012. Manifold exploration: a Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport. ACM Trans. Graph. 31, 4, Article 58 (jul 2012), 13 pages. https://doi.org/10.1145/2185520.2185554
[24]
Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, and Delio Vicini. 2022. DR.JIT: a just-in-time compiler for differentiable rendering. ACM Trans. Graph. 41, 4, Article 124 (jul 2022), 19 pages. https://doi.org/10.1145/3528223.3530099
[25]
Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization. arxiv:1412.6980 [cs.LG]
[26]
Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018a. Differentiable Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. 37, 6, Article 222 (dec 2018), 11 pages. https://doi.org/10.1145/3272127.3275109
[27]
Tzu-Mao Li, Michaël Gharbi, Andrew Adams, Frédo Durand, and Jonathan Ragan-Kelley. 2018b. Differentiable programming for image processing and deep learning in halide. ACM Trans. Graph. 37, 4, Article 139 (jul 2018), 13 pages. https://doi.org/10.1145/3197517.3201383
[28]
Zhengqin Li, Yu-Ying Yeh, and Manmohan Chandraker. 2020. Through the looking glass: Neural 3d reconstruction of transparent shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1262–1271.
[29]
Antoine McNamara, Adrien Treuille, Zoran Popović, and Jos Stam. 2004. Fluid control using the adjoint method. ACM Trans. Graph. 23, 3 (aug 2004), 449–456. https://doi.org/10.1145/1015706.1015744
[30]
Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. 2020. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3504–3515.
[31]
Merlin Nimier-David, Sébastien Speierer, Benoît Ruiz, and Wenzel Jakob. 2020. Radiative backpropagation: an adjoint method for lightning-fast differentiable rendering. ACM Trans. Graph. 39, 4, Article 146 (aug 2020), 15 pages. https://doi.org/10.1145/3386569.3392406
[32]
Marios Papas, Wojciech Jarosz, Wenzel Jakob, Szymon Rusinkiewicz, Wojciech Matusik, and Tim Weyrich. 2011. Goal-Based Caustics. Computer Graphics Forum (Proceedings of Eurographics) 30, 2 (June 2011), 503–511. https://doi.org/10.1111/j.1467-8659.2011.01876.x
[33]
Frank L Pedrotti, Leno M Pedrotti, and Leno S Pedrotti. 2017. Introduction to optics. Cambridge University Press.
[34]
Yingsi Qin, Wei-Yu Chen, Matthew O’Toole, and Aswin C. Sankaranarayanan. 2023. Split-Lohmann Multifocal Displays. ACM Trans. Graph. 42, 4, Article 57 (jul 2023), 18 pages. https://doi.org/10.1145/3592110
[35]
Daniel Reiley. 2014. Lens Designs. https://www.lens-designs.com/ Accessed: 2024-04-18.
[36]
Osborne Reynolds. 1903. Papers on Mechanical and Physical Subjects: The sub-mechanics of the universe. Vol. 3. The University Press.
[37]
Paul Rudolph. U.S. Patent 583 336, May. 1897. Object Glass.
[38]
Adriana Schulz, Harrison Wang, Eitan Grinspun, Justin Solomon, and Wojciech Matusik. 2018. Interactive exploration of design trade-offs. ACM Trans. Graph. 37, 4, Article 131 (jul 2018), 14 pages. https://doi.org/10.1145/3197517.3201385
[39]
Yuliy Schwartzburg, Romain Testuz, Andrea Tagliasacchi, and Mark Pauly. 2014. High-contrast computational caustic design. ACM Trans. Graph. 33, 4, Article 74 (jul 2014), 11 pages. https://doi.org/10.1145/2601097.2601200
[40]
Nicholas Sharp and Keenan Crane. 2018. Variational surface cutting. ACM Trans. Graph. 37, 4, Article 156 (jul 2018), 13 pages. https://doi.org/10.1145/3197517.3201356
[41]
Warren J Smith. 2008. Modern optical engineering: the design of optical systems. McGraw-Hill Education.
[42]
Jos Stam. 2020. Computing Light Transport Gradients using the Adjoint Method. arXiv:2006.15059 (2020).
[43]
Shlomi Steinberg, Nandor Bokor, and Nir Davidson. 2022. Two-mirror compact system for ideal concentration of diffuse light. Journal of the Optical Society of America A 39, 4 (Mar 2022), 628. https://doi.org/10.1364/josaa.447493
[44]
Shlomi Steinberg and Ling-Qi Yan. 2021. A generic framework for physical light transport. ACM Trans. Graph. 40, 4, Article 139 (jul 2021), 20 pages. https://doi.org/10.1145/3450626.3459791
[45]
Libin Sun, Brian Guenter, Neel Joshi, Patrick Therien, and James Hays. 2015. Lens Factory: Automatic Lens Generation Using Off-the-shelf Components. arxiv:1506.08956 [cs.GR]
[46]
Synopsis. 2023. Code V Optical Design Software. https://www.synopsys.com/optical-solutions/codev.html
[47]
Huixuan Tang and Kiriakos N. Kutulakos. 2013. What does an aberrated photo tell us about the lens and the scene?. In IEEE International Conference on Computational Photography (ICCP). 1–10. https://doi.org/10.1109/ICCPhot.2013.6528316
[48]
Arjun Teh, Matthew O’Toole, and Ioannis Gkioulekas. 2022. Adjoint nonlinear ray tracing. ACM Trans. Graph. 41, 4, Article 126 (jul 2022), 13 pages. https://doi.org/10.1145/3528223.3530077
[49]
Ethan Tseng, Ali Mosleh, Fahim Mannan, Karl St-Arnaud, Avinash Sharma, Yifan Peng, Alexander Braun, Derek Nowrouzezahrai, Jean-François Lalonde, and Felix Heide. 2021. Differentiable Compound Optics and Processing Pipeline Optimization for End-to-end Camera Design. ACM Trans. Graph. 40, 2, Article 18 (jun 2021), 19 pages. https://doi.org/10.1145/3446791
[50]
Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2021. Path replay backpropagation: differentiating light paths using constant memory and linear time. ACM Trans. Graph. 40, 4, Article 108 (jul 2021), 14 pages. https://doi.org/10.1145/3450626.3459804
[51]
Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2022. Differentiable signed distance function rendering. ACM Trans. Graph. 41, 4, Article 125 (jul 2022), 18 pages. https://doi.org/10.1145/3528223.3530139
[52]
Bruce Walter, Shuang Zhao, Nicolas Holzschuch, and Kavita Bala. 2009. Single scattering in refractive media with triangle mesh boundaries. In ACM SIGGRAPH 2009 Papers (New Orleans, Louisiana) (SIGGRAPH ’09). Article 92, 8 pages. https://doi.org/10.1145/1576246.1531398
[53]
Congli Wang, Ni Chen, and Wolfgang Heidrich. 2022. dO: A differentiable engine for Deep Lens design of computational imaging systems. IEEE Transactions on Computational Imaging 8 (2022), 905–916.
[54]
Raymond N Wilson. 2013. Reflecting telescope optics II: manufacture, testing, alignment, Modern Techniques. Springer Science & Business Media.
[55]
Tizian Zeltner, Iliyan Georgiev, and Wenzel Jakob. 2020. Specular manifold sampling for rendering high-frequency caustics and glints. ACM Trans. Graph. 39, 4, Article 149 (aug 2020), 15 pages. https://doi.org/10.1145/3386569.3392408
[56]
Zemax. 2023. OpticStudio. https://www.ansys.com/products/optics/ansys-zemax-opticstudio
[57]
Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020. Path-space differentiable rendering. ACM Trans. Graph. 39, 4, Article 143 (aug 2020), 19 pages. https://doi.org/10.1145/3386569.3392383
[58]
Ziyi Zhang, Nicolas Roussel, and Wenzel Jakob. 2023. Projective Sampling for Differentiable Rendering of Geometry. ACM Trans. Graph. 42, 6, Article 212 (dec 2023), 14 pages. https://doi.org/10.1145/3618385
Index Terms
- Aperture-Aware Lens Design
Recommendations
Shape and topology optimization of acoustic lens system using phase field method
A layout optimization method for a two-dimensional acoustic lens system used in underwater imaging is presented. To this end, a shape and topology optimization is formulated for the design problem of a lens system for the first time. The layout of a ...
A hybrid artificial intelligence system for optical lens design
The current approach to optical lens design still relies very much on the lens designers' experience and knowledge. In the initial lens design, it is normally performed by lens designers and then is optimised by using some optimisation techniques. This ...
Application of hybrid learning algorithm for optimization of LED lens design
AbstractIn this study was performed optimization procedure of LED lens design by adaptive neuro fuzzy inference system (ANFIS). There are two objective functions in the optimization procedure: viewing angle and luminance uniformity. Optical design ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
1106 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
SIGGRAPH '24
Sponsor:
SIGGRAPH '24: Special Interest Group on Computer Graphics and Interactive Techniques Conference
July 27 - August 1, 2024
CO, Denver, USA
Acceptance Rates
Overall Acceptance Rate 1,822 of 8,601 submissions, 21%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 547Total Downloads
- Downloads (Last 12 months)547
- Downloads (Last 6 weeks)141
Reflects downloads up to 11 Dec 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in