[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods

  • 3DR Review
  • Published:
3D Research

Abstract

This paper provides an introduction to photometric methods for image-based 3D shape reconstruction and a survey of photometric stereo techniques. We begin with taxonomy of active and passive shape acquisition techniques. Then we describe the methodical background of photometric 3D reconstruction, define the canonical setting of photometric stereo (Lambertian surface reflectance, parallel incident light, known illumination direction, known surface albedo, absence of cast shadows), discuss the 3D reconstruction of surfaces from local gradients, summarize the concept of the bidirectional reflectance distribution function (BRDF), and outline several important empirically and physically motivated reflectance models. We provide a detailed treatment of several generalizations of the canonical setting of photometric stereo, namely non-distant light sources, unknown illumination directions, and, in some detail, non-Lambertian surface reflectance functions. An important special case is purely specular reflections, where an extended light source allows capturing a surface that consists of perfectly specular surface patches. Linear combinations of purely Lambertian and purely specular reflectance components are favorably used for reconstructing smooth surfaces and also human skin. Nonuniform surface reflectance properties are estimated based on a simultaneous 3D reconstruction and determination of the locally variable parameters of the reflectance function based on a multitude of images. Assuming faceted surfaces, the effective resolution of the 3D reconstruction result can be increased to some extent beyond that of the underlying images. Other approaches separate specular and diffuse reflectance components based on polarization data or color information. The specular reflections can be used additionally to estimate the direction from which the surface is illuminated. Finally, we describe methods to combine photometric 3D reconstruction techniques with active and passive triangulation-based approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. Agrawal, R. Raskar, R. Chellappa (2006) What is the range of surface reconstructions from a gradient field? Proceedings of the European Conference on Computer Vision (ECCV 2006), 1(TR2006-021): 578–591.

    Google Scholar 

  2. N. Alldrin, T. Zickler, D. Kriegman (2008) Photometric stereo with non-parametric and spatiallyvarying reflectance, CVPR’08.

  3. N. G. Alldrin (2006) Reflectance estimation under natural illumination. Technical report, University of California, San Diego.

  4. N. G. Alldrin and D. J. Kriegman (2007) Toward reconstructing surfaces with arbitrary isotropic reflectance: A stratified photometric stereo approach, ICCV’07.

  5. N. G. Alldrin, S. P. Mallick, D. J. Kriegman (2007) Resolving the generalized bas-relief ambiguity by entropy minimization, CVPR’07, doi: http://dx.doi.org/10.1109/CVPR.2007.383208.

  6. S. Barsky and M. Petrou (2003) The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows, PAMI’03, 25(10):1239–1252.

    Article  Google Scholar 

  7. R. Basri, D. W. Jacobs, I. Kemelmacher (2007) Photometric stereo with general, unknown lighting, IJCV’07, 72(3):239–257.

    Article  Google Scholar 

  8. P. Beckmann and A. Spizzichino (1987) The Scattering of Electromagnetic Waves from Rough Surfaces, Number ISBN-13: 987-0890062382, Artech House Radar Library.

  9. P. N. Belhumeur, D. J. Kriegman, A. L. Yuille (1999) The bas-relief ambiguity, IJCV’99, 35(1):1040–1046, doi: http://dx.doi.org/10.1023/A:1008154927611.

    Google Scholar 

  10. M. K. Chandraker, F. Kahl, D. Kriegman (2005) Reflections on the generalized bas-relief ambiguity, CVPR’05, 1:788–795.

    Google Scholar 

  11. J. J. Clark (1992) Active photometric stereo, CVPR’92, pages 29–34, doi:http://dx.doi.org/10.1109/CVPR.1992.223231.

  12. R. L. Cook and K. E. Torrance (1981) A reflectance model for computer graphics. Proceedings of the 8th annual conference on Computer graphics and interactive techniques, 15(3):307–316, doi: http://doi.acm.org/10.1145/800224.806819.

    Google Scholar 

  13. B. L. Curless (1997) New Methods for Surface Reconstruction from Range Images. PhD thesis, Stanford University.

  14. d’Angelo and C. Wöhler (2008) Image-based 3d surface reconstruction by combination of photometric, geometric, and real-aperture models, ISPRS Journal of Photo-grammetry and Remote Sensing, 63(3):297–321, doi: http://dx.doi.org/10.1016/j.isprsjprs.2007.09.005.

    Article  Google Scholar 

  15. P. Debevec (1999) Modeling and rendering architecture from photographs, Technical report.

  16. O. Drbohlav and M. Chantler (2005) Can two specular pixels calibrate photometric stereo? ICCV’05, 2:1850–1857.

    Google Scholar 

  17. O. Drbohlav and R. Sara (2002) Specularities reduce ambiguity of uncalibrated photometric stereo, ECCV’02, 2:46–60.

    Google Scholar 

  18. R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Recognition of surface reflectance properties from a single image under unknown real-world illumination, Proceedings of the IEEE Workshop on Identifying Objects Across Variations in Lighting: Psychophysics & Computation.

  19. R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Surface reflectance estimation and natural illumination statistics, Proceedings of the IEEE Workshop on Statistical and Computational Theories of Vision.

  20. R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Estimating surface reflectance properties from images under unknown illumination, Proceedings of the SPIE Conference on Human Vision and Electronic Imaging IV, 4.

  21. R. O. Dror, T. K. Leung, E. H. Adelson, A. S. Willsky (2001) Statistics of real-world illumination, CVPR’01, 2:164–171.

    Google Scholar 

  22. J.-D. Duroua, M. Falconeb, M. Sagona (2007) Numerical methods for shape-from-shading: A new survey with benchmarks, Computer Vision and Image Understanding, 109(1).

  23. P. Fechteler, P. Eisert, J. Rurainsky (2007) Fast and high resolution 3d face scanning, ICIP’07, 3:81–84, doi:http://dx.doi.org/10.1109/ICIP.2007.4379251.

    Google Scholar 

  24. W. T. Freeman (1994) The generic viewpoint assumption in a framework for visual perception, Nature, 368:542–545.

    Article  Google Scholar 

  25. J. Garding (1992) Shape from texture for smooth curved surfaces in perspective projection. Journal of Mathematical Imaging and Vision, 2:630–638.

    Article  Google Scholar 

  26. A. S. Georghiades (2003) Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo, ICCV’03, 2:816–823.

    Google Scholar 

  27. A. S. Georghiades, P. N. Belhumeur, D. J. Kriegman (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose, PAMI’01, 23:643–660.

    Article  Google Scholar 

  28. D. B. Goldman, B. Curless, A. Hertzmann, S. Seitz (2005) Shape and spatially-varying BRDFs from photometric stereo, ICCV’05, 1:341–348.

    Google Scholar 

  29. B. Goldman, B. Curless, A. Hertzmann, S. Seitz (2010) Shape and spatially varying BRDFs from photometric stereo, PAMI’10, 32(6):1060–1071.

    Article  Google Scholar 

  30. A. Grumpe, S. Herbort, C. Wöhler (2011) 3D reconstruction of non-Lambertian surfaces with nonuniform reflectance parameters by fusion of photometrically estimated surface normal data with active range scanner data, Oldenburger 3D Tage 2011, 10.

  31. H. Hayakawa (1994) Photometric stereo under a light source with arbitrary motion. Journal of Optical Society of America A (JOSA A), 11:3079–3089, doi:10.1364/JOSAA.11.003079.

    Article  Google Scholar 

  32. C. Hernandez and G. Vogiatzis (2010) Selfcalibrating a real-time monocular 3D facial capture system, Fifth International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT2010).

  33. C. Hernandez, G. Vogiatzis, R. Cipolla (2008) Shadows in three-source photometric stereo, ECCV’08, pages 290–303, doi: http://dx.doi.org/10.1007/978-3-540-88682-2_23.

  34. C. Hernández, G. Vogiatzis, G. J. Brostow, B. Stenger, R. Cipolla (2007) Non-rigid photometric stereo with colored lights, ICCV’07.

  35. A. Hertzmann and S. M. Seitz (2003) Shape and materials by example: A photometric stereo approach, CVPR’03, 1:533–540.

    Google Scholar 

  36. A. Hertzmann and S. M. Seitz (2005) Example-based photometric stereo: Shape reconstruction with general, varying BRDFs, PAMI’05, 27(8):1254–1264.

    Article  Google Scholar 

  37. B. K. P. Horn (1970) Shape from shading: A method for obtaining the shape of a smooth opaque object from one view, Technical Report 232, MIT.

  38. B. K. P. Horn (1975) Determining Shape from Shading.

  39. B. K. P. Horn (1975) Image intensity understanding, Technical Report 335, MIT, Artificial Intelligence Laboratory.

  40. B. K. P. Horn (1977) Understanding image intensities, Artificial Intelligence, 11(2):201–231.

    Article  MathSciNet  Google Scholar 

  41. B. K. P. Horn (1989) Height and gradient from shading, Technical Report 1105A, MIT, Artificial Intelligence Laboratory.

  42. B. K. P. Horn and R. W. Sjoberg (1978) Calculating the reflectance map, Technical Report 498, MIT, Artificial Intelligence Laboratory.

  43. K. Ikeuchi (1980) Numerical shape from shading and occluding contours in a single view, Technical Report 566, MIT, Artificial Intelligence Laboratory.

  44. K. Ikeuchi (1981) Determining surface orientations of specular surfaces by using the photometric stereo method, PAMI, 3(6):661–669, doi: http://dx.doi.org/10.1109/TPAMI.1981.4767167.

    Article  Google Scholar 

  45. Ikeuchi and B. K. P. Horn (1981) Numerical shape from shading and occluding boundaries, Artificial Intelligence, 17:141–184.

    Article  Google Scholar 

  46. A. B. Israel and T. N. E. Greville (2003) Generalized Inverses Theory & Applications, Springer, 2nd edition.

  47. Y. Iwahori, H. Sugie, N. Ishii (1990) Reconstructing shape from shading images under point light source illumination, ICPR’90, 1:83–87.

    Google Scholar 

  48. N. Joshi and D. J. Kriegman (2007) Shape from varying illumination and viewpoint, ICCV’07.

  49. R. Kimmel and J. A. Sethian (2001) Optimal algorithm for shape from shading and path planning, Journal of Mathematical Imaging and Vision, 14: 237–244.

    Article  MATH  MathSciNet  Google Scholar 

  50. J.-H. Lambert (1760) Photometria, sive de mensura et gradibus luminis, colorum et umbrae, Vidae Eberhardi Klett.

  51. D. Lanman and G. Taubin (2009) Build your own 3D scanner: 3D photography for beginners, Technical Report, Brown University.

  52. J. Lawrence, A. Ben-Artzi, C. DeCoro, W. Matusik, H. Pfister, R. Ramamoorthi, S. Rusinkiewicz (2006) Inverse shade trees for non-parametric material representation and editing, ACM Transactions on Graphics (TOG’06), 25(3):735–745, doi: http://doi.acm.org/10.1145/1141911.1141949.

    Article  Google Scholar 

  53. J. Lim, J. Ho, M.-H. Yang, D. Kriegman (2005) Passive photometric stereo from motion, ICCV’05, 2:1635–1642, doi:http://dx.doi.org/10.1109/ICCV.2005.185.

    Google Scholar 

  54. S. P. Mallick, T. Zickler, D. J. Kriegman, P. N. Belhumeur (2005) Beyond Lambert: Reconstructing specular surfaces using color, CVPR’05, 1:619–626.

    Google Scholar 

  55. S. Marschner, E. P. F. Lafortune, S. H. Westin, K. E. Torrance, D. P. Greenberg (1999) Image based BRDF measurement, Applied Optics, 39:16.

    Google Scholar 

  56. S. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, D. P. Green-berg (1999) Imagebased BRDF measurement including human skin, Proceedings of 10th Eurographics Workshop on Rendering, pages 139–152.

  57. S. R. Marschner (1998) Inverse Rendering for Computer Graphics, PhD thesis, Cornell University.

  58. W. Matusik, H. Pfister, M. Brand, L. McMillan (2003) A data-driven reflectance model, ACM Transactions on Graphics, 22(3):759–769.

    Article  Google Scholar 

  59. W. Matusik, H. Pfister, M. Brand, L. McMillan (2003) Efficient isotropic BRDF measurement, 14th Eurographics Workshop on Rendering, 44:241–247.

    Google Scholar 

  60. Morel, F. Meriaudeau, C. Stolz, P. Gorria (2005) Polarization imaging applied to 3D reconstruction of specular metallic surfaces.

  61. S. K. Nayar, K. Ikeuchi, T. Kanade (1988) Extracting shape and reflectance of Lam-bertian, specular and hybrid surfaces, Technical Report CMU-FU-TR-88-14, The Robotics Institute, Carnegie Mellon University.

  62. S. K. Nayar, K. Ikeuchi, T. Kanade (1990) Determining shape and reflectance of hybrid surfaces by photometric sampling, IEEE Transactions on Robotics and Automation, 6(1):418–431.

    Article  Google Scholar 

  63. S. K. Nayar, K. Ikeuchi, T. Kanade (1990) Shape from interreflections, Technical Report CMU-RI-TR-90-14, Carnegie-Mellon University of Pittsburgh, PA, Robotics Institute.

  64. S. K. Nayar, K. Ikeuchi, T. Kanade (1991) Surface reflection: Physical and geometrical perspectives, PAMI’99, 13:611–634.

    Article  Google Scholar 

  65. S. K. Nayar, X.-S. Fang, T. Boult (1997) Separation of reflection components using color and polarization, IJCV’97, 21(3):163–186, doi: 10.1023/A:1007937815113.

    Article  Google Scholar 

  66. D. Nehab, S. Rusinkiewicz, J. Davis, R. Ramamoorthi (2005) Efficiently combining positions and normals for precise 3d geometry, SIGGRAPH’05, 24(3):536–543, doi: http://doi.acm.org/10.1145/1073204.1073226.

    Google Scholar 

  67. F. Nicodemus, J. Richmond, J. Hsia, I. Ginsberg, T. Limperis (1977) Geometrical considerations and nomenclature for reflectance, Technical report, U.S. Department of Commerce, National Bureau of Standards.

  68. E. North Coleman, Jr. and R. Jain (1982) Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry, Computer Graphics and Image Processing, 18:309–328, doi: http://dx.doi.org/10.1016/0146-664X(82)90001-6.

    Article  Google Scholar 

  69. T. Peng (2006) Algorithms and models for 3-D shape measurement using digital fringe projections, PhD thesis, University of Maryland, Department for Mechanical Engineering.

  70. R. Penrose (1955) A generalized inverse for matrices, Proceedings of the Cambridge Philosophical Society, 51:406–413.

    Article  MATH  MathSciNet  Google Scholar 

  71. B. T. Phong (1975) Illumination for computer generated pictures. Communications of the ACM, 18(6):311–317, doi: http://doi.acm.org/10.1145/360825.360839.

    Article  Google Scholar 

  72. E. Prados and O. Faugeras (2003) Perspective shape from shading and viscosity solutions, ICCV’03, 2:826–831.

    Google Scholar 

  73. R. Ramamoorthi (2002) A signal processing framework for forward and inverse rendering, PhD thesis, Stanford University.

  74. Y. Sato, M. D. Wheeler, K. Ikeuchi (1997) Object shape and reflectance modeling from observation. Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pages 379–387, doi:http://doi.acm.org/10.1145/258734.258885.

  75. M. Seitz (1999) An overview of passive vision techniques. Technical report, The Robotics Institute, Carnegie Mellon University.

  76. L. Shen and P. Tan (2009) Photometric stereo and weather estimation using internet images, CVPR’09, 1:1850–1857.

    Google Scholar 

  77. B. Shi, Y. Matsushita, Y. Wei, C. Xu, P. Tan (2010) Self-calibrating photometric stereo, CVPR’10.

  78. W. M. Silver (1980) Determining shape and reflectance using multiple images, Master’s thesis, MIT, Computer Science and Artificial Intelligence Laboratory.

  79. D. Simakov, D. Frolova, R. Basri (2003) Dense shape reconstruction of a moving object under arbitrary, unknown lighting, ICCV’03, 2:1202.

    Google Scholar 

  80. M. M. Stark, J. Arvo, B. Smits (2005) Barycentric parameterizations for isotropic BRDFs, IEEE Transactions on Visualization and Computer Graphics, 11(2):126–138, doi: http://dx.doi.org/10.1109/TVCG.2005.26.

    Article  Google Scholar 

  81. R. Szeliski (2010) Computer Vision Algorithms and Applications, online course material.

  82. P. Tan and T. Zickler (2009) A projective framework for radiometric image analysis, CVPR 2009, pages 2977–2984, doi: http://dx.doi.org/10.1109/CVPR.2009.5206731.

  83. P. Tan, S. Lin, L. Quan, H.-Y. Shum (2003) Highlight removal by illumination-constraint inpainting, ICCV’03, 1:164–169.

    Google Scholar 

  84. P. Tan, S. Lin, L. Quan (2006) Resolution-enhanced photometric stereo, ECCV’06, 3:58–71.

    Google Scholar 

  85. P. Tan, S. P. Mallick, L. Quan, D. J. Kriegman, T. Zickler (2007) Isotropy, reciprocity and the generalized bas-relief ambiguity, CVPR 2007, pages 1–8.

  86. P. Tan, S. Lin, L. Quan (2008) Subpixel photometric stereo, PAMI’08, 30(8):1460–1471.

    Google Scholar 

  87. R. T. Tan and K. Ikeuchi (2003) Separating reflection components of textured surfaces using a single image, ICCV 2003, 1:870–877.

    Google Scholar 

  88. R. T. Tan and K. Ikeuchi (2005) Separating reflection components of textured surfaces using a single image, PAMI’05, 27(2):179–193.

    Google Scholar 

  89. A. Tankus, N. Sochen, Y. Yeshurun (2005) Shapefrom-shading under perspective projection, IJCV’05, 63(1):21–43.

    Article  Google Scholar 

  90. D. Thomas and A. Sugimoto (2010) Range image registration of specular objects under complex illumination, Fifth International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT2010).

  91. C. Tomasi and T. Kanade (1992) Shape and motion from image streams under orthography: a factorization method, IJCV’92, 9(2):137–154, doi: http://dx.doi.org/10.1007/BF00129684.

    Article  Google Scholar 

  92. E. Torrance and E. M. Sparrow (1967) Theory for off-specular reflection from roughened surfaces, Journal of the Optical Society of America A (JOSA A), 57(9):1105–1114.

    Article  Google Scholar 

  93. H. von Helmholtz (1924) Handbuch der Physiologischen Optik, Optical Society of America.

  94. G. J. Ward (1992) Measuring and modeling anisotropic reflection, ACM SIGGRAPH’92, 26(2):265–272.

    Article  Google Scholar 

  95. T. Weise, B. Leibe, L. Van Gool (2007) Fast 3d scanning with automatic motion compensation, CVPR’07, pages 1–8.

  96. T. Weyrich, J. Lawrence, H. Lensch, S. Rusinkiewicz, T. Zickler (2008) Principles of appearance acquisition and representation, ACM SIGGRAPH 2008 classes, none: 1–70, doi: http://doi.acm.org/10.1145/1401132.1401234.

  97. D. R. White, P. Saunders, S. J. Bonsey, J. van de Ven, H. Edgar (1998) Reflectometer for measuring the bidirectional reflectance of rough surfaces, Applied Optics, 37(16):3450–3454, doi: doi:10.1364/AO.37.003450.

    Article  Google Scholar 

  98. C. Wöhler (2009) 3D Computer Vision — Efficient Methods and Applications, Springer, 1st edition.

  99. C. Wöhler and P. d’Angelo (2009) Stereo image analysis of non-Lambertian surfaces, IJCV’09, 81(2):529–540, doi: http://dx.doi.org/10.1007/s11263-008-0157-1.

    Google Scholar 

  100. L. B. Wolff (1989) Using polarization to separate reflection components, CVPR’89, 1(1):363–369, doi: http://dx.doi.org/10.1109/CVPR.1989.37873.

    Google Scholar 

  101. L. B. Wolff and T. E. Boult (1991) Constraining object features using a polarization reflectance model, PAMI’91, 13(7):635–657, doi:http://dx.doi.org/10.1109/34.85655.

    Article  Google Scholar 

  102. R. J. Woodham (1980) Photometric method for determining surface orientation from multiple images, Optical Engineering, 19(1):139–144.

    Google Scholar 

  103. Y. Yu, P. Debevec, J. Malik, T. Hawkins (1999) Inverse global illumination: Recovering reflectance models of real scenes from photographs, SIGGRAPH1999, pages 215–224, doi: http://doi.acm.org/10.1145/311535.311559.

  104. A. L. Yuille, J. M. Coughlan, S. Konishi (2000) The generic viewpoint constraint resolves the generalized bas relief ambiguity, Conference on Information Science and Systems.

  105. L. Zhang, B. Curless, A. Hertzmann, S. M. Seitz (2003) Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo, ICCV’03, 1:618–626.

    Google Scholar 

  106. R. Zhang, P.-S. Tsai, J. E. Cryer, M. Shah (1999) Shape from shading: A survey, PAMI’99, 21(8):690–706.

    Article  Google Scholar 

  107. Zhou and P. Tan (2010) Ring-light photometric stereo, ECCV’10, pages 1–14.

  108. T. Zickler, P. N. Belhumeur, D. J. Kriegman (2002) Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction, ECCV’02, 3:869–884.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Image Analysis Group, TU Dortmund University, Otto-Hahn-Str. 4, 44227, Dortmund, Germany

    Steffen Herbort & Christian Wöhler

Authors
  1. Steffen Herbort
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Wöhler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Wöhler.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Herbort, S., Wöhler, C. An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods. 3D Res 2, 4 (2011). https://doi.org/10.1007/3DRes.03(2011)4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/3DRes.03(2011)4

Keywords

Search

Navigation