[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Log in

Color Subspaces as Photometric Invariants

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe a class of such invariants that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from illuminant-dependent ‘subspaces’ of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, photometric stereo, material-based segmentation, and motion estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Ahmed, A., & Farag, A. (2006). A new formulation for shape from shading for non-lambertian surfaces. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1817–1824).

  • Bakshi, S., & Yang, Y.-H. (1994). Shape from shading for non-lambertian surfaces. In Proceedings of IEEE international conference on image processing (Vol. 2, pp. 130–134).

  • Barnard, K., Cardei, V., & Funt, B. (2002a). A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(9).

  • Barnard, K., Martin, L., Coath, A., & Funt, B. (2002b). A comparison of computational color constancy algorithms. II. Experiments with image data. IEEE Transactions on Image Processing, 11(9), 985–996.

    Article  Google Scholar 

  • Barron, J. L., & Klette, R. (2002). Quantitative color optical flow. In Proceedings international conference on pattern recognition (Vol. 4, pp. 251–255). Washington: IEEE Computer Society.

    Google Scholar 

  • Barsky, S., & Petrou, M. (2001). Colour photometric stereo: Simultaneous reconstruction of local gradient and colour of rough textured surfaces. In Proceedings of IEEE international conference on computer vision (pp. 600–605).

  • Barsky, S., & Petrou, M. (2003). The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1239–1252.

    Article  Google Scholar 

  • Bhat, D., & Nayar, S. (1998). Stereo and specular reflection. International Journal of Computer Vision, 26(2), 91–106.

    Article  Google Scholar 

  • Birchfield, S., & Tomasi, C. (1998). Depth discontinuities by pixel-to-pixel stereo. In Proceedings of IEEE international conference on computer vision (pp. 1073–1080).

  • Black, M. J., & Anandan, P. (1993). A framework for the robust estimation of optical flow. In Proceedings of IEEE international conference on computer vision (pp. 231–236).

  • Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9).

  • Boykov, Y., Veksler, O., & Zabih, R. (1998). Markov random fields with efficient approximations. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 648–655).

  • Brainard, D. H., & Freeman, W. T. (1997). Bayesian color constancy. Journal of Optical Society of America A, 14, 1393–1411.

    Article  Google Scholar 

  • Brelstaff, G., & Blake, A. (1988). Detecting specular reflection using lambertian constraints. In Proceedings of IEEE international conference on computer vision (pp. 297–302).

  • Coleman, E., & Jain, R. (1982). Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry. Computer Vision, Graphics and Image Processing, 18(4), 309–328.

    Article  Google Scholar 

  • Davis, J., Yang, R., & Wang, L. (2005a). BRDF invariant stereo using light transport constancy. In Proceedings of IEEE international conference on computer vision (Vol. 1).

  • Davis, J. E., Yang, R., & Wang, L. (2005b). BRDF invariant stereo using light transport constancy. In ICCV ’05: Proceedings of the tenth IEEE international conference on computer vision (ICCV’05) (Vol. 1, pp. 436–443). Washington: IEEE Computer Society.

    Chapter  Google Scholar 

  • Finlayson, G. D. (1996). Color in perspective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18, 1034–1036.

    Article  Google Scholar 

  • Finlayson, G., & Schaefer, G. (2001). Constrained dichromatic colour constancy. In Proceedings of European conference on computer vision (Vol. 1, pp. 342–358).

  • Finlayson, G. D., Hordley, S. D., & Hubel, P. M. (2001). Color by correlation: A simple, unifying framework for color constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 1209–1221.

    Article  Google Scholar 

  • Gershon, R. (1987). The use of color in computational vision. Ph.D. thesis, University of Toronto.

  • Grossberg, M. D., & Nayar, S. K. (2003). High dynamic range from multiple images: Which exposures to combine? In Proceedings of IEEE workshop on color and photometric methods in computer vision (CPMCV).

  • Healey, G. (1989). Using color for geometry-insensitive segmentation. Journal of Optical Society of America A, 6(6), 920–937.

    MathSciNet  Google Scholar 

  • Hertzmann, A., & Seitz, S. (2003). Shape and material by example: a photometric stereo approach. In Proceedings of IEEE conference on computer vision and pattern recognition.

  • Hordley, S., & Finlayson, G. (2006). Reevaluation of color constancy algorithm performance. Journal of Optical Society of America A, 23(5), 1008–1020.

    Article  Google Scholar 

  • Hordley, S. D., Finlayson, G. D., & Drew, M. S. (2002). Removing shadows from images. In Proceedings of European conference on computer vision (pp. 823–836).

  • Ikeuchi, K. (1981). Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 3(6), 661–669.

    Article  Google Scholar 

  • Jin, H., Cremers, D., Yezzi, A., & Soatto, S. (2004). Shedding light on stereoscopic segmentation. In Proceedings of IEEE conference on computer vision and pattern recognition.

  • Jin, H., Soatto, S., & Yezzi, A. J. (2005). Multi-view stereo reconstruction of dense shape and complex appearance. International Journal of Computer Vision, 63(3), 175–189.

    Article  Google Scholar 

  • Kim, J., Kolmogorov, V., & Zabih, R. (2003). Visual correspondence using energy minimization and mutual information.

  • Klinker, G., Shafer, S., & Kanade, T. (1988). The measurement of highlights in color images. International Journal of Computer Vision, 2(1), 7–32.

    Article  Google Scholar 

  • Koudelka, M., Magda, S., Belhumeur, P., & Kriegman, D. (2001). Image-based modeling and rendering of surfaces with arbitrary BRDFs. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 568–575).

  • Lee, H.-S. (1986). Method for computing the scene-illuminant chromaticity from specular highlights. Journal of Optical Society of America A, 3(10), 1694–1699.

    Google Scholar 

  • Lee, H. C., Breneman, E. J., & Schulte, C. P. (1990). Modeling light relfection for computer color vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(4), 402–409.

    Article  Google Scholar 

  • Lehmann, T. M., & Palm, C. (2001). Color line search for illuminant estimation in real-world scenes. Journal of Optical Society of America A, 18(11), 2679–2691.

    Article  Google Scholar 

  • Li, Y., Lin, S., Lu, H., Kang, S. B., & Shum, H.-Y. (2002). Multibaseline stereo in the presence of specular reflections. In ICPR ’02: Proceedings of the 16th international conference on pattern recognition (ICPR’02) (Vol. 3, p. 30573). Washington: IEEE Computer Society.

    Google Scholar 

  • Lim, J., Ho, J., Yang, M.-H., & Kriegman, D. (2005). Passive photometric stereo from motion. In Proceedings of IEEE international conference on computer vision.

  • Lin, S., Li, Y., Kang, S. B., Tong, X., & Shum, H.-Y. (2002). Diffuse-specular separation and depth recovery from image sequences. In ECCV ’02: proceedings of the 7th European conference on computer vision—Part III (pp. 210–224). London: Springer.

    Google Scholar 

  • Lu, J., & Little, J. (1999). Reflectance and shape from images using a collinear light source. International Journal of Computer Vision, 32(3), 1–28.

    Article  Google Scholar 

  • Magda, S., Kriegman, D., Zickler, T., & Belhumeur, P. (2001). Beyond Lambert: Reconstructing surfaces with arbitrary BRDFs. In Proceedings of IEEE international conference on computer vision (pp. 391–398).

  • Mallick, S. P., Zickler, T. E., Belhumeur, P. N., & Kriegman, D. J. (2006). Specularity removal in images and videos: A PDE approach. In Proceedings of European conference on computer vision.

  • Narasimhan, S. G., Ramesh, V., & Nayar, S. K. (2003). A class of photometric invariants: Separating material from shape and illumination. In Proceedings of IEEE international conference on computer vision (Vol. 2, pp. 1387–1394).

  • Nayar, S. K., & Bolle, M. (1996). Reflectance based object recognition. International Journal of Computer Vision, 17(3), 219–240.

    Article  Google Scholar 

  • Nayar, S., Ikeuchi, K., & Kanade, T. (1990). Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE Journal of Robotics and Automation, 6(4), 418–431.

    Article  Google Scholar 

  • Nayar, S., Fang, X., & Boult, T. (1997). Separation of reflection components using color and polarization. International Journal of Computer Vision, 21(3), 163–186.

    Article  Google Scholar 

  • Park, J. B. (2003). Efficient color representation for image segmentation under nonwhite illumination. In SPIE (Vol. 5267, pp. 163–174).

  • Ragheb, H., & Hancock, E. (2001). Separating lambertian and specular reflectance components using iterated conditional modes. In Proceedings of British machine vision conference (pp. 541–522).

  • Rosenberg, C., Hebert, M., & Thrun, S. (2001). Color constancy using KL-divergence. In Proceedings of IEEE international conference on computer vision (pp. 239–247).

  • Sapiro, G. (1999). Color and illumination voting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 1210–1215.

    Article  Google Scholar 

  • Sato, Y., & Ikeutchi, K. (1994). Temporal-color space analysis of reflection. Journal of Optical Society of America A, 11(11), 2990–3002.

    Google Scholar 

  • Schlüns, K., & Wittig, O. (1993). Photometric stereo for non-Lambertian surfaces using color information. In Proceedings of international conference on image analysis and processing (pp. 505–512).

  • Shafer, S. (1985). Using color to separate reflection components. COLOR Research and Applications, 10(4), 210–218.

    Article  Google Scholar 

  • Silver, W. (1980). Determining shape and reflectance using multiple images. Master’s thesis, MIT.

  • Tagare, H., & deFigueiredo, R. (1991). A theory of photometric stereo for a class of diffuse non-lambertian surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(2), 133–152.

    Article  Google Scholar 

  • Tan, P., Lin, S., & Quan, L. (2006). Separation of highlight reflections on textured surfaces. In Proceedings of IEEE conference on computer vision and pattern recognition.

  • Tan, R. T., & Ikeuchi, K. (2003). Separating reflection components of textured surface using a single image. In Proceedings of IEEE international conference on computer vision (pp. 870–877).

  • Tan, R. T., Nishino, K., & Ikeutchi, K. (2004). Color constancy through inverse-intensity chromaticity space. Journal of Optical Society of America A, 21(3), 321–334.

    Article  Google Scholar 

  • Tian, Y., & Tsui, H. (1997). Shape recovery from a color image for non-lambertian surfaces. Journal of Optical Society of America A, 14(2), 397–404.

    Article  Google Scholar 

  • Tominga, S., & Wandell, B. (1989). Standard surface-reflectance model and illuminant estimation. Journal of Optical Society of America A, 6(4), 576–584.

    Google Scholar 

  • Tominga, S., & Wandell, B. A. (2002). Natural scene-illuminant estimation using sensor correlation. Proceedings of IEEE, 90, 42–56.

    Article  Google Scholar 

  • Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., & Miyake, Y. (2003). Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. In Proceedings of ACM SIGGRAPH (pp. 770–779).

  • Tu, P., & Mendonça, P. (2003). Surface reconstruction via Helmholtz reciprocity with a single image pair. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 541–547).

  • van de Weijer, J., & Gevers, T. (2004). Robust optical flow from photometric invariants. In Proceedings of IEEE international conference on image processing (pp. 1835–1838).

  • Wann Jensen, H., Marschner, S., Levoy, M., & Hanrahan, P. (2001). A practical model for subsurface light transport. In Proceedings of ACM SIGGRAPH (pp. 511–518).

  • Wolff, L., & Angelopoulou, E. (1994). Three-dimensional stereo by photometric ratios. Journal of Optical Society of America A, 11, 3069–3078.

    Google Scholar 

  • Wolff, L. B., & Boult, T. E. (1991). Constraining object features using a polarization reflectance model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 635–657.

    Article  Google Scholar 

  • Woodham, R. (1978). Photometric stereo: A reflectance map technique for determining surface orientation from image intesity. In Proceedings of SPIE (Vol. 155, pp. 136–143).

  • Yang, R., Pollefeys, M., & Welch, G. (2003). Dealing with textureless regions and specular highlights-a progressive space carving scheme using a novel photo-consistency measure. In ICCV ’03: Proceedings of the ninth IEEE international conference on computer vision (p. 576). Washington: IEEE Computer Society.

    Chapter  Google Scholar 

  • Yoon, K., & Kweon, I. (2006a). Correspondence search in the presence of specular highlights using specular-free two-band images. In Proceedings of Asian conference on computer vision (pp. 761–770).

  • Yoon, K.-J., & Kweon, I. S. (2006b). Adaptive support-weight approach for correspondence search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4), 650–656.

    Article  Google Scholar 

  • Zhang, L., Curless, B., Hertzmann, A., & Seitz, S. M. (2003). Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. In Proceedings of IEEE international conference on computer vision (pp. 618–625).

  • Zheng, Q., & Chellappa, R. (1991). Estimation of illuminant direction, albedo, and shape from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 680–702.

    Article  Google Scholar 

  • Zickler, T., Belhumeur, P., & Kriegman, D. (2002). Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction. In Proceedings of European conference on computer vision (pp. 869–884).

  • Zickler, T. E., Ho, J., Kriegman, D. J., Ponce, J., & Belhumeur, P. N. (2003). Binocular helmholtz stereopsis. In Proceedings of IEEE international conference on computer vision (p. 1411). Washington: IEEE Computer Society.

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Todd Zickler.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zickler, T., Mallick, S.P., Kriegman, D.J. et al. Color Subspaces as Photometric Invariants. Int J Comput Vis 79, 13–30 (2008). https://doi.org/10.1007/s11263-007-0087-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-007-0087-3

Keywords

Navigation