Abstract
Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe a class of such invariants that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from illuminant-dependent ‘subspaces’ of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, photometric stereo, material-based segmentation, and motion estimation.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ahmed, A., & Farag, A. (2006). A new formulation for shape from shading for non-lambertian surfaces. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1817–1824).
Bakshi, S., & Yang, Y.-H. (1994). Shape from shading for non-lambertian surfaces. In Proceedings of IEEE international conference on image processing (Vol. 2, pp. 130–134).
Barnard, K., Cardei, V., & Funt, B. (2002a). A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(9).
Barnard, K., Martin, L., Coath, A., & Funt, B. (2002b). A comparison of computational color constancy algorithms. II. Experiments with image data. IEEE Transactions on Image Processing, 11(9), 985–996.
Barron, J. L., & Klette, R. (2002). Quantitative color optical flow. In Proceedings international conference on pattern recognition (Vol. 4, pp. 251–255). Washington: IEEE Computer Society.
Barsky, S., & Petrou, M. (2001). Colour photometric stereo: Simultaneous reconstruction of local gradient and colour of rough textured surfaces. In Proceedings of IEEE international conference on computer vision (pp. 600–605).
Barsky, S., & Petrou, M. (2003). The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1239–1252.
Bhat, D., & Nayar, S. (1998). Stereo and specular reflection. International Journal of Computer Vision, 26(2), 91–106.
Birchfield, S., & Tomasi, C. (1998). Depth discontinuities by pixel-to-pixel stereo. In Proceedings of IEEE international conference on computer vision (pp. 1073–1080).
Black, M. J., & Anandan, P. (1993). A framework for the robust estimation of optical flow. In Proceedings of IEEE international conference on computer vision (pp. 231–236).
Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9).
Boykov, Y., Veksler, O., & Zabih, R. (1998). Markov random fields with efficient approximations. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 648–655).
Brainard, D. H., & Freeman, W. T. (1997). Bayesian color constancy. Journal of Optical Society of America A, 14, 1393–1411.
Brelstaff, G., & Blake, A. (1988). Detecting specular reflection using lambertian constraints. In Proceedings of IEEE international conference on computer vision (pp. 297–302).
Coleman, E., & Jain, R. (1982). Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry. Computer Vision, Graphics and Image Processing, 18(4), 309–328.
Davis, J., Yang, R., & Wang, L. (2005a). BRDF invariant stereo using light transport constancy. In Proceedings of IEEE international conference on computer vision (Vol. 1).
Davis, J. E., Yang, R., & Wang, L. (2005b). BRDF invariant stereo using light transport constancy. In ICCV ’05: Proceedings of the tenth IEEE international conference on computer vision (ICCV’05) (Vol. 1, pp. 436–443). Washington: IEEE Computer Society.
Finlayson, G. D. (1996). Color in perspective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18, 1034–1036.
Finlayson, G., & Schaefer, G. (2001). Constrained dichromatic colour constancy. In Proceedings of European conference on computer vision (Vol. 1, pp. 342–358).
Finlayson, G. D., Hordley, S. D., & Hubel, P. M. (2001). Color by correlation: A simple, unifying framework for color constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 1209–1221.
Gershon, R. (1987). The use of color in computational vision. Ph.D. thesis, University of Toronto.
Grossberg, M. D., & Nayar, S. K. (2003). High dynamic range from multiple images: Which exposures to combine? In Proceedings of IEEE workshop on color and photometric methods in computer vision (CPMCV).
Healey, G. (1989). Using color for geometry-insensitive segmentation. Journal of Optical Society of America A, 6(6), 920–937.
Hertzmann, A., & Seitz, S. (2003). Shape and material by example: a photometric stereo approach. In Proceedings of IEEE conference on computer vision and pattern recognition.
Hordley, S., & Finlayson, G. (2006). Reevaluation of color constancy algorithm performance. Journal of Optical Society of America A, 23(5), 1008–1020.
Hordley, S. D., Finlayson, G. D., & Drew, M. S. (2002). Removing shadows from images. In Proceedings of European conference on computer vision (pp. 823–836).
Ikeuchi, K. (1981). Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 3(6), 661–669.
Jin, H., Cremers, D., Yezzi, A., & Soatto, S. (2004). Shedding light on stereoscopic segmentation. In Proceedings of IEEE conference on computer vision and pattern recognition.
Jin, H., Soatto, S., & Yezzi, A. J. (2005). Multi-view stereo reconstruction of dense shape and complex appearance. International Journal of Computer Vision, 63(3), 175–189.
Kim, J., Kolmogorov, V., & Zabih, R. (2003). Visual correspondence using energy minimization and mutual information.
Klinker, G., Shafer, S., & Kanade, T. (1988). The measurement of highlights in color images. International Journal of Computer Vision, 2(1), 7–32.
Koudelka, M., Magda, S., Belhumeur, P., & Kriegman, D. (2001). Image-based modeling and rendering of surfaces with arbitrary BRDFs. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 568–575).
Lee, H.-S. (1986). Method for computing the scene-illuminant chromaticity from specular highlights. Journal of Optical Society of America A, 3(10), 1694–1699.
Lee, H. C., Breneman, E. J., & Schulte, C. P. (1990). Modeling light relfection for computer color vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(4), 402–409.
Lehmann, T. M., & Palm, C. (2001). Color line search for illuminant estimation in real-world scenes. Journal of Optical Society of America A, 18(11), 2679–2691.
Li, Y., Lin, S., Lu, H., Kang, S. B., & Shum, H.-Y. (2002). Multibaseline stereo in the presence of specular reflections. In ICPR ’02: Proceedings of the 16th international conference on pattern recognition (ICPR’02) (Vol. 3, p. 30573). Washington: IEEE Computer Society.
Lim, J., Ho, J., Yang, M.-H., & Kriegman, D. (2005). Passive photometric stereo from motion. In Proceedings of IEEE international conference on computer vision.
Lin, S., Li, Y., Kang, S. B., Tong, X., & Shum, H.-Y. (2002). Diffuse-specular separation and depth recovery from image sequences. In ECCV ’02: proceedings of the 7th European conference on computer vision—Part III (pp. 210–224). London: Springer.
Lu, J., & Little, J. (1999). Reflectance and shape from images using a collinear light source. International Journal of Computer Vision, 32(3), 1–28.
Magda, S., Kriegman, D., Zickler, T., & Belhumeur, P. (2001). Beyond Lambert: Reconstructing surfaces with arbitrary BRDFs. In Proceedings of IEEE international conference on computer vision (pp. 391–398).
Mallick, S. P., Zickler, T. E., Belhumeur, P. N., & Kriegman, D. J. (2006). Specularity removal in images and videos: A PDE approach. In Proceedings of European conference on computer vision.
Narasimhan, S. G., Ramesh, V., & Nayar, S. K. (2003). A class of photometric invariants: Separating material from shape and illumination. In Proceedings of IEEE international conference on computer vision (Vol. 2, pp. 1387–1394).
Nayar, S. K., & Bolle, M. (1996). Reflectance based object recognition. International Journal of Computer Vision, 17(3), 219–240.
Nayar, S., Ikeuchi, K., & Kanade, T. (1990). Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE Journal of Robotics and Automation, 6(4), 418–431.
Nayar, S., Fang, X., & Boult, T. (1997). Separation of reflection components using color and polarization. International Journal of Computer Vision, 21(3), 163–186.
Park, J. B. (2003). Efficient color representation for image segmentation under nonwhite illumination. In SPIE (Vol. 5267, pp. 163–174).
Ragheb, H., & Hancock, E. (2001). Separating lambertian and specular reflectance components using iterated conditional modes. In Proceedings of British machine vision conference (pp. 541–522).
Rosenberg, C., Hebert, M., & Thrun, S. (2001). Color constancy using KL-divergence. In Proceedings of IEEE international conference on computer vision (pp. 239–247).
Sapiro, G. (1999). Color and illumination voting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 1210–1215.
Sato, Y., & Ikeutchi, K. (1994). Temporal-color space analysis of reflection. Journal of Optical Society of America A, 11(11), 2990–3002.
Schlüns, K., & Wittig, O. (1993). Photometric stereo for non-Lambertian surfaces using color information. In Proceedings of international conference on image analysis and processing (pp. 505–512).
Shafer, S. (1985). Using color to separate reflection components. COLOR Research and Applications, 10(4), 210–218.
Silver, W. (1980). Determining shape and reflectance using multiple images. Master’s thesis, MIT.
Tagare, H., & deFigueiredo, R. (1991). A theory of photometric stereo for a class of diffuse non-lambertian surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(2), 133–152.
Tan, P., Lin, S., & Quan, L. (2006). Separation of highlight reflections on textured surfaces. In Proceedings of IEEE conference on computer vision and pattern recognition.
Tan, R. T., & Ikeuchi, K. (2003). Separating reflection components of textured surface using a single image. In Proceedings of IEEE international conference on computer vision (pp. 870–877).
Tan, R. T., Nishino, K., & Ikeutchi, K. (2004). Color constancy through inverse-intensity chromaticity space. Journal of Optical Society of America A, 21(3), 321–334.
Tian, Y., & Tsui, H. (1997). Shape recovery from a color image for non-lambertian surfaces. Journal of Optical Society of America A, 14(2), 397–404.
Tominga, S., & Wandell, B. (1989). Standard surface-reflectance model and illuminant estimation. Journal of Optical Society of America A, 6(4), 576–584.
Tominga, S., & Wandell, B. A. (2002). Natural scene-illuminant estimation using sensor correlation. Proceedings of IEEE, 90, 42–56.
Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., & Miyake, Y. (2003). Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. In Proceedings of ACM SIGGRAPH (pp. 770–779).
Tu, P., & Mendonça, P. (2003). Surface reconstruction via Helmholtz reciprocity with a single image pair. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 541–547).
van de Weijer, J., & Gevers, T. (2004). Robust optical flow from photometric invariants. In Proceedings of IEEE international conference on image processing (pp. 1835–1838).
Wann Jensen, H., Marschner, S., Levoy, M., & Hanrahan, P. (2001). A practical model for subsurface light transport. In Proceedings of ACM SIGGRAPH (pp. 511–518).
Wolff, L., & Angelopoulou, E. (1994). Three-dimensional stereo by photometric ratios. Journal of Optical Society of America A, 11, 3069–3078.
Wolff, L. B., & Boult, T. E. (1991). Constraining object features using a polarization reflectance model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 635–657.
Woodham, R. (1978). Photometric stereo: A reflectance map technique for determining surface orientation from image intesity. In Proceedings of SPIE (Vol. 155, pp. 136–143).
Yang, R., Pollefeys, M., & Welch, G. (2003). Dealing with textureless regions and specular highlights-a progressive space carving scheme using a novel photo-consistency measure. In ICCV ’03: Proceedings of the ninth IEEE international conference on computer vision (p. 576). Washington: IEEE Computer Society.
Yoon, K., & Kweon, I. (2006a). Correspondence search in the presence of specular highlights using specular-free two-band images. In Proceedings of Asian conference on computer vision (pp. 761–770).
Yoon, K.-J., & Kweon, I. S. (2006b). Adaptive support-weight approach for correspondence search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4), 650–656.
Zhang, L., Curless, B., Hertzmann, A., & Seitz, S. M. (2003). Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. In Proceedings of IEEE international conference on computer vision (pp. 618–625).
Zheng, Q., & Chellappa, R. (1991). Estimation of illuminant direction, albedo, and shape from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 680–702.
Zickler, T., Belhumeur, P., & Kriegman, D. (2002). Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction. In Proceedings of European conference on computer vision (pp. 869–884).
Zickler, T. E., Ho, J., Kriegman, D. J., Ponce, J., & Belhumeur, P. N. (2003). Binocular helmholtz stereopsis. In Proceedings of IEEE international conference on computer vision (p. 1411). Washington: IEEE Computer Society.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zickler, T., Mallick, S.P., Kriegman, D.J. et al. Color Subspaces as Photometric Invariants. Int J Comput Vis 79, 13–30 (2008). https://doi.org/10.1007/s11263-007-0087-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0087-3