More Web Proxy on the site http://driver.im/

article

Camera Spectral Sensitivity and White Balance Estimation from Sky Images

Authors:

Katsushi IkeuchiAuthors Info & Claims

International Journal of Computer Vision, Volume 105, Issue 3

Pages 187 - 204

https://doi.org/10.1007/s11263-013-0632-1

Published: 01 December 2013 Publication History

Abstract

Photometric camera calibration is often required in physics-based computer vision. There have been a number of studies to estimate camera response functions (gamma function), and vignetting effect from images. However less attention has been paid to camera spectral sensitivities and white balance settings. This is unfortunate, since those two properties significantly affect image colors. Motivated by this, a method to estimate camera spectral sensitivities and white balance setting jointly from images with sky regions is introduced. The basic idea is to use the sky regions to infer the sky spectra. Given sky images as the input and assuming the sun direction with respect to the camera viewing direction can be extracted, the proposed method estimates the turbidity of the sky by fitting the image intensities to a sky model. Subsequently, it calculates the sky spectra from the estimated turbidity. Having the sky $$RGB$$ RGB values and their corresponding spectra, the method estimates the camera spectral sensitivities together with the white balance setting. Precomputed basis functions of camera spectral sensitivities are used in the method for robust estimation. The whole method is novel and practical since, unlike existing methods, it uses sky images without additional hardware, assuming the geolocation of the captured sky is known. Experimental results using various real images show the effectiveness of the method.

References

[1]

Barnard, K., & Funt, B. (2002). Camera characterization for color research. Color Research and Application , 27 (3), 153-164.

[2]

Buil, C. (2005). Comparative test between Canon 10D and Nikon D70 . Retrieved May 31, 2013 from http://www.astrosurf.com/buil/d70v10d/eval.htm.

[3]

Chaiwiwatworakul, P., & Chirarattananon, S. (2004). An investigation of atmospheric turbidity of thai sky. Energy and Buildings , 36 , 650- 659.

[4]

Chakrabarti, A., Scharstein, D., & Zickler, T. (2009). An empirical camera model for internet color vision. In Proceedings of British Machine Vision Conference .

[5]

Debevec, P., Hawkins, T., Tchou, C., Duiker, H. P., Sarokin, W., & Sagar, M. (2000). Acquiring the reflectance field of a human face. In ACM Transactions on Graphics (SIGGRAPH).

[6]

Ebner, M. (2007). Estimating the spectral sensitivity of a digital sensor using calibration targets. In Proceedings of Conference on Genetic and Evolutionary Computation (pp. 642-649).

[7]

Finlayson, G., Hordley, S., & Hubel, P. (1998). Recovering device sensitivities with quadratic programming. In Proceedings of Color Science, System, and Application (pp. 90-95).

[8]

Frieden, B. R. (1983). Probability, statistical optics and data testing: A problem solving approach . Springer.

[9]

Haber, T., Fuchs, C., Bekaert, P., Seidel, H. P., Geoesele, M., & Lensch, H. P. A. (2009). Relighting objects from image collections. In Proceedings of Computer Vision and Pattern Recognition (pp. 627-634).

[10]

Hara, K., Nishino, K., & Ikeuchi, K. (2005). Light source position and reflectance estimation from a single view without the distant illumination assumption. IEEE Transactions on Pattern Analysis and Machine Intelligence , 27 (4), 493-505.

Digital Library

[11]

Hordley, S. D. (2006). Scene illuminant estimation: Past, present, and future. Color Research and Application , 31 (4), 303-314.

[12]

Hubel, P. M., Sherman, D., & Farrell, J. E. (1994). A comparison of method of sensor spectral sensitivity estimation. In Proceedings of Color Science, System, and Application (pp. 45-48).

[13]

Ikeuchi, K. (1981). Determining surface orientations of specular surfaces by using the photometric stereomethod. IEEE Transactions on Pattern Analysis and Machine Intelligence , 3 (6), 661-669.

Digital Library

[14]

Ikeuchi, K., & Horn, B. K. P. (1981). Numerical shape from shading and occluding boundaries. Artificial Intelligence , 17 (1-3), 141-184.

Digital Library

[15]

Judd, D. B., Macadam, D. L., Wyszecki, G., Budde, H. W., Condit, H. R., Henderson, S. T., et al. (1964). Spectral distribution of typical daylight as a function of correlated color temperature. Journal of the Optical Society of America , 54 (8), 1031-1036.

[16]

Kawakami, R., & Ikeuchi, K. (2009). Color estimation from a single surface color. In Proceedings of Computer Vision and Pattern Recognition (pp. 635-642).

[17]

Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of IEEE International Conference on Neural Networks (pp. 1942-1948).

[18]

Kuthirummal, S., Agarwala, A., Goldman, D. B., & Nayar, S. K. (2008). Priors for large photo collections andwhat they reveal about cameras. In Proceedings of European Conference on Computer Vision (pp. 74-87).

[19]

Lalonde, J. F., Efros, A. A., & Narasimhan, S. G. (2009). Estimating natural illumination from a single outdoor image. In Proceedings of International Conference on Computer Vision (pp. 183-190).

[20]

Lalonde, J. F., Narasimhan, S. G., & Efros, A. A. (2010). What do the sun and the sky tell us about the camera? International Journal of Computer Vision , 88 (1), 24-51.

Digital Library

[21]

Li, Y., Lin, S., Lu, H., & Shum, H. Y. (2003). Multiple-cue illumination estimation in textured scenes. In Proceedings of International Conference on Computer Vision (pp. 1366-1373).

[22]

Lin, S., Gu, J. W., Yamazaki, S., & Shum, H. Y. (2004). Radiometric calibration using a single image. In Proceedings of Computer Vision and Pattern Recognition (pp. 938-945).

[23]

Lopez-Moreno, J., Hadap, S., Reinhard, E., & Gutierrez, D. (2010). Compositing images through light source detection. Computer and Graphics , 6 (34), 698-707.

[24]

McCartney, E. J. (1976). Optics of the atmosphere: Scattering by molecules and particles . New York: Wiley.

[25]

Parkkinen, J. P. S., Hallikainen, J., & Jaaskelainen, T. (1989). Characteristic spectra of Munsell colors. Journal of the Optical Society of America , 6 (2), 318-322.

[26]

Perez, R., Seals, R., & Michalsky, J. (1993). An all weather model for sky luminance distribution. Solar Energy , 50 (3), 235-245.

[27]

Pratt, W. K., & Mancill, C. E. (1976). Spectral estimation techniques for the spectral calibration of a color image scanner. Applied Optics , 15 (1), 73-75.

[28]

Preetham, A. J., Shirley, P., & Smits, B. (1999). A practical analytic model for daylight. In ACM Transactions on Graphics (SIGGRAPH).

[29]

Ramanath, R., Snyder, W., Yoo, Y., & Drew, M. (2005). Color image processing pipeline. IEEE Signal Processing Magazine , 22 (1), 34- 43.

[30]

Reinhard, E., Adhikhmin, M., Gooch, B., & Shirley, P. (2001). Color transfer between images. Computer Graphics and Application , 21 (5), 34-41.

Digital Library

[31]

Sato, I., Sato, Y., & Ikeuchi, K. (2003). Illumination from shadows. IEEE Transactions on Pattern Analysis and Machine Intelligence , 25 (3), 290-300.

Digital Library

[32]

Sharma, G., & Trussell, H. J. (1993). Characterization of scanner sensitivity. In Proceedings of Transforms and Transportability of Color (pp. 103-107).

[33]

Slater, D., & Healey, G. (1998). What is the spectral dimensionality of the illumination functions in outdoor scenes?. In Proceedings of Computer Vision and Pattern Recognition (pp. 105-110).

[34]

Snavely, N., Seitz, S., Szeliski, R. (2006). Photo tourism: Exploring image collections in 3D. In ACM Transactions on Graphics (SIGGRAPH).

[35]

Takamatsu, J., Matsushita, Y., & Ikeuchi, K. (2008). Estimating camera response functions using probabilistic intensity similarity. In Proceedings of Computer Vision and Pattern Recognition .

[36]

Tan, R. T., Nishino, K., & Ikeuchi, K. (2004). Color constancy through inverse intensity-chromaticity space. Journal of the Optical Society of America , 21 (3), 321-334.

[37]

Thomson, M., & Westland, S. (2001). Colour-imager characterization by parametric fitting of sensor response. Color Research and Application , 26 (6), 442-449.

[38]

van de Weijer, J., Gevers, T., & Gijsenij, A. (2007). Edge-based color constancy. IEEE Transactions on Image Processing , 16 (9), 2207-2214.

Digital Library

[39]

Vora, P. L., Farrell, J. E., Tietz, J. D., & Brainard, D. H. (1997). Digital color cameras. 2. Spectral response. Technical, Report HPL-97-54.

[40]

Woodham, R. J. (1980). Photometric method for determining surface orientation from multiple images. Optical Engineering , 19 (1), 139-144.

[41]

Wyszecki, G., & Stiles, W. S. (1982). Color science . New York: Wiley Interscience Publication.

[42]

Yu, Y., & Malik, J. (1998). Recovering photometric properties of architectural scenes from photographs. In ACM Transactions on Graphics (SIGGRAPH).

[43]

Zhang, R., Tsai, P., Cryer, J. E., & Shah, M. (1999). Numerical shape from shading and occluding boundaries. IEEE Transactions on Pattern Analysis and Machine Intelligence , 21 (8), 690-706.

Digital Library

[44]

Zhao, H. (2013). Spectral sensitivity database . Retrieved May 31, 2013 from http://www.cvl.iis.u-tokyo.ac.jp/~zhao/database.html.

[45]

Zhao, H., Kawakami, R., Tan, R. T., & Ikeuchi, K. (2009). Estimating basis functions for spectral sensitivity of digital cameras. In Meeting on Image Recognition and Understanding .

Cited By

Fu YZhang TZheng YZhang DHuang H(2022)Joint Camera Spectral Response Selection and Hyperspectral Image RecoveryIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2020.300999944:1(256-272)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TPAMI.2020.3009999
Zhang TFu YZhang J(2022)Guided Hyperspectral Image Denoising with Realistic DataInternational Journal of Computer Vision10.1007/s11263-022-01660-2130:11(2885-2901)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1007/s11263-022-01660-2
Zhou ZLiu DWang YZhu Z(2022)Illumination correction via optimized random vector functional link using improved Harris hawks optimizationMultimedia Tools and Applications10.1007/s11042-022-11986-181:18(25007-25027)Online publication date: 22-Mar-2022
https://dl.acm.org/doi/10.1007/s11042-022-11986-1
Show More Cited By

Camera Spectral Sensitivity and White Balance Estimation from Sky Images
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Computer graphics

Recommendations

What Do the Sun and the Sky Tell Us About the Camera?

As the main observed illuminant outdoors, the sky is a rich source of information about the scene. However, it is yet to be fully explored in computer vision because its appearance in an image depends on the sun position, weather conditions, photometric ...
A New In-Camera Imaging Model for Color Computer Vision and Its Application

We present a study of in-camera image processing through an extensive analysis of more than 10,000 images from over 30 cameras. The goal of this work is to investigate if image values can be transformed to physically meaningful values, and if so, when ...
EMCCD color correction based on spectral sensitivity analysis

The sensor response function of a color camera is very essential to understand an overall camera imaging pipeline and to process captured images. It is also true when we characterize underlying imaging behaviors of the electron multiplying charge ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 105, Issue 3

December 2013

111 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2013 Springer Science+Business Media New York.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 December 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fu YZhang TZheng YZhang DHuang H(2022)Joint Camera Spectral Response Selection and Hyperspectral Image RecoveryIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2020.300999944:1(256-272)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TPAMI.2020.3009999
Zhang TFu YZhang J(2022)Guided Hyperspectral Image Denoising with Realistic DataInternational Journal of Computer Vision10.1007/s11263-022-01660-2130:11(2885-2901)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1007/s11263-022-01660-2
Zhou ZLiu DWang YZhu Z(2022)Illumination correction via optimized random vector functional link using improved Harris hawks optimizationMultimedia Tools and Applications10.1007/s11042-022-11986-181:18(25007-25027)Online publication date: 22-Mar-2022
https://dl.acm.org/doi/10.1007/s11042-022-11986-1
Fascione LHanika JLeone MDroske MSchwarzhaupt JDavidovič TWeidlich AMeng J(2018)ManukaACM Transactions on Graphics10.1145/318216137:3(1-18)Online publication date: 7-Aug-2018
https://dl.acm.org/doi/10.1145/3182161
Wei RRobles-Kelly AÁlvarez J(2018)Context Free Band Reduction Using a Convolutional Neural NetworkStructural, Syntactic, and Statistical Pattern Recognition10.1007/978-3-319-97785-0_9(86-96)Online publication date: 17-Aug-2018
https://dl.acm.org/doi/10.1007/978-3-319-97785-0_9
Fu YZhang TZheng YZhang DHuang H(2018)Joint Camera Spectral Sensitivity Selection and Hyperspectral Image RecoveryComputer Vision – ECCV 201810.1007/978-3-030-01219-9_48(812-828)Online publication date: 8-Sep-2018
https://dl.acm.org/doi/10.1007/978-3-030-01219-9_48
Zhang LFu YZheng YHuang H(2017)Camera spectral sensitivity, illumination and spectral reflectance estimation for a hybrid hyperspectral image capture system2017 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP.2017.8296340(545-545)Online publication date: 17-Sep-2017
https://dl.acm.org/doi/10.1109/ICIP.2017.8296340
LeGendre CYu XLiu DBusch JJones APattanaik SDebevec P(2016)Practical multispectral lighting reproductionACM Transactions on Graphics10.1145/2897824.292593435:4(1-11)Online publication date: 11-Jul-2016
https://dl.acm.org/doi/10.1145/2897824.2925934
Robles-Kelly AZhou XSmeaton ATian QBulterman DShen HMayer-Patel KYan S(2015)Single Image Spectral Reconstruction for Multimedia ApplicationsProceedings of the 23rd ACM international conference on Multimedia10.1145/2733373.2806223(251-260)Online publication date: 13-Oct-2015
https://dl.acm.org/doi/10.1145/2733373.2806223
Gryaditskya YPouli TReinhard ESeidel H(2014)Sky Based Light Metering for High Dynamic Range ImagesComputer Graphics Forum10.1111/cgf.1247433:7(61-69)Online publication date: 1-Oct-2014
https://dl.acm.org/doi/10.1111/cgf.12474

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents