[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Automatic Flare Spot Artifact Detection and Removal in Photographs

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Flare spot is one type of flare artifact caused by a number of conditions, frequently provoked by one or more high-luminance sources within or close to the camera field of view. When light rays coming from a high-luminance source reach the front element of a camera, it can produce intra-reflections within camera elements that emerge at the film plane forming non-image information or flare on the captured image. Even though preventive mechanisms are used, artifacts can appear. In this paper, we propose a robust computational method to automatically detect and remove flare spot artifacts. Our contribution is threefold: firstly, we propose a characterization which is based on intrinsic properties that a flare spot is likely to satisfy; secondly, we define a new confidence measure able to select flare spots among the candidates; and, finally, a method to accurately determine the flare region is given. Then, the detected artifacts are removed by using exemplar-based inpainting. We show that our algorithm achieves top-tier quantitative and qualitative performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S.M., Szeliski, R.: Building rome in a day. Commun. ACM 54(10), 105–112 (2011)

    Article  Google Scholar 

  2. Aljabar, P., Heckemann, R.A., Hammers, A., Hajnal, J.V., Rueckert, D.: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. Neuroimage 46(3), 726–738 (2009)

    Article  Google Scholar 

  3. Arias, P., Facciolo, G., Caselles, V., Sapiro, G.: A variational framework for exemplar-based image inpainting. Int. J. Comput. Vis. 93, 319–347 (2011). https://doi.org/10.1007/s11263-010-0418-7

    Article  MathSciNet  MATH  Google Scholar 

  4. Bay, H., Fasel, B., Van Gool, L.: Interactive museum guide: fast and robust recognition of museum objects. In: Proc. Int. Workshop on Mobile Vision (2006)

  5. Boynton, P.A., Kelley, E.F.: Liquid-filled camera for the measurement of high-contrast images. In: Cockpit Displays X, vol. 5080, pp. 370–379. International Society for Optics and Photonics (2003)

  6. Chabert, F.: Automated lens flare removal. Technical report, Stanford University, Department of Electrical Engineering (2015)

  7. Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  8. Demanet, L., Song, B., Chan, T.: Image inpainting by correspondence maps: a deterministic approach. Appl. Comput. Math. 1100(217–50), 99 (2003)

    Google Scholar 

  9. Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)

    Article  Google Scholar 

  10. Evans, E. D.: An analysis and reduction of flare light in optical systems. PhD thesis, The Ohio State University (1988)

  11. Fedorov, V., Facciolo, G., Arias, P.: Variational framework for non-local inpainting. Image Process. On Line 5, 362–386 (2015)

    Article  MathSciNet  Google Scholar 

  12. Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings, vol. 2, pp. II–II. IEEE (2003)

  13. Gu, J., Ramamoorthi, R., Belhumeur, P., Nayar, S.: Removing image artifacts due to dirty camera lenses and thin occluders. ACM Trans. Graphics 28, 144 (2009)

    Article  Google Scholar 

  14. Hara, T., Saito, H., Kanade, T.: Removal of glare caused by water droplets. In: CVMP’09. Conference for Visual Media Production, pp 144–151. IEEE (2009)

  15. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  16. Kawai, N., Sato, T., Yokoya, N.: Image inpainting considering brightness change and spatial locality of textures and its evaluation. In: Pacific-Rim Symposium on Image and Video Technology, pp 271–282. Springer (2009)

  17. Kojima, T., Matsumaru, T., Banno, M.: Computer-simulation analysis of ghost images in photographic objectives. In: 1980 International Lens Design Conference, vol. 237, pp. 504–512. International Society for Optics and Photonics (1980)

  18. Kondo, H., Chiba, Y., Yoshida, T.: Veiling glare in photographic systems. Opt. Eng. 21(2), 212343 (1982)

    Article  Google Scholar 

  19. Lindeberg, T.: Scale-space theory: a basic tool for analyzing structures at different scales. J. Appl. Stat. 21(1–2), 225–270 (1994)

    Article  Google Scholar 

  20. Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 79–116 (1998)

    Article  Google Scholar 

  21. Lowe, D.G.: Object recognition from local scale-invariant features. In: The proceedings of the seventh IEEE international conference on Computer vision, vol. 2, pp. 1150–1157. IEEE (1999)

  22. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  23. Macleod, H.A.: Thin-Film Optical Filters. CRC Press, Boca Raton (2010)

    Book  Google Scholar 

  24. Martin, S.: Glare characteristics of lenses. Opt. Acta 19, 499–513 (1972)

    Article  Google Scholar 

  25. Matsuda, S., Nitoh, T.: Flare as applied to photographic lenses. Appl. Opt. 11(8), 1850–1856 (1972)

    Article  Google Scholar 

  26. Mikolajczyk, K.: Detection of local features invariant to affines transformations. PhD Thesis, Institut National Polytechnique de Grenoble-INPG (2002)

  27. Nussberger, A., Grabner, H., Van Gool, L.: Robust aerial object tracking from an airborne platform. IEEE Aerosp. Electron. Syst. Mag. 31(7), 38–46 (2016)

    Article  Google Scholar 

  28. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. In: ACM Trans. Graphics (TOG), vol. 23, pp. 664–672. ACM (2004)

  29. Prokopetc, K., Bartoli, A.: Slim (slit lamp image mosaicing): handling reflection artifacts. Int. J. Comput. Assist. Radiol. Surg. 12, 1–10 (2017)

    Article  Google Scholar 

  30. Psotny, D.: Removing lens flare from digital photographs. Technical Report, Charles Univ. Prague (2009)

  31. Raskar, R., Agrawal, A., Wilson, C.A., Veeraraghavan, A.: Glare aware photography: 4d ray sampling for reducing glare effects of camera lenses. ACM Trans. Graphics 27(3), 56 (2008)

    Article  Google Scholar 

  32. Ray, S. F.: Applied photographic optics: lenses and optical systems for photography, film, video, electronic and digital imaging, (2002)

  33. Rey-Otero, I., Delbracio, M.: Anatomy of the sift method. Image Process. On Line 4, 370–396 (2014). https://doi.org/10.5201/ipol.2014.82

    Article  Google Scholar 

  34. Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3d. ACM Trans. Graphics 25, 835–846 (2006)

    Article  Google Scholar 

  35. Steenstrup, S., Munk, O.: Automatic apparatus for measuring veiling-glare distribution. Opt. Acta 27, 939–947 (1980)

    Article  Google Scholar 

  36. Talvala, E.-V., Adams, A., Horowitz, M., Levoy, M.: Veiling glare in high dynamic range imaging. ACM Trans. Graphics 26, 37 (2007)

    Article  Google Scholar 

  37. Tan, R. T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2):178–193 (2005). ISSN 0162-8828. https://doi.org/10.1109/TPAMI.2005.36

  38. Tuytelaars, T., Mikolajczyk, K., et al.: Local invariant feature detectors: a survey. Found. Trends® Comput. Graphics Vis. 3(3), 177–280 (2008)

    Article  Google Scholar 

  39. Von Gioi, R.G., Monasse, P., Morel, J.-M., Tang, Z.: Towards high-precision lens distortion correction. In: 2010 17th IEEE International Conference on Image Processing, pp 4237–4240. IEEE (2010)

  40. Wels, M., Zheng, Y., Carneiro, G., Huber, M., Hornegger, J., Comaniciu, D.: Fast and robust 3-d mri brain structure segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 575–583. Springer (2009)

  41. Wexler, Y., Shechtman, E., Irani, M.: Space-time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 463–476 (2007)

    Article  Google Scholar 

  42. Wu, T.-P., Tang, C.-K.: A bayesian approach for shadow extraction from a single image. In: ICCV 2005. Tenth IEEE International Conference on Computer Vision, 2005, vol. 1, pp. 480–487. IEEE (2005)

  43. Zhang, J., Marszałek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object categories: a comprehensive study. Int. J. Comput. Vis. 73(2), 213–238 (2007)

    Article  Google Scholar 

  44. Zinemanas, P., Arias, P., Haro, G., Gomez, E.: Visual music transcription of clarinet video recordings trained with audio-based labelled data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 463–470 (2017)

Download references

Acknowledgements

The authors acknowledge partial support by MINECO/FEDER UE project, with reference TIN2015-70410-C2-1-R, and by H2020-MSCA-RISE-2017 project with reference 777826 NoMADS.

Author information

Authors and Affiliations

  1. Universitat Pompeu Fabra, C/ Roc Boronat 138, 08018, Barcelona, Spain

    Patricia Vitoria & Coloma Ballester

Authors
  1. Patricia Vitoria
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Coloma Ballester
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Patricia Vitoria.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vitoria, P., Ballester, C. Automatic Flare Spot Artifact Detection and Removal in Photographs. J Math Imaging Vis 61, 515–533 (2019). https://doi.org/10.1007/s10851-018-0859-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-018-0859-0

Keywords

Search

Navigation