More Web Proxy on the site http://driver.im/

research-article

Fisheye Image Calibration and Super-Resolution Method Based On Deep Learning

Authors:

Chang KongAuthors Info & Claims

ICIGP '21: Proceedings of the 2021 4th International Conference on Image and Graphics Processing

Pages 102 - 108

https://doi.org/10.1145/3447587.3447602

Published: 04 June 2021 Publication History

Abstract

Fisheye lens is a kind of ultra-wide-angle lens, which can get more field of view information than ordinary perspective lens. However, there is radial lens distortion in the photos taken by fisheye lens. In order to improve the visual quality of fisheye images and to perform subsequent computer vision tasks, fisheye images must be calibrated first. Traditional calibration methods are often based on a specific type of fisheye lens, which is not universal, and often need to take multiple images of a calibration pattern (typically a checkerboard) for calibration. And limited by hardware, many fisheye cameras take photos with low resolution. The method proposed in this paper not only overcomes the above limitations, but also jointly solves the problem of image super-resolution, which makes the final result more appreciable. This method uses a depth convolution neural network model to predict the fisheye image correction parameters, and then completes the correction process, but also completes the image super-resolution processing. Our quantitative experiments have proved the feasibility and superiority of our method.

References

[1]

Devernay, F., Faugeras, O.: Straight Lines Have to Be Straight: Automatic Calibration and Removal of Distortion from Scenes of Structured Environments. Int'l J. Machine Vision and Applications, vol.13, no.1. 2011: 1424.

[2]

G. P. Stein, "Lens distortion calibration using point correspondences," Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, USA, 1997, pp. 602-608.

[3]

Ciaran Hughes, Patrick Denny, Martin Glavin, and Edward Jones. Equidistant fisheye calibration and rectification by vanishing point extraction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(12):2289–2296, 2010.

Digital Library

[4]

Edward Rosten and Rohan Loveland. Camera distortion self-calibration using the plumb-line constraint and minimal hough entropy. Machine Vision and Applications, 22(1):77–85, 2011.

Digital Library

[5]

Simone Gasparini, Peter Sturm, Joao Barreto. Plane-Based Calibration of Central Catadioptric Cameras. ICCV 2009 - 12th International Conference on Computer Vision, Sep 2009, Kyoto, Japan.pp.1195-1202, ff10.1109/ICCV.2009.5459336ff. ffinria-00586972f

[6]

Ying, X., Zha, H. Identical Projective Geometric Properties of Central Catadioptric Line Images and Sphere Images with Applications to Calibration. Int J Comput Vis 78, 89–105 (2008).

[7]

A. W. Fitzgibbon, "Simultaneous linear estimation of multiple view geometry and lens distortion," Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, Kauai, HI, USA, 2001, pp. I-I.

[8]

B. Micusik and T. Pajdla, "Estimation of omnidirectional camera model from epipolar geometry," 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., Madison, WI, USA, 2003, pp. I-I.

[9]

Jiang F., Kuang Y., Solem J.E., Åström K. (2015) A Minimal Solution to Relative Pose with Unknown Focal Length and Radial Distortion. In: Cremers D., Reid I., Saito H., Yang MH. (eds) Computer Vision – ACCV 2014. ACCV 2014. Lecture Notes in Computer Science, vol 9004. Springer, Cham.

[10]

Mi Zhang, Jian Yao, Menghan Xia, Kai Li, Yi Zhang, and Yaping Liu. Line-based multi-label energy optimization for fisheye image rectification and calibration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4137–4145, 2015.

[11]

Michel Antunes, João P. Barreto, Djamila Aouada, and Björn Ottersten. 2017. Unsupervised Vanishing Point Detection and Camera Calibration from a Single Manhattan Image with Radial Distortion. In CVPR

[12]

Rong, J., Huang, S., Shang, Z., Ying, X.: Radial lens distortion correction using convolutional neural networks trained with synthesized images. In: Asian Conference on Computer Vision. Springer, 2016: 35-49.

[13]

Fitzgibbon, A.W.: Simultaneous linear estimation of multiple view geometry and lens distortion. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), vol.1. 2001: 1-125.

[14]

Yin X., Wang X., Yu J., Zhang M., Fua P., Tao D. (2018) FishEyeRecNet: A Multi-context Collaborative Deep Network for Fisheye Image Rectification. In: Ferrari V., Hebert M., Sminchisescu C., Weiss Y. (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol 11214. Springer, Cham.

[15]

Dong, Chao & Loy, Chen Change & He, Kaiming & Tang, Xiaoou. (2014). Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence. 38. 10.1109/TPAMI.2015.2439281.

[16]

J. Kim, J. K. Lee and K. M. Lee, "Accurate Image Super-Resolution Using Very Deep Convolutional Networks," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 1646-1654.

[17]

T. Tong, G. Li, X. Liu and Q. Gao, "Image Super-Resolution Using Dense Skip Connections," 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 4809-4817.

[18]

Zhou Biao, Zhao Xin, Qi Zhihong, Yang Jinhua. A Common Fisheye Correction Algorithm Based on Paraboloidal Model. Journal of Changchun University of Technology (Natural Science Edition), 2014, 37 (02): 22-25

[19]

Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Semantic understanding of scenes through the ade20k dataset. arXiv preprint arXiv:1608.05442, 2016.

[20]

R. Timofte, V. De and L. V. Gool, "Anchored Neighborhood Regression for Fast Example-Based Super-Resolution," 2013 IEEE International Conference on Computer Vision, Sydney, NSW, 2013, pp. 1920-1927.

Digital Library

Cited By

Landro NGallo IPelosi FLa Grassa RRehman A(2024)Distortion-aware super-resolution for planetary exploration imagesSixteenth International Conference on Machine Vision (ICMV 2023)10.1117/12.3023378(21)Online publication date: 3-Apr-2024
https://doi.org/10.1117/12.3023378
Nair PK M(2023)Recent Trends in Deep Learning Based Omnidirectional Image SuperResolution2023 3rd Asian Conference on Innovation in Technology (ASIANCON)10.1109/ASIANCON58793.2023.10269912(1-6)Online publication date: 25-Aug-2023
https://doi.org/10.1109/ASIANCON58793.2023.10269912
Kim BLee DMin KChong JJoe I(2022)Global Convolutional Neural Networks With Self-Attention for Fisheye Image RectificationIEEE Access10.1109/ACCESS.2022.322829710(129580-129587)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3228297

Recommendations

A fisheye distortion correction method based on deep learning
EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering

As machine vision involves many fields such as photogrammetry, national defense and military, and security surveillance, fisheye cameras have also been widely used as an ultra-wide-angle lens. Different from ordinary cameras, fisheye cameras can obtain ...
DeepCalib: a deep learning approach for automatic intrinsic calibration of wide field-of-view cameras
CVMP '18: Proceedings of the 15th ACM SIGGRAPH European Conference on Visual Media Production

Calibration of wide field-of-view cameras is a fundamental step for numerous visual media production applications, such as 3D reconstruction, image undistortion, augmented reality and camera motion estimation. However, existing calibration methods ...
Calibration and Correction for Omnidirectional Image with a Fisheye Lens
ICNC '08: Proceedings of the 2008 Fourth International Conference on Natural Computation - Volume 06

Omnidirectional vision (omni-vision) with a fisheye lens holds promise of various applications such as global surveillance, robotic navigation and so on. The calibration for the vision system and correction for the distorted image are critical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICIGP '21: Proceedings of the 2021 4th International Conference on Image and Graphics Processing

January 2021

231 pages

ISBN:9781450389105

DOI:10.1145/3447587

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Grant from Natural Science Foundation of Shenzhen City
National Key Research and Development Program

Conference

ICIGP 2021

ICIGP 2021: 2021 The 4th International Conference on Image and Graphics Processing

January 1 - 3, 2021

Sanya, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
173
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Landro NGallo IPelosi FLa Grassa RRehman A(2024)Distortion-aware super-resolution for planetary exploration imagesSixteenth International Conference on Machine Vision (ICMV 2023)10.1117/12.3023378(21)Online publication date: 3-Apr-2024
https://doi.org/10.1117/12.3023378
Nair PK M(2023)Recent Trends in Deep Learning Based Omnidirectional Image SuperResolution2023 3rd Asian Conference on Innovation in Technology (ASIANCON)10.1109/ASIANCON58793.2023.10269912(1-6)Online publication date: 25-Aug-2023
https://doi.org/10.1109/ASIANCON58793.2023.10269912
Kim BLee DMin KChong JJoe I(2022)Global Convolutional Neural Networks With Self-Attention for Fisheye Image RectificationIEEE Access10.1109/ACCESS.2022.322829710(129580-129587)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3228297

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents