[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Deep learning-based RGB-thermal image denoising: review and applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, vision-based detection (VD) technology has been well-developed, and its general-purpose object detection algorithms have been applied in various scenes. VD can be divided into two categories based on the type of modality: single-modal (single RGB or single thermal) and bimodal. Image denoising is typically the first stage of image processing in VD, where redundant information and noisy data are removed to produce clearer images for effective object detection. This study reviews deep learning-based image denoising for RGB and thermal images, investigating the denoising procedure, methodologies, and performances of algorithms tested with benchmark datasets. After introducing denoising models, the main results on public RGB and thermal datasets are presented and analyzed, and conclusions of objective comparison in practical effect are drawn. This review can serve as a reference for researchers in RGB–infrared denoising, image restoration, and related fields.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The datasets analysed during the current study are available in the BSD68, CBSD68, Kodak24, McMaster, OSU and FLIR repositories.

References

  1. Abascal JFPJ, Bussod S, Ducros N, Si-Mohamed S, Douek P, Chappard C, Peyrin F (January 2021) A residual U-net network with image prior for 3D image denoising. In: 2020 28th European Signal Processing Conference (EUSIPCO), Amsterdam, Netherlands. IEEE, pp 1264–1268

  2. Amal MFI, Wicaksono HPA, Prasetyo H (2020) Deep residual networks for impulsive noise suppression. In: 2020 27th International Conference on Telecommunications (ICT), Bali, Indonesia. IEEE, pp 1–5

  3. Anwar S, Barnes N, Petersson L (2021) Attention-based real image restoration. IEEE Trans Neural Netw Learn Syst 1–11

  4. Ashraf H, Jeong Y, Lee CH (2021) Underwater ambient-noise removing GAN based on magnitude and phase spectra. IEEE Access 9:24513–24530

    Google Scholar 

  5. Aspandi D, Martinez O, Sukno F, Binefa X (2019) Robust facial alignment with internal denoising auto-encoder. In: 2019 16th Conference on Computer and Robot Vision (CRV). IEEE, pp 143–150

  6. Bao L, Yang Z, Wang S, Bai D, Lee J (2020) Real image denoising based on multi-scale residual dense block and cascaded U-net with block-connection. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA. IEEE, pp 1823–1831

  7. Batchuluun G, Kang JK, Nguyen DT, Pham TD, Arsalan M, Park KR (2020) Deep learning-based thermal image reconstruction and object detection. IEEE Access 9:5951–5971

    Google Scholar 

  8. Batson J, Royer L (2019) Noise2self: blind denoising by self-supervision. In: International Conference on Machine Learning. PMLR, pp 524–533

  9. Borkar TS, Karam LJ (2019) DeepCorrect: correcting DNN models against image distortions. IEEE Trans Image Process 28(12):6022–6034

    MathSciNet  Google Scholar 

  10. Brown MS, Abdelhamed A, Lin S. Sidd dataset. https://paperswithcode.com/dataset/sidd. Accessed 2nd Feb. 2023

  11. Buades A, Li X, Zhang L, Wu X. Mcmaster dataset. https://paperswithcode.com/dataset/mcmaster. Accessed 2nd Feb. 2023

  12. Cha S, Moon T (2019) Fully convolutional pixel adaptive image denoiser. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). pp 4159–4168

  13. Chan W, Salimans T, Fleet DJ, Norouzi M, Saharia C, Ho J (2021) Image super-resolution via iterative refinement. Preprint at http://arxiv.org/abs/2104.07636

  14. Chen Y-J, Tsai C-Y, Xu X, Shi Y, Ho T-Y, Huang M, Yuan H, Zhuang J (2021) CT image denoising with encoder-decoder based graph convolutional networks. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE, pp 400–404

  15. Chen Y-S, Wang Y-C, Kao M-H, Chuang Y-Y (2018) Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 6306–6314

  16. Chen H, Zhang Y, Kalra MK, Lin F, Chen Y, Liao P, Zhou J, Wang G (2017) Low-dose CT with a residual encoder-decoder convolutional neural network. IEEE Trans Med Imaging 36(12):2524–2535

    Google Scholar 

  17. Chen L-H, Bampis CG, Li Z, Bovik AC (2020) Learning to distort images using generative adversarial networks. IEEE Signal Process Lett 27:2144–2148

    Google Scholar 

  18. Chen Y, Bruzzone L, Jiang L, Sun Q (2021) Aru-net: reduction of atmospheric phase screen in SAR interferometry using attention-based deep residual U-net. IEEE Trans Geosci Remote Sens 59(7):5780–5793

    Google Scholar 

  19. Chen J, Chen J, Chao H, Yang M (2018) Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 3155–3164

  20. Cheng W, Jinling L, Zhu X, Hong J, Liu X, Li M, Li P (2020) Dilated residual learning with skip connections for real-time denoising of laser speckle imaging of blood flow in a log-transformed domain. IEEE Trans Med Imaging 39(5):1582–1593

    Google Scholar 

  21. Chen Y, Meng D, Zhang L, Zhang K, Zuo WM. Set12 dataset. https://paperswithcode.com/dataset/set12. Accessed 14th Feb. 2023

  22. Chopra A, Maan A, Kesharwani A (2021) Low light GAN-based photo enhancement. In: 2021 Second International Conference on Electronics and Sustainable Communication Systems (ICESC), Coimbatore, India. IEEE, pp 103–110

  23. Chrysostomou C (2021) Sinogram denoise based on generative adversarial networks. In: 2020 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC). IEEE, pp 1–4

  24. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095

    MathSciNet  Google Scholar 

  25. Dang A, Vu TH, Wang J-C (2020) Encoder-recurrent decoder network for single image dehazing. In: ICASSP 2020–2020 IEEE International Conference on Acoustics. Speech and Signal Processing (ICASSP), Barcelona, Spain. IEEE, pp 4432–4436

  26. David JW. OSU dataset. http://vcipl-okstate.org/pbvs/bench/. Accessed 14th Feb. 2023

  27. Deepak S, Sahoo S, Patra D (2021) Super-resolution of thermal images using GAN network. In: 2021 Advanced Communication Technologies and Signal Processing (ACTS). IEEE, pp 1–5

  28. Deng S, Wei M, Wang J, Feng Y, Liang L, Xie H, Wang FL, Wang M (2020) Detail-recovery image deraining via context aggregation networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA. IEEE, pp 14548–14557

  29. DND. DND dataset. https://paperswithcode.com/dataset/dnd. Accessed 14th Feb. 2023

  30. Dong W, Zhang L, Shi G, Li X (2012) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630

    MathSciNet  Google Scholar 

  31. Duong HD, Tinh DT (2013) An efficient method for vision-based fire detection using SYM classification. In: 2013 International Conference on Soft Computing and Pattern Recognition (SoCPaR). IEEE, pp 190–195

  32. FLIR. Flir dataset. https://paperswithcode.com/dataset/flir-aligned. Accessed 14th Feb. 2023

  33. Fu X, Qi Q, Zha Z-J, Ding X, Wu F, Paisley J (2021) Successive graph convolutional network for image de-raining. Int J Comput Vision 129(5):1691–1711

    Google Scholar 

  34. Fukushima K, Miyake S (1982) Neocognitron: a self-organizing neural network model for a mechanism of visual pattern recognition. In: Competition and Cooperation in Neural Nets. Springer, pp 267–285

  35. Fu Z, Yu X, Ge C, Aziz MZ, Liu L (2021) ADGAN: an asymmetric despeckling generative adversarial network for unpaired OCT image speckle noise reduction. In: 2021 IEEE 6th Optoelectronics Global Conference (OGC). IEEE, pp 212–216

  36. Gao M, Fessler JA, Chan H-P (2021) Deep convolutional neural network with adversarial training for denoising digital breast tomosynthesis images. IEEE Trans Med Imaging 40(7):1805–1816

    Google Scholar 

  37. Gautam A, Singh S (2020) A comparative analysis of deep learning based super-resolution techniques for thermal videos. In: 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT). IEEE, pp 919–925

  38. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Proces Syst 27

  39. Goyal N, Chaudhary V, Wenzek G, Guzmán F, Grave E, Ott M, Zettlemoyer L, Stoyanov V, Conneau A, Khandelwal K. CC dataset. https://paperswithcode.com/dataset/cc100. Accessed 16th Feb. 2023

  40. Guo Y, Davy A, Facciolo G, Morel J-M, Jin Q (2021) Fast, nonlocal and neural: a lightweight high quality solution to image denoising. IEEE Signal Process Lett 28:1515–1519

    Google Scholar 

  41. Gurrola-Ramos J, Dalmau O, Alarcon TE (2021) A residual dense U-net neural network for image denoising. IEEE Access 9:31742–31754

    Google Scholar 

  42. Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 2862–2869

  43. Han Z, Shangguan H, Zhang X, Zhang P, Cui X, Ren H (2022) A dual-encoder-single-decoder based low-dose CT denoising network. IEEE J Biomed Health Inform 1–1

  44. Han Z, Shangguan H, Zhang X, Zhang P, Cui X, Ren H (2022) A dual-encoder-single-decoder based low-dose CT denoising network. IEEE J Biomed Health Inform

  45. Ho LT, Tran ST, Dinh D (2021) Nom document background removal using generative adversarial network. In: 2021 IEEE International Conference on Signal and Image Processing Applications (ICSIPA). IEEE, pp 100–104

  46. Hou R, Li F, Zhang G (2022) Truncated residual based plug-and-play ADMM algorithm for MRI reconstruction. IEEE Trans Comput Imaging 8:96–108

    MathSciNet  Google Scholar 

  47. Hou Z, Kung S-Y (2021) Hierarchically aggregated residual transformation for single image super resolution. In: 2020 25th International Conference on Pattern Recognition (ICPR), Italy, Milan. IEEE, pp 2248–2255

  48. Huang Z, Zhang J, Zhang Y, Shan H (2022) DU-GAN: generative adversarial networks with dual-domain U-net-based discriminators for low-dose CT denoising. IEEE Trans Instrum Meas 71:1–12

    Google Scholar 

  49. Huang Z, Zhao R, Leung FHF, Lam K-M, Ling SH, Lyu J, Banerjee S, Tin-Yan Lee T, Yang D, Zheng Y-P (2021) DA-GAN: learning structured noise removal in ultrasound volume projection imaging for enhanced spine segmentation. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE, pp 770–774

  50. Issa TB, Vinegoni C, Shaw A, Feruglio PF, Weissleder R, Uminsky D (2020) Video-rate acquisition fluorescence microscopy via generative adversarial networks. In: 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE). IEEE, pp 569–576

  51. Jadhav S, Kulkarni P (2021) Image denoising using deep auto-encoder network for production monitoring in real-time. In: 2021 International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT). IEEE, pp 1–7

  52. Jia X, Liu S, Feng X, Zhang L (2019) FOCNet: a fractional optimal control network for image denoising. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 6054–6063

  53. Jiang X, Lu L, Zhu M, Hao Z, Gao W (2021) Haze relevant feature attention network for single image dehazing. IEEE Access 9:106476–106488

    Google Scholar 

  54. Jimenez MFM, DeGuchy O, Marcia RF (2020) Deep convolutional autoencoders for deblurring and denoising low-resolution images. In: 2020 International Symposium on Information Theory and Its Applications (ISITA). IEEE, pp 549–553

  55. Karypis G, Wale N. NCI1 dataset. https://paperswithcode.com/dataset/nci1. Accessed 16th Feb. 2023

  56. Khamassi M, Kaaniche M, Benazza-Benyahia A (2021) Joint denoising of stereo images using 3D CNN. In: 2020 10th International Symposium on Signal, Image, Video and Communications (ISIVC), Saint-Etienne, France. IEEE, pp 1–6

  57. Khan S, Hayat M, Khan FS, Yang M-H, Shao L, Zamir SW, Arora A (2020) CycleISP: real image restoration via improved data synthesis. Preprint at http://arxiv.org/abs/2003.07761

  58. Khor HG, Ning G, Zhang X, Liao H (2022) Ultrasound speckle reduction using wavelet-based generative adversarial network. IEEE J Biomed Health Inform 1–1

  59. Kim D-W, Chung JR, Jung S-W. GRDN: grouped residual dense network for real image denoising and gan-based real-world noise modeling. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA, June 2019. IEEE, pp 2086–2094

  60. Kim Y, Soh JW, Park GY, Cho NI (2020) Transfer learning from synthetic to real-noise denoising with adaptive instance normalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 3482–3492

  61. Kodak. Kodak24 dataset. https://r0k.us/graphics/kodak/

  62. Krull A, Vičar T, Prakash M, Lalit M, Jug F (2020) Probabilistic noise2void: unsupervised content-aware denoising. Front Comp Sci 2:5. Accessed 16th Feb. 2023

  63. Krull A, Buchholz T-O, Jug F (2019) Noise2void-learning denoising from single noisy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 2129–2137

  64. Kuanar S, Athitsos V, Mahapatra D, Rao KR, Akhtar Z, Dasgupta D (2019) Low dose abdominal CT image reconstruction: an unsupervised learning based approach. In: 2019 IEEE International Conference on Image Processing (ICIP). IEEE, pp 1351–1355

  65. Kuang X, Sui X, Liu Y, Chen Q, Guohua GU (2017) Single infrared image optical noise removal using a deep convolutional neural network. IEEE Photonics J 10(2):1–15

    Google Scholar 

  66. Lee J, Kim J (2021) Edge profile super resolution. 9:11

  67. Lehtinen J, Munkberg J, Hasselgren J, Laine S, Karras T, Aittala M, Aila T (2018) Noise2noise: learning image restoration without clean data. Preprint at http://arxiv.org/abs/1803.04189

  68. Li Z, Zhang J, Fang Z, Huang B, Jiang X, Gao Y, Hwang J-N (2019) Single image snow removal via composition generative adversarial networks. IEEE Access 7:25016–25025

    Google Scholar 

  69. Li T, Zhao Y, Li Y, Zhou G (2021) Non-uniformity correction of infrared images based on improved CNN with long-short connections. IEEE Photonics J 13(3):1–13

    Google Scholar 

  70. Li Y, Luo X, Wu N, Dong X (2021) The application of semisupervised attentional generative adversarial networks in desert seismic data denoising. IEEE Geosci Remote Sens Lett 19:1–5

    Google Scholar 

  71. Liang J, Cao J, Sun G, Zhang K, Van Gool L, Timofte R (2021) Swinir: image restoration using swin transformer. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada. IEEE, pp 1833–1844

  72. Liang J, Liu R (2015) Stacked denoising autoencoder and dropout together to prevent overfitting in deep neural network. In: 2015 8th International Congress on Image And Signal processing (CISP). IEEE, pp 697–701

  73. Liang Z, Zhang D, Zhang L, Xu J, Li H. Polyu dataset. https://paperswithcode.com/dataset/polyu-dataset. Accessed 16th Feb. 2023

  74. Li M, Cheung Y-M (2021) Identity-preserved complete face recovering network for partial face image. IEEE Trans Emerg Top Comput Intell

  75. Li Y, Fu X, Zha Z-J (2021) Cross-patch graph convolutional network for image denoising. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp 4651–4660

  76. Li S, Hou Y, Yue H, Guo Z (2019) Single image de-raining via generative adversarial nets. In: 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp 1192–1197

  77. Lin C-H, Liao W-M, Liang J-W, Chen P-H, Ko C-E, Yang C-H, Lu C-K (2021) Denoising performance evaluation of automated age-related macular degeneration detection on optical coherence tomography images. IEEE Sens J 21(1):790–801

    Google Scholar 

  78. Lin S, Han X, Wang Y, Lu Z, Zhang Y, Jia T (2021) A convolutional neural network for small sample’s ring structured light denoising. In: 2021 33rd Chinese Control and Decision Conference (CCDC). IEEE, pp 1063–1068

  79. Lin S, Yang H (2021) Dual-mode iterative denoiser: tackling the weak label for anomaly detection. In: 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, pp 6742–6749

  80. Liu X, Mei S, Zhang Z, Zhang Y, Ji J, Du Q (2019) DECS-Net: convolutional self-encoding network for hyperspectral image denoising. In: IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium. IEEE, pp 1951–1954

  81. Liu Y, Wang Z, Zeng Y, Zeng H, Zhao D (2021) PD-GAN: perceptual-details GAN for extremely noisy low light image enhancement. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 1840–1844

  82. Liu Y, Wang Z, Zeng Y, Zeng H, Zhao D. PD-GAN: perceptual-details GAN for extremely noisy low light image enhancement. p 5

  83. Liu Y, Zhang X, Wang S, Ma S, Gao W (2020) Progressive multi-scale residual network for single image super-resolution. Preprint at http://arxiv.org/abs/2007.09552

  84. Ma H, Sun Y, Wu N, Li Y (2021) Relative attributes-based generative adversarial network for desert seismic noise suppression. IEEE Geosci Remote Sens Lett 19:1–5

    Google Scholar 

  85. Matsui T, Ikehara M (2020) Gan-based rain noise removal from single-image considering rain composite models. IEEE Access 8:40892–40900

    Google Scholar 

  86. Matsushita Y, Kim SJ, Nam S, Hwang Y. Nam dataset. https://github.com/daooshee/Image-Processing-Datasets. Accessed 16th Feb. 2023

  87. Mehranian A, Reader AJ (2021) Model-based deep learning pet image reconstruction using forward-backward splitting expectation-maximization. IEEE Trans Radiat Plasma Med Sci 5(1):54–64

    Google Scholar 

  88. Mehta A, Sinha H, Mandal M, Narang P (2021) Domain-aware unsupervised hyperspectral reconstruction for aerial image dehazing. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp 413–422

  89. Mok GSP, Sun J, Zhang Q, Du Y. Comparison of projection-based and reconstruction-based low dose spect image denoising using a conditional generative adversarial network. In: 2020 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC). IEEE, pp 1–3

  90. Moran N, Schmidt D, Zhong Y, Coady P (2020) Noisier2noise: Learning to denoise from unpaired noisy data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 12064–12072

  91. Motwani MC, Gadiya MC, Motwani RC, Harris FC (2004) Survey of image denoising techniques. Proceedings of GSPX 27:27–30

    Google Scholar 

  92. Muneeb U, Koyuncu E, Keshtkarjahromi Y, Seferoglu H, Erden MF, Cetin AE (2020) Robust and computationally-efficient anomaly detection using powers-of-two networks. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 2992–2996

  93. Nasonov A, Krylov A (2018) An improvement of BM3D image denoising and deblurring algorithm by generalized total variation. In: 2018 7th European workshop on visual information processing (EUVIP). IEEE, pp 1–4

  94. Nasrin S, Alom MZ, Burada R, Taha TM, Asari VK (2019) Medical image denoising with recurrent residual U-net (R2U-net) base auto-encoder. In: 2019 IEEE National Aerospace and Electronics Conference (NAECON). IEEE, pp 345–350

  95. Newey M, Sharma P (2021) Self-supervised speckle reduction GAN for synthetic aperture radar. In: 2021 IEEE Radar Conference (RadarConf21), Atlanta, GA, USA. IEEE, pp 1–6

  96. Othman A, Iqbal N, Hanafy SM, Waheed UB (2022) Automated event detection and denoising method for passive seismic data using residual deep convolutional neural networks. IEEE Trans Geosci Remote Sens 60:1–11

    Google Scholar 

  97. Pistilli F, Fracastoro G, Valsesia D, Magli E (2020) Learning robust graph-convolutional representations for point cloud denoising. IEEE J Sel Top Sign Proces 15(2):402–414

    Google Scholar 

  98. Prajapati K, Chudasama V, Patel H, Sarvaiya A, Upla KP, Raja K, Ramachandra R, Busch C (2021) Channel split convolutional neural network (ChasNet) for thermal image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 4368–4377

  99. Prakash M, Lalit M, Tomancak P, Krul A, Jug F (2020) Fully unsupervised probabilistic noise2void. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, pp 154–158

  100. Quan Y, Chen M, Pang T, Ji H (2020) Self2self with dropout: learning self-supervised denoising from single image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 1890–1898

  101. Que Y, Li S, Lee HJ (2021) Attentive composite residual network for robust rain removal from single images. IEEE Trans Multimedia 23:3059–3072

    Google Scholar 

  102. Ren D, Shang W, Zhu P, Hu Q, Meng D, Zuo W (2020) Single image deraining using bilateral recurrent network. IEEE Trans Image Process 29:6852–6863

    Google Scholar 

  103. Ren W, Liu S, Zhang H, Pan J, Cao X, Yang M-H (2016) Single image dehazing via multi-scale convolutional neural networks. In: European Conference on Computer Vision. Springer, pp 154–169

  104. Saranya A, Kottilingam K (2021) An efficient combined approach for denoising fibrous dysplasia images. In: 2021 International Conference on System, Computation, Automation and Networking (ICSCAN). IEEE, pp 1–6

  105. Shahidi F (2021) Breast cancer histopathology image super-resolution using wide-attention GAN with improved Wasserstein gradient penalty and perceptual loss. IEEE Access 9:32795–32809

    Google Scholar 

  106. Shao D, Zhao Y, Li Y, Li T (2022) Noisy2noisy: denoise pre-stack seismic data without paired training data with labels. IEEE Geosci Remote Sens Lett 19:1–5

    Google Scholar 

  107. Sharma M, Sarma KK, Mastorakis N (2018) AE and SAE based aircraft image denoising. In: 2018 5th International Conference on Mathematics and Computers in Sciences and Industry (MCSI). IEEE, pp 81–85

  108. Shen J, Chen H (2021) CT denoising by multi-feature CONCAT residual network with cross-domain attention BLCOK. In: 2021 2nd International Symposium on Computer Engineering and Intelligent Communications (ISCEIC), Nanjing, China. IEEE, pp 112–116

  109. Shobha Rani N, Nair BJ B, Karthik SK, Srinidhi A (2021) Binarization of degraded photographed document images-a variational denoising auto encoder. In: 2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA). IEEE, pp 119–124

  110. Siddiqua M, Akhter N, Khurshid J (2021) Comparative study of image to image translation models for underwater image enhancement. In: 2021 International Conference on Robotics and Automation in Industry (ICRAI). IEEE, pp 1–4

  111. Song T-A, Dutta J. Noise2void denoising of pet images. In: 2020 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC). IEEE, pp 1–2

  112. Suryanarayana G, Chandran K, Khalaf OI, Alotaibi Y, Alsufyani A, Alghamdi SA (2021) Accurate magnetic resonance image super-resolution using deep networks and Gaussian filtering in the stationary wavelet domain. IEEE Access 9:71406–71417

    Google Scholar 

  113. Tal D, Malik J, Martin D, Fowlkes C. Bsd300 dataset. https://github.com/BSD300/BSD300Dataset. Accessed 20th Feb. 2023

  114. Tal D, Malik J, Martin D, Fowlkes C. Bsd68 dataset. https://paperswithcode.com/dataset/bsd. Accessed 20th Feb. 2023

  115. Tal D, Malik J, Martin D, Fowlkes C. Cbsd68 dataset. https://paperswithcode.com/dataset/cbsd68. Accessed 20th Feb. 2023

  116. Tian M, Song K (2021) Boosting magnetic resonance image denoising with generative adversarial networks. IEEE Access 9:62266–62275

    Google Scholar 

  117. Tian C, Fei L, Zheng W, Xu Y, Zuo W, Lin C-W (2020) Deep learning on image denoising: an overview. Neural Netw 131:251–275

    Google Scholar 

  118. Tian C, Xu Y, Zuo W, Du B, Lin C-W, Zhang D (2021) Designing and training of a dual CNN for image denoising. Knowl-Based Syst 226:106949

    Google Scholar 

  119. Tomosada H, Kudo T, Fujisawa T, Ikehara M (2021) Gan-based image deblurring using DCT loss with customized datasets. IEEE Access 9:135224–135233

    Google Scholar 

  120. Tomosada H, Kudo T, Fujisawa T, Ikehara M (January 2021) GAN-based image deblurring using DCT discriminator. In: 2020 25th International Conference on Pattern Recognition (ICPR), Italy, Milan. IEEE, pp 3675–3681

  121. Valsesia D, Fracastoro G, Magli E (2020) Deep graph-convolutional image denoising. IEEE Trans Image Process 29:8226–8237

    MathSciNet  Google Scholar 

  122. Wang X, Li Z, Shan H, Tian Z, Ren Y, Zhou W (2020) FastDerainNet: a deep learning algorithm for single image deraining. IEEE Access 8:127622–127630

    Google Scholar 

  123. Wang Z, Li J, Song G (2019) DTDN: dual-task de-raining network. In: Proceedings of the 27th ACM International Conference on Multimedia. pp 1833–1841

  124. Wang Q, Liu H, Xie G, Zhang Y (2021) Image denoising using an improved generative adversarial network with Wasserstein distance. In: 2021 40th Chinese Control Conference (CCC). IEEE, pp 7027–7032

  125. Wang Q, Liu H, Xie G, Zhang Y (2021) Image denoising using an improved generative adversarial network with Wasserstein distance. In: 2021 40th Chinese Control Conference (CCC), Shanghai, China. IEEE, pp 7027–7032

  126. Wang X, Pan Y, Lun DPK (2020) Stereoscopic image reflection removal based on wasserstein generative adversarial network. In: 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP). IEEE, pp 148–151

  127. Wang X, Sebe N, Xu D, Ouyang W (2018) Pad-net: multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing. Preprint at http://arxiv.org/abs/1805.04409

  128. Wang M, Zhu W, Yu K, Chen Z, Shi F, Zhou Y, Ma Y, Peng Y, Bao D, Feng S, Ye L, Xiang D, Chen X (2021) Semi-supervised capsule CGAN for speckle noise reduction in retinal OCT images. IEEE Trans Med Imaging 40(4):1168–1183

    Google Scholar 

  129. Wei J, Ying C (2020) Aggregative adversarial network for still-to-video face recognition. In: 2020 5th International Conference on Computer and Communication Systems (ICCCS). IEEE, pp 266–270

  130. Wei X, Zhang X, Li Y. SARN: a lightweight stacked attention residual network for low-light image enhancement. p 5

  131. Wu M, Bu Y, Pan J, Yi Z, Kong X (2020) Spectra-GANs: a new automated denoising method for low-s/n stellar spectra. IEEE Access 8:107912–107926

    Google Scholar 

  132. Wu S, Dong C, Qiao Y (2022) Blind image restoration based on cycle-consistent network. IEEE Trans Multimedia

  133. Wu Q, Wang Z, Yong H, Li H, Zhang L, Ma K, Duanmu Z. Waterloo exploration dataset. https://ece.uwaterloo.ca/~k29ma/exploration/. Accessed 22nd Feb. 2023

  134. Xiao P, Guo Y, Zhuang P (2018) Removing stripe noise from infrared cloud images via deep convolutional networks. IEEE Photonics J 10(4):1–14

    Google Scholar 

  135. Xu J, Huang Y, Cheng M-M, Liu L, Zhu F, Xu Z, Shao L (2020) Noisy-as-clean: Learning self-supervised denoising from corrupted image. IEEE Trans Image Process 29:9316–9329

    Google Scholar 

  136. Xu Z, Lan J, Meng C, Wang W, Gu Z, Chen H (2022) Diffusioninst: diffusion model for instance segmentation. Preprint at http://arxiv.org/abs/2212.02773

  137. Xu W, Lee M, Zhang Y, You J, Suk S, Choi J-Y. Deep residual convolutional network for natural image denoising and brightness enhancement. p 6

  138. Yang Y, Cao L, Liu Q, Yang P (2019) A stacked multi-granularity convolution denoising auto-encoder. IEEE Access 7:83888–83899

    Google Scholar 

  139. Yang Z, Pan D, Shi P (2021) Joint image dehazing and super-resolution: closed shared source residual attention fusion network. IEEE Access 9:105477–105492

    Google Scholar 

  140. Yang X, Yu L, Wang H, Wang L, Zhang H (2023) Facial feature embedded CycleGAN for VIS-NIR translation. Multidimension Syst Signal Process 9:1573–0824

    Google Scholar 

  141. Yang X, Wang X, Wang N, Gao X (2021) SRDN: a unified super-resolution and motion deblurring network for space image restoration. IEEE Tran Geosci Remote Sens

  142. Yimin L, Sophie M, Jiang K, Haikun Q, Kuberan P, Kawal R (2021) Ultra-dense denoising network: application to cardiac catheter-based x-ray procedures. IEEE Trans Biomed Eng 68(9):2626–2636

    Google Scholar 

  143. Yuan D, Fan N, He Z (2020) Learning target-focusing convolutional regression model for visual object tracking. Knowl-Based Syst 194:105526

    Google Scholar 

  144. Yuan Y, Ma H, Liu G (2022) A new multiscale residual learning network for HSI inconsistent noise removal. IEEE Geosci Remote Sens Lett 19:1–5

    Google Scholar 

  145. Yu L, Shen L, Yang H, Wang L, An P (2019) Quality enhancement network via multi-reconstruction recursive residual learning for video coding. IEEE Signal Process Lett 26(4):557–561

    Google Scholar 

  146. Yu N, Wang H, Xu Q, Lin J (2021) Defect detection of rubber gloves based on normal samples. IN: 2021 IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), Chengdu, China. IEEE, pp 612–618

  147. Zeng J, Pang J, Sun W, Cheung G (2019) Deep graph laplacian regularization for robust denoising of real images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp 0–0

  148. Zeng Y, Zhang Z, Zhou X, Liu Y (2019) High dynamic range infrared image compression and denoising. In: 2019 International Conference on Information Technology and Computer Application (ITCA). IEEE, pp 65–69

  149. Zhang Z, Wang L, Kai A, Yamada T, Li W, Iwahashi M (2015) Deep neural network-based bottleneck feature and denoising autoencoder-based dereverberation for distant-talking speaker identification. EURASIP J Audio Speech Music Process 1:1–13

    Google Scholar 

  150. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process 26(7):3142–3155

    MathSciNet  Google Scholar 

  151. Zhang K, Zuo W, Zhang L (2018) FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans Image Process 27(9):4608–4622

    MathSciNet  Google Scholar 

  152. Zhang Y, Li K, Li K, Sun G, Kong Y, Fu Y (2021) Accurate and fast image denoising via attention guided scaling. IEEE Trans Image Process 30:6255–6265

    Google Scholar 

  153. Zhang Y, Deng W, Wang M, Hu J, Li X, Zhao D, Wen D (2020) Global-local GCN: large-scale label noise cleansing for face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 7731–7740

  154. Zhang H, Lan Y, Dai T, Qiao R, Xu Y, Yao Y, Xia S-T (2019) Residual frame for noisy video classification according to perceptual quality in convolutional neural networks. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), Shanghai, China. IEEE, pp 242–247

  155. Zhao Y, Jiang Z, Men A, Ju G (2019) Pyramid real image denoising network. In: 2019 IEEE Visual Communications and Image Processing (VCIP). IEEE, pp 1–4

  156. Zhao C, Li C, Feng S, Su N (2021) Hyperspectral anomaly detection using bilateral-filtered generative adversarial networks. In: 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. IEEE, pp 4408–4411

  157. Zhao Y, Zhai D, Jiang J, Liu X (2020) ADRN: attention-based deep residual network for hyperspectral image denoising. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain. IEEE, pp 2668–2672

  158. Zhong T, Cheng M, Lu S, Dong X, Li Y (2022) RCEN: a deep-learning-based background noise suppression method for DAS-VSP records. IEEE Geosci Remote Sens Lett 19:1–5

    Google Scholar 

  159. Zhong Y, Jia S, Hu Y (2022) Denoising auto-encoder network combined classfication module for brain tumors detection. In: 2022 3rd International Conference on Electronic Communication and Artificial Intelligence (IWECAI). IEEE, pp 540–543

  160. Zhong L, Liu G, Yang G (2021) Blind denoising of fluorescence microscopy images using GAN-based global noise modeling. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE, pp 863–867

  161. Zhou J, Leong CT, Li C (2021) Multi-scale and attention residual network for single image dehazing. In: 2021 6th International Conference on Intelligent Computing and Signal Processing (ICSP), Xi’an, China. IEEE, pp 483–487

  162. Zhu H, Zhang D, Kou Y (2021) Dual attention fusion network for single image dehazing. In: 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP), Changsha, China. IEEE, pp 1–5

  163. Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: 2011 International Conference on Computer Vision. IEEE, pp 479–486

Download references

Acknowledgements

This research was supported by the Zhejiang Provincial Natural Science Foundation of China under Grant No. LQ21F020024; This work is supported by Ningbo Science and Technology Bureau under Major S &T Programme with project code 2021Z037; This work was supported by a research grant funded by National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1A4A1019463).

Author information

Authors and Affiliations

  1. School of Computer Science, University of Nottingham Ningbo China, Ningbo, 315100, Zhejiang, China

    Yuan Yu, Boon Giin Lee, Matthew Pike & Qian Zhang

  2. Nottingham Ningbo China Beacons of Excellence Research and Innovation Institute, Ningbo, 315000, Zhejiang, China

    Boon Giin Lee

  3. Department of Electronic Engineering, Pukyong National University, Busan, 48513, Republic of Korea

    Wan-Young Chung

Authors
  1. Yuan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Boon Giin Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew Pike
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wan-Young Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Boon Giin Lee.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, Y., Lee, B.G., Pike, M. et al. Deep learning-based RGB-thermal image denoising: review and applications. Multimed Tools Appl 83, 11613–11641 (2024). https://doi.org/10.1007/s11042-023-15916-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15916-7

Keywords

Search

Navigation