Lightweight multi-scale generative adversarial network with attention for image denoising
Authors: 
Xuegang Hu, Wei ZhaoAuthors Info & Claims
Abstract
Most existing image denoising methods focus on improving denoising effect while neglecting computational cost. In this paper, a lightweight multi-scale generative adversarial network with attention is proposed to achieve superior denoising performance while reducing computational redundancy. Specifically, A novel generator with two sub-networks is designed for end-to-end image denoising. In the upper branch network, an attention-guided multi-scale convolution (AMC) block and a U-shaped dilated convolution (UDC) block are utilized to filter and extract noise features at multiple scales, enhancing the model’s feature representation capability and generalization ability while reducing computational complexity. The lower branch network directly predicts denoised images, consisting mainly of a dilated convolution residual (DCR) block and a residual attention (RA) block. DCR is composed of residual network and dilated convolution, which can capture multi-scale features stably and effectively without increasing the number of parameters. RA can be regarded as a novel attention mechanism to further guide the network for image denoising and image recovery. Experimental results show that compared with some state-of-the-art methods, the proposed method has better performance in terms of parameter overhead, visual effect and objective metrics such as PSNR and SSIM.
References
[1]
Zhu M, Fu C, and Wang X Rkseg+: make full use of runge-kutta methods in medical image segmentation Multimed. Syst. 2024 30 2 65
[2]
Yelleni SH and Kumari D Monte carlo dropblock for modeling uncertainty in object detection Pattern Recogn. 2024 146
[3]
Hu X, Feng J, and Gong J Lffnet: lightweight feature-enhanced fusion network for real-time semantic segmentation of road scenes Pattern Anal. Appl. 2024 27 1 27
[4]
Dabov K, Foi A, Katkovnik V, and Egiazarian K Image denoising by sparse 3-d transform-domain collaborative filtering IEEE Trans. Image Process. 2007 16 8 2080-2095
[5]
Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: 2011 International Conference on Computer Vision, pp. 479–486 (2011).
[6]
Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2862–2869 (2014).
[7]
Gao J, Chen M, and Xu C Vectorized evidential learning for weakly-supervised temporal action localization IEEE Trans. Pattern Anal. Mach. Intell. 2023 45 12 15949-15963
[8]
Gao, J., Zhang, T., Xu, C.: I know the relationships: Zero-shot action recognition via two-stream graph convolutional networks and knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8303–8311 (2019).
[9]
Gao J and Xu C Learning video moment retrieval without a single annotated video IEEE Trans. Circuits Syst Video Technol. 2022 32 3 1646-1657
[10]
Gao J, Zhang T, and Xu C Learning to model relationships for zero-shot video classification IEEE Trans. Pattern Anal. Mach. Intell. 2021 43 10 3476-3491
[11]
Zhang K, Zuo W, Chen Y, Meng D, and Zhang L Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising IEEE Trans. Image Process. 2017 26 7 3142-3155
[12]
Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep cnn denoiser prior for image restoration. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3929–3938 (2017).
[13]
Zhang K, Zuo W, and Zhang L Ffdnet: Toward a fast and flexible solution for cnn-based image denoising IEEE Trans. Image Process. 2018 27 9 4608-4622
[14]
Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1712–1722 (2019).
[15]
El Helou M and Süsstrunk S Blind universal bayesian image denoising with gaussian noise level learning IEEE Trans. Image Process. 2020 29 4885-4897
[16]
Li, B., Liu, X., Hu, P., Wu, Z., Lv, J., Peng, X.: All-in-one image restoration for unknown corruption. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17431–17441 (2022).
[17]
Anwar, S., Barnes, N.: Real image denoising with feature attention. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3155–3164 (2019).
[18]
Yue, Z., Zhao, Q., Zhang, L., Meng, D.: Dual adversarial network: Toward real-world noise removal and noise generation. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16, pp. 41–58 (2020)
[19]
Vo DM, Nguyen DM, Le TP, and Lee S-W Hi-gan: A hierarchical generative adversarial network for blind denoising of real photographs Inform. Sci. 2021 570 225-240
[20]
Yu X, Fu Z, and Ge C A multi-scale generative adversarial network for real-world image denoising Signal Image Video Process. 2022 16 1 257-264
[21]
Thakur, R.K., Maji, S.K.: Image denoising based on multi domain feature inspired convolutional neural network. In: TENCON 2022 - 2022 IEEE Region 10 Conference (TENCON), pp. 1–6 (2022).
[22]
Thakur, R.K., Maji, S.K.: Blind gaussian deep denoiser network using multi-scale pixel attention. In: 2022 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp. 1–5 (2022).
[23]
Multi scale pixel attention and feature extraction based neural network for image denoising. Pattern Recognition 141, 109603 (2023).
[24]
Sheng J, Lv G, Wang Z, and Feng Q Srnet: Sparse representation-based network for image denoising Digital Signal Process. 2022 130 103702
[25]
Tian C, Zheng M, Zuo W, Zhang B, Zhang Y, and Zhang D Multi-stage image denoising with the wavelet transform Pattern Recogn. 2023 134 109050
[26]
Wu W, Lv G, Liao S, and Zhang Y Feunet: a flexible and effective u-shaped network for image denoising Signal Image Video Process. 2023 17 5 2545-2553
[27]
Jia X, Peng Y, Ge B, Li J, Liu S, and Wang W A multi-scale dilated residual convolution network for image denoising Neural Process. Lett. 2023 55 2 1231-1246
[28]
Wu, W., Lv, G., Duan, Y., Liang, P., Zhang, Y., Xia, Y.: Dcanet: Dual convolutional neural network with attention for image blind denoising. arXiv preprint (2023).
[29]
Wu W, Liu S, Xia Y, and Zhang Y Dual residual attention network for image denoising Pattern Recogn. 2024 149 110291
[30]
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
[31]
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 214–223 (2017)
[32]
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein gans 30 (2017)
[33]
Tassew T and Xuan N Dc-gan with feature attention for single image dehazing Signal Image Video Process. 2024 18 2167-2182
[34]
Mahmoud AA, Sayed HA, and Mohamed SS Variant wasserstein generative adversarial network applied on low dose ct image denoising Comput. Mater. Continua 2023 75 2 4535-4552
[35]
Tian C, Xu Y, Zuo W, Lin C-W, and Zhang D Asymmetric cnn for image superresolution IEEE Trans. Syst. Man Cybern. 2022 52 6 3718-3730
[36]
Guo X, Tu Z, Li G, Shen Z, and Wu W A novel lightweight multi-dimension feature fusion network for single-image super-resolution reconstruction Visual Comput. 2024 40 1685-1696
[37]
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint (2015).
[38]
Tian C, Xu Y, Li Z, Zuo W, Fei L, and Liu H Attention-guided cnn for image denoising Neural Netw. 2020 124 117-129
[39]
Cai J, Duan Z, Wang L, Meng J, and Yao Z Multiscale dilated denoising convolution with channel attention mechanism for micro-seismic signal denoising J. Petroleum Explor. Product. Technol. 2024 14 883-908
[40]
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)
[41]
Shan H, Fu X, Lv Z, Xu X, Wang X, and Zhang Y Synthetic aperture radar images denoising based on multi-scale attention cascade convolutional neural network Measurement Sci. Technol. 2023 34 8 085403
[42]
Zhao S, Lin S, Cheng X, Zhou K, Zhang M, and Wang H Dual-gan complementary learning for real-world image denoising IEEE Sensors J. 2024 24 1 355-366
[43]
Zhu, S., Xu, G., Cheng, Y., Han, X., Wang, Z.: Bdgan: Image blind denoising using generative adversarial networks. In: Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 241–252 (2019)
[44]
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1125–1134 (2017).
[45]
Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., Cottrell, G.: Understanding convolution for semantic segmentation. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1451–1460 (2018).
[46]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016).
[47]
Lai, W.-S., Huang, J.-B., Ahuja, N., Yang, M.-H.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 624–632 (2017).
[48]
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint (2014).
[49]
Wang Z, Bovik AC, Sheikh HR, and Simoncelli EP Image quality assessment: from error visibility to structural similarity IEEE Trans. Image Process. 2004 13 4 600-612
[50]
Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: Dataset and study. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1122–1131 (2017).
[51]
Abdelhamed, A., Lin, S., Brown, M.S.: A high-quality denoising dataset for smartphone cameras. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1692–1700 (2018).
[52]
Roth, S., Black, M.J.: Fields of experts: A framework for learning image priors. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 860–867 (2005).
[53]
Franzen, R.: Kodak lossless true color image suite. source: http://r0k.us/graphics/kodak 4(2), 9 (1999)
[54]
Huang, J.-B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5197–5206 (2015).
[55]
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint (2014).
Index Terms
- Lightweight multi-scale generative adversarial network with attention for image denoising
Index terms have been assigned to the content through auto-classification.
Recommendations
Pyramidal convolution attention generative adversarial network with data augmentation for image denoising
AbstractGenerative adversarial networks (GANs) have shown remarkable effects for various computer vision tasks. Standard convolution plays an important role in the GAN-based model. However, the single type of kernel with a single spatial size limits the ...
A Multi-scale Dilated Residual Convolution Network for Image Denoising
AbstractDue to the excellent performance of deep learning, more and more image denoising methods based on convolutional neural networks (CNN) are proposed, including dilated convolution method and multi-scale convolution method. A fundamental issue is how ...
Inter-scale Correlation Image Denoising Based on Non-aliasing Contourlet Transform
ICMTMA '10: Proceedings of the 2010 International Conference on Measuring Technology and Mechatronics Automation - Volume 02A novel image denoising method based on non-aliasing Contourlet transform(NACT) is presented according to coefficient inter-scale correlation. A noisy image was decomposed into a low frequency approximation sub-image and a series of high frequency ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 24 September 2024
Accepted: 17 September 2024
Received: 03 May 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Jan 2025