[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Dual convolutional neural network with attention for image blind denoising

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Noise removal of images is an essential preprocessing procedure for many computer vision tasks. Currently, many denoising models based on deep neural networks can perform well in removing the noise with known distributions (i.e. the additive Gaussian white noise). However eliminating real noise is still a very challenging task, since real-world noise often does not simply follow one single type of distribution, and the noise may spatially vary. In this paper, we present a novel dual convolutional neural network (CNN) with attention for image blind denoising, named as the DCANet. To the best of our knowledge, the proposed DCANet is the first work that integrates both the dual CNN and attention mechanism for image denoising. The DCANet is composed of a noise estimation network, a spatial and channel attention module (SCAM), and a dual CNN. The noise estimation network is utilized to estimate the spatial distribution and the noise level in an image. The noisy image and its estimated noise are combined as the input of the SCAM, and a dual CNN contains two different branches is designed to learn the complementary features to obtain the denoised image. The experimental results have verified that the proposed DCANet can suppress both synthetic and real noise effectively. The code of DCANet is available at https://github.com/WenCongWu/DCANet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

If necessary, data involved in this work can be provided by the corresponding author.

References

  1. Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. IEEE Conference on Computer Vision and Pattern Recognition 60–65 (2005)

  2. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.O.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    MathSciNet  Google Scholar 

  3. Xu, J., Zhang, L., Zhang, D.: A trilateral weighted sparse coding scheme for real-world image denoising. European Conference on Computer Vision 11212, 21–38 (2018)

    Google Scholar 

  4. Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. IEEE Conference on Computer Vision and Pattern Recognition 2862–2869 (2014)

  5. Xu, J., Zhang, L., Zhang, D., Feng, X.: Multi-channel weighted nuclear norm minimization for real color image denoising. International Conference on Computer Vision 1105–1113 (2017)

  6. Ou, Y., Swamy, M.N.S., Luo, J., Li, B.: Single image denoising via multi-scale weighted group sparse coding. Signal Process. 200, 108650 (2022)

    Google Scholar 

  7. Shan, Y., Hu, D., Wang, Z., Jia, T.: Multi-channel nuclear norm minus frobenius norm minimization for color image denoising. Signal Process. 207, 108959 (2023)

    Google Scholar 

  8. Wang, Y., Yao, Q., Kwok, J.T.: A scalable, adaptive and sound nonconvex regularizer for low-rank matrix learning. International World Wide Web Conference 1798–1808 (2021)

  9. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    MathSciNet  Google Scholar 

  10. Chatterjee, P., Milanfar, P.: Is denoising dead? IEEE Trans. Image Process. 19(4), 895–911 (2010)

    MathSciNet  Google Scholar 

  11. Zhang, K., Zuo, W., Zhang, L.: Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)

    MathSciNet  Google Scholar 

  12. Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. IEEE Conference on Computer Vision and Pattern Recognition 1712–1722 (2019)

  13. Helou, M.E., Süsstrunk, S.: Blind universal bayesian image denoising with gaussian noise level learning. IEEE Trans. Image Process. 29, 4885–4897 (2020)

    Google Scholar 

  14. Li, B., et al.: All-in-one image restoration for unknown corruption. IEEE Conference on Computer Vision and Pattern Recognition 17431–17441 (2022)

  15. Tian, C., Xu, Y., Zuo, W.: Image denoising using deep CNN with batch renormalization. Neural Netw. 121, 461–473 (2020)

    Google Scholar 

  16. Pan, J., et al.: Learning dual convolutional neural networks for low-level vision. IEEE Conference on Computer Vision and Pattern Recognition 3070–3079 (2018)

  17. Anwar, S., Barnes, N.: Real image denoising with feature attention. International Conference on Computer Vision 3155–3164 (2019)

  18. Tian, C., et al.: Attention-guided CNN for image denoising. Neural Netw. 124, 117–129 (2020)

    Google Scholar 

  19. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. IEEE Conference on Computer Vision and Pattern Recognition 2808–2817 (2017)

  20. Peng, Y., et al.: Dilated residual networks with symmetric skip connection for image denoising. Neurocomputing 345, 67–76 (2019)

    Google Scholar 

  21. Soh, J.W., Cho, N.I.: Deep universal blind image denoising. International Conference on Pattern Recognition 747–754 (2020)

  22. Yue, Z., Yong, H., Zhao, Q., Meng, D., Zhang, L.: Variational denoising network: Toward blind noise modeling and removal. Advances in Neural Information Processing Systems 1688–1699 (2019)

  23. Kim, Y., Soh, J.W., Park, G.Y., Cho, N.I.: Transfer learning from synthetic to real-noise denoising with adaptive instance normalization. IEEE Conference on Computer Vision and Pattern Recognition 3479–3489 (2020)

  24. Zuo, Y., et al.: Cfnet: Conditional filter learning with dynamic noise estimation for real image denoising. Knowl.-Based Syst. 284, 111320 (2024)

    Google Scholar 

  25. Gao, J., Zhang, T., Xu, C.: I know the relationships: Zero-shot action recognition via two-stream graph convolutional networks and knowledge graphs. AAAI Conference on Artificial Intelligence 8303–8311 (2019)

  26. Gao, J., Zhang, T., Xu, C.: Learning to model relationships for zero-shot video classification. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3476–3491 (2021)

    Google Scholar 

  27. Gao, J., Xu, C.: Learning video moment retrieval without a single annotated video. IEEE Trans. Circuits Syst. Video Technol. 32(3), 1646–1657 (2022)

    Google Scholar 

  28. Gao, J., Chen, M., Xu, C.: Vectorized evidential learning for weakly-supervised temporal action localization. IEEE Trans. Pattern Anal. Mach. Intell. 45(12), 15949–15963 (2023)

    Google Scholar 

  29. Du, B., Wei, Q., Liu, R.: An improved quantum-behaved particle swarm optimization for endmember extraction. IEEE Trans. Geosci. Remote Sens. 57(8), 6003–6017 (2019)

    Google Scholar 

  30. Li, J., Lu, G., Zhang, B., You, J., Zhang, D.: Shared linear encoder-based multikernel gaussian process latent variable model for visual classification. IEEE Trans. Cybern. 51(2), 534–547 (2021)

    Google Scholar 

  31. Liang, X., Zhang, D., Lu, G., Guo, Z., Luo, N.: A novel multicamera system for high-speed touchless palm recognition. IEEE Trans. Syst. Man Cybern. Syst. 51(3), 1534–1548 (2021)

    Google Scholar 

  32. Li, Y., et al.: Attention-guided unified network for panoptic segmentation. IEEE Conference on Computer Vision and Pattern Recognition 7026–7035 (2019)

  33. Larochelle, H., Hinton, G.E.: Learning to combine foveal glimpses with a third-order boltzmann machine. Advances in Neural Information Processing Systems 1243–1251 (2010)

  34. Hou, G., Yang, Y., Xue, J.: Residual dilated network with attention for image blind denoising. International Conference on Multimedia and Expo 248–253 (2019)

  35. Zamir, S.W., et al.: Multi-stage progressive image restoration. IEEE Conference on Computer Vision and Pattern Recognition 14821–14831 (2021)

  36. Zhang, Y., Li, K., Li, K., Zhong, B., Fu, Y.: Residual non-local attention networks for image restoration. International Conference on Learning Representations (2019)

  37. Wang, Y., Song, X., Chen, K.: Channel and space attention neural network for image denoising. IEEE Signal Process. Lett. 28, 424–428 (2021)

    Google Scholar 

  38. Ren, C., He, X., Wang, C., Zhao, Z.: Adaptive consistency prior based deep network for image denoising. IEEE Conference on Computer Vision and Pattern Recognition 8596–8606 (2021)

  39. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. IEEE Conference on Computer Vision and Pattern Recognition 2472–2481 (2018)

  40. Tian, C., et al.: Designing and training of a dual CNN for image denoising. Knowl. Based Syst. 226, 106949 (2021)

    Google Scholar 

  41. Zhang, Q., et al.: A parallel and serial denoising network. Expert Syst. Appl. 231, 120628 (2023)

    Google Scholar 

  42. Pan, J., et al.: Dual convolutional neural networks for low-level vision. Int. J. Comput. Vis. 130(6), 1440–1458 (2022)

    Google Scholar 

  43. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. International Conference on Machine Learning 37, 448–456 (2015)

    Google Scholar 

  44. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 1106–1114 (2012)

  45. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. British Machine Vision Conference 1–10 (2012)

  46. Malfliet, W., Hereman, W.: The tanh method: I. exact solutions of nonlinear evolution and wave equations. Phys. Scr. 54(6), 563 (1996)

    MathSciNet  Google Scholar 

  47. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. International Conference on Computer Vision 1026–1034 (2015)

  48. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. IEEE Conference on Computer Vision and Pattern Recognition 7132–7141 (2018)

  49. Woo, S., Park, J., Lee, J., Kweon, I.S.: CBAM: convolutional block attention module. European Conference on Computer Vision 11211, 3–19 (2018)

    Google Scholar 

  50. Han, J., Moraga, C.: The influence of the sigmoid function parameters on the speed of backpropagation learning. International Workshop on Artificial Neural Networks 930, 195–201 (1995)

    Google Scholar 

  51. Szegedy, C., et al.: Going deeper with convolutions. IEEE Conference on Computer Vision and Pattern Recognition 1–9 (2015)

  52. Ranzato, M., Boureau, Y., LeCun, Y.: Sparse feature learning for deep belief networks. Advances in Neural Information Processing Systems 1185–1192 (2007)

  53. Huang, W., Xue, Y., Hu, L., Liuli, H.: S-eegnet: Electroencephalogram signal classification based on a separable convolution neural network with bilinear interpolation. IEEE Access 8, 131636–131646 (2020)

    Google Scholar 

  54. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (2016)

  55. Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. IEEE Conference on Computer Vision and Pattern Recognition 2261–2269 (2017)

  56. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. International Conference on Learning Representations (2016)

  57. Yu, J., Yang, X., Gao, F., Tao, D.: Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans. Cybern. 47(12), 4014–4024 (2017)

    Google Scholar 

  58. Wang, P., et al.: Understanding convolution for semantic segmentation. IEEE Winter Conference on Applications of Computer Vision 1451–1460 (2018)

  59. Yang, W., Liu, J., Yang, S., Guo, Z.: Scale-free single image deraining via visibility-enhanced recurrent wavelet learning. IEEE Trans. Image Process. 28(6), 2948–2961 (2019)

    MathSciNet  Google Scholar 

  60. Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. IEEE Conference on Computer Vision and Pattern Recognition 695–704 (2018)

  61. Lai, W., Huang, J., Ahuja, N., Yang, M.: Deep laplacian pyramid networks for fast and accurate super-resolution. IEEE Conference on Computer Vision and Pattern Recognition 5835–5843 (2017)

  62. Jiang, K., et al.: Multi-scale progressive fusion network for single image deraining. IEEE Conference on Computer Vision and Pattern Recognition 8343–8352 (2020)

  63. Wu, W., Liu, S., Xia, Y., Zhang, Y.: Dual residual attention network for image denoising. Pattern Recogn. 149, 110291 (2024)

    Google Scholar 

  64. Kamgar-Parsi, B., Rosenfeld, A.: Optimally isotropic laplacian operator. IEEE Trans. Image Process. 8(10), 1467–1472 (1999)

    MathSciNet  Google Scholar 

  65. Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: Dataset and study. IEEE Conference on Computer Vision and Pattern Recognition Workshops 1122–1131 (2017)

  66. Roth, S., Black, M.J.: Fields of experts: A framework for learning image priors. IEEE Conference on Computer Vision and Pattern Recognition 860–867 (2005)

  67. Franzen, R.: Kodak lossless true color image suite (1999). http://r0k.us/graphics/kodak/

  68. Zhang, L., Wu, X., Buades, A., Li, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J. Electron. Imaging 20(2), 1–17 (2011)

    Google Scholar 

  69. Abdelhamed, A., Lin, S., Brown, M.S.: A high-quality denoising dataset for smartphone cameras. IEEE Conference on Computer Vision and Pattern Recognition 1692–1700 (2018)

  70. Anaya, J., Barbu, A.: RENOIR - A dataset for real low-light image noise reduction. J. Vis. Commun. Image Represent. 51, 144–154 (2018)

    Google Scholar 

  71. Plötz, T., Roth, S.: Benchmarking denoising algorithms with real photographs. IEEE Conference on Computer Vision and Pattern Recognition 2750–2759 (2017)

  72. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. International Conference on Learning Representations (2015)

  73. Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. International Conference on Learning Representations (2017)

  74. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Google Scholar 

  75. Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1256–1272 (2017)

    Google Scholar 

  76. Yang, X., Xu, Y., Quan, Y., Ji, H.: Image denoising via sequential ensemble learning. IEEE Trans. Image Process. 29, 5038–5049 (2020)

    Google Scholar 

  77. Tian, C., et al.: Multi-stage image denoising with the wavelet transform. Pattern Recognit. 134, 109050 (2023)

    Google Scholar 

  78. Zhang, Q., Xiao, J., Tian, C., Lin, J.C., Zhang, S.: A robust deformed convolutional neural network (CNN) for image denoising. CAAI Trans. Intell. Technol. 8(2), 331–342 (2023)

    Google Scholar 

  79. Thakur, R.K., Maji, S.K.: Multi scale pixel attention and feature extraction based neural network for image denoising. Pattern Recognit. 141, 109603 (2023)

    Google Scholar 

  80. Deng, J., Hu, C.: Recovering a clean background: a new progressive multi-scale cnn for image denoising. Signal, Image and Video Processing 1–12 (2024)

  81. Jiang, B., Lu, Y., Zhang, B., Lu, G.: Few-shot learning for image denoising. IEEE Trans. Circuits Syst. Video Technol. 33(9), 4741–4753 (2023)

    Google Scholar 

  82. Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32(200), 675–701 (1937)

    Google Scholar 

  83. Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11(1), 86–92 (1940)

    MathSciNet  Google Scholar 

  84. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  Google Scholar 

  85. Nemenyi, P.B.: Distribution-free multiple comparisons. (Princeton University, 1963)

  86. Jiang, B., Lu, Y., Wang, J., Lu, G., Zhang, D.: Deep image denoising with adaptive priors. IEEE Trans. Circuits Syst. Video Technol. 32(8), 5124–5136 (2022)

    Google Scholar 

  87. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. IEEE International Conference on Computer Vision 3397–3405 (2015)

  88. Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. IEEE Conference on Computer Vision and Pattern Recognition 2736–2744 (2016)

  89. Gu, S., Meng, D., Zuo, W., Zhang, L.: Joint convolutional analysis and synthesis sparse representation for single image layer separation. IEEE International Conference on Computer Vision 1717–1725 (2017)

  90. Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.W.: Clearing the skies: A deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)

    MathSciNet  Google Scholar 

  91. Fu, X., et al.: Removing rain from single images via a deep detail network. IEEE Conference on Computer Vision and Pattern Recognition 1715–1723 (2017)

  92. Qian, R., Tan, R.T., Yang, W., Su, J., Liu, J.: Attentive generative adversarial network for raindrop removal from a single image. IEEE Conference on Computer Vision and Pattern Recognition 2482–2491 (2018)

  93. Wei, W., Meng, D., Zhao, Q., Xu, Z., Wu, Y.: Semi-supervised transfer learning for image rain removal. IEEE Conference on Computer Vision and Pattern Recognition 3877–3886 (2019)

  94. Chen, D., et al.: Gated context aggregation network for image dehazing and deraining. IEEE Winter Conference on Applications of Computer Vision 1375–1383 (2019)

  95. Wang, T., et al.: Spatial attentive single-image deraining with a high quality real rain dataset. IEEE Conference on Computer Vision and Pattern Recognition 12270–12279 (2019)

  96. Fu, X., Liang, B., Huang, Y., Ding, X., Paisley, J.W.: Lightweight pyramid networks for image deraining. IEEE Trans. Neural Networks Learn. Syst. 31(6), 1794–1807 (2020)

    Google Scholar 

  97. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3943–3956 (2020)

    Google Scholar 

  98. Deng, S., et al.: Detail-recovery image deraining via context aggregation networks. IEEE Conference on Computer Vision and Pattern Recognition 14548–14557 (2020)

  99. Jiang, K., et al.: Rain-free and residue hand-in-hand: A progressive coupled network for real-time image deraining. IEEE Trans. Image Process. 30, 7404–7418 (2021)

    Google Scholar 

  100. Hsu, W., Chang, W.: Recurrent wavelet structure-preserving residual network for single image deraining. Pattern Recognit. 137, 109294 (2023)

    Google Scholar 

Download references

Acknowledgements

This work is funded by the Applied Basic Research Foundation of Yunnan Province under grant No. 202001AT070077, the Yunnan Fundamental Research Projects under grant No. 202401AU070052, and the Natural Science Foundation of China No. 61863037, No. 41971392.

Author information

Authors and Affiliations

  1. School of Information Science, Yunnan Normal University, No. 798, Juxian Street, Kunming, 650500, Yunnan Province, China

    Wencong Wu, Guannan Lv, Yingying Duan, Peng Liang, Yungang Zhang & Yuelong Xia

Authors
  1. Wencong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guannan Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yingying Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yungang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuelong Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Wencong Wu conceived and designed the study. Wencong Wu, Guannan Lv and Yingying Duan performed the experiments. Guannan Lv were responsible for drawing figures and tables. Data analysis and collation were carried out by Wencong Wu and Peng Liang. Wencong Wu, Yungang Zhang and Yuelong Xia wrote the paper. Yungang Zhang and Yuelong Xia provided the funding support. All authors read and approved the manuscript.

Corresponding authors

Correspondence to Yungang Zhang or Yuelong Xia.

Ethics declarations

Conflict of interest

The authors declare that they have no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, W., Lv, G., Duan, Y. et al. Dual convolutional neural network with attention for image blind denoising. Multimedia Systems 30, 263 (2024). https://doi.org/10.1007/s00530-024-01469-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01469-8

Keywords

Search

Navigation