[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

FPPN: fast pixel purification network for single-image super-resolution

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

With the development of deep learning, convolution neural networks (CNNs) have drawn increasing attention in the field of single-image super-resolution (SISR). The previous methods have achieved excellent performance. However, in order to boost the performance, researchers often stack the basic blocks and deepen the networks, which leads to training difficulties, memory consumption, slow running, and other problems. The main goal of our work is to design a lightweight convolutional neural network to make a trade-off between performance and network capacity, thus relieving the aforementioned problems. On the one hand, we elaborately design a lightweight fast pixel purification block (FPPB), which is used to extract deep pixel features through multi-step pixel attention. On the other hand, our network adopts the structure of two path. One path takes LR images as input while the other takes gradient images as input. Because gradient images, which are rich in high-frequency information, can promote the recovery and reconstruction of image edges without the introduction of additional parameters. Experiments demonstrate that our method achieves superior SR performance while maintaining a small number of parameters and the computations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: Dataset and study. CVPR Workshops pp. 1122–1131 (2017)

  2. Ahn, N., Kang, B., Sohn, K.A.: Fast, accurate, and lightweight super-resolution with cascading residual network. ECCV, pp. 256–272 (2018)

  3. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. BMVC pp. 1–10 (2012)

  4. Dai, T., Cai, J., Zhang, Y., Xia, S.T., Zhang, L.: Second-order attention network for single image super-resolution. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

  5. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. ECCV 4, 184–199 (2014)

    Google Scholar 

  6. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  7. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. European Conference on Computer Vision, pp. 391–407 (2016)

  8. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. IEEE Conference on Computer Vision and Pattern Recognition (2015)

  9. Hui, Z., Wang, X., Gao, X.: Fast and accurate single image super-resolution via information distillation network. CVPR, pp. 723–731 (2018)

  10. Jie, H., Li, S., Gang, S., Albanie, S.: Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 99 (2017)

  11. Jin, X., Xiong, Q., Xiong, C., Li, Z., Gao, Z.: Single image super-resolution with multi-level feature fusion recursive network. Neurocomputing 370, 166–173 (2019)

    Article  Google Scholar 

  12. Kim, J., Lee, K.J., Lee, M.K.: Accurate image super-resolution using very deep convolutional networks. CVPR, pp. 1646–1654 (2016)

  13. Kim, J., Lee, K.J., Lee, M.K.: Deeply-recursive convolutional network for image super-resolution. CVPR (2016)

  14. Kingma, P.D., Ba, L.J.: Adam: A method for stochastic optimization. international conference on learning representations (2015)

  15. Kumar, K., Shrimankar, D.D.: F-des: Fast and deep event summarization. IEEE Trans. Multimed. 20(2), 323–334 (2017)

    Article  Google Scholar 

  16. Kumar, K., Shrimankar, D.D.: Deep event learning boost-up approach: delta. Multimed. Tools Appl. 77(20), 26635–26655 (2018)

    Article  Google Scholar 

  17. Kumar, K., Shrimankar, D.D.: Esumm: Event summarization on scale-free networks. Iete Technical Review 36(3), 265–274 (2019)

    Article  Google Scholar 

  18. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep laplacian pyramid networks for fast and accurate super-resolution. CVPR (2017)

  19. Liu, J., Zhang, W., Tang, Y., Tang, J., Wu, G.: Residual feature aggregation network for image super-resolution. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2359–2368 (2020)

  20. Mao, X.J., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Neural information processing systems, 29, pp. 2810–2818 (2016)

  21. Martin, R.D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference pp. 416–423, vol.2 (2001)

  22. Matsui, Y., Ito, K, Aramaki, Y., Yamasaki, T., Aizawa, K.: Sketch-based manga retrieval using manga109 dataset. Multimedia Tools Appl. 76(20), 21811–21838 (2017)

    Article  Google Scholar 

  23. Meng, B., Wang, L., He, Z., Jeon, G., Dou, Q., Yang, X.: Gradient information distillation network for real-time single-image super-resolution. J Real-Time Image Process. 18(2), 33–34 (2021)

    Article  Google Scholar 

  24. Park, J., Woo, S., Lee, J.Y., Kweon, I.S.: Bam: Bottleneck attention module. BMVC, p.147 (2018)

  25. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017)

  26. Roy, A.G., Navab, N., Wachinger, C.: Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks. International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 421–429 (2018)

  27. Salvador, J., Perez-Pellitero, E.: Naive bayes super-resolution forest. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 325–333 (2015)

  28. Sharma, S., Kumar, K.: Asl-3dcnn: American sign language recognition technique using 3-d convolutional neural networks. Multimed. Tools Appl. 18(17), 1–13 (2021)

    Google Scholar 

  29. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, P.A., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. CVPR, pp. 1874–1883 (2016)

  30. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. CVPR pp. 2790–2798 (2017)

  31. Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: A persistent memory network for image restoration. ICCV 4549–4557 (2017)

  32. Tian, C., Fei, L., Zheng, W., Xu, Y., Zuo, W., Lin, C.W.: Deep learning on image denoising: an overview. Neural Netw. 131, 251–275 (2020)

    Article  Google Scholar 

  33. Tian, C., Xu, Y., Li, Z., Zuo, W., Fei, L., Liu, H.: Attention-guided CNN for image denoising. Neural Netw. 124, 117–129 (2020)

    Article  Google Scholar 

  34. Tian, C., Xu, Y., Zuo, W.: Image denoising using deep CNN with batch renormalization. Neural Netw. 121, 461–473 (2020)

    Article  Google Scholar 

  35. Tian, C., Xu, Y., Zuo, W., Du, B., Lin, C.W., Zhang, D.: Designing and training of a dual CNN for image denoising. Knowl. Based Syst. 226, 106949 (2021)

    Article  Google Scholar 

  36. Tian, C., Xu, Y., Zuo, W., Lin, C.W., Zhang, D.: Asymmetric cnn for image super-resolution. IEEE Transactions on Systems, Man, and Cybernetics 1–13 (2021)

  37. Tian, C., Xu, Y., Zuo, W., Zhang, B., Fei, L., Lin, C.W.: Coarse-to-fine CNN for image super-resolution. IEEE Trans. Multimed. 23, 1489–1502 (2021)

    Article  Google Scholar 

  38. Tian, C., Zhuge, R., Wu, Z., Xu, Y., Zuo, W., Chen, C., Lin, C.W.: Lightweight image super-resolution with enhanced CNN. Knowl. Based Syst. 205, 106235 (2020)

    Article  Google Scholar 

  39. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., Hu, Q.: Eca-net: Efficient channel attention for deep convolutional neural networks. CVPR, pp. 11534–11542 (2019)

  40. Wang, Z., Bovik, C.A., Sheikh, R.H., Simoncelli, P.E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  41. Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: Cbam: Convolutional block attention module. Springer, Cham (2018)

    Google Scholar 

  42. Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21(8), 3467–3478 (2012)

    Article  MathSciNet  Google Scholar 

  43. Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  44. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  Google Scholar 

  45. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. Curves and surfaces 711–730 (2010)

  46. Zhang, K., Zuo, W., Zhang, L.: Learning a single convolutional super-resolution network for multiple degradations. Computer vision and pattern recognition, pp. 3262–3271 (2018)

  47. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks, pp. 56–72 (2018)

  48. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018)

  49. Zhao, H., Kong, X., He, J., Qiao, Y., Dong, C.: Efficient image super-resolution using pixel attention, pp. 56–72 (2020)

Download references

Acknowledgements

The research in our paper is funded from Sichuan University under grant 2020SCUNG205.

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu, Sichuan, 610064, China

    Bin Meng, Xiaomin Yang & Rongzhu Zhang

  2. School of Aeronautics and Astronautics, Sichuan University, Chengdu, 610064, China

    Bin Meng & Xiaomin Yang

  3. College of Electrical Engineering, Sichuan University, Chengdu, Sichuan, 610065, China

    Kai Liu

Authors
  1. Bin Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaomin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rongzhu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaomin Yang.

Additional information

Communicated by C. Yan.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, B., Yang, X., Zhang, R. et al. FPPN: fast pixel purification network for single-image super-resolution. Multimedia Systems 28, 281–293 (2022). https://doi.org/10.1007/s00530-021-00838-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-021-00838-x

Keywords

Search

Navigation