Abstract
With the rapid development of artificial intelligence in recent years, deep learning has shown great potential in the field of image denoising. However, most of the work is based on supervised learning, and the lack of clean images in the real application will limit neural network training. For this reason, self-supervised learning in the absence of clean images is getting more and more attentions. Nevertheless, since both the source and target in self-supervised training are the limited noisy image itself, such denoising methods suffer from overfitting. To this end, a stochastic triplet neighboring approach, thereafter referred to as STN, is proposed in this paper. Given an input noisy image, the source fed to the STN is the downsized sub-image via 4-neighbor sampling, whereas the target in STN training is a stochastic combination from the two neighbored sub-images. Such a mechanism is actually the augmentation of training data, which leads to the relief of the overfitting problem. Extensive experimental results show that our proposed STN approach outperforms the state-of-the-art image denoising methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Batson, J., Royer, L.: Noise2self: Blind denoising by self-supervision. In: International Conference on Machine Learning, pp. 524–533. PMLR (2019)
Bora, A., Price, E., Dimakis, A.G.: Ambientgan: Generative models from lossy measurements. In: International Conference on Learning Representations (2018)
Buchholz, T.O., Jordan, M., Pigino, G., Jug, F.: Cryo-CARE: content-aware image restoration for cryo-transmission electron microscopy data. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 502–506. IEEE (2019)
Ehret, T., Davy, A., Morel, J.M., Facciolo, G., Arias, P.: Model-blind video denoising via frame-to-frame training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11369–11378 (2019)
Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2019 (2019). https://doi.org/10.1109/CVPR.2019.00181
Hariharan, S.G., et al.: Learning-based x-ray image denoising utilizing model-based image simulations. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 549–557. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_61
Huang, T., Li, S., Jia, X., Lu, H., Liu, J.: Neighbor2Neighbor: self-supervised denoising from single noisy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14781–14790 (2021)
Izadi, S., Mirikharaji, Z., Zhao, M., Hamarneh, G.: Whitenner-blind image denoising via noise whiteness priors. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2019)
Krull, A., Buchholz, T.O., Jug, F.: Noise2void-Learning denoising from single noisy images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2019, pp. 2124–2132 (2019). https://doi.org/10.1109/CVPR.2019.00223
Krull, A., Vičar, T., Prakash, M., Lalit, M., Jug, F.: Probabilistic noise2void: Unsupervised content-aware denoising. Front. Comput. Sci. 2, 5 (2020)
Laine, S., Karras, T., Lehtinen, J., Aila, T.: High-quality self-supervised deep image denoising. Adv. Neural. Inf. Process. Syst. 32, 6970–6980 (2019)
Lehtinen, J., et al.: Noise2Noise: learning Image Restoration without Clean Data. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 2965–2974. PMLR (2018)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings 8th International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)
Moran, N., Schmidt, D., Zhong, Y., Coady, P.: Noisier2noise: learning to denoise from unpaired noisy data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12064–12072 (2020)
Nam, S., Hwang, Y., Matsushita, Y., Kim, S.J.: A holistic approach to cross-channel image noise modeling and its application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1683–1691 (2016)
Pang, T., Zheng, H., Quan, Y., Ji, H.: Recorrupted-to-Recorrupted: unsupervised deep learning for image denoising. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2043–2052 (2021)
Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. Adv. Neural. Inf. Process. Syst. 32, 8026–8037 (2019)
Quan, Y., Chen, M., Pang, T., Ji, H.: Self2self with dropout: Learning self-supervised denoising from single image. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1887–1895 (2020). https://doi.org/10.1109/CVPR42600.2020.00196
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Wu, D., Gong, K., Kim, K., Li, X., Li, Q.: Consensus neural network for medical imaging denoising with only noisy training samples. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 741–749. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_81
Xie, Y., Wang, Z., Ji, S.: Noise2same: optimizing a self-supervised bound for image denoising. arXiv preprint arXiv:2010.11971 (2020)
Xu, J., Li, H., Liang, Z., Zhang, D., Zhang, L.: Real-world noisy image denoising: a new benchmark. arXiv preprint arXiv:1804.02603 (2018)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)
Zhang, Y., et al.: A poisson-gaussian denoising dataset with real fluorescence microscopy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11710–11718 (2019)
Zhang K., Z.W., Zhang, L.: FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)
Zhussip, M., Soltanayev, S., Chun, S.Y.: Extending stein’s unbiased risk estimator to train deep denoisers with correlated pairs of noisy images. Adv. Neural. Inf. Process. Syst. 32, 1465–1475 (2019)
Acknowledgement
This work is supported by Ministry of Science and Technology China (MOST) Major Program on New Generation of Artificial Intelligence 2030 No. 2018AAA0102200. It is also supported by Natural Science Foundation China (NSFC) Major Project No. 61827814 and Shenzhen Science and Technology Innovation Commission (SZSTI) project No. JCYJ20190808153619413.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wan, B., Shi, D., Liu, Y. (2023). STN: Stochastic Triplet Neighboring Approach to Self-supervised Denoising from Limited Noisy Images. In: Dang-Nguyen, DT., et al. MultiMedia Modeling. MMM 2023. Lecture Notes in Computer Science, vol 13833. Springer, Cham. https://doi.org/10.1007/978-3-031-27077-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-27077-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27076-5
Online ISBN: 978-3-031-27077-2
eBook Packages: Computer ScienceComputer Science (R0)