[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content
Springer Nature Link
Log in

MGRLN-Net: Mask-Guided Residual Learning Network for Joint Single-Image Shadow Detection and Removal

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13843))

Included in the following conference series:

Abstract

Although significant progress has been made in single-image shadow detection or single-image shadow removal, only few works consider these two problems together. However, the two problems are complementary and can benefit from each other. In this work, we propose a Mask-Guided Residual Learning Network (MGRLN-Net) that jointly estimates shadow mask and shadow-free image. In particular, MGRLN-Net first generates a shadow mask, then utilizes a feature reassembling module to align the features from the shadow detection module to the shadow removal module. Finally, we leverage the learned shadow mask as guidance to generate a shadow-free image. We formulate shadow removal as a masked residual learning problem of the original shadow image. In this way, the learned shadow mask is used as guidance to produce better transitions in penumbra regions. Extensive experiments on ISTD, ISTD+, and SRD benchmark datasets demonstrate that our method outperforms current state-of-the-art approaches on both shadow detection and shadow removal tasks. Our code is available at https://github.com/LeipingJie/MGRLN-Net.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 71.50
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 89.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Chen, Z., Zhu, L., Wan, L., Wang, S., Feng, W., Heng, P.A.: A multi-task mean teacher for semi-supervised shadow detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5611–5620 (2020)

    Google Scholar 

  2. Chen, Z., Long, C., Zhang, L., Xiao, C.: CANet: a context-aware network for shadow removal. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4743–4752 (2021)

    Google Scholar 

  3. Cun, X., Pun, C.M., Shi, C.: Towards ghost-free shadow removal via dual hierarchical aggregation network and shadow matting GAN. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 10680–10687 (2020)

    Google Scholar 

  4. Ding, B., Long, C., Zhang, L., Xiao, C.: ARGAN: attentive recurrent generative adversarial network for shadow detection and removal. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  5. Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. Int. J. Comput. Vision 85(1), 35–57 (2009)

    Article  Google Scholar 

  6. Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 59–68 (2006). https://doi.org/10.1109/TPAMI.2006.18

    Article  Google Scholar 

  7. Fu, L., et al.: Auto-exposure fusion for single-image shadow removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10571–10580 (2021)

    Google Scholar 

  8. Goodfellow, I., et al.: Generative adversarial nets. In: Proceedings of International Conference on Neural Information Processing Systems (NeurIPS), pp. 2672–2680 (2014)

    Google Scholar 

  9. Gryka, M., Terry, M., Brostow, G.J.: Learning to remove soft shadows. ACM Trans. Graph. (TOG) 34(5), 1–15 (2015)

    Article  Google Scholar 

  10. Guo, R., Dai, Q., Hoiem, D.: Paired regions for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2956–2967 (2012)

    Article  Google Scholar 

  11. Hu, X., Fu, C.W., Zhu, L., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 42(11), 2795–2808 (2019)

    Article  Google Scholar 

  12. Hu, X., Jiang, Y., Fu, C.W., Heng, P.A.: Mask-shadowGAN: learning to remove shadows from unpaired data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2472–2481 (2019)

    Google Scholar 

  13. Hu, X., Zhu, L., Fu, C.W., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7454–7462 (2018)

    Google Scholar 

  14. Inoue, N., Yamasaki, T.: Learning from synthetic shadows for shadow detection and removal. IEEE Trans. Circuits Syst. Video Technol. 31(11), 4187–4197 (2021). https://doi.org/10.1109/TCSVT.2020.3047977

    Article  Google Scholar 

  15. Jie, L., Zhang, H.: A fast and efficient network for single image shadow detection. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2634–2638 (2022)

    Google Scholar 

  16. Jie, L., Zhang, H.: RMLANet: random multi-level attention network for shadow detection. In: 2022 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2022)

    Google Scholar 

  17. Jin, Y., Sharma, A., Tan, R.T.: DC-ShadowNet: single-image hard and soft shadow removal using unsupervised domain-classifier guided network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5027–5036 (2021)

    Google Scholar 

  18. Khan, S.H., Bennamoun, M., Sohel, F., Togneri, R.: Automatic feature learning for robust shadow detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1939–1946 (2014)

    Google Scholar 

  19. Lalonde, J.-F., Efros, A.A., Narasimhan, S.G.: Detecting ground shadows in outdoor consumer photographs. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 322–335. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15552-9_24

    Chapter  Google Scholar 

  20. Le, H., Samaras, D.: Shadow removal via shadow image decomposition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8578–8587 (2019)

    Google Scholar 

  21. Le, H., Samaras, D.: From shadow segmentation to shadow removal. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12356, pp. 264–281. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_16

    Chapter  Google Scholar 

  22. Nguyen, V., Vicente, T.F.Y., Zhao, M., Hoai, M., Samaras, D.: Shadow detection with conditional generative adversarial networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4510–4518 (2017)

    Google Scholar 

  23. Qu, L., Tian, J., He, S., Tang, Y., Lau, R.W.H.: DeshadowNet: a multi-context embedding deep network for shadow removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4067–4075 (2017)

    Google Scholar 

  24. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2015)

    Google Scholar 

  25. Shen, L., Chua, T.W., Leman, K.: Shadow optimization from structured deep edge detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2067–2074 (2015)

    Google Scholar 

  26. Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional lstm network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems 28 (2015)

    Google Scholar 

  27. Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: Proceedings of the 36th International Conference on Machine Learning (ICML), pp. 6105–6114 (2019)

    Google Scholar 

  28. Vicente, T.F.Y., Hoai, M., Samaras, D.: Leave-one-out kernel optimization for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 682–695 (2018)

    Article  Google Scholar 

  29. Wang, J., Li, X., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1788–1797 (2018)

    Google Scholar 

  30. Xiao, C., She, R., Xiao, D., Ma, K.L.: Fast shadow removal using adaptive multi-scale illumination transfer. Comput. Graph. Forum 32, 6105–6114 (2019)

    Google Scholar 

  31. Yang, Q., Tan, K.H., Ahuja, N.: Shadow removal using bilateral filtering. IEEE Trans. Image Process. 21(10), 4361–4368 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  32. Zhang, L., Zhang, Q., Xiao, C.: Shadow remover: image shadow removal based on illumination recovering optimization. IEEE Trans. Image Process. 24(11), 4623–4636 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  33. Zheng, Q., Qiao, X., Cao, Y., Lau, R.W.: Distraction-aware shadow detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5167–5176 (2019)

    Google Scholar 

  34. Zhu, J., Samuel, K.G., Masood, S.Z., Tappen, M.F.: Learning to recognize shadows in monochromatic natural images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 223–230 (2010)

    Google Scholar 

  35. Zhu, L., et al.: Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 122–137. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_8

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (62076029), Guangdong Science and Technology Department (2017A030313362), Guangdong Key Lab of AI and Multi-modal Data Processing (2020KSYS007). and internal funds of the United International College (R202012, R201802, R5201904, UICR0400025-21).

Author information

Authors and Affiliations

  1. Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China

    Leiping Jie

  2. Department of Computer Science and Technology, BNU-HKBU United International College, Zhuhai, China

    Leiping Jie & Hui Zhang

Authors
  1. Leiping Jie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Zhang .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jie, L., Zhang, H. (2023). MGRLN-Net: Mask-Guided Residual Learning Network for Joint Single-Image Shadow Detection and Removal. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13843. Springer, Cham. https://doi.org/10.1007/978-3-031-26313-2_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26313-2_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26312-5

  • Online ISBN: 978-3-031-26313-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation