More Web Proxy on the site http://driver.im/

research-article

Single Image Deraining via Scale-space Invariant Attention Neural Network

Authors:

Xianming LiuAuthors Info & Claims

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Pages 375 - 383

https://doi.org/10.1145/3394171.3413554

Published: 12 October 2020 Publication History

Abstract

Image enhancement from degradation of rainy artifacts plays a critical role in outdoor visual computing systems. In this paper, we tackle the notion of scale that deals with visual changes in appearance of rain steaks with respect to the camera. Specifically, we revisit multi-scale representation by scale-space theory, and propose to represent the multi-scale correlation in convolutional feature domain, which is more compact and robust than that in pixel domain. Moreover, to improve the modeling ability of the network, we do not treat the extracted multi-scale features equally, but design a novel scale-space invariant attention mechanism to help the network focus on parts of the features. In this way, we summarize the most activated presence of feature maps as the salient features. Extensive experiments results on synthetic and real rainy scenes demonstrate the superior performance of our scheme over the state-of-the-arts. The source code of our method can be found in: https://github.com/pangbo1997/RainRemoval.

Supplementary Material

MP4 File (3394171.3413554.mp4)

Presentation Video of "Single Image Deraining via Scale-space Invariant Attention Neural Network"

Download
397.92 MB

References

[1]

Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley. 2017a. Clearing the skies: A deep network architecture for single-image rain removal. IEEE Transactions on Image Processing, Vol. 26, 6 (2017), 2944--2956.

Digital Library

[2]

Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. 2017b. Removing rain from single images via a deep detail network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3855--3863.

[3]

X. Fu, B. Liang, Y. Huang, X. Ding, and J. Paisley. 2019. Lightweight Pyramid Networks for Image Deraining. IEEE Transactions on Neural Networks and Learning Systems (2019), 1--14.

[4]

Shuhang Gu, Deyu Meng, Wangmeng Zuo, and Lei Zhang. 2017. Joint convolutional analysis and synthesis sparse representation for single image layer separation. In Proceedings of the IEEE International Conference on Computer Vision. 1708--1716.

[5]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

[6]

Xiaowei Hu, Chi-Wing Fu, Lei Zhu, and Pheng-Ann Heng. 2019. Depth-attentional features for single-image rain removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8022--8031.

[7]

Laurent Itti, Christof Koch, and Ernst Niebur. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis & Machine Intelligence 11 (1998), 1254--1259.

Digital Library

[8]

Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Baojin Huang, Yimin Luo, Jiayi Ma, and Junjun Jiang. 2020. Multi-Scale Progressive Fusion Network for Single Image Deraining. (2020), 1--8.

[9]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv (2014).

[10]

Minghan Li, Xiangyong Cao, Qian Zhao, Lei Zhang, Chenqiang Gao, and Deyu Meng. 2019. Video Rain/Snow Removal by Transformed Online Multiscale Convolutional Sparse Coding. arXiv (2019).

[11]

Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha. 2018. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In Proceedings of the European Conference on Computer Vision (ECCV). 254--269.

Digital Library

[12]

Yu Li, Robby T Tan, Xiaojie Guo, Jiangbo Lu, and Michael S Brown. 2016. Rain streak removal using layer priors. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2736--2744.

[13]

Jiaying Liu, Wenhan Yang, Shuai Yang, and Zongming Guo. 2018. Erase or Fill? Deep Joint Recurrent Rain Removal and Reconstruction in Videos. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1--8.

[14]

G Lowe. 2004. SIFT-the scale invariant feature transform. Int. J, Vol. 2 (2004), 91--110.

[15]

Yu Luo, Yong Xu, and Hui Ji. 2015. Removing rain from a single image via discriminative sparse coding. In Proceedings of the IEEE International Conference on Computer Vision. 3397--3405.

Digital Library

[16]

Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng. 2019. Progressive image deraining networks: a better and simpler baseline. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3937--3946.

[17]

Julius T Tou and Rafael C Gonzalez. 1974. Pattern recognition principles. (1974).

[18]

Paul Viola, Michael Jones, et almbox. 2001. Robust real-time object detection. International journal of computer vision, Vol. 4, 34--47 (2001), 4.

[19]

Hong Wang, Yichen Wu, Minghan Li, Qian Zhao, and Deyu Meng. 2019 a. A Survey on Rain Removal from Video and Single Image. arXiv preprint arXiv:1909.08326 (2019), 1--8.

[20]

Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson WH Lau. 2019 b. Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 12270--12279.

[21]

Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P. Simoncelli. 2004. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans Image Process, Vol. 13, 4 (2004).

Digital Library

[22]

Wei Wei, Deyu Meng, Qian Zhao, Zongben Xu, and Ying Wu. 2019. Semi-Supervised Transfer Learning for Image Rain Removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3877--3886.

[23]

SHI Xingjian, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. 2015. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Advances in neural information processing systems. 802--810.

[24]

Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. 2017. Deep joint rain detection and removal from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1357--1366.

[25]

Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Shuicheng Yan, and Zongming Guo. 2019. Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks. IEEE transactions on pattern analysis and machine intelligence (2019).

[26]

Wenhan Yang, Robby T Tan, Shiqi Wang, Yuming Fang, and Jiaying Liu. 2020. Single Image Deraining: From Model-Based to Data-Driven and Beyond. IEEE Transactions on pattern analysis and machine intelligence (2020), 1--1.

[27]

He Zhang and Vishal M Patel. 2018. Density-aware single image de-raining using a multi-stream dense network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 695--704.

Cited By

Lotfi FBeheshti AFarhood HPooshideh MJamzad MBeigy H(2023)Storytelling with Image Data: A Systematic Review and Comparative Analysis of Methods and ToolsAlgorithms10.3390/a1603013516:3(135)Online publication date: 2-Mar-2023
https://doi.org/10.3390/a16030135
Yu CChen SChang YSong YYan L(2023)Both Diverse and Realism Matter: Physical Attribute and Style Alignment for Rainy Image Generation2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01138(12353-12363)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.01138
Cui XWang CRen DChen YZhu P(2022)Semi-Supervised Image Deraining Using Knowledge DistillationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.319051632:12(8327-8341)Online publication date: Dec-2022
https://doi.org/10.1109/TCSVT.2022.3190516
Show More Cited By

Index Terms

Single Image Deraining via Scale-space Invariant Attention Neural Network
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

Multi-scale Attentive Residual Network for Single Image Deraining
Human Centered Computing
Abstract
Removing rain streaks from a single image is extremely challenging since the appearance of rain streaks in shapes, scales and densities is ever changing. Therefore, we propose a novel end-to-end two- stage multi-scale attentive residual network ...
Multi-scale large kernel convolution and hybrid attention network for remote sensing image dehazing
Abstract
Remote sensing (RS) image dehazing holds significant importance in enhancing the quality and information extraction capability of RS imagery. The enhancement in image dehazing quality has progressively advanced alongside the evolution of ...
Highlights
- Multi-scale large kernel convolution is introduced to enhance the effective receptive field.
- Hybrid attention is used to enhance the extraction of features.
- The feature fusion attention is used to enhance the interaction between ...
Single image deraining using multi-scales context information and attention network
Highlights
- We proposed a novel CNN-based framework for image deraining.
- We designed a ...
Abstract
The existing deraining methods based on convolutional neural networks (CNNs) have made great success, but some remaining rain streaks can degrade images drastically. In this work, we proposed an end-to-end multi-scale context ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

October 2020

4889 pages

ISBN:9781450379885

DOI:10.1145/3394171

General Chairs:
Chang Wen Chen
Chinese University of Hong Kong, Shenzhen, China
,
Rita Cucchiara
UNIMORE, Italy
,
Xian-Sheng Hua
Alibaba Group, China
,
Program Chairs:
Guo-Jun Qi
Futurewei Technologies, USA
,
Elisa Ricci
UNITN & Fondazione Bruno Kessler, Italy
,
Zhengyou Zhang
Tencent, China
,
Roger Zimmermann
National University of Singapore, Singapore

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation of China

Conference

MM '20

Sponsor:

SIGMM

MM '20: The 28th ACM International Conference on Multimedia

October 12 - 16, 2020

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
173
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lotfi FBeheshti AFarhood HPooshideh MJamzad MBeigy H(2023)Storytelling with Image Data: A Systematic Review and Comparative Analysis of Methods and ToolsAlgorithms10.3390/a1603013516:3(135)Online publication date: 2-Mar-2023
https://doi.org/10.3390/a16030135
Yu CChen SChang YSong YYan L(2023)Both Diverse and Realism Matter: Physical Attribute and Style Alignment for Rainy Image Generation2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01138(12353-12363)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.01138
Cui XWang CRen DChen YZhu P(2022)Semi-Supervised Image Deraining Using Knowledge DistillationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.319051632:12(8327-8341)Online publication date: Dec-2022
https://doi.org/10.1109/TCSVT.2022.3190516
崔艳(2021)A Single Image Derain Method Based on Improved Residual NetworkJournal of Image and Signal Processing10.12677/JISP.2021.10100110:01(1-8)Online publication date: 2021
https://doi.org/10.12677/JISP.2021.101001
Zhang ZZhu YFu XXiong ZZha ZWu FShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Multifocal Attention-Based Cross-Scale Network for Image De-rainingProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475444(3673-3681)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475444
Jiang KWang ZYi PChen CHan ZLu THuang BJiang J(2021)Decomposition Makes Better Rain Removal: An Improved Attention-Guided Deraining NetworkIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2020.304488731:10(3981-3995)Online publication date: Oct-2021
https://doi.org/10.1109/TCSVT.2020.3044887
Wang HWu YLi MZhao QMeng D(2021)Survey on rain removal from videos or a single imageScience China Information Sciences10.1007/s11432-020-3225-965:1Online publication date: 27-Dec-2021
https://doi.org/10.1007/s11432-020-3225-9
Cui XShang WRen DZhu PGao Y(2021)Semi-supervised Single Image Deraining with Discrete Wavelet TransformPRICAI 2021: Trends in Artificial Intelligence10.1007/978-3-030-89370-5_20(265-278)Online publication date: 1-Nov-2021
https://doi.org/10.1007/978-3-030-89370-5_20

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents