More Web Proxy on the site http://driver.im/

research-article

Contour-Aware Loss: Boundary-Aware Learning for Salient Object Segmentation

Authors:

Xiaohua XieAuthors Info & Claims

IEEE Transactions on Image Processing, Volume 30

Pages 431 - 443

https://doi.org/10.1109/TIP.2020.3037536

Published: 01 January 2021 Publication History

Abstract

We present a learning model that makes full use of boundary information for salient object segmentation. Specifically, we come up with a novel loss function, i.e., Contour Loss, which leverages object contours to guide models to perceive salient object boundaries. Such a boundary-aware network can learn boundary-wise distinctions between salient objects and background, hence effectively facilitating the salient object segmentation. Yet the Contour Loss emphasizes the boundaries to capture the contextual details in the local range. We further propose the hierarchical global attention module (HGAM), which forces the model hierarchically to attend to global contexts, thus captures the global visual saliency. Comprehensive experiments on six benchmark datasets show that our method achieves superior performance over state-of-the-art ones. Moreover, our model has a real-time speed of 26 fps on a TITAN X GPU.

References

[1]

L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254–1259, Nov. 1998.

Digital Library

[2]

M.-M. Cheng, N. J. Mitra, X. Huang, P. H. S. Torr, and S.-M. Hu, “Global contrast based salient region detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 569–582, Mar. 2015.

Digital Library

[3]

C. Yang, L. Zhang, H. Lu, X. Ruan, and M.-H. Yang, “Saliency detection via graph-based manifold ranking,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 3166–3173.

[4]

J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 3431–3440.

[5]

O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assisted Intervent. (MICCAI), 2015, pp. 234–241.

[6]

T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 2117–2125.

[7]

Z. Luo, A. Mishra, A. Achkar, J. Eichel, S. Li, and P.-M. Jodoin, “Non-local deep features for salient object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 6609–6617.

[8]

X. Li, F. Yang, H. Cheng, W. Liu, and D. Shen, “Contour knowledge transfer for salient object detection,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 370–385.

[9]

N. Liu, J. Han, and M.-H. Yang, “PiCANet: Learning pixel-wise contextual attention for saliency detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 3089–3098.

[10]

T. Wanget al., “Detect globally, refine locally: A novel approach to saliency detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 3127–3135.

[11]

X. Zhang, T. Wang, J. Qi, H. Lu, and G. Wang, “Progressive attention guided recurrent network for salient object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 714–722.

[12]

T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 2980–2988.

[13]

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. Learn. Represent., 2015, pp. 1–14.

[14]

A. Borji, M.-M. Cheng, Q. Hou, H. Jiang, and J. Li, “Salient object detection: A survey,” Comput. Vis. Media, vol. 5, pp. 117–150, Jun. 2019.

[15]

A. Borji, M.-M. Cheng, H. Jiang, and J. Li, “Salient object detection: A benchmark,” IEEE Trans. Image Process., vol. 24, no. 12, pp. 5706–5722, Dec. 2015.

Digital Library

[16]

W. Wang, Q. Lai, H. Fu, J. Shen, H. Ling, and R. Yang, “Salient object detection in the deep learning era: An in-depth survey,” 2019, arXiv:1904.09146. [Online]. Available: http://arxiv.org/abs/1904.09146

[17]

D. A. Klein and S. Frintrop, “Center-surround divergence of feature statistics for salient object detection,” in Proc. Int. Conf. Comput. Vis., Nov. 2011, pp. 2214–2219.

[18]

Z. Jiang and L. S. Davis, “Submodular salient region detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 2043–2050.

[19]

Q. Yan, L. Xu, J. Shi, and J. Jia, “Hierarchical saliency detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 1155–1162.

[20]

H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, and S. Li, “Salient object detection: A discriminative regional feature integration approach,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 2083–2090.

[21]

Q. Hou, M.-M. Cheng, X. Hu, A. Borji, Z. Tu, and P. Torr, “Deeply supervised salient object detection with short connections,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 3203–3212.

[22]

F. Yang, X. Li, H. Cheng, Y. Guo, L. Chen, and J. Li, “Multi-scale bidirectional FCN for object skeleton extraction,” in Proc. 32nd AAAI Conf. Artif. Intell. (AAAI), 2018, pp. 1–8.

[23]

J. Kuen, Z. Wang, and G. Wang, “Recurrent attentional networks for saliency detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 3668–3677.

[24]

N. Liu and J. Han, “DHSNet: Deep hierarchical saliency network for salient object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 678–686.

[25]

L. Wang, L. Wang, H. Lu, P. Zhang, and X. Ruan, “Saliency detection with recurrent fully convolutional networks,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2016, pp. 825–841.

[26]

Y. Tang, X. Wu, and W. Bu, “Deeply-supervised recurrent convolutional neural network for saliency detection,” in Proc. ACM Multimedia Conf., 2016, pp. 397–401.

[27]

P. Zhang, D. Wang, H. Lu, H. Wang, and X. Ruan, “Amulet: Aggregating multi-level convolutional features for salient object detection,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 202–211.

[28]

X. Li, F. Yang, H. Cheng, J. Chen, Y. Guo, and L. Chen, “Multi-scale cascade network for salient object detection,” in Proc. ACM Multimedia Conf. (MM), 2017, pp. 439–447.

[29]

L. Zhang, J. Dai, H. Lu, Y. He, and G. Wang, “A bi-directional message passing model for salient object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 1741–1750.

[30]

Z. Chen, C. Guo, J. Lai, and X. Xie, “Motion-appearance interactive encoding for object segmentation in unconstrained videos,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 6, pp. 1613–1624, Jun. 2020.

[31]

X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 7794–7803.

[32]

S. Chen, X. Tan, B. Wang, and X. Hu, “Reverse attention for salient object detection,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 236–252.

[33]

L. Wang, H. Lu, X. Ruan, and M.-H. Yang, “Deep networks for saliency detection via local estimation and global search,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 3183–3192.

[34]

R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “SLIC superpixels compared to state-of-the-art superpixel methods,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2274–2282, Nov. 2012.

Digital Library

[35]

P. Hu, B. Shuai, J. Liu, and G. Wang, “Deep level sets for salient object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 2300–2309.

[36]

X. Liet al., “DeepSaliency: Multi-task deep neural network model for salient object detection,” IEEE Trans. Image Process., vol. 25, no. 8, pp. 3919–3930, Aug. 2016.

[37]

G. Li and Y. Yu, “Visual saliency based on multiscale deep features,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 5455–5463.

[38]

G. Lee, Y.-W. Tai, and J. Kim, “Deep saliency with encoded low level distance map and high level features,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 660–668.

[39]

Y. Tang and X. Wu, “Saliency detection via combining region-level and pixel-level predictions with CNNs,” in Eur. Conf. Comput. Vis. (ECCV), 2016, pp. 809–825.

[40]

G. Li and Y. Yu, “Deep contrast learning for salient object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 478–487.

[41]

A. Manno-Kovacs, “Direction selective contour detection for salient objects,” IEEE Trans. Circuits Syst. Video Technol., vol. 29, no. 2, pp. 375–389, Feb. 2019.

Digital Library

[42]

H. Ding, X. Jiang, A. Q. Liu, N. M. Thalmann, and G. Wang, “Boundary-aware feature propagation for scene segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2019, pp. 6819–6829.

[43]

M. Feng, H. Lu, and E. Ding, “Attentive feedback network for boundary-aware salient object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 1623–1632.

[44]

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 248–255.

[45]

J.-F. Rivest, P. Soille, and S. Beucher, “Morphological gradients,” J. Electron. Imag., vol. 2, no. 4, pp. 326–337, 1993.

[46]

A. N. Evans and X. U. Liu, “A morphological gradient approach to color edge detection,” IEEE Trans. Image Process., vol. 15, no. 6, pp. 1454–1463, Jun. 2006.

Digital Library

[47]

H. Law and J. Deng, “Cornernet: Detecting objects as paired keypoints,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 734–750.

[48]

Y. Li, X. Hou, C. Koch, J. M. Rehg, and A. L. Yuille, “The secrets of salient object segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 280–287.

[49]

L. Wanget al., “Learning to detect salient objects with image-level supervision,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 136–145.

[50]

V. Movahedi and J. H. Elder, “Design and perceptual validation of performance measures for salient object segmentation,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 49–56.

[51]

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 770–778.

[52]

T. Wang, A. Borji, L. Zhang, P. Zhang, and H. Lu, “A stagewise refinement model for detecting salient objects in images,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 4019–4028.

[53]

P. Zhang, D. Wang, H. Lu, H. Wang, and B. Yin, “Learning uncertain convolutional features for accurate saliency detection,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 212–221.

[54]

S. Woo, J. Park, J.-Y. Lee, and I. So Kweon, “CBAM: Convolutional block attention module,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 3–19.

[55]

A. Paszkeet al., “Automatic differentiation in PyTorch,” in Proc. Neural Inf. Process. Syst. (NIPS), 2017, pp. 1–4.

[56]

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, “Frequency-tuned salient region detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1597–1604.

[57]

P. Krähenbühl and V. Koltun, “Efficient inference in fully connected CRFs with Gaussian edge potentials,” in Proc. Neural Inf. Process. Syst. (NIPS), 2011, pp. 109–117.

[58]

Z. Wu, L. Su, and Q. Huang, “Cascaded partial decoder for fast and accurate salient object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 3907–3916.

[59]

S. Xie and Z. Tu, “Holistically-nested edge detection,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2015, pp. 1395–1403.

[60]

F. Visin, K. Kastner, K. Cho, M. Matteucci, A. Courville, and Y. Bengio, “ReNet: A recurrent neural network based alternative to convolutional networks,” 2015, arXiv:1505.00393. [Online]. Available: http://arxiv.org/abs/1505.00393

Cited By

Lu XYuan YLiu XWang LZhou XYang Y(2024)Low-Light Salient Object Detection by Learning to Highlight the Foreground ObjectsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337710834:8(7712-7724)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3377108
Zhang LZhang Q(2024)Salient Object Detection With Edge-Guided Learning and Specific AggregationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328716734:1(534-548)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3287167
Zhu GLi JGuo Y(2024)Separate first, then segmentPattern Recognition10.1016/j.patcog.2024.110328150:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.patcog.2024.110328
Show More Cited By

Recommendations

Salient object detection: From pixels to segments

In this paper we propose a novel approach to the task of salient object detection. In contrast to previous salient object detectors that are based on a spotlight attention theory, we follow an object-based attention theory and incorporate the notion of ...
Evaluation on fusion of saliency and objectness for salient object segmentation
ICIMCS '15: Proceedings of the 7th International Conference on Internet Multimedia Computing and Service

Saliency detection measures the probability how a region attracts human visual attention, and objectness estimates the probability that a rectangle window may contain potential objects. Can a salient object segmentation method which utilizes both ...
Depth-aware salient object detection using anisotropic center-surround difference

Most previous works on salient object detection concentrate on 2D images. In this paper, we propose to explore the power of depth cue for predicting salient regions. Our basic assumption is that a salient object tends to stand out from its surroundings ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Image Processing

IEEE Transactions on Image Processing Volume 30, Issue

2021

5053 pages

ISSN:1057-7149

Issue’s Table of Contents

1057-7149 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 January 2021

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lu XYuan YLiu XWang LZhou XYang Y(2024)Low-Light Salient Object Detection by Learning to Highlight the Foreground ObjectsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337710834:8(7712-7724)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3377108
Zhang LZhang Q(2024)Salient Object Detection With Edge-Guided Learning and Specific AggregationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328716734:1(534-548)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3287167
Zhu GLi JGuo Y(2024)Separate first, then segmentPattern Recognition10.1016/j.patcog.2024.110328150:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.patcog.2024.110328
Zhou HLin YYang LLai JXie X(2024)Benchmarking deep models on salient object detectionPattern Recognition10.1016/j.patcog.2023.109951145:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.patcog.2023.109951
Zhu GWang LTang J(2024)Learning discriminative context for salient object detectionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107820131:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107820
Qu HDi LLiang JLiu H(2024)U-SMREngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107094126:PDOnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107094
Bi HTong YZhang PZhang JZhang C(2024)Dual cross-enhancement network for highly accurate dichotomous image segmentationComputer Vision and Image Understanding10.1016/j.cviu.2024.104122248:COnline publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1016/j.cviu.2024.104122
Wu QZhu PChai ZGuo G(2024)Joint learning of foreground, background and edge for salient object detectionComputer Vision and Image Understanding10.1016/j.cviu.2023.103915240:COnline publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1016/j.cviu.2023.103915
Liu CZhang WLiu YLi YJing R(2024)Generating crisp boundaries using multi-scale features and mixed loss functionApplied Intelligence10.1007/s10489-024-05784-554:21(10731-10747)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05784-5
Zhou CZhou YPan C(2024)FocusNet: Cascaded Lightweight Networks and Ascending Feature Enhancement for Efficient Salient Object DetectionComputer Vision – ACCV 202410.1007/978-981-96-0960-4_20(330-345)Online publication date: 8-Dec-2024
https://dl.acm.org/doi/10.1007/978-981-96-0960-4_20
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents