More Web Proxy on the site http://driver.im/

research-article

Learning discriminative context for salient object detection

Authors:

Jinping TangAuthors Info & Claims

Volume 131, Issue C

https://doi.org/10.1016/j.engappai.2023.107820

Published: 01 May 2024 Publication History

Abstract

Context understanding is important for salient object detection (SOD) in complex scenes. To alleviate visual confusion, we propose a context aware network that full use of the same type contextual information for SOD. Specifically, we introduce the pixel relationships into decoder, which can extract the explicit contextual information to alleviate visual confusion by learning the similarities and differences between pixels. Furthermore, we explore the optimal embedding position of the pixel relationship in network to maximize its benefits, thereby reducing the introduction of noise. Compared with 20 counterparts, experimental results on five datasets show that our approach has better generalization, which averagely improves 1.3% and 2.8% over the advanced CNN-based model and Transformer-based model on three evaluation metrics. The improvements demonstrate effectiveness of the proposed context learning strategies, which are helpful for dealing with various complex scenes. Moreover, our model is more efficient with an inference speed of 56.7 FPS on a single NVIDIA 2080TI GPU. Codes are available at https://github.com/lesonly/CANet.

References

[1]

Ali Abdelmgeid A., El-Hafeez Tarek Abd, Mohany Yosra Khalaf, An accurate system for face detection and recognition, J. Adv. Math. Comput. Sci. 33 (3) (2019) 1–19.

[2]

Ali Abdelmgeid A., El-Hafeez Tarek Abd, Mohany Yosra Khalaf, A robust and efficient system to detect human faces based on facial features, Asian J. Res. Comput. Sci. 2 (4) (2019) 1–12.

[3]

Amirkhani Abdollah, Khosravian Amir, Masih-Tehrani Masoud, Kashiani Hossein, Robust semantic segmentation with multi-teacher knowledge distillation, IEEE Access 9 (2021) 119049–119066.

[4]

Banitalebi-Dehkordi Amin, Amirkhani Abdollah, Mohammadinasab Alireza, EBCDet: Energy-based curriculum for robust domain adaptive object detection, IEEE Access 11 (2023) 77810–77825.

[5]

de Boer Pieter-Tjerk, Kroese Dirk P., Mannor Shie, Rubinstein Reuven Y., A tutorial on the cross-entropy method, Ann. Oper. Res. 134 (1) (2005) 19–67.

[6]

Borji Ali, Cheng Ming-Ming, Hou Qibin, Jiang Huaizu, Li Jia, Salient object detection: A survey, Comput. Vis. Media 5 (2) (2019) 117–150.

[7]

Borji Ali, Cheng Ming-Ming, Jiang Huaizu, Li Jia, Salient object detection: A benchmark, IEEE Trans. Image Process. 24 (12) (2015) 5706–5722.

Digital Library

[8]

Chen Liang-Chieh, Papandreou George, Kokkinos Iasonas, Murphy Kevin, Yuille Alan L., DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs, IEEE Trans. Pattern Anal. Mach. Intell. 40 (4) (2018) 834–848.

[9]

Chen Zuyao, Xu Qianqian, Cong Runmin, Huang Qingming, Global context-aware progressive aggregation network for salient object detection, in: AAAI, AAAI Press, 2020, pp. 10599–10606.

[10]

Chen Zixuan, Zhou Huajun, Lai Jianhuang, Yang Lingxiao, Xie Xiaohua, Contour-aware loss: Boundary-aware learning for salient object segmentation, IEEE Trans. Image Process. 30 (2021) 431–443.

Digital Library

[11]

Cong Runmin, Lei Jianjun, Fu Huazhu, Cheng Ming-Ming, Lin Weisi, Huang Qingming, Review of visual saliency detection with comprehensive information, IEEE Trans. Circuits Syst. Video Technol. 29 (10) (2019) 2941–2959.

[12]

Dai Xiyang, Chen Yinpeng, Xiao Bin, Chen Dongdong, Liu Mengchen, Yuan Lu, Zhang Lei, Dynamic head: Unifying object detection heads with attentions, in: CVPR, Computer Vision Foundation / IEEE, 2021, pp. 7373–7382.

[13]

Dai, Yimian, Gieseke, Fabian, Oehmcke, Stefan, Wu, Yiquan, Barnard, Kobus, 2021b. Attentional feature fusion. In: WACV. pp. 3560–3569.

[14]

Deng Jia, Dong Wei, Socher Richard, Li Li-Jia, Li Kai, Li Fei-Fei, ImageNet: A large-scale hierarchical image database, in: CVPR, IEEE Computer Society, 2009, pp. 248–255.

[15]

Dong Xingping, Shen Jianbing, Porikli Fatih, Luo Jiebo, Shao Ling, Adaptive siamese tracking with a compact latent network, IEEE Trans. Pattern Anal. Mach. Intell. (2022).

[16]

Dong Xingping, Shen Jianbing, Wang Wenguan, Shao Ling, Ling Haibin, Porikli Fatih, Dynamical hyperparameter optimization via deep reinforcement learning in tracking, IEEE Trans. Pattern Anal. Mach. Intell. 43 (5) (2021) 1515–1529.

[17]

Eman Mohammed, Mahmoud Tarek M, Ibrahim Mostafa M, Abd El-Hafeez Tarek, Innovative hybrid approach for masked face recognition using pretrained mask detection and segmentation, robust PCA, and KNN classifier, Sensors 23 (15) (2023) 6727.

[18]

Everingham Mark, Gool Luc Van, Williams Christopher K.I., Winn John M., Zisserman Andrew, The pascal visual object classes (VOC) challenge, Int. J. Comput. Vis. 88 (2) (2010) 303–338.

Digital Library

[19]

Fan Deng-Ping, Cheng Ming-Ming, Liu Yun, Li Tao, Borji Ali, Structure-measure: A new way to evaluate foreground maps, in: ICCV, IEEE Computer Society, 2017, pp. 4558–4567.

[20]

Han Wencheng, Dong Xingping, Khan Fahad Shahbaz, Shao Ling, Shen Jianbing, Learning to fuse asymmetric feature maps in siamese trackers, in: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, Virtual, June 19-25, 2021, Computer Vision Foundation / IEEE, 2021, pp. 16570–16580.

[21]

He Kaiming, Zhang Xiangyu, Ren Shaoqing, Sun Jian, Deep residual learning for image recognition, in: CVPR, IEEE Computer Society, 2016, pp. 770–778.

[22]

Hoyer Lukas, Munoz Mauricio, Katiyar Prateek, Khoreva Anna, Fischer Volker, Grid saliency for context explanations of semantic segmentation, in: NeurIPS, 2019, pp. 6459–6470.

[23]

Hu Jie, Shen Li, Sun Gang, Squeeze-and-excitation networks, in: CVPR, Computer Vision Foundation / IEEE Computer Society, 2018, pp. 7132–7141.

[24]

Huang Zilong, Wang Xinggang, Huang Lichao, Huang Chang, Wei Yunchao, Liu Wenyu, CCNet: Criss-cross attention for semantic segmentation, in: ICCV, IEEE, 2019, pp. 603–612.

[25]

Khosravian Amir, Amirkhani Abdollah, Kashiani Hossein, Masih-Tehrani Masoud, Generalizing state-of-the-art object detectors for autonomous vehicles in unseen environments, Expert Syst. Appl. 183 (2021).

[26]

Krähenbühl, Philipp, Koltun, Vladlen, 2011. Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials. In: NIPS. pp. 109–117.

[27]

Lee Min Seok, Shin WooSeok, Han Sung Won, TRACER: extreme attention guided salient object tracing network (student abstract), in: AAAI, AAAI Press, 2022, pp. 12993–12994.

[28]

Li Yin, Hou Xiaodi, Koch Christof, Rehg James M., Yuille Alan L., The secrets of salient object segmentation, in: CVPR, IEEE Computer Society, 2014, pp. 280–287.

[29]

Li Jia, Su Jinming, Xia Changqun, Ma Mingcan, Tian Yonghong, Salient object detection with purificatory mechanism and structural similarity loss, IEEE Trans. Image Process. 30 (2021) 6855–6868.

[30]

Li Guanbin, Yu Yizhou, Visual saliency detection based on multiscale deep CNN features, IEEE Trans. Image Process. 25 (11) (2016) 5012–5024.

Digital Library

[31]

Liu Nian, Han Junwei, Yang Ming-Hsuan, PiCANet: Pixel-wise contextual attention learning for accurate saliency detection, IEEE Trans. Image Process. 29 (2020) 6438–6451.

Digital Library

[32]

Liu Jiangjiang, Hou Qibin, Cheng Ming-Ming, Feng Jiashi, Jiang Jianmin, A simple pooling-based design for real-time salient object detection, in: CVPR, Computer Vision Foundation / IEEE, 2019, pp. 3917–3926.

[33]

Liu Nian, Zhang Ni, Wan Kaiyuan, Shao Ling, Han Junwei, Visual saliency transformer, in: ICCV, IEEE, 2021, pp. 4702–4712.

[34]

Ma Mingcan, Xia Changqun, Li Jia, Pyramidal feature shrinking for salient object detection, in: AAAI, AAAI Press, 2021, pp. 2311–2318.

[35]

Ma Mingcan, Xia Changqun, Xie Chenxi, Chen Xiaowu, Li Jia, Boosting broader receptive fields for salient object detection, IEEE Trans. Image Process. 32 (2023) 1026–1038.

[36]

Mohammadi Sina, Noori Mehrdad, Bahri Ali, Majelan Sina Ghofrani, Havaei Mohammad, CAGNet: Content-aware guidance for salient object detection, Pattern Recognit. 103 (2020).

[37]

Pang Youwei, Zhao Xiaoqi, Zhang Lihe, Lu Huchuan, Multi-scale interactive network for salient object detection, in: CVPR, IEEE, 2020, pp. 9410–9419.

[38]

Qin Xuebin, Zhang Zichen Vincent, Huang Chenyang, Gao Chao, Dehghan Masood, Jägersand Martin, Basnet: Boundary-aware salient object detection, in: CVPR, Computer Vision Foundation / IEEE, 2019, pp. 7479–7489.

[39]

Saabia, Abd AL-BastRashed, El-Hafeez, TarekAbd, Zaki, Alaa M., 2019. Face recognition based on Grey Wolf Optimization for feature selection. In: Proceedings of the International Conference on Advanced Intelligent Systems and Informatics. pp. 273–283.

[40]

Shen Jianbing, Liu Yuanpei, Dong Xingping, Lu Xiankai, Khan Fahad Shahbaz, Hoi Steven C.H., Distilled siamese networks for visual tracking, IEEE Trans. Pattern Anal. Mach. Intell. 44 (12) (2022) 8896–8909.

[41]

Singh Krishna Kumar, Yu Hao, Sarmasi Aron, Pradeep Gautam, Lee Yong Jae, Hide-and-seek: A data augmentation technique for weakly-supervised localization and beyond, 2018, CoRR, abs/1811.02545.

[42]

Su Jinming, Li Jia, Zhang Yu, Xia Changqun, Tian Yonghong, Selectivity or invariance: Boundary-aware salient object detection, in: ICCV, IEEE, 2019, pp. 3798–3807.

[43]

Wang Tiantian, Borji Ali, Zhang Lihe, Zhang Pingping, Lu Huchuan, A stagewise refinement model for detecting salient objects in images, in: ICCV, IEEE Computer Society, 2017, pp. 4039–4048.

[44]

Wang Wenguan, Lai Qiuxia, Fu Huazhu, Shen Jianbing, Ling Haibin, Yang Ruigang, Salient object detection in the deep learning era: An in-depth survey, IEEE Trans. Pattern Anal. Mach. Intell. 44 (6) (2022) 3239–3259.

[45]

Wang Lijun, Lu Huchuan, Wang Yifan, Feng Mengyang, Wang Dong, Yin Baocai, Ruan Xiang, Learning to detect salient objects with image-level supervision, in: CVPR, IEEE Computer Society, 2017, pp. 3796–3805.

[46]

Wang Zhengwei, She Qi, Smolic Aljosa, ACTION-Net: Multipath excitation for action recognition, in: CVPR, Computer Vision Foundation / IEEE, 2021, pp. 13214–13223.

[47]

Wang Wenguan, Sun Guolei, Van Gool Luc, Looking beyond single images for weakly supervised semantic segmentation learning, IEEE Trans. Pattern Anal. Mach. Intell. (2022).

Digital Library

[48]

Wang Yi, Wang Ruili, Fan Xin, Wang Tianzhu, He Xiangjian, Pixels, regions, and objects: Multiple enhancement for salient object detection, in: CVPR, IEEE, 2023, pp. 10031–10040.

[49]

Wang Wenguan, Zhao Shuyang, Shen Jianbing, Hoi Steven C.H., Borji Ali, Salient object detection with pyramid attention and salient edges, in: CVPR, Computer Vision Foundation / IEEE, 2019, pp. 1448–1457.

[50]

Wei Jun, Wang Shuhui, Huang Qingming, F3: Fusion, feedback and focus for salient object detection, in: AAAI, AAAI Press, 2020, pp. 12321–12328.

[51]

Wei Jun, Wang Shuhui, Wu Zhe, Su Chi, Huang Qingming, Tian Qi, Label decoupling framework for salient object detection, in: CVPR, IEEE, 2020, pp. 13022–13031.

[52]

Woo Sanghyun, Park Jongchan, Lee Joon-Young, Kweon In So, CBAM: convolutional block attention module, in: ECCV, Vol. 11211, Springer, 2018, pp. 3–19.

[53]

Wu Yu-Huan, Liu Yun, Zhang Le, Cheng Ming-Ming, Ren Bo, EDN: salient object detection via extremely-downsampled network, IEEE Trans. Image Process. 31 (2022) 3125–3136.

[54]

Wu Zhe, Su Li, Huang Qingming, Cascaded partial decoder for fast and accurate salient object detection, in: CVPR, Computer Vision Foundation / IEEE, 2019, pp. 3907–3916.

[55]

Wu Zhe, Su Li, Huang Qingming, Stacked cross refinement network for edge-aware salient object detection, in: ICCV, IEEE, 2019, pp. 7263–7272.

[56]

Wu Zhe, Su Li, Huang Qingming, Decomposition and completion network for salient object detection, IEEE Trans. Image Process. 30 (2021) 6226–6239.

[57]

Xu Binwei, Liang Haoran, Liang Ronghua, Chen Peng, Locate globally, segment locally: A progressive architecture with knowledge review network for salient object detection, in: AAAI, AAAI Press, 2021, pp. 3004–3012.

[58]

Yan Qiong, Xu Li, Shi Jianping, Jia Jiaya, Hierarchical saliency detection, in: CVPR, IEEE Computer Society, 2013, pp. 1155–1162.

[59]

Yang Chuan, Zhang Lihe, Lu Huchuan, Ruan Xiang, Yang Ming-Hsuan, Saliency detection via graph-based manifold ranking, in: CVPR, IEEE Computer Society, 2013, pp. 3166–3173.

[60]

Yu Jiahui, Jiang Yuning, Wang Zhangyang, Cao Zhimin, Huang Thomas S., UnitBox: An advanced object detection network, in: ACMMM, ACM, 2016, pp. 516–520.

[61]

Yu Changqian, Wang Jingbo, Gao Changxin, Yu Gang, Shen Chunhua, Sang Nong, Context prior for scene segmentation, in: CVPR, Computer Vision Foundation / IEEE, 2020, pp. 12413–12422.

[62]

Yun Yi Ke, Lin Weisi, Towards a complete and detail-preserved salient object detection, IEEE Trans. Multimed. (2023).

[63]

Yun Yi Ke, Tsubono Takahiro, Recursive contour-saliency blending network for accurate salient object detection, in: WACV, IEEE, 2022, pp. 1360–1370.

[64]

Zeng Yu, Zhuge Yun-Zhi, Lu Huchuan, Zhang Lihe, Joint learning of saliency detection and weakly supervised semantic segmentation, in: ICCV, IEEE, 2019, pp. 7222–7232.

[65]

Zhang Lu, Dai Ju, Lu Huchuan, He You, Wang Gang, A bi-directional message passing model for salient object detection, in: CVPR, IEEE Computer Society, 2018, pp. 1741–1750.

[66]

Zhang Miao, Liu Tingwei, Piao Yongri, Yao Shunyu, Lu Huchuan, Auto-MSFNet: Search multi-scale fusion network for salient object detection, in: ACMMM, ACM, 2021, pp. 667–676.

[67]

Zhang Xiaoning, Wang Tiantian, Qi Jinqing, Lu Huchuan, Wang Gang, Progressive attention guided recurrent network for salient object detection, in: CVPR, Computer Vision Foundation / IEEE Computer Society, 2018, pp. 714–722.

[68]

Zhang Liqian, Zhang Qing, Zhao Rui, Progressive dual-attention residual network for salient object detection, IEEE Trans. Circuits Syst. Video Technol. 32 (9) (2022) 5902–5915.

[69]

Zhao Jiaxing, Liu Jiangjiang, Fan Deng-Ping, Cao Yang, Yang Jufeng, Cheng Ming-Ming, Egnet: Edge guidance network for salient object detection, in: ICCV, IEEE, 2019, pp. 8778–8787.

[70]

Zhao Xiaoqi, Pang Youwei, Zhang Lihe, Lu Huchuan, Zhang Lei, Suppress and balance: A simple gated network for salient object detection, in: ECCV (2), in: Lecture Notes in Computer Science, vol. 12347, Springer, 2020, pp. 35–51.

[71]

Zhao Hengshuang, Shi Jianping, Qi Xiaojuan, Wang Xiaogang, Jia Jiaya, Pyramid scene parsing network, in: CVPR, IEEE Computer Society, 2017, pp. 6230–6239.

[72]

Zhao Ting, Wu Xiangqian, Pyramid feature attention network for saliency detection, in: CVPR, Computer Vision Foundation / IEEE, 2019, pp. 3085–3094.

[73]

Zhao Zhirui, Xia Changqun, Xie Chenxi, Li Jia, Complementary trilateral decoder for fast and accurate salient object detection, in: ACMMM, ACM, 2021, pp. 4967–4975.

[74]

Zhou Huajun, Xie Xiaohua, Lai Jian-Huang, Chen Zixuan, Yang Lingxiao, Interactive two-stream decoder for accurate and fast saliency detection, in: CVPR, IEEE, 2020, pp. 9138–9147.

[75]

Zhu Lei, Chen Jiaxing, Hu Xiaowei, Fu Chi-Wing, Xu Xuemiao, Qin Jing, Heng Pheng-Ann, Aggregating attentional dilated features for salient object detection, IEEE Trans. Circuits Syst. Video Technol. 30 (10) (2020) 3358–3371.

[76]

Zhu Ge, Li Jinbao, Guo Yahong, PriorNet: Two deep prior cues for salient object detection, IEEE Trans. Multimed. (2023) 1–13.

[77]

Zhu Ge, Li Jinbao, Guo Yahong, Sharp eyes: A salient object detector working the same way as human visual characteristics, 2023, arXiv preprint arXiv:2301.07431.

[78]

Zhu Ge, Li Jinbao, Guo Yahong, Supplement and suppression: Both boundary and nonboundary are helpful for salient object detection, IEEE Trans. Neural Netw. Learn. Syst. 34 (9) (2023) 6615–6627.

[79]

Zhuge Mingchen, Fan Deng-Ping, Liu Nian, Zhang Dingwen, Xu Dong, Shao Ling, Salient object detection via integrity learning, IEEE Trans. Pattern Anal. Mach. Intell. 45 (3) (2023) 3738–3752.

Recommendations

Salient object detection: From pixels to segments

In this paper we propose a novel approach to the task of salient object detection. In contrast to previous salient object detectors that are based on a spotlight attention theory, we follow an object-based attention theory and incorporate the notion of ...
Audio-Visual Salient Object Detection
Intelligent Computing Theories and Application
Abstract
This paper studies audio-visual salient object detection. The task of salient object detection is to detect and mark the objects that are most concerned by people in the visual scene. Traditionally, visual salient object detection uses only images ...
Salient object detection via multiple saliency weights

Salient object detection aims to emulate the extraordinary capability of human visual system, which has the ability to find the most visually attractive objects in a complex visual scene. The human visual attention is often complicated and affected by ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Engineering Applications of Artificial Intelligence

Engineering Applications of Artificial Intelligence Volume 131, Issue C

May 2024

1508 pages

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 May 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents