More Web Proxy on the site http://driver.im/

research-article

Ranking Saliency

Authors:

Ming-Hsuan YangAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 39, Issue 9

Pages 1892 - 1904

https://doi.org/10.1109/TPAMI.2016.2609426

Published: 01 September 2017 Publication History

Abstract

Most existing bottom-up algorithms measure the foreground saliency of a pixel or region based on its contrast within a local context or the entire image, whereas a few methods focus on segmenting out background regions and thereby salient objects. Instead of only considering the contrast between salient objects and their surrounding regions, we consider both foreground and background cues in this work. We rank the similarity of image elements with foreground or background cues via graph-based manifold ranking. The saliency of image elements is defined based on their relevances to the given seeds or queries. We represent an image as a multi-scale graph with fine superpixels and coarse regions as nodes. These nodes are ranked based on the similarity to background and foreground queries using affinity matrices. Saliency detection is carried out in a cascade scheme to extract background regions and foreground salient objects efficiently. Experimental results demonstrate the proposed method performs well against the state-of-the-art methods in terms of accuracy and speed. We also propose a new benchmark dataset containing 5,168 images for large-scale performance evaluation of saliency detection methods.

References

[1]

S. Goferman, L. Zelnik-Manor, and A. Tal, “ Context-aware saliency detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 2376–2383.

[2]

U. Rutishauser, D. Walther, C. Koch, and P. Perona, “Is bottom-up attention useful for object recognition?” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , 2004, vol. 2, pp. 37–44.

[3]

L. Itti, “Automatic foveation for video compression using a neurobiological model of visual attention,” IEEE Trans. Image Process., vol. 13, no. 10, pp. 1304–1318, Oct. 2004.

Digital Library

[4]

T. Chen, M. Cheng, P. Tan, A. Shamir, and S. Hu, “Sketch2photo: Internet image montage,” ACM Trans. Graphics., vol. 28, no. 5, 2009, Art. no. 124.

[5]

L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell, vol. 20, no. 11, pp. 1254 –1259, Nov. 1998.

Digital Library

[6]

J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” in Proc. Adv. Neural Inf. Process. Syst., 2006, pp. 545–552.

[7]

Y. Ma and H. Zhang, “ Contrast-based image attention analysis by using fuzzy growing,” in Proc. 11th ACM Int. Conf. Multimedia, 2003, pp. 374–381.

[8]

X. Hou and L. Zhang, “ Saliency detection: A spectral residual approach,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2007, pp. 1–8 .

[9]

N. Bruce and J. Tsotsos, “Saliency based on information maximization,” in Proc. Adv. Neural Inf. Process. Syst., 2006, pp. 155–162.

[10]

W. Wang, Y. Wang, Q. Huang, and W. Gao, “Measuring visual saliency by site entropy rate,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 2368–2375.

[11]

D. Klein and S. Frintrop, “Center-surround divergence of feature statistics for salient object detection,” in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 2214 –2219.

[12]

V. Gopalakrishnan, Y. Hu, and D. Rajan, “Random walks on graphs for salient object detection in images,” IEEE Trans. Image Process., vol. 19, no. 12, pp. 3232–3242, Dec. 2010.

Digital Library

[13]

R. Achanta, F. Estrada, P. Wils, and S. Susstrunk, “Salient region detection and segmentation,” in Proc. Int. Conf. Comput. Vis. Syst., 2008, pp. 66–75.

[14]

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, “Frequency-tuned salient region detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , 2009, pp. 1597–1604.

[15]

Y. Zhai and M. Shah, “Visual attention detection in video sequences using spatiotemporal cues,” in Proc. ACM Int. Conf. Multimedia, 2006, pp. 815–824 .

Digital Library

[16]

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, “Global contrast based salient region detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2011, pp. 409–416.

Digital Library

[17]

F. Perazzi, P. Krahenbuhl, Y. Pritch, and A. Hornung, “Saliency filters: Contrast based filtering for salient region detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 733–740.

[18]

L. Wang, J. Xue, N. Zheng, and G. Hua, “Automatic salient object extraction with contextual cue,” in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 105–112.

[19]

K.-Y. Chang, T.-L. Liu, H.-T. Chen, and S.-H. Lai, “Fusing generic objectness and visual saliency for salient object detection,” in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 914–921.

[20]

Y. Lu, W. Zhang, H. Lu, and X. Xue, “Salient object detection using concavity context,” in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 233–240.

[21]

J. Feng, Y. Wei, L. Tao, C. Zhang, and J. Sun, “Salient object detection by composition,” in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 1028 –1035.

[22]

J. Yang and M. Yang, “Top-down visual saliency via joint CRF and dictionary learning,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 2296–2303.

[23]

T. Liu, et al., “Learning to detect a salient object,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353–367, Feb. 2011.

Digital Library

[24]

T. Judd, K. Ehinger, F. Durand, and A. Torralba, “Learning to predict where humans look,” in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 2106–2113.

[25]

A. Borji and L. Itti, “Exploiting local and global patch rarities for saliency detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 478 –485.

[26]

R. Margolin, A. Tal, and L. Zelnik-Manor, “What makes a patch distinct?” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 1139–1146.

[27]

M. Cheng, J. Warrell, W. Lin, S. Zheng, V. Vineet, and N. Crook, “ Efficient salient region detection with soft image abstraction,” in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1529–1536.

[28]

C. Yang, L. H. Zhang, and H. C. Lu, “ Graph-regularized saliency detection with convex-hull-based center prior,” IEEE Signal Process. Lett., vol. 20, no. 7, pp. 637–640, Jul. 2013.

[29]

P. Jiang, H. Ling, J. Yu, and J. Peng, “Salient region detection by UFO: Uniqueness, focusness and objectness,” in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1976–1983.

[30]

Y. Jia and M. Han, “ Category-independent object-level saliency detection,” in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1761 –1768.

[31]

Y. Wei, F. Wen, W. Zhu, and J. Sun, “Geodesic saliency using background priors,” in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 29– 42.

[32]

B. Jiang, L. Zhang, H. Lu, C. Yang, and M.-H. Yang, “Saliency detection via absorbing Markov chain,” in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1665 –1672.

[33]

W. Zhu, S. Liang, Y. Wei, and J. Sun, “Saliency optimization from robust background detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 1–8.

[34]

V. Lempitsky, P. Kohli, C. Rother, and T. Sharp, “Image segmentation with a bounding box prior,” in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 277–284.

[35]

L. Grady, M. Jolly, and A. Seitz, “Segmenation from a box,” in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 367–374.

[36]

T. Kim, K. Lee, and S. Lee, “Learning full pairwise affinities for spectral segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 2101–2108.

[37]

A. Borji and L. Itti, “State-of-the-art in visual attention modeling,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 185–207, Jan. 2013.

Digital Library

[38]

A. Triesman and G. Gelade, “A feature-integration theory of attention,” Cogn. Psychol., vol. 12, no. 1, pp. 97–136, 1980.

[39]

E. Rahtu, J. Kannala, M. Salo, and J. Heikkilä, “Segmenting salient objects from images and videos,” in Proc. Eur. Conf. Comput. Vis., 2010, pp. 366–379.

[40]

Y. Xie, H. Lu, and M.-H. Yang, “Bayesian saliency via low and mid level cues,” IEEE Trans. Image Process., vol. 22, no. 5, pp. 1689–1698, May 2013.

Digital Library

[41]

C. Guo, Q. Ma, and L. Zhang, “Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1–8 .

[42]

C. Lang, G. Liu, J. Yu, and S. Yan, “Saliency detection by multitask sparsity pursuit,” IEEE Trans. Image Process., vol. 21, no. 3, pp. 1327–1338, Mar. 2012.

Digital Library

[43]

X. Shen and Y. Wu, “A unified approach to salient object detection via low rank matrix recovery,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 853–860.

[44]

X. Li, H. Lu, L. Zhang, X. Ruan, and M.-H. Yang, “Saliency detection via dense and sparse reconstruction,” in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 2976 –2983.

[45]

B. Alexe, T. Deselares, and V. Ferrai, “What is an object?” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 73–80.

[46]

L. Mai, Y. Niu, and F. Liu, “Saliency aggregation: A data-driven approach,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 1131–1138.

[47]

Z. Jiang and L. Davis, “Submodular salient region detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 2043–2050.

[48]

X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-supervised learning using gaussian fields and harmonic functions,” in Proc. Int. Conf. Mach. Learn. , 2003, pp. 912–919.

[49]

B. Tatler, “The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions,” J. Vis. , vol. 7, no. 14, pp. 1–17, 2007.

[50]

D. Zhou, J. Weston, A. Gretton, O. Bousquet, and B. Scholkopf, “Ranking on data manifolds,” in Proc. Adv. Neural Inf. Process. Syst., 2004, pp. 1–8.

[51]

F.-R. Chung, Spectral Graph Theory. Providence, RI, USA : Amer. Math. Soc., 1997.

[52]

S. Brin and L. Page, “The anatomy of a large-scale hypertextual web search engine,” Comp. Netw. ISDN Syst., vol. 30, no. 1, pp. 107– 117, 1998.

Digital Library

[53]

A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” in Proc. Adv. Neural Inf. Process. Syst., 2002, pp. 849–856.

[54]

D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Scholkopf, “Learning with local and global consistency,” in Proc. Adv. Neural Inf. Process. Syst., 2003, pp. 321–328.

[55]

B. Scholkopf, J. Platt, J. Shawe-Taylor, A. Smola, and R. Williamson, “Estimating the support of a high-dimensional distribution,” Neural Comput., vol. 13, pp. 1443– 1471, 2001.

Digital Library

[56]

R. Achanta, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk, “SLIC superpixels,” EPFL, Lausanne, Switzerland, EPFL-REPORT-149300, 2010.

Digital Library

[57]

A. Borji, D. Sihite, and L. Itti, “Salient object detection: A benchmark,” in Proc. Eur. Conf. Comput. Vis., 2012, pp. 414–429.

[58]

M. Cerf, J. Harel, W. Einhauser, and C. Koch, “Predicting human gaze using low-level saliency combined with face detection,” in Proc. Adv. Neural Inf. Process. Syst., 2007, pp. 241–248.

[59]

H. Jiang, J. Wang, Z. Yuan, T. Liu, N. Zheng, and S. Li, “ Automatic salient object segmentation based on contex and shape prior,” in Proc. Brit. Mach. Vis. Conf., 2011, pp. 1–12.

[60]

C. Yang, L. Zhang, H. Lu, X. Ruan, and M.-H. Yang, “Saliency detection via graph-based manifold ranking,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 3166–3173.

[61]

M. Cheng, N. Mitra, X. Huang, P. Torr, and S.-M. Hu, “Global Contrast based Salient Region Detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 569–582, Mar. 2015.

Digital Library

[62]

T. Judd, F. Durand, and A. Torralba, “A benchmark of computational models of saliency to predict human fixations,” MIT, Cambridge, MA, USA, MIT-CSAIL-TR-2012-001, 2012.

[63]

M. Kummerer, L. Theis, and M. Bethge, “Deep gaze I: Boosting saliency prediction with feature maps trained on imagenet,” in Proc. Int. Conf. Learn. Representations, 2015, pp. 1–11.

[64]

E. Vig, M. Dorr, and D. Co, “Large-scale optimization of hierarchical features for saliency prediction in natural images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 2798–2805.

[65]

J. Pan, E. Sayrol, X. G. i Nieto, K. McGuinness, and N. E. O’Connor, “Shallow and deep convolutional networks for saliency prediction,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 598–606.

[66]

N. Liu, J. Han, D. Zhang, S. Wen, and T. Liu, “Predicting eye fixations using convolutional neural networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 362 –370.

[67]

R. J. Peters, A. Iyer, L. Itti, and C. Koch, “ Components of bottom-up gaze allocation in natural images,” Vis. Res., vol. 45, no. 18, pp. 2397–2416, 2005.

[68]

H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, and S. Li, “ Salient object detection: A discriminative regional feature integration approach,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 2083–2090.

[69]

X. Hou, J. Harel, and C. Koch, “Image signature: Highlighting sparse salient regions,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 1, pp. 194–201, Jan. 2012.

Digital Library

[70]

Z. Bylinskii, et al., “MIT saliency benchmark,” (2012). [Online]. Avaialble: http://saliency.mit.edu/

Cited By

Khan HUsman MRida IKoo J(2024)Attention enhanced machine instinctive vision with human-inspired saliency detectionImage and Vision Computing10.1016/j.imavis.2024.105308152:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.imavis.2024.105308
Gao WFan SLi GLin W(2023)A Thorough Benchmark and a New Model for Light Field Saliency DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.323541545:7(8003-8019)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3235415
Pang YWu CWu HYu X(2023)Unsupervised Multi-Subclass Saliency Classification for Salient Object DetectionIEEE Transactions on Multimedia10.1109/TMM.2022.314407025(2189-2202)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3144070
Show More Cited By

Index Terms

Ranking Saliency
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
2. Information systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Saliency Detection via Graph-Based Manifold Ranking
CVPR '13: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition

Most existing bottom-up methods measure the foreground saliency of a pixel or region based on its contrast within a local context or the entire image, whereas a few methods focus on segmenting out background regions and thereby salient objects. Instead ...
Co-saliency detection via inter and intra saliency propagation

The goal of salient object detection from an image is to extract the regions which capture the attention of the human visual system more than other regions of the image. In this paper a novel method is presented for detecting salient objects from a set ...
Mesh saliency

Research over the last decade has built a solid mathematical foundation for representation and analysis of 3D meshes in graphics and geometric modeling. Much of this work however does not explicitly incorporate models of low-level human visual ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 39, Issue 9

Sept. 2017

208 pages

ISSN:0162-8828

Issue’s Table of Contents

0162-8828 © 2016 IEEE.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 September 2017

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Khan HUsman MRida IKoo J(2024)Attention enhanced machine instinctive vision with human-inspired saliency detectionImage and Vision Computing10.1016/j.imavis.2024.105308152:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.imavis.2024.105308
Gao WFan SLi GLin W(2023)A Thorough Benchmark and a New Model for Light Field Saliency DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.323541545:7(8003-8019)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3235415
Pang YWu CWu HYu X(2023)Unsupervised Multi-Subclass Saliency Classification for Salient Object DetectionIEEE Transactions on Multimedia10.1109/TMM.2022.314407025(2189-2202)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3144070
Feng GMeng JZhang LLu H(2022)Encoder deep interleaved network with multi-scale aggregation for RGB-D salient object detectionPattern Recognition10.1016/j.patcog.2022.108666128:COnline publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1016/j.patcog.2022.108666
Singh SSrivastava R(2022)A robust RGBD saliency method with improved probabilistic contrast and the global reference surfaceThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-020-02050-w38:3(797-809)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1007/s00371-020-02050-w
Xia CGao XFang XLi KSu SZhang H(2021)RLP-AGMCImage Communication10.1016/j.image.2021.11637298:COnline publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1016/j.image.2021.116372
Zeeshan MMajid M(2021)HEVC compatible perceptual multiple description video coding for reliable video transmission over packet networksTelecommunications Systems10.1007/s11235-020-00702-976:1(63-83)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1007/s11235-020-00702-9
Shi JUlrich SRuel S(2021)Real-time saliency detection for greyscale and colour imagesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-020-01865-x37:6(1277-1296)Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1007/s00371-020-01865-x
Deng CYang XNie FTao D(2020)Saliency Detection via a Multiple Self-Weighted Graph-Based Manifold RankingIEEE Transactions on Multimedia10.1109/TMM.2019.293483322:4(885-896)Online publication date: 24-Mar-2020
https://dl.acm.org/doi/10.1109/TMM.2019.2934833
Chen STan XWang BLu HHu XFu Y(2020)Reverse Attention-Based Residual Network for Salient Object DetectionIEEE Transactions on Image Processing10.1109/TIP.2020.296598929(3763-3776)Online publication date: 29-Jan-2020
https://dl.acm.org/doi/10.1109/TIP.2020.2965989
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents