More Web Proxy on the site http://driver.im/

research-article

Objective Object Segmentation Visual Quality Evaluation: Quality Measure and Pooling Method

Authors:

Tong QiaoAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 18, Issue 3

Article No.: 73, Pages 1 - 19

https://doi.org/10.1145/3491229

Published: 04 March 2022 Publication History

Abstract

Objective object segmentation visual quality evaluation is an emergent member of the visual quality assessment family. It aims to develop an objective measure instead of a subjective survey to evaluate the object segmentation quality in agreement with human visual perception. It is an important benchmark for assessing and comparing the performances of object segmentation methods in terms of visual quality. Despite its essential role, sufficient study compared with other visual quality evaluation studies is still lacking. In this article, we propose a novel full-reference objective measure that includes a two-level single object segmentation visual quality measure and a pooling method for multiple object segmentation overall visual quality. The single object segmentation visual quality measure combines a pixel-level sub-measure and a region-level sub-measure for evaluating the similarity of area, shape, and object completeness between the segmentation result and the ground truth in terms of human visual perception. For the proposed multiple object segmentation overall visual quality pooling method, the rank of each object’s segmentation quality as a novel factor is integrated into the weighted harmonic mean to evaluate the overall quality. To evaluate the performance of our proposed measure, we tested it on an object segmentation subjective visual quality assessment database. The experimental results demonstrate that our proposed two-level measure and pooling method with good robustness perform better in matching subjective assessments compared with other state-of-the-art objective measures.

References

[1]

Shai Avidan and Ariel Shamir. 2007. Seam carving for content-aware image resizing. In ACM SIGGRAPH 2007 Papers. 10–es.

[2]

Oleh Berezsky, Grygoriy Melnyk, Yuriy Batko, and Oleh Pitsun. 2016. Regions matching algorithms analysis to quantify the image segmentation results. In Proceedings of the 2016 11th International Scientific and Technical Conference on Computer Sciences and Information Technologies (CSIT’16). IEEE, Los Alamitos, CA, 33–36.

[3]

Zhenzhong Chen and Han Zhu. 2019. Visual quality evaluation for semantic segmentation: Subjective assessment database and objective assessment measure. IEEE Transactions on Image Processing 28, 12 (2019), 5785–5796.

[4]

Ming-Ming Cheng, Niloy J. Mitra, Xiaolei Huang, Philip H. S. Torr, and Shi-Min Hu. 2015. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 3 (2015), 569–582.

Digital Library

[5]

Gabriela Csurka, Diane Larlus, Florent Perronnin, and France Meylan. 2013. What is a good evaluation measure for semantic segmentation? In Proceedings of the British Machine Vision Conference, Vol. 27.

[6]

Debi Prosad Dogra, Arun K. Majumdar, and Shamik Sural. 2012. Evaluation of segmentation techniques using region area and boundary matching information. Journal of Visual Communication and Image Representation 23, 1 (2012), 150–160.

Digital Library

[7]

Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, Tao Li, and Ali Borji. 2017. Structure-measure: A new way to evaluate foreground maps. In Proceedings of the IEEE International Conference on Computer Vision.

[8]

Deng-Ping Fan, Cheng Gong, Yang Cao, Bo Ren, Ming-Ming Cheng, and Ali Borji. 2018. Enhanced-alignment measure for binary foreground map evaluation. In Proceedings of the International Joint Conferences on Artificial Intelligence.

[9]

Mengyang Feng, Huchuan Lu, and Errui Ding. 2019. Attentive feedback network for boundary-aware salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1623–1632.

[10]

Zhenqi Fu, Feng Shao, Qiuping Jiang, Xiangchao Meng, and Yo-Sung Ho. 2021. Subjective and objective quality assessment for stereoscopic image retargeting. IEEE Transactions on Multimedia 23 (2021), 2100–2113.

[11]

Feng Ge, Song Wang, and Tiecheng Liu. 2007. New benchmark for image segmentation evaluation. Journal of Electronic Imaging 16, 3 (2007), 033011.

[12]

E. Drelie Gelasca. 2005. Full-Reference Objective Quality Metrics for Video Watermarking, Video Segmentation and 3D Model Watermarking. Ph.D. Dissertation. Lausanne, EPFL.

[13]

Elisa Drelie Gelasca and Touradj Ebrahimi. 2009. On evaluating video object segmentation quality: A perceptually driven objective metric. IEEE Journal of Selected Topics in Signal Processing 3, 2 (2009), 319–335.

[14]

Jungong Han, Eric J. Pauwels, and Paul De Zeeuw. 2013. Fast saliency-aware multi-modality image fusion. Neurocomputing 111 (2013), 70–80.

Digital Library

[15]

Chao Huang, Qingbo Wu, and Fanman Meng. 2016. QualityNet: Segmentation quality evaluation with deep convolutional networks. In Proceedings of the 2016 Conference on Visual Communications and Image Processing (VCIP’16). IEEE, Los Alamitos, CA, 1–4.

[16]

Tanveer Hussain, Saeed Anwar, Amin Ullah, Khan Muhammad, and Sung Wook Baik. 2021. Densely deformable efficient salient object detection network. Arxiv Preprint arXiv:2102.06407 (2021).

[17]

Pushmeet Kohli, L’ubor Ladicky, and Philip H. S. Torr. 2009. Robust higher order potentials for enforcing label consistency. International Journal of Computer Vision 82, 3 (2009), 302–324.

Digital Library

[18]

Yin Li, Xiaodi Hou, Christof Koch, James M. Rehg, and Alan L. Yuille. 2014. The secrets of salient object segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 280–287.

Digital Library

[19]

Jiang-Jiang Liu, Qibin Hou, Ming-Ming Cheng, Jiashi Feng, and Jianmin Jiang. 2019. A simple pooling-based design for real-time salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3917–3926.

[20]

Ran Margolin, Lihi Zelnik-Manor, and Ayellet Tal. 2014. How to evaluate foreground maps? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 248–255.

Digital Library

[21]

Kevin McGuinness and Noel E. O’Connor. 2010. A comparative evaluation of interactive segmentation algorithms. Pattern Recognition 43, 2 (2010), 434–444.

Digital Library

[22]

Vida Movahedi and James H. Elder. 2010. Design and perceptual validation of performance measures for salient object segmentation. In Proceedings of the Computer Vision and Pattern Recognition Workshops (CVPRW’10). 49–56.

[23]

Bo Peng, Macmillan Simfukwe, and Tianrui Li. 2018. Region-based image segmentation evaluation via perceptual pooling strategies. Machine Vision and Applications 29, 3 (2018), 477–488.

Digital Library

[24]

Jordi Pont-Tuset and Ferran Marques. 2015. Supervised evaluation of image segmentation and object proposal techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 7 (2015), 1465–1478.

[25]

David Martin Powers. 2011. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies 2, 1 (2011), 37–63.

[26]

Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, and Martin Jagersand. 2019. BASNet: Boundary-aware salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7479–7489.

[27]

Qingbo, Wu, Hongliang, Li, Fanman, Meng, and King N. Ngan. 2018. A perceptually weighted rank correlation indicator for objective image quality assessment. IEEE Transactions on Image Processing 27, 5 (2018), 2499–2513.

[28]

Shi Ran, Xiong Jian, and Qiao Tong. 2020. Objective object segmentation visual quality evaluation based on pixel-level and region-level characteristics. In Proceedings of the 2nd ACM International Conference on Multimedia in Asia (MMAsia’20).

[29]

Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. GrabCut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics 23 (2004), 309–314.

[30]

Ran Shi, King Ngi Ngan, Songnan Li, Raveendran Paramesran, and Hongliang Li. 2015. Visual quality evaluation of image object segmentation: Subjective assessment and objective measure. IEEE Transactions on Image Processing 24, 12 (2015), 5033–5045.

Digital Library

[31]

Wen Shi, Fanman Meng, and Qingbo Wu. 2017. Segmentation quality evaluation based on multi-scale convolutional neural networks. In Proceedings of the 2017 IEEE Conference on Visual Communications and Image Processing (VCIP’17). IEEE, Los Alamitos, CA, 1–4.

[32]

Kyohoon Sim, Jiachen Yang, Wen Lu, and Xinbo Gao. 2020. MaD-DLS: Mean and deviation of deep and local similarity for image quality assessment. IEEE Transactions on Multimedia 23 (2020), 4037–4048.

[33]

Konstantin Sofiiuk, Ilia Petrov, Olga Barinova, and Anton Konushin. 2020. F-BRS: Rethinking backpropagating refinement for interactive segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20).

[34]

Paulo Villegas and Xavier Marichal. 2004. Perceptually-weighted evaluation criteria for segmentation masks in video sequences. IEEE Transactions on Image Processing 13, 8 (2004), 1092–1103.

Digital Library

[35]

Final Report From the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment II, Video Quality Expert Group (VQEG), 2003. [Online]. Available http://www.vqeg.org/.

[36]

Wenguan Wang, Qiuxia Lai, Huazhu Fu, Jianbing Shen, Haibin Ling, and Ruigang Yang. 2021. Salient object detection in the deep learning era: An in-depth survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. Early access, January 12, 2021.

[37]

Wenguan Wang, Jianbing Shen, Ming-Ming Cheng, and Ling Shao. 2019. An iterative and cooperative top-down and bottom-up inference network for salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5968–5977.

[38]

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612.

Digital Library

[39]

Felix A. Wichmann and N. Jeremy Hill. 2001. The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics 63, 8 (2001), 1293–1313.

[40]

Qingbo Wu, Lei Wang, King Ngi Ngan, Hongliang Li, Fanman Meng, and Linfeng Xu. 2020. Subjective and objective de-raining quality assessment towards authentic rain image. IEEE Transactions on Circuits and Systems for Video Technology 30, 11 (2020), 3883–3897.

[41]

Runmin Wu, Mengyang Feng, Wenlong Guan, Dong Wang, Huchuan Lu, and Errui Ding. 2019. A mutual learning method for salient object detection with intertwined multi-supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8150–8159.

[42]

Zhe Wu, Li Su, and Qingming Huang. 2019. Cascaded partial decoder for fast and accurate salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3907–3916.

[43]

Yu Zeng, Yunzhi Zhuge, Huchuan Lu, Lihe Zhang, Mingyang Qian, and Yizhou Yu. 2019. Multi-source weak supervision for saliency detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6074–6083.

[44]

Dingwen Zhang, Junwei Han, Yu Zhang, and Dong Xu. 2019. Synthesizing supervision for learning deep saliency network without human annotation. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 7 (2019), 1755–1769.

[45]

Lu Zhang, Jianming Zhang, Zhe Lin, Huchuan Lu, and You He. 2019. Capsal: Leveraging captioning to boost semantics for salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6024–6033.

[46]

Bartosz Ziółko, David Emms, and Mariusz Ziółko. 2017. Fuzzy evaluations of image segmentations. IEEE Transactions on Fuzzy Systems 26, 4 (2017), 1789–1799.

Cited By

Li JXie QChang XXu JLiu Y(2024)Mutually-Guided Hierarchical Multi-Modal Feature Learning for Referring Image SegmentationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369877120:12(1-18)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3698771
Hou WLi GTian YHu D(2024)Towards Long Form Audio-visual Video UnderstandingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3672079Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3672079
Yang YJoukovsky BOramas Mogrovejo JTuytelaars TDeligiannis N(2024)SNIPPET: A Framework for Subjective Evaluation of Visual Explanations Applied to DeepFake DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366524820:8(1-29)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3665248
Show More Cited By

Index Terms

Objective Object Segmentation Visual Quality Evaluation: Quality Measure and Pooling Method
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Objective object segmentation visual quality evaluation based on pixel-level and region-level characteristics
MMAsia '20: Proceedings of the 2nd ACM International Conference on Multimedia in Asia

Objective object segmentation visual quality evaluation is an emergent member of the visual quality assessment family. It aims at developing an objective measure instead of a subjective survey to evaluate the object segmentation quality in agreement ...
Visual Quality Evaluation of Image Object Segmentation: Subjective Assessment and Objective Measure
A visual quality evaluation of image object segmentation as one member of the visual quality evaluation family has been studied over the years. Researchers aim at developing the objective measures that can evaluate the visual quality of object ...
Modeling visual attention's modulatory aftereffects on visual sensitivity and quality evaluation

With the fast development of visual noise-shaping related applications (visual compression, error resilience, watermarking, encryption, and display), there is an increasingly significant demand on incorporating perceptual characteristics into these ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 18, Issue 3

August 2022

478 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3505208

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2022

Accepted: 01 September 2021

Revised: 01 August 2021

Received: 01 May 2021

Published in TOMM Volume 18, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
397
Total Downloads

Downloads (Last 12 months)81
Downloads (Last 6 weeks)8

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li JXie QChang XXu JLiu Y(2024)Mutually-Guided Hierarchical Multi-Modal Feature Learning for Referring Image SegmentationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369877120:12(1-18)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3698771
Hou WLi GTian YHu D(2024)Towards Long Form Audio-visual Video UnderstandingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3672079Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3672079
Yang YJoukovsky BOramas Mogrovejo JTuytelaars TDeligiannis N(2024)SNIPPET: A Framework for Subjective Evaluation of Visual Explanations Applied to DeepFake DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366524820:8(1-29)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3665248
Roy K(2024)Multimodal Score Fusion with Sparse Low-rank Bilinear Pooling for Egocentric Hand Action RecognitionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365604420:7(1-22)Online publication date: 16-May-2024
https://dl.acm.org/doi/10.1145/3656044
Syu JLin JSrivastava G(2024)Distributed Learning Mechanisms for Anomaly Detection in Privacy-Aware Energy Grid Management SystemsACM Transactions on Sensor Networks10.1145/3640341Online publication date: 17-Jan-2024
https://dl.acm.org/doi/10.1145/3640341
Liu ZLi DZhang XZhang ZZhang PShan CHan J(2024)Pedestrian Attribute Recognition via Spatio-temporal Relationship Learning for Visual SurveillanceACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363262420:6(1-15)Online publication date: 8-Mar-2024
https://dl.acm.org/doi/10.1145/3632624
Muthu Kumara Swamy SHan Q(2024)Quality Evaluation of Image Segmentation in Mobile Augmented RealityMobile and Ubiquitous Systems: Computing, Networking and Services10.1007/978-3-031-63992-0_27(415-427)Online publication date: 19-Jul-2024
https://doi.org/10.1007/978-3-031-63992-0_27
Liang SMa WXie C(2023)Relation with Free Objects for Action RecognitionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/361759620:2(1-19)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1145/3617596
Wang HWang YYu BZhan YYuan CYang W(2023)Attentional Composition Networks for Long-Tailed Human Action RecognitionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/360325320:1(1-18)Online publication date: 9-Jun-2023
https://dl.acm.org/doi/10.1145/3603253
Tang ZChen ZLi ZZhong BZhang XZhang X(2023)Unifying Dual-Attention and Siamese Transformer Network for Full-Reference Image Quality AssessmentACM Transactions on Multimedia Computing, Communications, and Applications10.1145/359743419:6(1-24)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597434
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents