Abstract
Image segmentation is an essential step for many computer vision tasks. Evaluating the quality of image segmentations becomes indispensable for choosing an appropriate output of the image segmentation algorithms. To quantitatively evaluate the segmentation quality, various evaluation measures have been proposed to produce a quality map, and a spatial pooling algorithm is followed to combine the quality map into a single quality score. In this paper, we propose two pooling strategies instead of using the conventional spatial average operation. By assigning perceptual meaningful weights to the quality maps, we obtain evaluation measures that are correlated with the human perception of segmentation quality. Specifically, a quality-based and a visual importance-based pooling strategies are designed and tested on some popular evaluation measures, respectively. To the best of our knowledge, this is the first work that applies perceptual pooling strategies for segmentation evaluation. Extensive experiments are conducted on a subjective evaluation benchmark and the Berkeley Segmentation Dataset (BSDS500). The results indicate that the proposed strategies can improve the performance of existing evaluation measures and produce a more perceptually meaningful judgment on the segmentation quality.
Similar content being viewed by others
Notes
Preliminary results of this work are presented in [18].
References
Arbelaez, P., Maire, M., Fowlkes, C.C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)
Borsotti, M., Campadelli, P., Schettini, R.: Quantitative evaluation of color image segmentation results. Pattern Recognit. Lett. 19(8), 741–747 (1998)
Bruce, N., Tsotsos, J.: Saliency, attention, and visual search: an information theoretic approach. J. Vis. 9(3), 1–24 (2009)
Chen, P., Krim, H., Mendoza, O.: Multiphase joint segmentation–registration and object tracking for layered images. IEEE Trans. Image Process. 19(7), 1706–1719 (2010)
Christensen, H., Phillips, P.: Empirical Evaluation Methods in Computer Vision. World Scientific Publishing Company, Singapore (2002)
Comanicu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)
Felzenszwalb, P., Huttenlocher, D.: Efficient graph based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)
Hou, X., Harel, J., Koch, C.: Image signature: highlighting sparse salient regions. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 194–201 (2012)
Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2336–2343 (2007)
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)
Jiang, X., Marti, C., Irniger, C., Bunke, H.: Distance measures for image segmentation evaluation. EURASIP J. Appl. Signal Process. 2006, 209 (2006)
Kanungo, T., Dom, B., Niblack, W., Steele, D.: A fast algorithm for mdl-based multi-band image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 609–616 (1994)
Martin, D.: An empirical approach to grouping and segmentation. Ph.D. thesis, EECS Department, University of California, Berkeley (2002)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE International Conference on Computer Vision, pp. 416–424 (2001)
Meila, M.: Comparing clusterings: an axiomatic view. In: International Conference on Machine Learning, pp. 577–584 (2005)
Moorthy, A.K., Bovik, A.C.: Visual importance pooling for image quality assessment. IEEE J. Image Inf. Vis. Qual. 3(2), 193–C201 (2009)
Pantofaru, C., Schmid, C., Hebert, M.: Object recognition by integrating multiple image segmentations. In: European Conference on Computer Vision, pp. 481–494 (2008)
Peng, B., Simfukwe, M., Yang, Y., Li., T.: Perceptual pooling strategies for image segmentation quality evaluation. In: The 12th conference on uncertainty modeling in knowledge engineering and decision making (FLINS), pp. 918–923 (2016)
Peng, B., Veksler, O.: Parameter selection for graph cut based image segmentation. In: British Machine Vision Conference, pp. 153–162 (2008)
Pont-Tuset, J., Marques, F.: Measures and meta-measures for the supervised evaluation of image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2131–2138 (2013)
Rao, S., Mobahi, H., Yang, A., Sastry, S., Ma, Y.: Natural image segmentation with adaptive texture and boundary encoding. In: Asian Conference of Computer Vision, pp. 135–146 (2009)
Ren, X., Malik, J.: Learning a classification model for segmentation. In: IEEE International Conference on Computer Vision, pp. 10–17 (2003)
Smistad, E., Falch, T., Bozorgi, M., Elster, A., Lindseth, F.: Medical image segmentation on gpus—a comprehensive review. Med. Image Anal. 20(1), 1–18 (2015)
Tong, N., Lu, H., Zhang, L., Ruan, X.: Saliency detection with multi-scale superpixels. IEEE Signal Process. Lett. 21(9), 1035–1039 (2014)
Unnikrishnan, R., Pantofaru, C., Hebert, M.: Toward objective evaluation of image segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 929–944 (2007)
Wang, Z., Bovik, A.: Modern Image Quality Assessment. Morgan and Claypool Publishing Company, New York (2006)
Yang, A., Wright, J., Ma, Y., Sastry, S.: Unsupervised segmentation of natural images via lossy data compression. Comput. Vis. Image Underst. 11(2), 212–225 (2008)
Yang, C., Zhang, L., Lu, H., Ruan, X., Yan, M.: Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)
Zhang, H., Fritts, J., Goldman, S.: An entropy-based objective segmentation evaluation method for image segmentation. In: SPIE Sotrage and Retrieval Methods and Applicaitons for Multimedia, pp. 38–49 (2004)
Zhang, H., Fritts, J., Goldman, S.: A co-evaluation framework for improving segmentation evaluation. In: SPIE Signal Processing and Target Recognition, pp. 420–430 (2005)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the National Science Foundation of China under Grants 61772435.
Rights and permissions
About this article
Cite this article
Peng, B., Simfukwe, M. & Li, T. Region-based image segmentation evaluation via perceptual pooling strategies. Machine Vision and Applications 29, 477–488 (2018). https://doi.org/10.1007/s00138-017-0903-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-017-0903-x