Abstract
The extension of Mathematical Morphology to colour and multivariate images is challenging due to the need to define a total ordering in the colour space. No one general way of ordering multivariate data exists and, therefore, there is no single, definitive way of performing morphological operations on colour images. In this paper, we propose an extension to mathematical morphology, based on reduced ordering, specifically the morphological Hit-or-Miss Transform which is used for object detection. The reduced ordering employed transforms multivariate observations to scalar comparisons allowing for an order to be derived and for both flat and non-flat structuring elements to be used. We also compare other definitions of the Hit-or-Miss Transform and test alternative colour ordering schemes presented in the literature. Our proposed method is shown to be intuitive and outperforms other approaches to multivariate Hit-or-Miss Transforms. Furthermore, methods of setting the parameters of the proposed Hit-or-Miss Transform are introduced in order to make the transform robust to noise and partial occlusion of objects and, finally, a set of design tools are presented in order to obtain optimal values for setting these parameters accordingly.
References
[1] H. M. Al-Otum. Morphological operators for color image processing based on Mahalanobis distance measure. volume 42, page 12. SPIE, 2003.10.1117/1.1594727Search in Google Scholar
[2] J. Angulo. Morphological colour operators in totally ordered lattices based on distances: Application to image filtering, enhancement and analysis. Computer Vision and Image Understanding, 107(1):56–73, 2007.Search in Google Scholar
[3] E. Aptoula, S. Lefevre, and C. Ronse. A hit-or-miss transform for multivariate images. Pattern Recognition Letters, 30(8):760–764, 2009.10.1016/j.patrec.2009.02.007Search in Google Scholar
[4] E. Aptoula and S. Lefèvre. A comparative study on multivariate mathematical morphology. Pattern Recognition, 40(11):2914–2929, 2007.10.1016/j.patcog.2007.02.004Search in Google Scholar
[5] V. Barnett. The ordering of multivariate data. Journal of the Royal Statistical Society, 139(3):318–355, 1976.10.2307/2344839Search in Google Scholar
[6] B. Burgeth and A. Kleefeld. An approach to color-morphology based on Einstein addition and Loewner order. Pattern Recognition Letters, 47(Supplement C):29–39, 2014.10.1016/j.patrec.2014.01.018Search in Google Scholar
[7] M. L. Comer and E. J. Delp. Morphological operations for color image processing. J. Electronic Imaging, 8(3):279–289, 1999.10.1117/1.482677Search in Google Scholar
[8] A. N. Evans and X. U. Liu. A morphological gradient approach to color edge detection. IEEE Transactions on Image Processing, 15(6):1454–1463, 2006.10.1109/TIP.2005.864164Search in Google Scholar
[9] G. Franchi, A. Fehri, and A. Yao. Deep morphological networks. Pattern Recognition, page 107246, 2020.10.1016/j.patcog.2020.107246Search in Google Scholar
[10] Grupo de Inteligencia Computacional. Hyperspectral imagery synthesis (EIAs) toolbox. Universidad del País Vasco/Euskal Herriko Unibertsitatea (UPV/EHU), Spain.Search in Google Scholar
[11] N. R. Harvey, R. B. Porter, and J. Theiler. Ship detection in satellite imagery using rank-order greyscale hit-or-miss transforms. Report, 2010.10.1117/12.850886Search in Google Scholar
[12] K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask R-CNN. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2980–2988, 2017.10.1109/ICCV.2017.322Search in Google Scholar
[13] H. Heijmans. Morphological Image Operators. Academic Press, 1994.Search in Google Scholar
[14] M. A. Islam, B. Murray, A. Buck, D. T. Anderson, G. Scott, M. Popescu, and J. Keller. Deep morphological hit-or-miss transform neural network. arXiv preprint arXiv:1912.02259, 2019.Search in Google Scholar
[15] M. Ivanovici, A. Căliman, N. Richard, and C. Fernandez-Maloigne. Towards a multivariate probabilistic morphology for colour images. In Conference on Colour in Graphics, Imaging, and Vision, volume 2012, pages 189–193. Society for Imaging Science and Technology, 2012.Search in Google Scholar
[16] L. Jiao, F. Zhang, F. Liu, S. Yang, L. Li, Z. Feng, and R. Qu. A survey of deep learning-based object detection. IEEE Access, 7:128837–128868, 2019.10.1109/ACCESS.2019.2939201Search in Google Scholar
[17] R. F. Kokaly, R. N. Clark, G. A. Swayze, K. E. Livo, T. M. Hoefen, N. C. Pearson, R. A. Wise, W. M. Benzel, H. A. Lowers, and R. L. Driscoll. USGS spectral library version 7. Report 2327-638X, 2017.10.3133/ds1035Search in Google Scholar
[18] P. Lambert and J. Chanussot. Extending mathematical morphology to color image processing. Proc. CGIP, pages 158–163, 2000.Search in Google Scholar
[19] A. Ledoux, N. Richard, and A.-S. Capelle-Laizé. Color hit-or-miss transform (cmomp). In Signal Processing Conference (EUSIPCO), 2012 Proceedings of the 20th European, pages 2248–2252. IEEE, 2012.Search in Google Scholar
[20] S. Lefèvre, E. Aptoula, B. Perret, and J. Weber. Morphological template matching in color images, pages 241–277. Springer, 2014.10.1007/978-94-007-7584-8_8Search in Google Scholar
[21] O. Lezoray, C. Charrier, and A. Elmoataz. Learning complete lattices for manifold mathematical morphology. In International Symposium on Mathematical Morphology, pages 1–4, 2009. Computer Science [cs]/Image ProcessingConference papers.Search in Google Scholar
[22] R. P. Loce and E. R. Dougherty. Facilitation of optimal binary morphological filter design via structuring element libraries and design constraints. Optical Engineering, 31(5):1008–1025, 18, 1992.10.1117/12.56144Search in Google Scholar
[23] F. Macfarlane, P. Murray, S. Marshall, B. Perret, A. Evans, and H. White. A colour hit-or-miss transform based on a rank ordered distance measure. In 2018 26th European Signal Processing Conference (EUSIPCO), pages 588–592, 2018.10.23919/EUSIPCO.2018.8553050Search in Google Scholar
[24] J. Masci, J. Angulo, and J. Schmidhuber. A learning framework for morphological operators using counter–harmonic mean. In International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing, pages 329–340. Springer, 2013.10.1007/978-3-642-38294-9_28Search in Google Scholar
[25] G. Matheron. Random Sets and Integral Geometry. Wiley, New York, NY, 1975.Search in Google Scholar
[26] R. Mondal, M. S. Dey, and B. Chanda. Image restoration by learning morphological opening-closing network. Mathematical Morphology - Theory and Applications, 4(1):87–107, 2020.10.1515/mathm-2020-0103Search in Google Scholar
[27] P. Murray and S. Marshall. A new design tool for feature extraction in noisy images based on grayscale hit-or-miss transforms. IEEE Transactions on Image Processing, 20(7):1938–1948, 2011.10.1109/TIP.2010.2103952Search in Google Scholar PubMed
[28] P. Murray and S. Marshall. A Review of Recent Advances in the Hit-or-Miss Transform, volume 175, book section 5, pages 221–282. Elsevier, 2013.10.1016/B978-0-12-407670-9.00005-6Search in Google Scholar
[29] P. Murray, S. Marshall, and E. Bullinger. The percentage occupancy hit or miss transform. In Signal Processing Conference, 2009 17th European, pages 253–257. IEEE, 2009.Search in Google Scholar
[30] B. Naegel, N. Passat, and C. Ronse. Grey-level hit-or-miss transforms—part I: unified theory. Pattern Recognition, 40(2):635–647, 2007.Search in Google Scholar
[31] K. Nogueira, J. Chanussot, M. D. Mura, W. R. Schwartz, and J. A. d. Santos. An introduction to deep morphological networks. arXiv preprint arXiv:1906.01751, 2019.Search in Google Scholar
[32] B. Perret, S. Lefevre, and C. Collet. A robust hit-or-miss transform for template matching applied to very noisy astronomical images. Pattern Recognition, 42(11):2470–2480, 2009.10.1016/j.patcog.2009.02.013Search in Google Scholar
[33] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems, pages 91–99, 2015.Search in Google Scholar
[34] C. Ronse. Why mathematical morphology needs complete lattices. Signal Processing, 21(2):129–154, 1990.10.1016/0165-1684(90)90046-2Search in Google Scholar
[35] M. Sangalli and M. E. Valle. Color mathematical morphology using a fuzzy color-based supervised ordering. Fuzzy Information Processing, pages 278–289. Springer International Publishing, 2018.10.1007/978-3-319-95312-0_24Search in Google Scholar
[36] M. Sangalli and M. E. Valle. Approaches to multivalued mathematical morphology based on uncertain reduced orderings. In International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing, pages 228–240. Springer, 2019.10.1007/978-3-030-20867-7_18Search in Google Scholar
[37] J. Serra. Image Analysis and Mathematical Morphology, volume 1. Academic press, New York, 1982.Search in Google Scholar
[38] J. Serra. Image Analysis and Mathematical Morphology, Vol. 2: Theoretical Advances, volume 2. Academic Press, New York, NY, 1988.Search in Google Scholar
[39] X. Shen and W. Bao. Hyperspectral endmember extraction using spatially weighted simplex strategy. Remote Sensing, 11(18):2147, 2019.10.3390/rs11182147Search in Google Scholar
[40] P. Soille. Advances in the analysis of topographic features on discrete images. In Discrete Geometry for Computer Imagery, pages 271–296. Springer, 2002.10.1007/3-540-45986-3_16Search in Google Scholar
[41] P. Soille. Morphological Image Analysis: Principles and Applications. Springer Science & Business Media, 2 edition, 2013.Search in Google Scholar
[42] K. Stankov and D.-C. He. Detection of buildings in multispectral very high spatial resolution images using the percentage occupancy hit-or-miss transform. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(10):4069–4080, 2014.10.1109/JSTARS.2014.2308301Search in Google Scholar
[43] S. Velasco-Forero and J. Angulo. Hit-or-miss transform in multivariate images. In International Conference on Advanced Concepts for Intelligent Vision Systems, pages 452–463. Springer, 2010.10.1007/978-3-642-17688-3_42Search in Google Scholar
[44] S. Velasco-Forero and J. Angulo. Supervised ordering in ℝp: Application to morphological processing of hyperspectral images. IEEE Transactions on Image Processing, 20(11):3301–3308, 2011.Search in Google Scholar
[45] J. Weber and S. Lefèvre. Spatial and spectral morphological template matching. Image and Vision Computing, 30(12):934–945, 2012.10.1016/j.imavis.2012.07.002Search in Google Scholar
[46] S. Wu and Y. Xu. DSN: A new deformable subnetwork for object detection. IEEE Transactions on Circuits and Systems for Video Technology, pages 1–1, 2019.10.1109/TCSVT.2019.2905373Search in Google Scholar
[47] G.-S. Xia, X. Bai, J. Ding, Z. Zhu, S. Belongie, J. Luo, M. Datcu, M. Pelillo, and L. Zhang. DOTA: A large-scale dataset for object detection in aerial images. In Proc. CVPR, 2018.10.1109/CVPR.2018.00418Search in Google Scholar
[48] H. Xu, Y. Zhang, and H. Zhao. Edge detection of color image using mathematical morphology in HSV color space. In Proceedings of 2012 2nd International Conference on Computer Science and Network Technology, pages 2112–2116, 2012.10.1109/ICCSNT.2012.6526335Search in Google Scholar
[49] C.-W. Yeh. Colour morphology and its approaches. Thesis, 2015.Search in Google Scholar
[50] C. W. Yeh and D. Pycock. Similarity-based colour morphology. In 2013 5th Computer Science and Electronic Engineering Conference (CEEC), pages 71–76, 2013.10.1109/CEEC.2013.6659448Search in Google Scholar
[51] Y. Zhang, S. Blusseau, S. Velasco-Forero, I. Bloch, and J. Angulo. Max-plus operators applied to filter selection and model pruning in neural networks. In International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing, pages 310–322. Springer, 2019.10.1007/978-3-030-20867-7_24Search in Google Scholar
© 2020 Fraser Macfarlane et al., published by De Gruyter
This work is licensed under the Creative Commons Attribution 4.0 International License.