Open AccessLetter

AssesSeg—A Command Line Tool to Quantify Image Segmentation Quality: A Test Carried Out in Southern Spain from Satellite Imagery

Antonio Novelli

^1,*

Manuel A. Aguilar

Fernando J. Aguilar

Abderrahim Nemmaoui

and

Eufemia Tarantino

DICATECh, Politecnico di Bari, Via Orabona 4, Bari 70125, Italy

Escuela Superior de Ingeniería de la Universidad de Almeria (UAL), Almeria, Ctra. Sacramento, s/n, La Cañada de San Urbano Almería 04120, Spain

Author to whom correspondence should be addressed.

Remote Sens. 2017, 9(1), 40; https://doi.org/10.3390/rs9010040

Submission received: 4 November 2016 / Revised: 9 December 2016 / Accepted: 28 December 2016 / Published: 5 January 2017

Download

Browse Figures

Versions Notes

Abstract

This letter presents the capabilities of a command line tool created to assess the quality of segmented digital images. The executable source code, called AssesSeg, was written in Python 2.7 using open source libraries. AssesSeg (University of Almeria, Almeria, Spain; Politecnico di Bari, Bari, Italy) implements a modified version of the supervised discrepancy measure named Euclidean Distance 2 (ED2) and was tested on different satellite images (Sentinel-2, Landsat 8, and WorldView-2). The segmentation was applied to plastic covered greenhouse detection in the south of Spain (Almería). AssesSeg outputs were utilized to find the best band combinations for the performed segmentations of the images and showed a clear positive correlation between segmentation accuracy and the quantity of available reference data. This demonstrates the importance of a high number of reference data in supervised segmentation accuracy assessment problems.

Keywords:

AssesSeg; segmentation quality; greenhouses; Sentinel-2 Multi Spectral Instrument (MSI); Landsat 8 Operational Land Imager (OLI); WorldView-2 (WV2)

Graphical Abstract

1. Introduction

Environmental modeling and spatial planning need the development of operational tools capable to simplify the extraction of information from available data. This is especially true when the considered data are remotely sensed and the extracted information has an economic relevance [1].

The results presented in this letter are aimed at the enhancement of Object-Based Image Analysis (OBIA) products. OBIA starts by aggregating similar pixels to obtain homogeneous objects by means of a crucial stage called image segmentation. Many reasons justify the use of an OBIA approach (e.g., geometric and neighbourhood information availability and the reduction of salt and pepper effect in classifications). A general review of pros and cons of OBIA can be found in Blaschke [1] and Blaschke et al. [2].

In remote sensing OBIA applications, the segmentation phase is commonly part of the initial stage of the workflow. Scientific literature proposes different methods for assessing the segmentation quality both from unsupervised and supervised approaches (e.g., Drăguţ et al. [3], Liu et al. [4]). However, the proposed methods are not always coupled with tools able to automatically compute the described metrics. To overcome the limited availability of software assessing segmentation accuracy, this paper presents a free of charge command line tool (AssesSeg) that implements a modified version of the supervised discrepancy measure named Euclidean Distance 2 (ED2) proposed by Liu et al. [4]. AssesSeg was used to detect the best band combinations for the detection of plastic-covered greenhouses from three multispectral satellite data: Sentinel-2 (S2), Landsat 8 (L8), and WorldView-2 (WV2) images. The authors’ choice to test AssesSeg on plastic-covered greenhouses segmentation was due to their sound experience on this active research topic using both pixel-based approaches [5,6,7,8] and OBIA approaches [9,10,11,12,13,14]. The performed test also demonstrates the importance of the number of reference objects in segmentation quality assessment.

2. Method

2.1. Background

The modified ED2 works with a set of reference objects (ROs), so being considered a supervised segmentation quality metric. In this study, the ROs were manually digitized polygons. ED2 index (Equation (1)) computations, in their original formulation, start by defining the corresponding segment dataset characterized by

segments that spatially overlap the ROs;
overlapping criteria to be respected [15]: the intersection area between a RO and a candidate corresponding segment is more than half (50%) the area of either the RO or the corresponding segment of the polygon.

When the corresponding segments dataset is defined, the ED2 index evaluates the segmentation quality by computing the potential segmentation error (PSE) and the number-of-segments ratio (NSR). The potential segmentation error (PSE) measures the geometric discrepancy computed through the ratio between the total area of under-segments and ROs. A high PSE indicates macroscopic differences between the ROs and the corresponding segments datasets. In particular, the under-segmentation error occurs when the contour of a RO,

r_{k}

, splits the corresponding segment,

s_{i}

, into at least two parts (Equation (2)). The number-of-segments ratio (NSR) measures the arithmetic discrepancy between the ROs and the corresponding segments, being defined as the absolute difference between the number of ROs (m) and the number of corresponding segments (v) divided by the number of ROs (Equation (3)).

A high ED2 value can indicate a significant geometric discrepancy, a significant arithmetic discrepancy, or both. On the contrary, a small ED2 value can indicate a good segmentation quality.

ED 2 = \sqrt{{(PSE)}^{2} + {(NSR)}^{2}}

(1)

PSE = \frac{\sum | s_{i} - r_{k} |}{\sum | r_{k} |}

(2)

NSR = \frac{| m - v |}{m} .

(3)

The ED2 index was modified since the overlapping criteria acts as a filter both on the candidate corresponding segment and the ROs. In fact, when a RO does not have a corresponding segment candidate, the true number of employed ROs is lower than the original one. To take into account this biasing effect, both the PSE and NSR values were increased considering the actual number of ROs employed. In this way, n being the number of excluded ROs, the new PSE (Equation (4)) and NSR (Equation (5)) values were computed as follows:

{PSE}_{new} = \frac{\sum | s_{i} - r_{k} | + [n \times \max (| s_{i} - r_{k} |)]}{\sum | r_{k} |}

(4)

{NSR}_{new} = \frac{| m - v - [n \times v_{\max}] |}{m - n}

(5)

where

$\max (| s_{i} - r_{k} |)$ is the maximum under-segmented area found for a single RO;
$v_{\max}$ represents the maximum number of corresponding segment found for one single RO;
$\sum | r_{k} |$ here is referred to the total area of the m − n ROs.

2.2. AssesSeg Tool Description

“AssesSeg.exe” (see supplementary material) is a standalone command line tool (.exe extension) that implements the rules described in Section 2. AssesSeg deals only with the ESRI polygon shapefile (it does not depend on the segmentation software), and its source code was written in Python 2.7 given the large availability of open source optimization, data analysis, control, and numerical analysis libraries (e.g., NumPy and SciPy) [16]. It requires two different typologies of input in a working folder (see supplementary material): a RO input shapefile and a set of segmentation shapefiles (at least composed by one single file). The output of the executable is a spreadsheet file (see Appendix A). Figure 1 shows the iterated process for each segmentation file within AssesSeg.

AssesSeg can also deal with different prefixed overlapping percentages between ROs and input segments. However, in this letter, the overlapping criteria, described in Section 2, was always applied. Lastly, AssesSeg is a very powerful tool if coupled with automatic or semi-automatic algorithms capable of producing many segmentation files following a certain criterion. In this work, this was accomplished by means of a semi-automatic eCognition rule set characterized by a looping process among prefixed multi-resolution segmentation (MRS) algorithm parameters.

Appendix A shows an example output table (see supplementary material) and software details (including download links in Table A1).

3. Results: Test Carried Out on Satellite Imagery of Southern Spain

3.1. Study Area and Satellite Data

The test area (Figure 2) is located in the “Sea of Plastic”, province of Almería (Southern Spain) characterized by the greatest concentration of greenhouses in the world.

Three cloud-free images taken with different sensors were used to test AssesSeg within the context of plastic-covered greenhouses segmentations: (i) the recently launched S2 Multi Spectral Instrument (MSI); (ii) the L8 Operational Land Imager (OLI); and (iii) the WV2 satellite (bundle combination of Panchromatic (PAN) and MultiSpectral (MS) bands).

Satellite data were atmospherically and geometrically corrected before segmentation. Particularly, L8 and S2 data were co-registered to the WV2 PAN image given the much higher resolution and geometric accuracy of WV2 imagery. The WV2 data (5 July 2015) and the L8 OLI scene (8 January 2016) images were geometrically and atmospherically corrected using the capabilities of Geomatica v. 2014 (PCI Geomatics, Richmond Hill, ON, Canada). Further details can be found in [12] where a similar pre-processing scheme was undertaken. In particular, the OLI panchromatic band was not used in this test.

Level 1C (L1C) S2 MSI image (12 January 2016, orbit R051) product is a top of atmosphere reflectance dataset with cartographic projection (UTM/WGS84), 12-bit dynamic range, and tiles/granules consisting of 100 km² ortho-images. The MSI sensor collects up to 13 bands with three different geometric resolutions: 60 m, 20 m, and 10 m ground sample distance (GSD). The sen2core algorithm [17] was used to produce a Level 2A (L2A) S2 bottom of an atmosphere reflectance image. Lastly, the study area was extracted from the selected S2 granule (Figure 2) and co-registered to the WV2 PAN image by using Geomatica v. 2014. In this test, 60 m GSD bands were excluded from the segmentations.

3.2. Segmentation Results

The MRS algorithm provided by eCognition v. 8.8 was used to produce different segmentation datasets (see [18] for a complete mathematical description of the algorithm). MRS requires user driven parameters (scale, shape, compactness, and band combinations) to obtain satisfactory results. For this purpose, 400 plastic-covered greenhouse ROs (“400.shp” in supplementary material, the same ROs for all three satellite images), manually delimited on the corrected WV2 PAN scene, were extracted to find the best modified ED2 index. To deal with the scale parameters of the same order of magnitude among the three satellite data, the S2 and L8 initial geometric resolutions (10 m and 20 m GSD for S2, and 30 m GSD for L8) were increased to 1.875 m and 2 m, respectively, by simply splitting pixels in equal parts without any resampling. Then, AssesSeg was used for every modified ED2 index computation.

The test for S2 and L8 started by only varying the scale parameter values (from 10 to 120 with a step of 1) and fixing the shape and compactness to 0.5. In the case of the S2 data, only the 10 m GSD bands (Blue, Green, Red, NIR) were used, whereas for the L8 data the SWIR bands were also tested in consideration of their high sensitivity to plastic coverings [19]. Table 1 depicts the best ED2 results achieved and the corresponding scale parameter values for the tested L8 and S2 band combinations. It is noteworthy that the best band combinations common for S2 and L8 datasets are Blue-Green-NIR and Blue-Green-Red-NIR. Although the S2 Red-NIR combination was characterized by an ED2 value smaller than the Blue-Green-NIR one, the same does not occur for the L8 Red-NIR band combination that features almost the highest L8 measured ED2. The chosen band combinations were subsequently applied to extend and refine the search of an optimal segmentation for S2 and L8 images by varying the scale parameter values, within an interval of the local optimum, and the shape values from 0.1 to 0.9 (with a step of 0.1). The compactness parameter was always set to 0.5 since the literature underlined its minor contribution when compared to shape and, above all, scale parameters [3]. Table 2 summarizes the best achieved results for the two band combinations and the two tested satellite images. It is worth noting that the S2 image always performed better segmentations than the L8 one considering the ED2 metric.

The amplitude of the tested scale parameter values was reduced in the case of the WV2 image (further details can be found in a recent study performed by Aguilar et al. [9] over the same area). In fact, the chosen test scale parameter values ranged from 25 to 45, whereas the tested shape parameters varied from 0.1 to 0.9. The scale, shape, and compactness parameters followed the same rules defined for the S2 and L8 images. Table 3 reports the best results achieved, for each considered band combination, under the performed WV2 tests. In particular, Table 3 shows that the best WV2 segmentation performances were achieved with the Blue-Green-NIR1 combination. Table 2 and Table 3 show that all the best ED2 values (high quality segmentations) were attained, for the three atmospherically corrected satellite data, with a very similar band combination.

Lastly, Figure 3 shows a visual comparison over the same area between the best segmentations derived from S2 and L8 images (Blue-Green-NIR band combination) and WV2 image (Blue-Green-NIR2 band combination).

The segmentation features a high visual quality for WV2 data and an acceptable visual quality for S2 data, whereas L8 segmentation performed the worst. This visual comparison allows readers to fully appreciate the use of the modified ED2 index, implemented in AssesSeg, to extract the best potential segmentation from both VHR and medium resolution satellite images.

4. Discussion

The experimental design of this study shows one possible implementation of the freely available tool AssesSeg. Being a command line “.exe” tool, it can be used both with programming languages (e.g., Matlab, IDL, and Python) that implement the possibility to call the operating system to execute a specified command, and as a standalone tool able to compare one RO input file with many produced segmentation files. Particularly, the above results were achieved by using many segmentation files produced by an eCognition rule set and compared with a large (400 manually digitized polygons) RO input file.

Table 2 and Table 3 proved the stability of the resulting best common band combinations for plastic-covered greenhouses segmentations regardless of the tested satellite image. In particular, S2 and L8 featured the same best band combination, whereas the WV2 dataset showed very good results with either the Blue-Green-NIR1 and Blue-Green-NIR2 band combinations. The same did not occur when the Blue and the Green bands are used at the same time with the two WV2 NIR bands. The results also show that the Blue-Green-Red-NIR band combination is characterized by very good ED2 indices for the three sensors, whereas, for the WV2 dataset, the contemporary use of the two NIR bands leads to higher ED2 results if compared with an analogue single NIR band combination. Moreover, Table 1, Table 2 and Table 3 show that the use of all bands has never been the best option in the selected area.

Figure 3 shows, as already reported in Aguilar et al. [11], that the ED2 metric can be characterized by a very good relationship with the visual quality of plastic-covered greenhouses segmentations. This is especially true with very high resolution data. Lastly, the readers should bear in mind that the above results were achieved using always the same RO input data and with atmospherically corrected band data. Different results can be attained using atmospherically uncorrected data.

Another important finding achieved with AssesSeg is the ability to implement a large RO input file (with hundreds of polygons) avoiding extremely time-consuming manual computations. Indeed, in previous scientific works, only a reduced number of ROs were used to assess the segmentation accuracy (e.g., in [4,9,11,20], this number was less than 100). The improvement achieved through the use of a large RO input can be demonstrated in the chosen test area. For this reason, 50 independent sets of ROs were randomly extracted (with replacement) from the initial population of 400 ROs in groups of variable sample sizes ranging from 25 to 200 ROs with a step of five (see Figure 4). Each single independent set was extracted without replacement. This experiment shows that uncertainty in the computation of the modified ED2 index decreases when the number of ROs employed increases.

Figure 4 and Figure 5 respectively show, for the selected best band combinations of the three satellite data, the scatter plots and the corresponding 95% confidence intervals for the modified ED2 index according to the RO sample size. In this sense, both figures demonstrate that the ED2 variability is excessively high when working with a low number of ROs. In fact, based on the results obtained in this specific work, a good segmentation would require more than 100 ROs. It is relevant that only around 30 ROs per class were used in previous segmentation quality studies [4,20] and that, in this case study, 30 ROs were not enough to guarantee stable results.

5. Conclusions

By introducing AssesSeg, a command line user-friendly tool to assess digital image segmentation quality becomes freely available both for scientific and technical purposes.

The use of the very popular ESRI shapefiles data format makes AssesSeg widely compatible with the outputs of the most common OBIA software. Thanks to AssesSeg, the optimal bands combination and MRS parameters, headed up to carry out plastic-covered greenhouses segmentation, were easily found when working on S2, L8, and WV2 satellite images. Moreover, the test results also indicate the importance of increasing the number of ROs to diminish the uncertainty in assessing the segmentation quality through the ED2 modified metric. This demonstrates the importance of a high number of reference data in supervised segmentation accuracy assessment problems.

Future development will be aimed both at the implementation of a further optimization of the AssesSeg source code, including the development of a graphical user interface, and at the improvement of the performance of segmentation quality metrics indices.

Supplementary Materials

The following are available online at www.mdpi.com/2072-4292/9/1/40/s1. The software “AssesSeg.exe”, test material (a ground truth shapefile “400.shp” and the segmentation shapefiles showed in Table A1: “Scl10_Shp0.5_Comp0.5.shp”, “Scl11_Shp0.5_Comp0.5.shp”, “Scl12_Shp0.5_Comp0.5.shp”, “Scl43_Shp0.3_Com.shp”), the user guide “AssesSeg_manual.pdf”, the structure of the working folder and the results of the computation for the test material (file “test.xlsx”) are provided as supplementary material (as “.zip” archive) for this paper.

Acknowledgments

This work was supported by the Spanish Ministry of Economy and Competitiveness (Spain) and the European Union FEDER funds (Grant Reference AGL2014-56017-R), under the general research lines promoted by the Agrifood Campus of International Excellence ceiA3 (http://www.ceia3.es/). We also thank the journal’s anonymous reviewers for their valuable comments, which greatly improved our paper.

Author Contributions

Antonio Novelli wrote the paper, contributed to manuscript revision, and designed the experiments and the proposed tool with Manuel A. Aguilar. Moreover, Manuel A. Aguilar and Fernando J. Aguilar supervised the experiments and contributed extensively in manuscript revision. Abderrahim Nemmaoui collected ground truth data and collaborated in manuscript revision. Eufemia Tarantino collaborated in manuscript revision.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

The output of the executable AssesSeg.exe, according to Section 2, is an Excel file (.xlsx) containing for each processed segmentation file the following:

number of selected ROs that has at least one corresponding segment according to the selection criteria;
number of segmented geometries (objects) that respect the selection criteria;
total area of selected ROs based on the measure unit of the internal reference system provided by the reference shapefile;
under segmentation area [4];
NSR (modified or original);
PSE (modified or original);
ED2 (modified or original).
name of the i-th output segmentation shapefile processed;
scale, shape, compactness (optional): if the i-th output segmentation shapefile derives from a multi-resolution segmentation (MSR) algorithm (for example, by using a looping process in eCognition) and has a specific name, then these three columns will record the input scale, the shape, and the compactness values of the i-th output shapefile (see Section 3.2). Particularly, AssesSeg recognizes scale, shape, and compactness if the shapefiles are named “SclXX_ShpY.Y_CompZ.Z.shp” or “SclXXX_ShpY.Y_CompZ.Z.shp.” If the name of the selected output segmentation shapefile does not respect the required syntax rule, then the scale, the shape, and the compactness values will be set to 0.
area of ground truth geometries: the total area of selected ground truth geometries expressed in the square of the same unit of the internal reference system of the reference shapefile (if reference system is UTM, the unit is expressed in meters).

Further details on AssesSeg outputs, including the structure of the working folder can be found in “AssesSeg_manual.pdf” (provided as supplementary material)

Table A1 shows an output example table of AssesSeg and details about the software. The shapefiles showed in Table A1 are provided as supplementary material and stored in the file “test.xlsx”.

Table A1. AssesSeg output table example. From left to right are enumerated the following columns: name, scale parameter (SC, for MRS outputs), shape parameter (SP, for MRS outputs), compactness parameter (C, for MRS outputs), number (n) of ground truth (gt) geometries, number of segmented (seg) geometries, area gt (total area of gt geometries), under seg area (total under segmented area), number-of-segments ratio (NSR, modified or original), potential segmentation error (PSE, modified or original), and Euclidean Distance 2 (ED2, modified or original).

**Table A1.** AssesSeg output table example. From left to right are enumerated the following columns: name, scale parameter (SC, for MRS outputs), shape parameter (SP, for MRS outputs), compactness parameter (C, for MRS outputs), number (n) of ground truth (gt) geometries, number of segmented (seg) geometries, area gt (total area of gt geometries), under seg area (total under segmented area), number-of-segments ratio (NSR, modified or original), potential segmentation error (PSE, modified or original), and Euclidean Distance 2 (ED2, modified or original).
Name	SC	SP	C	n. gt Geometries	n. seg Geometries	Area gt	under seg Area	NSR	PSE	ED2
Scl10_Shp0.5_Comp0.5.shp	10	0.5	0.5	400	5943	4670869	4288389	13.86	0.09	13.86
Scl11_Shp0.5_Comp0.5.shp	11	0.5	0.5	400	4930	4670869	450101	11.33	0.10	11.33
Scl12_Shp0.5_Comp0.5.shp	12	0.5	0.5	400	4068	4670869	472559	9.17	0.10	9.17
Scl43_Shp0.3_Com.shp	0	0	0	398	501	4623737	1496193	0.26	0.34	0.42
Details
Name of the software			AssesSeg
Concept			Antonio Novelli, Manuel A. Aguilar
Programming			Antonio Novelli
Availability			https://www.ual.es/Proyectos/GreenhouseSat/index_archivos/links.htm

References

Blaschke, T. Object based image analysis for remote sensing. ISPRS J. Photogramm. Remote Sens. 2010, 65, 2–16. [Google Scholar] [CrossRef]
Blaschke, T.; Hay, G.J.; Kelly, M.; Lang, S.; Hofmann, P.; Addink, E.; Queiroz Feitosa, R.; van der Meer, F.; van der Werff, H.; van Coillie, F.; et al. Geographic object-based image analysis—Towards a new paradigm. ISPRS J. Photogramm. Remote Sens. 2014, 87, 180–191. [Google Scholar] [CrossRef] [PubMed]
Drăguţ, L.; Csillik, O.; Eisank, C.; Tiede, D. Automated parameterisation for multi-scale image segmentation on multiple layers. ISPRS J. Photogramm. Remote Sens. 2014, 88, 119–127. [Google Scholar] [CrossRef] [PubMed]
Liu, Y.; Bian, L.; Meng, Y.; Wang, H.; Zhang, S.; Yang, Y.; Shao, X.; Wang, B. Discrepancy measures for selecting optimal combination of parameter values in object-based image analysis. ISPRS J. Photogramm. Remote Sens. 2012, 68, 144–156. [Google Scholar] [CrossRef]
Agüera, F.; Aguilar, F.J.; Aguilar, M.A. Using texture analysis to improve per-pixel classification of very high resolution images for mapping plastic greenhouses. ISPRS J. Photogramm. Remote Sens. 2008, 63, 635–646. [Google Scholar] [CrossRef]
Agüera, F.; Aguilar, M.; Aguilar, F. Detecting greenhouse changes from QB imagery on the Mediterranean Coast. Int. J. Remote Sens. 2006, 27, 4751–4767. [Google Scholar] [CrossRef]
Novelli, A.; Tarantino, E. The Contribution of Landsat 8 TIRS Sensor Data to the Identification of Plastic Covered Vineyards. In Proceedings of the 2015 Third International Conference on Remote Sensing and Geoinformation of the Environment, Paphos, Cyprus, 16–19 March 2015.
Novelli, A.; Tarantino, E. Combining ad hoc spectral indices based on Landsat-8 OLI/TIRS sensor data for the detection of plastic cover vineyard. Remote Sens. Lett. 2015, 6, 933–941. [Google Scholar] [CrossRef]
Aguilar, M.; Aguilar, F.; García Lorca, A.; Guirado, E.; Betlej, M.; Cichon, P.; Nemmaoui, A.; Vallario, A.; Parente, C. Assessment of multiresolution segmentation for extracting greenhouses from WorldView-2 imagery. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2016, XLI-B7, 145–152. [Google Scholar] [CrossRef]
Aguilar, M.; Bianconi, F.; Aguilar, F.; Fernández, I. Object-based greenhouse classification from GeoEye-1 and WorldView-2 stereo imagery. Remote Sens. 2014, 6, 3554–3582. [Google Scholar] [CrossRef]
Aguilar, M.A.; Nemmaoui, A.; Novelli, A.; Aguilar, F.J.; García Lorca, A. Object-based greenhouse mapping using very high resolution satellite data and Landsat 8 time series. Remote Sens. 2016, 8, 513. [Google Scholar] [CrossRef]
Aguilar, M.A.; Vallario, A.; Aguilar, F.J.; Lorca, A.G.; Parente, C. Object-based greenhouse horticultural crop identification from multi-temporal satellite imagery: A case study in almeria, spain. Remote Sens. 2015, 7, 7378–7401. [Google Scholar] [CrossRef]
Novelli, A.; Aguilar, M.A.; Nemmaoui, A.; Aguilar, F.J.; Tarantino, E. Performance evaluation of object based greenhouse detection from Sentinel-2 MSI and Landsat 8 OLI data: A case study from Almería (Spain). Int. J. Appl. Earth Obs. Geoinf. 2016, 52, 403–411. [Google Scholar] [CrossRef]
Tarantino, E.; Figorito, B. Mapping rural areas with widespread plastic covered vineyards using true color aerial data. Remote Sens. 2012, 4, 1913–1928. [Google Scholar] [CrossRef]
Clinton, N.; Holt, A.; Scarborough, J.; Yan, L.; Gong, P. Accuracy assessment measures for object-based image segmentation goodness. Photogramm. Eng. Remote Sens. 2010, 76, 289–299. [Google Scholar] [CrossRef]
Riaño-Briceño, G.; Barreiro-Gomez, J.; Ramirez-Jaime, A.; Quijano, N.; Ocampo-Martinez, C. Matswmm—An open-source toolbox for designing real-time control of urban drainage systems. Environ. Model. Softw. 2016, 83, 143–154. [Google Scholar] [CrossRef]
Muller-Wilm, U.; Louis, J.; Richter, R.; Gascon, F.; Niezette, M. Sentinel-2 Level 2a Prototype Processor: Architecture, Algorithms and First Results. In Proceedings of the 2013 ESA Living Planet Symposium, Edinburgh, UK, 9–13 September 2013.
Tian, J.; Chen, D.M. Optimization in multi-scale segmentation of high-resolution satellite images for artificial feature recognition. Int. J. Remote Sens. 2007, 28, 4625–4644. [Google Scholar] [CrossRef]
Lu, L.; Di, L.; Ye, Y. A decision-tree classifier for extracting transparent plastic-mulched landcover from Landsat-5 tm images. IEEE J. Appl. Earth Obs. Remote Sens. 2014, 7, 4548–4558. [Google Scholar] [CrossRef]
Witharana, C.; Civco, D.L. Optimizing multi-resolution segmentation scale using empirical methods: Exploring the sensitivity of the supervised discrepancy measure Euclidean Distance 2 (ED2). ISPRS J. Photogramm. Remote Sens. 2014, 87, 108–121. [Google Scholar] [CrossRef]

Figure 1. Flowchart of the core algorithm implemented in AssesSeg.

Figure 2. Location of the study area depicted by means of the Red band of the Sentinel-2 image (T30SWF granule). Coordinate system: ETRS89 UTM Zone 30N.

Figure 3. Visual comparison of the best achieved segmentations over the WorldView-2 orthoimage (true colour visualization): (a) Landsat 8; (b) Sentinel-2; (c) WorldView-2. Coordinate system: ETRS89 UTM Zone 30N.

Figure 4. Scatter plots ED2—number of reference objects (ROs): (a) Landsat 8; (b) Sentinel-2; (c) WorldView-2.

Figure 5. 95% confidence intervals computed from the scatterplots depicted in Figure 4: (a) Landsat 8; (b) Sentinel-2; (c) WorldView-2.

Table 1. Modified ED2 and its associated scale parameter (SP) (shape and compactness were fixed to 0.5) for the tested Sentinel-2 (S2) and Landsat 8 (L8) equal-weighted band combinations. In grey, the best common results achieved.

**Table 1.** Modified ED2 and its associated scale parameter (SP) (shape and compactness were fixed to 0.5) for the tested Sentinel-2 (S2) and Landsat 8 (L8) equal-weighted band combinations. In grey, the best common results achieved.
Band Combination	S2 ED2/SP	L8 ED2/SP
All bands	0.427/36	0.451/39
Blue-Green-NIR	0.406/37	0.448/43
Blue-Green-Red-NIR	0.373/36	0.438/40
Costal-Blue-SWIR1-SWIR2	-	0.512/39
Blue-Green-Red	0.413/41	-
Red-NIR	0.378/37	0.461/43
Red-NIR-SWIR1	-	0.465/40

Table 2. Modified ED2 and its associated scale and shape parameters (compactness was fixed to 0.5) for the tested Sentinel-2 (S2) and Landsat 8 (L8) equal-weighted band combinations. In grey, the best results achieved.

**Table 2.** Modified ED2 and its associated scale and shape parameters (compactness was fixed to 0.5) for the tested Sentinel-2 (S2) and Landsat 8 (L8) equal-weighted band combinations. In grey, the best results achieved.
Band Combination	S2 ED2/SP/Shape	L8 ED2/SP/Shape
Blue-Green-NIR	0.319/39/0.2	0.424/43/0.3
Blue-Green-Red-NIR	0.333/38/0.4	0.429/43/0.3

Table 3. The modified ED2 and its associated scale (SP) and shape parameters (compactness was fixed to 0.5) for the tested WorldView2 (WV2) equal-weighted band combinations. In grey, the best results achieved.

**Table 3.** The modified ED2 and its associated scale (SP) and shape parameters (compactness was fixed to 0.5) for the tested WorldView2 (WV2) equal-weighted band combinations. In grey, the best results achieved.
Band Combination	WV2 ED2/SP/Shape
Blue-Green-NIR2	0.198/37/0.4
Blue-Green-NIR1	0.200/38/0.3
Blue-Green-NIR1-NIR2	0.216/42/0.2
Blue-Green-Red-NIR1-NIR2	0.221/38/0.2
Blue-Green-Red-NIR1	0.204/39/0.2
Blue-Green-Red-NIR2	0.203/38/0.3
Red-NIR2	0.231/39/0.2
Red-NIR1	0.238/40/0.3
Red-NIR1-NIR2	0.233/35/0.4
All	0.222/38/0.3

© 2017 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Novelli, A.; Aguilar, M.A.; Aguilar, F.J.; Nemmaoui, A.; Tarantino, E. AssesSeg—A Command Line Tool to Quantify Image Segmentation Quality: A Test Carried Out in Southern Spain from Satellite Imagery. Remote Sens. 2017, 9, 40. https://doi.org/10.3390/rs9010040

AMA Style

Novelli A, Aguilar MA, Aguilar FJ, Nemmaoui A, Tarantino E. AssesSeg—A Command Line Tool to Quantify Image Segmentation Quality: A Test Carried Out in Southern Spain from Satellite Imagery. Remote Sensing. 2017; 9(1):40. https://doi.org/10.3390/rs9010040

Chicago/Turabian Style

Novelli, Antonio, Manuel A. Aguilar, Fernando J. Aguilar, Abderrahim Nemmaoui, and Eufemia Tarantino. 2017. "AssesSeg—A Command Line Tool to Quantify Image Segmentation Quality: A Test Carried Out in Southern Spain from Satellite Imagery" Remote Sensing 9, no. 1: 40. https://doi.org/10.3390/rs9010040

APA Style

Novelli, A., Aguilar, M. A., Aguilar, F. J., Nemmaoui, A., & Tarantino, E. (2017). AssesSeg—A Command Line Tool to Quantify Image Segmentation Quality: A Test Carried Out in Southern Spain from Satellite Imagery. Remote Sensing, 9(1), 40. https://doi.org/10.3390/rs9010040

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu