[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Next Article in Journal
Spatial Transformation of Equality – Generalized Travelling Salesman Problem to Travelling Salesman Problem
Previous Article in Journal
OSM Data Import as an Outreach Tool to Trigger Community Growth? A Case Study in Miami
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJGI
Volume 7
Issue 3
10.3390/ijgi7030114
ijgi-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Accuracy Assessment of Different Digital Surface Models

by
Ugur Alganci
1,*,
Baris Besol
2 and
Elif Sertel
1
1
Geomatics Engineering Department, Civil Engineering Faculty, Istanbul Technical University, ITU Ayazaga Campus, Sariyer 34469, Istanbul, Turkey
2
Graduate School of Science Engineering and Technology, Institute of Science and Technology, Istanbul Technical University, ITU Ayazaga Campus, Sariyer 34469, Istanbul, Turkey
*
Author to whom correspondence should be addressed.
ISPRS Int. J. Geo-Inf. 2018, 7(3), 114; https://doi.org/10.3390/ijgi7030114
Submission received: 21 January 2018 / Revised: 19 February 2018 / Accepted: 12 March 2018 / Published: 15 March 2018
Browse Figures
Figure 1
<p>Geographic representation of the study area with a Pleiades high-resolution (PHR) image and a general overview of the region from Google Earth©.</p> "> Figure 2
<p>The gray levels represent the DSM dataset and the reference DSM data.</p> "> Figure 3
<p>(<b>a</b>) Land cover map of the study area; (<b>b</b>) reference elevation map from HGK data and the locations of checkpoints.</p> "> Figure 4
<p>The height difference distribution of the DSMs derived from 1000 checkpoints.</p> "> Figure 5
<p>Positions of the vertical profiles. (1) North/South; (2) Northeast/Southwest; (3) East/West; (4) Southeast/Northwest.</p> "> Figure A1
<p>Visuals and cross-sections of Profile 3a.</p> "> Figure A2
<p>Visuals and cross-sections of Profile 3b.</p> "> Figure A3
<p>Visuals and cross-sections of Profile 3c.</p> "> Figure A4
<p>Visuals and cross-sections of Profile 4a.</p> "> Figure A5
<p>Visuals and cross-sections of Profile 4b.</p> ">
Versions Notes

Abstract

:
Digital elevation models (DEMs), which can occur in the form of digital surface models (DSMs) or digital terrain models (DTMs), are widely used as important geospatial information sources for various remote sensing applications, including the precise orthorectification of high-resolution satellite images, 3D spatial analyses, multi-criteria decision support systems, and deformation monitoring. The accuracy of DEMs has direct impacts on specific calculations and process chains; therefore, it is important to select the most appropriate DEM by considering the aim, accuracy requirement, and scale of each study. In this research, DSMs obtained from a variety of satellite sensors were compared to analyze their accuracy and performance. For this purpose, freely available Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) 30 m, Shuttle Radar Topography Mission (SRTM) 30 m, and Advanced Land Observing Satellite (ALOS) 30 m resolution DSM data were obtained. Additionally, 3 m and 1 m resolution DSMs were produced from tri-stereo images from the SPOT 6 and Pleiades high-resolution (PHR) 1A satellites, respectively. Elevation reference data provided by the General Command of Mapping, the national mapping agency of Turkey—produced from 30 cm spatial resolution stereo aerial photos, with a 5 m grid spacing and ±3 m or better overall vertical accuracy at the 90% confidence interval (CI)—were used to perform accuracy assessments. Gross errors and water surfaces were removed from the reference DSM. The relative accuracies of the different DSMs were tested using a different number of checkpoints determined by different methods. In the first method, 25 checkpoints were selected from bare lands to evaluate the accuracies of the DSMs on terrain surfaces. In the second method, 1000 randomly selected checkpoints were used to evaluate the methods’ accuracies for the whole study area. In addition to the control point approach, vertical cross-sections were extracted from the DSMs to evaluate the accuracies related to land cover. The PHR and SPOT DSMs had the highest accuracies of all of the testing methods, followed by the ALOS DSM, which had very promising results. Comparatively, the SRTM and ASTER DSMs had the worst accuracies. Additionally, the PHR and SPOT DSMs captured man-made objects and above-terrain structures, which indicated the need for post-processing to attain better representations.

1. Introduction

Digital elevation models (DEMs) are important data sources for several applications that require surface height information [1]. A DEM is a 3D projection of the Earth that can be categorized into two groups: digital terrain models (DTMs), which are free of trees, buildings, and all types of objects, and digital surface models (DSMs), which reflect the Earth’s surface, including all man-made and natural objects [2]. A DEM can be found in a raster data format, which is an array of square cells (i.e., pixels) with a height value associated with each pixel [3]. DEMs are used as elevation data sources in various geospatial studies and applications, such as topography, geomorphology, plant cover research, tsunami assessments, urban studies, archeology, and glacier observations [4,5]. Contour lines, topographic maps, global positioning system (GPS) measurements, photogrammetry techniques, radar interferometry, stereo satellite images, and laser scanning are the main data sources that produce DEMs [3]. These data sources can be evaluated in four different aspects: cost, accuracy, resolution, and preprocessing. Moreover, each of these techniques has both advantages and disadvantages. Today, studies mostly use DEMs obtained by remote sensing methods instead of direct measurement techniques due to the increased number of observation satellites with stereo capabilities and increased spatial and temporal resolution, as well as the reduced cost of the production of new DEMs [6]. DEM data produced from synthetic aperture radar (SAR) or optical satellite images are initially in the DSM form [7,8]. DSMs can be used in their original form or they can be processed to obtain a DTM by applying the necessary filters according to the purpose of use. DSMs are mostly used for landscape modeling, visualization applications, and 3D digital city applications, while DTMs are usually used for flood or drainage modeling, land use studies, geological applications, and orthorectification of satellite images or aerial photographs [9,10,11]. Location accuracy and quality of DSM/DTM data are crucial, as these metrics have direct impacts on the analyses that use those data as sources. There are many studies in the literature on DSM data generation from optical/SAR satellite images and/or their quality assessments in recent years [12,13,14,15,16,17,18,19,20,21,22,23,24,25]. Table 1 summarizes these studies in terms of region, data source, and DSM generation method and/or accuracy metrics.
DSM data can be categorized into two groups based on coverage extension, resolution, and delivery options. The first group is produced from medium-resolution spatial sensors, which are available worldwide and are mainly distributed free of charge. The Shuttle Radar Topography Mission (SRTM), the Advanced Spaceborne Thermal Emission and Reflection Radiometer Global DEM (ASTER GDEM), and the Advanced Land Observing Satellite (ALOS) World 3D (AW3D) are categorized in this group and capture nearly all of Earth’s landmass free of charge. The second group is composed of local DEMs that are produced from medium- to very high-resolution optical or SAR satellite image data for a limited area of interest. In this study, the vertical accuracies of DSMs belonging to these two groups were evaluated together.
Although global accuracy metrics are available for medium-resolution DSM data, to our knowledge, there has not been a detailed comparison of local DSMs that are produced from high-resolution stereo/tri-stereo satellite images that considers the effects of different land cover characteristics on vertical accuracy.
The main objective of this research is to evaluate the relative vertical agreement of different DEMs that are in the DSM form compared to a reference 5 m grid spaced DSM. For this purpose, the ASTER 30 m, SRTM 30 m, and ALOS 30 m DSMs were obtained for the research area using a 3 m resolution DSM produced from tri-stereo images of the SPOT 6 satellite and a 1 m resolution DSM produced from tri-stereo images of the Pleiades high-resolution (PHR) 1A satellite. The reference DSM was produced from 30 cm spatial resolution aerial photos and provides ±1 m and ±3 m vertical accuracy at the 90% confidence interval (CI) in flat and hilly areas, respectively. The relative vertical agreement of the DSMs was tested with different accuracy assessment approaches in order to provide information about the following aspects:
  • The comparative and quantitative vertical accuracy of the DSMs in the study region.
  • The ranking of the comparative accuracy of the DSMs for specific land cover classes.
  • The performance of the DSMs in bare lands (i.e., terrain representation).

2. Study Area and Data

The study area selected for this research is inside the Istanbul metropolitan area in Turkey. The study area was selected according to data availability, and it includes forested, residential, and industrial areas of Istanbul that have experienced minimal change in recent decades. Specifically, the residential areas that consisted of high-rises and different types of buildings were good candidates for evaluation of extreme conditions (Figure 1).
In this research, freely available ASTER 30 m, SRTM 30 m, and ALOS 30 m DEMs were used as the first dataset group. These data were in DSM form, and a general description of each dataset is provided below.
NASA released the latest version of the SRTM DEM (v.3.0) in 2015. SAR images with a very small base=to-height ratio were processed to obtain DSMs by using interferometric SAR principles. The cartographic products, which were generated from the SRTM data, were sampled numerically on a grid of 1 arcsec (approximately 30 m) [26]. Several studies were performed in order to verify the accuracy of the SRTM DEMs by comparing the results with various reference data and other DEM products [27,28,29,30]. The validation report provided a 6.6 m absolute vertical error, according to kinematic GPS-based ground control point (GCP) comparison, and an 8.5 m absolute vertical error, according to the land GCP comparison for Eurasia [31]. More recent studies reported 12.4 m and 11.9 m vertical root mean square errors (RMSEs) for Europe and Eurasia, respectively [32].
The first version of the ASTER GDEM data was introduced to the global user community in July 2009, and an enhanced version (v.2), which was produced with additional data, improved water masking, and improved vertical accuracy, was distributed on 17 October 2011, by NASA and the METI. The ASTER GDEM v.2 vertical and horizontal RMSEs were calculated as approximately 12 m and 6 m, respectively, which showed an important quality improvement over the previous version [27]. In particular, the ASTER and SRTM DEMs can have faulty elevation values due to radar shadows, clouds, or low contrast [30].
A new global DSM dataset was produced from the 2.5 m spatial resolution data acquired by the Panchromatic Remote-sensing Instrument for Stereo Mapping (PRISM) onboard the ALOS. The AW3D project provided the DSM with a decent resolution of 0.15 arcsec (approximate 5 m), which is currently the most precise global-scale elevation. The first version of the AW3D DSM was distributed to commercial bases by the NTT DATA and RESTEC in March 2016 [33]. In 2015, the Japan Aerospace Exploration Agency (JAXA) released a free-of-charge DSM named the AW3D–30 m, which was a global DSM dataset with a horizontal resolution of approximately 30 m (1 arcsec). In fact, these data were a resampled version of the 5 m mesh version of the AW3D [34]. A recent study performed by Takuku et al. reported a 3.28 m vertical RMSE worldwide and a 3.69 m vertical RMSE for Turkey [35].
The second group of DSM datasets in this research was produced from tri-stereo images acquired by the PHR 1A and 1B and the SPOT 6 and 7 optical satellites. The PHR 1A and PHR 1B satellites were successfully launched into orbit in 2011 and 2012, respectively. The PHR satellites have agile sensors that enable tri-stereo along-track acquisitions, which result in three images over the same area with different acquisition angles during a single pass, one of which is almost in nadir. This configuration provides a promising dataset for modeling topography within the observed scene [36]. The spatial resolutions of the panchromatic (PAN) and multispectral (MS) images are 0.70 m and 2.80 m, respectively, with a theoretical swath of 21 km. The SPOT 6 satellite was launched in 2012 and was followed by the SPOT 7 satellite in 2014. These twin satellites also have the capability of tri-stereo along-track imaging and provide 1.5 m PAN spatial resolution and 6 m MS spatial resolution products with a 60 km swath width [37].
The acquisition parameters of the tri-stereo satellite images used in this research are given in Table 2.
To perform the tests, elevation reference DSM data were provided by the General Command of Mapping (Harita Genel Komutanlığı—HGK). The HGK DSM data were produced from 30 cm spatial resolution stereo aerial photographs acquired with a Microsoft UltraCam Eagle multispectral camera onboard a Beechcraft Super King Air B-200 aircraft [38]. The reference DSM was produced using automated image matching and was edited manually to mask gross errors and water surfaces (i.e., seas, lakes, and wide streambeds). The 5 m grid spaced DSM provides ±1 m and ±3 m vertical accuracy at the 90% (CI) in flat and hilly areas, respectively [39]. As the HGK DSM was produced from the highest spatial resolution input data and underwent a manual editing process, it was selected as the reference data when compared to the other datasets used in this research. All DSM data used in this study are presented in Figure 2.

3. Methods

3.1. DSM Generation from PHR 1A and 1B and SPOT 6 and 7 Satellite Images

There are different types of commercial and scientific software that process high-resolution satellite optical imagery for DSM generation. The tri-stereo satellite images used in this research were processed in the OrthoEngine module of the PCI Geomatica software for DSM extraction. The DSM was generated by image correlation of the epipolar geometry and application of the acquisition geometry with a rational function model. The Wallis filter was applied to the DSM generation in order to improve the contrast and matching ratio. The automated image matching algorithm was applied to produce matching points from the pixels in the image pairs. The algorithm used a mean-normalized cross-correlation with a multiscale strategy. In the next step, three epipolar images—fore-nadir, nadir-after, and fore-after—were produced at the spatial resolution of the source images. In the last step, the DSM was produced according to the mean sea level datum, with a sampling factor of 2 that was equivalent to two times that of the spatial resolution [20].

3.2. Production of Land Cover Map and Independent Checkpoints

The land cover classification of the study area was performed by the visual interpretation of on-screen digitization, using the 30 cm orthophotos, that was produced from the same aerial photographs used in the reference HGK DSM data. These HGK orthophotos provide ±2.5 m horizontal positional accuracy according to unpublished reports from the producer. Generated classes were generalized according to the density of the land cover inside the patch (Figure 3a). A resulting map was used to perform a comparative accuracy analysis of the DSMs related to the land cover classes. Figure 3b shows the elevation map generated from the reference HGK DSM data. The tested area was between 0 m (sea level) and 190 m, according to the reference data. Checkpoints were generated randomly with a stratified sampling strategy based on the elevation intervals.

3.3. Geometric Registration

For the vertical accuracy assessment of several DSMs, it is important to ensure horizontal location matching. The ASTER, SRTM, and ALOS data have acceptable horizontal accuracies regarding their spatial resolution. During the production of the PHR and SPOT DSMs, the geometric accuracy was improved by using GCPs. The co-registration of the DSMs was controlled visually by checking the locational fit of streambeds and roads that were observable in the DSM data. As a result, all of the DSM data obtained from different sources were horizontally matched with each other.

3.4. Accuracy Assessment

After the preprocessing steps were completed, the vertical accuracy assessment of the DSMs was performed in three different aspects.
In the first approach, a point-based assessment was performed on bare lands in order to evaluate the accuracy of the DSMs for terrain representation. For this purpose, 25 checkpoints that were located in bare lands were collected from 30 cm resolution HGK orthophotos. Then, height values belonging to these GCPs were derived for each DSM datum, including the reference DSM, using an overlay analysis. After deriving the height information, RMSE, accuracy (at the 95% CI), and standard deviation were calculated separately for all of the DSM data (Equations (1) and (2)).
R M S E = ∑ ​ ( Z İ − Z İ ′ ) 2 N
Accuracy = RMSE × 1.96
where Zi corresponds to the height value measured from the reference DSM and Z’i corresponds to the height value measured from the test DSMs.
In the second approach, 1030 randomly selected checkpoints were selected from the whole study region to assess the vertical accuracy of the DSMs. Stratified sampling was performed according to the different elevation intervals derived from the reference DSM. The spatial distribution of checkpoints is provided in Figure 3b. The accuracy metrics provided in the first approach were also calculated for these points, and their land cover labels were created by using an intersection analysis with the land cover map provided in Figure 3a. The accuracy metric given in Equation (2) is based on the assumption that vertical errors are normally distributed. To test this condition, frequency histograms of height differences (theoretically, the errors) were produced for each DSM (Figure 4). According to the results, the ALOS, ASTER, and SPOT DSMs provide normal distribution characteristics, and the PHR DSM is very close to a normal distribution with a very slight positive skew, while the same situation is observed for the SRTM DSM with a very slight negative skew. These results indicate that the accuracy metric can be used for this research.
In the last approach, vertical profiles were produced for different geographic directions that corresponded to 45° angular intervals for the whole DSM dataset. In the next step, elevation change characteristics for each DSM in 1 km length portions of the profiles under several topographic conditions and land cover scenarios were determined. Figure 5 presents the locations of the profiles over the study region.

4. Results and Discussion

4.1. Results of the Point-Based Assessment

After the preprocessing steps, the vertical accuracy of the DSMs was evaluated. For the first approach, elevation information extracted from 25 checkpoints was compared with that corresponding to the reference DSM. According to the comparison results given in Table 3, the DSM produced from the tri-stereo PHR images provided the lowest overall RMSE and the highest accuracy in bare terrain conditions. The ALOS and SPOT DSMs also provided acceptable RMSEs at approximately 2 m, while the SRTM DSM ranked fourth, and the ASTER DSM ranked last with the highest RMSE and the lowest accuracy. When the standard deviation of the errors was examined, the PHR, ALOS, and SPOT DSMs provided lower values, which implies that the error magnitudes for the different checkpoints were similar, while the other DSMs showed variations among each other.
In the second approach, 1030 randomly selected points were used in the accuracy assessment, and the same metrics used in the first approach were extracted. Because these points were distributed homogenously over the study region, the results provided a generalized quantification of accuracy for different land cover and topography types. It is important to note that the PHR and SPOT DSMs were not post-processed and included defects due to improper image matching. Some of these defects were observable in the Northwest side of the PHR DSM in Figure 2. Considering the defects mentioned above, a secondary analysis was performed by removing 30 points, which provided extreme absolute errors (±30 to 80 m) in the PHR and SPOT DSMs. After removal of these points, the RMSEs decreased and the accuracy increased in all of the DSMs, which produced a significant improvement in the PHR DSM (Table 4). These results indicate a need for the post-processing of local DSMs from high-resolution images to remove artifacts. After the removal of points with extreme error, the PHR and SPOT DSMs provided the highest accuracy and the lowest RMSEs. The PHR DSM had some elevation defects due to high-rise buildings, although the detection of these objects was successful. This DSM was followed by the ALOS, SRTM, and ASTER DSMs, respectively. Standard deviation metrics showed similar behavior to the RMSE and accuracy, indicating that the variations in error magnitude were directly proportional to the RMSE and accuracy.
The accuracies of the DSMs for different land cover types were evaluated by grouping the checkpoints according to their land cover source, then calculating the overall RMSE for each land cover type for all of the DSMs (Table 5). The evaluation results showed that the SPOT DSM provided closer RMSE values for all classes and the best accuracies on average. The PHR DSM provided similar results to the SPOT DSM (excluding the high-rise buildings) and ranked second in terms of average accuracy. The ALOS DSM provided very promising results by achieving similar accuracies to the local DSMs for four different land cover types; however, the accuracies in forests and over high-rise buildings were significantly lower, resulting in a third-place ranking on average. The SRTM and ASTER DSMs provided similar accuracies, with a slightly lower RMSE in the SRTM DSM on average. A table is produced according to the RMSE values, which ranks the DSMs based on land cover type (Table 6).

4.2. Accuracy Assessment by Profile

For the second part of the accuracy assessment, five different 1000 m length profile portions were evaluated, the directions of which are defined in Figure 4. For the evaluation, profiles from each DSM were overlaid on their respective reference profiles to detect the differences visually. Land cover type and distance are presented along the horizontal axis, and elevation is presented along the vertical axis.
According to Profile 3a, from the residential area review, although the ALOS, ASTER, and SRTM DSMs were close to the reference DSM, their spatial resolution was not enough to detect the individual buildings. The SPOT and PHR DSMs successfully captured the buildings; however, the main problem with those DSMs was the changing elevation values that were observed along the same buildings, specifically towards the edges and corners. Therefore, further post-processing is needed for high-resolution DSMs, specifically for man-made object elevations (Figure A1).
According to Profile 3b, there was an 80 m elevation change in the second profile route. The ASTER DSM provided the worst result for this profile, with a 10 m shift compared to the reference DSM. There was an important height increase with respect to the reference in the 2630th meter of the SPOT DSM profile that most likely corresponded to new building construction. This change was not available in the PHR DSM (Figure A2).
According to the route in Profile 3c, the PHR and SPOT DSMs were able to detect high buildings. In addition, the SPOT and PHR DSMs provided similar results as compared to the reference data in forest patches. The ALOS, ASTER, and SRTM DSMs failed to represent buildings in the 3200th and 3650th meters, which was most likely due to differences in image acquisition and building construction dates (Figure A3).
In the Profile 4a route, a sudden and similar altitude change was detected in all of the DSMs. The height change between 2340 m and 2440 m could not be detected with the ASTER DSM or the SRTM DSM. Moreover, buildings that were observable in the 2750 m to 3000 m interval of the profile could only be determined by the PHR and SPOT DSMs (Figure A4).
In Profile 4b, the accuracy differences of the DSMs in the mountainous, dense forest area were obvious. Errors in this area were relatively high for the SRTM and ASTER DSMs. It turns out that the SPOT and PHR DSMs were extremely accurate in this area, followed by the ALOS DSM. Differences in the ASTER DSM reached 20 m, and errors were observable in a majority of the route. In the SRTM DSM, there was a soft transition between heights, which caused faults to increase in the top and pit areas (Figure A5).
As a general observation, the DSMs produced from the high-resolution satellite images provided comparatively higher accuracies according to several evaluation methods applied in this research. While the SPOT DSM ranked first, followed by the PHR DSM in complex land cover, the PHR DSM provided the highest accuracy in bare terrain conditions. Local DSMs, such as the SPOT and PHR DSMs mentioned in this research, provided up-to-date elevation information for the region of interest and were more successful in the detection of above-terrain objects by means of their high spatial resolutions. On the other hand, the production of these local DSMs required GCPs to ensure horizontal and vertical accuracies, and post-processing was needed to remove artifacts for better quality.
Concerning the freely available global DSMs, the ALOS DSM provided more than satisfactory results, with acceptable accuracy on bare lands, producing results that were similar to the local, high-resolution DSMs in four out of the seven land cover classes. It was ranked third in the average evaluation. The higher accuracy of the ALOS DSM over the SRTM and ASTER DSMs can be explained by the original 5 m spatial resolution that was down-sampled to 30 m for free distribution. Another factor influencing the higher accuracy of the ALOS DSM was that it was produced from the most recent dataset of these three DSMs. The SRTM and ASTER DSMs provided similar accuracies in most situations, while the SRTM DSM had slightly better accuracy in five out of the seven land cover classes when compared to the ASTER DSM. It should be noted that the freely available DSMs were produced from datasets that were acquired in a single period, and their accuracy is limited for areas that have been subject to significant land cover changes in recent years.

5. Conclusions

DSMs are very important data sources for several remote sensing and geospatial applications; therefore, it is important to analyze the accuracy of DSMs. This research provided comparative evaluation of DSMs in terms of relative vertical agreement using accuracy metrics. The results of this study illustrated higher accuracy values for the PHR and SPOT DSMs, which was coherent with the spatial resolution of the input dataset. In most cases, a high-resolution DSM improved vertical accuracy; however, there were several noisy effects in these DSMs, specifically at the borders and corners of man-made structures, which require further processing of high-resolution DSMs. The ALOS DSM produced very good results, specifically compared to other freely available DSMs. Although the ALOS DSM had a 30 m grid spacing, it could be deduced that this was due to the acquisition of strong signals from the original 5 m DSM, which was produced from the 2.5 m images. The accuracies of the DSMs varied with respect to different land cover categories. The DSMs produced better accuracy values for rare residential and road classes when the elevation differences were not considerable. While comparing the different DSMs, the source of the data became more important, especially for dynamic regions. It was not always possible to find different datasets that were obtained at similar times; therefore, it is important to find stable regions within the study region to obtain reliable evaluations from the different DSMs. Absolute accuracy metrics can be derived with the presence of highly accurate reference data such as LIDAR-based point clouds.

Acknowledgments

The authors acknowledge the support of the Istanbul Technical University—Center for Satellite Communications and Remote Sensing (ITU-CSCRS) for providing the PHR 1A and SPOT 6 tri-stereo satellite images and the HGK for providing the reference DSM and orthophotos. The authors also acknowledge the support of the anonymous reviewers who improved the article with their comments and suggestions.

Author Contributions

U.A. and E.S. conceived and designed the experiments; B.B. performed the analysis; U.A., E.S., and B.B. wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Appendix A provides the visuals and vertical cross-sections related to five different profiles derived from the DSMs.
Ijgi 07 00114 g0a1 550
Figure A1. Visuals and cross-sections of Profile 3a.
Figure A1. Visuals and cross-sections of Profile 3a.
Ijgi 07 00114 g0a1
Ijgi 07 00114 g0a2 550
Figure A2. Visuals and cross-sections of Profile 3b.
Figure A2. Visuals and cross-sections of Profile 3b.
Ijgi 07 00114 g0a2
Ijgi 07 00114 g0a3 550
Figure A3. Visuals and cross-sections of Profile 3c.
Figure A3. Visuals and cross-sections of Profile 3c.
Ijgi 07 00114 g0a3
Ijgi 07 00114 g0a4 550
Figure A4. Visuals and cross-sections of Profile 4a.
Figure A4. Visuals and cross-sections of Profile 4a.
Ijgi 07 00114 g0a4
Ijgi 07 00114 g0a5 550
Figure A5. Visuals and cross-sections of Profile 4b.
Figure A5. Visuals and cross-sections of Profile 4b.
Ijgi 07 00114 g0a5

References

  1. Amans, O.C.; Beiping, W.; Yevenyo, Z.Y. Assessing Vertical Accuracy of SRTM Ver 4.1 and ASTER GDEM Ver 2 Using Differential GPS Measurements—Case Study in Ondo State Nigeria. Int. J. Sci. Eng. Res. 2013, 4, 523–531. [Google Scholar]
  2. Martha, T.R.; Kerle, N.; Jetten, V.; van Westen, C.J.; Kumar, K.V. Characterising spectral, spatial and morphometric properties of landslides for semi-automatic detection using object-oriented methods. Geomorphology 2010, 116, 24–36. [Google Scholar] [CrossRef]
  3. Peralvo, M. Influence of DEM Interpolation Methods in Drainage Analysis. Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.4759&rep=rep1&type=pdf (accessed on 12 December 2017).
  4. Erasmi, S.; Rosenbauer, R.; Buchbach, R.; Busche, T.; Rutishauser, S. Evaluating the quality and accuracy of TanDEM-X digital elevation models at archaeological sites in the Cilician Plain, Turkey. Remote Sens. 2014, 6, 9475–9493. [Google Scholar] [CrossRef] [Green Version]
  5. Pope, A.; Murray, T.; Luckman, A. DEM quality assessment for quantification of glacier surface change. Ann. Glaciol. 2007, 46, 189–194. [Google Scholar] [CrossRef]
  6. Kobrick, M. On the toes of giants—How SRTM was born. Photogramm. Eng. Remote Sens. 2006, 72, 206–210. [Google Scholar]
  7. Frey, H.; Paul, F. On the suitability of the SRTM DEM and ASTER GDEM for the compilation of topographic parameters in glacier inventories. Int. J. Appl. Earth Obs. Geoinf. 2012, 18, 480–490. [Google Scholar] [CrossRef]
  8. ASPRS. American Society for Photogrammetry and Remote Sensing: Accuracy standards for large-scale maps. Photogramm. Eng. Remote Sens. 1990, 56, 1068–1070. [Google Scholar]
  9. Groeger, G.; Kolbe, T.H.; Czerwinski, A.; Nagel, C. Open GIS City Geography Markup Language (CityGML) Encoding Standard, version 1.0.0.; Technical Representative, OGC-08-007r1; Open Geospatial Consortium Inc.: Wayland, MA, USA, 2008. [Google Scholar]
  10. Rayburg, S.; Thoms, M.; Neave, M. A comparison of digital elevation models generated from different data sources. Geomorphology 2009, 106, 261–270. [Google Scholar] [CrossRef]
  11. Koch, A.; Lohmann, P. Quality assessment and validation of digital surface models derived from the shuttle radar topography mission (SRTM). In Proceedings of the ISPRS Congress, Amsterdam, The Netherlands, 16–23 July 2000; Volume 33. [Google Scholar]
  12. Habib, A.; Kim, E.M.; Morgan, M.F.; Couloigner, I. DEM Generation from High Resolution Satellite Imagery Using Parallel Projection Model. In Proceedings of the 20th ISPRS Congress, Istanbul, Turkey, 12–23 July 2004; pp. 393–398. [Google Scholar]
  13. Jacobsen, K. Digital surface models of city areas by very high resolution space Imagery. In Proceedings of the 1st EARSeL Workshop of the SIG Urban Remote Sensing, Berlin, Germany, 2–3 March 2006. [Google Scholar]
  14. Toutin, T. Generation of DSMs from SPOT-5 in-track HRS and across-track HRG stereo data using spatiotriangulation and autocalibration. ISPRS J. Photogramm. Remote Sens. 2006, 60, 170–181. [Google Scholar] [CrossRef]
  15. Toutin, T. Comparison of 3D physical and empirical models for generating DSMs from stereo HR images. Photogramm. Eng. Remote Sens. 2006, 72, 597–604. [Google Scholar] [CrossRef]
  16. Zhang, L.; Gruen, A. Multi-image matching for DSM generation from IKONOS imagery. ISPRS J. Photogramm. Remote Sens. 2006, 60, 195–211. [Google Scholar] [CrossRef]
  17. Buyuksalih, G.; Jacobsen, K. Digital surface models in build up areas based on very high resolution space images. In Proceedings of the ASPRS 2007 Annual Conference, Tampa, Florida, 7–11 May 2007. [Google Scholar]
  18. Alobeid, A.; Jacobsen, K. Automatic Generation of Digital Surface Models from IKONOS Stereo Imagery and Related Application. In Proceedings of the GORS 16th International Symposium, Damascus, Syria, 10–12 November 2018. [Google Scholar]
  19. D’Angelo, P.; Lehner, M.; Krauss, T.; Hoja, D.; Peter, R. Towards Automated DEM Generation from High Resolution Stereo Satellite Images. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2008, 37, 1137–1142. [Google Scholar]
  20. Crespi, M.; Capaldo, P.; Fratarcangeli, F.; Nascetti, A.; Pieralice, F. DSM Generation from Very High Optical and Radar Sensors: Problems and Potentialities along the Road from the 3D Geometric Modeling to the Surface Model. In Proceedings of the International Geoscience and Remote Sensing Symposium (IGARSS), Honolulu, HI, USA, 25–30 July 2010; pp. 3596–3599. [Google Scholar]
  21. Capaldo, P.; Crespi, M.; Fratarcangeli, F.; Nascetti, A.; Pieralice, F.; Agugiaro, G.; Poli, D.; Remondino, F. DSM Generation from Optical and SAR High Resolution Satellite Imagery: Methodology, Problems and Potentialities. In Proceedings of the International Geoscience and Remote Sensing Symposium (IGARSS), Munich, German, 22–27 July 2012; pp. 6936–6939. [Google Scholar]
  22. Gong, K.; Fritsch, D. A Detailed Study about Digital Surface Model Generation Using High Resolution Satellite Stereo Imagery. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2016, 3, 69–76. [Google Scholar] [CrossRef]
  23. Yu, M.; Huang, Y.; Xu, Q.; Guo, P.; Dai, Z. Application of virtual earth in 3D terrain modeling to visual analysis of large-scale geological disasters in mountainous areas. Environ. Earth. Sci. 2016, 75, 563. [Google Scholar] [CrossRef]
  24. Huang, Y.; Yu, M.; Xu, Q.; Sawada, K.; Moriguchi, S.; Yashima, A.; Liu, C.; Xue, L. InSAR-derived digital elevation models for terrain change analysis of earthquake-triggered flow-like landslides based on ALOS/PALSAR imagery. Environ. Earth. Sci. 2015, 73, 7661–7668. [Google Scholar] [CrossRef]
  25. Purinton, B.; Bookhagen, B. Validation of digital elevation models (DEMs) and comparison of geomorphic metrics on the southern Central Andean Plateau. Earth Surf. Dynam. 2017, 5, 211–237. [Google Scholar] [CrossRef]
  26. Merryman Boncori, J.P. Caveats Concerning the use of SRTM DEM Version 4.1 (CGIAR-CSI). Remote Sens. 2016, 8, 793. [Google Scholar] [CrossRef]
  27. Arefi, H.; Reinartz, P. Accuracy enhancement of ASTER global digital elevation models using ICESat data. Remote Sens. 2011, 3, 1323–1343. [Google Scholar] [CrossRef]
  28. Hu, Z.; Peng, J.; Hou, Y.; Shan, J. Evaluation of Recently, Released Open Global Digital Elevation Models of Hubei, China. Remote Sens. 2017, 9, 262. [Google Scholar] [CrossRef]
  29. Suwandana, E.; Kawamura, K.; Sakuno, Y.; Kustiyanto, E.; Raharjo, B. Evaluation of ASTER GDEM2 in Comparison with GDEM1, SRTM DEM and Topographic-Map-Derived DEM Using Inundation Area Analysis and RTK-dGPS Data. Remote Sens. 2012, 4, 2419–2431. [Google Scholar] [CrossRef]
  30. De Oliveira, C.G.; Paradella, W.R. An assessment of the altimetric information derived from spaceborne SAR (RADARSAT-1, SRTM3) and optical (ASTER) data for cartographic application in the Amazon region. Sensors 2008, 8, 3819–3829. [Google Scholar] [CrossRef] [PubMed]
  31. Rodriguez, E.; Morris, C.S.; Belz, J.E.; Chapin, E.C.; Martin, J.M.; Daffer, W.; Hensley, S. An Assessment of the SRTM Topographic Products. Validation Report by NASA and JPL. Available online: https://www2.jpl.nasa.gov/srtm/SRTM_D31639.pdf (accessed on 18 November 2017).
  32. Mukul, M.; Srivastava, V.; Mukul, M. Analysis of the accuracy of Shuttle Radar Topography Mission (SRTM) height models using International Global Navigation Satellite System Service (IGS) Network. J. Earth Syst. Sci. 2015, 124, 1343–1357. [Google Scholar] [CrossRef]
  33. Tadono, T.; Nagai, H.; Ishida, H.; Oda, F.; Naito, S.; Minakawa, K.; Iwamoto, H. Generation of the 30 m-mesh global digital surface model by ALOS PRISM. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2016, XLI-B4, 157–162. [Google Scholar] [CrossRef]
  34. Santillan, J.R.; Makinano-Santillan, M.; Makinano, R.M. Vertical accuracy assessment of ALOS World 3D-30M Digital Elevation Model over northeastern Mindanao, Philippines. In Proceedings of the Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 10–15 July 2016; pp. 5374–5377. [Google Scholar]
  35. Takaku, J.; Tadono, T.; Tsutsui, K.; Ichikawa, M. Validation of ‘AW3D’ global DSM generated from ALOS PRISM. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2016, 3, 25–31. [Google Scholar] [CrossRef]
  36. Durand, A.; Michel, J.; De Franchis, C.; Allenbach, B.; Giros, A. Qualitative assessment of four DSM generation approaches using Pléiades-HR data. In Proceedings of the 33th EARSeL Symposium, Matera, Italy, 3–6 June 2013. [Google Scholar]
  37. SPOT 6 & SPOT 7 Technical Sheet. Available online: http://www.intelligence-airbusds.com/files/pmedia/public/r12317_9_spot6-7_technical_sheet.pdf (accessed on 21 November 2017).
  38. Cam, A.; Firat, O.; Yilmaz, A. The orthophoto and digital surface model production activities in the General Command of Mapping. In Proceedings of the TMMOB Geographical Information Systems Congress, Ankara, Turkey, 11–13 November 2013. [Google Scholar]
  39. Yilmaz, A.; Alp, O.; Okul, A.; Eker, O.; Erdogan, M. Türkiye için hassas sayısal yukseklik modeli üretimi. In Proceedings of the TUFUAB 3 Technical Symposium, Konya, Turkey, 21–23 May 2015. [Google Scholar]
Ijgi 07 00114 g001 550
Figure 1. Geographic representation of the study area with a Pleiades high-resolution (PHR) image and a general overview of the region from Google Earth©.
Figure 1. Geographic representation of the study area with a Pleiades high-resolution (PHR) image and a general overview of the region from Google Earth©.
Ijgi 07 00114 g001
Ijgi 07 00114 g002 550
Figure 2. The gray levels represent the DSM dataset and the reference DSM data.
Figure 2. The gray levels represent the DSM dataset and the reference DSM data.
Ijgi 07 00114 g002
Ijgi 07 00114 g003 550
Figure 3. (a) Land cover map of the study area; (b) reference elevation map from HGK data and the locations of checkpoints.
Figure 3. (a) Land cover map of the study area; (b) reference elevation map from HGK data and the locations of checkpoints.
Ijgi 07 00114 g003
Ijgi 07 00114 g004 550
Figure 4. The height difference distribution of the DSMs derived from 1000 checkpoints.
Figure 4. The height difference distribution of the DSMs derived from 1000 checkpoints.
Ijgi 07 00114 g004
Ijgi 07 00114 g005 550
Figure 5. Positions of the vertical profiles. (1) North/South; (2) Northeast/Southwest; (3) East/West; (4) Southeast/Northwest.
Figure 5. Positions of the vertical profiles. (1) North/South; (2) Northeast/Southwest; (3) East/West; (4) Southeast/Northwest.
Ijgi 07 00114 g005
Table
Table 1. Summary of previous studies on the production/accuracy assessment of digital surface models (DSMs).
Table 1. Summary of previous studies on the production/accuracy assessment of digital surface models (DSMs).
Author NameDateRegionData SourceGeneration Method
Habib, A., et al.2004Korea, BelgiumSPOT-5 HRSParallel projection model
Jacobsen, K.2006Maras and Zonguldak, Turkey; Phoenix, United StatesIKONOS, QuickBird and OrbView-3Automatic image matching
Toutin, T.2006North of Québec City, Québec, Canada SPOT-5 in-track HRS and across-track HRGArea-based multiscale image matching method
Toutin, T.2006North of Québec City, Québec, CanadaIKONOS, QuickBirdPhysical and empirical models
Zhang, L., and Gruen, A.2006Thun, SwitzerlandIKONOSMulti-image matching
Büyüksalih, G., and Jacobsen, K.2007Maras and Zonguldak, Turkey; Phoenix, United StatesIKONOS, QuickBird, OrbView-3, Cartosat-1Automatic image matching
Alobeid, A., and Jacobsen, K.2008Maras and Istanbul in TurkeyIKONOSAutomatic image matching
d’Angelo, P., et al.2008Catalonia, SpainCartosat-1Towards automated digital elevation model (DEM) generation
Crespi, M., et al.2010Rome and Merano, ItalyGeoeye-1 and Cosmo-SkyMedRigorous model and RPC model
Capaldo, P., et al.2012Trento, ItalyGeoEye-1 and TerraSAR-XRPC models for optical, radargrammetry for synthetic aperture radar (SAR)
Gong, K., and D. Fritsch2016Munich, GermanyWorldView-2Bias-compensated RPC bundle block-adjusted Epipolar images generation, dense image matching, and DSM generation
Yu, M., et al.2016Guangyuan City, ChinaGoogle Earth (GE)Terrain extraction from GE
Huang, Y., et al.2015Guangyuan City, ChinaAdvanced Land Observing Satellite (ALOS)/PALSARDEM extraction with InSAR technique
Purinton, B. and Bookhagen, B.2017Central Andean Plateu, ArgentinaAdvanced Spaceborne Thermal Emission and Reflection Radiometer Global DEM version 2 (ASTER GDEM v.2), Shuttle Radar Topography Mission (SRTM-C), TerrasarX, ALOS World 3D (ALOS W3D)Vertical accuracy by dGPS and morpometric comp
Table
Table 2. Acquisition properties of the PHR 1A and SPOT 6 satellites’ tri-stereo images.
Table 2. Acquisition properties of the PHR 1A and SPOT 6 satellites’ tri-stereo images.
PHR1ASPOT6
Acquisition DateIncidence Angles (°)Acquisition DateIncidence Angles (°)
28 August 201519.1925 April 201719.30
14.202.74
23.1415.06
Table
Table 3. Accuracy metrics derived from the 25 checkpoints under bare terrain conditions (m).
Table 3. Accuracy metrics derived from the 25 checkpoints under bare terrain conditions (m).
Accuracy MetricsPHR DSMALOSSPOT DSMSRTMASTER
Root mean square error (RMSE)1.572.142.263.535.72
Accuracy3.084.194.436.9211.21
SD1.051.411.482.203.32
Table
Table 4. Accuracy metrics derived from 1000 randomly selected points after extreme removal (m).
Table 4. Accuracy metrics derived from 1000 randomly selected points after extreme removal (m).
Accuracy MetricsSPOT DSMPHR DSMALOSSRTMASTER
RMSE4.235.095.916.496.92
Accuracy8.299.9711.5812.7213.56
SD3.173.464.494.574.68
Table
Table 5. RMSE of the DSMs with respect to land cover classes (m).
Table 5. RMSE of the DSMs with respect to land cover classes (m).
DSM TypeForestIndustryRare ResidentialResidentialRoadsHigh BuildingAverage
SPOT DSM4.194.163.723.024.211.404.23
PHR DSM4.814.364.523.783.487.235.09
ALOS7.444.503.183.603.777.125.91
SRTM7.548.813.433.814.367.436.49
ASTER8.287.415.134.165.336.536.92
Table
Table 6. Accuracy ranking of the DSMs for each land cover class (sorted by increasing RMSE values).
Table 6. Accuracy ranking of the DSMs for each land cover class (sorted by increasing RMSE values).
ForestIndustryRare ResidentialResidentialRoadsHigh BuildingAverage
SPOT DSMSPOT DSMALOSSPOT DSMPHR DSMSPOT DSMSPOT DSM
PHR DSMPHR DSMSRTMALOSALOSASTERPHR DSM
ALOSALOSSPOT DSMPHR DSMSPOT DSMALOSALOS
SRTMASTERPHR DSMSRTMSRTMPHR DSMSRTM
ASTERSRTMASTERASTERASTERSRTMASTER

Share and Cite

MDPI and ACS Style

Alganci, U.; Besol, B.; Sertel, E. Accuracy Assessment of Different Digital Surface Models. ISPRS Int. J. Geo-Inf. 2018, 7, 114. https://doi.org/10.3390/ijgi7030114

AMA Style

Alganci U, Besol B, Sertel E. Accuracy Assessment of Different Digital Surface Models. ISPRS International Journal of Geo-Information. 2018; 7(3):114. https://doi.org/10.3390/ijgi7030114

Chicago/Turabian Style

Alganci, Ugur, Baris Besol, and Elif Sertel. 2018. "Accuracy Assessment of Different Digital Surface Models" ISPRS International Journal of Geo-Information 7, no. 3: 114. https://doi.org/10.3390/ijgi7030114

APA Style

Alganci, U., Besol, B., & Sertel, E. (2018). Accuracy Assessment of Different Digital Surface Models. ISPRS International Journal of Geo-Information, 7(3), 114. https://doi.org/10.3390/ijgi7030114

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop