research-article

Open access

Improving Building Segmentation Using Uncertainty Modeling and Metadata Injection

Authors:

Edward J. DelpAuthors Info & Claims

SIGSPATIAL '21: Proceedings of the 29th International Conference on Advances in Geographic Information Systems

Pages 117 - 120

https://doi.org/10.1145/3474717.3483918

Published: 04 November 2021 Publication History

PDF eReader

Abstract

Automatic building segmentation is an important task for satellite imagery analysis and scene understanding. Most existing segmentation methods focus on the case where the images are taken from directly overhead (i.e., low off-nadir/viewing angle). These methods often fail to provide accurate results on satellite images with larger off-nadir angles due to the higher noise level and lower spatial resolution. In this paper, we propose a method that is able to provide accurate building segmentation for satellite imagery captured from a large range of off-nadir angles. Based on Bayesian deep learning, we explicitly design our method to learn the data noise via aleatoric and epistemic uncertainty modeling. Satellite image metadata (e.g., off-nadir angle and ground sample distance) is also used in our model to further improve the result. We show that with uncertainty modeling and metadata injection, our method achieves better performance than the baseline method, especially for noisy images taken from large off-nadir angles1.

References

[1]

Mang Tik Chiu, Xingqian Xu, Yunchao Wei, Zilong Huang, Alexander G. Schwing, Robert Brunner, Hrant Khachatrian, Hovnatan Karapetyan, Ivan Dozier, Greg Rose, David Wilson, Adrian Tudor, Naira Hovakimyan, Thomas S. Huang, and Honghui Shi. 2020. Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis. IEEE/CVF Conference on Computer Vision and Pattern Recognition (June 2020), 2825--2835. https://doi.org/10.1109/CVPR42600.2020.00290 Seoul, Korea.

Crossref

Google Scholar

[2]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. IEEE/CVF Conference on Computer Vision and Pattern Recognition (June 2009), 248--255. https://doi.org/10.1109/CVPR.2009.5206848 Miami, FL.

Crossref

Google Scholar

[3]

Yarin Gal and Zoubin Ghahramani. 2016. Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference. International Conference on Machine Learning (May 2016). https://arxiv.org/abs/1506.02158 San Juan, Puerto Rico.

Google Scholar

[4]

Ritwik Gupta, Bryce Goodman, Nirav Patel, Ricky Hosfelt, Sandra Sajeev, Eric Heim, Jigar Doshi, Keane Lucas, Howie Choset, and Matthew Gaston. 2019. Creating xBD: A Dataset for Assessing Building Damage from Satellite Imagery. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (June 2019). https://arxiv.org/abs/1911.09296 Long Beach, California.

Google Scholar

[5]

Hanxiang Hao, Sriram Baireddy, Emily Bartusiak, Mridul Gupta, Kevin LaTourette, Latisha Konz, Moses Chan, Mary L. Comer, and Edward J. Delp. 2021. Building Height Estimation via Satellite Metadata and Shadow Instance Detection. Automatic Target Recognition XXXI 11729 (April 2021), 175--190. https://doi.org/10.1117/12.2585012

Crossref

Google Scholar

[6]

Hanxiang Hao, Sriram Baireddy, Kevin LaTourette, Latisha Konz, Moses Chan, Mary L. Comer, and Edward J. Delp. 2021. Improving Building Segmentation for Off-Nadir Satellite Imagery. arXiv 2109.03961 (September 2021). https://arxiv.org/abs/2109.03961

Google Scholar

[7]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. IEEE/CVF Conference on Computer Vision and Pattern Recognition (June 2016), 770--778. https://doi.org/10.1109/CVPR.2016.90 Las Vegas, NV.

Crossref

Google Scholar

[8]

Alex Kendall and Yarin Gal. 2017. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? Conference on Neural Information Processing Systems 30 (December 2017). https://arxiv.org/abs/1703.04977 Long Beach, CA.

Google Scholar

[9]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. International Conference on Learning Representations (May 2015). http://arxiv.org/abs/1412.6980 San Diego, CA.

Google Scholar

[10]

Bowen Li, Xiaojuan Qi, Thomas Lukasiewicz, and Philip H.S. Torr. 2020. ManiGAN: Text-Guided Image Manipulation. IEEE/CVF Conference on Computer Vision and Pattern Recognition 30 (June 2020), 7877--7886. https://doi.org/10.1109/CVPR42600.2020.00790 Seattle, WA.

Crossref

Google Scholar

[11]

Gregoris Liasis and Stavros Stavrou. 2016. Satellite images analysis for shadow detection and building height estimation. ISPRS Journal of Photogrammetry and Remote Sensing 19 (2016), 437--450. https://doi.org/10.1016/j.isprsjprs.2016.07.006

Crossref

Google Scholar

[12]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention 9351 (June 2015), 234--241. https://arxiv.org/abs/1505.04597

Google Scholar

[13]

Alexey Trekin, Vladimir Ignatiev, and Pavel Yakubovskiy. 2020. Deep Neural Networks for Determining the Parameters of Buildings from Single-Shot Satellite Imagery. Computer and Systems Sciences Internationa 59 (2020), 755--767. https://doi.org/10.1134/S106423072005007X

Digital Library

Google Scholar

[14]

Nicholas Weir, David Lindenbaum, Alexei Bastidas, Adam Etten, Varun Kumar, Sean Mcpherson, Jacob Shermeyer, and Hanlin Tang. 2019. SpaceNet MVOI: A Multi-View Overhead Imagery Dataset. IEEE/CVF International Conference on Computer Vision (October 2019), 992--1001. https://doi.org/10.1109/ICCV.2019.00108 Seoul, Korea.

Crossref

Google Scholar

Cited By

View all

Ulku IAkagunduz EGhamisi P(2022)Deep Semantic Segmentation of Trees Using Multispectral ImagesIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2022.320314515(7589-7604)Online publication date: 2022
https://doi.org/10.1109/JSTARS.2022.3203145

Index Terms

Improving Building Segmentation Using Uncertainty Modeling and Metadata Injection
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation

Recommendations

A high-resolution panchromatic-multispectral satellite image fusion method assisted with building segmentation
Abstract
The main difficulty of panchromatic-multispectral image fusion is to balance the quality of spatial information and the spectral fidelity. Most of the practical fusion methods determine the optimal parameters based on the spatial and ...
Highlights
- Satellite image fusion assisted with deep learning building segmentation techniques.
Modeling Positional Uncertainty Acquired Through Street Geocoding

This article describes how modeling positional uncertainty helps to understand potential factors of uncertainty, and to identify impacts of uncertainty on spatial analysis results. However, modeling geocoding positional uncertainty still is limited in ...
Leveraging Uncertainty Estimates to Improve Segmentation Performance in Cardiac MR
Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis
Abstract
In medical image segmentation, several studies have used Bayesian neural networks to segment and quantify the uncertainty of the images. These studies show that there might be an increased epistemic uncertainty in areas where there are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGSPATIAL '21: Proceedings of the 29th International Conference on Advances in Geographic Information Systems

November 2021

700 pages

ISBN:9781450386647

DOI:10.1145/3474717

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 November 2021

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SIGSPATIAL '21

Sponsor:

SIGSPATIAL

SIGSPATIAL '21: 29th International Conference on Advances in Geographic Information Systems

November 2 - 5, 2021

Beijing, China

Acceptance Rates

Overall Acceptance Rate 257 of 1,238 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
329
Total Downloads

Downloads (Last 12 months)81
Downloads (Last 6 weeks)14

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ulku IAkagunduz EGhamisi P(2022)Deep Semantic Segmentation of Trees Using Multispectral ImagesIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2022.320314515(7589-7604)Online publication date: 2022
https://doi.org/10.1109/JSTARS.2022.3203145

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A high-resolution panchromatic-multispectral satellite image fusion method assisted with building segmentation

Modeling Positional Uncertainty Acquired Through Street Geocoding

Leveraging Uncertainty Estimates to Improve Segmentation Performance in Cardiac MR