[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3615886.3627747acmconferencesArticle/Chapter ViewAbstractPublication PagesgisConference Proceedingsconference-collections
research-article

Assessment of a new GeoAI foundation model for flood inundation mapping

Published: 20 November 2023 Publication History

Abstract

Vision foundation models are a new frontier in Geospatial Artificial Intelligence (GeoAI), an interdisciplinary research area that applies and extends AI for geospatial problem solving and geographic knowledge discovery, because of their potential to enable powerful image analysis by learning and extracting important image features from vast amounts of geospatial data. This paper evaluates the performance of the first-of-its-kind geospatial foundation model, IBM-NASA's Prithvi, to support a crucial geospatial analysis task: flood inundation mapping. This model is compared with convolutional neural network and vision transformer-based architectures in terms of mapping accuracy for flooded areas. A benchmark dataset, Sen1Floods11, is used in the experiments, and the models' predictability, generalizability, and transferability are evaluated based on both a test dataset and a dataset that is completely unseen by the model. Results show the good transferability of the Prithvi model, highlighting its performance advantages in segmenting flooded areas in previously unseen regions. The findings also indicate areas for improvement for the Prithvi model in terms of adopting multi-scale representation learning, developing more end-to-end pipelines for high-level image analysis tasks, and offering more flexibility in terms of input data bands.

References

[1]
Luc Anselin, Sergio J Rey, and Wenwen Li. 2014. Metadata and provenance for spatial analysis: the case of spatial weights. International Journal of Geographical Information Science 28, 11 (2014), 2261--2280.
[2]
Samantha T Arundel, Wenwen Li, and Sizhe Wang. 2020. GeoNat v1. 0: A dataset for natural feature mapping with artificial intelligence and supervised learning. Transactions in GIS 24, 3 (2020), 556--572.
[3]
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. 2017. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 12 (2017), 2481--2495.
[4]
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021).
[5]
Derrick Bonafilia, Beth Tellman, Tyler Anderson, and Erica Issenberg. 2020. Sen1Floods11: A georeferenced dataset to train and test deep learning flood algorithms for sentinel-1. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 210--211.
[6]
Keumgang Cha, Junghoon Seo, and Taekyung Lee. 2023. A billion-scale foundation model for remote sensing images. arXiv preprint arXiv:2304.05215 (2023).
[7]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 4 (2017), 834--848.
[8]
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. 2022. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1290--1299.
[9]
MMSegmentation Contributors. 2020. MMSegmentation: OpenMMLab Semantic Segmentation Toolbox and Benchmark. https://github.com/open-mmlab/mmsegmentation.
[10]
Foivos I Diakogiannis, François Waldner, Peter Caccetta, and Chen Wu. 2020. ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing 162 (2020), 94--114.
[11]
Michael F Goodchild and Wenwen Li. 2021. Replication across space and time must be weak in the social and environmental sciences. Proceedings of the National Academy of Sciences 118, 35 (2021), e2015759118.
[12]
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16000--16009.
[13]
Chia-Yu Hsu and Wenwen Li. 2023. Explainable GeoAI: can saliency maps help interpret artificial intelligence's learning process? An empirical study on natural feature detection. International Journal of Geographical Information Science 37, 5 (2023), 963--987.
[14]
Chia-Yu Hsu, Wenwen Li, and Sizhe Wang. 2021. Knowledge-driven GeoAI: Integrating spatial knowledge into multi-scale deep learning for Mars Crater detection. Remote Sensing 13, 11 (2021), 2116.
[15]
Yingjie Hu, Song Gao, Dalton Lunga, Wenwen Li, Shawn Newsam, and Budhendra Bhaduri. 2019. GeoAI at ACM SIGSPATIAL: progress, challenges, and future directions. Sigspatial Special 11, 2 (2019), 5--15.
[16]
Johannes Jakubik, Linsong Chu, Paolo Fraccaro, Carlos Gomes, Gabby Nyirjesy, Ranjini Bangalore, Devyani Lambhate, Kamal Das, Dario Oliveira Borges, Daiki Kimura, Naomi Simumba, Daniela Szwarcman, Michal Muszynski, Kommy Weldemariam, Bianca Zadrozny, Raghu Ganti, Carlos Costa, Campbell Edwards, Blair & Watson, Karthik Mukkavilli, Hendrik Schmude, Johannes & Hamann, Parkin Robert, Sujit Roy, Christopher Phillips, Kumar Ankur, Muthukumaran Ramasubramanian, Iksha Gurung, Wei Ji Leong, Ryan Avery, Rahul Ramachandran, Manil Maskey, Pontus Olofossen, Elizabeth Fancher, Tsengdar Lee, Kevin Murphy, Dan Duffy, Mike Little, Hamed Alemohammad, Michael Cecil, Steve Li, Sam Khallaghi, Denys Godwin, Maryam Ahmadi, Fatemeh Kordi, Bertrand Saux, Neal Pastick, Peter Doucette, Rylie Fleckenstein, Dalton Luanga, Alex Corvin, and Erwan Granger. 2023. Prithvi-100M. https://doi.org/10.57967/hf/0952
[17]
Peter Kedron, Wenwen Li, Stewart Fotheringham, and Michael Goodchild. 2021. Reproducibility and replicability: Opportunities and challenges for geospatial research. International Journal of Geographical Information Science 35, 3 (2021), 427--445.
[18]
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. Segment anything. arXiv preprint arXiv:2304.02643 (2023).
[19]
Goutam Konapala, Sujay V Kumar, and Shahryar Khalique Ahmad. 2021. Exploring Sentinel-1 and Sentinel-2 diversity for flood inundation mapping using deep learning. ISPRS Journal of Photogrammetry and Remote Sensing 180 (2021), 163--173.
[20]
Wenwen Li. 2020. GeoAI: Where machine learning and big data converge in GIScience. Journal of Spatial Information Science 20 (2020), 71--77.
[21]
Wenwen Li, Michael Batty, and Michael F Goodchild. 2020. Real-time GIS for smart cities. International Journal of Geographical Information Science 34, 2 (2020), 311--324.
[22]
Wenwen Li and Chia-Yu Hsu. 2022. GeoAI for large-scale image analysis and machine vision: Recent progress of artificial intelligence in geography. ISPRS International Journal of Geo-Information 11, 7 (2022), 385.
[23]
Wenwen Li, Chia-Yu Hsu, and Maosheng Hu. 2021. Tobler's First Law in GeoAI: A spatially explicit deep learning model for terrain feature detection under weak supervision. Annals of the American Association of Geographers 111, 7 (2021), 1887--1905.
[24]
Wenwen Li, Chia-Yu Hsu, Sizhe Wang, Chandi Witharana, and Anna Liljedahl. 2022. Real-time GeoAI for high-resolution mapping and segmentation of arctic permafrost features: the case of ice-wedge polygons. In Proceedings of the 5th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery. 62--65.
[25]
Wenwen Li, Miaomiao Song, Bin Zhou, Kai Cao, and Song Gao. 2015. Performance improvement techniques for geospatial web services in a cyberinfrastructure environment--A case study with a disaster management portal. Computers, Environment and Urban Systems 54 (2015), 314--325.
[26]
Wenwen Li, Sizhe Wang, Samantha T Arundel, and Chia-Yu Hsu. 2023. GeoImageNet: a multi-source natural feature benchmark dataset for GeoAI and supervised machine learning. GeoInformatica 27, 3 (2023), 619--640.
[27]
Wenwen Li, Sizhe Wang, Xiao Chen, Yuanyuan Tian, Zhining Gu, Anna Lopez-Carr, Andrew Schroeder, Kitty Currier, Mark Schildhauer, and Rui Zhu. 2023. Geographvis: a knowledge graph and geovisualization empowered cyberinfrastructure to support disaster response and humanitarian aid. ISPRS International Journal of Geo-Information 12, 3 (2023), 112.
[28]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10012--10022.
[29]
Yang Long, Gui-Song Xia, Shengyang Li, Wen Yang, Michael Ying Yang, Xiao Xiang Zhu, Liangpei Zhang, and Deren Li. 2021. On creating benchmark dataset for aerial image interpretation: Reviews, guidances, and million-aid. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 14 (2021), 4205--4230.
[30]
Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017).
[31]
Thomas R Loveland and James R Irons. 2016. Landsat 8: The plans, the reality, and the legacy. Remote Sensing of Environment 185 (2016), 1--6.
[32]
Jun Rentschler, Melda Salhab, and Bramka Arga Jafino. 2022. Flood risk already affects 1.81 billion people. Climate change and unplanned urbanization could worsen exposure. http://web.archive.org/web/20080207010024/http://www.808multimedia.com/winnt/kernel.htm. Accessed: 2023-09-01.
[33]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention--MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, 234--241.
[34]
Hans Thisanke, Chamli Deshan, Kavindu Chamith, Sachith Seneviratne, Rajith Vidanaarachchi, and Damayanthi Herath. 2023. Semantic segmentation using Vision Transformers: A survey. Engineering Applications of Artificial Intelligence 126 (2023), 106669.
[35]
Sizhe Wang and Wenwen Li. 2021. GeoAI in terrain analysis: Enabling multi-source deep learning and data fusion for natural feature detection. Computers, Environment and Urban Systems 90 (2021), 101715.
[36]
John P Wilson, Kevin Butler, Song Gao, Yingjie Hu, Wenwen Li, and Dawn J Wright. 2021. A five-star guide for achieving replicability and reproducibility when working with GIS software and algorithms. Annals of the American Association of Geographers 111, 5 (2021), 1311--1317.
[37]
Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. 2021. SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems 34 (2021), 12077--12090.
[38]
Ritu Yadav, Andrea Nascetti, and Yifang Ban. 2022. Deep attentive fusion network for flood detection on uni-temporal Sentinel-1 data. Frontiers in Remote Sensing 3 (2022), 1060144.
[39]
Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, et al. 2021. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6881--6890.
[40]
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2017. Scene parsing through ade20k dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 633--641.

Cited By

View all
  • (2024)Urban Resilience Index for Critical Infrastructure: A Scenario-Based Approach to Disaster Risk Reduction in Road NetworksSustainability10.3390/su1610414316:10(4143)Online publication date: 15-May-2024
  • (2024)Advancing Arctic Sea Ice Remote Sensing with AI and Deep Learning: Opportunities and ChallengesRemote Sensing10.3390/rs1620376416:20(3764)Online publication date: 10-Oct-2024
  • (2024)Mapping Geospatial AI Flood Risk in National Road NetworksISPRS International Journal of Geo-Information10.3390/ijgi1309032313:9(323)Online publication date: 7-Sep-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
GeoAI '23: Proceedings of the 6th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery
November 2023
135 pages
ISBN:9798400703485
DOI:10.1145/3615886
Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 November 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Artificial Intelligence
  2. GeoAI
  3. Segformer
  4. U-Net
  5. semantic segmentation

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

SIGSPATIAL '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 17 of 25 submissions, 68%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)227
  • Downloads (Last 6 weeks)29
Reflects downloads up to 12 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Urban Resilience Index for Critical Infrastructure: A Scenario-Based Approach to Disaster Risk Reduction in Road NetworksSustainability10.3390/su1610414316:10(4143)Online publication date: 15-May-2024
  • (2024)Advancing Arctic Sea Ice Remote Sensing with AI and Deep Learning: Opportunities and ChallengesRemote Sensing10.3390/rs1620376416:20(3764)Online publication date: 10-Oct-2024
  • (2024)Mapping Geospatial AI Flood Risk in National Road NetworksISPRS International Journal of Geo-Information10.3390/ijgi1309032313:9(323)Online publication date: 7-Sep-2024
  • (2024)Mountain Streambed Roughness and Flood Extent Estimation from Imagery Using the Segment Anything Model (SAM)Hydrology10.3390/hydrology1102001711:2(17)Online publication date: 31-Jan-2024
  • (2024)Large language models can help boost food production, but be mindful of their risksFrontiers in Artificial Intelligence10.3389/frai.2024.13261537Online publication date: 25-Oct-2024
  • (2024)Potential Application of Bayesian Changepoint Detection for Near-Real-Time Flood MonitoringIGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium10.1109/IGARSS53475.2024.10641333(3749-3751)Online publication date: 7-Jul-2024
  • (2024)UrbanSARFloods: Sentinel-1 SLC-Based Benchmark Dataset for Urban and Open-Area Flood Mapping2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00047(419-429)Online publication date: 17-Jun-2024
  • (2024)GeoAI Reproducibility and Replicability: A Computational and Spatial PerspectiveAnnals of the American Association of Geographers10.1080/24694452.2024.2373787114:9(2085-2103)Online publication date: 23-Jul-2024
  • (2024)MapGPT: an autonomous framework for mapping by integrating large language model and cartographic toolsCartography and Geographic Information Science10.1080/15230406.2024.240486851:6(717-743)Online publication date: 3-Oct-2024
  • (2024)Detecting floating litter in freshwater bodies with semi-supervised deep learningWater Research10.1016/j.watres.2024.122405266(122405)Online publication date: Nov-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media