Abstract
Semantic segmentation is fundamental for enabling scene understanding in several robotics applications, such as aerial delivery and autonomous driving. While scenarios in autonomous driving mainly comprise roads and small viewpoint changes, imagery acquired from aerial platforms is usually characterized by extreme variations in viewpoint. In this paper, we focus on aerial delivery use cases, in which a drone visits the same places repeatedly from distinct viewpoints. Although such applications are already under investigation (e.g. transport of blood between hospitals), current approaches depend heavily on ground personnel assistance to ensure safe delivery. Aiming at enabling safer and more autonomous operation, in this work, we propose a novel deep-learning-based semantic segmentation approach capable of running on small aerial vehicles, as well as a practical dataset-capturing method and a network-training strategy that enables greater viewpoint tolerance in such scenarios. Our experiments show that the proposed method greatly outperforms a state-of-the-art network for embedded computers while maintaining similar inference speed and memory consumption. In addition, it achieves slightly better accuracy compared to a much larger and slower state-of-the-art network, which is unsuitable for small aerial vehicles, as considered in this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., Kalinin, A.A.: Albumentations: fast and flexible image augmentations. Information (2020)
Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017)
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)
Cheng, B., et al.: Panoptic-deeplab: a simple, strong, and fast baseline for bottom-up panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)
Contributors, M.: MMSegmentation: openmmlab semantic segmentation toolbox and benchmark (2020). https://github.com/open-mmlab/mmsegmentation
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2016)
DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2018)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The kitti vision benchmark suite. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2012)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2016)
Howard, A., et al.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2019)
Howard, A.G., et al.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2018)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2015)
Lyu, Y., Vosselman, G., Xia, G.S., Yilmaz, A., Yang, M.Y.: Uavid: a semantic segmentation dataset for uav imagery. ISPRS J. Photogramm. Remote Sens. (2020)
Marmanis, D., Schindler, K., Wegner, J.D., Galliani, S., Datcu, M., Stilla, U.: Classification with an edge: improving semantic image segmentation with boundary detection. ISPRS J. Photogramm. Remote Sens. (2018)
Mottaghi, R., et al.: The role of context for object detection and semantic segmentation in the wild. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2014)
Mou, L., Hua, Y., Zhu, X.X.: A relation-augmented fully convolutional network for semantic segmentation in aerial scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates Inc. (2019)
Poudel, R.P., Liwicki, S., Cipolla, R.: Fast-scnn: fast semantic segmentation network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention (2015)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2018)
Sarlin, P.E., Cadena, C., Siegwart, R., Dymczyk, M.: From coarse to fine: robust hierarchical localization at large scale. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2019)
Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2015)
Zhang, E.: Fast semantic segmentation (2020). https://github.com/ekzhang/fastseg
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2017)
Zhu, Y., et al.: Improving semantic segmentation via video propagation and label relaxation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)
Acknowledgments
This work was supported by the Swiss National Science Foundation (SNSF, NCCR Robotics, NCCR Digital Fabrication), the Amazon Research Awards and IDEA League Student Grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, S., Maffra, F., Mascaro, R., Teixeira, L., Chli, M. (2021). Viewpoint-Tolerant Semantic Segmentation for Aerial Logistics. In: Bauckhage, C., Gall, J., Schwing, A. (eds) Pattern Recognition. DAGM GCPR 2021. Lecture Notes in Computer Science(), vol 13024. Springer, Cham. https://doi.org/10.1007/978-3-030-92659-5_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-92659-5_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92658-8
Online ISBN: 978-3-030-92659-5
eBook Packages: Computer ScienceComputer Science (R0)