[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

Improving Contrastive Learning on Visually Homogeneous Mars Rover Images

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Abstract

Contrastive learning has recently demonstrated superior performance to supervised learning, despite requiring no training labels. We explore how contrastive learning can be applied to hundreds of thousands of unlabeled Mars terrain images, collected from the Mars rovers Curiosity and Perseverance, and from the Mars Reconnaissance Orbiter. Such methods are appealing since the vast majority of Mars images are unlabeled as manual annotation is labor intensive and requires extensive domain knowledge. Contrastive learning, however, assumes that any given pair of distinct images contain distinct semantic content. This is an issue for Mars image datasets, as any two pairs of Mars images are far more likely to be semantically similar due to the lack of visual diversity on the planet’s surface. Making the assumption that pairs of images will be in visual contrast — when they are in fact not — results in pairs that are falsely considered as negatives, impacting training performance. In this study, we propose two approaches to resolve this: 1) an unsupervised deep clustering step on the Mars datasets, which identifies clusters of images containing similar semantic content and corrects false negative errors during training, and 2) a simple approach which mixes data from different domains to increase visual diversity of the total training dataset. Both cases reduce the rate of false negative pairs, thus minimizing the rate in which the model is incorrectly penalized during contrastive training. These modified approaches remain fully unsupervised end-to-end. To evaluate their performance, we add a single linear layer trained to generate class predictions based on these contrastively-learned features and demonstrate increased performance compared to supervised models; observing an improvement in classification accuracy of \(3.06\%\) using only \(10\%\) of the labeled data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 79.50
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 99.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://pds.nasa.gov/.

References

  1. Caron, M., et al.: Emerging properties in self-supervised vision transformers. In: Proceedings of the International Conference on Computer Vision (ICCV) (2021)

    Google Scholar 

  2. Chakravarthy, A.S., Roy, R., Ravirathinam, P.:MRSCAtt: a spatio-channel attention-guided network for mars rover image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1961–1970 (2021)

    Google Scholar 

  3. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)

    Google Scholar 

  4. Chen, T., Kornblith, S., Swersky, K., Norouzi, M., Hinton, G.: Big self-supervised models are strong semi-supervised learners. arXiv preprint arXiv:2006.10029 (2020)

  5. Chen, T.S., Hung, W.C., Tseng, H.Y., Chien, S.Y., Yang, M.H.: Incremental false negative detection for contrastive learning. arXiv preprint arXiv:2106.03719 (2021)

  6. Chen, X., Fan, H., Girshick, R., He, K.: Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 (2020)

  7. Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  8. Goh, E., Chen, J., Wilson, B.: Mars terrain segmentation with less labels. arXiv preprint arXiv:2202.00791 (2022)

  9. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. arXiv preprint arXiv:1911.05722 (2019)

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  11. Helber, P., Bischke, B., Dengel, A., Borth, D.: Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J. Select. Top. Appl. Earth Obser Remote Sens. 12(7), 2217–2226 (2019)

    Article  Google Scholar 

  12. Huynh, T., Kornblith, S., Walter, M.R., Maire, M., Khademi, M.: Boosting contrastive self-supervised learning with false negative cancellation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2785–2795 (2022)

    Google Scholar 

  13. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: 25th Proceedings of Advances in Neural Information Processing Systems (2012)

    Google Scholar 

  14. Kucik, A.S., Meoni, G.: Investigating spiking neural networks for energy-efficient on-board AI applications. a case study in land cover and land use classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2020–2030 (2021)

    Google Scholar 

  15. NASA: Mars exploration rover mission goals, http://mars.nasa.gov/mer/mission/science/goals

  16. Panambur, T., Chakraborty, D., Meyer, M., Milliken, R., Learned-Miller, E., Parente, M.: Self-supervised learning to guide scientifically relevant categorization of Martian terrain images. arXiv preprint arXiv:2204.09854 (2022)

  17. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  18. Sprinks, J.C., Wardlaw, J., Houghton, R., Bamford, S., Marsh, S.: Mars in motion: an online citizen science platform looking for changes on the surface of mars. In: AAS/Division for Planetary Sciences Meeting Abstracts# 48. vol. 48, pp. 426–401 (2016)

    Google Scholar 

  19. Swan, R.M., et al.: Ai4mars: a dataset for terrain-aware autonomous driving on mars. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1982–1991 (2021)

    Google Scholar 

  20. Wagstaff, K., et al.: Mars image content classification: three years of NASA deployment and recent advances. arXiv preprint arXiv:2102.05011 (2021)

  21. Wagstaff, K.L., Lu, Y., Stanboli, A., Grimes, K., Gowda, T., Padams, J.: Deep mars: CNN classification of mars imagery for the PDS imaging atlas. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  22. Wang, W., Lin, L., Fan, Z., Liu, J.: Semi-supervised learning for mars imagery classification. In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 499–503. IEEE (2021)

    Google Scholar 

  23. Wang, Y., Braham, N.A.A.A., Albrecht, C.M., Xiong, Z., Liu, C., Zhu, X.X.: Ssl4eo-s12: a large-scale multimodal multitemporal dataset for self-supervised learning in earth observation (2022)

    Google Scholar 

  24. Wightman, R.: Pytorch image models (2019). http://github.com/rwightman/pytorch-image-models. https://doi.org/10.5281/zenodo.4414861

  25. Xue, J., Zhang, H., Dana, K.: Deep texture manifold for ground terrain recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 558–567 (2018)

    Google Scholar 

Download references

Acknowledgments

This research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004), and was funded by the Data Science Working Group (DSWG). The authors also acknowledge the Extreme Science and Engineering Discovery Environment (XSEDE) Bridges at Pittsburgh Supercomputing Center for providing GPU resources through allocation TG-CIS220027. U.S. Government sponsorship acknowledged.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Isaac Ronald Ward .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ward, I.R., Moore, C., Pak, K., Chen, J., Goh, E. (2023). Improving Contrastive Learning on Visually Homogeneous Mars Rover Images. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13801. Springer, Cham. https://doi.org/10.1007/978-3-031-25056-9_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25056-9_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25055-2

  • Online ISBN: 978-3-031-25056-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics