[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

A Feature-Based Out-of-Distribution Detection Approach in Skin Lesion Classification

  • Conference paper
  • First Online:
Intelligent Systems (BRACIS 2023)

Abstract

When dealing with Deep Learning applications in open-set problems, accurately classifying known classes seen in the training phase is not the only aspect to be taken into account. In such a context, detecting Out-of-Distribution (OOD) samples plays an important role as an auxiliary task, generally solved by OOD detection methods. For medical applications, detecting unknown samples may in classification problems can be beneficial for many aspects, such as a better understanding of the diagnosis and probably a more adequate treatment. In this article, we evaluate a feature space-based approach, named as OpenPCS-Class, for OOD detection in medical applications, more specifically skin lesion classification. We compare the OpenPCS-Class against important OOD detection methods, evaluating different model architectures and OOD datasets. The OpenPCS-Class outperformed other methods at 48.4% and 5.3% in terms of FPR95 and AUROC, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 51.99
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 64.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Code available at https://github.com/mdrs-thiago/skin-lesion-ood-detection.

References

  1. Ali, S.N., et al.: Monkeypox skin lesion detection using deep learning models: a preliminary feasibility study. arXiv preprint arXiv:2207.03342 (2022)

  2. Berger, C., Paschali, M., Glocker, B., Kamnitsas, K.: Confidence-based out-of-distribution detection: a comparative study and analysis. In: Sudre, C.H., et al. (eds.) UNSURE/PIPPI -2021. LNCS, vol. 12959, pp. 122–132. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87735-4_12

    Chapter  Google Scholar 

  3. Calderon-Ramirez, S., Yang, S., Elizondo, D., Moemeni, A.: Dealing with distribution mismatch in semi-supervised deep learning for COVID-19 detection using chest X-ray images: a novel approach using feature densities. Appl. Soft Comput. 123, 108983 (2022)

    Article  Google Scholar 

  4. Cao, T., Huang, C.W., Hui, D.Y.T., Cohen, J.P.: A benchmark of medical out of distribution detection. arXiv preprint arXiv:2007.04250 (2020)

  5. Carvalho, T., Vellasco, M., Amaral, J.F.: Out-of-distribution detection in deep learning models: a feature space-based approach. In: International Joint Conference on Neural Networks (2023)

    Google Scholar 

  6. Cho, W., Park, J., Choo, J.: Training auxiliary prototypical classifiers for explainable anomaly detection in medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2624–2633 (2023)

    Google Scholar 

  7. Dosovitskiy, A., et al.: An image is worth 16 \(\times \)16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  8. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  9. Hendrycks, D., et al.: Scaling out-of-distribution detection for real-world settings. In: International Conference on Machine Learning, pp. 8759–8773. PMLR (2022)

    Google Scholar 

  10. Hendrycks, D., Carlini, N., Schulman, J., Steinhardt, J.: Unsolved problems in ML safety. arXiv preprint arXiv:2109.13916 (2021)

  11. Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)

  12. Kamoi, R., Kobayashi, K.: Why is the mahalanobis distance effective for anomaly detection? arXiv preprint arXiv:2003.00402 (2020)

  13. Karimi, D., Gholipour, A.: Improving calibration and out-of-distribution detection in deep models for medical image segmentation. IEEE Trans. Artif. Intell. 4, 383–397 (2022)

    Article  Google Scholar 

  14. Lambert, B., Forbes, F., Doyle, S., Tucholka, A., Dojat, M.: Improving uncertainty-based out-of-distribution detection for medical image segmentation. arXiv preprint arXiv:2211.05421 (2022)

  15. Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In: Advances in neural information processing systems, vol. 31 (2018)

    Google Scholar 

  16. Liu, W., Wang, X., Owens, J., Li, Y.: Energy-based out-of-distribution detection. Adv. Neural. Inf. Process. Syst. 33, 21464–21475 (2020)

    Google Scholar 

  17. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976–11986 (2022)

    Google Scholar 

  18. Martinez, J.A.C., Oliveira, H., dos Santos, J.A., Feitosa, R.Q.: Open set semantic segmentation for multitemporal crop recognition. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021)

    Article  Google Scholar 

  19. Muhammad, K., et al.: Vision-based semantic segmentation in scene understanding for autonomous driving: recent achievements, challenges, and outlooks. IEEE Trans. Intell. Transp. Syst. 23, 22694–22715 (2022)

    Article  Google Scholar 

  20. Nunes, I., Pereira, M.B., Oliveira, H., Santos, J.A.D., Poggi, M.: Fuss: Fusing superpixels for improved segmentation consistency. arXiv preprint arXiv:2206.02714 (2022)

  21. Oliveira, H., Silva, C., Machado, G.L., Nogueira, K., Dos Santos, J.A.: Fully convolutional open set segmentation. Mach. Learn. 112, 1733–1784 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  22. Pawlowski, N., Glocker, B.: Abnormality detection in histopathology via density estimation with normalising flows. In: Medical Imaging with Deep Learning (2021)

    Google Scholar 

  23. Podolskiy, A., Lipin, D., Bout, A., Artemova, E., Piontkovskaya, I.: Revisiting Mahalanobis distance for transformer-based out-of-domain detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 13675–13682 (2021)

    Google Scholar 

  24. Ren, J., Fort, S., Liu, J., Roy, A.G., Padhy, S., Lakshminarayanan, B.: A simple fix to mahalanobis distance for improving near-OOD detection. arXiv preprint arXiv:2106.09022 (2021)

  25. Roy, A.G., et al.: Does your dermatology classifier know what it doesn’t know? detecting the long-tail of unseen conditions. Med. Image Anal. 75, 102274 (2022)

    Article  Google Scholar 

  26. Swetha, P., Srilatha, J.: Applications of speech recognition in the agriculture sector: a review. ECS Trans. 107(1), 19377 (2022)

    Article  Google Scholar 

  27. Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5(1), 1–9 (2018)

    Article  Google Scholar 

  28. Uwimana, A., Senanayake, R.: Out of distribution detection and adversarial attacks on deep neural networks for robust medical image analysis. arXiv preprint arXiv:2107.04882 (2021)

  29. Wightman, R., Touvron, H., Jégou, H.: ResNet strikes back: an improved training procedure in timm. arXiv preprint arXiv:2110.00476 (2021)

  30. Wollek, A., Willem, T., Ingrisch, M., Sabel, B., Lasser, T.: A knee cannot have lung disease: out-of-distribution detection with in-distribution voting using the medical example of chest X-ray classification. arXiv preprint arXiv:2208.01077 (2022)

  31. Wright, J., Ma, Y.: High-Dimensional Data Analysis with Low-Dimensional Models: Principles, Computation, and Applications. Cambridge University Press (2022)

    Google Scholar 

  32. Wu, Y., et al.: Revisit overconfidence for OOD detection: reassigned contrastive learning with adaptive class-dependent threshold. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4165–4179 (2022)

    Google Scholar 

  33. Wu, Y., et al.: Disentangling confidence score distribution for out-of-domain intent detection with energy-based learning. arXiv preprint arXiv:2210.08830 (2022)

  34. Yang, J., Zhou, K., Li, Y., Liu, Z.: Generalized out-of-distribution detection: a survey. arXiv preprint arXiv:2110.11334 (2021)

  35. Ye, N., et al.: OOD-bench: quantifying and understanding two dimensions of out-of-distribution generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7947–7958 (2022)

    Google Scholar 

  36. Zadorozhny, K., Thoral, P., Elbers, P., Ciná, G.: Out-of-distribution detection for medical applications: guidelines for practical evaluation. In: Shaban-Nejad, A., Michalowski, M., Bianco, S. (eds.) Multimodal AI in Healthcare. Studies in Computational Intelligence, vol. 1060, pp. 137–153. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-14771-5_10

  37. Zhang, O., Delbrouck, J.-B., Rubin, D.L.: Out of distribution detection for medical images. In: Sudre, C.H., et al. (eds.) UNSURE/PIPPI -2021. LNCS, vol. 12959, pp. 102–111. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87735-4_10

    Chapter  Google Scholar 

  38. Zimmerman, D.W., Zumbo, B.D.: Rank transformations and the power of the student T test and welch T’test for non-normal populations with unequal variances. Can. J. Exp. Psychol. 47(3), 523 (1993)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001, Conselho Nacional de Desenvolvimento e Pesquisa (CNPq) under Grants 140254/2021-8 and 308717/2020-1, and Fundação de Amparo à Pesquisa do Rio de Janeiro (FAPERJ)

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thiago Carvalho .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Carvalho, T., Vellasco, M., Amaral, J.F., Figueiredo, K. (2023). A Feature-Based Out-of-Distribution Detection Approach in Skin Lesion Classification. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14196. Springer, Cham. https://doi.org/10.1007/978-3-031-45389-2_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45389-2_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45388-5

  • Online ISBN: 978-3-031-45389-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics