Abstract
Skin diseases are among the most prevalent health issues, and accurate computer-aided diagnosis methods are of importance for both dermatologists and patients. However, most of the existing methods overlook the essential domain knowledge required for skin disease diagnosis. A novel multi-task model, namely DermImitFormer, is proposed to fill this gap by imitating dermatologists’ diagnostic procedures and strategies. Through multi-task learning, the model simultaneously predicts body parts and lesion attributes in addition to the disease itself, enhancing diagnosis accuracy and improving diagnosis interpretability. The designed lesion selection module mimics dermatologists’ zoom-in action, effectively highlighting the local lesion features from noisy backgrounds. Additionally, the presented cross-interaction module explicitly models the complicated diagnostic reasoning between body parts, lesion attributes, and diseases. To provide a more robust evaluation of the proposed method, a large-scale clinical image dataset of skin diseases with significantly more cases than existing datasets has been established. Extensive experiments on three different datasets consistently demonstrate the state-of-the-art recognition performance of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Binder, M., et al.: Epiluminescence microscopy: a useful tool for the diagnosis of pigmented skin lesions for formally trained dermatologists. Arch. Dermatol. 131(3), 286–291 (1995)
Chen, K., Lei, W., Zhang, R., Zhao, S., Zheng, W.S., Wang, R.: PCCT: progressive class-center triplet loss for imbalanced medical image classification. arXiv preprint arXiv:2207.04793 (2022)
Chen, X., Li, D., Zhang, Y., Jian, M.: Interactive attention sampling network for clinical skin disease image classification. In: Ma, H., et al. (eds.) PRCV 2021. LNCS, vol. 13021, pp. 398–410. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88010-1_33
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Gupta, K., Krishnan, M., Narayanan, A., Narayan, N.S., et al.: Dual stream network with selective optimization for skin disease recognition in consumer grade images. In: Proceedings of the International Conference on Pattern Recognition (ICPR), pp. 5262–5269. IEEE (2021)
Jalaboi, R., Faye, F., Orbes-Arteaga, M., Jørgensen, D., Winther, O., Galimzianova, A.: Dermx: an end-to-end framework for explainable automated dermatological diagnosis. Med. Image Anal. 83, 102647 (2023)
Kittler, H., Pehamberger, H., Wolff, K., Binder, M.: Diagnostic accuracy of dermoscopy. Lancet Oncol. 3(3), 159–165 (2002)
Kshirsagar, P.R., Manoharan, H., Shitharth, S., Alshareef, A.M., Albishry, N., Balachandran, P.K.: Deep learning approaches for prognosis of automated skin disease. Life 12(3), 426 (2022)
Lei, W., Zhang, R., Yang, Y., Wang, R., Zheng, W.S.: Class-center involved triplet loss for skin disease classification on imbalanced data. In: Proceedings of the International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2020)
Li, L.F., Wang, X., Hu, W.J., Xiong, N.N., Du, Y.X., Li, B.S.: Deep learning in skin disease image recognition: a review. IEEE Access 8, 208264–208280 (2020)
Liu, Y., et al.: A deep learning system for differential diagnosis of skin diseases. Nat. Med. 26(6), 900–908 (2020)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10012–10022 (2021)
Nast, A., Griffiths, C.E., Hay, R., Sterry, W., Bolognia, J.L.: The 2016 international league of dermatological societies’ revised glossary for the description of cutaneous lesions. Br. J. Dermatol. 174(6), 1351–1358 (2016)
Ou, C., et al.: A deep learning based multimodal fusion model for skin lesion diagnosis using smartphone collected clinical images and metadata. Front. Surg. 9, 1029991 (2022)
Pacheco, A.G., Krohling, R.A.: The impact of patient clinical information on automated skin cancer detection. Comput. Biol. Med. 116, 103545 (2020)
Rogers, H.W., Weinstock, M.A., Feldman, S.R., Coldiron, B.M.: Incidence estimate of nonmelanoma skin cancer (keratinocyte carcinomas) in the us population, 2012. JAMA Dermatol. 151(10), 1081–1086 (2015)
Siegel, R.L., Miller, K.D., Fuchs, H.E., Jemal, A.: Cancer statistics, 2022. CA Cancer J. Clin. 72(1), 7–33 (2022)
Sun, X., Yang, J., Sun, M., Wang, K.: A benchmark for automatic visual classification of clinical skin disease images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 206–222. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_13
Wang, J., Yu, X., Gao, Y.: Feature fusion vision transformer for fine-grained visual categorization. arXiv preprint arXiv:2107.02341 (2021)
Wu, J., et al.: Learning differential diagnosis of skin conditions with co-occurrence supervision using graph convolutional networks. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 335–344. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_33
Xu, J., et al.: Remixformer: a transformer model for precision skin tumor differential diagnosis via multi-modal imaging and non-imaging data. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13433, pp. 624–633. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16437-8_60
Xu, Z., Zhuang, J., Zhang, R., Wang, R., Guo, X., Zheng, W.S.: Auxiliary decoder and classifier for imbalanced skin disease diagnosis. J. Phys. Conf. Ser. 1631(1), 012046 (2020)
Yang, J., et al.: Self-paced balance learning for clinical skin disease recognition. IEEE Trans. Neural Netw. Learn. Syst. 31(8), 2832–2846 (2019)
Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: Cutmix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6023–6032 (2019)
Zhang, J., Xie, Y., Wu, Q., Xia, Y.: Medical image classification using synergic deep learning. Med. Image Anal. 54, 10–19 (2019)
Zhang, J., Xie, Y., Xia, Y., Shen, C.: Attention residual learning for skin lesion classification. IEEE Trans. Med. Imaging 38(9), 2092–2103 (2019)
Acknowledgement
This work was supported by National Key R &D Program of China (2020YFC2008703) and the Project of Intelligent Management Software for Multimodal Medical Big Data for New Generation Information Technology, the Ministry of Industry and Information Technology of the People’s Republic of China (TC210804V).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, YJ. et al. (2023). A Novel Multi-task Model Imitating Dermatologists for Accurate Differential Diagnosis of Skin Diseases in Clinical Images. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14225. Springer, Cham. https://doi.org/10.1007/978-3-031-43987-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-43987-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43986-5
Online ISBN: 978-3-031-43987-2
eBook Packages: Computer ScienceComputer Science (R0)