More Web Proxy on the site http://driver.im/

research-article

Breast tumor segmentation and shape classification in mammograms using generative adversarial and convolutional neural network

Authors:

Vivek Kumar Singh,

Hatem A. Rashwan,

Santiago Romani,

Md. Mostafa Kamal Sarker,

Meritxell Arenas,

Jordina Torrents-BarrenaAuthors Info & Claims

Volume 139, Issue C

https://doi.org/10.1016/j.eswa.2019.112855

Published: 01 January 2020 Publication History

Highlights

•

A conditional generative adversarial network (cGAN) is proposed to segment the breast tumor.

•

A convolutional neural network (CNN) based shape classification descriptor is proposed.

•

The segmented regions are classified into four different shapes and correlated with four molecular subtypes.

Abstract

Mammogram inspection in search of breast tumors is a tough assignment that radiologists must carry out frequently. Therefore, image analysis methods are needed for the detection and delineation of breast tumors, which portray crucial morphological information that will support reliable diagnosis. In this paper, we proposed a conditional Generative Adversarial Network (cGAN) devised to segment a breast tumor within a region of interest (ROI) in a mammogram. The generative network learns to recognize the tumor area and to create the binary mask that outlines it. In turn, the adversarial network learns to distinguish between real (ground truth) and synthetic segmentations, thus enforcing the generative network to create binary masks as realistic as possible. The cGAN works well even when the number of training samples are limited. As a consequence, the proposed method outperforms several state-of-the-art approaches. Our working hypothesis is corroborated by diverse segmentation experiments performed on INbreast and a private in-house dataset. The proposed segmentation model, working on an image crop containing the tumor as well as a significant surrounding area of healthy tissue (loose frame ROI), provides a high Dice coefficient and Intersection over Union (IoU) of 94% and 87%, respectively. In addition, a shape descriptor based on a Convolutional Neural Network (CNN) is proposed to classify the generated masks into four tumor shapes: irregular, lobular, oval and round. The proposed shape descriptor was trained on DDSM, since it provides shape ground truth (while the other two datasets does not), yielding an overall accuracy of 80%, which outperforms the current state-of-the-art.

References

[1]

F. Akram, J. Kim, C. Lee, K.N. Choi, Segmentation of regions of interest using active contours with SPF function, Computational and Mathematical Methods in Medicine 2015 (2015) 710326:1–710326:14,.

[2]

M.A. Al-antari, M.A. Al-masni, M.-T. Choi, S.-M. Han, T.-S. Kim, A fully integrated computer-aided diagnosis system for digital x-ray mammograms via deep learning detection, segmentation, and classification, International Journal of Medical Informatics 117 (2018) 44–54.

[3]

V. Badrinarayanan, A. Kendall, R. Cipolla, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (12) (2017) 2481–2495.

[4]

M. Boisserie-Lacroix, G. Hurtevent-Labrot, S. Ferron, N. Lippa, H. Bonnefoi, G. Mac Grogan, Correlation between imaging and molecular classification of breast cancers, Diagnostic and Interventional Imaging 94 (11) (2013) 1069–1080.

[5]

J.S. Cardoso, I. Domingues, H.P. Oliveira, Closed shortest path in the original coordinates with an application to breast cancer, International Journal of Pattern Recognition and Artificial Intelligence 29 (1) (2015).

[6]

J.S. Cardoso, N. Marques, N. Dhungel, G. Carneiro, A. Bradley, Mass segmentation in mammograms: A cross-sensor comparison of deep and tailored features, Proceeding of the ieee international conference on image processing (icip), 2017, pp. 1737–1741.

[7]

H. Cheng, X. Cai, X. Chen, L. Hu, X. Lou, Computer-aided detection and classification of microcalcifications in mammograms: a survey, Pattern Recognition 36 (12) (2003) 2967–2991,.

[8]

N. Cho, Molecular subtypes and imaging phenotypes of breast cancer, Ultrasonography 35 (4) (2016) 281.

[9]

N. Dhungel, G. Carneiro, A. Bradley, Tree re-weighted belief propagation using deep learning potentials for mass segmentation from mammograms, 12th IEEE international symposium on biomedical imaging, ISBI 2015, Brooklyn, NY, USA, April 16–19, 2015, 2015, pp. 760–763,.

[10]

N. Dhungel, G. Carneiro, A.P. Bradley, Deep learning and structured prediction for the segmentation of mass in mammograms, Proceedings of the international conference on medical image computing and computer-assisted intervention (MICCAI), 2015, pp. 605–612.

[11]

N. Dhungel, G. Carneiro, A.P. Bradley, A deep learning approach for the analysis of masses in mammograms with minimal user intervention, Medical Image Analysis 37 (2017) 114–128.

[12]

Y. Fang, J. Xie, G. Dai, M. Wang, F. Zhu, T. Xu, 3d deep shape descriptor, Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 2319–2328.

[13]

Fu, H., Cheng, J., Xu, Y., Wong, D. W. K., Liu, J., & Cao, X. (2018). Joint optic disc and cup segmentation based on multi-label deep network and polar transformation. arXiv: 1801.00926.

[14]

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, Generative adversarial nets, Advances in neural information processing systems, 2014, pp. 2672–2680.

[15]

A. Hamidinekoo, E. Denton, A. Rampun, K. Honnor, R. Zwiggelaar, Deep learning in mammography and breast histology, an overview and future trends, Medical Image Analysis 47 (2018) 45–67.

[16]

M. Hazarika, L.B. Mahanta, A new breast border extraction and contrast enhancement technique with digital mammogram images for improved detection of breast cancer, Asian Pacific Journal of Cancer Prevention: APJCP 19 (8) (2018) 2141.

[17]

C. Hofer, R. Kwitt, M. Niethammer, A. Uhl, Deep learning with topological signatures, Advances in neural information processing systems, 2017, pp. 1634–1644.

[18]

P. Isola, J.-Y. Zhu, T. Zhou, A.A. Efros, Image-to-image translation with conditional adversarial networks, Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp. 5967–5976.

[19]

Z. Jiao, X. Gao, Y. Wang, J. Li, A parasitic metric learning net for breast mass classification based on mammography, Pattern Recognition 75 (2018) 292–301.

[20]

S.T. Kim, H. Lee, H.G. Kim, Y.M. Ro, ICADx: Interpretable computer aided diagnosis of breast masses, Proceedings of the SPIE - medical imaging 2018: Computer-aided diagnosis, 10575, 2018.

[21]

P. Kisilev, E. Sason, E. Barkan, S. Hashoul, Medical image description using multi-task-loss cnn, Deep learning and data labeling for medical applications, Springer, 2016, pp. 121–129.

[22]

P. Kisilev, E. Walach, S.Y. Hashoul, E. Barkan, B. Ophir, S. Alpert, Semantic description of medical image findings: Structured learning approach, Proceedings of the british machine vision conference (BMVC), 2015, pp. 171.1–171.11.

[23]

Kozegar, E., Soryani, M., Minaei, B., Domingues, I. et al. (2013). Assessment of a novel mass detection algorithm in mammograms.

[24]

Kshema, M.J. George, D.A.S. Dhas, Preprocessing filters for mammogram images: A review, 2017 conference on emerging devices and smart systems (ICEDSS), 2017, pp. 1–7,.

[25]

L. Kurnianggoro, K.-H. Jo, et al., A survey of 2d shape representation: Methods, evaluations, and future research directions, Neurocomputing 300 (2018) 1–16.

[26]

S. Lankton, A. Tannenbaum, Localizing region-based active contours, IEEE Transactions on Image Processing 17 (11) (2008) 2029–2039.

[27]

B. Lauby-Secretan, C. Scoccianti, D. Loomis, L. Benbrahim-Tallaa, V. Bouvard, F. Bianchini, Breast-cancer screening-viewpoint of the IARC working group, New England Journal of Medicine 372 (24) (2015) 2353–2358.

[28]

G. Litjens, T. Kooi, B.E. Bejnordi, A.A.A. Setio, F. Ciompi, M. Ghafoorian, A survey on deep learning in medical image analysis, Medical Image Analysis 42 (2017) 60–88.

[29]

S. Liu, X.-D. Wu, W.-J. Xu, Q. Lin, X.-J. Liu, Y. Li, Is there a correlation between the presence of a spiculated mass on mammogram and luminal a subtype breast cancer?, Korean Journal of Radiology 17 (6) (2016) 846–852.

[30]

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, Ssd: Single shot multibox detector, European conference on computer vision, Springer, 2016, pp. 21–37.

[31]

Z. Liu, C. Zhuo, X. Xu, Efficient segmentation method using quantised and non-linear cenn for breast tumour classification, Electronics Letters (2018).

[32]

J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, Proceedings of the IEEE conference on computer vision and pattern recognition (cvpr), 2015, pp. 3431–3440.

[33]

L. Luciano, A.B. Hamza, Deep learning with geodesic moments for 3d shape classification, Pattern Recognition Letters 105 (2018) 182–190.

[34]

M. Masoumi, A.B. Hamza, Spectral shape classification: A deep learning approach, Journal of Visual Communication and Image Representation 43 (2017) 198–211.

[35]

C.E.F. Matos, J.C. Souza, J.O.B. Diniz, G.B. Junior, A.C. de Paiva, J.D.S. de Almeida, Diagnosis of breast tissue in mammography images based local feature descriptors, Multimedia Tools and Applications (2018) 1–26.

[36]

C.R. Qi, H. Su, K. Mo, L.J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, Proceedings of Computer Vision and Pattern Recognition (CVPR), IEEE 1 (2) (2017) 4.

[37]

R.M. Rangayyan, S. Banik, J.L. Desautels, Computer-aided detection of architectural distortion in prior mammograms of interval cancer, Journal of Digital Imaging 23 (5) (2010) 611–631.

[38]

J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: Unified, real-time object detection, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.

[39]

S. Ren, K. He, R. Girshick, J. Sun, Faster r-cnn: Towards real-time object detection with region proposal networks, Advances in neural information processing systems, 2015, pp. 91–99.

[40]

O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, Proceedings of the international conference on medical image computing and computer-assisted intervention (MICCAI), 2015, pp. 234–241.

[41]

R. Rouhi, M. Jafari, S. Kasaei, P. Keshavarzian, Benign and malignant breast tumors classification based on region growing and cnn segmentation, Expert Systems with Applications 42 (3) (2015) 990–1002.

[42]

M.M.K. Sarker, H.A. Rashwan, F. Akram, S.F. Banu, A. Saleh, V.K. Singh, SLSDeep: Skin lesion segmentation based on dilated residual and pyramid pooling networks, Medical image computing and computer assisted intervention - MICCAI 2018 - 21st international conference, Granada, Spain, september 16–20, 2018, proceedings, part II, 2018, pp. 21–29,.

Digital Library

[43]

J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks 61 (2015) 85–117.

Digital Library

[44]

R.L. Siegel, K.D. Miller, A. Jemal, Cancer statistics, 2017, CA: A Cancer Journal for Clinicians 67 (1) (2017) 7–30.

[45]

Singh, V. K., Rashwan, H., Akram, F., Pandey, N., Sarker, M., & Kamal, M., et al. (2018). Retinal optic disc segmentation using conditional generative adversarial network. arXiv: 1806.03905.

[46]

V.K. Singh, S. Romani, H.A. Rashwan, F. Akram, N. Pandey, M.M.K. Sarker, Conditional generative adversarial and convolutional networks for x-ray breast mass segmentation and shape classification, Medical image computing and computer assisted intervention - MICCAI 2018 - 21st international conference, Granada, Spain, september 16–20, 2018, proceedings, part II, 2018, pp. 833–840,.

Digital Library

[47]

V.K. Singh, S. Romani, J. Torrents-Barrena, F. Akram, N. Pandey, M.M.K. Sarker, Classification of breast cancer molecular subtypes from their micro-texture in mammograms using a vggnet-based convolutional neural network, Recent advances in artificial intelligence research and development: Proceedings of the 20th international conference of the catalan association for artificial intelligence, deltebre, terres de l’ebre, Spain, october 25–27, 2017, 300, IOS Press, 2017, p. 76.

[48]

K. Tamaki, T. Ishida, M. Miyashita, M. Amari, N. Ohuchi, N. Tamaki, Correlation between mammographic findings and corresponding histopathology: potential predictors for biological characteristics of breast diseases, Cancer Science 102 (12) (2011) 2179–2185.

[49]

J. Tang, R.M. Rangayyan, J. Xu, I. El Naqa, Y. Yang, Computer-aided detection and diagnosis of breast cancer with mammography: recent advances, IEEE Transactions on Information Technology in Biomedicine 13 (2) (2009) 236–251.

[50]

D. Yang, D. Xu, S.K. Zhou, B. Georgescu, M. Chen, S. Grbic, Automatic liver segmentation using an adversarial image-to-image network, Proceedings of the international conference on medical image computing and computer-assisted intervention (MICCAI), 2017, pp. 507–515.

[51]

N.I. Yassin, S. Omran, E.M. El Houby, H. Allam, Machine learning techniques for breast cancer computer aided diagnosis using different image modalities: A systematic review, Computer Methods and Programs in Biomedicine 156 (2018) 25–45.

[52]

W. Zhu, X. Xiang, T.D. Tran, G.D. Hager, X. Xie, Adversarial deep structured nets for mass segmentation from mammograms, Proceedings of the ieee 15th international symposium on biomedical imaging (ISBI), 2018, pp. 847–850.

Cited By

Oza UGohel BKumar POza P(2024)Presegmenter Cascaded Framework for Mammogram Mass SegmentationJournal of Biomedical Imaging10.1155/2024/94220832024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/9422083
Chen YWu WWu HChen DHuang Y(2024)Tumor segmentation of breast tomosynthesis in breast-conserving surgery with deep learningProceedings of the 2024 8th International Conference on Medical and Health Informatics10.1145/3673971.3673985(18-23)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1145/3673971.3673985
Zamani MSharifian S(2024)Distributed edge to cloud ensemble deep learning architecture to diagnose Covid-19 from lung image in IoT based e-Health systemThe Journal of Supercomputing10.1007/s11227-024-06163-080:13(18492-18520)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11227-024-06163-0
Show More Cited By

Index Terms

Breast tumor segmentation and shape classification in mammograms using generative adversarial and convolutional neural network
1. Applied computing
  1. Life and medical sciences
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Index terms have been assigned to the content through auto-classification.

Recommendations

Computer-aided detection of breast cancer on mammograms

PSOWNN - Particle Swarm Optimized Wavelet Neural Network. DB - Database.Display Omitted We propose a CAD system for detecting breast cancer in mammograms.Swarm intelligence optimized wavelet neural network detects the cancers.We focus on optimized ...
Reinforcement learning (RL)-based semantic segmentation and attention based backpropagation convolutional neural network (ABB-CNN) for breast cancer identification and classification using mammogram images
Abstract
Breast cancer poses a threat to women’s health and contributes to an increase in mortality rates. Mammography has proven to be an effective tool for the early detection of breast cancer. However, it faces many challenges in early breast cancer ...
Brain Tumor Segmentation on MR Images Using Anisotropic Deeply Supervised Convolutional Neural Network
ICVIP '18: Proceedings of the 2018 2nd International Conference on Video and Image Processing

Glioma is the most common and cancerous type of early stage brain tumors that arise from glial cells. Deep learning based solutions are flexible in brain tumor image analysis and computer-assisted diagnosis but it does not gain the desired accuracy. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 139, Issue C

Jan 2020

492 pages

ISSN:0957-4174

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 January 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Oza UGohel BKumar POza P(2024)Presegmenter Cascaded Framework for Mammogram Mass SegmentationJournal of Biomedical Imaging10.1155/2024/94220832024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/9422083
Chen YWu WWu HChen DHuang Y(2024)Tumor segmentation of breast tomosynthesis in breast-conserving surgery with deep learningProceedings of the 2024 8th International Conference on Medical and Health Informatics10.1145/3673971.3673985(18-23)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1145/3673971.3673985
Zamani MSharifian S(2024)Distributed edge to cloud ensemble deep learning architecture to diagnose Covid-19 from lung image in IoT based e-Health systemThe Journal of Supercomputing10.1007/s11227-024-06163-080:13(18492-18520)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11227-024-06163-0
El-Mawla NBerbar MEl-Fishawy NEl-Rashidy M(2024)A novel deep learning approach (Bi-xBcNet-96) considering green AI to discover breast cancer using mammography imagesNeural Computing and Applications10.1007/s00521-024-09815-736:21(12701-12723)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00521-024-09815-7
Subaramani NSasikala E(2024)An attention-based dense network model for cardiac image segmentation using learning approachesSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-023-09482-128:1(765-775)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s00500-023-09482-1
Enobun AAnakwenze UTaherkhani AAnastassi ZCaraffini FEshkiki H(2024)Segmenting Breast Ultrasound Scans Using a Generative Adversarial Network Embedding U-NetArtificial Intelligence in Healthcare10.1007/978-3-031-67285-9_11(149-159)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-67285-9_11
Zhang QXiao JZheng B(2023)Image Segmentation of Triple-Negative Breast Cancer by Incorporating Multiscale and Parallel Attention MechanismsScientific Programming10.1155/2023/66291892023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/6629189
Zhang BRigall EHuang YZou XZhang SDong JYu H(2023)A Method for Breast Mass Segmentation using Image Augmentation with SAM and Receptive Field ExpansionProceedings of the 2023 12th International Conference on Computing and Pattern Recognition10.1145/3633637.3633698(387-394)Online publication date: 27-Oct-2023
https://dl.acm.org/doi/10.1145/3633637.3633698
Fatima MKhan MShaheen SAlmujally NWang S(2023)B2C3NetF2CAAI Transactions on Intelligence Technology10.1049/cit2.122198:4(1374-1390)Online publication date: 13-Apr-2023
https://dl.acm.org/doi/10.1049/cit2.12219
Luo JTang YWang JLu H(2023)USMLPImage and Vision Computing10.1016/j.imavis.2023.104761137:COnline publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1016/j.imavis.2023.104761
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents