Abstract
Controlled despeckling (structure/edges/feature preservation with smoothing the homogeneous areas) is a desired pre-processing step for the design of computer-aided diagnostic (CAD) systems using ultrasound images as the presence of speckle noise masks diagnostically important information making interpretation difficult even for experienced radiologist. For efficiently classifying the breast tumors, the conventional CAD system designs use hand-crafted features. However, these features are not robust to the variations in size, shape and orientation of the tumors resulting in lower sensitivity. Thus deep feature extraction and classification of breast ultrasound images have recently gained attention from research community. The deep networks come with an advantage of directly learning the representative features from the images. However, these networks are difficult to train from scratch if the representative training data is small in size. Therefore transfer learning approach for deep feature extraction and classification of medical images has been widely used. In the present work the performance of four pre-trained convolutional neural networks VGG-19, SqueezeNet, ResNet-18 and GoogLeNet has been evaluated for differentiating between benign and malignant tumor types. From the results of the experiments, it is noted that CAD system design using GoogLeNet architecture for deep feature extraction followed by correlation based feature selection and fuzzy feature selection using ANFC-LH yields highest accuracy of 98.0% with individual class accuracy value of 100% and 96% for benign and malignant classes respectively. For differentiating between the breast tumors, the proposed CAD system design can be utilized in routine clinical environment.
Similar content being viewed by others
References
Ahmed SS, Dey N, Ashour AS, Pistolla DS, Timar DB, Balas VE, Tavares JMRS (2017) Effect of fuzzy partitioning in Crohn’s disease classification: a neuro-fuzzy-based approach. Med Biol Eng Comput 55(1):101–115
Ai H, Hu X, Zhang J (2019) Medical tumor image classification algorithm and its application in breast cancer. Investig Clin 60(5):1169–1179
Aja-Fernández S, AlberolaLópez C (2006) On the estimation of the coefficient of variation for anisotropic diffusion speckle filtering. IEEE Trans Image Process 15(9):2694–2701
Al-Dhabyani W, Fahmy A, Gomma M, Khaled H (2019) Deep learning approaches for data augmentation and classification of breast masses using ultrasound images. Int J Adv Comput Sci Appl 10(5):618–627
Alvarenga AV, Infantosi AFC, Pereira WCA, Azevedo CM (2012) Assessing the combined performance of texture and morphological parameters in distinguishing breast tumors in ultrasound images. Medical Analysis 39(12):7350–7358
Antropova N, Huynh BQ, Giger ML (2017) A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys 44(10):5162–5171
Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK (2018) Medical image analysis using convolutional neural networks: a review. J Med Syst 42:226
Bakkouri I, Afdel K (2019) Computer-aided diagnosis (CAD) system based on multi-layer feature fusion network for skin lesion recognition in dermoscopy images. Multimed Tools Appl. https://doi.org/10.1007/s11042-019-07988-1
Balagourouchetty L, Pragatheeswaran JK, Pottakat B, Ramkumar G (2019) GoogLeNet based ensemble FCNet classifier for focal liver lesion diagnosis. IEEE J Biomed Health Informatics 24:1686–1694. https://doi.org/10.1109/JBHI.2019.2942774
Byra M (2018) Discriminant analysis of neural style representations for breast lesion classification in ultrasound. Biocybernetics Biomed Eng 38(3):684–690
Byra M, Galperin M, Olson L, O’Boyle M, Comstock C, Andre M (2019) Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion. Med Phys 46(2):746–755. https://doi.org/10.1002/mp.13361
Cao Z, Duan L, Yang G, Yue T, Chen Q (2019). An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures. BMC Med Imaging, 19, doi: https://doi.org/10.1186/s12880-019-0349-x.
Cetisli B (2010) Development of an adaptive neuro-fuzzy classifier using linguistic hedges: part 1. Expert Syst Appl 37:6093–6101
Cetisli B (2010) Development of an adaptive neuro-fuzzy classifier using linguistic hedges: part 2. Expert Syst Appl 37:6102–6108
Chan TF, Vese LA (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277
Chen CY, Chiou HJ, Chou SY, Chiou SY, Wang HK, Chou YH, Chiang HK (2009) Computer-aided diagnosis of soft-tissue tumors using sonographic morphologic and texture features. Acad Radiol 16(2):1531–1538
Cheng JZ, Ni D, Chou YH, Qin J, Tiu CM, Chang YC, Huang CS, Shen D, Chen CM (2016). Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT Scientific Reports, 6, doi: https://doi.org/10.1038/srep24454
Chi J, Walia E, Babyn P, Wang J, Groot G, Eramian M (2017) Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J Digit Imaging 30(4):477–486
Chiao JY, Chen KY, Liao KYK, Hsieh PH, Zhang G, Huang TC (2019) Detection and classification the breast tumors using mask R-CNN on sonograms. Medicine 98(19):e15200
Ciritsis A, Rossi C, Eberhard M, Marcon M, Becker AS, Boss A (2019) Automatic classification of ultrasound breast lesions using a deep convolutional neural network. Eur Radiol 29:5458–5468. https://doi.org/10.1007/s00330-019-06118-7
Consensus document for management of breast cancer. Indian Council of Medical Research, [online], http://www.icmr.nic.in/guide/cancer/Breast_Cancer.pdf (Accessed: April 2018).
Crytsal P, Strano SD, Shcharynski S, Koretz MJ (2003) Using sonography to screen women with mammographically dense breasts. Am J Roentgenol 181:177–182
Fletcher SW, Elmore JG (2003) Mammographic screening for breast cancer. N Engl J Med 348(17):1672–1680
Fujioka T, Kubota K, Mori M, Kikuchi Y, Katsuta L, Kasahara M, Oda G, Ishiba T, Nakagawa T, Tateishi U (2019) Distinction between benign and malignant breast masses at breast ultrasound using deep learning method with convolutional neural network. Jpn J Radiol 37:466–472
Garcia AH, Konig P (2018) Further advantages of data augmentation on convolutional neural networks. In: Kurkova V et al (eds) ICANN, vol 11139. Springer, Switzerland, pp 95–103
Gardezi SJS, Elazab A, Lei B, Wang T (2019) Breast cancer detection and diagnosis using mammographic data: systematic review. J Med Internet Res 21(7):e14464
Gokhale S (2009) Ultrasound characterization of breast masses. Int J Radiol Imaging 19(3):242–247
Gordon PB, Goldenberg SL (1995) Malignant breast masses detected only by ultrasound. Cancer 76(4):626–630
Han HK, Kang JY, Jeong MH, Park W, Kim WC, Bang WC, Seong YK (2017) A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 62:7714–7728
Han S, Lee S, Lee JR (2019) A practical implementation of deep learning method for supporting the classification of breast lesions in ultrasound images. Int J Advanc Smart Convergence 8(1):24–34
He K, Zhang X, Ren S, Sun J (2016). Deep residual learning for image recognition. In Proc. of IEEE conference on computer vision and Pattern Recognition, pp. 770–778.
Hooda R, Mittal A, Sofat S (2019). Automated TB classification using ensemble of deep architectures. Multimedia tools and applications, doi: https://doi.org/10.1007/s11042-019-07984-5
http://ultrasoundcases.info/category.aspx?cat=67, Accessed December 2016.
Hu Z, Tang J, Wang Z, Zhang K, Zhang L, Sun Q (2018) Deep learning for image-based cancer detection and diagnosis- a survey. Pattern Recogn 83:134–149
Huang Y, Han L, Dou H, Luo H, Yuan Z, Zhang J, Fin G (2019). Two-stage CNNs for computerized BI-RADS categorization in breast ultrasound images. BioMedical Eng OnLine, 18, doi: https://doi.org/10.1186/s12938-019-0626-5.
Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016). SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and < 0.5 MB model size, arXiv:1602.07360.
Jain I, Jain VK, Jain R (2018) Correlation feature selection based improved-binary particle swarm optimization for gene selection and cancer classification. Appl Soft Comput 62:203–215
Kadah YM, Farag AA, Zurada JM, Badawi AM, Youssef ABM (1996) Classification algorithms for quantitative tissue characterization of diffuse liver disease from ultrasound. IEEE Trans Med Imaging 14(4):466–478
Kameswari SSD, Vijayakumar V (2019) A CNN based breast tumor classifier using Mendeley BUS dataset. Int J Innov Technol Explor Eng 8(6):1340–1343
Kar S, Das S, Ghosh PK (2015) Applications of neuro fuzzy systems: a brief review and future outline. Appl Soft Comput 15:243–259
Karimi B, Krzyzak A (2014) Computer-aided system for automatic classification of suspicious lesions in breast ultrasound images. In: Rutkowski I et al (eds) ICAISC 2014, vol 8468. Springer, Cham, pp 131–142
Key TJ, Verkasalo PK, Banks E (2001) Epidemiology of breast cancer. The Lancet Oncology 2(3):133–140
Kriti, Virmani J, Agarwal R (2019) Effect of despeckle filtering on classification of breast tumors using ultrasound images. Biocybernetics Biomed Eng 39:536–560
Kriti, Virmani J, Agarwal R (2019) Assessment of despeckle filtering algorithms for segmentation of breast tumors from ultrasound images. Biocybernetics Biomed Eng 39:100–121
Kumar A, Kim J, Lyndon D, Fulham M, Feng D (2016) An ensemble of fine-tuned convolutional neural networks for medical image classification. IEEE J Biomed Health Informatics 21(1):31–40
Kumar I, Bhadauria HS, Virmani J, Thakur S (2017) A hybrid hierarchical framework for classification of breast density using digitized film screen mammograms. Multimed Tools Appl 76(18):18789–18813
Lakhani P, Sundaram B (2017) Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 284(2):574–582
Lee CY, Chen GL, Zhang ZX, Chou YH, Hsu CC (2018) Is intensity inhomogeneity correction useful for classification of breast cancer in sonograms using deep neural network ? J Healthcare Eng 2018:1–10
Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van der Laak JA, Ginneken BV, Sanchez CI (2017) A survey of deep, learning in medical image analysis. Med Image Anal 42:60–88
Manogaran G, Varatharajan R, Priyan MK (2018) Hybrid recommendation system for heart disease diagnosis based on kernel learning with adaptive neuro-fuzzy inference system. Multimed Tools Appl 77(4):4379–4399
Mateen M, Wen J, Song S, Huang Z (2019). Fundus image classification using VGG-19 architecture with PCA and SVD. Symmetry, 11 (1): doi: 20.3390/sym11010001.
Meng D, Zhang L, Cao G, Cao W, Zhang G, Hu B (2017) Liver fibrosis classification based on transfer learning and FCNet for ultrasound images. IEEE Access 5:5804–5810
Michalak K, Kwasnicka H (2006) Correlation-based feature selection strategy in classification problems. Int J Appl Math Comput Sci 16:503–511
Moon WK, Lo CM, Huang CS, Chen JH, Chang RF (2012) Computer-aided diagnosis based on speckle patterns in ultrasound images. Ultrasound Med Biol 38(7):1251–1261
Nibali A, He Z, Wollersheim D (2017) Pulmonary nodule classification with deep residual networks. Int J Comput Assist Radiol Surg 12(10):1799–1808
Prabusankarlal KM, Thirumoorthy MR (2015) Assessment of combined textural and morphological features for diagnosis of breast masses in ultrasound. Human-Centric Comput Inform Sci 5:12–28
Priego-Torres BM, Morillo DS, Granero MAF, Rojo MG (2020) Automatic segmentation of whole-slide H&E stained breast histopathology images using a deep convolutional neural network architecture. Expert Syst Appl 151:113387
Qi X, Zhang L, Chen Y, Pi Y, Chen Y, Lv Q, Yi Z (2019) Automated diagnosis of breast ultrasonography images using deep learning neural networks. Med Image Anal 52:185–198
Qian X, Patton EW, Swaney J, Xing Q, Zeng TH (2018). Machine learning on cataracts classification using SqueezeNet. In Proc. of 4th international conference on Universal Village, doi: https://doi.org/10.1109/UV.2018.8642133.
R Core Team. A language and environment for statistical computing. R foundation for statistical computing, Vienna, Austria, 2013, http://www.R-project.org/.
Rani VMK, Dhanekaran SS (2019) Classification of ultrasound breast cancer tumor images using neural learning and predicting the tumor growth rate. Multimed Tools Appl 79:16967–16985. https://doi.org/10.1007/s11042-019-7487-6
Ravishankar H, Sudhakar P, Venkataramani R, Thiruvenkadam S, Annangi P, Babu N, Vaidya V (2016) Understanding the mechanisms of deep transfer learning for medical images. In: Carneiro G et al (eds) Deep learning and data labeling for medical applications, vol 10008. Springer, Cham, pp 188–196
Rawat J, Singh A, Bhadauria HS, Virmani J, Devgun JS (2017) Classification of acute lymphoblastic leukaemia using hybrid hierarchical classifiers. Multimed Tools Appl 76(18):19057–19085
Rawat J, Singh A, Bhadauria HS, Virmani J, Devgun JS (2018) Leukocyte classification using adaptive neuro-fuzzy inference system in microscopic blood images. Arab J Sci Eng 43(12):7041–7058
Rehman M, Khan SH, Abbas Z, Rizvi SMD (2019). Classification of diabetic retinopathy images based on customized CNN architecture. In Proc. of Amity International conference on artificial intelligence, pp. 244–248.
Sharma A, Kumar R (2019) A constrained framework for context-aware remote E-healthcare (CARE) services. Trans Emerg Telecommun Technol. https://doi.org/10.1002/ett.3649
Sharma A, Ansari MD, Kumar R (2017). A comparative study of edge detectors in digital image processing. In Proc. of 4th international conference on signal processing, computing and control, doi: https://doi.org/10.1109/ISPCC.2017.8269683.
Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298
Simonyan K, Zisserman A (2014). Very deep convolutional networks for large-scale image recognition, arXiv: 1409.1556.
Singh BK (2019) Determining relevant biomarkers for prediction of breast cancer using anthropometric and clinical features: a comparative investigation in machine learning paradigm. Biocybernetics and Biomedical Engineering 39(2):393–409
Spampinato C, Palazzo S, Giordano D, Aldinucci M, Leonardi R (2017) Deep learning for automated skeletal bone age assessment in X-ray images. Med Image Anal 36:41–51
Su Y, Wang Y, Jiao J, Guo Y (2011) Automatic detection and classification of breast tumors in ultrasonic images using texture and morphological features. Open Med Inform J 5(Suppl1-M3):26–37
Sudharshan PJ, Petitjean C, Spanhol F, Oliviera LE, Huette L, Honeine P (2019) Multiple instance learning for histopathological breast cancer image classification. Expert Syst Appl 117:103–111
Sun J, Wan C, Vheng J, Yu F, Liu J (2017) Retinal image quality classification using fine-tuned CNN. In: Cardoso M et al (eds) Fetal, infant and ophthalmic medical image analysis, vol 10554. Springer, Cham, pp 126–133
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015). Going deeper with convolutions. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9.
Tajbaksh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312
Tohno E, Ueno E, Watanabe H (2009) Ultrasound screening of breast cancer. Breast Cancer 16:18–22
Ubeyli ED (2009) Adaptive neuro-fuzzy inference systems for automatic detection of breast cancer. J Med Syst 33:353–358
Vesal S, Ravikumar N, Davari AA, Ellmann S, Maier A (2018) Classification of breast cancer histology images using transfer learning. In: Campilho A et al (eds) Image analysis and recognition, vol 10882. Springer, Cham, pp 812–819
Vianna VP (2018). Study and development of a computer-aided diagnosis system for classification of chest X-ray images using convolutional neural networks pre-trained for ImageNet and data augmentation, arXiv preprint arXiv: 1806.00839.
Virmani J, Kumar V, Kalra N, Khandelwal N (2013) SVM-based characterization of liver ultrasound images using wavelet packet texture descriptors. J Digit Imaging 26(3):530–543
Wan S, Liang Y, Zhang Y (2018) Deep convolutional neural networks for diabetic retinopathy detection by image classification. Comput Electr Eng 72:274–282
Wang J, Perez L (2017). The effectiveness of data augmentation in image classification using deep learning. arXiv: 1712.04621v1.
Weiss N, Kost H, Homeyer A (2018). Towards interactive breast tumor classification using transfer learning. In: Campilho a. et al. (eds.), ICIAR 2018, vol. 10882, pp. 727-736, springer.
Wu WJ, Moon WK (2008) Ultrasound breast tumor image computer-aided diagnosis with texture and morphological features. Acad Radiol 15(7):873–880
Wu WJ, Lin SW, Moon WK (2015) An artificial immune system-based support vector machine approach for classifying ultrasound breast tumor images. J Digit Imaging 28(5):576–585
Xiao T, Liu L, Li K, Qin W, Yu S, Li Z (2018) Comparison of transferred deep neural networks in ultrasonic breast masses discrimination. BioMed Res Int. https://doi.org/10.1155/2018/4605191
Xie X, Shi F, Niu J, Tang X (2018) Breast ultrasound image classification and segmentation using convolutional neural networks. In: Hong R et al (eds) Pacific rim conference on multimedia, vol 11166. Springer, Cham, pp 200–211
Yoshida H, Casalino DD, Keserci B, Coskun A, Ozturk O, Savranlar A (2003) Wavelet-packet based texture analysis for differentiation between benign and malignant liver tumors in ultrasound images. Phys Med Biol 48(22):3735–3753
Yu L, Chen H, Dou Q, Heng PA (2016) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4):994–1004
Zadeh LA (1975) The concept of a linguistic variable and its application to approximate reasoning-I. Inf Sci 8(3):199–249
Zadeh LA (1975) The concept of a linguistic variable and its application to approximate reasoning-II. Inf Sci 8(4):301–357
Zadeh LA (1975) The concept of a linguistic variable and its application to approximate reasoning-III. Inf Sci 9(1):43–80
Zakeri FS, Behnam H, Ahmadinejad N (2012) Classification of benign and malignant breast masses based on shape and texture features in sonography images. J Med Syst 36(3):1621–1627
Zeimarani B, Costa MGF, Nyrani NZ, Filho CFFC (2019) A novel breast tumor classification in ultrasound images, using deep convolutional neural network. In: Costa RF, Machado J, Alvarenga A (eds) XXVI Brazilian congress on biomedical engineering, vol 70/2. Springer, Singapore, pp 89–94
Zhang E, Seiler S, Chen M, Lu W, Gu X (2019). BIRADS features oriented semi-supervised deep learning for breast ultrasound computer-aided diagnosis. arXiv:1904.01076.
Zhou Z, Shin J, Zhang L, Gurudu S, Gotway M, Liang J (2017). Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In Proc. of IEEE conference on computer vision and Pattern Recognition, pp. 7340–7351.
Acknowledgments
The authors would like to thank Dr. Shruti Thakur, Kamla Nehru Hospital, Shimla for grading the filtered images and marking the tumor contours in the original ultrasound images. The authors would also like to thank Director, Thapar Institute of Engineering and Technology, Patiala and Director, CSIR-CSIO, Chandigarh for constant patronage and support in carrying out the present research.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kriti, Virmani, J. & Agarwal, R. Deep feature extraction and classification of breast ultrasound images. Multimed Tools Appl 79, 27257–27292 (2020). https://doi.org/10.1007/s11042-020-09337-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09337-z