CN117237328A - Bladder cancer TNM staging method and system based on digital pathological section - Google Patents
Bladder cancer TNM staging method and system based on digital pathological section Download PDFInfo
- Publication number
- CN117237328A CN117237328A CN202311341209.XA CN202311341209A CN117237328A CN 117237328 A CN117237328 A CN 117237328A CN 202311341209 A CN202311341209 A CN 202311341209A CN 117237328 A CN117237328 A CN 117237328A
- Authority
- CN
- China
- Prior art keywords
- bladder cancer
- image
- pathological section
- model
- pathological
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000001575 pathological effect Effects 0.000 title claims abstract description 151
- 206010005003 Bladder cancer Diseases 0.000 title claims abstract description 135
- 208000007097 Urinary Bladder Neoplasms Diseases 0.000 title claims abstract description 135
- 201000005112 urinary bladder cancer Diseases 0.000 title claims abstract description 135
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000012549 training Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 19
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 238000005070 sampling Methods 0.000 claims description 37
- 230000002159 abnormal effect Effects 0.000 claims description 14
- 238000013136 deep learning model Methods 0.000 claims description 13
- 206010009944 Colon cancer Diseases 0.000 claims description 12
- 238000003709 image segmentation Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 208000001333 Colorectal Neoplasms Diseases 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 6
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 5
- 238000002372 labelling Methods 0.000 abstract description 5
- 210000001519 tissue Anatomy 0.000 description 40
- 238000010586 diagram Methods 0.000 description 12
- 238000001914 filtration Methods 0.000 description 7
- 238000013103 analytical ultracentrifugation Methods 0.000 description 6
- 230000007170 pathology Effects 0.000 description 5
- 206010028980 Neoplasm Diseases 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000003745 diagnosis Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 210000000981 epithelium Anatomy 0.000 description 3
- 238000003711 image thresholding Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 2
- 201000011510 cancer Diseases 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 210000004877 mucosa Anatomy 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 206010054094 Tumour necrosis Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 210000001165 lymph node Anatomy 0.000 description 1
- 210000004698 lymphocyte Anatomy 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000001338 necrotic effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 210000002460 smooth muscle Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000005748 tumor development Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a bladder cancer TNM staging method and system based on digital pathological sections, which relate to the technical field of medical image processing and comprise the following steps: identifying a tissue region of the bladder cancer pathological section image; pre-training the ResNet-50 model to enable the ResNet-50 model to have the feature extraction capability of bladder cancer pathological section images; dividing a tissue region of a bladder cancer pathological section image into a plurality of image blocks; feature extraction is carried out on each segmented image block by utilizing a pre-trained ResNet-50 model, so that each image block feature is obtained, and the pathological section feature of the bladder cancer is obtained according to each image block feature; according to the pathological section characteristics of the bladder cancer, predicting the TNM stage of the bladder cancer. The method can solve the problems of subjectivity, difficulty in manual labeling, time consumption and the like of the traditional bladder cancer TNM staging method, thereby realizing more accurate, more consistent and more efficient bladder cancer TNM staging.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a bladder cancer TNM staging method and system based on digital pathological sections.
Background
In the current medical field, TNM staging of bladder cancer (TNM is a staged form of tumor in oncology) plays a key role in cancer diagnosis. The traditional bladder cancer TNM staging methods suffer from several drawbacks, including but not limited to:
1. subjectivity and operator variability: traditional bladder cancer TNM staging methods often rely on subjective judgment of doctors, so that stage results among different doctors can be different, and the accuracy and consistency of stage are affected.
2. Difficulty of manual labeling: the traditional bladder cancer TNM staging method generally requires a large number of manual marks, and has a heavy workload on doctors. Meanwhile, due to the complexity of the bladder cancer pathology image, accurate manual labeling may be limited.
3. Time and effort consuming: the conventional bladder cancer TNM staging method requires a great deal of time and effort from a doctor to observe and analyze pathological sections, thereby reducing the efficiency of diagnosis.
The existing CLAM method (clustering constraint attention multi-instance learning method) realizes high-performance pathological section classification by a weak supervision learning mode when processing the digitalized pathological section, and the method utilizes the clustering constraint and attention mechanism when analyzing the pathological section, so that important characteristics in the section can be automatically learned and captured. Although the method can realize classification and analysis of pathological sections without a large number of manual labels, the accuracy and consistency of the method stage results still need to be improved.
To solve these problems, a new method for the TNM staging of bladder cancer based on digitized pathological sections is needed.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides a bladder cancer TNM (total length, width and length) staging method based on a digital pathological section, which can solve the problems of subjectivity, difficulty in manual labeling, time consumption and the like of the traditional bladder cancer TNM staging method, thereby realizing more accurate, consistent and efficient bladder cancer TNM staging.
In order to achieve the above purpose, the present invention adopts the following technical scheme, including:
a bladder cancer TNM staging method based on digitized pathological sections, comprising the steps of:
s1, identifying a tissue region of a bladder cancer pathological section image;
s2, pre-training the deep learning model to enable the deep learning model to have the feature extraction capability of pathological section images;
s3, dividing a tissue region of the bladder cancer pathological section image into a plurality of image blocks;
s4, extracting features of each segmented image block by utilizing a pre-trained deep learning model to obtain features of each image block, and obtaining pathological section features of bladder cancer according to the features of each image block;
s5, predicting TNM stage of the bladder cancer according to pathological section characteristics of the bladder cancer.
Preferably, in step S1, an image of a pathological section of bladder cancer is obtained, and downsampling is performed on the image by selecting downsampling size to obtain a sampling map of the pathological section of bladder cancer; identifying the tissue outline of the pathological section based on the sampling graph, and extracting the tissue area of the bladder cancer pathological section image; and correcting the sampling image of the pathological section of the bladder cancer for the abnormal pathological section image of the bladder cancer.
Preferably, the deep learning model uses a ResNet-50 model.
Preferably, in step S2, the ResNet-50 model is trained using the ImageNet dataset, the human colorectal cancer and healthy tissue image dataset, and the bladder pathological section image dataset.
Preferably, the ResNet-50 model comprises, in order in the input to output direction: the convolution layer, the maximum pooling layer, 4 residual convolution blocks used for extracting the characteristics, and finally, the average pooling layer is utilized and matched with the full connection layer to output a classification result;
in step S4, when feature extraction is carried out on each segmented image block by utilizing the pre-trained ResNet-50 model, the last residual block and the full connection layer of the ResNet-50 model are removed so as to obtain the image features extracted in the middle process;
in step S4, the features of all the image blocks are overlapped and fused to obtain the pathological section features of the bladder cancer.
Preferably, in step S3, the pathological slice image of bladder cancer is segmented to obtain a plurality of equal-sized tiles; based on the identified tissue region of the bladder cancer pathological section image, whether each image block belongs to the tissue region or not is judged, and the image block belonging to the tissue region is extracted from the bladder cancer pathological section image.
Preferably, in step S5, the TNM stage of bladder cancer is predicted using a CLAM model.
The invention also provides a bladder cancer TNM staging system based on the digital pathological section, which is suitable for the bladder cancer TNM staging method based on the digital pathological section, and comprises the following steps: the device comprises an image reading module, a ResNet-50 model pre-training module, an image segmentation module, a feature extraction module and a stage prediction module.
The image reading module is used for acquiring a sampling image of the pathological section of the bladder cancer, identifying a tissue area of the pathological section and processing abnormal pathological section images of the bladder cancer;
the ResNet-50 model pre-training module is used for pre-training the ResNet-50 model and comprises the steps of training the ResNet-50 model by utilizing a data set and optimizing the structure of the ResNet-50 model;
the image segmentation module is used for carrying out image segmentation and segmenting a tissue region of the bladder cancer pathological section image into a plurality of image blocks;
the feature extraction module is a pre-trained ResNet-50 model and is used for extracting bladder cancer pathological section features from the segmented image blocks;
the stage prediction module is used for predicting TNM stages of the bladder cancer according to pathological section characteristics of the bladder cancer.
The invention has the advantages that:
(1) The bladder cancer TNM staging method based on the digital pathological section can solve the problems of subjectivity, difficulty in manual marking, time consumption and the like of the traditional bladder cancer TNM staging method, and has the beneficial effects of reducing marking cost, improving time efficiency and the like in the bladder cancer TNM staging field, thereby realizing more accurate, consistent and efficient bladder cancer TNM staging.
(2) According to the invention, by introducing a machine learning technology and an artificial intelligence technology, key features are automatically extracted from the digital pathological section, and the influence of subjective judgment of doctors and operator difference on the staging result is reduced, so that the accuracy and consistency of the bladder cancer TNM staging are improved.
(3) The invention utilizes the CLAM model, not only can automatically learn and capture important features in the slice, but also can effectively classify and analyze the slice under the condition of a small quantity of marked samples, thereby reducing the requirement on a large quantity of manual marking, further reducing the burden of doctors on tedious manual marking and reducing the marking cost in the stage process.
(4) According to the invention, the bladder cancer TNM stage can be rapidly and accurately evaluated by an automatic method, so that the stage efficiency is improved, and the time and effort of doctors are reduced.
(5) The method based on the CLAM design can be effectively applied to the TNM stage of the bladder cancer, provides a more accurate and rapid stage result for a clinician, is hopefully popularized and applied to clinical practice, and promotes the early diagnosis and treatment of the bladder cancer.
Drawings
FIG. 1 is a flow chart of a method for TNM staging of bladder cancer based on digitized pathological sections.
Fig. 2 is a schematic view of downsampling of bladder cancer pathological section images.
Fig. 3 is a schematic diagram of the result of median filtering treatment on a bladder cancer pathological section sampling chart.
Fig. 4 is a schematic diagram of the result of image thresholding of a sample image of a pathological section of bladder cancer.
Fig. 5 is a schematic diagram of contour recognition of a pathological section sampling map of bladder cancer.
Fig. 6 is a schematic diagram of abnormal outline recognition of a sample image of a pathological section of bladder cancer.
Fig. 7 is a schematic diagram after abnormal contour recognition processing of a pathological section sample map of bladder cancer.
FIG. 8 is a diagram of pre-training of the ResNet-50 model.
Fig. 9 is a schematic diagram of segmentation and feature extraction of bladder cancer pathological section images.
Fig. 10 is a schematic diagram of the architecture of the CLAM model.
Fig. 11 is a graph showing AUC of different data sets for predicting bladder cancer TNM stage.
Fig. 12 is a schematic representation of ROC curves of different data sets predicting TNM staging.
Fig. 13 is a block diagram of a bladder cancer TNM staging system based on digitized pathological sections.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention provides a bladder cancer TNM (tumor necrosis machine) staging method based on a digital pathological section, which aims to solve the subjectivity and operator difference problems of the traditional staging method, reduce the manual labeling burden and improve the accuracy and efficiency of staging. By introducing a clustering constraint attention multi-instance learning (CLAM) design, the invention can automatically extract key features from digital pathological sections, and realize accurate stage of bladder cancer, thereby providing a more accurate, consistent and reliable clinical diagnosis tool for doctors.
As shown in fig. 1, the method for classifying bladder cancer TNM based on digital pathological sections of the present invention comprises the following steps:
s1, acquiring a sampling image of a pathological section of bladder cancer, identifying a tissue region of the pathological section image of bladder cancer, and processing the abnormal pathological section image of bladder cancer;
the method comprises the steps of obtaining an image of a pathological section of the bladder cancer, and selecting a downsampling size to downsample the image to obtain a sampling image of the pathological section of the bladder cancer; identifying the tissue outline of the pathological section based on the sampling graph, and extracting the tissue area of the bladder cancer pathological section image; and correcting the sampling image of the pathological section of the bladder cancer for the abnormal pathological section image of the bladder cancer.
S2, pre-training a ResNet-50 model: training a ResNet-50 model by using an ImageNet data set, a human colorectal cancer and healthy tissue image data set and a bladder pathological section image data set, so that the ResNet-50 model has the feature extraction capability of bladder cancer pathological section images, the structure of the ResNet-50 model is optimized, and the applicability and the effectiveness of the ResNet-50 model in bladder cancer pathological section feature extraction are further improved by adjusting the structure of the ResNet-50 model;
s3, segmenting the bladder cancer pathological section image, and segmenting the tissue region of the bladder cancer pathological section image into a plurality of blocks based on the tissue region of the bladder cancer pathological section image;
s4, feature extraction is carried out on each segmented image block by utilizing a pre-trained ResNet-50 model to obtain each image block feature, and each image block feature is stacked and fused to obtain pathological section features of bladder cancer;
s5, predicting TNM stage of the bladder cancer by using a CLAM model according to pathological section characteristics of the bladder cancer.
The human colorectal cancer and healthy tissue image dataset is a public dataset comprising two datasets, NCT-CRC-HE-100K and CRC-VAL-HE-7K, which in this example are used as training dataset, CRC-VAL-HE-7K is used as test dataset, and the ResNet-50 model is trained using this dataset. The image dataset of human colorectal cancer and healthy tissue includes nine categories of fat, background, debris, lymphocytes, mucosa, smooth muscle, normal mucosa, stroma, cancer epithelium, all images being 224 x 224 pixels in size.
The bladder cancer image data set is a tissue image data set which is manually screened from 50 sections in digital pathological sections of II-IV phase of a patient diagnosis result, is divided, screened and classified and comprises 32740 images and is divided into nine categories of abnormal epithelium, normal epithelium tissue, necrotic tissue, lymph node, normal interstitium, abnormal interstitium, blood stain, muscle and unknown tumor. The training set and the test set are divided in such a way that half of each data is extracted, and all images have a size of 224×224 pixels.
As shown in fig. 13, a bladder cancer TNM staging system based on digitized pathological sections of the invention includes: the device comprises an image reading module, a ResNet-50 model pre-training module, an image segmentation module, a feature extraction module and a stage prediction module;
the image reading module is used for acquiring a sampling image of the pathological section of the bladder cancer, identifying a tissue area of the pathological section of the bladder cancer and processing the abnormal pathological section of the bladder cancer;
the ResNet-50 model pre-training module is used for pre-training the ResNet-50 model and comprises the steps of training the ResNet-50 model by utilizing a data set and optimizing the structure of the ResNet-50 model;
the image segmentation module is used for carrying out image segmentation and segmenting a tissue region of the bladder cancer pathological section image into a plurality of image blocks;
the feature extraction module is a pre-trained ResNet-50 model and is used for extracting bladder cancer pathological section features from the segmented image blocks;
the stage prediction module is a CLAM model and is used for predicting TNM stages of the bladder cancer according to pathological section characteristics of the bladder cancer.
The present invention generally requires processing of the data sets prior to utilizing each data set to meet the data format required by the deep learning model. In the subsequent TNM staging experiments of bladder cancer, the pathological section characteristics of bladder cancer need to be extracted, and the specific modes are as follows: the bladder cancer pathological section image is divided into a plurality of blocks, then a deep learning model is used for extracting block characteristics, and finally the plurality of block characteristics after stacking and fusing are used as bladder cancer pathological section characteristics for carrying out bladder cancer TNM stage prediction. Since each pathology image may be segmented into thousands to tens of thousands of tiles, and not all of one pathology image includes a tissue region, it is often necessary to determine the tissue region in the pathology image in order to save the loss of computer power. When determining a tissue region, pathological section images have the condition that part of the tissue region is polluted or affected by boundary shadow, which can lead researchers to be incapable of well identifying the tissue region. For this part of the pathological section image, the image is usually processed in a scaling mode, namely in a downsampling mode, and since the common computer equipment cannot process the image with ultra-high pixels, it is necessary to select a proper scaling and a proper size and re-identify the tissue area on the basis of the scaling and the size. After the tissue area is determined, feature extraction is carried out on the image blocks of the tissue area through a deep learning model, and the image blocks are used as a data base for subsequent processing.
The specific method for determining the tissue region in the pathological section image in this embodiment is as follows:
s11, opening pathological sections by using an OpenSlide library, opening pathological sections in a svs format by using an OpenSlide method, and obtaining a sampling graph of the pathological sections by combining a level_dimensions method for obtaining sampling levels, the level_dimensions method for obtaining different levels of picture sizes and a read_region method for sampling the pathological sections. FIG. 2 is a view of a bladder cancer pathological section taken from below;
s12, after a sampling image of the pathological section is obtained, the sampling image is processed by using a morphological method. The sampling image of the pathological section is a photo stored in an RGB format, and for convenience of processing, the sampling image is converted from the RGB format to the HSV format, and in this embodiment, conversion is performed by using a cvtdcolor method in an opencv library. After the sampling graph is converted into the HSV format, the noise in the sampling graph needs to be processed, and the sampling graph is processed by using a median filtering technology in the embodiment, and particularly, the sampling graph is processed by using an opencv library media blue method. The sample graph after median filtering is shown in fig. 3;
s13, thresholding, namely image thresholding, is carried out on the sampling image after median filtering processing so as to separate a tissue area and a background area of the sampling image, and the embodiment adopts a global Ojin method (maximum inter-class variance method) algorithm which can calculate the contrast value of the maximum class of pixel values, so that the sampling image is subjected to pixel segmentation to obtain a target and a background. In the embodiment, the sampling graph after median filtering processing is processed by using a threshold method and an OTSU method in an opencv library, and finally, images with distinct background and foreground are obtained. The image thresholded sampling graph is shown in FIG. 4;
s14, acquiring contour information of the sampling graph, including region contour and hole contour information of the sampling graph, by using the sampling graph after median filtering and pixel thresholding. In the embodiment, the contour information of the sampling map is obtained by using the findContours method in the opencv library, the information in the gray scale map is converted into the contour information, fig. 5 is a schematic diagram of contour recognition of the pathological section sampling map, and the extracted contour information comprises a region contour and a hole contour;
the recognition process is affected by boundary shadow or stain, so that the recognition abnormality occurs, and therefore, it is necessary to process and re-recognize the digital pathological section of the recognition abnormality. The pathological section boundary shadow is that the produced digital pathological section has problems due to the pathological section occupying the edge position of the slide; the pathological section is stained, and a pathologist or other digital section users use a marker pen to mark the pathological section, so that the marks are scanned in the process of manufacturing the digital pathological section later, and the stains of the pathological section are formed. Fig. 6 is a schematic diagram of abnormal contour recognition after two factors influence the digital pathological section.
In the stage of acquiring the downsampled image of the digital pathological section, the invention processes the sampled image of the contour recognition abnormal section by using graphic processing software to make the sampled image become an image favorable for contour recognition. Removing boundary shadow parts by using graphic processing software aiming at the boundary shadow, and then carrying out subsequent graphic processing; in the case of being affected by the stains, the stain areas are removed by using graphic processing software, the subsequent processing of converting the image format, median filtering, image thresholding and recognizing the image contour is performed, and finally the contour of the brand-new digital pathological section is obtained. Fig. 7 is a schematic diagram of the result of the contour recognition abnormal pathological section sample processed by the graphic processing software and the result of the redrawing of the contour.
As shown in FIG. 8, the ResNet-50 model was trained using the ImageNet dataset, the human colorectal cancer and healthy tissue image dataset, and the bladder pathology image dataset, resulting in training parameters for the three models, respectively. In a subsequent experiment, a tile of a pathological slice image of bladder cancer, 224 x 224 pixels in size, was input. Firstly, through a 7×7 convolution layer and a 3×3 maximum pooling layer, obtaining a 64×56×56 dimension characteristic; then, extracting features through 4 residual convolution blocks, wherein each residual block is formed by combining a convolution layer of 1×1 and a convolution layer of 3×3, and after the processing is finished, the output size of each residual convolution block is 256×56×56, 512×28×28, 1024×14×14, 2048×7×7 respectively; finally, a classification result is output by utilizing the averaging layer and matching with the full-connection layer, the size of the characteristics in the model is changed from 2048 multiplied by 1 to n multiplied by 1, and n is the category of the classification result.
In order to obtain features in the middle process when feature extraction is performed by using the ResNet-50 model, the last residual block of ResNet-50 and the full connection layer need to be removed. After the structure of ResNet-50 is changed, the 1024×14×14 features output from the third residual block are subjected to averaging and pooling, and finally 1024×1×1 features are obtained.
The method comprises the steps of segmenting a bladder cancer digital pathological section, extracting features of each image block by using a deep learning model, namely a ResNet-50 model, and superposing the features extracted from all the image blocks to obtain bladder cancer pathological section features. Firstly, dividing a block from a tissue region of a pathological section by combining the acquired contour information; then, processing the image block by using a pre-trained ResNet-50 model to acquire characteristic information of the image block; finally, the features of all the image blocks are fused into the features of pathological sections. As shown in fig. 9, the specific procedure is as follows:
s21, image segmentation is carried out, namely, a plurality of image blocks with the same size are obtained by segmenting the digital pathological section of the bladder cancer; and according to the identified tissue region and the tissue contour information, judging whether each image block belongs to the tissue region of the pathological section in sequence, and acquiring a sampling graph of the image block belonging to the tissue region of the pathological section from the pathological section. In this embodiment, according to the contour information of the pathological section, including the starting position of the contour and the range of the contour, the contour information is enlarged or reduced to be matched with the hierarchy of the processing block; then, generating a block coordinate matrix according to the range of the outline and the size of the block to be cut; finally, verifying whether each generated image block is in the outline range or not by combining an algorithm, and removing image block coordinates in the outline range;
s22, extracting block features: it is necessary to extract features of the tiles using a deep learning method. Before starting, the deep learning model needs to be loaded and corresponding model parameters are read, and the embodiment uses the torch deep learning framework to load the model. Similarly, the read_region method using openslide acquires the information of the tiles according to the coordinates and processing level of the tiles, and in order to fully utilize the computing power resources, the present embodiment uses a batch processing method to sequentially process the tiles. Specifically, a block with 224×224 size is converted into a one-dimensional vector through deep learning;
s23, feature summarization: for convenience of use, feature vectors extracted from each image block are sequentially saved to a local file as features of the digital pathological section.
As one of the indexes measured in the tumor development stage, TNM staging has a reliable guiding effect on the doctor's formulation of a treatment scheme, but due to the lack of a pathologist, pathological section diagnosis cannot be well applied clinically, so that it is very necessary to apply a deep learning method to the staging of TNM to alleviate the problem of insufficient pathologists. The invention utilizes the CLAM model to realize the prediction task of TNM stage II-IV, and adopts a ten-time cross verification mode when testing the performance of the model. For each cross-validated set, the bladder cancer digital slice dataset was randomly divided into a training set (50% of pathological slices), a validation set (50% of pathological slices). The core architecture of the CLAM model is shown in fig. 10.
The performance of this example for TNM staging of bladder cancer is shown in FIGS. 11 and 12. Wherein fig. 11 is an AUC (model evaluation index) of the model on the test set, showing the effect of TNM staging on the validation set using features extracted from the ResNet-50 model pre-trained by the ImageNet dataset, the human colorectal cancer and healthy tissue image dataset, i.e., NCT-CRC-HE dataset, and the bladder cancer image dataset, i.e., TCGA-BLCA dataset, respectively. 12a, 12b, 12c in FIG. 12 correspond to stage II, stage III, stage IV prediction tasks, respectively, and broken lines, dashed lines, and dotted lines in FIG. 12 represent ROC results after completion of TNM staging tasks for features extracted by the ResNet-50 model pre-trained on the ImageNet dataset, the human colorectal and healthy tissue image dataset, and the bladder cancer image dataset, respectively.
As can be seen from fig. 11, AUCs of ImageNet, NCT-CRC-HE, and TCGA-BLCA for TNM staging tasks were 0.640±0.0200, 0.646±0.0240, 0.592±0.0258, respectively, demonstrating that it is basically feasible to extract features of pathological electronic slices for TNM classification prediction based on a pre-trained ResNet-50 model.
As can be seen from FIG. 12, the AUC for the stage II prediction tasks are 0.703+ -0.0358, 0.693+ -0.0341, 0.664+ -0.0229, respectively, while the AUC for the stage III prediction tasks are 0.552+ -0.0332, 0.584+ -0.0273, 0.531+ -0.0372, respectively, and the AUC for the stage IV prediction tasks are 0.665+ -0.0315, 0.662+ -0.0256, 0.582+ -0.0253, respectively.
The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention, and any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A bladder cancer TNM staging method based on digitized pathological sections, comprising the steps of:
s1, identifying a tissue region of a bladder cancer pathological section image;
s2, pre-training the deep learning model to enable the deep learning model to have the feature extraction capability of pathological section images;
s3, dividing a tissue region of the bladder cancer pathological section image into a plurality of image blocks;
s4, extracting features of each segmented image block by utilizing a pre-trained deep learning model to obtain features of each image block, and obtaining pathological section features of bladder cancer according to the features of each image block;
s5, predicting TNM stage of the bladder cancer according to pathological section characteristics of the bladder cancer.
2. The method for classifying bladder cancer TNM based on digitized pathological sections according to claim 1, wherein in step S1, bladder cancer pathological section images are acquired, downsampling dimensions are selected to downsample the images, and a sampling map of bladder cancer pathological sections is obtained; identifying the tissue outline of the pathological section based on the sampling graph, and extracting the tissue area of the bladder cancer pathological section image; and correcting the sampling image of the pathological section of the bladder cancer for the abnormal pathological section image of the bladder cancer.
3. The method for the TNM staging of bladder cancer based on digitized pathological sections of claim 1 wherein said deep learning model uses a ResNet-50 model.
4. A method according to claim 3, wherein in step S2, the res net-50 model is trained using the ImageNet dataset, the human colorectal cancer and healthy tissue image dataset, and the bladder pathological section image dataset.
5. The method for TNM staging bladder cancer based on digitized pathological sections of claim 4 wherein the ResNet-50 model comprises, in order, in an input-to-output direction: the convolution layer, the maximum pooling layer, 4 residual convolution blocks used for extracting the characteristics, and finally, the average pooling layer is utilized and matched with the full connection layer to output a classification result;
in step S4, when feature extraction is carried out on each segmented image block by utilizing the pre-trained ResNet-50 model, the last residual block and the full connection layer of the ResNet-50 model are removed so as to obtain the image features extracted in the middle process;
in step S4, the features of all the image blocks are overlapped and fused to obtain the pathological section features of the bladder cancer.
6. The method for classifying bladder cancer TNM based on digitized pathological sections according to claim 1, wherein in step S3, the image of the pathological section of bladder cancer is segmented to obtain a plurality of equal-sized image blocks; based on the identified tissue region of the bladder cancer pathological section image, whether each image block belongs to the tissue region or not is judged, and the image block belonging to the tissue region is extracted from the bladder cancer pathological section image.
7. The method for TNM staging bladder cancer based on digitized pathological sections according to claim 1, characterized in that in step S5, TNM staging of bladder cancer is predicted by using a CLAM model.
8. A bladder cancer TNM staging system based on digitized pathological sections, adapted for use in a method of staging bladder cancer TNM based on digitized pathological sections as claimed in any one of claims 1 to 7, comprising: the device comprises an image reading module, a ResNet-50 model pre-training module, an image segmentation module, a feature extraction module and a stage prediction module.
The image reading module is used for acquiring a sampling image of the pathological section of the bladder cancer, identifying a tissue area of the pathological section and processing abnormal pathological section images of the bladder cancer;
the ResNet-50 model pre-training module is used for pre-training the ResNet-50 model and comprises the steps of training the ResNet-50 model by utilizing a data set and optimizing the structure of the ResNet-50 model;
the image segmentation module is used for carrying out image segmentation and segmenting a tissue region of the bladder cancer pathological section image into a plurality of image blocks;
the feature extraction module is a pre-trained ResNet-50 model and is used for extracting bladder cancer pathological section features from the segmented image blocks;
the stage prediction module is used for predicting TNM stages of the bladder cancer according to pathological section characteristics of the bladder cancer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311341209.XA CN117237328A (en) | 2023-10-17 | 2023-10-17 | Bladder cancer TNM staging method and system based on digital pathological section |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311341209.XA CN117237328A (en) | 2023-10-17 | 2023-10-17 | Bladder cancer TNM staging method and system based on digital pathological section |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117237328A true CN117237328A (en) | 2023-12-15 |
Family
ID=89091057
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311341209.XA Pending CN117237328A (en) | 2023-10-17 | 2023-10-17 | Bladder cancer TNM staging method and system based on digital pathological section |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117237328A (en) |
-
2023
- 2023-10-17 CN CN202311341209.XA patent/CN117237328A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109154978B (en) | System and method for detecting plant diseases | |
| CN111448582B (en) | System and method for single channel whole cell segmentation | |
| CN111986150B (en) | The method comprises the following steps of: digital number pathological image Interactive annotation refining method | |
| Valkonen et al. | Metastasis detection from whole slide images using local features and random forests | |
| Chekkoury et al. | Automated malignancy detection in breast histopathological images | |
| CN110472676A (en) | Stomach morning cancerous tissue image classification system based on deep neural network | |
| Veta et al. | Detecting mitotic figures in breast cancer histopathology images | |
| CN108537751B (en) | Thyroid ultrasound image automatic segmentation method based on radial basis function neural network | |
| Xu et al. | Computerized classification of prostate cancer gleason scores from whole slide images | |
| CN108010013A (en) | A kind of lung CT image pulmonary nodule detection methods | |
| CN110598030A (en) | Oracle bone rubbing classification method based on local CNN framework | |
| CN113393454A (en) | Method and device for segmenting pathological target examples in biopsy tissues | |
| Xu et al. | Using transfer learning on whole slide images to predict tumor mutational burden in bladder cancer patients | |
| CN112703531A (en) | Generating annotation data for tissue images | |
| CN113160175B (en) | Tumor lymphatic vessel infiltration detection method based on cascade network | |
| CN112990214A (en) | Medical image feature recognition prediction model | |
| CN104933723B (en) | Tongue image dividing method based on rarefaction representation | |
| KR20240012738A (en) | Cluster analysis system and method of artificial intelligence classification for cell nuclei of prostate cancer tissue | |
| WO2014006421A1 (en) | Identification of mitotic cells within a tumor region | |
| CN116468690B (en) | Subtype analysis system of invasive non-mucous lung adenocarcinoma based on deep learning | |
| Kiyuna et al. | Automatic classification of hepatocellular carcinoma images based on nuclear and structural features | |
| CN117237328A (en) | Bladder cancer TNM staging method and system based on digital pathological section | |
| CN113870194B (en) | Breast tumor ultrasonic image processing device with fusion of deep layer characteristics and shallow layer LBP characteristics | |
| Yancey | Deep Feature Fusion for Mitosis Counting | |
| CN111414956B (en) | Multi-example learning identification method for fuzzy mode in lung CT image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |