TWI745940B - Medical image analyzing system and method thereof - Google Patents

Abstract

The present invention provides a medical image analyzing system and method thereof, comprising: capturing a processed image having a segmentation label corresponding to organ location to generate a plurality of image patches, and training a deep-learning model with the plurality of image patches to obtain a prediction value and draw a corresponding operating characteristic curve to determine the threshold with which the invention determines whether the patches harbor cancer. The present invention can effectively improve the detection rate of pancreatic cancer.

Description

Medical image analysis system and method

The present invention relates to an image analysis system and method, in particular to a medical image analysis system and method.

In the current medical standards, pancreatic cancer is one of the cancers that are difficult to detect early, and once the tumor size exceeds 2 cm, the survival rate will drop significantly. In the prior art, computerized tomography (CT) imaging is currently the main method for detecting and evaluating pancreatic cancer, but the detection efficiency still depends on the radiologist’s personal experience. For example, when the tumor is less than 2 cm, about 40% Could not be detected. This reflects that the manual method of reading and judgment will be too subjective, and it is easy to cause misjudgment due to human factors.

Therefore, how to propose a medical image analysis system and method that can be used to identify pancreatic cancer to improve the identification rate, for example, is one of the issues to be solved urgently.

The main purpose of the present invention is to provide a medical image analysis system, including: an image preprocessing module for processing at least one image corresponding to an organ to generate at least one processed image, wherein the processed image is marked with a corresponding dirty Label for the position of the device; the block cutting module is used to target The processed image marked with the segmentation label is captured to generate a plurality of image blocks; the analysis module trains a deep learning model through the plurality of image blocks to obtain each of the plurality of image blocks A plurality of first predicted values corresponding to each block; and a threshold selection module, which draws a first curve for the plurality of first predicted values, so as to determine whether each of the plurality of image blocks is determined from the first curve Has the first threshold of cancer.

Another object of the present invention is to provide a medical image analysis method, including: processing at least one image corresponding to an organ to generate at least one processed image, wherein the processed image is marked with a segmentation label corresponding to the position of the organ; The processed image marked with the segmentation label is captured to generate a plurality of image blocks; a deep learning model is trained by the plurality of image blocks to obtain the corresponding corresponding to each of the plurality of image blocks A plurality of first predicted values; and a first curve is drawn for the plurality of first predicted values, so as to determine from the first curve a first threshold for judging whether each of the plurality of image blocks has cancer.

In the aforementioned medical image analysis system and method, the block cutting module uses a square sub-region to capture along the x-axis and y-axis of the processed image marked with the segmentation label to generate the plurality of images Block.

In the aforementioned medical image analysis system and method, the plurality of image blocks do not overlap each other or partially overlap each other.

In the aforementioned medical image analysis system and method, the ratio of the overlapping area of the plurality of image blocks to each other ranges from 20% to 80%.

In the aforementioned medical image analysis system and method, the analysis module uses a convolutional neural network to train the deep learning model.

In the aforementioned medical image analysis system and method, the first curve is the receiver's operating characteristic curve, and the first threshold is the threshold corresponding to the maximum value of the Youden Index.

In the aforementioned medical image analysis system and method, the image is a two-dimensional image or a three-dimensional image of computed tomography or magnetic resonance imaging.

The aforementioned medical image analysis system and method further includes a computer-aided diagnosis module for inputting at least one patient image to the image preprocessing module and the block cutting module to generate a plurality of patient image blocks And input the plurality of patient image blocks into the deep learning model to obtain a plurality of first predictive values corresponding to each of the plurality of patient image blocks.

In the aforementioned medical image analysis system and method, the computer-aided diagnosis module further enables the threshold selection module to calculate corresponding values for the plurality of first predictive values corresponding to each of the plurality of patient image blocks. At least one second predictive value of at least one patient image, and a second curve is drawn according to the at least one second predictive value to determine from the second curve a second threshold for judging whether the at least one patient image has cancer .

In the aforementioned medical image analysis system and method, the at least one second predictive value is generated after the plurality of first predictive values corresponding to each of the plurality of patient image blocks are judged by the first threshold The ratio of the number of patient image blocks with cancer in the at least one patient image to the total number of the plurality of patient image blocks.

In the aforementioned medical image analysis system and method, the second curve is the receiver's operating characteristic curve, and the second threshold is the threshold corresponding to the maximum value of the Youden Index.

In summary, the medical image analysis system and method of the present invention have higher sensitivity than radiologists in identifying pancreatic cancer, which means that the medical image analysis system and method of the present invention are effective Assisting radiologists to reduce their clinical missed diagnosis rate, especially in areas less than 2 cm The size of the tumor can be effectively improved when the tumor is less than 2 cm, about 40% of the cases cannot be detected.

1: Medical image analysis system

11: Medical image analysis device

111: Image preprocessing module

112: Block cutting module

113: Analysis Module

114: Threshold selection module

115, 122: processing unit

116, 123: communication unit

117, 124: storage unit

118, 125: display unit

12: Computer device

121: Computer Aided Diagnosis Module

13: Internet

2: image

2’: Processing images

21: Organs

22: Split label

24: Square sub-area

23, 231, 232, 233, 25, 251, 252, 253: image block

254, 255, 256: overlapping part

30: Deep learning model

31: Convolutional Neural Network

311: Convolutional layer

312: Pooling layer

313: Fully Connected Layer

40: receiver operating characteristic curve

D1, D2, D1’, D2’: distance

S1~S5: steps

Figure 1A is a schematic diagram of the first embodiment of the medical image analysis system of the present invention.

Figure 1B is a schematic diagram of the second embodiment of the medical image analysis system of the present invention.

Figure 1C is a schematic diagram of the third embodiment of the medical image analysis system of the present invention.

Figure 2A is a schematic diagram of the computer tomography image of the training image used in the medical image analysis system of the present invention.

Figures 2B to 2D are schematic diagrams of the simplification of Figure 2A and image preprocessing and image block generation.

Figure 3 is a schematic diagram of training a deep learning model in the medical image analysis system of the present invention.

Figure 4 is a schematic diagram of the receiver operating characteristic curve drawn by the medical image analysis system of the present invention.

Figure 5 is a schematic flow diagram of the medical image analysis method of the present invention.

The following specific examples illustrate the implementation of the present invention, and those skilled in the art can easily understand the other advantages and effects of the present invention from the contents disclosed in this specification, and can also use other different specific embodiments. To implement or apply.

Figure 1A is a schematic diagram of the first embodiment of the medical image analysis system of the present invention. The medical image analysis system 1 may include a medical image analysis device 11 and a computer device 12 electrically connected to the medical image analysis device 11, wherein, The medical image analysis device 11 and the computer device 12 communicate through a wired or wireless network 13.

The medical image analysis device 11 includes an image preprocessing module 111, a block cutting module 112, an analysis module 113, and a threshold selection module 114, and includes a processing unit 115, a communication unit 116, and a storage unit 117. The communication unit 116 and the storage unit 117 are coupled to the processing unit 115. In addition, the medical image analysis device 11 may be, for example, a mobile phone, a tablet computer, a notebook computer, a desktop computer, a server, or a cloud server, and the present invention is not limited to this. In addition, the medical image analysis device 11 may also include a display unit (not shown) such as a screen or a display.

In this embodiment, the processing unit 115 may be a central processing unit (Central Processing Unit, CPU), a microprocessor (Microprocessor), a graphics processing unit (Graphics Processing Unit, GPU), or an Application Specific Integrated Circuit (Application Specific Integrated Circuit). , ASIC). The communication unit 116 can support signal transmission of various mobile communication systems (such as GSM, PHS, CDMA, WCDMA, LTE, WiMAX, 4G, 5G, etc.), Wi-Fi systems, Bluetooth systems, or Ethernet. element. The storage unit 117 can be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, floppy disk ( A combination of the above components of soft disk, database or similar components. However, the present invention is not limited to this.

In this embodiment, the image preprocessing module 111, the block cutting module 112, the analysis module 113, and the threshold selection module 114 can be code fragments, software or firmware stored in the storage unit 117, respectively. It can be executed by the processing unit 115. However, the present invention is not limited to this. The medical image analysis device 11 The internal image preprocessing module 111, block cutting module 112, analysis module 113, and threshold selection module 114 can also be implemented using other hardware or a combination of software and hardware.

The computer device 12 may include a computer-aided diagnosis module 121, and also includes a processing unit 122, a communication unit 123, a storage unit 124, and a display unit 125. In this embodiment, the processing unit 122, the communication unit 123, and the storage unit 124 may be the same or similar components as the above-mentioned processing unit 115, the communication unit 116, and the storage unit 117, and will not be repeated here. The computer-aided diagnosis module 121 is also a code fragment, software, or firmware stored in the storage unit 124. It can also use other hardware or a combination of software and hardware, and can be executed by the processing unit 122. In addition, the computer device 12 can also be, for example, a mobile phone, a tablet computer, a notebook computer, or a desktop computer. The display unit 125 can be a screen or a display, but the invention is not limited thereto.

Please refer to FIG. 1B again, which is a schematic diagram of the second embodiment of the medical image analysis system of the present invention. The second embodiment is different from the aforementioned first embodiment only in that the computer-aided diagnosis module 121 is located in the medical image analysis device 11 instead of the computer device 12. In this way, all operations can be concentrated on the medical image analysis device 11, and the computer device 12 can become a device that simply receives output from the medical image analysis device 11 for display, so that the computer device 12 does not require higher-level hardware.

Please refer to Figure 1C again, which is a schematic diagram of the third embodiment of the medical image analysis system of the present invention. The difference between the third embodiment and the foregoing second embodiment is that the medical image analysis system of the present invention can only include a medical image analysis device without a computer device. The medical image analysis device 11 of the medical image analysis system of the present invention can be a computer tomography device or a magnetic computer in addition to the aforementioned mobile phone, tablet computer, notebook computer, desktop computer, server or cloud server. The resonance imaging device, that is, the medical image analysis system of the present invention can be installed in a computer tomography device or a magnetic resonance imaging device, but the present invention is not limited thereto.

The detailed technical content of the modules used in the medical image analysis system shown in Figures 1A to 1C will be described below.

FIG. 2A is a schematic diagram of a computed tomography image of the image 2 used in the medical image analysis system of the present invention. The organ 21 may be, for example, a pancreas. Fig. 2B is a simplified schematic diagram of Fig. 2A, and the simplification in Fig. 2B is only for convenience of description and does not impose any limitation on the present invention. Please refer to FIGS. 2A and 2B together. The image preprocessing module 111 is used to process at least one image 2 corresponding to an organ 21 to generate at least one processed image 2', wherein the processed image 2'is marked with a corresponding The segmentation label 22 of the position of the organ 21. This segmentation label 22 may generally be referred to as a region of interest (ROI). In this embodiment, the image preprocessing module 111 processes the image 2 by reconstruction, expansion, windowing, and normalization. In detail, the image 2 is a two-dimensional image or a three-dimensional image of computed tomography or magnetic resonance imaging. Take the two-dimensional images of computed tomography as an example. Generally, patients will have multiple two-dimensional images of computed tomography, and they must first use linear interpolation and nearest-neighbor interpolation. ) The image is reconstructed into slices of uniform thickness (for example, 5mm), where the linear interpolation method can be used for the entire image, and the nearest neighbor interpolation method can be used for the region of interest. Next, expand the label area of the corresponding organ 21 in image 2, for example, increase the range of the organ 21 outward by 3x3 pixels on the xy plane, so that the label area of the corresponding organ 21 in image 2 is enlarged, and this expanded The label area will correspond to the segmentation label 22 in the subsequent processed image 2'. The purpose of expanding the label area is to prevent the original label area from not completely corresponding to the organ 21, and to increase the judgment information of the module. After that, the image 2 is windowed, for example, the window width and the window level of the image 2 are set to 250 and 75 Hounsfield (HU) units, respectively. Finally, image 2 is normalized, that is, the pixel intensity value of image 2 is set between 0 and 1. After reconstruction and expansion The image 2 processed by windowing, windowing and normalization will become the processed image 2'. In one embodiment, there are multiple images for image 2, so there are multiple images for processing image 2', but the present invention is not limited to this.

Please refer to FIG. 2C. The block cutting module 112 is used to capture the processed image 2'marked with the segmentation label 22 to generate a plurality of image patches 23. In this embodiment, the block cutting module 112 uses a square sub-region 24 to sequentially capture along the x-axis and y-axis of the processed image 2'marked with the segmentation label 22 to generate a plurality of image blocks. twenty three. For example, the processed image 2'may have a size of 512x512 pixels, and the square sub-region 24 may have a size of 50x50 pixels. Generally speaking, the block cutting module 112 sets the square sub-areas 24 at the left and upper borders (i.e., the upper left corner) of the processed image 2'as the starting point, and captures to generate the image block 231. Then, the square sub-region 24 is moved to the right along the x-axis of the processed image 2'by a distance D1 (for example, 50 pixels), and then captured to generate an image block 232, and this step is repeated until the square sub-region 24 touches the processed image 2'to the right boundary. After that, the square sub-area 24 will move to the left border of the processed image 2'and 50 pixels from the upper border (that is, move down along the y axis by a distance D2), capture to generate an image block 233, and repeat until The square sub-region 24 touches the right border of the processed image 2'. Eventually, the square sub-region 24 will move to the right border and the lower border (ie, the lower right corner) of the processed image 2', and a plurality of image blocks 23 will be captured and generated. In this embodiment, the block cutting module 112 will capture each processed image 2'to generate a large number of image blocks 23, and the image blocks 23 do not overlap with each other. In addition, the above-mentioned processed image 2'and the size of the square sub-region 24 are only examples, and the present invention is not limited to this.

In another embodiment, the block cutting module 112 may not set the square sub-area 24 at the left and upper borders (that is, the upper left corner) of the processed image 2'as the starting point, but set the square sub-area 24 It is assumed that the vicinity of the segment label 22 in the processed image 2'is used as the starting point, but the present invention is not limited to this.

In another embodiment, as shown in FIG. 2D, the plurality of image blocks 25 generated by the block cutting module 112 may also partially overlap with each other. In detail, the block cutting module 112 uses a square sub-region 24 to sequentially capture along the x-axis and y-axis of the processed image 2'marked with the segmentation label 22 to generate a plurality of image blocks 25. In the above-mentioned embodiment, the square sub-region 24 can also be set at the left and upper borders (ie, the upper left corner) of the processed image 2'as a starting point, or set in the processed image 2'with a segmentation label 22 As a starting point. The difference between this embodiment and the above-mentioned embodiment lies in the moving distances D1' and D2' of the square sub-region 24. In the above embodiment, for example, the square sub-region 24 has a size of 50×50 pixels, and each movement is 50 pixels, so the captured image blocks 23 do not overlap with each other. In this embodiment, for example, the square sub-region 24 has a size of 50×50 pixels, but each movement is less than 50 pixels, so the captured image blocks 25 will partially overlap each other. As shown in Figure 2D, the square sub-region 24 only moves 25 pixels at a time (distance D1', D2'), so the image block 251 and the image block 252 will have overlapping parts 254, 256, and overlap each other in areas The ratio is 50%, and the image block 251 and the image block 253 will have overlapping parts 255 and 256, and the overlapping area ratio is 50%. The present invention does not limit the moving distance of the square sub-region 24 and the ratio of the overlapping area of the image blocks to each other, and the ratio of the overlapping area of the image blocks can be between 20% and 80% (optimally 50%). The purpose of image block overlap is to allow the deep learning model to be repeatedly tested when it is subsequently provided to the deep learning model for training. For example, the overlapping parts 254 and 255 are repeatedly tested twice, and the overlapping part 256 can be repeatedly tested three times. To improve accuracy.

The image 2 mentioned in each of the foregoing embodiments uses computer tomography images of diagnosed patients when training the model. In other words, the radiologist can clearly know the location of the tumor from the image 2, which enables the plurality of image blocks 23 and 25 generated by the block cutting module 112 to have cancer markers or non-cancer markers, respectively. As long as part of the tumor is covered in image blocks 23 and 25, that is Cancer markers can be marked, and the image blocks 23 and 25 must not cover the tumor at all before they can be marked with non-cancer markers. In one embodiment, if the plurality of image blocks 23 and 25 are marked as non-cancer markers and do not include the segmentation label 22, they may not be included in the subsequent training and prediction process, but the invention is not limited to this.

Please refer to Figure 3. After obtaining a plurality of image blocks 23 and 25, the analysis module 113 trains a deep learning model 30 through the plurality of image blocks 23 and 25 to obtain each plurality of image blocks. The plural first predicted values corresponding to 23 and 25 respectively. In this embodiment, the analysis module 113 can use a Convolutional Neural Network (Convolutional Neural Network) 31 to train the deep learning model 30. The convolutional neural network 31 generally includes a plurality of layers, such as a plurality of convolution layers (convolution layer) 311, a plurality of pooling layers (pooling layer) 312, a plurality of fully-connected layers (fully-connected layer) 313, etc., the present invention The number of layers of the convolutional layer 311, pooling layer 312, and fully connected layer 313 is not limited, and the convolutional layer 311, pooling layer 312, and fully connected layer 313 must be used at the same time. For example, only the convolutional layer 311 and the fully connected layer 313 can be used. The connection layer 313, or only two convolutional layers 311 and a fully connected layer 313, etc. are used.

In one embodiment, the convolutional neural network 31 used by the analysis module 113 may include two convolutional layers 311, followed by a rectified linear unit (ReLu) as the activation function, and then connected to the fully connected layer 313 Before, the pooling layer 312 is connected first. After this step, the plurality of image blocks 23, 25 can be flattened into an array, and then the above steps can be repeated until the plurality of image blocks 23, 25 are flattened to a single value. The following Table 1 shows the processing process of the convolutional neural network 31 when the size of the plurality of image blocks is 50 pixels x 50 pixels:

Table 1

In one embodiment, the convolutional neural network 31 uses weighted binary cross-entropy as the loss function to consider the difference between the number of blocks with cancer markers and the number of blocks with non-cancer markers. Balance. For example, the loss function can be:

其中，N為影像區塊總數，第i個影像區塊之標記為有癌症則y_i是1，其標記為非癌症則y_i為0。p(y_i)為第i個影像區塊的模型預測值。權重w₁及w₂則設為有癌症標記與有非癌症標記之影像區塊數量的反比，即w ₁：w ₂=N _{non-cancerous} ：N _cancerous 。

Among them, N is the total number of image blocks. If the i-th image block is marked as having cancer, y _i is 1, and if it is marked as non-cancer, y _i is 0. p(y _i ) is the model prediction value of the i-th image block. The weights w ₁ and w ₂ are set to be inversely proportional to the number of image blocks with cancer markers and those with non-cancer markers, that is, w ₁ : w ₂ = N _{non-cancerous} : N _cancerous .

In one embodiment, when the analysis module 113 trains the deep learning model 30 through a plurality of image blocks 23 and 25, it can set hyperparameters and callback mechanisms to optimize the model. Training effectiveness. For example, one of the super-constant-batch size is set to 2560, which means that the deep learning model 30 will receive 2560 image blocks in each iteration, but the present invention is not limited to this. In addition, there are two callback mechanisms: one is to reduce the learning rate of the convolutional neural network 31 for the deep learning model 30 when the analysis module 113 verifies that the loss function is in an unreduced state in the past ten iterations; the other is In order for the analysis module 113 to verify that the loss function is in a stable state in nearly forty iterations, the training of the deep learning model 30 by the convolutional neural network 31 is stopped. However, the present invention is not limited to the above-mentioned callback mechanism, for example, the number of iterations is not limited.

In this embodiment, after the analysis module 113 trains the deep learning model 30, the deep learning model 30 can give a corresponding first prediction value for each image block, and this first prediction value can be used for classification. For example, each image block can be classified as having cancer or not having cancer through a first threshold, and the method for determining the first threshold will be as follows.

The threshold selection module 114 can draw a first curve for a plurality of first predicted values, so as to determine from the first curve a first threshold for judging whether each of the plurality of image blocks 23 and 25 has cancer. In detail, the plurality of image blocks 23 and 25 respectively have corresponding plurality of first prediction values, and the plurality of first prediction values are determined after a specific threshold value (for example, if the first prediction value is greater than the specific threshold value, it is determined The image block has cancer), and the sensitivity corresponding to the specific threshold can be calculated (Sensitivity) and specificity (Specificity) and other statistical indicators, and any value between 0 and 1 (for example, 0.1, 0.2, 0.3, 0.4... etc.) is the possible value of the specific threshold, so that According to the multiple sensitivity and specificity calculated based on the possible values of multiple specific thresholds, the receiver operating characteristic curve (ROC) 40 shown in Figure 4 can be drawn, and the receiver operating characteristic curve (ROC) 40 can be drawn from the The operator obtains statistical indicators such as Area Under Receiver Operating Characteristic Curve (AUC) and a plurality of Youden index (Youden index) from the operating characteristic curve 40. Among them, a plurality of Youden indexes (the formula is: Youden index = Sensitivity-(1-specificity)) can be calculated from the sensitivity and specificity corresponding to each point in the receiver's operating characteristic curve 40. The present invention uses the threshold corresponding to the maximum value of the plurality of Youden exponents as the first threshold. When the first predicted value of the image block is greater than the first threshold, the image block can be classified as having cancer (Positive), and when the first predictive value of the image block is less than or equal to the first threshold, the image block can be classified as having no cancer (negative).

In one embodiment, when the deep learning model of the present invention determines that the image block has cancer and the radiologist also determines that the image block has cancer, it is defined as a true positive; the deep learning model of the present invention determines that the image block does not have cancer And when the radiologist also judges that the image block does not have cancer, it is defined as true negative; the deep learning model of the present invention judges that the image block has cancer, but when the radiologist judges that the image block does not have cancer, it is defined as false Positive; the deep learning model of the present invention judges that the image block does not have cancer, but when the radiologist judges that the image block has cancer, it is defined as a false negative. The aforementioned sensitivity and specificity are defined by the following formula: sensitivity=true positive/(true positive+false negative); specificity=true negative/(true negative+false positive).

The computer-aided diagnosis module 121 is used to input at least one patient image to the image preprocessing module 111 and the block cutting module 112 to generate a plurality of patient images corresponding to the at least one patient image Image blocks, and input the plurality of patient image blocks into the deep learning model 30 to obtain a plurality of first predictive values corresponding to each of the plurality of patient image blocks.

In detail, the computer-assisted diagnosis module 121 may specifically be computer-assisted detection/diagnosis tools (CAD tools) software, and the computer-assisted diagnosis module 121 may use the analysis module 113 of the medical image analysis device 11 The trained deep learning model 30 is used to assist the clinician in the diagnosis of the patient. For example, a clinician may first obtain a patient image of a patient to be analyzed, and input the patient image to the image preprocessing module 111 and the image preprocessing module 111 of the medical image analysis device 11 through the computer-aided diagnosis module 121 of the computer device 12 The block cutting module 112 generates a plurality of patient image blocks. The processing methods of the image preprocessing module 111 and the block cutting module 112 on the patient image are the same as the image 2 described above, and will not be repeated here. Then, input the plurality of patient image blocks of the patient into the deep learning model 30 to obtain a plurality of first predictive values corresponding to the plurality of patient image blocks. Then, the computer-aided diagnosis module 121 can make the threshold selection module 114 calculate at least one second prediction value corresponding to at least one patient image for each of the plurality of first predictive values corresponding to each of the plurality of patient image blocks. Predictive value. In one embodiment, a patient system corresponds to a second predictive value. If a patient has multiple patient images, it also corresponds to a second predictive value, and the second predictive value is composed of multiple patient images. The first predicted value is calculated, but the present invention is not limited to this. In this embodiment, after the plurality of first predicted values corresponding to each of the plurality of patient image blocks are determined by the first threshold determined by the threshold selection module 114, the threshold selection module 114 The plurality of patient image blocks are classified as having cancer (positive) or not having cancer (negative), and the second predictive value is based on counting the patient image areas classified as having cancer in the at least one patient image For example, the second predicted value may be the number of patient image blocks classified as having cancer in the at least one patient image and the total number of patient image blocks in the at least one patient image The ratio of the amount. In this embodiment, the computer-assisted diagnosis The segmentation module 121 can be used to input a single patient image to obtain a single second predictive value for subsequent clinicians to obtain the computer-aided diagnosis module 121 to determine whether the patient image has cancer. The computer-aided diagnosis module 121 can also input multiple patient images (ie, different patients) to obtain multiple second predictive values for subsequent drawing of the second curve to determine the second threshold, but the present invention does not use this Is limited. In addition, the aforementioned single patient image can be one or more two-dimensional CT images taken by a single patient, so that the second predictive value can correspond to a single patient image, and the single patient image can also be taken by a single patient One or more three-dimensional CT images, which can be processed by the image preprocessing module 111 to generate multiple two-dimensional patient images, so that the second predictive value can also correspond to multiple patient images (which can also directly correspond to To the patient), the present invention is not limited to this.

Then, the computer-aided diagnosis module 121 can make the threshold selection module 114 draw a second curve for the plurality of second predictive values, so as to determine from the second curve the second curve for judging whether each of the plurality of patient images has cancer. Threshold, where the second curve is the receiver operating characteristic curve, and the second threshold is the threshold corresponding to the maximum value of the Youden index. The drawing and determining method of the second curve and the second threshold are the same as the first curve and the first threshold, and will not be repeated here. After the second threshold is determined, the computer-aided diagnosis module 121 can determine the second predictive value corresponding to the patient image according to the second threshold to determine whether the patient image has cancer. For example, after a patient image is processed by each module and the deep learning model, the second predictive value obtained is 0.7. If the second threshold is 0.5, the computer-aided diagnosis module 121 can provide the patient image Has cancer results. If the second threshold is 0.8, it can give the result that the image of the patient does not have cancer.

Please refer to Figure 5, which shows a schematic flow diagram of the medical image analysis method of the present invention, and the medical image analysis method of the present invention can be used for the aforementioned medical image analysis device 11 Medical image analysis system 1. The technical content of the medical image analysis method of the present invention is the same as that of the aforementioned medical image analysis system, and will not be repeated here.

First, the medical image analysis method of the present invention can process images to generate processed images (step S1). That is, the medical image analysis method of the present invention is to order the image preprocessing module 111 of the medical image analysis device 11 to process at least one image 2 corresponding to an organ 21 to generate at least one processed image 2', wherein the processing The image 2'is marked with a segmentation label 22 corresponding to the position of the organ 21.

Next, the medical image analysis method of the present invention can generate a plurality of image blocks (step S2). That is, the block cutting module 112 of the medical image analysis device 11 is allowed to capture the processed image 2'marked with the segmentation label 22 to generate a plurality of image blocks 23 and 25.

After that, the medical image analysis method of the present invention trains a deep learning model (step S3). That is, the analysis module 113 of the medical image analysis device 11 is allowed to train a deep learning model 30 through a plurality of image blocks to obtain a plurality of first predicted values corresponding to each of the plurality of image blocks.

After that, the medical image analysis method of the present invention draws a first curve to determine the first threshold (step S4). That is, the threshold selection module 114 of the medical image analysis device 11 is caused to draw a first curve for a plurality of first predicted values, so as to determine from the first curve whether each of the plurality of image blocks 23, 25 has cancer. The first threshold.

Finally, after the medical image analysis method of the present invention has trained the deep learning model and determined the first threshold, a second curve can be drawn to determine the second threshold (step S5). That is, the computer-aided diagnosis module 121 of the computer device 12 electrically connected to the medical image analysis device 11 or the computer-aided diagnosis module 121 in the medical image analysis device 11 input at least one patient image to the image preprocessing module 111 and the block cutting module 112 to generate a plurality of patient image blocks, and input the plurality of patient image blocks To the deep learning model 30 to obtain a plurality of first predictive values corresponding to each of the plurality of patient image blocks. The computer-aided diagnosis module 121 further enables the threshold selection module 114 to calculate at least one second predictive value corresponding to at least one patient image for each of the plurality of first predictive values corresponding to each of the plurality of patient image blocks. , And draw a second curve according to at least one second predicted value to determine from the second curve a second threshold for judging whether the at least one patient image has cancer, where the second curve is the receiver’s operating characteristic curve, The second threshold is the threshold corresponding to the maximum value of the Youden Index. In this embodiment, the plurality of second predictive values are the at least one patient image generated after the plurality of first predictive values corresponding to each of the plurality of patient image blocks are judged by the first threshold value The ratio of the number of patient image blocks with cancer to the total number of the plurality of patient image blocks.

The deep learning model trained by the medical image analysis system and method of the present invention has better accuracy if the image block is 50 x 50 pixels (that is, the square sub-region is 50 x 50 pixels) for training. It can be clearly seen from Table 2 below that the sensitivity of the 50 x 50 pixel image block size is 91.1±2.0%, the specificity is 86.5±2.6%, the accuracy is 87.3±1.9%, and the area under the curve is 0.96±0.001. If the size of the image block is less than 50 x 50 pixels, the accuracy will decrease. This may be because the size of the image block is too small to contain enough information about the tumor and adjacent tissues; if the size of the image block is larger than 50 x 50 pixels, the stability of the accuracy rate is insufficient, which may be related to the excessively large image block size and the introduction of more noise. It can be seen that an image block size of 50 x 50 pixels is a better choice.

Table 2

In addition, the efficacy of the medical image analysis system and method of the present invention is confirmed as follows: first provide 244,859 image blocks with cancer markers of 295 pancreatic cancer patients, and provide 256 cancer-free patients with non-cancer The marked 1,216,715 image blocks are used as training materials for the deep learning model. The number of the above image blocks is randomly divided into a training group and a verification group. When the deep learning model trained from the image blocks of the training group distinguishes image blocks with cancer markers from those with non-cancer markers in the validation group, the area under the receiver operating characteristic curve (AUC) is as high as 0.96. The sensitivity, specificity and accuracy rate are 91.3%, 84.5% and 85.6% respectively. The area under the receiver operating characteristic curve (AUC) obtained by distinguishing cancer patients and cancer-free patients was 1.00, and the sensitivity, specificity and accuracy were 97.3%, 100% and 98.5%, respectively.

In addition, the comparison between the medical image analysis system and method of the present invention and radiologists proved as follows: 75 cancer patients' images and 64 cancer-free patients' images were used to test the above-mentioned training data The deep learning model shows that the medical image analysis system of the present invention predicts the area under the operating characteristic curve (AUC) of the receiver for distinguishing cancer patients and cancer-free patients to 0.997. Sensitivity, specificity and accuracy are 97.3%, 100.0% and 98.6% respectively. In the same image, the radiologist's sensitivity is only 94.4%.

In addition, 101 images of cancer patients and 88 images of cancer-free patients were used to test the above-mentioned deep learning model obtained from the training data. It can be seen that the medical image analysis system of the present invention distinguishes cancer patients from cancer-free patients. The area under the operating characteristic curve (AUC) of the patient’s image predictions reached 0.999, and the sensitivity, specificity and accuracy were 99.0%, 98.9% and 98.9%, respectively. In the same image, the radiologist's sensitivity is only 91.7%.

After calculating the tumor size of the aforementioned cancer patients and the sensitivity between the medical image analysis system and method of the present invention and the radiologist, the sensitivity of the present invention in detecting tumor sizes between 4 cm and 2-4 cm can be obtained. Both are 100%, while the sensitivity of radiologists in detecting tumor sizes larger than 4 cm is 100%, and the sensitivity of tumor sizes between 2-4 cm is only 90.8%. In addition, the sensitivity of the present invention in detecting tumor sizes less than 2 cm is 92.1%, while the sensitivity of radiologists in detecting tumor sizes less than 2 cm is only 89.5%.

In summary, the medical image analysis system and method of the present invention have higher sensitivity than radiologists in identifying pancreatic cancer, which means that the medical image analysis system and method of the present invention are effective Assisting radiologists to reduce their clinical missed diagnosis rate, especially in the case of tumors less than 2 cm in size, so it can effectively improve the general clinical situation when the tumor is less than 2 cm, about 40% of the cases cannot be detected.

The above-mentioned embodiments are only illustrative to illustrate the technical principles, features and effects of the present invention, and are not intended to limit the scope of the present invention. Anyone familiar with this technology can do the same without departing from the spirit and scope of the present invention. The above embodiment is modified and changed. Any use of the invention The equivalent modifications and changes to the teaching content should still be covered by the following patent application scope. The scope of protection of the rights of the present invention shall be as listed in the following patent scope.

1: Medical image analysis system

11: Medical image analysis device

111: Image preprocessing module

112: Block cutting module

113: Analysis Module

114: Threshold selection module

115, 122: processing unit

116, 123: communication unit

117, 124: storage unit

12: Computer device

121: Computer Aided Diagnosis Module

125: display unit

13: Internet

Claims (22)

A medical image analysis system, including: The image preprocessing module is used to process at least one image corresponding to an organ to generate at least one processed image, wherein the processed image is marked with a segmentation label corresponding to the position of the organ; The block cutting module is used to capture the processed image marked with the segmentation label to generate a plurality of image blocks; The analysis module trains a deep learning model by using the plurality of image blocks to obtain a plurality of first predicted values corresponding to each of the plurality of image blocks; and The threshold selection module draws a first curve for the plurality of first predicted values to determine from the first curve a first threshold for judging whether each of the plurality of image blocks has cancer. For example, the medical image analysis system described in claim 1, wherein the block cutting module uses a square sub-region to capture along the x-axis and y-axis of the processed image marked with the segmentation label, To generate the plurality of image blocks. According to the medical image analysis system described in item 2 of the scope of patent application, the plurality of image blocks do not overlap each other or partially overlap each other. According to the medical image analysis system described in item 3 of the scope of patent application, the ratio of the overlapping area of the plurality of image blocks to each other ranges from 20% to 80%. According to the medical image analysis system described in item 1 of the scope of patent application, the analysis module uses a convolutional neural network to train the deep learning model. According to the medical image analysis system described in claim 1, wherein the first curve is a receiver operating characteristic curve, and the first threshold value is the threshold value corresponding to the maximum value of the Youden index. The medical image analysis system described in item 1 of the scope of patent application, wherein the image is a two-dimensional image or a three-dimensional image of computed tomography or magnetic resonance imaging. For example, the medical image analysis system described in item 1 of the scope of patent application further includes a computer-aided diagnosis module for inputting at least one patient image to the image preprocessing module and the block cutting module to generate multiple diseases And input the plurality of patient image blocks into the deep learning model to obtain a plurality of first predictive values corresponding to each of the plurality of patient image blocks. For example, the medical image analysis system described in item 8 of the scope of patent application, wherein the computer-aided diagnosis module further makes the threshold selection module for each of the plurality of patient image blocks corresponding to the plurality of first The predicted value calculates at least one second predicted value corresponding to at least one patient image, and draws a second curve according to the at least one second predicted value, so as to determine from the second curve whether the at least one patient image has The second threshold of cancer. According to the medical image analysis system described in item 9 of the scope of patent application, wherein the at least one second predictive value corresponds to the plurality of first predictive values corresponding to each of the plurality of patient image blocks through the first valve The ratio of the number of patient image blocks with cancer in the at least one patient image generated after value judgment to the total number of the plurality of patient image blocks. According to the medical image analysis system described in item 9 of the scope of patent application, the second curve is a receiver operating characteristic curve, and the second threshold value is the threshold value corresponding to the maximum value of the Youden index. A medical image analysis method, including: Processing at least one image corresponding to an organ to generate at least one processed image, wherein the processed image is marked with a segmentation label corresponding to the position of the organ; Capture the processed image marked with the segmentation label to generate a plurality of image blocks; Training a deep learning model through the plurality of image blocks to obtain a plurality of first predicted values corresponding to each of the plurality of image blocks; and A first curve is drawn for the plurality of first predicted values, so as to determine from the first curve a first threshold for judging whether each of the plurality of image blocks has cancer. The medical image analysis method described in item 12 of the scope of patent application, wherein a square sub-region is captured along the x-axis and y-axis of the processed image marked with the segmentation label to generate the plurality of images Block. According to the medical image analysis method described in item 13 of the scope of patent application, the plurality of image blocks do not overlap each other or overlap each other. According to the medical image analysis method described in item 14 of the scope of patent application, the ratio of the overlapping area of the plurality of image blocks to each other ranges from 20% to 80%. The medical image analysis method described in item 12 of the scope of patent application, wherein the deep learning model is trained using a convolutional neural network. According to the medical image analysis method described in claim 12, the first curve is a receiver operating characteristic curve, and the first threshold value is the threshold value corresponding to the maximum value of the Youden index. The medical image analysis method described in item 12 of the scope of patent application, wherein the image is a two-dimensional or three-dimensional image of computed tomography or magnetic resonance imaging. The medical image analysis method described in item 12 of the scope of patent application further includes inputting at least one patient image to generate a plurality of patient image blocks, and inputting the plurality of patient image blocks into the deep learning model In order to obtain a plurality of first predictive values corresponding to each of the plurality of patient image blocks. For example, the medical image analysis method described in item 19 of the scope of patent application further includes calculating at least one corresponding to at least one patient image for the plurality of first predictive values corresponding to each of the plurality of patient image blocks. And draw a second curve according to the at least one second predicted value, so as to determine from the second curve a second threshold for judging whether the at least one patient image has cancer. According to the medical image analysis method described in claim 20, wherein the at least one second predictive value corresponds to the plurality of first predictive values corresponding to each of the plurality of patient image blocks through the first valve The ratio of the number of patient image blocks with cancer in the at least one patient image generated after the value judgment to the total number of the plurality of patient image blocks. According to the medical image analysis method described in item 20 of the scope of patent application, the second curve is a receiver operating characteristic curve, and the second threshold value is the threshold value corresponding to the maximum value of the Youden index.

Images