CN111145209B - Medical image segmentation method, device, equipment and storage medium - Google Patents
Medical image segmentation method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111145209B CN111145209B CN201911367572.2A CN201911367572A CN111145209B CN 111145209 B CN111145209 B CN 111145209B CN 201911367572 A CN201911367572 A CN 201911367572A CN 111145209 B CN111145209 B CN 111145209B
- Authority
- CN
- China
- Prior art keywords
- image
- segmentation
- cell nucleus
- tissue
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003709 image segmentation Methods 0.000 title claims abstract description 102
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000011218 segmentation Effects 0.000 claims abstract description 146
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 107
- 210000003855 cell nucleus Anatomy 0.000 claims abstract description 107
- 210000001519 tissue Anatomy 0.000 claims abstract description 102
- 239000003550 marker Substances 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000013527 convolutional neural network Methods 0.000 claims description 10
- 210000004940 nucleus Anatomy 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 8
- 238000002372 labelling Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 abstract description 32
- 230000000694 effects Effects 0.000 abstract description 7
- 230000006870 function Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 210000004027 cell Anatomy 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000004195 computer-aided diagnosis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 230000007170 pathology Effects 0.000 description 1
- 238000004393 prognosis Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/155—Segmentation; Edge detection involving morphological operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention discloses a medical image segmentation method, a device, equipment and a storage medium. The method comprises the following steps: obtaining a tissue image of a detected object, and extracting a preliminary segmentation image of a cell nucleus from the tissue image based on a preset image segmentation algorithm; and inputting the tissue image and the preliminary segmentation image into a trained cell nucleus segmentation model, and obtaining a target segmentation image of the cell nucleus according to an output result of the cell nucleus segmentation model. According to the technical scheme, the traditional image segmentation algorithm and the deep learning algorithm are combined, so that the requirement of the deep learning algorithm on the manually marked perfect data can be greatly relieved on the basis of fully playing the good segmentation performance of the deep learning algorithm, the better segmentation performance can still be achieved under the condition that only a small amount of manually marked perfect data is available, and the effect of accurately segmenting the nuclear image from the medical image with lower labor cost and time cost is achieved.
Description
Technical Field
The embodiment of the invention relates to the technical field of medical image processing, in particular to a medical image segmentation method, a device, equipment and a storage medium.
Background
Histopathological images are important components of pathology and are important reference factors for medical staff to diagnose and prognosis various diseases. In histopathological image analysis, different forms of cell nuclei are an important basis for whether a plurality of diseases occur or not, so how to accurately extract segmented images of cell nuclei from histopathological images is an important subject for computer-aided diagnosis and automatic analysis of medical images.
Medical image segmentation is the process of segmenting a medical image into regions of similar nature, a pixel-by-pixel classification task. Aiming at a nuclear segmentation task in medical image segmentation, the prior art mainly comprises a traditional image segmentation algorithm and a deep learning algorithm. However, the traditional image segmentation algorithm has the defects of low segmentation precision, easiness in over-segmentation and under-segmentation, and poor generalization capability on pathological images of different tissues; the deep learning algorithm needs a large amount of manually marked perfect data to perform model training, especially for a cell nucleus segmentation task, and the deep learning algorithm needs pixel-level fine marking, so that the labor cost and the time cost for acquiring the manually marked perfect data are high.
Disclosure of Invention
The embodiment of the invention provides a medical image segmentation method, a device, equipment and a storage medium, which are used for realizing the effect of segmenting a cell nucleus image from a tissue image.
In a first aspect, an embodiment of the present invention provides a medical image segmentation method, which may include:
obtaining a tissue image of a detected object, and extracting a preliminary segmentation image of a cell nucleus from the tissue image based on a preset image segmentation algorithm;
and inputting the tissue image and the preliminary segmentation image into a trained cell nucleus segmentation model, and obtaining a target segmentation image of the cell nucleus according to an output result of the cell nucleus segmentation model.
Optionally, extracting the preliminary segmented image of the nucleus from the tissue image based on a preset image segmentation algorithm may include:
extracting a marked point image of the cell nucleus from the tissue image;
and extracting a preliminary segmentation image of the cell nucleus from the tissue image according to the mark point image based on a watershed algorithm of the mark point.
Optionally, extracting the marker point image of the nucleus from the tissue image may include:
acquiring a binary image of the tissue image;
obtaining a foreground image and a background image of the cell nucleus according to the binary image;
and carrying out connected domain calculation on the foreground image, and carrying out subtraction calculation on the foreground image and the background image to obtain a marker point image of the cell nucleus.
Optionally, inputting the tissue image and the preliminary segmentation image into the trained nuclear segmentation model may include:
splicing the tissue image and the preliminary segmentation image to obtain a spliced image;
inputting the spliced image into a trained cell nucleus segmentation model, wherein the cell nucleus segmentation model comprises a semantic segmentation network model.
Optionally, the medical image segmentation method may further include:
acquiring a sample tissue image of a sample object, and extracting a sample preliminary segmentation image of a sample cell nucleus from the sample tissue image based on a preset image segmentation algorithm;
and training the original convolutional neural network model based on a plurality of groups of training samples by taking the sample tissue image, the sample preliminary segmentation image and the artificial labeling mask image of the sample cell nucleus as a group of training samples to obtain a cell nucleus segmentation model.
Alternatively, the preset image segmentation algorithm may include at least one of a threshold-based image segmentation algorithm, an edge-based image segmentation algorithm, a region-based image segmentation algorithm, and an active contour model-based image segmentation algorithm.
Alternatively, the tissue image may comprise a stained tissue section image.
In a second aspect, an embodiment of the present invention further provides a medical image segmentation apparatus, which may include:
the primary segmentation image extraction module is used for acquiring a tissue image of the detected object and extracting a primary segmentation image of the cell nucleus from the tissue image based on a preset image segmentation algorithm;
the target segmentation image obtaining module is used for inputting the tissue image and the preliminary segmentation image into the trained cell nucleus segmentation model, and obtaining the target segmentation image of the cell nucleus according to the output result of the cell nucleus segmentation model.
In a third aspect, an embodiment of the present invention further provides an apparatus, which may include:
one or more processors;
a memory for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the medical image segmentation method provided by any embodiment of the present invention.
In a fourth aspect, embodiments of the present invention further provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the medical image segmentation method provided by any of the embodiments of the present invention.
According to the technical scheme, the primary segmentation image of the cell nucleus is extracted from the tissue image by acquiring the tissue image of the detected object and based on the preset image segmentation algorithm, and can be used as a priori knowledge to be input into the trained cell nucleus segmentation model together with the tissue image, so that the target segmentation image of the cell nucleus can be obtained according to the output result of the cell nucleus segmentation model. According to the technical scheme, the traditional image segmentation algorithm and the deep learning algorithm are combined, so that the requirement of the deep learning algorithm on the manually marked perfect data can be greatly relieved on the basis of fully playing the good segmentation performance of the deep learning algorithm, the better segmentation performance can still be achieved under the condition that only a small amount of manually marked perfect data is available, the segmentation performance of the pure deep learning algorithm can be improved, and the effect of accurately segmenting the nuclear image from the medical image with lower labor cost and time cost is achieved.
Drawings
FIG. 1 is a flow chart of a medical image segmentation method in accordance with a first embodiment of the present invention;
FIG. 2 is a schematic diagram showing the general structure of a medical image segmentation method according to a first embodiment of the present invention;
FIG. 3a is a flowchart of a watershed algorithm based on marker points in a medical image segmentation method according to a first embodiment of the present invention;
FIG. 3b is a schematic view of a distance transformation in a medical image segmentation method according to a first embodiment of the present invention;
FIG. 3c is a schematic diagram of a U-Net model in a medical image segmentation method according to a first embodiment of the present invention;
fig. 4 is a block diagram of a medical image segmentation apparatus according to a second embodiment of the present invention;
fig. 5 is a schematic structural view of an apparatus according to a third embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Before describing the embodiment of the present invention, an application scenario of the embodiment of the present invention is described in an exemplary manner: as described above, although the conventional image segmentation algorithm and the deep learning algorithm have certain limitations, they have respective advantages: traditional image segmentation algorithms, such as a threshold-based image segmentation algorithm, an edge-based image segmentation algorithm, a region-based image segmentation algorithm, an active contour model-based image segmentation algorithm and the like, have the advantages of strong interpretability, easiness in implementation and no need of manually noted perfect data; correspondingly, the deep learning algorithm is particularly a supervised learning algorithm which can be used for image segmentation, such as a full convolutional neural network (Fully Convolutional Network, FCN), an end-to-end full convolutional neural network (U-Net), a pyramid scene analysis network (PSPNet) and the like, and has the advantages of high segmentation precision and excellent performance. Therefore, the traditional image segmentation algorithm and the deep learning algorithm complement each other, and if the traditional image segmentation algorithm and the deep learning algorithm are combined, the accurate segmentation effect of the medical image can be realized with lower labor cost and time cost.
Example 1
Fig. 1 is a flowchart of a medical image segmentation method according to a first embodiment of the present invention. The present embodiment is applicable to a case of segmenting a nuclear image from a tissue image, and is particularly applicable to a case of combining a conventional image segmentation algorithm and a deep learning algorithm to segment a nuclear image from a tissue image. The method may be performed by a medical image segmentation apparatus provided by an embodiment of the present invention, which may be implemented in software and/or hardware, and which may be integrated on various devices.
Referring to fig. 1, the method of the embodiment of the present invention specifically includes the following steps:
s110, acquiring a tissue image of the detected object, and extracting a preliminary segmentation image of the cell nucleus from the tissue image based on a preset image segmentation algorithm.
Wherein a tissue image of the subject is acquired, optionally for a nuclear segmentation task in medical image segmentation, the tissue image may comprise a stained tissue slice image, in particular a stained histopathological slice image. On this basis, a preliminary segmented image of the nucleus may be extracted from the tissue image based on a preset image segmentation algorithm, which may be a threshold-based image segmentation algorithm, an edge-based image segmentation algorithm, a region-based image segmentation algorithm, a moving contour model-based image segmentation algorithm, or the like, which may be a mask image of the nucleus in the tissue image.
In one aspect, in general, cells are basic structural and functional units of an organism, and cells can be differentiated to form cell groups with different morphologies, structures and functions, and cell groups with similar morphologies, structures and functions can be called tissues, and several different tissues can be combined in a certain order to form an organ with a certain function.
On the other hand, as described above, the deep learning algorithm requires a large amount of manually-labeled perfect data for model training, and when the amount of manually-labeled perfect data is limited, the learning effect of the deep learning algorithm is more general. At this time, although the segmentation accuracy of the cell nucleus image and the generalization capability of the cell nucleus image to different types of tissue images are to be improved by the preset image segmentation algorithm, the segmentation result can be considered as a priori knowledge, and the segmentation result can be used as input data of the deep learning algorithm, that is, the deep learning algorithm can learn again on the preliminary segmentation image of the cell nucleus, so that the deep learning algorithm can still obtain better segmentation performance of the cell nucleus image under the condition that the data amount of manually marked perfect data is limited, and the segmentation performance of the pure deep learning algorithm can be improved.
From the above, under the condition that the data volume of the manually marked perfect data is the same, the learning precision of the deep learning algorithm combined with the preset image segmentation algorithm is better; or, if the same segmentation precision is to be achieved, the data volume of the manually marked perfect data required by the deep learning algorithm combined with the preset image segmentation algorithm during training is smaller than that of the pure deep learning algorithm.
S120, inputting the tissue image and the preliminary segmentation image into the trained cell nucleus segmentation model, and obtaining a target segmentation image of the cell nucleus according to an output result of the cell nucleus segmentation model.
The preliminary segmentation image can be used as a priori knowledge and is input into the trained cell nucleus segmentation model together with the tissue image, so that a target segmentation image of the cell nucleus can be obtained according to the output result of the cell nucleus segmentation model, and the target segmentation image is a cell nucleus image with higher segmentation precision extracted from the tissue image. The above-described cell nucleus segmentation model may include a semantic segmentation network model, which may be FCN, U-Net, PSPNet, mask-RCNN, segNet, or the like.
On the basis, optionally, inputting the tissue image and the preliminary segmentation image into a trained cell nucleus segmentation model can specifically include: stitching the tissue image and the preliminary segmented image, that is, the preliminary segmented image may be used as a channel of the tissue image, thereby obtaining a stitched image; the stitched image may then be directly input into the trained nuclear segmentation model.
Alternatively, the nucleus segmentation model may be pre-trained by: exemplary, as shown in fig. 2, a sample tissue image of a sample object is obtained, and a sample preliminary segmentation image of a sample cell nucleus is extracted from the sample tissue image based on a preset image segmentation algorithm; and training the original convolutional neural network model based on a plurality of groups of training samples by taking the sample tissue image, the sample preliminary segmentation image and the artificial labeling mask image of the sample cell nucleus as a group of training samples to obtain a cell nucleus segmentation model. At this time, the sample tissue image and the sample preliminary segmentation image can be spliced to obtain a sample spliced image; and then, inputting the sample spliced image into an original convolutional neural network model, comparing a predicted segmentation image output by the original convolutional neural network model with the artificial labeling mask image, and adjusting the original convolutional neural network model according to a comparison result, thereby generating a trained cell nucleus segmentation model.
According to the technical scheme, the primary segmentation image of the cell nucleus is extracted from the tissue image by acquiring the tissue image of the detected object and based on the preset image segmentation algorithm, and can be used as a priori knowledge to be input into the trained cell nucleus segmentation model together with the tissue image, so that the target segmentation image of the cell nucleus can be obtained according to the output result of the cell nucleus segmentation model. According to the technical scheme, the traditional image segmentation algorithm and the deep learning algorithm are combined, so that the requirement of the deep learning algorithm on the manually marked perfect data can be greatly relieved on the basis of fully playing the good segmentation performance of the deep learning algorithm, the better segmentation performance can still be achieved under the condition that only a small amount of manually marked perfect data is available, the segmentation performance of the pure deep learning algorithm can be improved, and the effect of accurately segmenting the nuclear image from the medical image with lower labor cost and time cost is achieved.
An optional technical solution, when the preset image segmentation algorithm is a watershed algorithm based on a marker point, extracts a preliminary segmentation image of a cell nucleus from a tissue image based on the preset image segmentation algorithm, which specifically may include: extracting a marked point image of the cell nucleus from the tissue image; and extracting a preliminary segmentation image of the cell nucleus from the tissue image according to the mark point image based on a watershed algorithm of the mark point.
The watershed algorithm is a classical image segmentation algorithm, and segments images by taking similarity between adjacent pixels as a segmentation standard, so that a plurality of pixels which are similar in spatial position and approximate in gray value form a closed contour together, and the characteristic is particularly suitable for segmenting objects with closed contours, such as cell nuclei and the like. The embodiment of the invention can adopt a watershed algorithm based on marking points, in particular to marking points which mark outlines of all pixel points in a medical image, wherein the outlines belong to different object areas, each outline can be provided with unique marking points, and the marking points can be used as initial seed points. For the pixel points of the non-marked points in the medical image, the object areas respectively belonging to the non-marked points can be judged according to a watershed algorithm so as to divide the nuclear image from the medical image.
On the basis, optionally, extracting the marker point image of the cell nucleus from the tissue image specifically comprises the following steps: acquiring a binary image of a tissue image, and acquiring a foreground image and a background image of a cell nucleus according to the binary image; and carrying out connected domain calculation on the foreground image, and carrying out subtraction calculation on the foreground image and the background image to obtain a marker point image of the cell nucleus. If the tissue image is a dyed tissue image, the tissue image can be subjected to processing operations such as graying, binarization, noise removal and the like to obtain a binary image of the tissue image; then, a foreground image and a background image can be obtained according to the binary image, wherein the pixel points in the foreground image are pixel points belonging to cell nuclei, and the pixel points in the background image are pixel points not belonging to the cell nuclei; in this way, the connected domain calculation is performed on the foreground image to obtain the marking points of the cell nucleus, the marking points can describe which pixel points in the foreground image can be used as marking points, and the subtraction calculation can be performed on the foreground image and the background image to obtain the unknown region image, wherein the pixel points in the unknown region image can be the pixel points belonging to the cell nucleus or the pixel points not belonging to the cell nucleus; and finally, obtaining a marker point image of the cell nucleus according to the marker points and the unknown region image.
In order to better understand the specific implementation procedure of the above steps, an exemplary description will be given below of the medical image segmentation method of the present embodiment in connection with the specific examples shown in fig. 3a to 3 c. The preset image segmentation algorithm can be a watershed algorithm based on marked points, and the calculation principle is particularly suitable for segmenting objects with closed outlines such as similar cell nuclei; correspondingly, the cell nucleus segmentation model is a U-Net model, is a typical end-to-end full convolution neural network, and is widely applied to the field of medical image segmentation.
Specifically, as shown in fig. 3a, the stained tissue image (i.e., the RGB image in fig. 3 a) is grayed to obtain a gray-scale image; carrying out Gaussian filtering denoising on the gray level image to obtain a filtered image; binarizing the filtered image to obtain a binary image; performing morphological open operation on the binary image, and removing noise such as isolated points, burrs, bridges and the like in the binary image to obtain an image after the open operation; then, a foreground image and a background image of the image after the open operation can be obtained based on a distance transformation algorithm and morphological dilation, wherein the distance transformation algorithm can be used to calculate a distance between each target pixel point (i.e., a non-zero value pixel point) and a nearest background pixel point (i.e., a zero value pixel point) in a binary image, the distance can be euclidean distance or non-euclidean distance, and the foreground image obtained therefrom is a gray distance image, and the images before and after the distance transformation are illustrated in fig. 3 b; subtracting the foreground image from the background image to obtain an unknown region image, and performing connected domain calculation on the foreground image to obtain a mark point, wherein the connected domain is a set formed by adjacent pixel points with the same pixel value, and the connected domain calculation is a process of finding out different front Jing Liantong domains in a binary image and giving a unique mark; combining the unknown region image with the mark points to obtain a mark point image; finally, the tissue image and the marker point image may be used to obtain a preliminary segmentation image of the nucleus (i.e., the segmentation mask image in fig. 3 a) using a marker point-based watershed algorithm, which may be a mask image. After the preliminary segmentation image of the cell nucleus is obtained, the preliminary segmentation image and the tissue image can be spliced to obtain a spliced image; then, the spliced image is input into the trained U-Net model, and therefore a target segmentation image of the cell nucleus can be obtained according to the output result of the U-Net model.
Taking fig. 3c as an example, the U-Net model is a symmetrical U-shaped architecture consisting of a left encoder and a right decoder. Specifically, in the encoder, stacked 3*3 convolutional layers (Conv 3x 3) and downsampling operations may be used to extract different levels of image features; in the decoder, stacked 3*3 convolutional layers and up-sampling operations may be used to recover the original size of the image; meanwhile, the U-Net model also introduces skip connection (Copy and connect in fig. 3 c) to feature-fuse the image features of the corresponding scale in the encoder and decoder.
On this basis, optionally, the training process of the U-Net model is shown in FIG. 3c, which can output the predictive segmented image via a Sigmoid function, where the definition of the Sigmoid function is
Furthermore, the loss function in the model training process can use cross entropy or Dice coefficient, the definition of cross entropy is +.>
Wherein n is the total number of training samples, y is the true value of the artificial annotation in the artificial annotation mask image, < >>
The predicted value is output for the U-Net model being trained; the Dice coefficient definition is +.>
Where |E.andgate Y| represents the intersection between the predicted segmented image E and the artificial annotation mask image Y output by the U-Net model, and |E| represent the number of elements of E and Y, respectively. Updating of the model parameters may be performed based on a Back Propagation (BP) optimization algorithm or a variant thereof.
Example two
Fig. 4 is a block diagram of a medical image segmentation apparatus according to a second embodiment of the present invention, which is configured to perform the medical image segmentation method according to any of the above embodiments. The device belongs to the same inventive concept as the medical image segmentation method of the above embodiments, and reference may be made to the above embodiments of the medical image segmentation method for details not described in detail in the embodiments of the medical image segmentation device. Referring to fig. 4, the apparatus may specifically include: a preliminary segmented image extraction module 210 and a target segmented image acquisition module 220.
The preliminary segmentation image extraction module 210 is configured to obtain a tissue image of the examined object, and extract a preliminary segmentation image of the nucleus from the tissue image based on a preset image segmentation algorithm;
the target segmentation image obtaining module 220 is configured to input the tissue image and the preliminary segmentation image into a trained cell nucleus segmentation model, and obtain a target segmentation image of the cell nucleus according to an output result of the cell nucleus segmentation model.
Optionally, the preliminary segmentation image extraction module 210 may specifically include:
an extraction unit for extracting a marker point image of the nucleus from the tissue image;
the segmentation unit is used for extracting a preliminary segmentation image of the cell nucleus from the tissue image according to the mark point image based on a watershed algorithm of the mark point.
Optionally, the extracting unit may specifically be configured to:
acquiring a binary image of the tissue image;
obtaining a foreground image and a background image of the cell nucleus according to the binary image;
and carrying out connected domain calculation on the foreground image, and carrying out subtraction calculation on the foreground image and the background image to obtain a marker point image of the cell nucleus.
Optionally, the target segmentation image obtaining module 220 may specifically include:
the obtaining unit is used for splicing the tissue image and the preliminary segmentation image to obtain a spliced image;
and the input unit is used for inputting the spliced image into the trained cell nucleus segmentation model, wherein the cell nucleus segmentation model comprises a semantic segmentation network model.
Optionally, on the basis of the above device, the device may further include:
the acquisition module is used for acquiring a sample tissue image of a sample object and extracting a sample preliminary segmentation image of a sample cell nucleus from the sample tissue image based on a preset image segmentation algorithm;
the training module is used for taking the sample tissue image, the sample preliminary segmentation image and the artificial labeling mask image of the sample cell nucleus as a group of training samples, and training the original convolutional neural network model based on a plurality of groups of training samples to obtain a cell nucleus segmentation model.
Alternatively, the preset image segmentation algorithm may include at least one of a threshold-based image segmentation algorithm, an edge-based image segmentation algorithm, a region-based image segmentation algorithm, and an active contour model-based image segmentation algorithm.
Alternatively, the tissue image may comprise a stained tissue section image.
According to the medical image segmentation device provided by the embodiment of the invention, the primary segmentation image extraction module and the target segmentation image obtaining module are matched with each other, so that the tissue image of the detected object can be obtained, the primary segmentation image of the cell nucleus is extracted from the tissue image based on the preset image segmentation algorithm, the primary segmentation image can be used as a priori knowledge and is input into the trained cell nucleus segmentation model together with the tissue image, and therefore, the target segmentation image of the cell nucleus can be obtained according to the output result of the cell nucleus segmentation model. According to the device, the traditional image segmentation algorithm and the deep learning algorithm are combined, so that the requirement of the deep learning algorithm on the manually marked perfect data can be greatly relieved on the basis of fully playing the good segmentation performance of the deep learning algorithm, the better segmentation performance can still be achieved under the condition that only a small amount of manually marked perfect data is available, the segmentation performance of the pure deep learning algorithm can be improved, and the effect of accurately segmenting the nuclear image from the medical image with lower labor cost and time cost is achieved.
The medical image segmentation device provided by the embodiment of the invention can execute the medical image segmentation method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
It should be noted that, in the embodiment of the medical image segmentation apparatus described above, each unit and module included are only divided according to the functional logic, but are not limited to the above-described division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.
Example III
Fig. 5 is a schematic structural diagram of an apparatus according to a third embodiment of the present invention, and as shown in fig. 5, the apparatus includes a memory 310, a processor 320, an input device 330 and an output device 340. The number of processors 320 in the device may be one or more, one processor 320 being taken as an example in fig. 5; the memory 310, processor 320, input 330 and output 340 in the device may be connected by a bus or other means, as exemplified by bus 350 in fig. 5.
The memory 310 is used as a computer readable storage medium for storing software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the medical image segmentation method in the embodiment of the present invention (e.g., the preliminary segmented image extraction module 210 and the target segmented image obtaining module 220 in the medical image segmentation apparatus). The processor 320 performs various functional applications of the device and data processing, i.e., implements the medical image segmentation method described above, by running software programs, instructions, and modules stored in the memory 310.
The input device 330 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the device. The output device 340 may include a display device such as a display screen.
Example IV
A fourth embodiment of the present invention provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are for performing a medical image segmentation method, the method comprising:
obtaining a tissue image of a detected object, and extracting a preliminary segmentation image of a cell nucleus from the tissue image based on a preset image segmentation algorithm;
and inputting the tissue image and the preliminary segmentation image into a trained cell nucleus segmentation model, and obtaining a target segmentation image of the cell nucleus according to an output result of the cell nucleus segmentation model.
Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the method operations described above, and may also perform the related operations in the medical image segmentation method provided in any embodiment of the present invention.
From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. In light of such understanding, the technical solution of the present invention may be embodied essentially or in part in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), FLASH Memory (FLASH), hard disk, optical disk, etc., of a computer, which may be a personal computer, a server, a network device, etc., and which includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in the embodiments of the present invention.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (9)
1. A medical image segmentation method, comprising:
obtaining a tissue image of a detected object, and extracting a preliminary segmentation image of a cell nucleus from the tissue image based on a preset image segmentation algorithm;
inputting the tissue image and the preliminary segmentation image into a trained cell nucleus segmentation model, and obtaining a target segmentation image of the cell nucleus according to an output result of the cell nucleus segmentation model;
acquiring a sample tissue image of a sample object, and extracting a sample preliminary segmentation image of a sample cell nucleus from the sample tissue image based on the preset image segmentation algorithm;
and taking the sample tissue image, the sample preliminary segmentation image and the artificial labeling mask image of the sample cell nucleus as a group of training samples, and training an original convolutional neural network model based on a plurality of groups of training samples to obtain the cell nucleus segmentation model.
2. The method of claim 1, wherein the extracting a preliminary segmented image of the nucleus from the tissue image based on a preset image segmentation algorithm comprises:
extracting a marker point image of the cell nucleus from the tissue image;
and extracting a preliminary segmentation image of the cell nucleus from the tissue image according to the mark point image based on a mark point watershed algorithm.
3. The method of claim 2, wherein the extracting the marker point image of the nucleus from the tissue image comprises:
acquiring a binary image of the tissue image, and acquiring a foreground image and a background image of a cell nucleus according to the binary image;
and carrying out connected domain calculation on the foreground image, and carrying out subtraction calculation on the foreground image and the background image to obtain the marker point image of the cell nucleus.
4. The method of claim 1, wherein said inputting the tissue image and the preliminary segmentation image into a trained nuclear segmentation model comprises:
splicing the tissue image and the preliminary segmentation image to obtain a spliced image;
inputting the spliced image into a trained cell nucleus segmentation model, wherein the cell nucleus segmentation model comprises a semantic segmentation network model.
5. The method of claim 1, wherein the preset image segmentation algorithm comprises at least one of a threshold-based image segmentation algorithm, an edge-based image segmentation algorithm, a region-based image segmentation algorithm, and an active contour model-based image segmentation algorithm.
6. The method of claim 1, wherein the tissue image comprises a stained tissue section image.
7. A medical image segmentation apparatus, comprising:
the primary segmentation image extraction module is used for acquiring a tissue image of a detected object and extracting a primary segmentation image of a cell nucleus from the tissue image based on a preset image segmentation algorithm;
the target segmentation image obtaining module is used for inputting the tissue image and the preliminary segmentation image into a trained cell nucleus segmentation model, and obtaining a target segmentation image of the cell nucleus according to an output result of the cell nucleus segmentation model;
the acquisition module is used for acquiring a sample tissue image of a sample object and extracting a sample preliminary segmentation image of a sample cell nucleus from the sample tissue image based on a preset image segmentation algorithm;
the training module is used for taking the sample tissue image, the sample preliminary segmentation image and the artificial labeling mask image of the sample cell nucleus as a group of training samples, and training the original convolutional neural network model based on a plurality of groups of training samples to obtain a cell nucleus segmentation model.
8. An apparatus, the apparatus comprising:
one or more processors;
a memory for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the medical image segmentation method as set forth in any one of claims 1-6.
9. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the medical image segmentation method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911367572.2A CN111145209B (en) | 2019-12-26 | 2019-12-26 | Medical image segmentation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911367572.2A CN111145209B (en) | 2019-12-26 | 2019-12-26 | Medical image segmentation method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111145209A CN111145209A (en) | 2020-05-12 |
| CN111145209B true CN111145209B (en) | 2023-06-02 |
Family
ID=70520678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911367572.2A Active CN111145209B (en) | 2019-12-26 | 2019-12-26 | Medical image segmentation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111145209B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA3141859A1 (en) | 2019-04-11 | 2020-10-15 | Agilent Technologies, Inc. | User interface configured to facilitate user annotation for instance segmentation within biological sample |
| CN111583282B (en) * | 2020-05-18 | 2024-04-23 | 联想(北京)有限公司 | Image segmentation method, device, equipment and storage medium |
| CN111696084B (en) * | 2020-05-20 | 2024-05-31 | 平安科技（深圳）有限公司 | Cell image segmentation method, device, electronic equipment and readable storage medium |
| CN112070722A (en) * | 2020-08-14 | 2020-12-11 | 厦门骁科码生物科技有限公司 | Fluorescence in situ hybridization cell nucleus segmentation method and system |
| CN112183556B (en) * | 2020-09-27 | 2022-08-30 | 长光卫星技术股份有限公司 | Port ore heap contour extraction method based on spatial clustering and watershed transformation |
| CN112200815B (en) * | 2020-10-12 | 2023-06-27 | 徐州医科大学附属医院 | Thyroid nodule ultrasound image segmentation method based on semantic segmentation network PSPNet |
| CN112446892A (en) * | 2020-11-18 | 2021-03-05 | 黑龙江机智通智能科技有限公司 | Cell nucleus segmentation method based on attention learning |
| CN113222944B (en) * | 2021-05-18 | 2022-10-14 | 湖南医药学院 | Cell nucleus segmentation method and cancer auxiliary analysis system and device based on pathological image |
| CN113177957B (en) * | 2021-05-24 | 2024-03-08 | 西交利物浦大学 | Cell image segmentation method and device, electronic equipment and storage medium |
| CN113837168A (en) * | 2021-09-22 | 2021-12-24 | 易联众智鼎(厦门)科技有限公司 | Image text detection and OCR recognition method, device and storage medium |
| CN113989294B (en) * | 2021-12-29 | 2022-07-05 | 北京航空航天大学 | Cell segmentation and typing method, device, equipment and medium based on machine learning |
| CN114862877B (en) * | 2022-05-27 | 2024-03-22 | 四川大学华西医院 | Cell adhesion segmentation method and device based on confidence score |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108364288A (en) * | 2018-03-01 | 2018-08-03 | 北京航空航天大学 | Dividing method and device for breast cancer pathological image |
| CN109886179A (en) * | 2019-02-18 | 2019-06-14 | 深圳视见医疗科技有限公司 | The image partition method and system of cervical cell smear based on Mask-RCNN |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103345748B (en) * | 2013-06-26 | 2016-08-10 | 福建师范大学 | A kind of locating segmentation method of human tissue cell two-photon micro-image |
| CN103679184A (en) * | 2013-12-06 | 2014-03-26 | 河海大学 | Method for leukocyte automatic identification based on relevant vector machine |
| CN107256558B (en) * | 2017-05-18 | 2019-12-17 | 深思考人工智能机器人科技（北京）有限公司 | Unsupervised automatic cervical cell image segmentation method and system |
| US10789451B2 (en) * | 2017-11-16 | 2020-09-29 | Global Life Sciences Solutions Usa Llc | System and method for single channel whole cell segmentation |
| US11164312B2 (en) * | 2017-11-30 | 2021-11-02 | The Research Foundation tor the State University of New York | System and method to quantify tumor-infiltrating lymphocytes (TILs) for clinical pathology analysis based on prediction, spatial analysis, molecular correlation, and reconstruction of TIL information identified in digitized tissue images |
| CN111417958A (en) * | 2017-12-07 | 2020-07-14 | 文塔纳医疗系统公司 | Deep learning system and method for joint cell and region classification in biological images |
| WO2019197509A1 (en) * | 2018-04-13 | 2019-10-17 | Ventana Medical Systems, Inc. | Systems for cell shape estimation |
| CN109389557B (en) * | 2018-10-20 | 2023-01-06 | 南京大学 | Cell image super-resolution method and device based on image prior |
-
2019
- 2019-12-26 CN CN201911367572.2A patent/CN111145209B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108364288A (en) * | 2018-03-01 | 2018-08-03 | 北京航空航天大学 | Dividing method and device for breast cancer pathological image |
| CN109886179A (en) * | 2019-02-18 | 2019-06-14 | 深圳视见医疗科技有限公司 | The image partition method and system of cervical cell smear based on Mask-RCNN |
Non-Patent Citations (1)
| Title |
|---|
| 基于多维PCNN的彩色枸杞细胞显微图像的边缘检测;李志强;程飞燕;安黎哲;马义德;高清祥;;兰州大学学报(自然科学版)(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111145209A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111145209B (en) | Medical image segmentation method, device, equipment and storage medium | |
| CN109154978B (en) | System and method for detecting plant diseases | |
| CN108364288B (en) | Segmentation method and device for breast cancer pathological image | |
| CN109389129B (en) | Image processing method, electronic device and storage medium | |
| US20210118144A1 (en) | Image processing method, electronic device, and storage medium | |
| CN110472676A (en) | Stomach morning cancerous tissue image classification system based on deep neural network | |
| CN109583345B (en) | Road recognition method, device, computer device and computer readable storage medium | |
| CN110705583A (en) | Cell detection model training method and device, computer equipment and storage medium | |
| CN109460735B (en) | Document binarization processing method, system and device based on graph semi-supervised learning | |
| Jiang et al. | A novel white blood cell segmentation scheme based on feature space clustering | |
| CN111402267A (en) | Segmentation method, device and terminal for epithelial cell nucleus in prostate cancer pathological image | |
| CN109241973B (en) | Full-automatic soft segmentation method for characters under texture background | |
| CN115482221A (en) | End-to-end weak supervision semantic segmentation labeling method for pathological image | |
| Tareef et al. | Automated three-stage nucleus and cytoplasm segmentation of overlapping cells | |
| Lv et al. | Nuclei R-CNN: improve mask R-CNN for nuclei segmentation | |
| CN110852327A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN110490159B (en) | Method, device, equipment and storage medium for identifying cells in microscopic image | |
| CN113609984A (en) | Pointer instrument reading identification method and device and electronic equipment | |
| CN116648723A (en) | Method and device for analyzing microstructure of material | |
| CN113780110A (en) | Method and device for detecting weak and small targets in image sequence in real time | |
| CN113033558A (en) | Text detection method and device for natural scene and storage medium | |
| Kausar et al. | Multi-scale deep neural network for mitosis detection in histological images | |
| Seyedhosseini et al. | Detection of neuron membranes in electron microscopy images using multi-scale context and radon-like features | |
| Lal et al. | A robust method for nuclei segmentation of H&E stained histopathology images | |
| CN113724235B (en) | Semi-automatic Ki67/ER/PR negative and positive cell counting system and method under condition of changing environment under mirror |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room B401, floor 4, building 1, No. 12, Shangdi Information Road, Haidian District, Beijing 100085 Applicant after: Tuxiang Medical Technology Co.,Ltd. Address before: Room B401, floor 4, building 1, No. 12, Shangdi Information Road, Haidian District, Beijing 100085 Applicant before: INFERVISION |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |