CN117218135B - Method and related equipment for segmenting plateau pulmonary edema chest film focus based on transducer - Google Patents
Method and related equipment for segmenting plateau pulmonary edema chest film focus based on transducer Download PDFInfo
- Publication number
- CN117218135B CN117218135B CN202311191474.4A CN202311191474A CN117218135B CN 117218135 B CN117218135 B CN 117218135B CN 202311191474 A CN202311191474 A CN 202311191474A CN 117218135 B CN117218135 B CN 117218135B
- Authority
- CN
- China
- Prior art keywords
- model
- image
- generate
- processing
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 206010037423 Pulmonary oedema Diseases 0.000 title claims abstract description 49
- 208000005333 pulmonary edema Diseases 0.000 title claims abstract description 48
- 230000011218 segmentation Effects 0.000 claims abstract description 141
- 238000012549 training Methods 0.000 claims abstract description 136
- 238000012545 processing Methods 0.000 claims abstract description 101
- 238000013528 artificial neural network Methods 0.000 claims abstract description 47
- 238000007781 pre-processing Methods 0.000 claims abstract description 23
- 238000005070 sampling Methods 0.000 claims description 74
- 238000013527 convolutional neural network Methods 0.000 claims description 40
- 238000013136 deep learning model Methods 0.000 claims description 19
- 238000010606 normalization Methods 0.000 claims description 15
- 230000003902 lesion Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 12
- 230000003321 amplification Effects 0.000 claims description 7
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 7
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 238000002601 radiography Methods 0.000 abstract description 5
- 210000000038 chest Anatomy 0.000 description 34
- 238000003745 diagnosis Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 230000009467 reduction Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 210000000115 thoracic cavity Anatomy 0.000 description 3
- 230000008649 adaptation response Effects 0.000 description 2
- 208000008445 altitude sickness Diseases 0.000 description 2
- 238000013434 data augmentation Methods 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000002685 pulmonary effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 206010021143 Hypoxia Diseases 0.000 description 1
- 206010030113 Oedema Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 210000000481 breast Anatomy 0.000 description 1
- 230000004856 capillary permeability Effects 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000005713 exacerbation Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000001146 hypoxic effect Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 210000004072 lung Anatomy 0.000 description 1
- 230000004089 microcirculation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000035699 permeability Effects 0.000 description 1
- 210000003456 pulmonary alveoli Anatomy 0.000 description 1
- 238000005057 refrigeration Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Abstract
The application provides a method for segmenting plateau pulmonary edema chest film focus based on a transducer and related equipment, which are applied to the technical field of data processing. The method comprises the steps of obtaining a training sample and a target image; preprocessing the training sample to generate a marked training sample; constructing a U-shaped full convolution neural network segmentation model; training the neural network segmentation model by the marked training sample to generate a trained neural network segmentation model; inputting the target image into the trained segmentation model for processing, and outputting a target case; and adding the target image into the training sample set for updating the neural network segmentation model. The method adopts the transducer network to extract the characteristics of the chest radiography image and divide the focus, thereby effectively solving the problems of low dividing precision, incapability of accurately identifying the focus area and the like in the prior art.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method for segmenting a plateau pulmonary edema chest film focus based on a transducer and related equipment.
Background
Plateau pulmonary edema is a common type of altitude sickness, which is a disease caused by abnormal adaptive response of humans in an altitude hypoxic environment. Under the condition of altitude hypoxia, the body adaptive response can cause symptoms such as pulmonary microcirculation disturbance, pulmonary alveolus permeability increase, pulmonary capillary permeability increase and the like, so that pulmonary edema is caused, and the edema is called altitude pulmonary edema. Early diagnosis and treatment of altitude pulmonary edema is important because if not treated in time, it can result in exacerbation of the patient's condition and, in severe cases, even life threatening. At present, the medical community mainly adopts medical imaging technology to diagnose and treat the altitude pulmonary edema. However, the conventional medical imaging technology often requires manual segmentation and diagnosis by a doctor, which is time-consuming and labor-consuming, and is prone to subjective errors, affecting the diagnosis effect and the treatment effect.
Therefore, an automatic plateau pulmonary edema focus segmentation method based on a computer vision technology is generated, the problems of manual segmentation, subjective errors and the like can be effectively solved, and the diagnosis and treatment effects are improved. The current automatic plateau pulmonary edema focus segmentation method is mainly based on a convolutional neural network method, and realizes automatic segmentation of plateau pulmonary edema focus by performing deep learning and image segmentation on medical images. However, the conventional convolutional neural network model often ignores different spatial position correlations in the training process.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The application aims to provide a method for segmenting a plateau pulmonary edema chest film focus based on a transducer and related equipment, wherein a transducer network is adopted to perform feature extraction and focus segmentation on chest film images, so that the problems that segmentation accuracy is low, focus areas cannot be accurately identified and the like in the prior art can be effectively solved, in addition, a local and global multi-level window attention mechanism is introduced by a Swin transducer, weights of different position information can be better captured, different weights are given to focus areas and non-focus areas, denseNet are connected in a dense mode between different convolution layers, the feature information of the previous layer can be better utilized, and the feature reduction capability is improved.
Other features and advantages of the application will be apparent from the following detailed description, or may be learned by the practice of the application.
According to one aspect of the present application, there is provided a method for segmenting a plateau pulmonary edema chest lesion based on a transducer, comprising acquiring a training sample and a target image; preprocessing the training sample to generate a marked training sample; constructing a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread Transformer deep learning model, and the up-sampling model is a DenseNet convolutional neural network model; training the neural network segmentation model by the marked training sample to generate a trained neural network segmentation model; inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus; and adding the target image into the training sample set for updating the neural network segmentation model.
In one embodiment of the present application, the upsampling model and the downsampling model are connected through a Skip Connection layer, comprising: the output end of the Swin transform deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection layer and the bottom feature layer; performing size amplification processing on the output end of the DenseNet convolutional neural network model through up-sampling to generate preprocessed model features; and connecting the preprocessed model features with the Skip Connection layer.
In one embodiment of the present application, the preprocessing the training sample to generate a labeled training sample includes: performing data enhancement processing on the training samples to generate enhanced training samples; and processing the enhanced training sample based on a preset partitioning rule to generate a focus area and a non-focus area.
In one embodiment of the present application, the preprocessing the training sample to generate a labeled training sample further includes: carrying out normalization pretreatment on the training sample to generate a training sample with target brightness; acquiring a preset classification rule, wherein the preset classification rule comprises segmentation data of a plurality of different focus types; and processing the training samples of the target brightness based on the preset classification rule to generate classified training samples.
In one embodiment of the present application, the inputting the target image into the trained segmentation model for processing, outputting a target case includes: graying treatment is carried out on the target image, and a target color image is generated; performing image normalization processing on the target color image to generate a color image with a preset size; slicing the color images to sequentially generate three continuous slice images; processing the slice picture to generate a three-channel image; and inputting the three-channel image into the trained segmentation model for processing, and outputting a target case.
In one embodiment of the present application, the slicing the color image sequentially generates three consecutive slice images, including: processing the color image based on a preset color division rule to generate a first color image containing different classification numbers; dividing the first color image based on the classification number to generate a plurality of initial slice images; preprocessing the initial slice image based on a target segmentation model to generate a preprocessed slice image; and processing the preprocessed slice images based on preset bilinear interpolation to generate a plurality of slice images with preset sizes.
In one embodiment of the present application, the slicing processing is performed on the color images, and three consecutive slice images are sequentially generated, and the method further includes: acquiring a preset size value to process the color image, and generating a panoramic image with a preset size; performing multi-layer slicing processing on the panoramic image to generate a plurality of slice image groups with different depth levels; and processing the slice image group based on time sequence or attribute information to generate a plurality of slice images.
In another aspect of the present application, a device for segmenting a plateau pulmonary edema chest lesion based on a transducer, the device comprising: the acquisition module is used for acquiring training samples and target images; the processing module is used for preprocessing the training samples and generating marked training samples; constructing a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread Transformer deep learning model, and the up-sampling model is a DenseNet convolutional neural network model; training the neural network segmentation model by the marked training sample to generate a trained neural network segmentation model; inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus; and adding the target image into the training sample set for updating the neural network segmentation model.
According to still another aspect of the present application, an electronic apparatus, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform a segmentation method implementing the above-described transducer-based plateau pulmonary edema chest lesion via execution of the executable instructions.
According to yet another aspect of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described method for segmenting a transducer-based altitude pulmonary edema chest lesion.
According to a further aspect of the present application, there is provided a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the above-mentioned method for segmenting a plateau pulmonary oedema chest lesion based on a transducer.
The application provides a method for segmenting plateau pulmonary edema chest film focus based on a transducer, which comprises the steps of obtaining a training sample and a target image; preprocessing the training sample to generate a marked training sample; constructing a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread Transformer deep learning model, and the up-sampling model is a DenseNet convolutional neural network model; training the neural network segmentation model by the marked training sample to generate a trained neural network segmentation model; inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus; and adding the target image into the training sample set for updating the neural network segmentation model. The method has the advantages that the problems that segmentation accuracy is low, focus areas cannot be accurately identified and the like in the prior art can be effectively solved by adopting a transducer network to perform feature extraction and focus segmentation on chest images, in addition, the Swin transducer introduces a local and global multi-level window attention mechanism, weights of different position information can be better captured, different weights are given to focus areas and non-focus areas, denseNet are connected densely between different convolution layers, the feature information of the previous layer can be better utilized, and the feature restoration capability is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
FIG. 1 is a flowchart of a method for segmenting a plateau pneumochysis chest film focus based on a transducer according to an embodiment of the present application;
Fig. 2 is a schematic structural diagram of a segmentation apparatus for a plateau pulmonary edema chest lesion based on a transducer according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an electronic device according to an embodiment of the present application;
Fig. 4 is a schematic diagram of a storage medium according to an embodiment of the present application.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
A method for segmenting a plateau pulmonary edema chest lesion based on a transducer according to an exemplary embodiment of the present application is described below with reference to fig. 1. It should be noted that the following application scenarios are only shown for facilitating understanding of the spirit and principles of the present application, and embodiments of the present application are not limited in this respect. Rather, embodiments of the application may be applied to any scenario where applicable.
In one embodiment, the application also provides a method for segmenting the plateau pulmonary edema chest film focus based on the transducer. Fig. 1 schematically shows a flow chart of a method for segmenting a plateau pulmonary edema chest lesion based on a transducer according to an embodiment of the present application. As shown in fig. 1, the method is applied to a server, and includes:
s101, acquiring a training sample and a target image.
In one embodiment, chest image data for training, validation and testing is acquired as a training sample, and patient chest image data to be detected is acquired as a target image.
S102, preprocessing the training sample to generate a labeled training sample.
In one embodiment, the training samples are subjected to data enhancement processing to generate enhanced training samples, and the enhanced training samples are processed based on preset partitioning rules to generate focus areas and non-focus areas. The invention adopts Cutout data enhancement mode, which carries out independent repeated evenly distributed sampling on the resolution of input pictures, then selects different scales with equal probability, and finally carries out clipping or random transformation. The method erases some key point information and even brings some confusing information, the forced model has to infer the erased information and distinguish the confusing information through the context, the robustness of the model is greatly improved, and the model has very good generalization capability on an unknown data set.
In another embodiment, the chest radiography image data is subjected to noise reduction, artificial and natural noise which is harmful to the determination of the focus position in the image is removed, and the data set is expanded by deforming and rotating the image subjected to noise reduction. In addition, each ultrasonic gray-scale image in the data set is respectively subjected to filling and standardization operation, and the training set is subjected to data augmentation, wherein filling refers to filling fixed pixel values around the image, so that the output size of the image after being trained by a convolutional neural network is consistent with the original image size, standardization refers to dividing the average value of each dimension of the data by the standard deviation, and the data augmentation mode comprises image translation, rotation or affine transformation.
In another embodiment, the training samples are subjected to normalization pretreatment to generate training samples with target brightness, where the normalization pretreatment specifically includes: (1) Acquiring a three-dimensional image gray matrix from an original sequence image; (2) Normalizing the pixel gray value of the three-dimensional image gray matrix between 0 and 1; (3) And (3) normalizing the three-dimensional pixel spacing of the image normalized in the step (2) to a preset value. The image illumination and contrast can be regulated by carrying out normalization pretreatment on chest radiography image data, so that the influence of great difference of the image illumination and contrast caused by different equipment is avoided. And acquiring a preset classification rule, wherein the preset classification rule comprises a plurality of pieces of segmentation data of different focus types, and processing a training sample of target brightness based on the preset classification rule to generate a classified training sample. And acquiring the segmentation result of the expert label and the classification result of the expert label by collecting segmentation and classification results of the same focus on a plurality of CT images containing the focus.
S103, constructing a U-shaped full convolution neural network segmentation model.
In one embodiment, the U-shaped full convolutional neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread transform deep learning model, and the up-sampling model is a DenseNet convolutional neural network model. The Swin transducer deep learning model is used as downsampling to conduct feature extraction, and the DenseNet convolutional neural network model is used as upsampling to conduct feature reduction. For a U-shaped network, a feature map is obtained every time the downsampling is performed, the size of the feature map is reduced once, the feature map is calculated as a scale, the low-level feature map may be focused on the large-scale feature such as the overall outline of the image, and the high-level feature map may be focused on the fine-grained feature of the image. For example, a low level may be of interest for the overall contour of the breast and a high level may be of interest for focal areas, particularly areas where the foci and normal tissue interface.
In another embodiment, the output end of the Swin transform deep learning model is connected to the input end of the DenseNet convolutional neural network model through a Skip Connection layer and a bottom feature layer, the output end of the DenseNet convolutional neural network model is subjected to size amplification processing through upsampling to generate preprocessed model features, and the preprocessed model features are connected with the Skip Connection layer to form a complete segmentation result. Through the design of the technical characteristics and the connection relation, the model can more efficiently and accurately realize the segmentation of the plateau pulmonary edema focus in the lung CT image. The bottom feature contains high-level semantic information and is a feature map with more channels and smaller size. Is the feature representation of the deepest hierarchy in the encoder. The bottom feature layer is the same as the original U-Net bottom feature layer, consists of 2 times 3x3 conv convolution and ReLU, and in the decoder, the bottom features are connected with features in the upsampling path to help preserve more detailed information. Through the join operation, the bottom feature may pass fine-grained information to the decoder, thereby enabling more accurate segmentation.
And S104, training the neural network segmentation model by the labeled training sample, and generating a trained neural network segmentation model.
In one embodiment, the segmentation model is trained using artificially labeled lesion images. During training, a doctor selects a part of patients as a training set, marks a focus of the plateau pulmonary edema in the CT image, and sets a focus area as 1 and a non-focus area as 0.
In another embodiment, the segmentation model is trained using the artificially labeled lesion image to obtain a semantic segmentation pre-training model, the learning rate of model training is adjusted, and the semantic segmentation pre-training model is continuously trained using the training set with semantic image content to obtain a neural network segmentation model with semantic recognition function.
S105, inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus.
In one embodiment, the target image is grayed to produce a target color image, wherein the target image includes a true customer condition. The method comprises the steps of performing image normalization processing on a target color image to generate a color image with a preset size, performing slicing processing on the color image, sequentially generating three continuous slice images, processing slice images to generate a three-channel image, inputting the three-channel image into a trained segmentation model for processing, and outputting a target case.
And S106, adding the target image into the training sample set so as to update the neural network segmentation model.
In one embodiment, the target image of the real illness state of the user is added into the training sample set, so that the training sample set is continuously updated, the neural network segmentation model can be continuously updated, and the recognition accuracy of the neural network segmentation model is improved.
The method comprises the steps of obtaining a training sample and a target image, preprocessing the training sample, generating a marked training sample, and constructing a U-shaped full convolution neural network segmentation model, wherein the U-shaped full convolution neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread transform deep learning model, and the up-sampling model is a DenseNet convolution neural network model. Training the neural network segmentation model by the labeled training sample, generating a trained neural network segmentation model, inputting a target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus, and adding the target image into a training sample set for updating the neural network segmentation model. The accuracy and precision of the segmentation of the plateau pulmonary edema chest film focus are improved, the focus area can be better identified, and more accurate diagnosis results and treatment suggestions are provided for clinicians, so that the medical level and treatment effect are improved. Meanwhile, the network based on the transducer has stronger self-adaptive capacity and generalization capacity, can adapt to different types of chest radiography images, and has good application prospect.
Optionally, in another embodiment of the method according to the present application, the slicing the color image sequentially generates three consecutive slice images, including:
Processing the color image based on a preset color division rule to generate a first color image containing different classification numbers;
dividing the first color image based on the classification number to generate a plurality of initial slice images;
preprocessing the initial slice image based on a target segmentation model to generate a preprocessed slice image;
and processing the preprocessed slice images based on preset bilinear interpolation to generate a plurality of slice images with preset sizes.
In one embodiment, the color images are annotated and divided according to the darkness of the color of the target image. Preprocessing all slices, inputting all the preprocessed slices into an optimal segmentation model, obtaining segmentation results, carrying out bilinear interpolation up-sampling on the segmentation results to restore to the original size, merging the segmentation results restored to the original size of all the slices of the chest three-dimensional CT scanning image to be segmented into three-dimensional data according to the position relationship, combining continuous 3 CT slices into a 3-channel image, and sending the 3-channel image into a trained segmentation model to quickly obtain segmentation results of a plateau pulmonary edema focus. The invention combines three continuous CT slices into a 3-channel image, mainly based on an ImageNet pre-training weight, wherein the image required by the pre-training weight is 3 channels, so that 3 continuous slices are combined into the 3-channel image.
Optionally, in another embodiment of the above method according to the present application, the slicing processing is performed on the color image, and three consecutive slice images are sequentially generated, and further includes:
Acquiring a preset size value to process the color image, and generating a panoramic image with a preset size;
performing multi-layer slicing processing on the panoramic image to generate a plurality of slice image groups with different depth levels;
And processing the slice image group based on time sequence or attribute information to generate a plurality of slice images.
In one embodiment, the corresponding slice images may be sequentially generated according to the order of slices or the type of slices. And slicing the color images to obtain a plurality of slice images with initial depth levels, generating a slice image group with initial depth levels according to the plurality of slice images with initial depth levels, slicing the plurality of slice images with current depth levels to obtain a plurality of slice images with next depth levels, and generating a slice image group with next depth levels according to the plurality of slice images with next depth levels. And repeatedly executing the previous step until the depth level reaches a preset first threshold value, stopping slicing processing, and obtaining a plurality of slice image groups with different depth levels.
By applying the technical scheme, the training sample and the target image are obtained, the data enhancement processing is carried out on the training sample, the enhanced training sample is generated, the enhanced training sample is processed based on the preset dividing rule, the focus area and the non-focus area are generated, the normalization preprocessing is carried out on the training sample, and the training sample with the target brightness is generated. The method comprises the steps of obtaining a preset classification rule, wherein the preset classification rule comprises a plurality of segmentation data of different focus types, processing training samples of target brightness based on the preset classification rule, generating classified training samples, constructing a U-shaped full-convolution neural network segmentation model, wherein the U-shaped full-convolution neural network segmentation model comprises an up-sampling model and a down-sampling model, the output end of a Swin transform deep learning model is connected to the input end of a DenseNet convolutional neural network model through a Skip Connection layer and a bottom feature layer, the output end of the DenseNet convolutional neural network model is subjected to size amplification processing through up-sampling, preprocessed model features are generated, the preprocessed model features are connected with the Skip Connection layer, the down-sampling model is the Swin transform deep learning model, the up-sampling model is the DenseNet convolutional neural network model, training the labeled training samples on the neural network segmentation model, and generating a trained neural network segmentation model.
In addition, the target image is subjected to graying processing to generate a target color image, the target color image is subjected to image normalization processing to generate a color image with a preset size, the color image is processed based on preset color division rules to generate a first color image containing different classification numbers, the first color image is divided based on the classification numbers to generate a plurality of initial slice images, the initial slice images are preprocessed based on a target segmentation model to generate a preprocessed slice image, the preprocessed slice images are processed based on preset bilinear interpolation to generate a plurality of slice images with a preset size, the color image is processed to obtain a preset size value, the panoramic image with the preset size is generated, the panoramic image is subjected to multi-layer slice processing to generate a plurality of slice image groups with different depth levels, the slice image groups are processed based on time sequence or attribute information to generate a plurality of slice images, the three-channel image is generated, the three-channel image is input into the trained segmentation model to process, and the target case is output, wherein the target case comprises a segmentation result of a plateau pulmonary edema, the target case is added into the training sample set for updating a neural network model. The method has the advantages that the problems that segmentation accuracy is low, focus areas cannot be accurately identified and the like in the prior art can be effectively solved by adopting a transducer network to perform feature extraction and focus segmentation on chest images, in addition, the Swin transducer introduces a local and global multi-level window attention mechanism, weights of different position information can be better captured, different weights are given to focus areas and non-focus areas, denseNet are connected densely between different convolution layers, the feature information of the previous layer can be better utilized, and the feature restoration capability is improved.
In one embodiment, as shown in fig. 2, the present application further provides a device for segmenting a plateau pulmonary edema chest film focus based on a transducer, which comprises:
an acquisition module 201, configured to acquire a training sample and a target image;
The processing module 202 is configured to pre-process the training sample to generate a labeled training sample; constructing a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread Transformer deep learning model, and the up-sampling model is a DenseNet convolutional neural network model; training the neural network segmentation model by the marked training sample to generate a trained neural network segmentation model; inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus; and adding the target image into the training sample set for updating the neural network segmentation model.
The method comprises the steps of obtaining a training sample and a target image, preprocessing the training sample, generating a marked training sample, and constructing a U-shaped full convolution neural network segmentation model, wherein the U-shaped full convolution neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread transform deep learning model, and the up-sampling model is a DenseNet convolution neural network model. Training the neural network segmentation model by the labeled training sample, generating a trained neural network segmentation model, inputting a target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus, and adding the target image into a training sample set for updating the neural network segmentation model. The accuracy and precision of the segmentation of the plateau pulmonary edema chest film focus are improved, the focus area can be better identified, and more accurate diagnosis results and treatment suggestions are provided for clinicians, so that the medical level and treatment effect are improved. Meanwhile, the network based on the transducer has stronger self-adaptive capacity and generalization capacity, can adapt to different types of chest radiography images, and has good application prospect.
In another embodiment of the present application, the processing module 202 is configured to connect the upsampling model and the downsampling model through a Skip Connection layer, including: the output end of the Swin transform deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection layer and the bottom feature layer; performing size amplification processing on the output end of the DenseNet convolutional neural network model through up-sampling to generate preprocessed model features; and connecting the preprocessed model features with the Skip Connection layer.
In another embodiment of the present application, the processing module 202 is configured to perform the preprocessing on the training samples to generate labeled training samples, and includes: performing data enhancement processing on the training samples to generate enhanced training samples; and processing the enhanced training sample based on a preset partitioning rule to generate a focus area and a non-focus area.
In another embodiment of the present application, the processing module 202 is configured to perform the preprocessing on the training samples to generate labeled training samples, and further includes: carrying out normalization pretreatment on the training sample to generate a training sample with target brightness; acquiring a preset classification rule, wherein the preset classification rule comprises segmentation data of a plurality of different focus types; and processing the training samples of the target brightness based on the preset classification rule to generate classified training samples.
In another embodiment of the present application, the processing module 202 is configured to input the target image into the trained segmentation model for processing, output a target case, and includes: graying treatment is carried out on the target image, and a target color image is generated; performing image normalization processing on the target color image to generate a color image with a preset size; slicing the color images to sequentially generate three continuous slice images; processing the slice picture to generate a three-channel image; and inputting the three-channel image into the trained segmentation model for processing, and outputting a target case.
In another embodiment of the present application, the processing module 202 is configured to perform slicing processing on the color image, and sequentially generate three consecutive slice images, including: processing the color image based on a preset color division rule to generate a first color image containing different classification numbers; dividing the first color image based on the classification number to generate a plurality of initial slice images; preprocessing the initial slice image based on a target segmentation model to generate a preprocessed slice image; and processing the preprocessed slice images based on preset bilinear interpolation to generate a plurality of slice images with preset sizes.
In another embodiment of the present application, the processing module 202 is configured to perform slicing processing on the color image, and sequentially generate three consecutive slice images, and further includes: acquiring a preset size value to process the color image, and generating a panoramic image with a preset size; performing multi-layer slicing processing on the panoramic image to generate a plurality of slice image groups with different depth levels; and processing the slice image group based on time sequence or attribute information to generate a plurality of slice images.
According to the application, a training sample and a target image are obtained, data enhancement processing is carried out on the training sample, the enhanced training sample is generated, the enhanced training sample is processed based on a preset division rule, a focus area and a non-focus area are generated, normalization preprocessing is carried out on the training sample, and the training sample with target brightness is generated. The method comprises the steps of obtaining a preset classification rule, wherein the preset classification rule comprises a plurality of segmentation data of different focus types, processing training samples of target brightness based on the preset classification rule, generating classified training samples, constructing a U-shaped full-convolution neural network segmentation model, wherein the U-shaped full-convolution neural network segmentation model comprises an up-sampling model and a down-sampling model, the output end of a Swin transform deep learning model is connected to the input end of a DenseNet convolutional neural network model through a Skip Connection layer and a bottom feature layer, the output end of the DenseNet convolutional neural network model is subjected to size amplification processing through up-sampling, preprocessed model features are generated, the preprocessed model features are connected with the Skip Connection layer, the down-sampling model is the Swin transform deep learning model, the up-sampling model is the DenseNet convolutional neural network model, training the labeled training samples on the neural network segmentation model, and generating a trained neural network segmentation model.
In addition, the target image is subjected to graying processing to generate a target color image, the target color image is subjected to image normalization processing to generate a color image with a preset size, the color image is processed based on preset color division rules to generate a first color image containing different classification numbers, the first color image is divided based on the classification numbers to generate a plurality of initial slice images, the initial slice images are preprocessed based on a target segmentation model to generate a preprocessed slice image, the preprocessed slice images are processed based on preset bilinear interpolation to generate a plurality of slice images with a preset size, the color image is processed to obtain a preset size value, the panoramic image with the preset size is generated, the panoramic image is subjected to multi-layer slice processing to generate a plurality of slice image groups with different depth levels, the slice image groups are processed based on time sequence or attribute information to generate a plurality of slice images, the three-channel image is generated, the three-channel image is input into the trained segmentation model to process, and the target case is output, wherein the target case comprises a segmentation result of a plateau pulmonary edema, the target case is added into the training sample set for updating a neural network model. The method has the advantages that the problems that segmentation accuracy is low, focus areas cannot be accurately identified and the like in the prior art can be effectively solved by adopting a transducer network to perform feature extraction and focus segmentation on chest images, in addition, the Swin transducer introduces a local and global multi-level window attention mechanism, weights of different position information can be better captured, different weights are given to focus areas and non-focus areas, denseNet are connected densely between different convolution layers, the feature information of the previous layer can be better utilized, and the feature restoration capability is improved.
The embodiment of the application provides an electronic device, as shown in fig. 3, which comprises a processor 300, a memory 301, a bus 302 and a communication interface 303, wherein the processor 300, the communication interface 303 and the memory 301 are connected through the bus 302; the memory 301 stores a computer program that can be executed by the processor 300, and when the processor 300 executes the method for segmenting a plateau pneumochysis chest film focus based on a transducer according to any of the above embodiments of the present application.
The memory 301 may include a high-speed random access memory (RAM: random Access Memory), and may further include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the system network element and at least one other network element is implemented via at least one communication interface 303 (which may be wired or wireless), the internet, a wide area network, a local network, a metropolitan area network, etc. may be used.
Bus 302 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be classified as address buses, data buses, control buses, etc. The memory 301 is configured to store a program, and the processor 300 executes the program after receiving an execution instruction, and the segmentation method of the plateau pulmonary edema chest focus based on the transducer disclosed in any of the foregoing embodiments of the present application may be applied to the processor 300 or implemented by the processor 300.
The processor 300 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 300 or by instructions in the form of software. The processor 300 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but may also be a Digital Signal Processor (DSP), application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied as a hardware decoding processor executing or a combination of hardware and software modules executing in the decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the memory 301, and the processor 300 reads the information in the memory 301, and in combination with its hardware, performs the steps of the above method.
The electronic device provided by the embodiment of the application and the method for segmenting the plateau pulmonary edema chest film focus based on the Transformer provided by the embodiment of the application have the same beneficial effects as the method adopted, operated or realized by the application program stored in the electronic device because of the same inventive concept.
An embodiment of the present application provides a computer readable storage medium, as shown in fig. 4, where the computer readable storage medium stores 401 a computer program, and when the computer program is read and executed by a processor 402, the method for segmenting a plateau pulmonary edema chest focus based on a transducer as described above is implemented.
The technical solution of the embodiment of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing an electronic device (which may be an air conditioner, a refrigeration device, a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method of the embodiment of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.
The computer readable storage medium provided by the above embodiment of the present application has the same beneficial effects as the method adopted, operated or implemented by the application program stored in the computer readable storage medium, because the same inventive concept is adopted by the segmentation method of the plateau pulmonary edema chest film focus based on the transducer provided by the embodiment of the present application.
Embodiments of the present application provide a computer program product comprising a computer program for execution by a processor to perform a method as described above.
The computer program product provided by the embodiment of the application and the segmentation method of the plateau pulmonary edema chest film focus based on the Transformer provided by the embodiment of the application have the same beneficial effects as the method adopted, operated or realized by the application program stored by the computer program product of the embodiment of the application are based on the same inventive concept.
It is noted that in the present application, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The embodiments of the present application are described in a related manner, and the same similar parts between the embodiments are all mutually referred, and each embodiment is mainly described in the differences from the other embodiments. In particular, regarding the method, the electronic device, the electronic apparatus, and the readable storage medium embodiments for segmenting the plateau pneumochysis thoracic film focus based on the transducer, since the method is substantially similar to the above-described embodiment of the method for segmenting the plateau pneumochysis thoracic film focus based on the transducer, the description is relatively simple, and the relevant points are referred to the part of the description of the above-described embodiment of the method for segmenting the plateau pneumochysis thoracic film focus based on the transducer.
Although the present application is disclosed above, the present application is not limited thereto. Various changes and modifications may be made by one skilled in the art without departing from the spirit and scope of the application, and the scope of the application should be assessed accordingly to that of the appended claims.
Claims (6)
1. A method for segmenting a plateau pulmonary edema chest lesion based on a transducer, comprising:
acquiring a training sample and a target image;
preprocessing the training sample to generate a marked training sample;
Constructing a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread Transformer deep learning model, and the up-sampling model is a DenseNet convolutional neural network model;
training the neural network segmentation model by the marked training sample to generate a trained neural network segmentation model;
Inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus;
The step of inputting the target image into the trained segmentation model for processing and outputting a target case comprises the following steps:
graying treatment is carried out on the target image, and a target color image is generated;
Performing image normalization processing on the target color image to generate a color image with a preset size;
slicing the color images to sequentially generate three continuous slice images;
Processing the slice picture to generate a three-channel image;
Inputting the three-channel image into the trained segmentation model for processing, and outputting a target case;
The slicing processing is performed on the color images, and three continuous slice images are sequentially generated, including:
Processing the color image based on a preset color division rule to generate a first color image containing different classification numbers;
dividing the first color image based on the classification number to generate a plurality of initial slice images;
preprocessing the initial slice image based on a target segmentation model to generate a preprocessed slice image;
processing the preprocessed slice images based on preset bilinear interpolation to generate a plurality of slice images with preset sizes;
adding the target image into the training sample set for updating the neural network segmentation model;
The up-sampling model and the down-sampling model are connected through a Skip Connection layer, and the method comprises the following steps:
the output end of the Swin transform deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection layer and the bottom feature layer;
performing size amplification processing on the output end of the DenseNet convolutional neural network model through up-sampling to generate preprocessed model features;
the preprocessed model features are connected with the Skip Connection layer;
The slicing processing is performed on the color images, three continuous slice images are sequentially generated, and the method further comprises the following steps:
Acquiring a preset size value to process the color image, and generating a panoramic image with a preset size;
performing multi-layer slicing processing on the panoramic image to generate a plurality of slice image groups with different depth levels;
And processing the slice image group based on time sequence or attribute information to generate a plurality of slice images.
2. The method of claim 1, wherein the preprocessing the training sample to generate the annotated training sample comprises:
Performing data enhancement processing on the training samples to generate enhanced training samples;
And processing the enhanced training sample based on a preset partitioning rule to generate a focus area and a non-focus area.
3. The method of claim 1, wherein the preprocessing the training sample to generate a labeled training sample further comprises:
carrying out normalization pretreatment on the training sample to generate a training sample with target brightness;
Acquiring a preset classification rule, wherein the preset classification rule comprises segmentation data of a plurality of different focus types;
and processing the training samples of the target brightness based on the preset classification rule to generate classified training samples.
4. A transducer-based segmentation apparatus for a plateau pulmonary edema chest film lesion, the apparatus comprising:
the acquisition module is used for acquiring training samples and target images;
The processing module is used for preprocessing the training samples and generating marked training samples; constructing a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model comprises an up-sampling model and a down-sampling model, the up-sampling model and the down-sampling model are connected through a Skip Connection layer, the down-sampling model is a thread Transformer deep learning model, and the up-sampling model is a DenseNet convolutional neural network model; training the neural network segmentation model by the marked training sample to generate a trained neural network segmentation model; Inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case comprises a segmentation result of a plateau pulmonary edema focus; the step of inputting the target image into the trained segmentation model for processing and outputting a target case comprises the following steps: graying treatment is carried out on the target image, and a target color image is generated; performing image normalization processing on the target color image to generate a color image with a preset size; slicing the color images to sequentially generate three continuous slice images; processing the slice picture to generate a three-channel image; inputting the three-channel image into the trained segmentation model for processing, and outputting a target case; The slicing processing is performed on the color images, and three continuous slice images are sequentially generated, including: processing the color image based on a preset color division rule to generate a first color image containing different classification numbers; dividing the first color image based on the classification number to generate a plurality of initial slice images; preprocessing the initial slice image based on a target segmentation model to generate a preprocessed slice image; processing the preprocessed slice images based on preset bilinear interpolation to generate a plurality of slice images with preset sizes; adding the target image into the training sample set for updating the neural network segmentation model; The up-sampling model and the down-sampling model are connected through a Skip Connection layer, and the method comprises the following steps: the output end of the Swin transform deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection layer and the bottom feature layer; performing size amplification processing on the output end of the DenseNet convolutional neural network model through up-sampling to generate preprocessed model features; the preprocessed model features are connected with the Skip Connection layer; The slicing processing is performed on the color images, three continuous slice images are sequentially generated, and the method further comprises the following steps: acquiring a preset size value to process the color image, and generating a panoramic image with a preset size; performing multi-layer slicing processing on the panoramic image to generate a plurality of slice image groups with different depth levels; and processing the slice image group based on time sequence or attribute information to generate a plurality of slice images.
5. An electronic device, comprising:
a processor; and a memory for storing executable instructions of the processor;
Wherein the processor is configured to perform the method of segmenting a transducer-based altitude pulmonary edema chest lesion of any one of claims 1-3 via execution of the executable instructions.
6. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements a method for segmenting a transducer-based chest lesion of altitude pulmonary edema according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311191474.4A CN117218135B (en) | 2023-09-15 | Method and related equipment for segmenting plateau pulmonary edema chest film focus based on transducer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311191474.4A CN117218135B (en) | 2023-09-15 | Method and related equipment for segmenting plateau pulmonary edema chest film focus based on transducer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117218135A CN117218135A (en) | 2023-12-12 |
| CN117218135B true CN117218135B (en) | 2024-11-19 |
Family
ID=
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114882047A (en) * | 2022-04-19 | 2022-08-09 | 厦门大学 | Medical image segmentation method and system based on semi-supervision and Transformers |
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114882047A (en) * | 2022-04-19 | 2022-08-09 | 厦门大学 | Medical image segmentation method and system based on semi-supervision and Transformers |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Pinaya et al. | Unsupervised brain imaging 3D anomaly detection and segmentation with transformers | |
| US11568533B2 (en) | Automated classification and taxonomy of 3D teeth data using deep learning methods | |
| CN111784671B (en) | Pathological image focus region detection method based on multi-scale deep learning | |
| CN109978037B (en) | Image processing method, model training method, device and storage medium | |
| US10043088B2 (en) | Image quality score using a deep generative machine-learning model | |
| JP6885517B1 (en) | Diagnostic support device and model generation device | |
| JP2020064609A (en) | Patient-specific deep learning image denoising methods and systems | |
| CN116097302A (en) | Connected machine learning model with joint training for lesion detection | |
| CN113539402B (en) | Multi-mode image automatic sketching model migration method | |
| KR102097743B1 (en) | Apparatus and Method for analyzing disease based on artificial intelligence | |
| JP2023515367A (en) | Out-of-distribution detection of input instances to model | |
| CN111091575B (en) | Medical image segmentation method based on reinforcement learning method | |
| CN117392082A (en) | Liver CT image segmentation method and system based on full-scale jump connection | |
| Goutham et al. | Automatic localization of landmarks in cephalometric images via modified U-Net | |
| CN114693671B (en) | Lung nodule semi-automatic segmentation method, device, equipment and medium based on deep learning | |
| CN114972266A (en) | Lymphoma ultrasonic image semantic segmentation method based on self-attention mechanism and stable learning | |
| CN117218135B (en) | Method and related equipment for segmenting plateau pulmonary edema chest film focus based on transducer | |
| CN113379770B (en) | Construction method of nasopharyngeal carcinoma MR image segmentation network, image segmentation method and device | |
| CN114049315B (en) | Joint recognition method, electronic device, storage medium, and computer program product | |
| CN113793316B (en) | Ultrasonic scanning area extraction method, device, equipment and storage medium | |
| CN117218135A (en) | Method and related equipment for segmenting plateau pulmonary edema chest film focus based on transducer | |
| CN115294023A (en) | Liver tumor automatic segmentation method and device | |
| CN115359005A (en) | Image prediction model generation method, device, computer equipment and storage medium | |
| CN115081637A (en) | In-situ training of machine learning algorithms for generating synthetic imaging data | |
| CN112967295A (en) | Image processing method and system based on residual error network and attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |