CN117078780A - Deep learning-based micro-fossil CT image preprocessing method and device - Google Patents
Deep learning-based micro-fossil CT image preprocessing method and device Download PDFInfo
- Publication number
- CN117078780A CN117078780A CN202310863239.0A CN202310863239A CN117078780A CN 117078780 A CN117078780 A CN 117078780A CN 202310863239 A CN202310863239 A CN 202310863239A CN 117078780 A CN117078780 A CN 117078780A
- Authority
- CN
- China
- Prior art keywords
- image
- super
- resolution
- network
- micro
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000013135 deep learning Methods 0.000 title claims abstract description 38
- 238000007781 pre-processing Methods 0.000 title claims abstract description 24
- 230000006870 function Effects 0.000 claims abstract description 23
- 238000012549 training Methods 0.000 claims description 28
- 239000013598 vector Substances 0.000 claims description 28
- 230000008447 perception Effects 0.000 claims description 14
- 230000007246 mechanism Effects 0.000 claims description 13
- 230000008485 antagonism Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 10
- 210000002500 microbody Anatomy 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 3
- 230000015556 catabolic process Effects 0.000 claims description 3
- 238000006731 degradation reaction Methods 0.000 claims description 3
- 230000008034 disappearance Effects 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- PMGQWSIVQFOFOQ-YKVZVUFRSA-N clemastine fumarate Chemical compound OC(=O)\C=C\C(O)=O.CN1CCC[C@@H]1CCO[C@@](C)(C=1C=CC(Cl)=CC=1)C1=CC=CC=C1 PMGQWSIVQFOFOQ-YKVZVUFRSA-N 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 claims description 2
- 238000010603 microCT Methods 0.000 description 9
- QTBSBXVTEAMEQO-UHFFFAOYSA-N Acetic acid Chemical compound CC(O)=O QTBSBXVTEAMEQO-UHFFFAOYSA-N 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 229910021487 silica fume Inorganic materials 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 210000001519 tissue Anatomy 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 241000251539 Vertebrata <Metazoa> Species 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000004927 clay Substances 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 229910052500 inorganic mineral Inorganic materials 0.000 description 1
- 238000000386 microscopy Methods 0.000 description 1
- 239000011707 mineral Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
- G06T11/005—Specific pre-processing for tomographic reconstruction, e.g. calibration, source positioning, rebinning, scatter correction, retrospective gating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a deep learning-based micro-fossil CT image preprocessing method and device, wherein the method comprises the following steps: acquiring CT images of the micro fossil; obtaining a super-resolution image of the CT image based on the CT image super-resolution reconstruction model; and obtaining a CT image with high signal-to-noise ratio corresponding to the super-resolution image based on the denoising network model. The invention can efficiently and objectively perform super-resolution reconstruction and noise removal functions on the micro-fossil CT image so as to solve the problems of low resolution of the micro-fossil CT image, low signal-to-noise ratio of the CT image and the like.
Description
Technical Field
The invention relates to the technical field of micro-fossil CT image preprocessing, in particular to a micro-fossil CT image preprocessing method and device based on deep learning.
Background
The microvesicle fossil is a microscopic ancient organism remains or remains stored in a stratum and cannot be directly observed by naked eyes. The micro-fossil contains a great deal of abundant geological information and has very important significance for the scientific research of the earth, life and environment. In recent years, thanks to the wide use of the micro-CT technique in micro-ancient biologies, ancient biologists can obtain more comprehensive three-dimensional data information of micro-fossils, including three-dimensional microstructures from the inside to the surface of the micro-fossils, through micro-CT scanning. However, in the process of acquiring the micro-fossil CT image, the micro-CT image is limited by software, hardware and environmental factors of the micro-CT device, and various problems that CT image noise and CT image resolution cannot meet the identification requirement of the micro-fossil structure are unavoidable.
The resolution of the micro-fossil CT image acquired by the micro-CT system is limited by factors such as the size of the focal spot of the X-ray source, the pixel of the imaging detector, the size of the micro-fossil sample, and the like. Thus, the archaeological specialist can only trade off between the micro-foster CT image field of view and the image voxel size. However, CT image voxel size is critical to describe the microstructure of the microposts (e.g., texture, medullary cavity holes, ridges, etc.). At present, the ancient biological specialist wants to observe the characteristics or further observe the super-microscopic structure of the micro-fossils, even the ancient tissue details, and the expensive hardware equipment is needed or the traditional image interpolation-based method is adopted to increase the voxel size of the CT image, so that the super-resolution reconstruction of the CT image of the micro-fossils is realized.
The signal-to-noise ratio of the micro fossil CT image acquired by the micro CT system is limited by factors such as the exposure time of an X-ray machine, the number of superimposed frames, the number of projection images and the like. Therefore, the archaeological specialist can only trade off between the time of CT scanning of the micro-fossils and the high signal-to-noise ratio of CT images. However, CT images with high signal-to-noise ratios are critical to distinguishing the texture detail features of the micro-fossils. Currently, ancient biological specialists want to acquire high signal-to-noise ratio micro-fossil CT images, and generally adopt a traditional denoising method based on artificial features, for example, a discrete cosine transform, a wavelet transform and the like.
The conventional micro-fossil CT image preprocessing method has the following problems and disadvantages: the traditional image interpolation-based method is adopted to realize super-resolution reconstruction of the micro-fossil CT image, although the method is simple and rough and has small operand, the pixel values of adjacent pixel points can be directly copied for filling, but the reconstruction effect is poor and the blocking effect is obvious by the method of moving surrounding pixels; the conventional denoising method generally needs to adjust parameters, so that experienced staff is required to repeatedly adjust the parameters to exert the maximum performance of the algorithm. In addition, the calculation amount of some algorithms is huge, and the application of the algorithms in the pretreatment of the micro-fossil CT images is limited.
Disclosure of Invention
In order to overcome the defects of the traditional micro-fossil CT image preprocessing method, the invention provides a novel micro-fossil CT image preprocessing method and device based on deep learning, which can efficiently and objectively perform super-resolution reconstruction and noise removal functions on micro-fossil CT images so as to solve the problems of low resolution of the micro-fossil CT images, low signal-to-noise ratio of the CT images and the like.
The technical scheme of the invention comprises the following steps:
a deep learning-based micro-fossils CT image preprocessing method, the method comprising:
acquiring CT images of the micro fossil;
based on a CT image super-resolution reconstruction model, obtaining a super-resolution image of the CT image; the CT image super-resolution reconstruction model is obtained by calculating perception loss, antagonism loss, total variation loss and content loss by using a high-definition-low-definition original CT image pair, wherein the perception loss is obtained based on a feature vector distance between a high-definition original CT image and a corresponding super-resolution image, the antagonism loss is obtained by discriminating a high-definition original CT image and a reconstruction result of the super-resolution network model, the total variation loss is obtained based on a sharpness difference between image gradients of the high-definition original CT image and the corresponding super-resolution image due to regional variation, and the content loss is obtained based on a reconstruction result of the super-resolution network model and a pixel level distance between the high-definition original CT image;
and obtaining the CT image with high signal-to-noise ratio corresponding to the super-resolution image based on the denoising network model.
Further, the backbone network of the CT image super-resolution reconstruction model is a WSRGAN network, and the WSRGAN network comprises: a generation network and a countermeasure network, the generation network being comprised of an encoding structure and a decoding structure, the encoding structure comprising a plurality of encoding modules, each encoding module comprising: residual block, convolution layer, model weight normalization layer, reLU activation layer, channel attention block, spatial attention block and upsampling function; the decoding structure comprises a plurality of decoding modules, and the countermeasure network establishes a structure comprising a VGG19 network model, a LeakyReLU activation function, a convolution layer and a model weight normalization layer based on a VGG architecture.
Further, training the CT image super-resolution reconstruction model includes:
constructing an experimental data set based on a plurality of high-definition-low-definition original CT image pairs, and dividing all images in the experimental data set into a plurality of batches;
loading a pre-trained WSRGAN network original weight file;
inputting a low-definition original CT image to a generating network so that the encoding module extracts a feature vector from the low-definition original CT image, and generates an output vector to be input to a next encoding module, wherein the encoding module extracts the feature vector from the output vector; the feature vector output by the encoding module of the next stage obtains higher-dimensional image feature information than the feature vector output by the encoding module of the previous stage;
inputting the final feature vector extracted by the coding structure into a decoding structure, and generating a super-resolution image corresponding to the original CT image by a last decoding module, and obtaining perception loss based on the Euclidean distance of the feature vector between the high-definition original CT image and the corresponding super-resolution image;
inputting the low-definition original CT image into a generating network to obtain a reconstruction result of a super-resolution network model, inputting the reconstruction result and the high-definition original CT image into an countermeasure network together, and distinguishing the reconstruction image of the high-definition original CT image and the super-resolution network model based on a discriminator to obtain countermeasure loss;
using variable step sizesAnd updating the weight W of each batch by a random gradient descent algorithm i+1 Restarting the optimization weight by using the LRS strategy to obtain an optimal weight file of the WSRGAN network model; wherein, table iShowing batch, j represents algebra of training;
the total variation loss is obtained by minimizing the total variation of gradient operators in the horizontal direction and the vertical direction of the pixel level of the super-resolution reconstructed image and the high-definition original CT image;
calculating a pixel-level distance between a super-resolution reconstructed image result and a high-definition original CT image based on the mean square error loss to obtain content loss;
and training and optimizing the WSRGAN network according to the perception loss, the antagonism loss, the total variation loss and the content loss, so as to obtain a CT image super-resolution reconstruction model.
Further, the variable step sizeWherein alpha is max Representing a variable step +.>Maximum value range of alpha min Representing a variable step +.>C is the cycle length of the cos function, S is the number of iterations of the restart step, b i The number of micro-fostered CT images per batch.
Further, the weightWherein (1)>Gradient value, X for the ith generation of training j For the high-definition original CT image set, Y j For a high resolution image set generated based on the low definition raw CT image set, B represents the total batch of the ith generation of training.
Further, the obtaining the CT image with high signal-to-noise ratio corresponding to the super-resolution image based on the denoising network model includes:
constructing a denoising network model, wherein a backbone network of the denoising network model is a modified Noise to Noise network, and the modified Noise to Noise network comprises: a downsampling portion, a skip connecting portion, and an upsampling portion; the feature images of the same stage of the downsampling part and the upsampling part have the same size and the same channel number, the downsampling part solves gradient disappearance and network degradation by introducing a residual error module, and the jump connection part fuses different layers of multi-scale feature images extracted by the convolutional neural network by introducing an attention mechanism;
training the improved Noise to Noise network by inputting micro-fossil CT images with different levels of Gaussian random Noise with zero mean as input data and tag data respectively, and monitoring the training process of the improved Noise to Noise network by the signal-to-Noise ratio of the denoised images to obtain denoising loss L noise ；
According to denoising loss L noise Performing the improved Noise to Noise network training and optimization to obtain a denoising network model;
and inputting the super-resolution image into a denoising network model to obtain a CT image with high signal-to-noise ratio.
Further, the denoising penaltyWherein y is t Represents the t-th noisy micro-body fossil CT data, θ represents network parameters, +.>Another noisy distribution of the t-th noisy micro-body CT data is represented, and N represents the total number of noisy micro-body CT data.
A deep learning based micro-fossils CT image preprocessing device, the device comprising:
the acquisition module is used for acquiring CT images of the micro fossil;
the generation module is used for obtaining a super-resolution image of the CT image based on the CT image super-resolution reconstruction model; the CT image super-resolution reconstruction model is obtained by calculating perception loss, antagonism loss, total variation loss and content loss by using a high-definition-low-definition original CT image pair, wherein the perception loss is obtained based on a feature vector distance between a high-definition original CT image and a corresponding super-resolution image, the antagonism loss is obtained by identifying a reconstruction result of the high-definition original CT image and the super-resolution network model, the total variation loss is obtained based on a sharpness difference generated by image gradients of the high-definition original CT image and the corresponding super-resolution image due to region change, and the content loss is obtained based on a reconstruction result of the super-resolution network model and a pixel level distance between the high-definition original CT image;
and the denoising module is used for obtaining the CT image with high signal-to-noise ratio corresponding to the super-resolution image based on the denoising network model.
A computer device, the computer device comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the deep learning-based micro-fossils CT image preprocessing method of any one of the above.
A computer readable storage medium, wherein computer program instructions are stored on the computer readable storage medium, and when executed by a processor, the computer program instructions implement the deep learning-based micro-fossils CT image preprocessing method according to any one of the above.
Compared with the prior art, the invention has the following advantages:
1) The deep learning technology is applied to develop super-resolution reconstruction work of the micro-fossil CT image, so that the spatial resolution and the density resolution of the micro-fossil CT image are improved, and the research requirement of ancient specialists on fine observation of the micro-fossil three-dimensional microstructure is met;
2) The traditional method for removing the micro-fossil CT image noise needs experienced staff to repeatedly adjust parameters to exert the maximum performance of the algorithm. The CT image denoising method based on the deep learning can play the maximum performance of the deep learning method without constructing a noiseless data set.
Drawings
Fig. 1 is a flowchart of a method for preprocessing a CT image of a micro-fossil according to an embodiment of the present invention.
Fig. 2 is a flow chart for creating a super-resolution reconstruction network model of a micro-fossil CT image based on deep learning according to an embodiment of the present invention.
Fig. 3 is a flowchart of creating a denoising network model of a micro-fossil CT image based on deep learning according to an embodiment of the present invention.
Fig. 4 is a block diagram of a micro-fossil CT image preprocessing apparatus according to an embodiment of the present invention.
Detailed Description
The invention will be described in further detail with the aim of making the objects, solutions and advantages of the invention more clear, taking experiments performed on a real dataset as an example. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The invention relates to a preprocessing method of a micro-fossil CT image, which comprises the following steps: acquiring a CT image of the micro-fossils, performing super-resolution reconstruction of the CT image of the micro-fossils based on deep learning, and denoising the CT image of the micro-fossils based on the deep learning.
Specifically, as shown in fig. 1, the present invention has the steps of:
step 1: CT images of the microvesicle fossils were acquired.
According to the invention, the micro-CT image is acquired by using the micro-CT according to the micro-CT image acquisition standard. The format of the micro-fossils CT image can be a DICOM-format micro-fossils CT image, or a non-DICOM common image BMP/TIF-format micro-fossils CT image.
In one example, the micro-fossils were derived from early vertebrate micro-fossils from the mid-juveniles to the clay pot of Yunnan China. After the treatment of the microsilica with diluted (3% -7%) acetic acid solution, CT images of the microsilica were acquired using GE v|tome| x m300&180 microscopy CT. Wherein, the parameters of GE v|tome| x m300&180 micro-industrial CT are set as follows: voltage: 90kV; current flow: 90uA; exposure time: 2000ms; superimposed frame number: 1 frame; number of projection images: 2500 sheets; voxel size: 3um.
Step 2: and obtaining a super-resolution image of the CT image based on the CT image super-resolution reconstruction model.
The CT image is directly input into a trained deep learning-based micro-fossil CT image super-resolution reconstruction WSRGAN network model, namely, the trained generation network is used for obtaining the micro-fossil CT image with high resolution, realizing the super-resolution reconstruction of the micro-fossil CT image and outputting the micro-fossil CT image super-resolution image.
The super-resolution reconstruction WSRGAN network model uses a weight-normalized generation type countermeasure network. The generation network part is used for generating the super-resolution micro-body fossil CT image, and the countermeasure network part judges the fidelity degree of the generation network generated image and feeds the fidelity degree back to the generation network. The generation network and the countermeasure network part are added with weight normalization layers, and the weight normalization layers WN are defined as follows:
wherein the vector iiv is the euclidean norm of V, W, V being the K-dimensional vector. g is a scalar quantity, and g is a scalar quantity, g= |w|. W is the minor state of V. The introduction of WN can obtain better training and testing accuracy.
The super-resolution reconstruction WSRGAN network model adopts a dual attention mechanism, namely a channel attention mechanism and a space attention mechanism are combined. The channel attention mechanism can adaptively adjust the weight of the channel, and effectively improve the super-resolution reconstruction index. The spatial attention mechanism is an effective supplement to the channel attention mechanism, and the introduction of the spatial attention mechanism can lead the feature extraction to be concentrated on the detail part of the high-frequency region, inhibit the weight of the low-frequency region, and be beneficial to reconstructing the high-frequency detail in the image, in particular to structures such as textures, medullary cavity holes, ridges and the like in the micro-fossils.
In one example, when the super-resolution reconstruction network model is trained, an experimental dataset is constructed by inputting a plurality of high-definition (2048 x 2048) -low-definition (512 x 512) CT image pairs, and then the super-resolution reconstruction WSRGAN network model of the micro-body fossil CT image based on deep learning is trained, verified and tested. Loading an original weight file of a pre-trained WSRGAN network model, performing countermeasure learning through a high-resolution micro-fossil CT image and a low-resolution micro-fossil CT image, continuously regressing, training a deep learning-based micro-fossil CT image super-resolution reconstruction WSRGAN network model, adjusting the reconstruction precision of the WSRGAN network model, and storing the parameter weights of the optimized WSRGAN network model. In the loss function part, in order to reduce adverse effects caused by feature matching errors, the invention adds total variation loss on the basis of the context loss function, so that gradients in the image can be changed in a smaller area, the image can generate a certain sharpness, the detail texture of the image can be improved to a certain extent, and the reconstruction of structures such as textures, medullary cavity holes and ridges in the micro-fossils is facilitated. And judging the relative reality of the micro-fossil CT image by adopting a relative discriminator, and optimizing the super-resolution effect of the micro-fossil CT image.
As shown in fig. 2, the embodiment of the invention provides a super-resolution reconstruction WSRGAN network model creation flow based on deep learning micro-fossil CT images. The low resolution 512 x 512 micro-fossils CT images in the experimental dataset were input into a codec-based generator. The generator has 6 modules, each comprising a residual block, a convolution layer, a model weight normalization layer, a ReLU activation layer, a channel attention block, a spatial attention block, and an upsampling function, etc. Each module may extract feature vectors from the low resolution image and then generate an output vector for input to the next module. The feature vectors represent the main features of the input low resolution image and the decoder is responsible for upsampling the feature vectors. The last decoding module can reconstruct the micro-fossil CT image with the resolution of 2048 x 2048.
And loading the high-resolution 2048 x 2048 micro-fossil CT image in the experimental data set and the super-resolution reconstruction result of the micro-fossil CT image in the VGG19 network model-based discriminator pre-trained in the ImageNet data set, and judging the relative authenticity of the micro-fossil CT image. The arbiter comprises a VGG19 network model, a LeakyReLU activation function, a convolution layer, a model weight normalization layer and other structures.
In the training process, the method is used for reconstructing the micro-fossil CT image in super-resolution in order to save the optimal weight of the WSRGAN network model. The invention adopts an LRS (learning rate decay strategy) of an optimization weight algorithm. The LRS is a strategy capable of restarting the optimization weight in each epoch stage, so that the convergence rate of the network model is further improved. In order to make the network model easier to converge to local minima, the present invention updates the weights of each epoch stage with a variable step-size α and random gradient descent (SGD) algorithm, while dividing all images in the training set into B batches. The weight value W of the WSRGAN network model updated by the SGD is as follows:
wherein,gradient value, X for the ith generation of training j For the original high resolution image set, Y j A set of high resolution images is generated for the generation network.
Restarting the optimization weight by using the LRS strategy, and minimizing the loss function of the WSRGAN network model. The variable step size α is defined as follows:
wherein,for the jth generation i batch, < ->The value range of (a) is [ alpha ] min ,α max ]，b i For the number of micro-fostered CT images per batch, c is the cycle length of the cos function, S is the number of iterations of the restart step.
Training deep learning-based micro fossil CTSuper-resolution reconstruction WSRGAN network model of image, iteration number set to 500, optimizer as Adam SGD function, batch size set to 16, initial learning rate set to [1×10 ] -5 ,1×10 -3 ]. The loss function of the countermeasure network model is a non-convex loss function, and the super-resolution reconstruction WSRGAN network model of the optimal deep learning-based micro-fossil CT image is obtained. The loss function includes four parts, namely, perceived loss, counterloss, content loss and total variation loss. Loss of perceptionThe method is based on VGG architecture establishment, and the VGG19 network is used for respectively extracting the original high-resolution image and the reconstructed super-resolution image feature vector, calculating the Euclidean distance between the original high-resolution image and the reconstructed super-resolution image feature vector, and focusing on the perception information of the image. Countering losses->And using a discriminator to discriminate the original high-resolution image and the reconstructed super-resolution image, and restricting a generator so as to improve the visual effect of the reconstructed image. Total variation loss->The gradient in the image can be changed in a smaller area, a certain sharpness is generated, and the detail texture of the image is improved to a certain extent. Content loss->In the image super-resolution reconstruction process, the pixel-level distance between the reconstructed super-resolution image and the original high-resolution image is calculated by using the mean square error loss, so that the training of the generator is constrained. The definition formula of the loss function L of the final model training is as follows:
wherein,for the perception of lost parts, ->To combat lost parts, let go of>In order to achieve the total variation loss,is the content loss part.
Step 3: and obtaining a CT image with high signal-to-noise ratio corresponding to the super-resolution image based on the denoising network model.
According to the invention, the micro-fossils CT image to be Noise-reduced is input into a trained deep learning-based micro-fossils CT image denoising Noise to Noise network model, so that a high signal-to-Noise ratio micro-fossils CT image is obtained, the Noise reduction treatment of the micro-fossils CT image is realized, and the high signal-to-Noise ratio micro-fossils CT image is output.
The improved Noise to Noise network model for processing the de-noising problem of the micro-fossil CT image mainly comprises the following three parts: downsampling to introduce residual modules, skip connections to introduce attention mechanism modules, and upsampling to introduce deconvolution modules. In the modified Noise to Noise network model, the feature map size and channel number of the same phase of the downsampling and upsampling portions are the same. When the feature map is input to the next stage of the downsampling section, the size is one half of the original size, and the number of channels is 2 times of the original size. Meanwhile, in the image denoising method based on deep learning, which is provided for the micro-fossil CT image, a residual error module is introduced into a sampling part of an original Noise to Noise network model. The residual error module solves the problems of gradient disappearance, network degradation and the like caused by deepening the layer number of the traditional convolutional neural network, and the network model with excellent performance is easier to obtain. Attention mechanism modules are introduced in the original Noise to Noise network model hopping connection section. The attention mechanism module can fuse different layers of multi-scale feature images extracted by the convolutional neural network, and is helpful for removing different types of noise in the micro-fossil CT image, such as hardening artifact, spiced salt noise, gaussian noise and the like. The network model is trained, verified and tested by inputting the micro-fossil CT images with different levels of Gaussian random noise with zero mean value as input data and label data respectively. Since the loss function loss does not gradually drop with iteration, but the Noise to Noise network model still converges, the invention monitors the network training process by the signal-to-Noise ratio (SNR) of the denoised image. The method comprises the steps of loading a pre-trained Noise to Noise network model original weight file, comparing micro-fossil CT images added with different levels of random Noise, continuously regressing, training a Noise to Noise network model based on deep learning of the micro-fossil CT images, adjusting and improving the signal to Noise ratio of the micro-fossil CT images after Noise removal, and storing the optimized parameter weight of the improved Noise to Noise network model.
In one example, the improved Noise to Noise network model uses a ResNet34 network model as a backbone network for downsampling, and the downsampling portion includes four phases, each phase is composed of a plurality of residual modules based on the ResNet34 network model, the first phase includes 3 residual modules, the second phase includes 4 residual modules, the third phase includes 6 residual modules, and the fourth phase includes 3 residual modules. The jump connection part adopts a feature pyramid and attention mechanism combined structure and is connected with the feature images with the same scale extracted in the same stage of downsampling encoding and upsampling decoding, so that the channel number of the feature images in the upsampling stage can be doubled to 128, 256, 512 and 1024 respectively. Each stage of up-sampling includes a deconvolution operation with deconvolution kernel size of 2 x 2, feature map concatenation, and two convolution operations with convolution kernel size of 3 x 3, with the 1 x 1 convolution layer outputting a high signal-to-noise ratio micro-fossilized CT image.
As shown in fig. 3, the embodiment of the invention provides a deep learning-based micro-fossil CT image denoising Noise to Noise network model creation flow. And (3) adding random Noise micromanipulation CT images of different grades into an input experimental data set to improve a downsampling stage of a Noise to Noise network model, wherein the stage consists of a plurality of residual modules based on a pre-trained ResNet34 network model. Loading in ImageNet datasetThe method comprises the steps of setting the initial weight file of a pre-trained ResNet34 network model, setting the iteration number to 100, setting an optimizer to an Adam function, setting the batch size to 16, setting the initial learning rate to 1e-3, training a deep learning-based micro-fossil CT image denoising Noise to Noise network model, and obtaining an optimal deep learning-based micro-fossil CT image denoising network model. Improving the loss function of a Noise to Noise network model to be L noise Function, loss function L noise The definition formula of (2) is as follows:
wherein y is i Represents noisy micro-body fossil CT data, theta represents network parameters,another noisy distribution representing noisy micro-fossil CT data.
In summary, aiming at the ancient biological specialists, the invention further observes the super-microscopic structure of the micro-foster and requires expensive hardware equipment, and acquires the micro-foster CT images with high resolution and low resolution through micro-CT to establish experimental data sets for training, verifying and testing the super-resolution reconstruction method of the micro-foster CT images based on the deep learning, which is provided by the invention; aiming at ancient tissue details of the micro-fossils further observed by ancient biological specialists, a CT image with high signal to noise ratio is needed. The deep learning-based micro-fossil CT image preprocessing method and device developed by the invention are expected to indicate potential directions for determining geologic ages, dividing and comparing strata and searching mineral resources.
Compared with the existing micro-fossil CT image pretreatment method, experiments show that:
in the process of super-resolution reconstruction of the micro-fossil CT image, compared with the traditional bilinear interpolation method, bicubic interpolation method, and EDSR, WDSR and SRGAN network models based on deep learning, as shown in table 1, the WSRGAN network model developed by using the end-to-end architecture improves the peak signal-to-noise ratio PSNR (the larger the numerical value is, the higher the image quality) of the similarity measurement peak and the LPIPS (the smaller the numerical value is, the higher the image quality is) of the image similarity measurement standard.
| Method | PSNR | LPIPS |
| Bilinear | 32.6754 | 0.4289 |
| Bicubic | 33.1636 | 0.4166 |
| EDSR | 33.1733 | 0.2281 |
| WDSR | 32.1440 | 0.2640 |
| SRGAN | 34.0559 | 0.1815 |
| The invention is that | 35.0615 | 0.0757 |
TABLE 1
In the process of denoising the micro-fossil CT image, compared with the traditional image denoising method of non-local mean value (NLM) image denoising method, as shown in table 2, the improved Noise to Noise network model adopted by the invention obviously improves the signal-to-Noise ratio SNR (the larger the numerical value is, the higher the image quality is).
TABLE 2
Referring to fig. 4, a block diagram of a micro-fossil CT image preprocessing apparatus according to an embodiment of the present invention is shown. The device can be a computer device or can be arranged in the computer device. As shown in fig. 4, the apparatus includes the following modules: an image acquisition module 410, an image reconstruction module 420, an image denoising module 430, and an image output module 440.
An image acquisition module 410 for acquiring CT images of the micro-fossils;
an image reconstruction module 420, configured to reconstruct the CT image to obtain a CT image super-resolution image;
the image denoising module 430 is configured to denoise the CT image super-resolution image to obtain a CT image with a high signal-to-noise ratio;
an image output module 440 for outputting a high resolution, high signal-to-noise ratio CT image;
for details of the specific implementation process, beneficial effects, etc. of the device module, please refer to the description of the above method embodiment, and the details are not repeated here.
In an exemplary embodiment, there is also provided a computer device including a memory and a processor, the memory storing a computer program loaded and executed by the processor to implement the above-described deep learning-based micro-fossils CT image preprocessing method.
In an exemplary embodiment, a computer readable storage medium is also provided, on which a computer program is stored, which when being executed by a processor implements a deep learning based micro-fossilized CT image preprocessing method as described above.
In an exemplary embodiment, a computer program product is also provided, which, when run on a computer device, causes the computer device to perform a deep learning based micro-fossilized CT image preprocessing method as described above.
The foregoing description of the embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.
Claims (10)
1. A deep learning-based micro-fossil CT image preprocessing method, the method comprising:
acquiring CT images of the micro fossil;
based on a CT image super-resolution reconstruction model, obtaining a super-resolution image of the CT image; the CT image super-resolution reconstruction model is obtained by calculating perception loss, antagonism loss, total variation loss and content loss by using a high-definition-low-definition original CT image pair, wherein the perception loss is obtained based on a feature vector distance between a high-definition original CT image and a corresponding super-resolution image, the antagonism loss is obtained by discriminating a high-definition original CT image and a reconstruction result of the super-resolution network model, the total variation loss is obtained based on a sharpness difference between image gradients of the high-definition original CT image and the corresponding super-resolution image due to regional variation, and the content loss is obtained based on a reconstruction result of the super-resolution network model and a pixel level distance between the high-definition original CT image;
and obtaining the CT image with high signal-to-noise ratio corresponding to the super-resolution image based on the denoising network model.
2. The method of claim 1, wherein the backbone network of the CT image super-resolution reconstruction model is a WSRGAN network, the WSRGAN network comprising: a generation network and a countermeasure network, the generation network being comprised of an encoding structure and a decoding structure, the encoding structure comprising a plurality of encoding modules, each encoding module comprising: residual block, convolution layer, model weight normalization layer, reLU activation layer, channel attention block, spatial attention block and upsampling function; the decoding structure comprises a plurality of decoding modules, and the countermeasure network establishes a structure comprising a VGG19 network model, a LeakyReLU activation function, a convolution layer and a model weight normalization layer based on a VGG architecture.
3. The method of claim 2, wherein training the CT image super-resolution reconstruction model comprises:
constructing an experimental data set based on a plurality of high-definition-low-definition original CT image pairs, and dividing all images in the experimental data set into a plurality of batches;
loading a pre-trained WSRGAN network original weight file;
inputting a low-definition original CT image to a generating network so that the encoding module extracts a feature vector from the low-definition original CT image, and generates an output vector to be input to a next encoding module, wherein the encoding module extracts the feature vector from the output vector; the feature vector output by the encoding module of the next stage obtains higher-dimensional image feature information than the feature vector output by the encoding module of the previous stage;
inputting the final feature vector extracted by the coding structure into a decoding structure, and generating a super-resolution image corresponding to the original CT image by a last decoding module, and obtaining perception loss based on the Euclidean distance of the feature vector between the high-definition original CT image and the corresponding super-resolution image;
inputting the low-definition original CT image into a generating network to obtain a reconstruction result of a super-resolution network model, inputting the reconstruction result and the high-definition original CT image into an countermeasure network together, and distinguishing the reconstruction image of the high-definition original CT image and the super-resolution network model based on a discriminator to obtain countermeasure loss;
using variable step sizesAnd updating the weight W of each batch by a random gradient descent algorithm i+1 Restarting the optimization weight by using the LRS strategy to obtain an optimal weight file of the WSRGAN network model; wherein i represents a batch, j represents a training algebra;
the total variation loss is obtained by minimizing the total variation of gradient operators in the horizontal direction and the vertical direction of the pixel level of the super-resolution reconstructed image and the high-definition original CT image;
calculating a pixel-level distance between a super-resolution reconstructed image result and a high-definition original CT image based on the mean square error loss to obtain content loss;
and training and optimizing the WSRGAN network according to the perception loss, the antagonism loss, the total variation loss and the content loss, so as to obtain a CT image super-resolution reconstruction model.
4. The method of claim 3, wherein the variable step size Wherein alpha is max Representing a variable step +.>Maximum value range of alpha min Representing a variable step +.>C is the cycle length of the cos function, S is the number of iterations of the restart step, b i The number of micro-fostered CT images per batch.
5. The method of claim 3, wherein the weightsWherein (1)>Gradient value, X for the ith generation of training j For the high-definition original CT image set, Y j For a high resolution image set generated based on the low definition raw CT image set, B represents the total batch of the ith generation of training.
6. The method of claim 1, wherein obtaining a high signal-to-noise ratio CT image corresponding to the super-resolution image based on a denoising network model comprises:
constructing a denoising network model, wherein a backbone network of the denoising network model is a modified Noise to Noise network, and the modified Noise to Noise network comprises: a downsampling portion, a skip connecting portion, and an upsampling portion; the feature images of the same stage of the downsampling part and the upsampling part have the same size and the same channel number, the downsampling part solves gradient disappearance and network degradation by introducing a residual error module, and the jump connection part fuses different layers of multi-scale feature images extracted by the convolutional neural network by introducing an attention mechanism;
training the improved Noise to Noise network by inputting micro-fossil CT images with different levels of Gaussian random Noise with zero mean as input data and tag data respectively, and monitoring the training process of the improved Noise to Noise network by the signal-to-Noise ratio of the denoised images to obtain denoising loss L noise ；
According to denoising loss L noise Performing the improved Noise to Noise network training and optimization to obtain a denoising network model;
and inputting the super-resolution image into a denoising network model to obtain a CT image with high signal-to-noise ratio.
7. The method of claim 1, wherein the denoising penaltyWherein y is t Represents the t-th noisy micro-body fossil CT data, θ represents network parameters, +.>Another noisy distribution of the t-th noisy micro-body CT data is represented, and N represents the total number of noisy micro-body CT data.
8. A deep learning-based micro-fossil CT image preprocessing device, the device comprising:
the acquisition module is used for acquiring CT images of the micro fossil;
the generation module is used for obtaining a super-resolution image of the CT image based on the CT image super-resolution reconstruction model; the CT image super-resolution reconstruction model is obtained by calculating perception loss, antagonism loss, total variation loss and content loss by using a high-definition-low-definition original CT image pair, wherein the perception loss is obtained based on a feature vector distance between a high-definition original CT image and a corresponding super-resolution image, the antagonism loss is obtained by identifying a reconstruction result of the high-definition original CT image and the super-resolution network model, the total variation loss is obtained based on a sharpness difference generated by image gradients of the high-definition original CT image and the corresponding super-resolution image due to region change, and the content loss is obtained based on a reconstruction result of the super-resolution network model and a pixel level distance between the high-definition original CT image;
and the denoising module is used for obtaining the CT image with high signal-to-noise ratio corresponding to the super-resolution image based on the denoising network model.
9. A computer device, the computer device comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the deep learning-based micro-fostering CT image preprocessing method according to any one of claims 1 to 7.
10. A computer readable storage medium, wherein computer program instructions are stored on the computer readable storage medium, which when executed by a processor, implement the deep learning based micro-fossilized CT image preprocessing method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310863239.0A CN117078780A (en) | 2023-07-14 | 2023-07-14 | Deep learning-based micro-fossil CT image preprocessing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310863239.0A CN117078780A (en) | 2023-07-14 | 2023-07-14 | Deep learning-based micro-fossil CT image preprocessing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117078780A true CN117078780A (en) | 2023-11-17 |
Family
ID=88708730
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310863239.0A Pending CN117078780A (en) | 2023-07-14 | 2023-07-14 | Deep learning-based micro-fossil CT image preprocessing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117078780A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118195903A (en) * | 2024-05-20 | 2024-06-14 | 四川新视创伟超高清科技有限公司 | Super-resolution image restoration method and restoration system based on deep learning |
-
2023
- 2023-07-14 CN CN202310863239.0A patent/CN117078780A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118195903A (en) * | 2024-05-20 | 2024-06-14 | 四川新视创伟超高清科技有限公司 | Super-resolution image restoration method and restoration system based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110443768B (en) | Single-frame image super-resolution reconstruction method based on multiple consistency constraints | |
| Trinh et al. | Novel example-based method for super-resolution and denoising of medical images | |
| CN112132959B (en) | Digital rock core image processing method and device, computer equipment and storage medium | |
| CN109214989B (en) | Single image super resolution ratio reconstruction method based on Orientation Features prediction priori | |
| CN111583285A (en) | Liver image semantic segmentation method based on edge attention strategy | |
| Min et al. | Blind deblurring via a novel recursive deep CNN improved by wavelet transform | |
| CN111583262A (en) | Blood vessel segmentation method and system | |
| CN113538616A (en) | Magnetic resonance image reconstruction method combining PUGAN and improved U-net | |
| Mansour et al. | Image-to-image MLP-mixer for image reconstruction | |
| CN116739899A (en) | Image super-resolution reconstruction method based on SAUGAN network | |
| CN115018728A (en) | Image fusion method and system based on multi-scale transformation and convolution sparse representation | |
| CN117274059A (en) | Low-resolution image reconstruction method and system based on image coding-decoding | |
| CN114913262B (en) | Nuclear magnetic resonance imaging method and system with combined optimization of sampling mode and reconstruction algorithm | |
| Rashid et al. | Single MR image super-resolution using generative adversarial network | |
| CN117078780A (en) | Deep learning-based micro-fossil CT image preprocessing method and device | |
| CN116258632A (en) | Text image super-resolution reconstruction method based on text assistance | |
| CN111986102B (en) | Digital pathological image deblurring method | |
| CN111311531A (en) | Image enhancement method and device, console equipment and medical imaging system | |
| Won et al. | Low-dose CT denoising using octave convolution with high and low frequency bands | |
| CN116778016A (en) | MRI image reconstruction method, system and medium | |
| WO2023164300A1 (en) | Machine learning enabled restoration of low resolution images | |
| Zhang et al. | Deep residual network based medical image reconstruction | |
| Song et al. | Super resolution reconstruction of medical image based on adaptive quad-tree decomposition | |
| Zhu et al. | Denoising of Magnetic Resonance Images with Deep Neural Regularizer Driven by Image Prior | |
| CN114862739B (en) | Intelligent medical image enhancement method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |