CN114912588A - Nerve fiber bundle reconstruction method and device based on deep learning - Google Patents
Nerve fiber bundle reconstruction method and device based on deep learning Download PDFInfo
- Publication number
- CN114912588A CN114912588A CN202210669939.1A CN202210669939A CN114912588A CN 114912588 A CN114912588 A CN 114912588A CN 202210669939 A CN202210669939 A CN 202210669939A CN 114912588 A CN114912588 A CN 114912588A
- Authority
- CN
- China
- Prior art keywords
- image
- fiber bundle
- map
- nerve fiber
- scanning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000004126 nerve fiber Anatomy 0.000 title claims abstract description 65
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013135 deep learning Methods 0.000 title claims abstract description 34
- 238000013528 artificial neural network Methods 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 14
- 230000006870 function Effects 0.000 claims description 49
- 230000004913 activation Effects 0.000 claims description 33
- 238000009792 diffusion process Methods 0.000 claims description 17
- 238000005070 sampling Methods 0.000 claims description 17
- 238000003860 storage Methods 0.000 claims description 16
- 210000002569 neuron Anatomy 0.000 claims description 15
- 238000003384 imaging method Methods 0.000 claims description 14
- 210000004556 brain Anatomy 0.000 claims description 11
- 239000011159 matrix material Substances 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 9
- 230000020411 cell activation Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 239000000835 fiber Substances 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000007547 defect Effects 0.000 abstract description 5
- 238000004422 calculation algorithm Methods 0.000 description 12
- 210000004885 white matter Anatomy 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 208000030173 low grade glioma Diseases 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000005304 joining Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000036982 action potential Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000007177 brain activity Effects 0.000 description 1
- 210000005013 brain tissue Anatomy 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 210000004884 grey matter Anatomy 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a nerve fiber bundle reconstruction method and a nerve fiber bundle reconstruction device based on deep learning, wherein the head or the designated position of the head is scanned by utilizing magnetic resonance according to scanning parameters to obtain MRI image data of a target area; performing DTI processing on the MRI image data to calculate a FAGS map or an MDG map; evaluating a first degree of difference between an FA map and an FAG map of the original DTI image or a second degree of difference between an MD map and an MDG map by using an MLP regression model of the DTI signal; if the first difference degree or the second difference degree is smaller than a preset threshold value, inputting the original DTI image into a deep learning neural network to obtain a reconstructed FA image and a reconstructed MD image; and reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image. The invention can overcome the defects of the prior fiber bundle reconstruction technology, accurately construct crossed and forked nerve fiber bundles, improve the accuracy and stability of reconstruction and shorten the reconstruction time on the premise of ensuring the reconstruction quality.
Description
Technical Field
The present application relates to the field of image reconstruction technologies, and in particular, to a method and an apparatus for reconstructing a nerve fiber bundle based on deep learning, a computer device, and a storage medium.
Background
The complex nervous activity of human brain is realized by the interconnection of different nerve centers, and the basis of the material forming the interconnection is white matter formed by the aggregation of countless nerve fibers, and the main function of the white matter is to communicate the gray matter (neurons) of different brain areas and transmit action potentials among the neurons.
High-order brain activity involves complex interactions between different brain regions, and therefore, analysis and processing of anatomical and functional properties of the fascicles of the white matter, especially nerve fiber tract reconstruction, are often required in medical imaging studies, and are the basis and important link for performing the analysis. However, there are a lot of intersections, convergence and branches in the nerve fiber bundles of the brain of a human body, so that it is important to select an appropriate nerve fiber bundle reconstruction method.
Currently commonly used neuro-fascicle reconstruction Algorithms include Deterministic neuro-fascicle imaging Algorithms (Deterministic White Matter Tractography Algorithms), Probabilistic neuro-fascicle imaging Algorithms (Probabilistic White Matter Tractography Algorithms) and Global Optimization Algorithms (Global Optimization Algorithms). The deterministic nerve fiber bundle imaging algorithm mainly comprises the steps of seed point selection, nerve fiber bundle track tracking, nerve fiber bundle selection strategy and the like, has the advantages that the main nerve fiber bundle can be reconstructed, the stable and accurate characteristic direction of the nerve fiber bundle can be obtained, the user reproducibility is high, the reconstruction effect is easily influenced by factors such as tracking direction, ROI selection, termination conditions and the like, and the defects of time consumption, artifacts and the like exist at the same time; the probabilistic nerve fiber bundle imaging algorithm takes probability distribution of the possibility of interconnection between two or more specific regions as an eyepoint, determines the main direction tracked by the nerve fiber bundle by using a probability distribution function, quantifies the correlation between voxels, and estimates the maximum possible nerve fiber bundle traveling direction of each voxel by using a tensor model. The method has the advantages that the tracking precision can be improved, smaller nerve fiber bundles can be displayed, and the fiber bundle branching calculation can be carried out, but the algorithm has the defects of larger calculation amount, long consumed time and limited clinical application; global optimization algorithms reconstruct the most likely path in brain space, mainly by optimizing global parameters calculated at the path level, making the path configuration consistent with the underlying diffusion data and satisfying specified constraints, have the advantage of being less affected by local irregularities in the diffusion data, and have the disadvantage that they are typically computationally intensive.
Therefore, it is desirable to provide a method for reconstructing a nerve fiber bundle based on deep learning, which overcomes the above disadvantages and shortcomings of the nerve fiber bundle reconstruction technique, and thus reconstructs a high-quality nerve fiber bundle quickly and efficiently.
Disclosure of Invention
The embodiment of the invention provides a nerve fiber bundle reconstruction method and device based on deep learning, computer equipment and a storage medium, which are used for solving the problems that the reconstruction effect in the related nerve fiber bundle reconstruction technology is easily influenced by various factors, the calculated amount is large, the consumed time is long, and the method is only suitable for intensive calculation.
In order to achieve the above object, in a first aspect of embodiments of the present invention, there is provided a deep learning-based nerve fiber bundle reconstruction method, including:
according to scanning parameters, scanning the head or the head designated position by using magnetic resonance to obtain MRI image data of a target region, wherein the scanning parameters comprise in-plane spatial resolution, matrix size, scanning layer number, imaging thickness, repetition time and echo time;
performing DTI processing on the MRI image data to calculate a FAGS map or an MDG map;
evaluating a first degree of difference between an FA map and an FAG map of the original DTI image or a second degree of difference between an MD map and an MDG map by using an MLP regression model of the DTI signal;
if the first difference degree or the second difference degree is smaller than a preset threshold value, inputting the original DTI image into a deep learning neural network to obtain a reconstructed FA image and a reconstructed MD image;
and reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image.
Optionally, in a possible implementation manner of the first aspect, the deep learning neural network includes:
adopting a UD-Net model as a prediction network, wherein an encoder layer and a decoder layer in the UD-Net model are transmitted through jump connection and are used for connecting feature mapping from a down-sampling layer to a corresponding layer of an up-sampling layer and fusing a low-resolution image and a high-resolution image;
the UD-Net model comprises four down-sampling and four up-sampling, wherein each down-sampling comprises two 3 x 3 convolution operators, an exponential linear unit activation function and a 2 x 2 maximum pool operator; each upsampling comprises a 2 x 2 convolution operator and two 3 x 3 convolution operators having an exponential linear unit activation function;
the output layer contains the FA value represented by the sigmoid activation function and the MD value represented by the corrected linear cell activation function.
Optionally, in a possible implementation manner of the first aspect, the method further includes:
the MLP regression model comprises a multilayer sensing layer and two hidden layers, wherein the multilayer sensing layer and the two hidden layers respectively comprise 150 neurons and 50 neurons, and a linear rectification function is used as an activation function, and the expression is as follows:
a＝g(x)＝max(0,z)；
the output layer of the neural network contains a hyperbolic tangent activation function, and the expression is as follows:
optionally, in a possible implementation manner of the first aspect, the method further includes:
training was performed using an Adam optimizer with a learning rate, where the initial learning rate was 0.001, which varies with the training process.
Optionally, in a possible implementation manner of the first aspect, the method further includes:
performing scanning training on the UD-Net model according to a first scanning condition, a second scanning condition and a third scanning condition respectively by using a whole brain slice, wherein the first scanning condition, the second scanning condition and the third scanning condition are different; the first scanning condition includes: 20 diffusion coding directions, b value of 0, an average value and scanning time range of 48-101 seconds; the second scanning condition includes: 20 diffusion coding directions, b value of 0, an average value and scanning time range of 16-34 seconds; the third scanning condition includes: 3 diffusion encoding directions, b value of 0, one average value and scan time range from 9-19 seconds.
In a second aspect of the embodiments of the present invention, there is provided a deep learning-based nerve fiber bundle reconstruction apparatus, including:
the scanning module is used for scanning the head or the head designated position by using magnetic resonance according to scanning parameters to obtain MRI image data of a target region, wherein the scanning parameters comprise in-plane spatial resolution, matrix size, scanning layer number, imaging thickness, repetition time and echo time;
the DTI processing module is used for carrying out DTI processing on the MRI image data so as to calculate a FAGS image or an MDG image;
the difference degree evaluation module is used for evaluating a first difference degree between an FA image and an FAG image of an original DTI image or a second difference degree between an MD image and an MDG image by using an MLP regression model of the DTI signal;
the reconstruction parameter map generation module is used for inputting the original DTI image into the deep learning neural network to obtain a reconstructed FA map and a reconstructed MD map if the first difference or the second difference is smaller than a preset threshold;
and the nerve fiber bundle reconstruction module is used for reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image.
Optionally, in a possible implementation manner of the second aspect, the reconstruction parameter map generating module includes:
adopting a UD-Net model as a prediction network, wherein an encoder layer and a decoder layer in the UD-Net model are transmitted through jump connection and are used for connecting feature mapping from a down-sampling layer to a corresponding layer of an up-sampling layer and fusing a low-resolution image and a high-resolution image;
the UD-Net model comprises four down-samples and four up-samples, wherein each down-sample comprises two 3 x 3 convolution operators, an exponential linear unit activation function and a 2 x 2 maximum pool operator; each upsampling comprises a 2 x 2 convolution operator and two 3 x 3 convolution operators having an exponential linear unit activation function;
the output layer contains the FA value represented by the sigmoid activation function and the MD value represented by the corrected linear cell activation function.
Optionally, in a possible implementation manner of the second aspect, the difference degree evaluation module includes:
the MLP regression model comprises a multilayer sensing layer and two hidden layers, wherein the multilayer sensing layer and the two hidden layers respectively comprise 150 neurons and 50 neurons, and a linear rectification function is used as an activation function, and the expression is as follows:
a＝g(x)＝max(0,z)
the output layer of the neural network contains a hyperbolic tangent activation function, and the expression is as follows:
in a third aspect of the embodiments of the present invention, a computer device is provided, which includes a memory and a processor, where the memory stores a computer program that is executable on the processor, and the processor implements the steps in the above method embodiments when executing the computer program.
A fourth aspect of the embodiments of the present invention provides a readable storage medium, in which a computer program is stored, which, when being executed by a processor, is adapted to carry out the steps of the method according to the first aspect of the present invention and various possible designs of the first aspect of the present invention.
According to the nerve fiber bundle reconstruction method, the nerve fiber bundle reconstruction device, the computer equipment and the storage medium based on the deep learning, the head or the head designated position is scanned by utilizing the magnetic resonance according to the scanning parameters to obtain the MRI image data of a target area, wherein the scanning parameters comprise in-plane spatial resolution, matrix size, scanning layer number, imaging thickness, repetition time and echo time; performing DTI processing on the MRI image data to calculate a FAGS map or an MDG map; evaluating a first degree of difference between an FA map and an FAG map of the original DTI image or a second degree of difference between an MD map and an MDG map by using an MLP regression model of the DTI signal; if the first difference degree or the second difference degree is smaller than a preset threshold value, inputting the original DTI image into a deep learning neural network to obtain a reconstructed FA image and a reconstructed MD image; and reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image. The invention can overcome the defects of the prior fiber bundle reconstruction technology, accurately construct crossed and forked nerve fiber bundles, improve the accuracy and stability of reconstruction and shorten the reconstruction time on the premise of ensuring the reconstruction quality.
Drawings
Fig. 1 is a flowchart of a deep learning-based nerve fiber bundle reconstruction method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the UD-net architecture model;
FIG. 3 is a schematic diagram of FA map generation from raw DTI images using UD-net neural network;
FIG. 4 is a FA and MD image reconstructed by a UD-Net neural network;
fig. 5 is a structural diagram of a deep learning-based nerve fiber bundle reconstruction device according to an embodiment of the present invention;
FIG. 6 is a graph comparing UD-Net and standard fiber bundle reconstitution for low grade gliomas;
FIG. 7 is a graph comparing 6-and 12-directional UD-Net with standard fiber bundle reconstruction for the same low-grade glioma patient.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein.
It should be understood that, in various embodiments of the present invention, the sequence numbers of the processes do not mean the execution sequence, and the execution sequence of the processes should be determined by the functions and the internal logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
It should be understood that in the present application, "comprising" and "having" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that, in the present invention, "a plurality" means two or more. "and/or" is merely an association relationship describing an associated object, meaning that there may be three relationships, for example, and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "comprises A, B and C" and "comprises A, B, C" means that all three of A, B, C comprise, "comprises A, B or C" means that one of A, B, C comprises, "comprises A, B and/or C" means that any 1 or any 2 or 3 of A, B, C comprises.
It should be understood that in the present invention, "B corresponding to a", "a corresponds to B", or "B corresponds to a" means that B is associated with a, and B can be determined from a. According to A
Determining B does not mean determining B based only on a, but may also be based on a and/or other information. And the matching of A and B means that the similarity of A and B is greater than or equal to a preset threshold value.
As used herein, "if" may be interpreted as "at … …" or "when … …" or "in response to a determination" or "in response to a detection", depending on the context.
The technical solution of the present invention will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
The invention provides a nerve fiber bundle reconstruction method based on deep learning, which is a flow chart shown in figure 1 and comprises the following steps:
and step S110, scanning the head or the head designated position by using magnetic resonance according to the scanning parameters to obtain MRI image data of the target area.
In the present step, imaging is performed on 1.5T (N ═ 4) or 3T (N ═ 52) MRI, and DTI imaging with high resolution can be performed with a field strength of 3T, where the scanning content includes 1.7-1.9mm in-plane spatial resolution, 128 × 128 matrix size, the number of scanning layers is 15-33, the imaging layer thickness is 4-5mm, the echo time TE is 67-104ms, and the repetition time TR is 3300-. These scans use a twice refocused eddy current-zero diffusion encoding scheme. 5 images were masked using the Robust blue Extraction (ROBEX) tool to exclude non-brain tissue.
Step S120, performing DTI processing on the MRI image data to calculate a FAGS map or an MDG map.
In step S120, each DTI scan needs to be reconstructed to compute the standard diffusion tensor (i.e., pixel averaging is repeated for all signals, and then conventional LLS fitting is performed using all 20 encoding directions obtained), and the standard diffusion tensor is decomposed into eigenvalues (λ 1, λ 2, λ 3) and eigenvectors (E1, E2, E3). The decomposed characteristic values are used for calculating standard FAGS and MDG (mean diffusivity) graphs; the brain MRI mapping was then used with both Keras and tensrflow neural networks to generate the corresponding FA and MD values.
The calculation formula is as follows:
MDG＝(λ 1 +λ 2 +λ 3 )/3
specifically, the canonical diffusion tensor is a mathematical structure D, having a three-dimensional space, with anisotropy having 3 × 3 secondary components, of which 3 components are identical (symmetry), while the remaining 6 factors (Dxx, Dyy, Dzz, Dxy, Dxz, Dyz) determine the features of the tensor, and the 3 non-0 factors reflect the shape of the mathematical structure along the principal diagonal of the tensor, called eigenvalues λ 1, λ 2, λ 3, with magnitude independent of direction, and with mathematical relationships representing 3 eigenvectors (E1, E2, E3).
The brain slices were then used to retrospectively sample the cranial nuclear magnetic scan under the following three scan conditions to simulate accelerated acquisition: wherein; the first scanning condition includes: 20 diffusion coding directions, b value of 0, an average value and scanning time range of 48-101 seconds; the second scanning condition includes: 20 diffusion coding directions, b value of 0, an average value and scanning time range of 16-34 seconds; the third scanning condition includes: 3 diffusion encoding directions, b value of 0, one average value and scan time range from 9-19 seconds.
Step S130, a first difference degree between an FA map and an FAG map of the original DTI image or a second difference degree between an MD map and an MDG map is evaluated by using an MLP regression model of the DTI signal.
In this step, the input layer of the MLP network is a single raw DTI image, each signal being normalized to a b-0 signal at that voxel. The output layer is the FA or MD value corresponding to the DTI signal. All fiber bundles in 46 DTI images were scanned and the FA and MD models were trained, 3.5X 106 DTI images. The MLP regression model comprises a multilayer sensing layer and two hidden layers, wherein the multilayer sensing layer and the two hidden layers respectively comprise 150 neurons and 50 neurons, and a linear rectification function is taken as an activation function, and the expression is as follows:
a＝g(x)＝max(0,z)
the output layer of the neural network contains a hyperbolic tangent activation function, and the expression is as follows:
the neural network is also provided with a loss function of the boundary weight, and the expression is as follows:
therefore, the neural network can perform iterative special optimization on the neuron number of each layer so as to improve the accuracy of the model.
Training with an Adam optimizer using a learning rate, wherein the initial learning rate is 0.001, the learning rate varies with the training process, and is formulated as:
m t ＝μ*m t-1 +(1-μ)*gt
MLP uses a functional regression model with mean and variance for evaluating the degree of difference between FA and reference FAG, or between MD and MDG values.
And S140, if the first difference degree or the second difference degree is smaller than a preset threshold value, inputting the original DTI image into a deep learning neural network to obtain a reconstructed FA image and a reconstructed MD image.
In step S140, when the first difference or the second difference is smaller than the preset threshold, the mri data of the patient is acquired, and the data is acquired by using a similar imaging protocol, and the data is input into the trained network model to obtain the final prediction result of the patient.
In particular, the deep learning neural network refers to a 2D UD-net architecture model. Adopting a UD-Net model as a prediction network, wherein an encoder layer and a decoder layer in the UD-Net model are transmitted through jump connection and are used for connecting feature mapping from a down-sampling layer to a corresponding layer of an up-sampling layer and fusing a low-resolution image and a high-resolution image; the UD-Net model comprises four down-samples and four up-samples, wherein each down-sample comprises two 3 x 3 convolution operators, an exponential linear unit activation function and a 2 x 2 maximum pool operator; each upsampling comprises a 2 x 2 convolution operator and two 3 x 3 convolution operators having an exponential linear unit activation function; the output layer contains the FA value represented by the sigmoid activation function and the MD value represented by the corrected linear cell activation function.
More specifically, the input layer of UD-net is a set of DTI pictures at a single slice position, with a single diffusion weighted picture (and b ═ 0 reference scan) occupying a different picture channel; the model output layer is the predicted FA or MD graph. The UD-net architecture model is shown in FIG. 2. The network includes a down-sampling path and an up-sampling path, each path having four convolutional layers. Each step on the downsampled path, which includes two 3 x 3 convolution operators, an exponential linear unit activation function, and a 2 x 2 max pool operator, doubles the number of channels computed and halves the x-y matrix size (i.e., 64 x 64, 32 x 32, 16 x 16, 8 x 8). Each step on the upsampling path includes a 2 x 2 convolution which halves the number of features, doubles the x-y matrix size, and two 3 x 3 convolution with exponential linear unit activation functions, including skip-joining, by joining the layers in the downsampling and upsampling paths. A skip connection is a key component in the UD-Net architecture that connects feature maps from a downsampling layer to a corresponding layer in an upsampling path, fusing the low resolution image with the high resolution image, as shown in fig. 3. Each step in the upsampling and downsampling paths contains missing layers to avoid overfitting. The output layer contains the FA value represented by the sigmoid activation function and the MD value represented by the corrected linear cell activation function. Before UD-Net is input, each DTI image scan intensity is increased to the maximum value of brain signal intensity, which results in the intensity of all images (across the b-value) being normalized from 0 to 1. Although all DTI data used in this application are acquired with a uniform 128 x 128 matrix size, any image acquired with a different size needs to be resized before entering UD-Net. Using whole brain slices, 3, 6 and 20 directions of scan training were performed on UD-Net models of FA and MD, which together contain 994 algorithms of different thicknesses, to reconstruct high resolution FA and MD images, three-dimensional nerve fiber bundles, as shown in FIG. 4.
And S150, reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image.
In this step, the obtained reconstructed FA map and the reconstructed MD map are described as the direction and distribution of the nerve fibers by continuous curve fitting, and a three-dimensional nerve fiber bundle is constructed.
According to the nerve fiber bundle reconstruction method based on deep learning, the head or the designated position of the head is scanned by using magnetic resonance according to scanning parameters, so that MRI image data of a target area are obtained; performing DTI processing on the MRI image data to calculate a FAGS map or an MDG map; evaluating a first degree of difference between an FA map and an FAG map of the original DTI image or a second degree of difference between an MD map and an MDG map by using an MLP regression model of the DTI signal; if the first difference degree or the second difference degree is smaller than a preset threshold value, inputting the original DTI image into a deep learning neural network to obtain a reconstructed FA image and a reconstructed MD image; and reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image. The invention overcomes the defects of three fiber bundle reconstruction technologies in the background technology, accurately constructs crossed and forked nerve fiber bundles, improves the accuracy and stability of reconstruction, shortens the reconstruction time and promotes the wide clinical application of nerve fiber bundle reconstruction on the premise of ensuring the reconstruction quality.
In addition, the present application can improve the reconstruction accuracy of the nerve fiber bundle (0.005 when Ndir is 20, and 0.01 when Ndir is 3). The FA image can be extracted from three diffusion coding directions only, and then the nerve fiber bundle is reconstructed, so that the efficiency is improved, and the reconstruction time is shortened. The noise-induced deviation is reduced, and the quality and stability of nerve fiber bundle reconstruction are improved.
An embodiment of the present invention further provides a nerve fiber bundle reconstruction device based on deep learning, as shown in fig. 5, including:
the scanning module is used for scanning the head or the head designated position by using magnetic resonance according to scanning parameters to obtain MRI image data of a target region, wherein the scanning parameters comprise in-plane spatial resolution, matrix size, scanning layer number, imaging thickness, repetition time and echo time;
the DTI processing module is used for carrying out DTI processing on the MRI image data so as to calculate a FAGS image or an MDG image;
the difference degree evaluation module is used for evaluating a first difference degree between an FA image and an FAG image of an original DTI image or a second difference degree between an MD image and an MDG image by using an MLP regression model of the DTI signal;
the reconstruction parameter map generation module is used for inputting the original DTI image into the deep learning neural network to obtain a reconstructed FA map and a reconstructed MD map if the first difference or the second difference is smaller than a preset threshold;
and the nerve fiber bundle reconstruction module is used for reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image.
In one embodiment, the reconstruction parameter map generating module includes:
adopting a UD-Net model as a prediction network, wherein an encoder layer and a decoder layer in the UD-Net model are transmitted through a jump connection and used for connecting feature mapping from a down-sampling layer to a corresponding layer of an up-sampling layer and fusing a low-resolution image with a high-resolution image;
the UD-Net model comprises four down-samples and four up-samples, wherein each down-sample comprises two 3 x 3 convolution operators, an exponential linear unit activation function and a 2 x 2 maximum pool operator; each upsampling comprises a 2 x 2 convolution operator and two 3 x 3 convolution operators having an exponential linear unit activation function;
the output layer contains the FA value represented by the sigmoid activation function and the MD value represented by the corrected linear cell activation function.
In one embodiment, the difference degree evaluation module includes:
the MLP regression model comprises a multilayer sensing layer and two hidden layers, wherein the multilayer sensing layer and the two hidden layers respectively comprise 150 neurons and 50 neurons, and a linear rectification function is used as an activation function, and the expression is as follows:
a＝g(x)＝max(0,z)
the output layer of the neural network contains a hyperbolic tangent activation function, and the expression is as follows:
the invention carries out nerve fiber bundle reconstruction aiming at low-grade glioma and carries out comparison research aiming at UD-Net and gold standard, as shown in figures 6 and 7, the UD-Net model can accurately construct crossed and forked nerve fiber bundle images, improve the accuracy and stability of reconstruction, shorten the reconstruction time and promote the wide clinical application of nerve fiber bundle reconstruction on the premise of ensuring the reconstruction quality.
The readable storage medium may be a computer storage medium or a communication medium. Communication media includes any medium that facilitates transfer of a computer program from one place to another. Computer storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, a readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may reside in an Application Specific Integrated Circuits (ASIC). Additionally, the ASIC may reside in user equipment. Of course, the processor and the readable storage medium may also reside as discrete components in a communication device. The readable storage medium may be a read-only memory (ROM), a random-access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The present invention also provides a program product comprising execution instructions stored in a readable storage medium. The at least one processor of the device may read the execution instructions from the readable storage medium, and the execution of the execution instructions by the at least one processor causes the device to implement the methods provided by the various embodiments described above.
In the above embodiments of the terminal or the server, it should be understood that the Processor may be a Central Processing Unit (CPU), other general-purpose processors, a Digital Signal Processor (DSP), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A nerve fiber bundle reconstruction method based on deep learning is characterized by comprising the following steps:
according to scanning parameters, scanning the head or the head designated position by using magnetic resonance to obtain MRI image data of a target region, wherein the scanning parameters comprise in-plane spatial resolution, matrix size, scanning layer number, imaging thickness, repetition time and echo time;
performing DTI processing on the MRI image data to calculate a FAGS map or an MDG map;
evaluating a first degree of difference between an FA map and an FAG map of the original DTI image or a second degree of difference between an MD map and an MDG map by using an MLP regression model of the DTI signal;
if the first difference degree or the second difference degree is smaller than a preset threshold value, inputting the original DTI image into a deep learning neural network to obtain a reconstructed FA image and a reconstructed MD image;
and reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image.
2. The deep learning based nerve fiber bundle reconstruction method according to claim 1, wherein the deep learning neural network comprises:
adopting a UD-Net model as a prediction network, wherein an encoder layer and a decoder layer in the UD-Net model are transmitted through jump connection and are used for connecting feature mapping from a down-sampling layer to a corresponding layer of an up-sampling layer and fusing a low-resolution image and a high-resolution image;
the UD-Net model comprises four down-samples and four up-samples, wherein each down-sample comprises two 3 x 3 convolution operators, an exponential linear unit activation function and a 2 x 2 maximum pool operator; each upsampling comprises a 2 x 2 convolution operator and two 3 x 3 convolution operators having an exponential linear unit activation function;
the output layer contains the FA value represented by the sigmoid activation function and the MD value represented by the corrected linear cell activation function.
3. The deep learning based nerve fiber bundle reconstruction method of claim 1, further comprising:
the MLP regression model comprises a multilayer sensing layer and two hidden layers, wherein the multilayer sensing layer and the two hidden layers respectively comprise 150 neurons and 50 neurons, and a linear rectification function is used as an activation function, and the expression is as follows:
a＝g(x)＝max(0,z)
the output layer of the neural network contains a hyperbolic tangent activation function, and the expression is as follows:
4. the deep learning based nerve fiber bundle reconstruction method of claim 3, further comprising:
training was performed using an Adam optimizer with a learning rate, where the initial learning rate was 0.001, which varies with the training process.
5. The deep learning based nerve fiber bundle reconstruction method of claim 2, further comprising:
performing scanning training on the UD-Net model according to a first scanning condition, a second scanning condition and a third scanning condition respectively by using a whole brain slice, wherein the first scanning condition, the second scanning condition and the third scanning condition are different; the first scanning condition includes: 20 diffusion coding directions, b value is 0, and an average value and scanning time range is 48-101 seconds; the second scanning condition includes: 20 diffusion coding directions, b value of 0, an average value and scanning time range of 16-34 seconds; the third scanning condition includes: 3 diffusion coding directions, b value of 0, one mean value and scan time range from 9 to 19 seconds.
6. A nerve fiber bundle reconstruction device based on deep learning, comprising:
the scanning module is used for scanning the head or the head designated position by using magnetic resonance according to scanning parameters to obtain MRI image data of a target region, wherein the scanning parameters comprise in-plane spatial resolution, matrix size, scanning layer number, imaging thickness, repetition time and echo time;
the DTI processing module is used for carrying out DTI processing on the MRI image data so as to calculate a FAGS image or an MDG image;
the difference degree evaluation module is used for evaluating a first difference degree between an FA image and an FAG image of an original DTI image or a second difference degree between an MD image and an MDG image by using an MLP regression model of the DTI signal;
the reconstruction parameter map generation module is used for inputting the original DTI image into the deep learning neural network to obtain a reconstructed FA map and a reconstructed MD map if the first difference or the second difference is smaller than a preset threshold;
and the nerve fiber bundle reconstruction module is used for reconstructing the nerve fiber bundle according to the reconstructed FA image and the reconstructed MD image.
7. The deep learning based nerve fiber bundle reconstruction device according to claim 6, wherein the reconstruction parameter map generation module comprises:
adopting a UD-Net model as a prediction network, wherein an encoder layer and a decoder layer in the UD-Net model are transmitted through jump connection and are used for connecting feature mapping from a down-sampling layer to a corresponding layer of an up-sampling layer and fusing a low-resolution image and a high-resolution image;
the UD-Net model comprises four down-samples and four up-samples, wherein each down-sample comprises two 3 x 3 convolution operators, an exponential linear unit activation function and a 2 x 2 maximum pool operator; each up-sample includes a 2 x 2 convolution operator and two 3 x 3 convolution operators with exponential linear unit activation functions;
the output layer contains the FA value represented by the sigmoid activation function and the MD value represented by the corrected linear cell activation function.
8. The deep learning based nerve fiber bundle reconstruction device according to claim 6, wherein the difference degree evaluation module comprises:
the MLP regression model comprises a multilayer sensing layer and two hidden layers, wherein the multilayer sensing layer and the two hidden layers respectively comprise 150 neurons and 50 neurons, and a linear rectification function is used as an activation function, and the expression is as follows:
a＝g(x)＝max(0,z)
the output layer of the neural network contains a hyperbolic tangent activation function, and the expression is as follows:
9. a computer device comprising a memory and a processor, the memory storing a computer program operable on the processor, wherein the processor implements the steps of the method of any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210669939.1A CN114912588A (en) | 2022-06-14 | 2022-06-14 | Nerve fiber bundle reconstruction method and device based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210669939.1A CN114912588A (en) | 2022-06-14 | 2022-06-14 | Nerve fiber bundle reconstruction method and device based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114912588A true CN114912588A (en) | 2022-08-16 |
Family
ID=82770781
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210669939.1A Pending CN114912588A (en) | 2022-06-14 | 2022-06-14 | Nerve fiber bundle reconstruction method and device based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114912588A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111028306A (en) * | 2019-11-06 | 2020-04-17 | 杭州电子科技大学 | AR2U-Net neural network-based rapid magnetic resonance imaging method |
| CN112085810A (en) * | 2020-09-21 | 2020-12-15 | 上海交通大学 | Brain tissue free water imaging reconstruction method and system, storage medium and terminal |
| CN113256536A (en) * | 2021-06-18 | 2021-08-13 | 之江实验室 | Ultrahigh-dimensional data reconstruction deep learning method based on wavelet analysis |
-
2022
- 2022-06-14 CN CN202210669939.1A patent/CN114912588A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111028306A (en) * | 2019-11-06 | 2020-04-17 | 杭州电子科技大学 | AR2U-Net neural network-based rapid magnetic resonance imaging method |
| CN112085810A (en) * | 2020-09-21 | 2020-12-15 | 上海交通大学 | Brain tissue free water imaging reconstruction method and system, storage medium and terminal |
| CN113256536A (en) * | 2021-06-18 | 2021-08-13 | 之江实验室 | Ultrahigh-dimensional data reconstruction deep learning method based on wavelet analysis |
Non-Patent Citations (4)
| Title |
|---|
| AURÉLIEN OLIVIER ET AL.: "Active learning strategy and hybrid training for infarct segmentation on diffusion MRI with a U-shaped network", 《JOURNAL OF MEDICAL IMAGING》, vol. 6, no. 4, 4 October 2019 (2019-10-04), pages 1 - 11 * |
| ERIC ALIOTTA ET AL.: "Extracting diffusion tensor fractional anisotropy and mean diffusivity from 3‐direction DWI scans using deep learning", 《MAGNETIC RESONANCE IN MEDICINE》, 18 August 2020 (2020-08-18), pages 1 - 10 * |
| 刘宇等: "《智能搜索和推荐系统 原理、算法与应用》", 31 January 2021, 机械工业出版社, pages: 116 - 117 * |
| 岳春林: "《太极拳延缓认知衰退研究》", 30 November 2020, 上海交通大学出版社, pages: 27 - 29 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109584254B (en) | Heart left ventricle segmentation method based on deep full convolution neural network | |
| García-Lorenzo et al. | Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging | |
| CN110689543A (en) | Improved convolutional neural network brain tumor image segmentation method based on attention mechanism | |
| CN114926477B (en) | Brain tumor multi-mode MRI image segmentation method based on deep learning | |
| WO2022121100A1 (en) | Darts network-based multi-modal medical image fusion method | |
| KR20230059799A (en) | A Connected Machine Learning Model Using Collaborative Training for Lesion Detection | |
| Vasilic et al. | A novel local thresholding algorithm for trabecular bone volume fraction mapping in the limited spatial resolution regime of in vivo MRI | |
| CN115496771A (en) | Brain tumor segmentation method based on brain three-dimensional MRI image design | |
| JP2004535874A (en) | Magnetic resonance angiography and apparatus therefor | |
| US9402562B2 (en) | Systems and methods for improved tractographic processing | |
| CN114119637B (en) | Brain white matter high signal segmentation method based on multiscale fusion and split attention | |
| CN109886944A (en) | A kind of white matter high signal intensity detection and localization method based on multichannel chromatogram | |
| Zou et al. | Multi-scale deformable transformer for multi-contrast knee MRI super-resolution | |
| Vakli et al. | Predicting body mass index from structural MRI brain images using a deep convolutional neural network | |
| CN109949288A (en) | Tumor type determines system, method and storage medium | |
| CN110853048A (en) | MRI image segmentation method, device and storage medium based on rough training and fine training | |
| CN117333750A (en) | Spatial registration and local global multi-scale multi-modal medical image fusion method | |
| Feng et al. | Automatic localization and segmentation of focal cortical dysplasia in FLAIR‐negative patients using a convolutional neural network | |
| Lim et al. | Motion artifact correction in fetal MRI based on a Generative Adversarial network method | |
| CN115830016A (en) | Medical image registration model training method and equipment | |
| CN115861464A (en) | Pseudo CT (computed tomography) synthesis method based on multimode MRI (magnetic resonance imaging) synchronous generation | |
| CN115018863A (en) | Image segmentation method and device based on deep learning | |
| CN114581459A (en) | Improved 3D U-Net model-based segmentation method for image region of interest of preschool child lung | |
| CN113160256A (en) | MR image placenta segmentation method for multitask generation confrontation model | |
| CN115272386A (en) | Multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatic generation label |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |