[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN110956617A - Brain nuclear magnetic resonance abnormal image visualization method based on circulation attention model - Google Patents

Brain nuclear magnetic resonance abnormal image visualization method based on circulation attention model Download PDF

Info

Publication number
CN110956617A
CN110956617A CN201911129218.6A CN201911129218A CN110956617A CN 110956617 A CN110956617 A CN 110956617A CN 201911129218 A CN201911129218 A CN 201911129218A CN 110956617 A CN110956617 A CN 110956617A
Authority
CN
China
Prior art keywords
network
glimpse
magnetic resonance
model
core network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911129218.6A
Other languages
Chinese (zh)
Inventor
柯丰恺
刘欢平
赵大兴
孙国栋
冯维
柳晨康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hubei University of Technology
Original Assignee
Hubei University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hubei University of Technology filed Critical Hubei University of Technology
Priority to CN201911129218.6A priority Critical patent/CN110956617A/en
Publication of CN110956617A publication Critical patent/CN110956617A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/54—Browsing; Visualisation therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a brain nuclear magnetic resonance abnormal image visualization method based on a circulation attention model, which comprises the steps of collecting a brain nuclear magnetic resonance abnormal image of a patient as a training sample, training the circulation attention model by using the training sample to obtain trained network parameters, and carrying out visualization detection on the brain nuclear magnetic resonance abnormal image by using a trained RAM model. The model is simple, the speed is fast, the precision is high, and the detection capability is stronger than that of the traditional CNN network.

Description

Brain nuclear magnetic resonance abnormal image visualization method based on circulation attention model
Technical Field
The invention belongs to the technical field of nuclear magnetic resonance abnormal image visualization, and particularly relates to a brain nuclear magnetic resonance abnormal image visualization method based on a circulation attention model.
Background
The most common model for deep learning is a network model based on CNN and RNN, and the deep learning is used for detecting defects of the magnetic resonance of the brain, generally, bottom layer features are extracted through operations such as convolution and pooling of input magnetic resonance images and nonlinear transformation is carried out, so that defects in the images are identified, and finally, the defects are classified and the like. However, as the Model input increases, the CNN and RNN training data linearly increase, and the training time is doubled, in order to solve this problem, a circular Attention Model (RAM) is proposed, which is actually an Attention mechanism simulating human vision to selectively process a certain region of interest of the Model based on the input of the RNN serialization, rather than processing the whole image, so that the computation time of the Model is greatly reduced, and the computation complexity is reduced.
The conventional Attention mechanism is generally divided into two forms, Soft Attention and Hard Attention. SoftAttention is mainly created by Saliency maps (salience Map) of visual signals. The salient image is obtained by processing the image features of the bottom layer, but the deep features such as the semantic features of the image cannot be considered, so that the effect of the constructed model on treating the magnetic resonance defects of the brain is not good, and although more high-level features can be extracted from the original image based on the CNN, the model is complex and the information amount of each processing is huge.
Disclosure of Invention
The invention aims to provide a rapid and high-precision brain nuclear magnetic resonance abnormal image visualization method based on a circulation attention model, aiming at the defects of the technology.
In order to achieve the purpose, the brain nuclear magnetic resonance abnormal image visualization method based on the circulation attention model comprises the following steps:
1) acquiring a brain nuclear magnetic resonance abnormal image of a patient as a training sample;
2) training the circulating attention model by using the training sample to obtain trained network parameters;
2.1) constructing a RAM model and randomly initializing network parameters
And constructing a RAM model, wherein the RAM model belongs to a new hard attention model algorithm based on a position and combining reinforcement learning and a recurrent neural network. The RAM model comprises five parts of a Glimpse network, a Core network, an Action network, a Location network and a Baseline network, and is randomly initialized, namely the parameters of the Glimpse network, the Core network, the Action network, the Location network and the Baseline network are initialized
Figure BDA0002277812300000021
Figure BDA0002277812300000022
l(ht|θl) And b (h)t|θb) Is a random initial value;
2.2) training the RAM model
2.2.1) random initialization of the first attention position l0；
2.2.2) according to the first attention position l0Obtaining Glimpse characteristics g0；
2.2.3) first hidden state h of the time-series Core network0Initialization is 0;
2.2.4) hidden state h of Core network0And features of the Glimpse network g0As Core network input, the output obtains a new hidden state h1；
2.2.5) hiding new Core network state h1As the input of the Action network, outputting the predicted classification result a1；
2.2.6) outputting the new hidden state h of the Core network1As the input of the Baseline network, the output is obtained as a one-dimensional vector b1Baseline networks reduce the variance to the threshold for gradientsA range of values ε;
2.2.7) outputting the new hidden state h of the Core network1As the input of the Location network, the Location network adopts a policy gradient algorithm to determine the attention position of the next time sequence, and the output attention position of the next time sequence is l1；
2.2.8) circulating the step 2.2.2) to the step 2.2.7), and repeating the step T times;
2.2.9) constructing a loss function for the network
Defining the overall loss function of the RAM training model as:
Figure BDA0002277812300000031
2.2.10) back-propagating the neural network according to the loss function, thereby updating the parameters of the network
According to the Loss function Loss (theta) in the step 2.2.9), training the Action network and reversely propagating the Action network to the Glimpse network and the Core network, wherein in the process of reversely propagating, the Location network and the Baseline network are trained by adopting a strategy gradient of reinforcement learning, although the input of the Location network and the Baseline network is the hidden state of the Core network, the input of the base network is not reversely propagated to the Core network and the Glimpse network, and the reinforcement learning weight value updating mode is as follows: thetat+1＝θt+αγtRt▽θlog(At|Stθ), θ is a parameter of the policy gradient, and a parameter θ of the Glimpse network, the Core network, the Location networkfg、θfh、θlThe parameters are all strategy parameters controlled by the parameter theta of the strategy gradient, so that the parameters of the whole network are updated;
2.2.11 repeating the training from step 2.2.1) to step 2.2.10) M times to obtain the final network parameters;
3) and carrying out visual detection on the brain nuclear magnetic resonance abnormal image by using the trained RAM model.
Further, in the step 2.2), the Glimpse network comprises a Glimpse sensor, and the Glimpse sensor samples the magnetic resonance image x of the brain of the patient to be processed and surrounds the magnetic resonance image xObtaining 4 square images with different lengths by taking the first attention position as the center of the image fixation area at the first attention position, and uniformly transforming the square images into a group of images with the size of 32 x 32 by using a nearest neighbor interpolation method, wherein the first attention position l of the image is0The middle region of (a) is a higher resolution image, and the larger regions outward from the middle region are progressively lower resolution images; the Glimpse sensor then acquires the set of images and the first attention location l0Carrying out feature extraction, and connecting through full-connection layers to obtain features g output by the Glimpse network0。
Further, in the step 2.2.4), the Core network is actually an RNN network, and the hidden state h output by the Core network in the previous time series is time-sequenced0And the feature g currently output via the Glimpse network0The two characteristics are combined to be used as the input of the Core network, and the output of the Core network obtains a new hidden state h in the RNN network1。
Further, in the step 2.2.5), the Action network outputs a new hidden state h output by the Core network1As input, the training target is the detection of the brain nuclear magnetic resonance image of the patient, the Action network is a classification network, and the output of the obtained prediction classification result is a1And further based on the predicted classification result a1And the actual label of the image obtains the reward function, wherein if the classification result a is correct, the reward function is 1, otherwise, the reward function is 0.
Further, in the step 2.2.9), the loss function is composed of three parts:
the first part is the last classification result aTAnd the cross entropy loss function formed by the actual category of the image, wherein the loss function formula is as follows:
Figure BDA0002277812300000041
wherein y isiFor the input magnetic resonance image true tag values,
Figure BDA0002277812300000042
predicting for a targetA tag value of (a);
the second part is a loss function of the Location network policy gradient algorithm, and the loss function formula is as follows:
Figure BDA0002277812300000043
wherein theta is a parameter of the strategy gradient, and parameters of the Glimpse network, the Core network and the Location network
Figure BDA0002277812300000044
θlAre all the strategy parameters controlled by the parameter theta of the strategy gradient,
Figure BDA0002277812300000045
the reward obtained for each sample in reinforcement learning,
Figure BDA0002277812300000046
the attenuation coefficient of the obtained reward is between 0 and 1,
Figure BDA0002277812300000047
the action behaviors sampled by the policy gradient algorithm,
Figure BDA0002277812300000048
the behavior states sampled by the policy gradient algorithm,
Figure BDA0002277812300000049
performing an action behavior
Figure BDA00022778123000000416
Accumulated awards obtained, biIs a reward benchmark value that depends on the behavior state
Figure BDA00022778123000000410
Independent of motion behavior
Figure BDA00022778123000000411
Figure BDA00022778123000000412
The representative is that the strategy gradient algorithm directly parameterizes the strategy, namely, the strategy is represented by a parameterized function to seek the optimal strategy;
the loss function of the third part is based on the classification result aTThe reward obtained whether the reward is correct and the baseline form a mean square error loss function, and the loss function formula is as follows:
Figure BDA00022778123000000413
using error to obtain training reference value biThe increase of the baseline network can reduce the probability of behavior actions below a baseline value and increase the probability of behavior actions above the baseline value, wherein,
Figure BDA00022778123000000417
the reward obtained for each sample in reinforcement learning if the classification result aTIf correct, the reward function is
Figure BDA00022778123000000414
Otherwise the reward function is
Figure BDA00022778123000000415
biA baseline value predicted for each sample; according to the loss functions of the first part, the second part and the third part, the loss function of the RAM training model is defined as follows:
Figure BDA0002277812300000051
compared with the prior art, the invention has the following advantages: the brain nuclear magnetic resonance abnormal image visualization method based on the circulation attention model is simple in model, high in speed and precision and stronger in detection capability than the traditional CNN network.
Drawings
FIG. 1 is a schematic view of a cyclic attention model in the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments.
A brain nuclear magnetic resonance abnormal image visualization method based on a circulation attention model comprises the following specific steps:
1) acquiring a brain nuclear magnetic resonance abnormal image of a patient as a training sample;
2) training a circulating attention model (RAM) by using a training sample to obtain trained network parameters;
2.1) constructing a RAM model and randomly initializing network parameters
And constructing an RAM model, wherein the RAM model belongs to a new hard attention model algorithm based on a position and combining reinforcement learning and a cycle neural network. As shown in fig. 1, the RAM model includes five parts, i.e., a Glimpse network, a Core network, an Action network, a Location network, and a base network, and is randomly initialized, i.e., the parameters of the Glimpse network, the Core network, the Action network, the Location network, and the base network are initialized
Figure BDA0002277812300000052
l(ht|θl) And b (h)t|θb) Is a random initial value;
2.2) training the RAM model
2.2.1) random initialization of the first attention position l0；
2.2.2) according to the first attention position l0Obtaining Glimpse characteristics g0；
The Glimpse network comprises a Glimpse sensor which samples a nuclear magnetic resonance image x of the brain of a patient to be processed around the attention position thereof to obtain 4 square images with different lengths taking the square images as the center of the image fixation area, and then uniformly transforms the square images into a group of images with the size of 32 x 32 by using a nearest neighbor interpolation method, wherein the first attention position l of the images0Is a higher resolution image, and larger areas from the middle area outwards are progressively lower resolution images, and then the Glimpse sensor is based on the set of images obtained and the first attention position/0The characteristic extraction is carried out, and the characteristic extraction is carried out,obtaining the characteristics g of Glimpse network output through full-connection layer connection0；
2.2.3) first hidden state h of the time-series Core network0Initialization is 0;
2.2.4) hidden state h of Core network0And features of the Glimpse network g0As Core network input, the output obtains a new hidden state h1
The Core network is actually an RNN network, and the hidden state h output by the Core network in the last time sequence is output in time sequence0And the feature g currently output via the Glimpse network0The two characteristics are combined to be used as the input of the Core network, and the output of the Core network obtains a new hidden state h in the RNN network1；
2.2.5) hiding new Core network state h1As the input of the Action network, outputting the predicted classification result a1
The Action network outputs a new hidden state h of the Core network1As input, the training target is the detection of the brain nuclear magnetic resonance image of the patient, the Action network is a classification network, and the output of the obtained prediction classification result is a1And further based on the predicted classification result a1Obtaining a reward function together with an actual label of the image, wherein if the classification result a is correct, the reward function is 1, otherwise, the reward function is 0;
2.2.6) outputting the new hidden state h of the Core network1As the input of the Baseline network, the output is obtained as a one-dimensional vector b1The Baseline network reduces the variance of the gradient to be within a threshold range epsilon;
2.2.7) outputting the new hidden state h of the Core network1As the input of the Location network, the Location network adopts a policy gradient algorithm to determine the attention position of the next time sequence, and the output attention position of the next time sequence is l1；
2.2.8) circulating the step 2.2.2) to the step 2.2.7), and repeating the step T times;
2.2.9) constructing a loss function for the network
The loss function consists of three parts:
the first part is the last classification result aTAnd the cross entropy loss function formed by the actual category of the image, wherein the loss function formula is as follows:
Figure BDA0002277812300000071
wherein y isiFor the input magnetic resonance image true tag values,
Figure BDA0002277812300000072
a predicted tag value for the target;
the second part is a loss function of the Location network policy gradient algorithm, and the loss function formula is as follows:
Figure BDA0002277812300000073
wherein theta is a parameter of the strategy gradient, and parameters of the Glimpse network, the Core network and the Location network
Figure BDA0002277812300000074
θlAre all the strategy parameters controlled by the parameter theta of the strategy gradient,
Figure BDA0002277812300000075
the reward obtained for each sample in reinforcement learning,
Figure BDA0002277812300000076
the attenuation coefficient of the obtained reward is between 0 and 1,
Figure BDA0002277812300000077
the action behaviors sampled by the policy gradient algorithm,
Figure BDA0002277812300000078
the behavior states sampled by the policy gradient algorithm,
Figure BDA0002277812300000079
performing an action behavior
Figure BDA00022778123000000710
Accumulated awards obtained, biIs a reward benchmark value that depends on the behavior state
Figure BDA00022778123000000711
Independent of motion behavior
Figure BDA00022778123000000712
Figure BDA00022778123000000713
The representative is that the strategy gradient algorithm directly parameterizes the strategy, namely, the strategy is represented by a parameterized function to seek the optimal strategy;
the loss function of the third part is based on the classification result aTThe reward obtained whether the reward is correct and the baseline form a mean square error loss function, and the loss function formula is as follows:
Figure BDA00022778123000000714
using error to obtain training reference value biThe increase of the baseline network can reduce the probability of behavior actions below a baseline value and increase the probability of behavior actions above the baseline value, wherein,
Figure BDA00022778123000000715
the reward obtained for each sample in reinforcement learning if the classification result aTIf correct, the reward function is
Figure BDA00022778123000000716
Otherwise the reward function is
Figure BDA00022778123000000717
biA baseline value predicted for each sample;
according to the loss functions of the first part, the second part and the third part, the loss function of the RAM training model is defined as follows:
Figure BDA0002277812300000081
2.2.10) back-propagating the neural network according to the loss function, thereby updating the parameters of the network
According to the Loss function Loss (θ) in step 2.2.9), the Action network is trained and propagated backward to the Glimpse network or the Core network, but it should be noted that, in the backward propagation process, the Location network and the Baseline network are always trained by using a strategy gradient of reinforcement learning, and although the input of the Location network and the Baseline network is the hidden state of the Core network, the input of the base network is not propagated backward to the Core network or the Glimpse network, because the algorithm is difficult to converge and has poor effect, the weight updating method of reinforcement learning is as follows: thetat+1＝θt+αγtRt▽θlog(At|Stθ), θ is a parameter of the policy gradient, and a parameter of the Glimpse network, the Core network, the Location network
Figure BDA0002277812300000082
θlThe parameters are all strategy parameters controlled by the parameter theta of the strategy gradient, so that the parameters of the whole network are updated;
2.2.12) repeating the training from the step 2.2.1) to the step 2.2.10) for M times to obtain the final network parameters;
3) and the trained RAM model can be used for carrying out visual detection on the brain nuclear magnetic resonance abnormal image.
Experimental data:
the experimental data is brain nuclear magnetic resonance abnormal images of patients with the size of about 12 thousands and the size of 128 x 128 as training samples; parameters in the experiment: the training times M of the RAM model are 60000 times, T is 7 times, batch is 256 groups, and the experimental results are as follows:
Figure BDA0002277812300000083
the training comparison experiment is carried out according to two different models in the table 1, and compared with a Convolutional Neural Network (CNN), the RAM model provided by the text has the advantages that the CNN network detection capability is poor, the RAM model detection operation speed is high, the accuracy is high, and the experiment effect is good.

Claims (5)

1. A brain nuclear magnetic resonance abnormal image visualization method based on a circulation attention model is characterized in that: the specific method comprises the following steps:
1) acquiring a brain nuclear magnetic resonance abnormal image of a patient as a training sample;
2) training the circulating attention model by using the training sample to obtain trained network parameters;
2.1) constructing a RAM model and randomly initializing network parameters
And constructing a RAM model, wherein the RAM model belongs to a new hard attention model algorithm based on a position and combining reinforcement learning and a recurrent neural network. The RAM model comprises five parts of a Glimpse network, a Core network, an Action network, a Location network and a Baseline network, and is randomly initialized, namely the parameters of the Glimpse network, the Core network, the Action network, the Location network and the Baseline network are initialized
Figure FDA0002277812290000011
Figure FDA0002277812290000012
l(ht|θl) And b (h)t|θb) Is a random initial value;
2.2) training the RAM model
2.2.1) random initialization of the first attention position l0；
2.2.2) according to the first attention position l0Obtaining Glimpse characteristics g0；
2.2.3) first hidden state h of the time-series Core network0Initialization is 0;
2.2.4) hidden state h of Core network0And features of the Glimpse network g0As Core network input, the output obtains a new hidden state h1
2.2.5) newly hiding Core networkHidden state h1As the input of the Action network, outputting the predicted classification result a1
2.2.6) outputting the new hidden state h of the Core network1As the input of the Baseline network, the output is obtained as a one-dimensional vector b1The Baseline network reduces the variance of the gradient to be within a threshold range epsilon;
2.2.7) outputting the new hidden state h of the Core network1As the input of the Location network, the Location network adopts a policy gradient algorithm to determine the attention position of the next time sequence, and the output attention position of the next time sequence is l1；
2.2.8) circulating the step 2.2.2) to the step 2.2.7), and repeating the step T times;
2.2.9) constructing a loss function for the network
Defining the overall loss function of the RAM training model as:
Figure FDA0002277812290000021
2.2.10) back-propagating the neural network according to the loss function, thereby updating the parameters of the network
According to the Loss function Loss (theta) in the step 2.2.9), training the Action network and reversely propagating the Action network to the Glimpse network and the Core network, wherein in the process of reversely propagating, the Location network and the Baseline network are trained by adopting a strategy gradient of reinforcement learning, although the input of the Location network and the Baseline network is the hidden state of the Core network, the input of the base network is not reversely propagated to the Core network and the Glimpse network, and the reinforcement learning weight value updating mode is as follows: thetat+1＝θt+αγtRt▽θlog(At|Stθ), θ is a parameter of the policy gradient, and a parameter of the Glimpse network, the Core network, the Location network
Figure FDA0002277812290000022
θlThe parameters are all strategy parameters controlled by the parameter theta of the strategy gradient, so that the parameters of the whole network are updated;
2.2.11 repeating the training from step 2.2.1) to step 2.2.10) M times to obtain the final network parameters;
3) and carrying out visual detection on the brain nuclear magnetic resonance abnormal image by using the trained RAM model.
2. The method for visualizing the abnormal nuclear magnetic resonance image based on the circulatory attention model as claimed in claim 1, wherein: in step 2.2), the Glimpse network comprises a Glimpse sensor, which samples the magnetic resonance image x of the brain of the patient to be treated, around the first attention position, to obtain 4 square images with different lengths, with the first attention position as the center of the image fixation area, and then uniformly transforms them into a group of images with the size of 32 × 32 by using nearest neighbor interpolation, wherein the first attention position l of the images0Is a higher resolution image, and larger areas from the middle area outwards are progressively lower resolution images, and then the Glimpse sensor is based on the set of images obtained and the first attention position/0Carrying out feature extraction, and connecting through full-connection layers to obtain features g output by the Glimpse network0。
3. The method for visualizing the abnormal nuclear magnetic resonance image based on the circulatory attention model as claimed in claim 1, wherein: in said step 2.2.4), the Core network is actually an RNN network, and the hidden state h output by the Core network in the previous time series is chronologically output0And the feature g currently output via the Glimpse network0The two characteristics are combined to be used as the input of the Core network, and the output of the Core network obtains a new hidden state h in the RNN network1。
4. The method for visualizing the abnormal nuclear magnetic resonance image based on the circulatory attention model as claimed in claim 1, wherein: in the step 2.2.5), the Action network outputs a new hidden state h output by the Core network1As input, the training target is the detection of nuclear magnetic resonance images of the brain of the patient, an Action networkIf the network is a classification network, the predicted classification result is output as a1And further based on the predicted classification result a1And the actual label of the image obtains the reward function, wherein if the classification result a is correct, the reward function is 1, otherwise, the reward function is 0.
5. The method for visualizing the abnormal nuclear magnetic resonance image based on the circulatory attention model as claimed in claim 1, wherein: said step 2.2.9), the loss function consists of three parts:
the first part is the last classification result aTAnd the cross entropy loss function formed by the actual category of the image, wherein the loss function formula is as follows:
Figure FDA0002277812290000031
wherein y isiFor the input magnetic resonance image true tag values,
Figure FDA0002277812290000032
a predicted tag value for the target;
the second part is a loss function of the Location network policy gradient algorithm, and the loss function formula is as follows:
Figure FDA0002277812290000033
wherein theta is a parameter of the strategy gradient, and parameters of the Glimpse network, the Core network and the Location network
Figure FDA0002277812290000034
θlAre all the strategy parameters controlled by the parameter theta of the strategy gradient,
Figure FDA0002277812290000035
the reward obtained for each sample in reinforcement learning,
Figure FDA0002277812290000036
the attenuation coefficient of the obtained reward is between 0 and 1,
Figure FDA0002277812290000037
the action behaviors sampled by the policy gradient algorithm,
Figure FDA0002277812290000038
the behavior states sampled by the policy gradient algorithm,
Figure FDA0002277812290000039
performing an action behavior
Figure FDA00022778122900000310
Accumulated awards obtained, biIs a reward benchmark value that depends on the behavior state
Figure FDA0002277812290000041
Independent of motion behavior
Figure FDA0002277812290000042
Figure FDA0002277812290000043
The representative is that the strategy gradient algorithm directly parameterizes the strategy, namely, the strategy is represented by a parameterized function to seek the optimal strategy;
the loss function of the third part is based on the classification result aTThe reward obtained whether the reward is correct and the baseline form a mean square error loss function, and the loss function formula is as follows:
Figure FDA0002277812290000044
using error to obtain training reference value biThe increase of the baseline network can reduce the probability of behavior actions below a baseline value and increase the probability of behavior actions above the baseline value, wherein,
Figure FDA0002277812290000045
obtained for each sample in reinforcement learningAwarding if the classification result aTIf correct, the reward function is
Figure FDA0002277812290000046
Otherwise the reward function is
Figure FDA0002277812290000047
biA baseline value predicted for each sample; according to the loss functions of the first part, the second part and the third part, the loss function of the RAM training model is defined as follows:
Figure FDA0002277812290000048
CN201911129218.6A 2019-11-18 2019-11-18 Brain nuclear magnetic resonance abnormal image visualization method based on circulation attention model Pending CN110956617A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911129218.6A CN110956617A (en) 2019-11-18 2019-11-18 Brain nuclear magnetic resonance abnormal image visualization method based on circulation attention model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911129218.6A CN110956617A (en) 2019-11-18 2019-11-18 Brain nuclear magnetic resonance abnormal image visualization method based on circulation attention model

Publications (1)

Publication Number Publication Date
CN110956617A true CN110956617A (en) 2020-04-03

Family

ID=69977767

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911129218.6A Pending CN110956617A (en) 2019-11-18 2019-11-18 Brain nuclear magnetic resonance abnormal image visualization method based on circulation attention model

Country Status (1)

Country Link
CN (1) CN110956617A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021243724A1 (en) * 2020-06-05 2021-12-09 北京嘀嘀无限科技发展有限公司 Image processing method, electronic device, vehicle traveling data recorder, and server
CN114743053A (en) * 2022-04-14 2022-07-12 电子科技大学 Magnetic resonance image auxiliary processing system based on graph neural network and self attention
CN116363512A (en) * 2023-03-24 2023-06-30 北京城市轨道交通咨询有限公司 Method and device for detecting stability of surrounding rock

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106022244A (en) * 2016-05-16 2016-10-12 广东工业大学 Unsupervised crowd abnormity monitoring and positioning method based on recurrent neural network modeling
CN106096729A (en) * 2016-06-06 2016-11-09 天津科技大学 A kind of towards the depth-size strategy learning method of complex task in extensive environment
CN107346430A (en) * 2017-06-29 2017-11-14 济南浪潮高新科技投资发展有限公司 A kind of nuclear magnetic resonance image auto-screening method
CN108170736A (en) * 2017-12-15 2018-06-15 南瑞集团有限公司 A kind of document based on cycle attention mechanism quickly scans qualitative method
CN109325517A (en) * 2018-08-14 2019-02-12 浙江大学 A kind of figure classification method of the Recognition with Recurrent Neural Network model based on Attention
US20190318648A1 (en) * 2018-04-12 2019-10-17 Baidu Usa Llc Systems and methods for interactive language acquisition with one-shot visual concept learning through a conversational game

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106022244A (en) * 2016-05-16 2016-10-12 广东工业大学 Unsupervised crowd abnormity monitoring and positioning method based on recurrent neural network modeling
CN106096729A (en) * 2016-06-06 2016-11-09 天津科技大学 A kind of towards the depth-size strategy learning method of complex task in extensive environment
CN107346430A (en) * 2017-06-29 2017-11-14 济南浪潮高新科技投资发展有限公司 A kind of nuclear magnetic resonance image auto-screening method
CN108170736A (en) * 2017-12-15 2018-06-15 南瑞集团有限公司 A kind of document based on cycle attention mechanism quickly scans qualitative method
US20190318648A1 (en) * 2018-04-12 2019-10-17 Baidu Usa Llc Systems and methods for interactive language acquisition with one-shot visual concept learning through a conversational game
CN109325517A (en) * 2018-08-14 2019-02-12 浙江大学 A kind of figure classification method of the Recognition with Recurrent Neural Network model based on Attention

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
VOLIDYMYR MNIH等: "Recurrent Models of Visual Attention", 《ADVANCED IN NEURAL INFORMATION PROCESSING SYSTEMS》 *
VOLIDYMYR MNIH等: "Recurrent Models of Visual Attention", 《ADVANCED IN NEURAL INFORMATION PROCESSING SYSTEMS》, 24 June 2014 (2014-06-24), pages 1 - 8 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021243724A1 (en) * 2020-06-05 2021-12-09 北京嘀嘀无限科技发展有限公司 Image processing method, electronic device, vehicle traveling data recorder, and server
CN114743053A (en) * 2022-04-14 2022-07-12 电子科技大学 Magnetic resonance image auxiliary processing system based on graph neural network and self attention
CN114743053B (en) * 2022-04-14 2023-04-25 电子科技大学 Magnetic resonance image auxiliary processing system based on graph neural network and self-attention
CN116363512A (en) * 2023-03-24 2023-06-30 北京城市轨道交通咨询有限公司 Method and device for detecting stability of surrounding rock

Similar Documents

Publication Publication Date Title
CN110378381B (en) Object detection method, device and computer storage medium
CN110335261B (en) CT lymph node detection system based on space-time circulation attention mechanism
CN110428428B (en) Image semantic segmentation method, electronic equipment and readable storage medium
CN110210551B (en) Visual target tracking method based on adaptive subject sensitivity
CN109409222B (en) Multi-view facial expression recognition method based on mobile terminal
CN110956126B (en) Small target detection method combined with super-resolution reconstruction
CN109711426B (en) Pathological image classification device and method based on GAN and transfer learning
CN113807355A (en) Image semantic segmentation method based on coding and decoding structure
CN107016406A (en) The pest and disease damage image generating method of network is resisted based on production
CN110956617A (en) Brain nuclear magnetic resonance abnormal image visualization method based on circulation attention model
CN112634211B (en) MRI image segmentation method, device and equipment based on multiple neural networks
CN113256592B (en) Training method, system and device of image feature extraction model
CN111127490A (en) Medical image segmentation method based on cyclic residual U-Net network
CN113706544B (en) Medical image segmentation method based on complete attention convolutional neural network
Zilvan et al. Denoising convolutional variational autoencoders-based feature learning for automatic detection of plant diseases
CN116310693A (en) Camouflage target detection method based on edge feature fusion and high-order space interaction
Du et al. Efficient network construction through structural plasticity
CN111401209B (en) Action recognition method based on deep learning
CN118196231A (en) Lifelong learning draft method based on concept segmentation
CN117711442A (en) Infant crying classification method based on CNN-GRU fusion model
CN111402180A (en) Image fusion method based on FBIEM and ISCM of NSCT domain
CN114886438B (en) Epileptic detection method based on EEG single sample deep learning
CN114565964B (en) Emotion recognition model generation method, emotion recognition device, emotion recognition medium and emotion recognition equipment
CN113516670B (en) Feedback attention-enhanced non-mode image segmentation method and device
Yang et al. UAV Landmark Detection Based on Convolutional Neural Network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200403

RJ01 Rejection of invention patent application after publication