[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN107610194A - MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN - Google Patents

MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN Download PDF

Info

Publication number
CN107610194A
CN107610194A CN201710689598.3A CN201710689598A CN107610194A CN 107610194 A CN107610194 A CN 107610194A CN 201710689598 A CN201710689598 A CN 201710689598A CN 107610194 A CN107610194 A CN 107610194A
Authority
CN
China
Prior art keywords
mrow
msup
layer
mri
convolutional neural
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710689598.3A
Other languages
Chinese (zh)
Other versions
CN107610194B (en
Inventor
刘昶
吴锡
周激流
郎方年
于曦
赵卫东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu University
Original Assignee
Chengdu University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu University filed Critical Chengdu University
Priority to CN201710689598.3A priority Critical patent/CN107610194B/en
Publication of CN107610194A publication Critical patent/CN107610194A/en
Application granted granted Critical
Publication of CN107610194B publication Critical patent/CN107610194B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Magnetic Resonance Imaging Apparatus (AREA)
  • Image Analysis (AREA)

Abstract

The present invention relates to a kind of MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN, low-resolution image and corresponding high-definition picture are pre-processed first, and build training dataset and label data collection, then the structure fusion full convolutional neural networks of multi-scale information, training dataset is input in the fusion multi-scale information convolutional neural networks of structure and be trained, convolutional neural networks model after being learnt, it will test in the convolutional neural networks after low-resolution image is input to study, obtain rebuilding high-definition picture.The present invention passes through Multiscale Fusion unit, the Feature Mapping of different convolutional layers is merged, overcome the flat layered structures that the multilayer convolutional layer of traditional convolutional neural networks stacks, the convergence rate of network can be accelerated, quickly reconstruct the image detail of low-resolution image loss, reconstruction time is reduced, improves and rebuilds efficiency, avoid the wasting of resources.

Description

MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN
Technical field
The invention belongs to image processing field, more particularly to a kind of MRI oversubscription based on Multiscale Fusion CNN Resolution method for reconstructing.
Background technology
Higher spatial resolution structure MRI has less artifact, directly affects successive image processing and medical treatment The precision of diagnosis, such as registration, segmentation etc..But due to physical equipment, the limitation in terms of acquisition technique and economic dispatch is existing The spatial resolution of MRI is influenceed by certain.
In image processing field, traditional super-resolution method for reconstructing mainly uses interpolation method, such as bilinear interpolation, B samples The methods of bar interpolation.These methods assume that regional area has smooth property, and the voxel of new interpolation is estimated according to neighboring voxel Value.But interpolation method is not suitable for Nonuniform Domain Simulation of Reservoir, easily causes image to obscure.
For MRI, according to different phase of regeneration, super resolution ratio reconstruction method is broadly divided into two kinds：The first It is reconstituted in gatherer process, directly K space data is rebuild；Second, which is rebuild, is passed in post-processing stages, generally use System method for reconstructing is applied to structure magnetic resonance image data.The most frequently used method is non-local mean method and sparse coding side Method.Because the priori that non-local mean method is rebuild still comes from topography's block, it can not obtain and preferably rebuild effect. Sparse coding method uses machine learning method, respectively from low-resolution image block and corresponding high-definition picture block learning Low resolution and high-resolution dictionary；Then linear group in low-resolution image rarefaction representation low-resolution dictionary space is thought Close, solve its sparse coefficient.And sparse coefficient is projected into high-resolution dictionary space, so as to the high-resolution after being rebuild Image.But the sparse expression based on image block can not ensure the optimal reconstruct of general image.
The training of traditional convolutional neural networks needs great amount of samples just to can guarantee that final preferable effect.Led in medical science Domain, it is difficult to substantial amounts of magnetic resonance image data is obtained, therefore directly using traditional convolutional neural networks it is difficult to ensure that the receipts of network Hold back and reconstruction precision.
The content of the invention
For the deficiency of prior art, the present invention proposes a kind of MRI super-resolution based on Multiscale Fusion CNN Rate method for reconstructing, methods described include：
Step 1：Low resolution structure MRI and corresponding high resolution structures MRI are carried out Pretreatment operation, and build training dataset and label data collection；
Step 11：The low resolution structure MRI and high resolution structures MRI of reference format are inputted, Enter row format conversion；
Step 12：By the low resolution structure MRI after being changed in step 11 and the high resolution structures MRI removes skull part, only retains brain area part；
Step 13：To the low resolution structure MRI after removal skull in step 12 and the high-resolution Structure MRI is normalized, and is normalized to [0-1] section；
Step 14：To the resolution structural MRI after normalized in step 13 and the high resolution structures MRI extracts multiple two dimensional image blocks respectively successively using sliding window mode on every layer, wherein by low resolution figure As block composing training data set, high-definition picture block forms label data collection；
Step 2：Structure fusion multi-scale information convolutional neural networks, the convolutional neural networks include an input layer, At least three Multiscale Fusion units stacked and a reconstruction of layer；
Step 21：The input layer is used to receive the training dataset；
Step 22：Build at least three Multiscale Fusion units；
Step 23：Reconstruction of layer is built, the reconstruction of layer is the convolutional layer that a convolution kernel is formed；
Step 3：The training dataset is input in the convolutional neural networks that step 2 builds and is trained, is learned Convolutional neural networks model after habit；
Step 31：The training dataset is divided into more batches of training datas, and the structure of initialization step 2 is described multiple dimensioned Convolution kernel weight and biasing in information convolutional neural networks in all convolutional layers are 0 to loss function inverse, i.e.,：
△W(l)=0
△b(l)=0
Wherein, W represents convolution kernel weight, and b represents that biasing represents l layers to loss function, l；
Step 32：A collection of training data is inputted every time to be counted with each node parameter in the Multiscale Fusion unit Calculate, realize the propagated forward of neural metwork training, finally by reconstruction of layer, obtain output high-resolution data；
Step 33：Using Euclidean distance, by the output high-resolution data obtained in step 32 and the label data collection Error：
Wherein, I, J represent the size of image block；
Step 34：Based on the error, using gradient descent method, backwards calculation convolution kernel weight and biasing are to loss function DerivativeWithAnd accumulate it △ W(l)With △ b(l), i.e.,：
Step 35：Repeat step 32 is to step 34, until all training datas are disposed, completes an iteration, according to Above-mentioned △ W(l)With △ b(l), using batch gradient descent algorithm, network parameter after being updated：
Wherein m represents the lot number of training data, and α is learning rate, and λ is kinetic energy；
Step 36：Repeat step 32 is to step 35, until reaching default iterations；
Step 4：Test low resolution structure MRI is input in the convolutional neural networks that step 3 trains, High resolution structures MRI is rebuild in output；
Step 41：The each layer for testing low resolution structure MRI is directly inputted into the convolution that step 3 trains Input layer in neural network model；
Step 42：The test low resolution structure MRI that step 41 receives is input to the convolutional Neural succeeded in school In network model, computing is carried out from front to back, is finally exported in reconstruction of layer and is rebuild high resolution structures MRI.
According to a kind of preferred embodiment, the Multiscale Fusion unit include main path, at least one subpath and Fused layer, the main path add a ReLU activation primitive to form by a convolutional layer, and the subpath is added by a convolutional layer One ReLU activation primitive is alternately formed successively, and last layer is convolutional layer, and the fused layer is by the main path and institute The output for stating subpath is merged to be output to next Multiscale Fusion unit by being added.
Compared with prior art, the beneficial effects of the present invention are：
1st, the Feature Mapping of different convolutional layers is merged by Multiscale Fusion unit, overcomes tradition by the present invention The flat layered structures that the multilayer convolutional layer of convolutional neural networks stacks, can accelerate the convergence rate of network, quickly rebuild low point The image detail of resolution missing image, the time rebuild and needed is reduced, reduces the wasting of resources.
2nd, compared with existing method for reconstructing, the reconstruction effect that method for reconstructing of the invention obtains is more preferable, the details of recovery Information and structural information are also higher closer to true high-definition picture, the Y-PSNR of acquisition.
Brief description of the drawings
Fig. 1 is the schematic flow sheet of super resolution ratio reconstruction method of the present invention；
Fig. 2 is the structural representation of Multiscale Fusion unit of the present invention；
Fig. 3 is the structural representation of present invention fusion multi-scale information convolutional neural networks；
Fig. 4 is the structural representation of traditional convolutional neural networks；
Fig. 5 is the Feature Mapping of Multiscale Fusion unit various pieces output；
Fig. 6 is the reconstruction design sketch of all kinds of methods on emulation data set Brainweb；With
The reconstruction design sketch of Fig. 7 all kinds of methods on True Data collection.
Embodiment
To make the object, technical solutions and advantages of the present invention of greater clarity, with reference to embodiment and join According to accompanying drawing, the present invention is described in more detail.It should be understood that these descriptions are exemplary, and it is not intended to limit this hair Bright scope.In addition, in the following description, the description to known features and technology is eliminated, to avoid unnecessarily obscuring this The concept of invention.
The Multiscale Fusion unit MFU of the present invention：The Multi-scale Fusion Unit.
Fig. 1 is the schematic flow sheet of super resolution ratio reconstruction method of the present invention.As shown in figure 1, one kind proposed by the present invention is more Yardstick merges CNN MRI super resolution ratio reconstruction method, and method includes：
Step 1：Low resolution structure MRI and corresponding high resolution structures MRI are carried out Pretreatment operation, and build training dataset and label data collection.High-definition picture derives from true using 3T magnetic resonance equipments The image collected in fact, low-resolution image, which derives from, carries out what down-sampling obtained to high-definition picture.Low point in step 1 Resolution structure MRI carrys out training convolutional neural networks as training sample.
Step 11：The low resolution structure MRI and high resolution structures MRI of reference format are inputted, Enter row format conversion.Original magnetic resonance image data format is DCM forms, and NII forms are converted into using SPM.Reason exists It is made up of in the MR data that original DCM forms are a people N number of DCM files, and after switching to NII forms, the magnetic of a people Resonance data is made up of 1 NII file, is convenient to data processing below.
Step 12：By low resolution structure MRI and high resolution structures magnetic resonance figure after being changed in step 11 As removing skull part, only retain brain area part.
Step 13：Low resolution structure MRI after removal skull in step 12 and high resolution structures magnetic are total to The image that shakes is normalized, and is normalized to [0-1] section.The magnetic resonance image data scope arrived due to acquired original From 0 to up to ten thousand, and image procossing is generally by its range conversion to [0-1], same in order to which all data are put into Scope.
Step 14：To the low resolution structure MRI and high resolution structures magnetic after normalized in step 13 Resonance image extracts multiple two dimensional image blocks respectively successively using sliding window mode on every layer, wherein by low-resolution image Block composing training data set, high-definition picture block form label data collection.The number of two dimensional image block by image size and The size of sliding window typically gets up to ten thousand come what is controlled.Specifically, the MR data of human brain is three-dimensional data M*N*S, Magnetic resonance machine scans scan human brain from level to level from top to bottom, and S represents the number of plies, and M*N represents the size of this layer of brain.
Step 2：The convolutional neural networks of structure fusion multi-scale information.
Fig. 3 is the structural representation that the present invention merges multiple dimensioned information convolutional neural networks.Fig. 4 is traditional convolutional Neural The structural representation of network.As shown in figure 4, traditional convolutional neural networks are the flat layered structures that multilayer convolutional layer stacks.Such as Fig. 3 Shown, multi-scale information convolutional neural networks include an input layer, the Multiscale Fusion unit of multiple stackings and a reconstruct Layer.Specifically, at least three Multiscale Fusion units stacked.The multiple dimensioned convolutional neural networks of fusion of the present invention overcome biography The deficiency of system convolutional neural networks, can accelerate the convergence rate of neutral net, quickly rebuild what low-resolution image was lost Image detail, the time rebuild and needed is reduced, it is more efficient, reduce the wasting of resources.
Step 21：Input layer is used to receive training dataset.
Step 22：Build at least three Multiscale Fusion units.Fig. 2 is the structural representation of Multiscale Fusion unit.Such as Shown in Fig. 2, Multiscale Fusion unit includes main path, at least one subpath and fused layer.Main path is added by a convolutional layer One ReLU activation primitive is formed, and subpath adds a ReLU activation primitive alternately to form successively by a convolutional layer, and most Later layer is convolutional layer.Fused layer is by the output of main path and subpath by being added fusion to be output to next multiple dimensioned melt Close unit.
Step 23：Reconstruction of layer is built, reconstruction of layer is the convolutional layer that a convolution kernel is formed.
Step 3：Training dataset is input in the convolutional neural networks that step 2 builds and is trained, after being learnt Convolutional neural networks model.
Step 31：Training dataset is divided into more batches of training datas.Because the data volume of training dataset is larger, construction Neutral net can not disposably handle all training datas, it is therefore desirable to training dataset is divided into multiple batches and handled. Specific lot number is depending on the number of training sample and every batch of number of samples, such as has 10,000 training samples, 100 every batch, Then it is divided into 100 batches of training datas.And in the multi-scale information convolutional neural networks of the structure of initialization step 2 in all convolutional layers Convolution kernel weight and biasing are 0 to loss function inverse, i.e.,：
△W(l)=0
△b(l)=0
Wherein, W represents convolution kernel weight, and b represents that biasing represents l layers to loss function, l.All convolutional layers include more The convolutional layer of convolutional layer and composition reconstruction of layer in yardstick integrated unit.
Step 32：A collection of training data is inputted every time to be calculated with each node parameter in Multiscale Fusion unit, it is real The propagated forward of existing neural metwork training, finally by reconstruction of layer, obtain output high-resolution data；
Step 33：Using Euclidean distance, the error for exporting high-resolution data and label data collection is calculated：
Wherein, I, J represent the size of image block.
Step 34：The error calculated based on step 33, using gradient descent method, backwards calculation convolution kernel weight and biasing pair The derivative of loss functionWithAnd △ W are added to it(l)With △ b(l), i.e.,：
Step 35：Repeat step S32-S34, until all training samples are disposed, complete an iteration.According to above-mentioned △W(l)With △ b(l), using batch gradient descent algorithm, network parameter after being updated.
Wherein, m represents the lot number of training sample, and α is learning rate, and λ is kinetic energy, is determined in parameter renewal process, upper one The influence size of secondary iterative parameter.
Step 36：Repeat step 32 is to step 35, until reaching default iterations.General iterations is taken as 10 5 powers, or loss are less than 0.02 or so, can be determined by loss function.After iteration stopping, that is, the convolution trained is refreshing Through network model.
Step 4：Low resolution structure MRI will be tested and be input to the convolutional neural networks model that step 3 trains In, high resolution structures MRI is rebuild in output.Low resolution structure MRI is tested as test sample.
Step 41：The each layer for testing low resolution structure MRI is directly inputted into the convolution that step 3 trains Input layer in neural network model.
Step 42：The test low resolution structure MRI that step 41 receives is input to the convolutional Neural net trained In network, computing is carried out from front to back, is finally exported in reconstruction of layer and is rebuild high resolution structures MRI.Rebuild high-resolution Structure MRI is the high resolution structures MRI learnt by Multiscale Fusion CNN.
Fig. 5 is the Feature Mapping figure of Multiscale Fusion unit various pieces output.
The first row image is low resolution structure MRI in Fig. 5, and the second row image is Multiscale Fusion unit master Path output Feature Mapping figure, the third line image be Multiscale Fusion unit subpath output Feature Mapping figure, fourth line Image is the Feature Mapping figure of Multiscale Fusion layer output.
Fig. 6 is the reconstruction design sketch of all kinds of methods on emulation data set Brainweb.Fig. 7 is all kinds of on True Data collection The reconstruction design sketch of method.HR Real Data represent true high-definition picture in Fig. 7, and LR Real Data represent true low Image in different resolution, according to true low-resolution image Reconstructing High.Fig. 6 and Fig. 7 the second row image is reconstruction image Partial enlarged drawing, can intuitively find out from Fig. 6 and Fig. 7, using the present invention method MFCN (Multi-scale Fusion Convolution Network) reconstruction effect it is best, the high-resolution edge detail information and structure of reconstruction and true high score Resolution image is closer, the part that oval circle marks in especially Fig. 6 and Fig. 7.Further, with Y-PSNR PSNR guest See and evaluate rebuilding effect, Y-PSNR is higher, represents that reconstruction effect is better.It can be seen from figures 6 and 7 that this The Y-PSNR that the method for invention obtains is higher compared to for existing several method.
The super resolution ratio reconstruction method of the present invention, the reconstruction of MRI is applicable not only to, while suitable for other necks The reconstruction of the image reconstruction in domain, such as Weather Radar, CT images, PET-CT image reconstructions.
Super resolution ratio reconstruction method of the invention based on Multiscale Fusion CNN, overcomes traditional convolutional neural networks and is difficult to Ensure the convergence of neutral net and the deficiency of reconstruction precision, without obtaining substantial amounts of magnetic resonance image data, by different convolutional layers Feature Mapping merged, overcome traditional convolutional neural networks multilayer convolutional layer stack flat layered structures, can accelerate The convergence rate of network, the image detail that low-resolution image is lost quickly is rebuild, reduce the time rebuild and needed, reduce The wasting of resources.Further, intuitively with can be seen that in objective indicator, the inventive method obtain it is higher than existing reconstruction technique Reconstruction precision.
It should be noted that above-mentioned specific embodiment is exemplary, those skilled in the art can disclose in the present invention Various solutions are found out under the inspiration of content, and these solutions also belong to disclosure of the invention scope and fall into this hair Within bright protection domain.It will be understood by those skilled in the art that description of the invention and its accompanying drawing are illustrative and are not Form limitations on claims.Protection scope of the present invention is limited by claim and its equivalent.

Claims (2)

  1. A kind of 1. MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN, it is characterised in that methods described bag Include：
    Step 1：Low resolution structure MRI and corresponding high resolution structures MRI are located in advance Reason operation, and build training dataset and label data collection；
    Step 11：The low resolution structure MRI and high resolution structures MRI of reference format are inputted, is carried out Form is changed；
    Step 12：The low resolution structure MRI after being changed in step 11 and the high resolution structures magnetic are total to The image that shakes removes skull part, only retains brain area part；
    Step 13：To the low resolution structure MRI after removal skull in step 12 and the high resolution structures MRI is normalized, and is normalized to [0-1] section；
    Step 14：To the low resolution structure MRI after normalized in step 13 and the high resolution structures magnetic Resonance image extracts multiple two dimensional image blocks respectively successively using sliding window mode on every layer, wherein by low-resolution image Block composing training data set, high-definition picture block form label data collection；
    Step 2：Structure fusion multi-scale information convolutional neural networks, the convolutional neural networks include an input layer, at least The Multiscale Fusion unit and a reconstruction of layer of three stackings；
    Step 21：The input layer is used to receive the training dataset；
    Step 22：Build at least three Multiscale Fusion units；
    Step 23：Reconstruction of layer is built, the reconstruction of layer is the convolutional layer that a convolution kernel is formed；
    Step 3：The training dataset is input in the convolutional neural networks that step 2 builds and is trained, after being learnt Convolutional neural networks model；
    Step 31：The training dataset is divided into more batches of training datas, and the multi-scale information that initialization step 2 is built Convolution kernel weight and biasing in convolutional neural networks in all convolutional layers are 0 to loss function inverse, i.e.,：
    △W(l)=0
    △b(l)=0
    Wherein, W represents convolution kernel weight, and b represents that biasing represents l layers to loss function, l；
    Step 32：A collection of training data is inputted every time to be calculated with each node parameter in the Multiscale Fusion unit, it is real The propagated forward of existing neural metwork training, finally by reconstruction of layer, obtain output high-resolution data；
    Step 33：Using Euclidean distance, by the mistake of the output high-resolution data and the label data collection that are obtained in step 32 Difference：
    <mrow> <mi>E</mi> <mo>=</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>=</mo> <mn>0</mn> </mrow> <mrow> <mi>I</mi> <mo>,</mo> <mi>J</mi> </mrow> </munderover> <mo>|</mo> <mo>|</mo> <msub> <mi>X</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>-</mo> <msub> <mover> <mi>X</mi> <mo>&amp;OverBar;</mo> </mover> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> </mrow>
    Wherein, I, J represent the size of image block；
    Step 34：Based on the error, using gradient descent method, backwards calculation convolution kernel weight and biasing are led to loss function NumberWithAnd accumulate it △ W(l)With △ b(l), i.e.,：
    <mrow> <msup> <mi>&amp;Delta;W</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>=</mo> <msup> <mi>&amp;Delta;W</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>+</mo> <msub> <mo>&amp;dtri;</mo> <msup> <mi>W</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> </msub> <mi>J</mi> <mrow> <mo>(</mo> <mi>W</mi> <mo>,</mo> <mi>b</mi> <mo>;</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mrow>
    <mrow> <msup> <mi>&amp;Delta;b</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>=</mo> <msup> <mi>&amp;Delta;b</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>+</mo> <msub> <mo>&amp;dtri;</mo> <msup> <mi>b</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> </msub> <mi>J</mi> <mrow> <mo>(</mo> <mi>W</mi> <mo>,</mo> <mi>b</mi> <mo>;</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mrow>
    Step 35：Repeat step 32 is to step 34, until all training datas are disposed, an iteration is completed, according to above-mentioned △W(l)With △ b(l), it is as follows using batch gradient descent algorithm, network parameter after being updated, mathematical notation：
    <mrow> <msup> <mi>W</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>=</mo> <msup> <mi>W</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>-</mo> <mi>&amp;alpha;</mi> <mo>&amp;lsqb;</mo> <mrow> <mo>(</mo> <mfrac> <mn>1</mn> <mi>m</mi> </mfrac> <msup> <mi>&amp;Delta;W</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>)</mo> </mrow> <mo>+</mo> <msup> <mi>&amp;lambda;W</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>&amp;rsqb;</mo> </mrow>
    <mrow> <msup> <mi>b</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>=</mo> <msup> <mi>b</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>-</mo> <mi>&amp;alpha;</mi> <mo>&amp;lsqb;</mo> <mfrac> <mn>1</mn> <mi>m</mi> </mfrac> <msup> <mi>&amp;Delta;b</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mo>&amp;rsqb;</mo> </mrow>
    Wherein, m represents the lot number of training data, and α is learning rate, and λ is kinetic energy；
    Step 36：Repeat step 32 is to step 35, until reaching default iterations；
    Step 4：Test low resolution structure MRI is input in the convolutional neural networks that step 3 trains, exported Rebuild high resolution structures MRI；
    Step 41：The each layer for testing low resolution structure MRI is directly inputted into the convolutional Neural that step 3 trains Input layer in network model；
    Step 42：The test low resolution structure MRI that step 41 receives is input to the convolutional neural networks succeeded in school In model, computing is carried out from front to back, is finally exported in reconstruction of layer and is rebuild high resolution structures MRI.
  2. 2. super resolution ratio reconstruction method as claimed in claim 1, it is characterised in that the Multiscale Fusion unit includes main road Footpath, at least one subpath and fused layer, the main path add a ReLU activation primitive to form by a convolutional layer, the son Path adds a ReLU activation primitive alternately to form successively by a convolutional layer, and last layer is convolutional layer, the fusion Layer merges the output of the main path and the subpath to be output to next Multiscale Fusion unit by being added.
CN201710689598.3A 2017-08-14 2017-08-14 Magnetic resonance image super-resolution reconstruction method based on multi-scale fusion CNN Active CN107610194B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710689598.3A CN107610194B (en) 2017-08-14 2017-08-14 Magnetic resonance image super-resolution reconstruction method based on multi-scale fusion CNN

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710689598.3A CN107610194B (en) 2017-08-14 2017-08-14 Magnetic resonance image super-resolution reconstruction method based on multi-scale fusion CNN

Publications (2)

Publication Number Publication Date
CN107610194A true CN107610194A (en) 2018-01-19
CN107610194B CN107610194B (en) 2020-08-04

Family

ID=61063987

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710689598.3A Active CN107610194B (en) 2017-08-14 2017-08-14 Magnetic resonance image super-resolution reconstruction method based on multi-scale fusion CNN

Country Status (1)

Country Link
CN (1) CN107610194B (en)

Cited By (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108335303A (en) * 2018-01-28 2018-07-27 浙江大学 A kind of multiple dimensioned palm bone segmentation method applied to palm X-ray
CN108416821A (en) * 2018-03-08 2018-08-17 山东财经大学 A kind of CT Image Super-resolution Reconstruction methods of deep neural network
CN108447062A (en) * 2018-02-01 2018-08-24 浙江大学 A kind of dividing method of the unconventional cell of pathological section based on multiple dimensioned mixing parted pattern
CN108492271A (en) * 2018-03-26 2018-09-04 中国电子科技集团公司第三十八研究所 A kind of automated graphics enhancing system and method for fusion multi-scale information
CN108596994A (en) * 2018-04-24 2018-09-28 朱高杰 A kind of Diffusion-weighted imaging method being in harmony certainly based on deep learning and data
CN108594146A (en) * 2018-04-24 2018-09-28 朱高杰 A kind of Diffusion-weighted imaging method based on deep learning and convex set projection
CN108828481A (en) * 2018-04-24 2018-11-16 朱高杰 A kind of magnetic resonance reconstruction method based on deep learning and data consistency
CN108898222A (en) * 2018-06-26 2018-11-27 郑州云海信息技术有限公司 A kind of method and apparatus automatically adjusting network model hyper parameter
CN109003229A (en) * 2018-08-09 2018-12-14 成都大学 Magnetic resonance super resolution ratio reconstruction method based on three-dimensional enhancing depth residual error network
CN109035356A (en) * 2018-07-05 2018-12-18 四川大学 A kind of system and method based on PET pattern imaging
CN109087273A (en) * 2018-07-20 2018-12-25 哈尔滨工业大学（深圳） Image recovery method, storage medium and the system of neural network based on enhancing
CN109146784A (en) * 2018-07-27 2019-01-04 徐州工程学院 A kind of image super-resolution rebuilding method based on multiple dimensioned generation confrontation network
CN109272443A (en) * 2018-09-30 2019-01-25 东北大学 A kind of PET based on full convolutional neural networks and CT method for registering images
CN109345449A (en) * 2018-07-17 2019-02-15 西安交通大学 A kind of image super-resolution based on converged network and remove non-homogeneous blur method
CN109410240A (en) * 2018-10-09 2019-03-01 电子科技大学中山学院 Method and device for positioning volume characteristic points and storage medium thereof
CN109447238A (en) * 2018-09-21 2019-03-08 广东石油化工学院 Multi-output regression depth network establishing method, structure, equipment and storage medium
CN109544488A (en) * 2018-10-08 2019-03-29 西北大学 A kind of image composition method based on convolutional neural networks
CN109685717A (en) * 2018-12-14 2019-04-26 厦门理工学院 Image super-resolution rebuilding method, device and electronic equipment
CN109741416A (en) * 2019-01-04 2019-05-10 北京大学深圳医院 Nuclear magnetic resonance image method for reconstructing, device, computer equipment and its storage medium
CN109859106A (en) * 2019-01-28 2019-06-07 桂林电子科技大学 A kind of image super-resolution rebuilding method based on the high-order converged network from attention
CN109903229A (en) * 2019-03-04 2019-06-18 科新(杭州)能源环境科技有限公司 A kind of μ-CT image reconstructing method based on convolutional neural networks
CN109919840A (en) * 2019-01-21 2019-06-21 南京航空航天大学 Image super-resolution rebuilding method based on dense feature converged network
CN110276736A (en) * 2019-04-01 2019-09-24 厦门大学 A kind of magnetic resonance image fusion method based on weight prediction network
CN110333076A (en) * 2019-06-19 2019-10-15 电子科技大学 Method for Bearing Fault Diagnosis based on CNN-Stacking
CN110598724A (en) * 2019-01-17 2019-12-20 西安理工大学 Cell low-resolution image fusion method based on convolutional neural network
CN110929521A (en) * 2019-12-06 2020-03-27 北京知道智慧信息技术有限公司 Model generation method, entity identification method, device and storage medium
CN110940944A (en) * 2019-12-04 2020-03-31 厦门大学 J coupling removing method for magnetic resonance signals based on deep learning
CN111127581A (en) * 2019-12-31 2020-05-08 东软医疗系统股份有限公司 Image reconstruction method and device, CT (computed tomography) equipment and CT system
CN111160413A (en) * 2019-12-12 2020-05-15 天津大学 Thyroid nodule classification method based on multi-scale feature fusion
CN111161182A (en) * 2019-12-27 2020-05-15 南方医科大学 MR structure information constrained non-local mean guided PET image partial volume correction method
CN111274865A (en) * 2019-12-14 2020-06-12 深圳先进技术研究院 Remote sensing image cloud detection method and device based on full convolution neural network
CN111275618A (en) * 2020-01-12 2020-06-12 杭州电子科技大学 Depth map super-resolution reconstruction network construction method based on double-branch perception
CN111292240A (en) * 2020-01-23 2020-06-16 上海交通大学 Magnetic resonance super-resolution imaging method based on imaging model and machine learning
CN111353944A (en) * 2018-12-20 2020-06-30 深圳市中兴微电子技术有限公司 Image reconstruction method and device and computer readable storage medium
WO2020135630A1 (en) * 2018-12-26 2020-07-02 Shanghai United Imaging Intelligence Co., Ltd. Systems and methods for image reconstruction
CN111798377A (en) * 2020-07-08 2020-10-20 广东工业大学 Magnetic resonance image super-resolution reconstruction method based on multi-resolution learning strategy
CN111956180A (en) * 2019-05-20 2020-11-20 华北电力大学（保定） Method for reconstructing photoacoustic endoscopic tomography image
CN112102388A (en) * 2020-09-18 2020-12-18 中国矿业大学 Method and device for acquiring depth image based on monocular image of inspection robot
CN112419437A (en) * 2019-11-29 2021-02-26 上海联影智能医疗科技有限公司 System and method for reconstructing magnetic resonance images
CN112767374A (en) * 2021-01-27 2021-05-07 天津理工大学 Alzheimer disease focus region semantic segmentation algorithm based on MRI
CN112789659A (en) * 2018-10-02 2021-05-11 皇家飞利浦有限公司 Generation of pseudo-radiographic image from optical image
CN112949636A (en) * 2021-03-31 2021-06-11 上海电机学院 License plate super-resolution identification method and system and computer readable medium
WO2021135474A1 (en) * 2020-01-02 2021-07-08 平安科技（深圳）有限公司 Method and apparatus for fusing data from multiple data sources, electronic device, and storage medium
CN113534031A (en) * 2020-04-21 2021-10-22 上海联影医疗科技股份有限公司 Image domain data generating method, computer device and readable storage medium
CN113935928A (en) * 2020-07-13 2022-01-14 四川大学 Rock core image super-resolution reconstruction based on Raw format
WO2022206021A1 (en) * 2021-03-30 2022-10-06 中国科学院深圳先进技术研究院 Image reconstruction model generation method and apparatus, image reconstruction method and apparatus, and device and medium
CN116805284A (en) * 2023-08-28 2023-09-26 之江实验室 Feature migration-based super-resolution reconstruction method and system between three-dimensional magnetic resonance planes

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106204449A (en) * 2016-07-06 2016-12-07 安徽工业大学 A kind of single image super resolution ratio reconstruction method based on symmetrical degree of depth network
CN106683067A (en) * 2017-01-20 2017-05-17 福建帝视信息科技有限公司 Deep learning super-resolution reconstruction method based on residual sub-images
US20170200067A1 (en) * 2016-01-08 2017-07-13 Siemens Healthcare Gmbh Deep Image-to-Image Network Learning for Medical Image Analysis
CN106952229A (en) * 2017-03-15 2017-07-14 桂林电子科技大学 Image super-resolution rebuilding method based on the enhanced modified convolutional network of data

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170200067A1 (en) * 2016-01-08 2017-07-13 Siemens Healthcare Gmbh Deep Image-to-Image Network Learning for Medical Image Analysis
CN106204449A (en) * 2016-07-06 2016-12-07 安徽工业大学 A kind of single image super resolution ratio reconstruction method based on symmetrical degree of depth network
CN106683067A (en) * 2017-01-20 2017-05-17 福建帝视信息科技有限公司 Deep learning super-resolution reconstruction method based on residual sub-images
CN106952229A (en) * 2017-03-15 2017-07-14 桂林电子科技大学 Image super-resolution rebuilding method based on the enhanced modified convolutional network of data

Cited By (77)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108335303A (en) * 2018-01-28 2018-07-27 浙江大学 A kind of multiple dimensioned palm bone segmentation method applied to palm X-ray
CN108447062B (en) * 2018-02-01 2021-04-20 浙江大学 Pathological section unconventional cell segmentation method based on multi-scale mixed segmentation model
CN108447062A (en) * 2018-02-01 2018-08-24 浙江大学 A kind of dividing method of the unconventional cell of pathological section based on multiple dimensioned mixing parted pattern
CN108416821A (en) * 2018-03-08 2018-08-17 山东财经大学 A kind of CT Image Super-resolution Reconstruction methods of deep neural network
CN108492271A (en) * 2018-03-26 2018-09-04 中国电子科技集团公司第三十八研究所 A kind of automated graphics enhancing system and method for fusion multi-scale information
CN108492271B (en) * 2018-03-26 2021-08-24 中国电子科技集团公司第三十八研究所 Automatic image enhancement system and method fusing multi-scale information
CN108828481A (en) * 2018-04-24 2018-11-16 朱高杰 A kind of magnetic resonance reconstruction method based on deep learning and data consistency
CN108596994B (en) * 2018-04-24 2022-05-03 朱高杰 Magnetic resonance diffusion weighted imaging method based on deep learning and data self-consistency
CN108596994A (en) * 2018-04-24 2018-09-28 朱高杰 A kind of Diffusion-weighted imaging method being in harmony certainly based on deep learning and data
CN108594146A (en) * 2018-04-24 2018-09-28 朱高杰 A kind of Diffusion-weighted imaging method based on deep learning and convex set projection
CN108594146B (en) * 2018-04-24 2020-07-28 朱高杰 Magnetic resonance diffusion weighted imaging method based on deep learning and convex set projection
CN108828481B (en) * 2018-04-24 2021-01-22 朱高杰 Magnetic resonance reconstruction method based on deep learning and data consistency
CN108898222A (en) * 2018-06-26 2018-11-27 郑州云海信息技术有限公司 A kind of method and apparatus automatically adjusting network model hyper parameter
CN109035356B (en) * 2018-07-05 2020-07-10 四川大学 System and method based on PET (positron emission tomography) graphic imaging
CN109035356A (en) * 2018-07-05 2018-12-18 四川大学 A kind of system and method based on PET pattern imaging
CN109345449A (en) * 2018-07-17 2019-02-15 西安交通大学 A kind of image super-resolution based on converged network and remove non-homogeneous blur method
US11928792B2 (en) 2018-07-17 2024-03-12 Xi'an Jiaotong University Fusion network-based method for image super-resolution and non-uniform motion deblurring
CN109345449B (en) * 2018-07-17 2020-11-10 西安交通大学 Image super-resolution and non-uniform blur removing method based on fusion network
CN109087273B (en) * 2018-07-20 2021-09-14 哈尔滨工业大学（深圳） Image restoration method, storage medium and system based on enhanced neural network
CN109087273A (en) * 2018-07-20 2018-12-25 哈尔滨工业大学（深圳） Image recovery method, storage medium and the system of neural network based on enhancing
CN109146784A (en) * 2018-07-27 2019-01-04 徐州工程学院 A kind of image super-resolution rebuilding method based on multiple dimensioned generation confrontation network
CN109146784B (en) * 2018-07-27 2020-11-20 徐州工程学院 Image super-resolution reconstruction method based on multi-scale generation countermeasure network
CN109003229A (en) * 2018-08-09 2018-12-14 成都大学 Magnetic resonance super resolution ratio reconstruction method based on three-dimensional enhancing depth residual error network
CN109003229B (en) * 2018-08-09 2022-12-13 成都大学 Magnetic resonance super-resolution reconstruction method based on three-dimensional enhanced depth residual error network
CN109447238A (en) * 2018-09-21 2019-03-08 广东石油化工学院 Multi-output regression depth network establishing method, structure, equipment and storage medium
CN109272443A (en) * 2018-09-30 2019-01-25 东北大学 A kind of PET based on full convolutional neural networks and CT method for registering images
CN112789659A (en) * 2018-10-02 2021-05-11 皇家飞利浦有限公司 Generation of pseudo-radiographic image from optical image
CN109544488B (en) * 2018-10-08 2021-06-01 西北大学 Image synthesis method based on convolutional neural network
CN109544488A (en) * 2018-10-08 2019-03-29 西北大学 A kind of image composition method based on convolutional neural networks
CN109410240A (en) * 2018-10-09 2019-03-01 电子科技大学中山学院 Method and device for positioning volume characteristic points and storage medium thereof
CN109685717A (en) * 2018-12-14 2019-04-26 厦门理工学院 Image super-resolution rebuilding method, device and electronic equipment
CN111353944B (en) * 2018-12-20 2024-05-28 深圳市中兴微电子技术有限公司 Image reconstruction method, device and computer readable storage medium
CN111353944A (en) * 2018-12-20 2020-06-30 深圳市中兴微电子技术有限公司 Image reconstruction method and device and computer readable storage medium
US11494877B2 (en) 2018-12-26 2022-11-08 Shanghai United Imaging Intelligence Co., Ltd. Systems and methods for image reconstruction
WO2020135630A1 (en) * 2018-12-26 2020-07-02 Shanghai United Imaging Intelligence Co., Ltd. Systems and methods for image reconstruction
CN109741416A (en) * 2019-01-04 2019-05-10 北京大学深圳医院 Nuclear magnetic resonance image method for reconstructing, device, computer equipment and its storage medium
CN110598724B (en) * 2019-01-17 2022-09-23 西安理工大学 Cell low-resolution image fusion method based on convolutional neural network
CN110598724A (en) * 2019-01-17 2019-12-20 西安理工大学 Cell low-resolution image fusion method based on convolutional neural network
CN109919840A (en) * 2019-01-21 2019-06-21 南京航空航天大学 Image super-resolution rebuilding method based on dense feature converged network
CN109859106A (en) * 2019-01-28 2019-06-07 桂林电子科技大学 A kind of image super-resolution rebuilding method based on the high-order converged network from attention
CN109903229A (en) * 2019-03-04 2019-06-18 科新(杭州)能源环境科技有限公司 A kind of μ-CT image reconstructing method based on convolutional neural networks
CN110276736B (en) * 2019-04-01 2021-01-19 厦门大学 Magnetic resonance image fusion method based on weight prediction network
CN110276736A (en) * 2019-04-01 2019-09-24 厦门大学 A kind of magnetic resonance image fusion method based on weight prediction network
CN111956180A (en) * 2019-05-20 2020-11-20 华北电力大学（保定） Method for reconstructing photoacoustic endoscopic tomography image
CN111956180B (en) * 2019-05-20 2023-06-27 华北电力大学（保定） Method for reconstructing photoacoustic endoscopic tomographic image
CN110333076A (en) * 2019-06-19 2019-10-15 电子科技大学 Method for Bearing Fault Diagnosis based on CNN-Stacking
CN110333076B (en) * 2019-06-19 2021-01-26 电子科技大学 Bearing fault diagnosis method based on CNN-Stacking
CN112419437A (en) * 2019-11-29 2021-02-26 上海联影智能医疗科技有限公司 System and method for reconstructing magnetic resonance images
CN110940944B (en) * 2019-12-04 2020-11-10 厦门大学 J coupling removing method for magnetic resonance signals based on deep learning
CN110940944A (en) * 2019-12-04 2020-03-31 厦门大学 J coupling removing method for magnetic resonance signals based on deep learning
CN110929521A (en) * 2019-12-06 2020-03-27 北京知道智慧信息技术有限公司 Model generation method, entity identification method, device and storage medium
CN110929521B (en) * 2019-12-06 2023-10-27 北京知道创宇信息技术股份有限公司 Model generation method, entity identification method, device and storage medium
CN111160413B (en) * 2019-12-12 2023-11-17 天津大学 Thyroid nodule classification method based on multi-scale feature fusion
CN111160413A (en) * 2019-12-12 2020-05-15 天津大学 Thyroid nodule classification method based on multi-scale feature fusion
CN111274865A (en) * 2019-12-14 2020-06-12 深圳先进技术研究院 Remote sensing image cloud detection method and device based on full convolution neural network
CN111274865B (en) * 2019-12-14 2023-09-19 深圳先进技术研究院 Remote sensing image cloud detection method and device based on full convolution neural network
CN111161182B (en) * 2019-12-27 2021-03-09 南方医科大学 MR structure information constrained non-local mean guided PET image partial volume correction method
CN111161182A (en) * 2019-12-27 2020-05-15 南方医科大学 MR structure information constrained non-local mean guided PET image partial volume correction method
CN111127581A (en) * 2019-12-31 2020-05-08 东软医疗系统股份有限公司 Image reconstruction method and device, CT (computed tomography) equipment and CT system
WO2021135474A1 (en) * 2020-01-02 2021-07-08 平安科技（深圳）有限公司 Method and apparatus for fusing data from multiple data sources, electronic device, and storage medium
CN111275618B (en) * 2020-01-12 2023-09-29 杭州电子科技大学 Depth map super-resolution reconstruction network construction method based on double-branch perception
CN111275618A (en) * 2020-01-12 2020-06-12 杭州电子科技大学 Depth map super-resolution reconstruction network construction method based on double-branch perception
CN111292240A (en) * 2020-01-23 2020-06-16 上海交通大学 Magnetic resonance super-resolution imaging method based on imaging model and machine learning
CN111292240B (en) * 2020-01-23 2022-01-07 上海交通大学 Magnetic resonance super-resolution imaging method based on imaging model and machine learning
CN113534031A (en) * 2020-04-21 2021-10-22 上海联影医疗科技股份有限公司 Image domain data generating method, computer device and readable storage medium
CN111798377B (en) * 2020-07-08 2023-07-28 广东工业大学 Magnetic resonance image super-resolution reconstruction method based on multi-resolution learning strategy
CN111798377A (en) * 2020-07-08 2020-10-20 广东工业大学 Magnetic resonance image super-resolution reconstruction method based on multi-resolution learning strategy
CN113935928B (en) * 2020-07-13 2023-04-11 四川大学 Rock core image super-resolution reconstruction based on Raw format
CN113935928A (en) * 2020-07-13 2022-01-14 四川大学 Rock core image super-resolution reconstruction based on Raw format
CN112102388B (en) * 2020-09-18 2024-03-26 中国矿业大学 Method and device for obtaining depth image based on inspection robot monocular image
CN112102388A (en) * 2020-09-18 2020-12-18 中国矿业大学 Method and device for acquiring depth image based on monocular image of inspection robot
CN112767374A (en) * 2021-01-27 2021-05-07 天津理工大学 Alzheimer disease focus region semantic segmentation algorithm based on MRI
WO2022206021A1 (en) * 2021-03-30 2022-10-06 中国科学院深圳先进技术研究院 Image reconstruction model generation method and apparatus, image reconstruction method and apparatus, and device and medium
CN112949636B (en) * 2021-03-31 2023-05-30 上海电机学院 License plate super-resolution recognition method, system and computer readable medium
CN112949636A (en) * 2021-03-31 2021-06-11 上海电机学院 License plate super-resolution identification method and system and computer readable medium
CN116805284A (en) * 2023-08-28 2023-09-26 之江实验室 Feature migration-based super-resolution reconstruction method and system between three-dimensional magnetic resonance planes
CN116805284B (en) * 2023-08-28 2023-12-19 之江实验室 Feature migration-based super-resolution reconstruction method and system between three-dimensional magnetic resonance planes

Also Published As

Publication number Publication date
CN107610194B (en) 2020-08-04

Similar Documents

Publication Publication Date Title
CN107610194A (en) MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN
Hu et al. Medical image reconstruction using generative adversarial network for Alzheimer disease assessment with class-imbalance problem
CN108460726B (en) Magnetic resonance image super-resolution reconstruction method based on enhanced recursive residual network
CN109727195B (en) Image super-resolution reconstruction method
CN110276736B (en) Magnetic resonance image fusion method based on weight prediction network
CN103295198B (en) Based on redundant dictionary and the sparse non-convex compressed sensing image reconstructing method of structure
Du et al. Accelerated super-resolution MR image reconstruction via a 3D densely connected deep convolutional neural network
CN110189253A (en) A kind of image super-resolution rebuilding method generating confrontation network based on improvement
CN108416821B (en) A kind of CT Image Super-resolution Reconstruction method of deep neural network
CN106204449A (en) A kind of single image super resolution ratio reconstruction method based on symmetrical degree of depth network
CN106683067A (en) Deep learning super-resolution reconstruction method based on residual sub-images
CN111583384B (en) Hair reconstruction method based on self-adaptive octree hair convolution neural network
CN108259994A (en) A kind of method for improving video spatial resolution
CN109035146A (en) A kind of low-quality image oversubscription method based on deep learning
CN107845065A (en) Super-resolution image reconstruction method and device
CN111899165A (en) Multi-task image reconstruction convolution network model based on functional module
CN112802046B (en) Image generation system for generating pseudo CT from multi-sequence MR based on deep learning
CN110110808A (en) A kind of pair of image carries out the method, apparatus and computer readable medium of target mark
CN107563434A (en) A kind of brain MRI image sorting technique based on Three dimensional convolution neutral net, device
CN109584164A (en) Medical image super-resolution three-dimensional rebuilding method based on bidimensional image transfer learning
CN109598676A (en) A kind of single image super-resolution method based on Hadamard transform
CN107220971A (en) A kind of Lung neoplasm feature extracting method based on convolutional neural networks and PCA
CN112489050A (en) Semi-supervised instance segmentation algorithm based on feature migration
CN110503699A (en) A kind of CT projection path reduce in the case of CT image rebuilding method
CN112102317A (en) Multi-phase liver lesion detection method and system based on anchor-frame-free

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant