WO2022206021A1

WO2022206021A1 - Image reconstruction model generation method and apparatus, image reconstruction method and apparatus, and device and medium

Info

Publication number: WO2022206021A1
Application number: PCT/CN2021/137623
Authority: WO
Inventors: 郭木子; 郑海荣; 朱燕杰; 梁栋; 刘新
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2021-03-30
Filing date: 2021-12-13
Publication date: 2022-10-06
Also published as: CN115147502A

Abstract

Disclosed in the embodiments of the present invention are an image reconstruction model generation method and apparatus, an image reconstruction method and apparatus, and a device and a medium. The image reconstruction model generation method comprises: acquiring a preliminary reconstructed magnetic resonance image, and reducing the resolution of the preliminary reconstructed magnetic resonance image according to a preset image resolution increase multiple, so as to obtain a low-resolution image; performing image interpolation processing on the low-resolution image, a multiple for image interpolation being the preset image resolution increase multiple; and training a high-resolution image reconstruction model by using the image having been subjected to interpolation processing as input data for model training and using the preliminary reconstructed magnetic resonance image as label data, and when a loss function of the high-resolution image reconstruction model converges to a preset value, generating a target high-resolution image reconstruction model. By means of the technical solution of the present embodiment, hidden issues such as a poor learning effect caused by data deviation are eliminated, and the resolution of a reconstructed image can be increased while shortening the imaging time, thereby achieving a better image effect.

Description

Image reconstruction model generation and image reconstruction method, device, device and medium

technical field

Embodiments of the present invention relate to the technical field of medical image processing, and in particular, to an image reconstruction model generation and image reconstruction method, apparatus, device, and medium.

Background technique

Magnetic Resonance Imaging (MRI) technology is widely used in clinical diagnosis and medical research due to its non-invasive, non-radiation, good soft tissue contrast and imaging at any level. At this stage, cardiac magnetic resonance cine has been regarded as the imaging gold standard for assessing cardiac function. However, when high-resolution image acquisition is performed clinically, a long acquisition time is often required. At the same time, it is often difficult to obtain high-resolution cardiac magnetic resonance images in the actual clinical process due to the influence of patient tolerance and respiratory movement.

At present, the methods for obtaining high-resolution magnetic resonance cardiac cine images mainly include image reconstruction methods based on interpolation, image reconstruction methods based on traditional machine learning, and image reconstruction methods based on deep learning. Among them, the method based on the deep learning network has good processing ability for linear and nonlinear methods, and can reconstruct the magnetic resonance image with higher image quality.

However, the existing deep learning-based methods for super-resolution image reconstruction have not solved their dependence on data, and still require a large number of paired low-resolution images and high-resolution images for learning. However, paired high-resolution and low-resolution cine images are difficult to obtain from practical and patient ethics perspectives, as well as the specificity of medical images. At the same time, for film imaging with different resolutions, different field strengths and different phases, standardized registration is required, and this process is often inaccurate due to the influence of machine and geometric deformation. This makes the existing deep learning-based super-resolution methods susceptible to data bias, which increases the difficulty of large-scale clinical promotion.

SUMMARY OF THE INVENTION

The embodiments of the present invention provide an image reconstruction model generation and image reconstruction method, device, device and medium, so as to shorten the imaging time and reduce the complexity of the image reconstruction network, and at the same time improve the reconstructed image resolution. better image effects.

In a first aspect, an embodiment of the present invention provides a method for generating an image reconstruction model, the method comprising:

Acquiring a preliminary magnetic resonance reconstruction image, increasing the resolution of the preliminary magnetic resonance image by a multiple, and reducing the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image;

Perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;

The image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained. When the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.

Optionally, a preliminary magnetic resonance reconstruction image is obtained, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a preset ratio to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image, including:

Input the preliminary magnetic resonance reconstruction image to a generator of a preset generative adversarial network, and the generator performs convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain the resolution reduction of the predetermined image resolution. Construct low-resolution images with increased rate;

A discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;

Perform layer-by-layer convolution on all convolutional layer parameters of the generator in the trained preset generative adversarial network to obtain a magnetic resonance image degradation kernel function;

Convolving the preliminary magnetic resonance reconstruction image and the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.

Preferably, the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.

Optionally, performing image interpolation processing on the low-resolution image includes:

Perform bicubic interpolation processing on the low-resolution image by the preset image resolution enhancement factor.

Optionally, before training the high-resolution image reconstruction model, the method further includes:

The image after image interpolation processing and the preliminary magnetic resonance reconstruction image are rotated or mirrored synchronously, and the image pair obtained after the rotation or mirror operation is used as the training sample data of the new high-resolution image reconstruction model.

Optionally, in the training process of the high-resolution image reconstruction model, a residual learning method is used to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the sum of the obtained data. The loss function between the described label data.

In a second aspect, an embodiment of the present invention further provides an image reconstruction method, the method comprising:

Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase;

Inputting the preliminary magnetic resonance reconstruction image that has undergone image interpolation processing into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.

In a third aspect, an embodiment of the present invention further provides a device for generating an image reconstruction model, the device comprising:

an image degradation module, configured to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a multiple according to a preset image resolution, reduce the resolution of the preliminary magnetic resonance reconstruction image, and obtain a corresponding low-resolution image;

an image interpolation module, configured to perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;

The model training module is used to use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model. When the loss function of the high-resolution image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.

Optionally, the image degradation module is specifically used for:

Optionally, the image interpolation module is specifically used for:

Optionally, the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.

Optionally, the model training module is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to combine the residual of the input data after passing through the convolution layer with the input. After the data itself is added, a loss function between the data and the label data is calculated.

In a fourth aspect, an embodiment of the present invention further provides an image reconstruction device, the device comprising:

an image preprocessing module, configured to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple;

The image reconstruction module is used for inputting the preliminary magnetic resonance reconstruction image subjected to image interpolation processing to the preset resolution enhancement factor obtained by the image reconstruction model generation method described in any one of the embodiments. In the target image reconstruction model of , the target magnetic resonance image reconstruction image is obtained.

In a fifth aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

one or more processors;

memory for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or the image reconstruction method provided by any embodiment of the present invention.

In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method or image provided by any embodiment of the present invention rebuild method.

The embodiments in the above invention have the following advantages or beneficial effects:

In this embodiment of the present invention, a preliminary magnetic resonance reconstruction image is acquired, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a multiple according to a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image, so as to obtain a corresponding low-resolution image; Perform image interpolation processing, wherein the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used. As the label data, a high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated. The problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for A large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.

Description of drawings

1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention;

2 is a schematic structural diagram of a generative adversarial network according to Embodiment 1 of the present invention;

3 is a schematic diagram of a network structure of a discriminator in a generative adversarial network provided by Embodiment 1 of the present invention;

4 is a schematic diagram of a generator network structure in a generative adversarial network according to Embodiment 1 of the present invention;

5 is a schematic diagram of a network training process of an image reconstruction model according to Embodiment 1 of the present invention;

6 is a flowchart of an image reconstruction method according to Embodiment 2 of the present invention;

7 is a schematic diagram of an image reconstruction process according to Embodiment 2 of the present invention;

8 is a comparison diagram of the effect of magnetic resonance image reconstruction performed by different image reconstruction methods according to Embodiment 2 of the present invention;

9 is a schematic structural diagram of an apparatus for generating an image reconstruction model according to Embodiment 3 of the present invention;

10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention;

FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

Example 1

FIG. 1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training. The method may be executed by an image reconstruction model generating apparatus, which may be implemented in software and/or hardware, and integrated into an electronic device with an application development function.

As shown in Figure 1, the image reconstruction model generation method includes the following steps:

S110. Acquire a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image.

The preliminary magnetic resonance reconstruction image is a magnetic resonance image that can be scanned and obtained by a current magnetic resonance imaging device, that is, an image with relatively low resolution and has not been reconstructed with improved resolution (super-resolution reconstruction). The preset image resolution enhancement multiple is a multiple that is expected to be able to improve the resolution of the preliminary magnetic resonance reconstruction image. Exemplarily, if the resolution of the preliminary magnetic resonance reconstruction image is 128*128, it is hoped that the preliminary magnetic resonance reconstruction image can be reconstructed to obtain a high-resolution image with a resolution of 512*512; then, the preset image resolution The enhancement factor can be obtained by dividing (512*512) by (128*128). The preset image resolution enhancement factor is 16.

In this embodiment, the resolution of the preliminary magnetic resonance reconstructed image is reduced according to the preset image resolution increase multiple to obtain a corresponding low-resolution image, and the purpose is to use the preliminary magnetic resonance reconstruction image itself as the training parameter of the image reconstruction network, Instead of matching the initial MR reconstruction image with the corresponding high-resolution high-quality MR reconstruction image. Therefore, there is no need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data to train the neural network, which can reduce the difficulty of obtaining training samples for image reconstruction models.

Specifically, the way to reduce the image resolution can be a down-sampling method to perform down-sampling processing on the preliminary magnetic resonance reconstructed image to obtain a corresponding low-resolution image.

In a preferred embodiment, a generative adversarial network can also be used to degrade the preliminary magnetic resonance image reconstruction image to obtain a low-resolution image whose resolution is reduced by a preset image resolution enhancement factor. During training of the generative adversarial network, a kernel function can be determined for performing the same image degradation process on one or more preliminary MRI reconstruction images. Specifically, first, the preliminary magnetic resonance reconstruction image is input into the preset generative adversarial network as shown in FIG. 2 . First, the generator G performs convolution and down-sampling processing on the preliminary magnetic resonance reconstruction image, and obtains a low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple. The obtained image can be regarded as the original magnetic resonance reconstruction image. Fake version image F; further, cut out image blocks with the same size and matching position as image F from the preliminary magnetic resonance reconstruction image, as real sample image T. After the image T and the image F are input to the discriminator D together, according to the image T, the discriminator analyzes the possibility that the image F is a real image pixel by pixel. The goal of the generator is to make the output image F fool the discriminator as much as possible. The two networks, the generator and the discriminator, constantly adjust the parameters in the process of confrontation with each other. Finally, when the discriminator cannot judge whether the output result F of the generator is real, the network training is completed. In the heat map output by the discriminator, the image F and the image T are compared pixel by pixel, and every two pixels compared with each other are the probability value of the same pixel point, when the probability value of the year in the heat map is satisfied. When the probability value of the judgment is affirmative, it can be determined that the discriminator cannot judge whether the output result F of the generator is true. Further, all convolutional layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function; thus, each preliminary magnetic resonance reconstruction image can be respectively combined with the magnetic resonance image. The image degradation kernel function performs convolution to obtain the corresponding low-resolution image. In this embodiment, the size and number of convolution kernels in the generator in the preset generative adversarial network can be changed and adjusted. The generator includes six convolution kernels, and the size of each convolution kernel is: 5x5, 3x3, 1x1, 1x1, 1x1 and 1x1. In a preferred embodiment, the structure of the discriminator in the preset Generative Adversarial Network may be the structure shown in FIG. 3 , which consists of 7 convolutional layers, and the convolution kernel sizes of each convolutional layer are: 7×7 and 1×1 respectively. , 1x1, 1x1, 1x1, 1x1 and 1x1. The network structure of the generator is shown in Figure 4, which consists of 6 convolution layers, and the convolution kernel sizes are: 7x7, 5x5, 3x3, 1x1, 1x1, 1x1. Among them, the value of the convolution step size of the last convolutional layer of the generator is the preset image resolution enhancement multiple, so as to achieve the effect of downsampling the input image.

It should be noted here that, if the magnetic resonance imaging object is the heart, the acquired preliminary magnetic resonance reconstruction image is a cardiac magnetic resonance cine image, which includes multiple frames of magnetic resonance images. During the training of the generative adversarial network, any frame of the cardiac magnetic resonance cine image can be taken as the input image.

S120. Perform image interpolation processing on the low-resolution image, where the multiple of image interpolation is the preset image resolution enhancement multiple.

In this step, image interpolation processing is performed on the low-resolution image, and the multiple of image interpolation is the preset image resolution enhancement multiple, so as to keep the size of the input image of the image reconstruction model consistent with the size of the output image, which can be shortened. Model training time, optimizing the model training process.

Specifically, in this embodiment, the interpolation method may adopt a bicubic interpolation (Bicubic) algorithm. This is because the bicubic interpolation can preserve more image details during the image enlargement process, and the enlarged image has the function of anti-aliasing. At the same time, the enlarged image has a more realistic effect than the source image.

In a preferred embodiment, the image that has undergone image interpolation processing and the corresponding preliminary magnetic resonance reconstruction image can also be rotated or mirrored synchronously, and the image pair obtained after the rotation or mirroring operation can be used as a new high-resolution image. Rate image reconstruction model training sample data. For example, rotate the image pair at 0, 90, 180, and 270 degrees, and then perform mirror symmetry operations in the horizontal and vertical directions, respectively, to obtain 8 sets of training data, so that the training data can be obtained. enhanced. Moreover, for the synchronized image pairs, the phases of the data are consistent, and no registration is required, thereby eliminating hidden dangers such as poor learning effects caused by data deviations.

S130. Use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model. When the loss function of the image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.

After the above steps, a sufficient amount of model training sample data is constructed, and then the model training process can begin. The image that has undergone image interpolation processing is used as the input data for model training, and the preliminary MRI reconstruction image corresponding to the low-resolution image is used as the label data. The to-be-obtained preliminary magnetic resonance image is used as the label data, and the low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple. Then, after the preliminary magnetic resonance reconstruction image is input into the trained image reconstruction model, a high-resolution image that increases the resolution of the preliminary magnetic resonance reconstruction image by a preset image resolution can be obtained from the output of the image reconstruction model.

Specifically, for the training process of the target image reconstruction model, reference may be made to the process shown in FIG. 5 . In the process shown in Figure 5, the image reconstruction network is a fully convolutional network with a total of 8 convolutional layers. The convolution kernel size of the first layer is 3x3, the number of channels is f, and the second to seventh layers are The size of the convolution kernel is 3x3, the number of channels is 64, and the size of the convolution kernel of the last layer is 3x3, and the number of channels is f. where f represents the number of image frames simultaneously input to the image reconstruction network. For an image, the f value is 1. When the acquired preliminary MRI reconstructed image is a cardiac cine image, the value of f is the number of image frames in the cardiac cine image. The number of convolutional layers of the image reconstruction network can also be changed to 6 or more, and the size of the convolution kernel is not limited to 3x3, and the parameters can be adjusted according to the calculation requirements.

Further, the model training process in Fig. 5 adopts a preferred deep learning method, that is, the residual learning method, which compares the residual of the image data of the input model after passing through the convolution layer with the input image data itself. After adding, calculate the loss function between and the label data. It should be noted here that the loss function may not only use the L1 loss function shown in FIG. 5 , but also use loss functions such as L2 loss and perceptual loss. Further, optimization algorithms such as Adam optimization algorithm, stochastic gradient descent algorithm or AdaGrad can also be used to optimize the network learning process.

It should be noted here that in the structure diagram of each network, conv represents the convolution kernel in the convolutional neural network, and ".mat" is the abbreviation of the file format. All the parameters of the convolutional layers of the device are convolved layer by layer to obtain the degraded kernel function of the magnetic resonance image.

In the technical solution of this embodiment, a corresponding low-resolution image is obtained by reducing the image resolution of the obtained preliminary magnetic resonance reconstruction image, and the resolution reduction factor is controllable; then, image interpolation processing is performed on the low-resolution image , where the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, The high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated. The problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for A large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.

Embodiment 2

Fig. 6 is a flowchart of an image reconstruction method provided in Embodiment 2 of the present invention, and this embodiment can be applied to the situation of reconstructing a collected low-resolution medical image to obtain a high-resolution image. The method may be performed by an image reconstruction apparatus, and the apparatus may be implemented in software and/or hardware, and integrated into a computer device with an application development function.

As shown in Figure 6, the image reconstruction method includes the following steps:

S210. Acquire a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image by a preset resolution increase multiple.

The preliminary magnetic resonance reconstruction image is a low-resolution image whose resolution needs to be improved. According to the resolution improvement requirement, the preliminary magnetic resonance reconstruction image can be interpolated and reconstructed to obtain a preprocessed image. The multiple of image interpolation is the preset resolution enhancement multiple.

S220. Input the preliminary magnetic resonance reconstruction image that has undergone image interpolation processing into the target image reconstruction obtained by the image reconstruction model generation method according to any one of the embodiments, and the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor In the model, the target magnetic resonance reconstruction image is obtained.

The target image reconstruction model is an image reconstruction model trained by the image reconstruction model generation method in the above-mentioned embodiment according to the requirement of increasing the preset resolution. Specifically, for the process of image reconstruction, reference may be made to the schematic diagram shown in FIG. 7 .

Further, Fig. 8 shows the reconstructed image obtained by performing image reconstruction on the same preliminary magnetic resonance reconstructed image by different graphic reconstruction methods, wherein (a) NN is the result obtained by the nearest neighbor interpolation algorithm, (b) Bicubic is the double The result obtained by the cubic interpolation algorithm, (c) zero-padding is the result of transforming the image back to the image domain after the image is transformed to the Fourier domain, and then inversely transforming back to the image domain after zero-filling the surrounding area, (d) SR (Super Resolution) is the image of this example The results obtained by the reconstruction method. Intuitively, from the results, the image reconstruction results obtained by the image reconstruction method in this embodiment are clearer.

In the technical solution of this embodiment, by inputting a low-resolution image into a pre-trained image reconstruction model that can increase the image resolution by a preset multiple, a reconstructed high-resolution image is obtained; Due to the problems of long image reconstruction time and slow imaging speed, it is possible to obtain high-resolution images with clear edges in a relatively short period of time without a lot of computation through a relatively simple image reconstruction network.

Embodiment 3

FIG. 9 is a schematic structural diagram of an image reconstruction model generating apparatus according to Embodiment 3 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training.

As shown in FIG. 9 , the image reconstruction model generating apparatus includes an image degradation module 310 , an image interpolation module 320 and a model training module 330 .

Wherein, the image degradation module 310 is used to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image; the image interpolation module 320, for performing image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple; model training module 330, for using the image subjected to image interpolation processing as model training The input data of the high-resolution image reconstruction model is trained by using the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as the label data. When the loss function of the high-resolution image reconstruction model converges to the preset value , to generate the target high-resolution image reconstruction model.

In the technical solution of this embodiment, by acquiring a preliminary magnetic resonance reconstruction image, and increasing the resolution of the preliminary magnetic resonance image by a multiple, the resolution of the preliminary magnetic resonance reconstruction image is reduced, and a corresponding low-resolution image is obtained; Perform image interpolation processing on the low-resolution images, wherein the multiple of image interpolation is the preset image resolution improvement multiple; the image after image interpolation processing is used as the input data for model training, and the preliminary magnetic resonance imaging corresponding to the low-resolution image is used. The reconstructed image is used as label data to train a high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated. The problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for A large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.

Optionally, the image degradation module 310 is specifically used for:

Optionally, the image interpolation module 320 is specifically used for:

Optionally, the model training module 330 is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to compare the residual of the input data after passing through the convolutional layer with the residual. After the input data itself is added, a loss function between the input data and the label data is calculated.

The image reconstruction model generation apparatus provided by the embodiment of the present invention can execute the image reconstruction model generation method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

Embodiment 4

FIG. 10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention. This embodiment can be applied to a situation in which a low-resolution medical image obtained by acquisition is reconstructed to obtain a high-resolution image.

As shown in FIG. 10 , the image reconstruction apparatus includes an image preprocessing module 410 and an image reconstruction module 420 .

Among them, the image preprocessing module 410 is used to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple; the image reconstruction module 420 is used for image interpolation processing. The preliminary magnetic resonance reconstruction image is input into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, and the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor, and the target magnetic resonance image is obtained. Rebuild the image.

The image reconstruction apparatus provided by the embodiment of the present invention can execute the image reconstruction method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

Embodiment 5

FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention. Figure 11 shows a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in FIG. 11 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present invention. The computer device 12 may be any terminal device with computing capability connected to the magnetic resonance scanning imaging device, such as an intelligent controller, a server, a mobile phone and other terminal devices.

As shown in FIG. 11, computer device 12 takes the form of a general-purpose computing device. Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16 , system memory 28 , and a bus 18 connecting various system components including system memory 28 and processing unit 16 .

Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.

Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including both volatile and nonvolatile media, removable and non-removable media.

System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32 . Computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. For example only, storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 11, commonly referred to as a "hard drive"). Although not shown in Figure 11, a disk drive for reading and writing to removable non-volatile magnetic disks (eg "floppy disks") and removable non-volatile optical disks (eg CD-ROM, DVD-ROM) may be provided or other optical media) to read and write optical drives. In these cases, each drive may be connected to bus 18 through one or more data media interfaces. System memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present invention.

A program/utility 40 having a set (at least one) of program modules 42, which may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and programs Data, each or some combination of these examples may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the described embodiments of the present invention.

Computer device 12 may also communicate with one or more external devices 14 (eg, keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with computer device 12, and/or communicate with Any device (eg, network card, modem, etc.) that enables the computer device 12 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 22 . Also, computer device 12 may communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet, through network adapters 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18 . It should be understood that, although not shown in FIG. 11, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tapes drives and data backup storage systems, etc.

The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, for example, to realize the steps of an image reconstruction model generation method provided by the embodiment of the present invention, and the method includes:

Alternatively, the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:

Embodiment 6

The sixth embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method provided by any embodiment of the present invention, including:

The computer storage medium in the embodiments of the present invention may adopt any combination of one or more computer-readable mediums. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. More specific examples (a non-exhaustive list) of computer readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .

Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional A procedural programming language, such as the "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (eg, through the Internet using an Internet service provider) connect).

Those of ordinary skill in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device, and they can be centralized on a single computing device, or distributed on a network composed of multiple computing devices. Optionally, they may be implemented in program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be fabricated separately into individual integrated circuit modules, or a plurality of modules of them Or the steps are made into a single integrated circuit module to realize. As such, the present invention is not limited to any specific combination of hardware and software.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made to those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.

Claims

A method for generating an image reconstruction model, comprising:

Acquiring a preliminary magnetic resonance reconstruction image, increasing the resolution of the preliminary magnetic resonance image by a multiple, and reducing the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image;

Perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;

The image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained. When the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
The method according to claim 1, wherein a preliminary magnetic resonance reconstruction image is obtained, and a resolution of the preliminary magnetic resonance reconstruction image is reduced according to a preset image resolution increase multiple to obtain a corresponding low-resolution image, include:

Input the preliminary magnetic resonance reconstruction image to a generator of a preset generative adversarial network, and the generator performs convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain the resolution reduction of the predetermined image resolution. Construct low-resolution images with increased rate;

A discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;

Perform layer-by-layer convolution on all convolutional layer parameters of the generator in the trained preset generative adversarial network to obtain a magnetic resonance image degradation kernel function;

Convolving the preliminary magnetic resonance reconstruction image and the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.
The method according to claim 2, wherein the generator of the preset generative adversarial network comprises six convolution layers, and the convolution step size of the last layer of the six convolution layers is the The preset image resolution is increased by a multiple; the discriminator of the preset generative adversarial network includes seven convolution layers.
The method according to any one of claims 1-3, wherein, performing image interpolation processing on the low-resolution image, comprising:

Perform bicubic interpolation processing on the low-resolution image by the preset image resolution enhancement factor.
The method according to any one of claims 1-3, wherein, before the high-resolution image reconstruction model is trained, the method further comprises:

The image after image interpolation processing and the preliminary magnetic resonance reconstruction image are rotated or mirrored synchronously, and the image pair obtained after the rotation or mirror operation is used as the training sample data of the new high-resolution image reconstruction model.
The method according to any one of claims 1-3, wherein, in the training process of the high-resolution image reconstruction model, a residual learning method is used to pass the input data through the residual after the convolution layer. After the difference is added to the input data itself, a loss function between the difference and the label data is calculated.
An image reconstruction method, comprising:

Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase;

Inputting the preliminary magnetic resonance reconstructed image that has undergone image interpolation processing into the target of increasing the resolution of the magnetic resonance image by the preset resolution enhancement multiple obtained by the image reconstruction model generation method according to any one of claims 1-6 In the image reconstruction model, the target magnetic resonance reconstruction image is obtained.
A device for generating an image reconstruction model, comprising:

an image degradation module, configured to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a multiple according to a preset image resolution, reduce the resolution of the preliminary magnetic resonance reconstruction image, and obtain a corresponding low-resolution image;

an image interpolation module, configured to perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;

The model training module is used to use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model. When the loss function of the high-resolution image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.
An image reconstruction device, comprising:

an image preprocessing module, configured to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple;

The image reconstruction module is used to input the preliminary magnetic resonance reconstruction image subjected to image interpolation processing to the preset magnetic resonance image resolution enhancement obtained by the image reconstruction model generation method according to any one of claims 1-6. In the target image reconstruction model of which the resolution is increased, the target magnetic resonance reconstruction image is obtained.
A computer device, characterized in that the computer device comprises:

one or more processors;

memory for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or the image reconstruction method according to any one of claims 1-7 .
A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the image reconstruction model generation method or the image reconstruction method according to any one of claims 1-7 is implemented.