CN114558251A - A deep learning-based automatic positioning method, device and radiotherapy equipment - Google Patents

Abstract

The invention discloses an automatic positioning method and device based on deep learning and radiotherapy equipment, wherein the automatic positioning method comprises the following steps: s1: fixing a patient to be treated with radiotherapy on a treatment couch, and collecting DR images of corresponding parts; s2: inputting the CT image acquired during planning into a trained U-Net model to obtain a segmentation result of the designated part and reconstructing to generate a DRR image; s3: respectively inputting the DR image obtained in the step S1 and the DRR image obtained in the step S2 into a cycleGAN model to obtain a DR image only containing a specified part; s4: registering the DR image only containing the designated part obtained in the step S3 with the DRR image to be registered to obtain a positioning deviation; s5: and controlling the treatment bed to move according to the swing deviation obtained in the step S4, so as to realize automatic swing. The method effectively improves the quality of the image to be registered, improves the registration precision, reduces the prediction time during positioning in a deep learning mode, finally realizes automatic positioning, avoids manual errors and improves the positioning efficiency.

Description

A deep learning-based automatic positioning method, device and radiotherapy equipment

technical field

The invention belongs to the technical field of radiotherapy, and in particular relates to an automatic positioning method, device and radiotherapy equipment based on deep learning.

Background technique

At present, tumor radiation therapy is generally carried out in stages. Each treatment will repeat the fixation of the patient with the corresponding thermoplastic film, negative pressure bag and other devices with laser light according to the position fixation during positioning CT scan and the reset situation during simulation. . However, due to various reasons, there will still be a certain deviation in the patient's position, and the deviation is mostly between a few millimeters and one centimeter, or even a few centimeters.

In order to realize image-guided radiation therapy, the existing technology mainly includes: selecting the ROI (Region of Interest) region according to the diagnostic CT and treatment plan, and combining the CBCT or other tomographic images scanned during the treatment to realize 3D/3D image registration. Or collect MV/KV images to realize 2D/2D or 2D/3D image registration problem.

In particular, MV image registration is the most difficult problem. The image contrast is low and the modal difference is large, which leads to the failure of the traditional registration algorithm based on image gray value density. Besides, the iterative registration algorithm based on gray value density is usually very time-consuming and has poor user experience. The feature-based registration algorithm is difficult to select and extract features, which requires technicians to perform manually, which is time-consuming and labor-intensive, and is prone to human errors.

SUMMARY OF THE INVENTION

In order to solve the above technical problems, the present invention proposes an automatic positioning method, device and radiotherapy equipment based on deep learning.

In order to achieve the above object, technical scheme of the present invention is as follows:

On the one hand, the present invention discloses an automatic positioning method based on deep learning, comprising the following steps:

S1: Fix the patient to be radiotherapy on the treatment couch, and collect the DR image of the corresponding part;

S2: Input the CT images collected during planning into the trained U-Net model, obtain the segmentation results of the specified parts, and reconstruct and generate DRR images;

S3: Input the DR image obtained by S1 and the DRR image obtained by S2 into the CycleGAN model, respectively, to obtain a DR image containing only the specified part;

S4: Register the DR image obtained in S3 that only contains the specified part and the DRR image to be registered to obtain the placement deviation;

S5: Control the movement of the treatment couch according to the positioning deviation obtained in S4 to realize automatic positioning.

On the basis of the above technical solutions, the following improvements can be made:

As a preferred solution, the training steps of the U-Net model are as follows:

T1: Collect CT images of various parts for clinical use;

T2: Divide CT images into training set, test set and validation set according to the specified proportion;

T3: Select the U-Net model, the input is CT image, the output is the segmentation result of the specified part, and use the data in the training set to train the deep learning model;

T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

As a preferred solution, the following steps are included between T1 and T2: preprocessing the collected CT images.

As a preferred solution, the preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing the pixel size, and scaling the image.

As a preferred solution, in S2, the reconstructed DRR image specifically includes the following content:

Through the DRR reconstruction algorithm, the simulated X-ray light source penetrates the CT voxel, and after attenuation and absorption, it is projected to the detector plane for imaging to obtain a new DRR image.

On the other hand, the present invention discloses an automatic positioning device based on deep learning, comprising:

The DR image acquisition module is used to collect the DR image of the corresponding part after the patient to be radiotherapy is fixed on the treatment couch;

The reconstruction DRR image generation module is used to input the CT images collected during planning into the trained U-Net model, obtain the segmentation results of the specified parts, and reconstruct and generate DRR images;

The DR image generation module is used to input the DR image collected by the DR image acquisition module and the DRR image generated by the reconstructed DRR image generation module into the CycleGAN model respectively to obtain the DR image containing only the specified parts;

The placement deviation generating module is used for registering the DR image generated by the DR image generating module and containing only the specified part with the DRR image to be registered to obtain the placement deviation;

The automatic positioning module is used to control the movement of the treatment couch according to the positioning deviation generated by the positioning deviation generating module to realize automatic positioning.

As a preferred solution, the automatic positioning device also includes: a U-Net model training module, and the U-Net model training module is used to train the U-Net model by adopting the following steps;

The training steps of the U-Net model are as follows:

T1: Collect CT images of various parts for clinical use;

T2: Divide CT images into training set, test set and validation set according to the specified proportion;

T3: Select the U-Net model, the input is CT image, the output is the segmentation result of the specified part, and use the data in the training set to train the deep learning model;

T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

As a preferred solution, the following steps are included between T1 and T2: preprocessing the collected CT images.

As a preferred solution, the preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing the pixel size, and scaling the image.

In addition, on the other hand, the present invention also discloses a radiotherapy device, which uses any of the above-mentioned automatic positioning methods to realize automatic positioning, or includes: any of the above-mentioned automatic positioning devices.

The invention discloses an automatic positioning method, device and radiotherapy equipment based on deep learning. U-Net model and CycleGAN model are used to effectively improve the quality of images to be registered, improve the accuracy of registration, and reduce the amount of time by means of deep learning. The predicted time during placement will eventually realize automatic placement, avoid manual errors and improve placement efficiency.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore do not It should be regarded as a limitation of the scope, and for those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a flowchart of an automatic positioning method provided by an embodiment of the present invention.

FIG. 2 is a flowchart of training a U-Net model provided by an embodiment of the present invention.

FIG. 3 is a schematic diagram of generating a DRR image according to an embodiment of the present invention.

Among them: 1-simulated light source, 2-CT voxel, 3-DRR plane.

Detailed ways

The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The expression "comprising" an element is an "open-ended" expression, which merely refers to the presence of corresponding parts or steps and should not be interpreted as excluding additional parts or steps.

In order to achieve the purpose of the present invention, in some embodiments of the deep learning-based automatic positioning method, device, and radiotherapy equipment, since the features of the spine in the neck, chest and abdomen are obvious, the present embodiment discloses The method performs registration based on the spine.

As shown in Figure 1, the automatic positioning method includes the following steps:

S1: Fix the neck, chest and abdomen of the patient to be radiotherapy on the treatment couch, and collect DR images of the corresponding parts;

S2: Input the CT image collected during planning into the trained U-Net model, obtain the result of spine segmentation and reconstruct the DRR image;

S3: Input the DR image obtained by S1 and the DRR image obtained by S2 into the CycleGAN model respectively to obtain a DR image containing only the spine;

S4: register the DR image containing only the spine obtained in S3 with the DRR image to be registered to obtain the placement deviation;

S5: Control the movement of the treatment couch according to the positioning deviation obtained in S4 to realize automatic positioning.

In order to further optimize the implementation effect of the present invention, in other embodiments, the remaining feature techniques are the same, the difference is that, as shown in FIG. 2 , the training steps of the U-Net model are as follows:

T1: Collect CT images (cross-sections) of the neck, chest and abdomen for clinical use, in nii format;

T2: data division, the CT images are divided into training set, test set and validation set according to the ratio of 6:3:1;

T3: Select the U-Net model, the input is CT image, the output is the segmentation result of the spine, and use the data in the training set to train the deep learning model;

T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

Further, on the basis of the above embodiment, the following steps are included between T1 and T2: preprocessing the collected CT images.

Preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling.

In order to further optimize the implementation effect of the present invention, in other embodiments, the remaining feature technologies are the same, the difference is that in S2, the reconstructed and generated DRR image specifically includes the following content:

Through the DRR reconstruction algorithm, the simulated X-ray light source penetrates the CT voxel, and after attenuation and absorption, it is projected to the detector plane for imaging to obtain a new DRR image.

It is worth noting that when performing image registration, the two registered entities should have the same dimensions, that is, either 3D-3D registration or 2D-2D registration. When performing 2D-3D registration, the 3D image needs to be registered. Descend to 2D, and then do 2D-2D registration. The above DRR reconstruction algorithm is a process of reducing the dimension of a 3D model to 2D. As shown in Figure 3, the simulated X-ray light source passes through 3D voxels, that is, 3D-CT voxels, and projects to the DRR plane to generate DRR images. The whole process is an optical The attenuation process conforms to the law of the optical absorption model, and the attenuation process can be described by the following expression.

Where: s is the length parameter of the optical projection direction;

I(s) is the optical intensity at distance s;

x(t) is the attenuation coefficient of optical intensity;

I ₀ is the optical intensity entering the CT voxel.

It can be seen from Figure 3 and the above formula that DRR is a process in which the simulated rays emitted by the simulated light source 1 pass through the CT voxel 2 and are projected to the imaging plane for accumulation after attenuation and absorption. Specifically:

1. Establish a 3-dimensional voxel matrix of the CT image group, which is composed of several CT voxels;

2. Send out several rays along the CT image group of the virtual light source image, and the number of rays is consistent with the number of pixels of the DRR plane 3;

3. Obtain the intersection points of each ray passing through this CT voxel, and accumulate the electron density values of these points;

4. Find the finite ray length of the projection line passing through the voxel matrix;

5. Multiply the accumulated value of the electron density by the ray length, and the obtained value is displayed as a grayscale value, which is the DRR image.

The embodiment of the present invention discloses an automatic positioning device based on deep learning, comprising:

The DR image acquisition module is used to collect the DR images of the corresponding parts after fixing the neck, chest and abdomen of the patient to be radiotherapy on the treatment bed;

The reconstruction DRR image generation module is used to input the CT images collected during planning into the trained U-Net model, obtain the segmentation results of the spine, and reconstruct and generate DRR images;

The DR image generation module is used to input the DR image collected by the DR image acquisition module and the DRR image generated by the reconstructed DRR image generation module into the CycleGAN model, respectively, to obtain a DR image containing only the spine;

The placement deviation generation module is used to register the DR image containing only the spine generated by the DR image generation module and the DRR image to be registered to obtain the placement deviation;

The automatic positioning module is used to control the movement of the treatment couch according to the positioning deviation generated by the positioning deviation generating module to realize automatic positioning.

Further, on the basis of the above-mentioned embodiment, the automatic positioning device further includes: a U-Net model training module, and the U-Net model training module is used to train the U-Net model by adopting the following steps;

The training steps of the U-Net model are as follows:

T1: Collect CT images (cross-sections) of the neck, chest and abdomen for clinical use, in nii format;

T2: Data division, dividing CT images into training set, test set and validation set according to the ratio of 6:3:1;

T3: Select the U-Net model, the input is CT image, the output is the segmentation result of the spine, and use the data in the training set to train the deep learning model;

T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

Further, on the basis of the above embodiment, the following steps are included between T1 and T2: preprocessing the collected CT images.

Preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling.

In addition, the embodiment of the present invention also discloses a radiotherapy device, which uses the automatic positioning method disclosed in any of the above embodiments to realize automatic positioning, or includes: the automatic positioning device disclosed in any of the above embodiments.

The present invention involves two deep learning models. The first model is a U-Net model, and the training enables it to automatically segment and extract the spine region in the CT image, and obtain the registered DRR image through the DRR reconstruction algorithm; the second model is the U-Net model. The model is the CycleGan model, which removes other redundant information in the DR image and preserves the spine, thereby reducing the interference of other factors in the image on the registration results. Extracting the ROI area by the above method is helpful to improve the placement accuracy and reduce the placement time.

The deep learning-based automatic positioning method, device and radiation therapy equipment of the present invention have the following beneficial effects:

First, use the U-Net deep learning model to automatically extract the spine area, use CycleGAN to remove the registration interference factors in the image, preserve the spine, and register the ROI area in a targeted manner, improve the quality of the image to be registered, and improve the registration. accuracy;

Second, the requirements for technicians are low, and the use is simple;

Third, the setup speed is fast, excluding the process of DRR reconstruction, the setup time is within 5s, while the manual setup method takes 5 minutes;

Fourth, it can be quickly iterated, and each upgrade can be upgraded online for hospitals without long-term training.

The invention can effectively improve the quality of images to be registered, improve the accuracy of registration, reduce the prediction time during placement by means of deep learning, finally realize automatic placement, avoid manual errors and improve placement efficiency.

It should be understood that the various techniques described herein can be implemented in conjunction with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or some aspects or portions of the method and apparatus of the present invention, may take the form of program code embedded in a tangible medium, such as a floppy disk, CD-ROM, hard drive, or any other machine-readable storage medium (ie, instructions), wherein when a program is loaded into a machine, such as a computer, and executed by the machine, the machine becomes an apparatus for practicing the invention.

The above-mentioned embodiments are only intended to illustrate the technical concept and characteristics of the present invention, and their purpose is to enable those of ordinary skill in the art to understand the content of the present invention and implement it, and cannot limit the scope of protection of the present invention with this. The equivalent changes or modifications made should be covered within the protection scope of the present invention.

Claims (10)

1. an automatic positioning method based on deep learning, is characterized in that, comprises the following steps: S1: Fix the patient to be radiotherapy on the treatment couch, and collect the DR image of the corresponding part; S2: Input the CT images collected during planning into the trained U-Net model, obtain the segmentation results of the specified parts, and reconstruct and generate DRR images; S3: Input the DR image obtained by S1 and the DRR image obtained by S2 into the CycleGAN model, respectively, to obtain a DR image containing only the specified part; S4: Register the DR image obtained in S3 that only contains the specified part with the DRR image to be registered to obtain the placement deviation; S5: Control the movement of the treatment couch according to the positioning deviation obtained in S4 to realize automatic positioning. 2. automatic positioning method according to claim 1, is characterized in that, the training step of U-Net model is as follows: T1: Collect CT images of various parts for clinical use; T2: Divide CT images into training set, test set and validation set according to the specified proportion; T3: Select the U-Net model, the input is CT image, the output is the segmentation result of the specified part, and use the data in the training set to train the deep learning model; T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained. 3 . The automatic positioning method according to claim 2 , wherein the following steps are included between T1 and T2 : preprocessing the collected CT images. 4 . 4 . The automatic positioning method according to claim 3 , wherein the preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling. 5 . 5. The automatic positioning method according to any one of claims 1-4, wherein in S2, the reconstruction and generation of a DRR image specifically include the following: Through the DRR reconstruction algorithm, the simulated X-ray light source penetrates the CT voxel, and after attenuation and absorption, it is projected to the detector plane for imaging to obtain a new DRR image. 6. An automatic positioning device based on deep learning, characterized in that, comprising: The DR image acquisition module is used to collect the DR image of the corresponding part after the patient to be radiotherapy is fixed on the treatment couch; The reconstruction DRR image generation module is used to input the CT images collected during planning into the trained U-Net model, obtain the segmentation results of the specified parts, and reconstruct and generate DRR images; The DR image generation module is used to input the DR image collected by the DR image acquisition module and the DRR image generated by the reconstructed DRR image generation module into the CycleGAN model respectively to obtain the DR image containing only the specified parts; The placement deviation generating module is used for registering the DR image generated by the DR image generating module and containing only the specified part with the DRR image to be registered to obtain the placement deviation; The automatic positioning module is used to control the movement of the treatment couch according to the positioning deviation generated by the positioning deviation generating module to realize automatic positioning. 7. automatic positioning device according to claim 6, is characterized in that, described automatic positioning device also comprises: U-Net model training module, U-Net model training module is used for adopting following steps to train U-Net model ; The training steps of the U-Net model are as follows: T1: Collect CT images of various parts for clinical use; T2: Divide CT images into training set, test set and validation set according to the specified proportion; T3: Select the U-Net model, the input is CT image, the output is the segmentation result of the specified part, and use the data in the training set to train the deep learning model; T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained. 8 . The automatic positioning device according to claim 7 , wherein the following step is included between T1 and T2 : preprocessing the collected CT images. 9 . 9 . The automatic positioning device according to claim 8 , wherein the preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling. 10 . 10. Radiation therapy equipment, characterized in that, using the automatic positioning method according to any one of claims 1-5 to realize automatic positioning, or, comprising: the automatic positioning according to any one of claims 6-9 bit device.

Images