CN116168097A - Method, device, equipment and medium for constructing CBCT delineation model and delineation CBCT image - Google Patents

Abstract

The present disclosure relates to a method, apparatus, device and medium for constructing a CBCT sketch model and sketching a CBCT image, the method comprising: acquiring training data and a training label, wherein the training data comprises CBCT images and CT images of a plurality of training subjects, and the training label comprises: the quality of a reference image of the CT image, and a reference sketching result of sketching a target area aiming at the CT image; the CBCT image is input into a first model for training, the output of the first model is a pseudo CT image, the training is finished under the condition that the difference between the image quality of the pseudo CT image and the quality of a reference image is smaller than a first set threshold value, and the trained first model is an image generation model; inputting the CT image into a second model for training, wherein the output of the second model is a prediction sketch result, and training is finished under the condition that the difference between the prediction sketch result and a reference sketch result is smaller than a second set threshold value, and the trained second model is a segmentation model; and generating a CBCT sketch model according to the image generation model and the segmentation model.

Description

Method, device, equipment and medium for constructing CBCT delineation model and delineation CBCT image

technical field

The present disclosure relates to the fields of image processing, medical technology and computer technology, and in particular to a method, device, device and medium for constructing a CBCT delineation model and delineating a CBCT image.

Background technique

Cone-beam computed tomography (CBCT, cone-beam computed tomography) is a widely used current image-guided device, which can be used for positioning correction of patients before radiotherapy to quantify factors such as tumor movement and organ movement within treatment fractions It can also implement online adaptive radiotherapy, which plays a major role in improving the accuracy of radiotherapy.

In each fraction of adaptive radiotherapy, doctors are required to delineate regions of interest such as tumor targets and organs at risk on CBCT images.

Contents of the invention

In order to solve or at least partially solve the following technical problems: manually delineating the region of interest on the CBCT image is not only very time-consuming, but also greatly increases the total time of adaptive radiotherapy, prolongs the waiting time of patients, and reduces the efficiency of clinical operations; Moreover, there are many artifacts in the CBCT image, and the image quality is poor. The direct delineation on the CBCT image largely depends on the experience and level of the doctor, and the subjective differences in delineation by different doctors lead to great differences in the delineation results; this disclosure The embodiments of the present invention provide a method, device, device and medium for constructing a CBCT delineation model and delineating a CBCT image.

In a first aspect, the embodiments of the present disclosure provide a method for constructing a CBCT delineation model. The above-mentioned method for constructing a CBCT delineation model includes: obtaining training data and training labels, the above-mentioned training data includes: CBCT images and CT images of a plurality of training objects, and the above-mentioned training labels include: the reference image quality of the above-mentioned CT images, for the above-mentioned CT images A reference delineation result for delineating the target area; the above-mentioned CBCT image is input to the first model for training, and the output of the above-mentioned first model is a pseudo CT image, and the difference between the image quality of the above-mentioned pseudo CT image and the above-mentioned reference image quality is smaller than the first When the threshold is set, the training ends, and the trained first model is an image generation model; the above-mentioned CT images are input to the second model for training, and the output of the above-mentioned second model is the prediction delineation result, and the above-mentioned prediction delineation result and the above-mentioned When the difference between the reference delineation results is less than the second set threshold, the training ends, and the trained second model is a segmentation model; a CBCT delineation model is generated according to the above-mentioned image generation model and the above-mentioned segmentation model.

According to an embodiment of the present disclosure, generating a CBCT delineation model based on the above-mentioned image generation model and the above-mentioned segmentation model includes: using the output of the above-mentioned image generation model as the input of the above-mentioned segmentation model to obtain a group including the above-mentioned image generation model and the above-mentioned segmentation model Delineate the CBCT delineation model; according to the CT image of the first object and the reference delineation result for the CT image of the first object, fine-tune the parameters of the above-mentioned segmentation model to obtain a personalized segmentation model adapted to the above-mentioned first object; The output of the image generation model is used as the input of the above-mentioned personalized segmentation model, and a personalized CBCT delineation model including the above-mentioned image generation model and the above-mentioned personalized segmentation model is obtained.

According to an embodiment of the present disclosure, according to the CT image of the first object and the reference delineation result for the CT image of the first object, fine-tuning is performed on the parameters of the segmentation model to obtain a personalized segmentation model adapted to the first object , comprising: inputting the CT image of the above-mentioned first object into the above-mentioned segmentation model, and outputting the prediction delineation result of the CT image of the above-mentioned first object; fine-tuning the parameters of the above-mentioned segmentation model, so that the reference of the CT image of the above-mentioned first object The difference between the delineation result and the predicted delineation result is smaller than the second set threshold, and the segmentation model after parameter fine-tuning is a personalized segmentation model adapted to the above-mentioned first object.

According to an embodiment of the present disclosure, the difference in image quality is analyzed from the four dimensions of noise level, artifact, tissue boundary definition, and gray value; the above-mentioned first set threshold includes: noise error threshold, image error threshold, Sharpness error threshold and grayscale error threshold; the difference between the image quality of the above-mentioned pseudo CT image and the above-mentioned reference image quality with respect to the noise level is less than the noise error threshold, the difference between the artifacts is smaller than the image error threshold, and the difference between the tissue boundary definition If it is less than the sharpness error threshold and the difference with respect to the grayscale value is smaller than the grayscale error threshold, it is considered that the image quality of the above-mentioned pseudo CT image is consistent with the above-mentioned reference image quality, and the condition for the end of training is met.

In a second aspect, embodiments of the present disclosure provide a method for delineating a CBCT image. The above-mentioned method for delineating a CBCT image includes: obtaining a CBCT image to be delineated of a target object; inputting the above-mentioned CBCT image to be delineated into a pre-trained target image generation model, and outputting a pseudo CT image; inputting the above-mentioned pseudo CT image into a pre-trained The target segmentation model, output the CBCT delineation result of the above-mentioned target object; wherein, the above-mentioned target image generation model includes the first network parameters mapped from the CBCT image to the pseudo-CT image; the above-mentioned target segmentation model includes the mapping from the CT image to the CT image The second network parameter of the delineation result of the target area; in the training stage of the above-mentioned target image generation model, the input is the CBCT image of the object for training, and the output is the pseudo-CT image of the object for the above-mentioned training; the pseudo-CT image of the above-mentioned training object The image quality was the same as that of the CT image of the above-mentioned training object.

According to an embodiment of the present disclosure, the first network parameters of the above-mentioned target image generation model are obtained in the following manner: using multiple CBCT images of training objects as the training data of the first model, and referring to the CT images of the above-mentioned multiple training objects The image quality is used as the training label of the first model, the trained first model is the target image generation model, and the trained parameters of the first model are the first network parameters. The second network parameters of the above-mentioned target segmentation model are obtained by one of the following methods: using the CT images of the above-mentioned multiple training objects as the training data of the second model, and delineating the target area for the CT images of the above-mentioned multiple training objects The reference delineation result of the above-mentioned second model is used as the training label of the above-mentioned second model, the second model after training is the above-mentioned target segmentation model, and the parameters trained by the above-mentioned second model are the above-mentioned second network parameters; or, the above-mentioned multiple training objects The CT image is used as the training data of the second model, and the reference delineation result of the target area delineation for the CT images of the above-mentioned multiple training objects is used as the training label of the above-mentioned second model, and the parameters obtained from the training are used as the intermediate parameters of the above-mentioned second model; According to the CT image of the above-mentioned target object and the reference delineation result for the CT image of the above-mentioned target object, the intermediate parameters of the above-mentioned second model are fine-tuned, the second model after fine-tuning is the above-mentioned target segmentation model, and the parameters after fine-tuning are obtained as the above-mentioned Second network parameters.

In a third aspect, embodiments of the present disclosure provide an apparatus for constructing a CBCT delineation model. The above-mentioned device for constructing a CBCT delineation model includes: a training data and label acquisition module, a first training module, a second training module and a delineation model generation module. The above-mentioned training data and label acquisition module is used to obtain training data and training labels. The above-mentioned training data includes: CBCT images and CT images of multiple training objects. The above-mentioned training labels include: the reference image quality of the above-mentioned CT images, for the above-mentioned CT images The reference delineation result for delineation of the target area. The above-mentioned first training module is used to input the above-mentioned CBCT image to the first model for training, the output of the above-mentioned first model is a pseudo CT image, and the difference between the image quality of the above-mentioned pseudo-CT image and the above-mentioned reference image quality is smaller than the first setting In the case of the threshold value, the training ends, and the first model after training is an image generation model. The second training module is used to input the CT image to the second model for training, the output of the second model is the predicted delineation result, and when the difference between the predicted delineation result and the reference delineation result is less than the second set threshold Next, the training ends, and the second model after training is a segmentation model. The above-mentioned delineation model generating module is configured to generate a CBCT delineation model according to the above-mentioned image generation model and the above-mentioned segmentation model.

In a fourth aspect, embodiments of the present disclosure provide an apparatus for delineating a CBCT image. The above device includes: a data acquisition module, a first processing module and a second processing module. The above-mentioned data acquisition module is used to acquire the CBCT image to be outlined of the target object. The above-mentioned first processing module is used to input the above-mentioned CBCT image to be delineated into a pre-trained target image generation model, and output a pseudo CT image. The second processing module is configured to input the pseudo CT image into a pre-trained target segmentation model, and output a CBCT delineation result of the target object. Wherein, the above-mentioned target image generation model includes the first network parameters mapped from the CBCT image to the pseudo-CT image; the above-mentioned target segmentation model includes the second network parameters mapped from the CT image to the delineation result of the target area of the CT image; in the above-mentioned target image In the training phase of the generated model, the input is the CBCT image of the training object, and the output is the pseudo CT image of the training object; the image quality of the pseudo CT image of the training object is consistent with the image quality of the CT image of the training object.

In a fifth aspect, embodiments of the present disclosure provide an electronic device. The above-mentioned electronic equipment includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete mutual communication through the communication bus; the memory is used to store computer programs; the processor is used to execute all programs on the memory. The stored program implements the method for constructing a CBCT delineation model or the method for delineating a CBCT image as described above.

In a sixth aspect, embodiments of the present disclosure provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for constructing a CBCT delineation model or the method for delineating a CBCT image as described above is implemented.

The above-mentioned technical solutions provided by the embodiments of the present disclosure have at least some or all of the following advantages:

Considering that the CBCT image and the CT image have different scanning times, the image structures presented by the two are similar but the image quality is different, and because the CBCT image has many artifacts, if the delineation result of the CT image is used to supervise the delineation of the CBCT image Training will cause problems such as delineation deformation and inaccurate delineation; in the method for constructing a CBCT delineation model and the method for delineating a CBCT image provided by the embodiment of the present disclosure, by training the first model and the second model, the CBCT image is obtained to represent the CBCT image. The image generation model of the mapping relationship of the pseudo CT image and the segmentation model used to represent the mapping relationship between the CT image and the prediction delineation result, and generate the CBCT delineation model according to the image generation model and the segmentation model; because in the supervised training process of the first model , the real image quality of the CT image is used as the reference image quality corresponding to the training label, so that the image quality of the pseudo CT image corresponding to the CBCT image through the image generation model is consistent with the reference image quality of the real CT image. Since the pseudo CT image and the real CT image The image quality of the image is consistent, so the delineation result of the CT image can be applied to the pseudo CT image, and because the image structure between the CBCT image and the pseudo CT image is consistent, the delineation on the pseudo CT image is equivalent to the tissue corresponding to the CBCT image The structure is matched and delineated, and the delineation result based on the real CT is realized as a whole to delineate the CBCT image relatively accurately and efficiently.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art , on the premise of not paying creative labor, other drawings can also be obtained based on these drawings.

Fig. 1 schematically shows a flowchart of a method for constructing a CBCT delineation model provided by an embodiment of the present disclosure;

Fig. 2 schematically shows a schematic diagram of a training process of an image generation model according to an embodiment of the present disclosure;

Fig. 3 schematically shows a schematic diagram of a training process of a segmentation model according to an embodiment of the present disclosure;

Fig. 4 schematically shows a schematic diagram of the process of generating a CBCT sketching model according to an embodiment of the present disclosure;

Fig. 5 schematically shows a comparison diagram of delineating the clinical target volume (CTV) of the tumor clinical target volume (CTV) on the CBCT image of a nasopharyngeal carcinoma patient. (a1) Real CTV delineation results of the CBCT images corresponding to patient X, (a2) CTV delineation results using the grouped CBCT delineation model, (a3) CTV delineation results using the individualized CBCT delineation model; (b) patients with nasopharyngeal carcinoma The CBCT image corresponding to Y, for the CBCT image corresponding to the nasopharyngeal carcinoma patient Y (b1) the real CTV delineation result, (b2) the CTV delineation result using the group CBCT delineation model, (b3) the delineation result using the personalized CBCT delineation model CTV delineation results;

Fig. 6 schematically shows the comparison diagram of delineating the nasopharyngeal tumor target volume (GTVnx) on the CBCT image of the nasopharyngeal carcinoma patient, respectively showing (a) the corresponding CBCT image of the nasopharyngeal carcinoma patient X, for the nasopharyngeal tumor (a1) Real GTVnx delineation results of CBCT images corresponding to cancer patient X, (a2) GTVnx delineation results using population CBCT delineation model, (a3) GTVnx delineation results using personalized CBCT delineation model; (b) nasopharyngeal carcinoma For the CBCT image corresponding to patient Y, the (b1) real GTVnx delineation results, (b2) the GTVnx delineation results using the group CBCT delineation model, and (b3) the personalized CBCT delineation model for the CBCT image corresponding to the nasopharyngeal carcinoma patient Y GTVnx sketch results;

FIG. 7 schematically shows a flowchart of a method for delineating a CBCT image according to an embodiment of the present disclosure;

FIG. 8 schematically shows a structural block diagram of a device for constructing a CBCT sketching model provided by an embodiment of the present disclosure;

FIG. 9 schematically shows a structural block diagram of a device for delineating a CBCT image provided by an embodiment of the present disclosure; and

Fig. 10 schematically shows a structural block diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments It is a part of embodiments of the present disclosure, but not all embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present disclosure.

The first exemplary embodiment of the present disclosure provides a method for constructing a CBCT delineation model. The above methods can be performed by electronic devices with computing capabilities.

Fig. 1 schematically shows a flowchart of a method for constructing a CBCT delineation model provided by an embodiment of the present disclosure.

Referring to FIG. 1 , the method for constructing a CBCT delineation model provided by an embodiment of the present disclosure includes the following steps: S110 , S120 , S130 and S140 .

In step S110, training data and training labels are acquired. The above training data includes: CBCT images and CT images of a plurality of training objects. The above training labels include: the reference image quality of the above CT images, and the target area delineation for the above CT images. Refer to sketching results.

The above-mentioned training objects refer to the objects of data collection in the training set. The training data are, for example, CBCT images and CT images of patients in the medical system database. The same body part, organ, tissue area, etc. of the same patient have CBCT images and CT images at the same time. .

For example, there are CBCT images and CT images captured for the head region, stomach region, thoracic region or abdomen region of a certain cancer patient _P1 ; all or part of the same type of cancer patients in the medical system database (such as nasal CBCT images and CT images of patients with pharyngeal cancer or lung cancer, etc.) can be used as training data. The above CBCT image and CT image are image data with the same pixel size and mutual registration (with a mapping matching relationship).

The target area is, for example, a target area, a target position, and the like in the CBCT image and CT image of the training object.

For each CT image, the real image quality of the CT image can be obtained; in order to distinguish it from the image quality of the pseudo CT image obtained through the output of the first model and explain the role of the label, the real image quality of the CT image Described as reference image quality.

For each CT image, a delineation result after delineation of the target area on the current CT image can be obtained; the above delineation result refers to a boundary line or a contour line of the target area. In order to differentiate in description from the predicted delineation result obtained through the subsequent output of the second model and to explain the function of the label, the delineation result of the target region delineation on the CT image is described as the reference delineation result.

Because CT images have the characteristics of fast scanning time and clear images, there are abundant CT images and corresponding accurate reference delineation results stored in the medical system database. Therefore, it is possible to consider using the reference delineation results of CT images to delineate CBCT images for reference. ; Further considering that due to the difference in scanning time between CBCT images and CT images, the image structure presented by the two is similar but the image quality is different, and because there are many artifacts in the CBCT image, if the delineation result of the CT image is used to delineate the CBCT image Supervised training will cause problems such as delineation deformation and inaccurate delineation. Therefore, in the embodiment of the present disclosure, through the training process of constructing two models, the output of the first model and the input of the second model are based on the consistency of image quality. Carry out association, by training the first model and the second model, the image generation model used to represent the mapping relationship between CBCT images and pseudo CT images and the segmentation model used to represent the mapping relationship between CT images and predicted delineation results are obtained, because the generated The pseudo-CT image and the CBCT image have the same anatomical structure (or described as image structure), and the pseudo-CT image and the CT image have the same image quality, so that the accuracy of the CBCT image can be realized according to the transformation of the image generation model and the delineation of the segmentation model. outline.

In step S120, the above-mentioned CBCT image is input to the first model for training, and the output of the above-mentioned first model is a pseudo CT image, when the difference between the image quality of the above-mentioned pseudo-CT image and the above-mentioned reference image quality is smaller than the first set threshold Next, the training ends, and the first model trained is an image generation model.

Fig. 2 schematically shows a schematic diagram of a training process of an image generation model according to an embodiment of the present disclosure.

Referring to FIG. 2 , the image generation model in the training phase is described as the first model, and the first model after training is the image generation model. In some embodiments, the first model may be a neural network model, for example, an image generation model may be obtained by training based on a deep learning method.

The input of the first model is the CBCT image in the training data, and the output is the pseudo CT image. Pseudo CT images refer to CT images generated by computer simulation or through neural network models, which are different from real CT images obtained by scanning with instruments.

In an embodiment, the training of the image generation model can be realized based on a part of the structure of the CycleGAN network. The CycleGAN network consists of two generators and two discriminators, described as generator A and generator B, discriminator C and discriminator D, respectively. The flow direction of data input and output is expressed as: real CBCT image → generator A → pseudo CT → generator B → reconstructed CBCT image, and the discriminator C and discriminator D are also included in the training process, and the discriminator C is used to judge the generated Whether the image quality between the fake CT image and the real CT image is consistent, the discriminator D is used to judge whether the reconstructed CBCT image is consistent with the real CBCT image. Among them, the network corresponding to the generator A and the discriminator C can be selected for training to obtain the image generation model.

During the training process, the training ends when the difference between the quality of the pseudo CT image and the quality of the reference image is smaller than the first set threshold.

In one embodiment, the difference in image quality is analyzed from the four dimensions of noise level, artifact, tissue boundary definition, and gray value; the above-mentioned first set threshold includes: noise error threshold, image error threshold, clarity The difference between the image quality of the above-mentioned pseudo CT image and the above-mentioned reference image quality about the noise level is less than the noise error threshold, the difference about the artifact is less than the image error threshold, and the difference about the definition of the tissue boundary is less than When the sharpness error threshold is lower than the grayscale error threshold, it is considered that the image quality of the above-mentioned pseudo CT image is consistent with the above-mentioned reference image quality, and the training end condition is met.

In step S130, the above-mentioned CT images are input to the second model for training, and the output of the above-mentioned second model is the predicted delineation result, and the training ends when the difference between the above-mentioned predicted delineation result and the above-mentioned reference delineation result is less than the second set threshold , the second model after training is the segmentation model.

Fig. 3 schematically shows a schematic diagram of a training process of a segmentation model according to an embodiment of the present disclosure.

Referring to FIG. 3 , the segmentation model in the training phase is described as the second model, and the second model after training is the segmentation model. In some embodiments, the second model may be a neural network model, for example, it may be trained based on a deep learning method to obtain a segmentation model.

The input of the second model is the CT image in the training data, and the output is the predicted delineation result; the training label is the reference delineation result for delineating the target area on the CT image.

In some embodiments, the above-mentioned second model may be, but not limited to, a Deeplab V3 network model.

In step S140, a CBCT delineation model is generated based on the image generation model and the segmentation model.

Fig. 4 schematically shows a schematic diagram of a process of generating a CBCT sketching model according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, as shown in FIG. 4 , in the above step S140, a CBCT sketching model is generated according to the above image generation model and the above segmentation model, including:

Using the output of the above-mentioned image generation model as the input of the above-mentioned segmentation model, a grouped CBCT delineation model 410 comprising the above-mentioned image generation model and the above-mentioned segmentation model is obtained;

Fine-tuning the parameters of the segmentation model according to the CT image of the first object and the reference delineation result for the CT image of the first object to obtain a personalized segmentation model adapted to the first object;

Using the output of the above-mentioned image generation model as the input of the above-mentioned personalized segmentation model, a personalized CBCT delineation model 420 including the above-mentioned image generation model and the above-mentioned personalized segmentation model is obtained.

In some embodiments, the image generation model and the segmentation model can be combined to obtain the grouped CBCT delineation model, and the output of the image generation model is used as the input of the segmentation model.

Considering that the segmentation model trained by the second model is a generalized model adapted to population, although the lesion areas of all patients are similar in general structure, there are still individual differences between the details of the target structure of different patients. difference. That is, it is difficult to accurately predict the delineated shape of the region of interest (target region) for individuals with large anatomical structure changes or special body shapes based on a grouped segmentation model trained on a limited data scale. Therefore, in other embodiments, the CBCT image of the first object (which can also be described as the target object) that needs to be delineated is used as the personalized input of the segmentation model to fine-tune the parameters of the segmentation model, so that the fine-tuned segmentation model Combined with the image generation model to obtain a personalized CBCT delineation model, the output of the image generation model is used as the input of the fine-tuned segmentation model. By establishing a personalized segmentation model, it is possible to quickly and objectively segment individual ROI structures with high precision on CBCT images.

According to an embodiment of the present disclosure, according to the CT image of the first object and the reference delineation result for the CT image of the first object, fine-tuning is performed on the parameters of the segmentation model to obtain a personalized segmentation model adapted to the first object , comprising: inputting the CT image of the above-mentioned first object into the above-mentioned segmentation model, and outputting the prediction delineation result of the CT image of the above-mentioned first object; fine-tuning the parameters of the above-mentioned segmentation model, so that the reference of the CT image of the above-mentioned first object The difference between the delineation result and the predicted delineation result is smaller than the second set threshold, and the segmentation model after parameter fine-tuning is a personalized segmentation model adapted to the above-mentioned first object.

In the application scenario, the above-mentioned grouped CBCT delineation model can be used to perform automatic delineation of CBCT images, and the above-mentioned personalized CBCT delineation model can also be used to perform automatic delineation of CBCT images.

In the embodiment including the above steps S110-S140, by training the first model and the second model, the image generation model used to represent the mapping relationship between the CBCT image and the pseudo CT image and the image generation model used to represent the CT image to the predicted delineation result are obtained. The segmentation model of the mapping relationship generates the CBCT delineation model according to the image generation model and the segmentation model; in the supervised training process of the first model, the real image quality of the CT image is used as the reference image quality corresponding to the training label, so that the CBCT image is The image quality of the pseudo CT image corresponding to the image generation model is consistent with the reference image quality of the real CT image. Since the image quality of the pseudo CT image and the real CT image are consistent, the delineation result of the CT image can be applied to the pseudo CT image. At the same time, because The image structure between the CBCT image and the pseudo-CT image is consistent, and the delineation on the pseudo-CT image is equivalent to matching and delineating the tissue structure corresponding to the CBCT image. On the whole, the delineation result based on the real CT is relatively accurate and efficient. Outline the CBCT image.

For example, when establishing an image generation model from CBCT images to pseudo CT images, the patient's positioning CT images and CBCT images collected during treatment are selected to form a training data set. The patient's CT images were selected from a Philip Brilliance CT Big Bore CT scanner or a Siemens SOMATOM Definition AS CT-Sim CT detector. Use medical image processing software (such as MIM software for medical image processing) to register CT images (also can be understood as mapping pairing) to CBCT images to form CT-CBCT pairing data with the same pixel size as CBCT images. Then use the MIM software to automatically outline the outer contour of the target area, and extract the part within the outer contour range for deep learning training and testing. Before inputting into the neural network, the CT image and the CBCT image can also be normalized so that the range is in the value interval [-1, 1]. Then, a deep learning network is used to learn the mapping from CBCT images to pseudo-CT images. The task is to input single-channel CBCT images and output single-channel pseudo-CT images. Supervised training is adopted, and the training label is the image quality of CT images.

During the application process of the method of the embodiment of the present disclosure, the CBCT delineation model is constructed by means of the deep learning method, and the CT positioning images and corresponding delineation results measured by each patient before radiotherapy are used to automatically delineate the grouped CBCT The model is personalized and fine-tuned, and there is no need to collect manual delineation data on CBCT images during the training process, and it can effectively improve the automatic delineation accuracy of the region of interest.

For example, when a new patient is admitted to the hospital, the doctor first completes the CT scan of the target area to obtain a CT image, and delineates the region of interest corresponding to the CT image. Before the patient undergoes the first CBCT scan, the positioning CT image and outline are input into the segmentation model, and the segmentation model is fine-tuned to obtain a personalized segmentation model (also can be described as an individualized segmentation model). After the CBCT scan is performed on the day of the patient, the CBCT image is input into the image generation model to generate the corresponding pseudo CT image, and the pseudo CT image is input into the fine-tuned personalized segmentation model to obtain high-precision automatic delineation of the region of interest Structure. Doctors and physicists can observe the patient's anatomical changes based on the quickly sketched structure, or perform adaptive radiotherapy, etc. Not only is the accuracy higher than that of the group model, but the deep learning method can also be greatly improved compared with manual sketching. Efficiency, save patient waiting time, improve clinical operation efficiency.

Fig. 5 schematically shows a comparison diagram of delineating the clinical target volume (CTV) of the tumor clinical target volume (CTV) on the CBCT image of a nasopharyngeal carcinoma patient. (a1) Real CTV delineation results of the CBCT images corresponding to patient X, (a2) CTV delineation results using the grouped CBCT delineation model, (a3) CTV delineation results using the individualized CBCT delineation model; (b) patients with nasopharyngeal carcinoma The CBCT image corresponding to Y, for the CBCT image corresponding to the nasopharyngeal carcinoma patient Y (b1) the real CTV delineation result, (b2) the CTV delineation result using the group CBCT delineation model, (b3) the delineation result using the personalized CBCT delineation model CTV sketches the results.

Take the image of nasopharyngeal carcinoma patients and the delineation of tumor clinical target volume (CTV) as an example. Referring to (a) and (b) in Figure 5, the CBCT images of two nasopharyngeal carcinoma patients X and Y are shown respectively, comparing (a1) and (a2) in Figure 5, or (b1) in Figure 5 From the results of and (b2), it can be seen that the CTV delineation result drawn by the grouped CBCT delineation model can basically draw the core part of the real tumor clinical target area, with acceptable accuracy, and the automatic delineation based on the grouped CBCT delineation model, It saves the manpower and time cost of manual CBCT delineation, and does not need to delineate and label CBCT data in advance, just use a large number of CT images in the medical system database and the corresponding CT delineation and labeling data (as a reference delineation result); comparison (a1) and (a3) in Figure 5 or comparing the results of (b1) and (b3) in Figure 5, it can be seen that the CTV delineation results based on the personalized CBCT delineation model delineation are almost close to the real delineation results, except for the group In addition to the advantages of the CBCT delineation model, it also improves the accuracy of the delineation results.

Fig. 6 schematically shows the comparison diagram of delineating the nasopharyngeal tumor target volume (GTVnx) on the CBCT image of the nasopharyngeal carcinoma patient, respectively showing (a) the corresponding CBCT image of the nasopharyngeal carcinoma patient X, for the nasopharyngeal tumor (a1) Real GTVnx delineation results of CBCT images corresponding to cancer patient X, (a2) GTVnx delineation results using population CBCT delineation model, (a3) GTVnx delineation results using personalized CBCT delineation model; (b) nasopharyngeal carcinoma For the CBCT image corresponding to patient Y, the (b1) real GTVnx delineation results, (b2) the GTVnx delineation results using the group CBCT delineation model, and (b3) the personalized CBCT delineation model for the CBCT image corresponding to the nasopharyngeal carcinoma patient Y GTVnx sketches the results.

Take the images of nasopharyngeal carcinoma patients and the delineation of nasopharyngeal tumor target volume (GTVnx) as an example. Referring to (a) and (b) shown in Figure 6, the CBCT images of two nasopharyngeal carcinoma patients X and Y are illustrated respectively, compared with (a1) and (a2) in Figure 6 or (b1) and ( From the results of b2), it can be seen that the GTVnx delineation results drawn by the group CBCT delineation model can basically draw the core part of the real nasopharyngeal tumor target area, with acceptable accuracy. Automatic delineation based on the population CBCT delineation model saves It saves the manpower and time cost of manual CBCT delineation, and does not need to delineate and label CBCT data in advance, just use a large number of CT images in the medical system database and corresponding CT delineation and labeling data (as a reference delineation result); comparison diagram The results of (a1) and (a3) in 6 or (b1) and (b3) in Figure 6 show that the GTVnx delineation results based on the personalized CBCT delineation model delineation are almost close to the real delineation results, except that the grouped CBCT In addition to the advantages of sketching the model, it also improves the accuracy of the sketching results.

A second exemplary embodiment of the present disclosure provides a method of delineating a CBCT image. The above methods can be performed by electronic devices with computing capabilities.

In some embodiments, this embodiment can directly use the grouped CBCT delineation model obtained in the first embodiment above or the personalized CBCT delineation model for data processing, and input the CBCT image to be delineated of the target object into the grouped CBCT delineation model. Model or personalized CBCT delineation model, and output the CBCT delineation result corresponding to the target object.

Fig. 7 schematically shows a flowchart of a method for delineating a CBCT image according to an embodiment of the present disclosure.

Referring to FIG. 7 , the method for delineating a CBCT image provided by an embodiment of the present disclosure includes the following steps: S710 , S720 and S730 .

In step S710, a CBCT image to be outlined of the target object is acquired.

In step S720, the above-mentioned CBCT image to be delineated is input into the pre-trained target image generation model, and a pseudo CT image is obtained as output.

In step S730, the pseudo CT image is input into the pre-trained target segmentation model, and the CBCT delineation result of the target object is output.

Wherein, the target image generation model includes first network parameters mapped from the CBCT image to the pseudo CT image; the target segmentation model includes second network parameters mapped from the CT image to the delineation result of the target area of the CT image. In the training stage of the above-mentioned target image generation model, the input is the CBCT image of the training object, and the output is the pseudo CT image of the above-mentioned training object; the image quality of the pseudo CT image of the above-mentioned training object and the CT image of the above-mentioned training object Image quality is consistent.

According to an embodiment of the present disclosure, the first network parameters of the above-mentioned target image generation model are obtained in the following manner: using multiple CBCT images of training objects as the training data of the first model, and referring to the CT images of the above-mentioned multiple training objects The image quality is used as the training label of the first model, the trained first model is the target image generation model, and the trained parameters of the first model are the first network parameters.

In some embodiments, the second network parameters of the above target segmentation model are obtained in one of the following ways:

The CT images of the above-mentioned multiple training objects are used as the training data of the second model, and the reference delineation results of target area delineation are performed on the CT images of the above-mentioned multiple training objects as the training labels of the above-mentioned second model, and the second model after training is completed. The model is the above-mentioned target segmentation model, and the parameters trained by the above-mentioned second model are the above-mentioned second network parameters; or,

The CT images of the above-mentioned multiple training objects are used as the training data of the second model, the reference delineation results of target area delineation are performed on the CT images of the above-mentioned multiple training objects as the training labels of the above-mentioned second model, and the parameters obtained after training are used as The intermediate parameters of the above-mentioned second model; according to the CT image of the above-mentioned target object and the reference delineation result for the CT image of the above-mentioned target object, the intermediate parameters of the above-mentioned second model are fine-tuned, and the second model after fine-tuning is the above-mentioned target segmentation model , the fine-tuned parameters obtained are the above-mentioned second network parameters.

In an embodiment, generating a CBCT delineation model based on the above-mentioned image generation model and the above-mentioned segmentation model includes: using the output of the above-mentioned image generation model as the input of the above-mentioned segmentation model to obtain the grouping comprising the above-mentioned image generation model and the above-mentioned segmentation model CBCT delineation model; according to the CT image of the first object and the reference delineation result for the CT image of the first object, fine-tune the parameters of the above-mentioned segmentation model to obtain a personalized segmentation model adapted to the above-mentioned first object; the above-mentioned image The output of the generation model is used as the input of the above-mentioned personalized segmentation model to obtain a personalized CBCT delineation model including the above-mentioned image generation model and the above-mentioned personalized segmentation model, which can refer to the detailed description of FIG. 4 in the first embodiment.

For the specific details of the second embodiment, reference may also be made to the related description of the first embodiment, which will not be repeated here.

A third exemplary embodiment of the present disclosure provides an apparatus for constructing a CBCT delineation model.

Fig. 8 schematically shows a structural block diagram of an apparatus for constructing a CBCT sketching model provided by an embodiment of the present disclosure.

Referring to FIG. 8 , an apparatus 800 for constructing a CBCT delineation model provided by an embodiment of the present disclosure includes: a training data and label acquisition module 801 , a first training module 802 , a second training module 803 and a delineation model generation module 804 .

The above-mentioned training data and label acquisition module 801 is used to obtain training data and training labels. The above-mentioned training data includes: CBCT images and CT images of a plurality of training objects, and the above-mentioned training labels include: the reference image quality of the above-mentioned CT images, for the above-mentioned CT The reference delineation result of delineating the target area on the image.

The above-mentioned first training module 802 is used to input the above-mentioned CBCT image into the first model for training, the output of the above-mentioned first model is a pseudo CT image, and the difference between the image quality of the above-mentioned pseudo CT image and the above-mentioned reference image quality is smaller than the first setting When the threshold is set, the training ends, and the first model after training is an image generation model.

The second training module 803 is used to input the CT images to the second model for training, the output of the second model is the predicted delineation result, and the difference between the predicted delineation result and the reference delineation result is less than the second set threshold In this case, the training ends, and the second model after training is a segmentation model.

The sketching model generation module 804 is configured to generate a CBCT sketching model according to the image generation model and the segmentation model.

For implementation details of the functional modules of this embodiment, reference may be made to the relevant description of the first embodiment, and details are not repeated here.

A fourth exemplary embodiment of the present disclosure provides an apparatus for delineating a CBCT image.

Fig. 9 schematically shows a structural block diagram of an apparatus for delineating a CBCT image provided by an embodiment of the present disclosure.

Referring to FIG. 9 , an apparatus 900 for delineating a CBCT image provided by an embodiment of the present disclosure includes: a data acquisition module 901 , a first processing module 902 and a second processing module 903 .

The above-mentioned data acquisition module 901 is used to acquire the CBCT image to be outlined of the target object.

The first processing module 902 is configured to input the CBCT image to be delineated into a pre-trained target image generation model, and output a pseudo CT image. In one embodiment, the first processing module includes a target image generation model; in another embodiment, the first processing module can communicate with a device storing the target image generation model, and invoke the target image generation model to execute the CBCT image to be delineated. processing steps.

The second processing module 903 is configured to input the pseudo CT image into a pre-trained target segmentation model, and output a CBCT delineation result of the target object. In one embodiment, the second processing module includes an object segmentation model. In another embodiment, the second processing module can communicate with the device storing the target segmentation model, and invoke the target segmentation model to execute the processing steps on the pseudo CT image. In some embodiments, the target image generation model and the target segmentation model are integrated in the same device as a whole CBCT delineation model; in other embodiments, the target image generation model and the target segmentation model can be dispersed in different devices , the two are correlated with each other based on the quality of CT images in the training phase.

Wherein, the above-mentioned target image generation model includes the first network parameters mapped from the CBCT image to the pseudo-CT image; the above-mentioned target segmentation model includes the second network parameters mapped from the CT image to the delineation result of the target area of the CT image; in the above-mentioned target image In the training phase of the generated model, the input is the CBCT image of the training object, and the output is the pseudo CT image of the training object; the image quality of the pseudo CT image of the training object is consistent with the image quality of the CT image of the training object.

Any number of the functional modules included in the above-mentioned apparatus 800 or apparatus 900 may be combined into one module for implementation, or any one of the modules may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. At least one of the functional modules contained in the above-mentioned device 800 or device 900 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate , a system on a package, an application-specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging circuits, such as hardware or firmware, or any of the three implementation methods of software, hardware, and firmware Or realize it with any suitable combination of any of them. Alternatively, at least one of the functional modules included in the above-mentioned apparatus 800 or apparatus 900 may be at least partially implemented as a computer program module, and when the computer program module is executed, corresponding functions may be performed.

A fifth exemplary embodiment of the present disclosure provides an electronic device.

Fig. 10 schematically shows a structural block diagram of an electronic device provided by an embodiment of the present disclosure.

Referring to FIG. 10 , an electronic device 1000 provided by an embodiment of the present disclosure includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004. The memory 1003 is used to store computer programs; the processor 1001 is used to implement the method for constructing a CBCT delineation model or the method for delineating a CBCT image as described above when executing the program stored in the memory.

The sixth exemplary embodiment of the present disclosure also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for constructing a CBCT delineation model or the method for delineating a CBCT image as described above is implemented.

The computer-readable storage medium may be included in the device/device described in the above embodiments; or it may exist independently without being assembled into the device/device. The above-mentioned computer-readable storage medium carries one or more programs, and when the above-mentioned one or more programs are executed, the method according to the embodiment of the present disclosure is realized.

According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as may include but not limited to: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM) , erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

It should be noted that in this article, relative terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these No such actual relationship or order exists between entities or operations. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The above descriptions are only specific implementation manners of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims (10)

1. A method of constructing a CBCT delineation model, comprising:

acquiring training data and training labels, wherein the training data comprises: a CBCT image and a CT image of a plurality of training subjects, the training tag comprising: the reference image quality of the CT image is used for carrying out a reference sketching result of target area sketching aiming at the CT image;

inputting the CBCT image into a first model for training, wherein the output of the first model is a pseudo CT image, and training is finished under the condition that the difference between the image quality of the pseudo CT image and the quality of the reference image is smaller than a first set threshold value, and the trained first model is an image generation model;

Inputting the CT image into a second model for training, wherein the output of the second model is a prediction sketch result, and training is finished under the condition that the difference between the prediction sketch result and the reference sketch result is smaller than a second set threshold value, and the trained second model is a segmentation model;

and generating a CBCT sketch model according to the image generation model and the segmentation model.

2. The method of claim 1, wherein generating a CBCT delineation model from the image generation model and the segmentation model comprises:

taking the output of the image generation model as the input of the segmentation model to obtain a clustered CBCT sketch model comprising the image generation model and the segmentation model;

performing parameter fine adjustment on the segmentation model according to the CT image of the first object and a reference sketching result of the CT image of the first object to obtain a personalized segmentation model adapted to the first object;

and taking the output of the image generation model as the input of the personalized segmentation model to obtain a personalized CBCT sketch model comprising the image generation model and the personalized segmentation model.

3. The method according to claim 2, wherein the performing parameter fine-tuning on the segmentation model according to the CT image of the first object and the reference delineation result of the CT image of the first object to obtain a personalized segmentation model adapted to the first object comprises:

Inputting the CT image of the first object into the segmentation model, and outputting a prediction sketch result of the CT image of the first object;

and fine-tuning parameters of the segmentation model, so that the difference between the reference sketching result and the prediction sketching result of the CT image of the first object is smaller than a second set threshold value, and the segmentation model subjected to parameter fine-tuning is a personalized segmentation model which is suitable for the first object.

4. The method of claim 1, wherein the difference in image quality is analyzed from four dimensions of noise level, artifacts, tissue boundary sharpness, gray values; the first set threshold includes: noise error threshold, image error threshold, sharpness error threshold, and gray error threshold;

and when the difference between the image quality of the pseudo CT image and the reference image quality is smaller than a noise error threshold, the difference between the image quality of the pseudo CT image and the reference image quality is smaller than an image error threshold, the difference between the image quality of the pseudo CT image and the reference image quality is smaller than a definition error threshold, the difference between the image quality of the pseudo CT image and the reference image quality is smaller than a gray level error threshold, and the difference between the image quality of the pseudo CT image and the reference image quality is smaller than a gray level error threshold.

5. A method of delineating a CBCT image, comprising:

acquiring a CBCT image to be sketched of a target object;

inputting the CBCT image to be sketched into a pre-trained target image generation model, and outputting to obtain a pseudo CT image;

inputting the pseudo CT image into a pre-trained target segmentation model, and outputting to obtain a CBCT sketching result of the target object;

wherein the target image generation model comprises first network parameters mapped from CBCT images to pseudo CT images; the target segmentation model comprises a second network parameter mapped from the CT image to a sketching result of a target area of the CT image; in a training stage of the target image generation model, a CBCT image of a training object is input, and a pseudo CT image of the training object is output; the image quality of the pseudo CT image of the training object is identical to the image quality of the CT image of the training object.

6. The method of claim 5, wherein the step of determining the position of the probe is performed,

the first network parameters of the target image generation model are obtained by: taking CBCT images of a plurality of training objects as training data of a first model, taking reference image quality of the CT images of the plurality of training objects as a training label of the first model, generating a model for the target image by the trained first model, wherein the trained parameters of the first model are the first network parameters;

The second network parameter of the target segmentation model is obtained by one of the following means:

taking CT images of the plurality of training objects as training data of a second model, taking a reference sketching result of target area sketching aiming at the CT images of the plurality of training objects as a training label of the second model, wherein the trained second model is the target segmentation model, and the trained parameters of the second model are the second network parameters; or,

taking CT images of the plurality of training objects as training data of a second model, taking a reference sketching result of target area sketching aiming at the CT images of the plurality of training objects as a training label of the second model, and taking a parameter obtained by training as an intermediate parameter of the second model; and fine-tuning the intermediate parameters of the second model according to the CT image of the target object and the reference sketching result of the CT image of the target object, wherein the fine-tuned second model is the target segmentation model, and the fine-tuned parameters are the second network parameters.

7. An apparatus for constructing a CBCT delineation model, comprising:

the training data and label acquisition module is used for acquiring training data and training labels, and the training data comprises: a CBCT image and a CT image of a plurality of training subjects, the training tag comprising: the reference image quality of the CT image is used for carrying out a reference sketching result of target area sketching aiming at the CT image;

The first training module is used for inputting the CBCT image into a first model for training, the output of the first model is a pseudo CT image, the training is finished under the condition that the difference between the image quality of the pseudo CT image and the reference image quality is smaller than a first set threshold value, and the first model after the training is an image generation model;

the second training module is used for inputting the CT image into a second model for training, the output of the second model is a prediction sketching result, the training is finished under the condition that the difference between the prediction sketching result and the reference sketching result is smaller than a second set threshold value, and the trained second model is a segmentation model;

and the sketch model generating module is used for generating a CBCT sketch model according to the image generating model and the segmentation model.

8. An apparatus for delineating CBCT images, comprising:

the data acquisition module is used for acquiring a CBCT image to be sketched of the target object;

the first processing module is used for inputting the CBCT image to be sketched into a pre-trained target image generation model and outputting to obtain a pseudo CT image;

the second processing module is used for inputting the pseudo CT image into a pre-trained target segmentation model and outputting a CBCT sketching result of the target object;

Wherein the target image generation model comprises first network parameters mapped from CBCT images to pseudo CT images; the target segmentation model comprises a second network parameter mapped from the CT image to a sketching result of a target area of the CT image; in a training stage of the target image generation model, a CBCT image of a training object is input, and a pseudo CT image of the training object is output; the image quality of the pseudo CT image of the training object is identical to the image quality of the CT image of the training object.

9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for implementing the method of any one of claims 1-6 when executing a program stored on a memory.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any of claims 1-6.

Images