CN102592137A - Multi-modality image registration method and operation navigation method based on multi-modality image registration - Google Patents

Abstract

A multimodal image registration method, comprising the following steps: sampling a reference image and an image to be registered according to a first sampling ratio to obtain a reference image with a first resolution and an image to be registered; using a B-spline surface as The deformation model registers the reference image with the first resolution and the image to be registered, calculates a registration measure, and obtains a first registration parameter; increases the sampling ratio and repeats the above steps until the registration measure reaches a preset threshold; Wherein, when the registration is performed after increasing the sampling ratio each time, the previous registration parameter is used as the initial registration parameter. In addition, a surgical navigation method based on multimodal image registration is provided.

Description

Multimodal image registration method and surgical navigation method based on multimodal image registration

【Technical field】

The invention relates to image processing, in particular to a multimodal image registration method and a surgical navigation method based on multimodal image registration.

【Background technique】

In the prior art, domestic interventional navigation systems are mainly aimed at bone and brain surgery, and registration based on ultrasound images. Due to the rigidity of the bone, the registration between the preoperative image and the patient is relatively easy to complete the rigid registration through the landmarks on the bone, and can also achieve better treatment accuracy. However, for surgical operations on soft tissues such as the liver, the displacement and deformation of the liver will inevitably be caused by the action of breathing or surgical instruments. At this point, rigid registration based on feature points is no longer applicable. Moreover, general ultrasound images are noisy and have unclear edges. Due to the limitations of ultrasound image imaging technology, high-precision image registration methods based on ultrasound are still in the research stage. Moreover, the registration speed is the technical focus and difficulty of the elastic registration technology for easily deformable tissues such as the liver.

Therefore, the current interventional navigation system is lacking in real-time registration, provision of three-dimensional image information, surgical path planning, and navigation accuracy. Since different imaging devices use different imaging methods to collect images, each imaging principle can only effectively reflect the information of a certain organ. For example, CT can clearly display human bones, while MRI equipment is good for displaying soft tissues. At present, there is no imaging technology that can comprehensively obtain human organ information and quickly and accurately register images of various organ information.

【Content of invention】

Based on this, it is necessary to provide a fast and accurate multimodal image registration method.

A multimodal image registration method, comprising the following steps:

Sampling the reference image and the image to be registered according to a first sampling ratio to obtain a reference image and an image to be registered with a first resolution;

Using a B-spline surface as a deformation model to register the reference image with the first resolution and the image to be registered, calculate a registration measure, and obtain a first registration parameter;

Increase the sampling ratio and repeat the above steps until the registration measure reaches the preset threshold; wherein, each time the registration is performed after increasing the sampling ratio, the previous registration parameter is used as the initial registration parameter.

Preferably, the step of sampling the reference image and the image to be registered according to the sampling ratio specifically includes:

Normalizing the pixel gray values of the reference image and the image to be registered;

256 gray levels are sampled at a sampling scale.

Preferably, the steps of sampling and registration are repeated once, wherein the first sampling ratio and the repeated sampling ratio are set according to the resolutions of the reference image and the image to be registered.

Preferably, the sampling method adopts one of the following methods: random sampling, equal interval sampling, and pyramid layered sampling.

Preferably, the gradient descent algorithm is used for the registration optimization algorithm, and the search step size of the first sampling and repeated sampling, the relaxation factor in the convergence condition, the maximum number of iterations, and the size of the B-spline surface are set.

Preferably, in the step of calculating the registration measure:

The normalized mutual information value is used as the registration measure.

Preferably, when the registration measure reaches a preset threshold, absolute registration is performed on the reference image and the image to be registered, wherein a B-spline control grid is defined in the absolute registration.

In addition, it is necessary to provide a fast and accurate method for surgical navigation based on multimodal image registration.

A surgical navigation method based on multimodal image registration, used for surgical navigation after registration of a three-dimensional image established from a preoperative CT image and an intraoperative MRI image, comprising the following steps:

Using digital image reconstruction technology to reconstruct preoperative CT images to obtain DRR (digitally reconstructed radiography, digitally reconstructed image) images;

Register the DRR image with the intraoperative MRI image and continuously change the spatial position parameters of the DRR image in the simulated X-ray imaging equipment to obtain the registration measure when the spatial position parameters are different;

The spatial position parameter when the registration measure reaches the threshold is searched by the optimization algorithm, which is the spatial positioning parameter of the DRR image in the simulated X-ray;

Obtain the Euler transformation parameter when the similarity between the DRR image and the intraoperative MRI image is the largest as the coordinate conversion parameter for converting the preoperative CT image data to the MRI image;

performing surgical navigation in combination with the preoperative CT image and the intraoperative MRI image according to the coordinate transformation parameters;

Wherein, the step of registering the DRR image with the MRI image in the operation comprises:

Sampling the DRR image and the intraoperative nuclear magnetic resonance image according to a first sampling ratio to obtain the DRR image and the intraoperative nuclear magnetic resonance image with a first resolution;

Using a B-spline surface as a deformation model to register the DRR image with the first resolution and the intraoperative MRI image, calculate a registration measure, and obtain a first registration parameter;

Increase the sampling ratio and repeat the above registration steps until the registration measure reaches the preset threshold; wherein, each time the registration is performed after increasing the sampling ratio, the previous registration parameter is used as the initial registration parameter.

Preferably, the step of sampling the DRR image and the intraoperative nuclear magnetic resonance image according to the first sampling ratio to obtain the DRR image and the intraoperative nuclear magnetic resonance image with the first resolution specifically includes:

Normalize the pixel gray value of the DRR image and the MRI image in the operation;

The 256 gray levels are sampled at the first sampling ratio.

Preferably, the steps of sampling and registration are repeated once, wherein the first sampling ratio and the repeated sampling ratio are set according to the resolutions of the reference image and the image to be registered.

Preferably, the gradient descent algorithm is used for the registration optimization algorithm, and the search step size of the first sampling and repeated sampling, the relaxation factor in the convergence condition, the maximum number of iterations, and the size of the B-spline surface are set.

Preferably, in the step of calculating the registration measure:

The normalized mutual information value is used as the registration measure.

Preferably, when the registration measure reaches a preset threshold, absolute registration is performed on the DRR image and the intraoperative MRI image, wherein a B-spline control grid is defined in the absolute registration.

Preferably, the step of performing surgical navigation according to the coordinate transformation parameters in combination with the preoperative CT image and the intraoperative MRI image includes:

The preoperative CT image with the largest similarity and the intraoperative MRI image are fused according to the spatial matching relationship, and a surgical navigation path is obtained according to the fused image.

The above multi-modal image registration method adopts a multi-layer resolution image registration implementation method, and at the same time uses a B-spline surface as a deformation model, which can quickly and accurately find the registration measurement threshold of the reference image and the image to be registered , to achieve accurate image registration.

The above-mentioned surgical navigation method based on multi-modal image registration adopts the multi-layer resolution image registration method, and at the same time uses the B-spline surface as the deformation model, which can quickly and accurately find the difference between the reference image and the image to be registered. The registration measure threshold is used to achieve accurate image registration, and the registered images are fused to obtain favorable information in each channel of the data about the same target collected by multi-source channels.

【Description of drawings】

Fig. 1 is the flowchart of multimodal image registration method;

Figure 2 is a schematic diagram of extracting a region of interest;

Fig. 3 is the schematic diagram of reducing the B-spline grid control area;

Figure 4(a) to Figure 4(c) are schematic diagrams of the implementation process of the multimodal image registration method;

Fig. 5 is a flowchart of a surgical navigation method based on multimodal image registration.

【Detailed ways】

Because multi-modal images are images taken from different imaging devices and at different shooting times. Therefore, the reference image, that is, the target image in the preoperative planning system, is not the same as the image to be registered, that is, the intraoperative image. For example, the two images are not on the same layer, or the two images will be deformed or shifted, so timely correction is required . Therefore, the present invention adopts the following multimodal image registration method.

As shown in Figure 1, it is a flowchart of a multimodal image registration method, including the following steps:

Step S110 , sampling the reference image and the image to be registered according to the first sampling ratio to obtain the reference image and the image to be registered with the first resolution.

In this embodiment, the step of sampling the reference image and the image to be registered according to the sampling ratio in step S110 specifically includes:

① Normalize the pixel gray values of the reference image and the image to be registered.

② Sampling 256 gray levels at the sampling ratio.

In this embodiment, the sampling method adopts one of the following methods: random sampling, equal interval sampling, and pyramid layered sampling. Preferably, pyramid layered sampling is used.

Image registration is the process of matching multiple images of the same scene taken at different times, from different sensors, or from different viewpoints. In general, the raw image resolutions of both CT images and MRI can reach various high resolutions. Therefore, the pixel gray value of the image is normalized, the data of all 256 gray levels are sampled in a certain proportion, and the sampled subset is used to replace all pixel data to calculate the similarity measure, that is, each registration adopts a different Image Resolution.

In this embodiment, the three-degree B-spline surface is used as the space conversion function in each registration process with different resolutions, and according to the registration requirements of different scales, the size of each layer can be used in the registration process Different B-spline surfaces are used as deformation models. Preferably, B-spline surfaces of different sizes are used as the deformation model according to the actual resolution of the multimodal image.

Step S120, using a B-spline surface as a deformation model to register the reference image with the first resolution and the image to be registered, calculate a registration measure, and obtain a first registration parameter.

In this embodiment, in the step of calculating the registration measure: the normalized mutual information value is used as the registration measure.

Mutual information is a basic concept in information theory, which is used to describe the statistical correlation between two systems, or the amount of information contained in one system in another system. Among them, in the registration measurement, since the variance and mean square error are the most important and commonly used indicators to measure the degree of data variation, the normalized form is often used.

In this embodiment, the gradient descent algorithm is used for the registration optimization algorithm, and the search step size of the first sampling and repeated sampling, the relaxation factor in the convergence condition, the maximum number of iterations, and the size of the B-spline surface are set.

In this embodiment, the registration optimization algorithm for the first sampling adopts the gradient descent algorithm, in which the search step size is 10 mm, the relaxation factor in the convergence condition is 0.7, the maximum number of iterations is 200, and a 5×5 B-spline is used The curved surface is used as the deformation model; the registration optimization algorithm for repeated sampling adopts the LBFG algorithm, in which the search step is 0.1mm, the gradient convergence tolerance is 0.05, the maximum number of iterations is 1000, and a 10×10 B-spline surface is used as the deformation model . The first registration parameters include: pixels, shrinkage coefficients, and amplitude lengths.

Step S130 , increasing the sampling ratio and repeating the above steps until the registration measure reaches the preset threshold; wherein, each time the registration is performed after increasing the sampling ratio, the previous registration parameter is used as the initial registration parameter.

Using the previous registration parameters as the next initial registration parameters can improve the continuity and stability of the two registrations.

In this embodiment, the steps of sampling and registration are repeated once, wherein the first sampling ratio and the repeated sampling ratio are set according to the resolutions of the reference image and the image to be registered.

In this embodiment, the steps of sampling and registration are repeated once, wherein the first sampling ratio is 0.3, and the repeated sampling ratio is 0.6.

As shown in Table 1, it is the image registration result, including the mutual information value before registration, the first registration result and the second registration result, and the comparison between the registration result after registration and the mutual information of the reference image.

Mutual information value before registration The first registration result The second registration result 1 1.24257 1.27905 1.28651

Table 1

Table 2 shows the registration results of CT (512*512) and MRI (512*416) images of two different modalities, where the maximum mutual information value is used as the similarity measure. The registration output parameters include the mutual information value of the reference image and the floating image before registration, the comparison of the mutual information value between the first registration result and the second registration result and the reference image, and the time required for registration. From the comparison of the mutual information value before and after registration, it can be seen that the mutual information value has been greatly improved, indicating that the consistency between the deformed floating image and the reference image has improved. This registration method has a certain registration effect for multi-modal images.

Mutual information value before registration The first registration result The second registration result registration time 1 0.666575 0.924476 0.924923 208

Table 2

Table 3 shows the results of experiments designed to compare the registration speed of the registration strategy in this embodiment with the traditional single-layer registration method. The traditional single registration method in this experiment uses the same registration method, including the registration strategy of using B-spline surface as the deformation model, mutual information as the similarity measure and LBFG optimization algorithm to find the optimal value. On this basis, it can be considered that the registration convergence conditions of the two methods are the same, that is, the registration accuracy is equivalent. When the same registration convergence conditions are set, the registration time required for the single registration method and the multi-layer registration method in this paper is shown in the following table:

Time(S)CT/MRI Traditional method multi-resolution 1 451 208

table 3

When the registration measure reaches the preset threshold, absolute registration is performed between the reference image and the image to be registered, wherein a B-spline control grid is defined in the absolute registration. When the difference between the registration parameters of the reference image and the image to be registered is within ±5%, it is considered that the registration measure reaches the preset threshold. Specifically, the method steps of adopting the B-spline grid to control the region include:

① Select the region of interest in the reference image, obtain its pixel information, and map the pixel information to the corresponding region of the image to be registered, while controlling the change of the B-spline grid.

Figure 2 is a schematic diagram of extracting the region of interest. The region of interest 10 is a deformed region in the reference image, and the region of interest 10 is extracted and mapped to the corresponding region of the image to be registered to control the change of the B-spline grid. Preferably, the region of interest 10 is reduced in a B-spline grid.

②Optimize and calculate the changed grid control area, and embed it into the image to be registered after reaching the matching threshold.

As shown in Figure 3, it is a schematic diagram of reducing the control area of the B-spline grid. The deformed area 20 is a larger deformed area in the reference image, and the reduced area 22 is a changed grid control area. Perform optimization calculation on the reduced area 22, and embed it into the image to be registered after reaching the matching threshold. The specific implementation process is shown in Figure 4: Figure 4a is a reference image. Figure 4b is the image to be registered. Figure 4c is the registration result.

Next, in order to compare the registration speed of the multi-layer registration strategy of multi-modal images and the traditional single-layer registration method, the following experiments are designed. The traditional single-layer registration method in this experiment uses the same registration parameters as the layered registration method, including the same registration parameters that use B-spline surface as the deformation model, mutual information as the registration measure, and LBFG optimization algorithm to find the optimal value. quasi-strategy. On this basis, it can be considered that the registration convergence conditions of the two methods are the same, that is, the registration accuracy is equivalent. When the same registration convergence conditions are set, the time required for registration of the single-layer registration method and the multi-layer registration method of multi-modal images is shown in Table 4.

multimodal image Traditional method registration time (s) Hierarchical registration time (s) 1 451 208 2 100 28 3 150 15

Table 4

In the B-spline order, control the B-spline grid spacing unchanged, the above method improves the calculation speed and accuracy, and is more suitable for correcting images with large local deformations.

Based on all the above-mentioned embodiments, as shown in FIG. 5 , it is a flow chart of a surgical navigation method based on multi-modal image registration, which is used for registration of a three-dimensional image established from a preoperative CT image and an intraoperative MRI image. The subsequent surgical navigation includes the following steps:

Step S210, using digital image reconstruction technology to reconstruct the preoperative CT image to obtain a DRR image.

Digital image reconstruction technology (DRR) is used to reconstruct the preoperative CT image, that is, the preoperative CT tomographic data is used to reconstruct the affine photographic image, and the physical process of the actual imaging is simulated.

In this embodiment, the purpose of reconstructing the preoperative CT image is to realize the three-dimensional reconstruction of the CT slice image by the three-dimensional preoperative planning system.

Step S220, registering the DRR image with the intraoperative MRI image and constantly changing the spatial position parameters of the DRR image in the simulated X-ray imaging device to obtain a registration measure when the spatial position parameters are different.

In this embodiment, Euler rotation is used to control parameters of six degrees of freedom such as rotation and translation of the three-dimensional image. Therefore, in the process of resampling, DRR images of different angles can be obtained by changing the Euler transformation parameters.

In this embodiment, the step of registering the DRR image with the intraoperative MRI image includes:

① Sampling the DRR image and the intraoperative MRI image according to the first sampling ratio to obtain the DRR image and the intraoperative MRI image with the first resolution.

② Using a B-spline surface as a deformation model to register the DRR image with the first resolution and the intraoperative MRI image, calculate a registration measure, and obtain a first registration parameter.

③Increase the sampling ratio and repeat the above registration steps until the registration measure reaches the preset threshold; wherein, when the registration is performed after increasing the sampling ratio each time, the previous registration parameters are used as the initial registration parameters.

In this embodiment, the DRR image and the intraoperative MRI image are processed according to the first sampling ratio

For example, the steps of sampling to obtain a DRR image with a first resolution and an intraoperative nuclear magnetic resonance image specifically include:

① Normalizing the pixel gray values of the DRR image and the intraoperative nuclear magnetic resonance image.

② Sampling 256 gray levels at the sampling ratio.

In this embodiment, the steps of sampling and registration are repeated once, wherein the first sampling ratio and the repeated sampling ratio are set according to the resolutions of the reference image and the image to be registered.

In this embodiment, the steps of sampling and registration are repeated once, wherein the first sampling ratio is 0.3, and the repeated sampling ratio is 0.6.

Commonly used resampling methods include nearest neighbor method, bilinear interpolation method and cubic convolution interpolation method. Among them, the nearest neighbor method is the simplest and has a fast calculation speed, but the visual effect is poor; bilinear interpolation will blur the image outline; the image produced by the cubic convolution method is smoother and has good visual effects, but the calculation is large and time-consuming . In this embodiment, in order to obtain an image with a good visual effect, a cubic convolution method is generally used for resampling.

In this embodiment, the gradient descent algorithm is used for the registration optimization algorithm, and the search step size of the first sampling and repeated sampling, the relaxation factor in the convergence condition, the maximum number of iterations, and the size of the B-spline surface are set.

In this embodiment, the registration optimization algorithm for the first sampling adopts the gradient descent algorithm, in which the search step size is 10 mm, the relaxation factor in the convergence condition is 0.7, the maximum number of iterations is 200, and a 5×5 B-spline is used The surface is used as the deformation model; the LBFG algorithm is used for the registration optimization algorithm during repeated sampling, in which the search step is 0.1mm, the gradient convergence tolerance is 0.05, the maximum number of iterations is 1000, and a 10×10 B-spline surface is used as the deformation Model.

In this embodiment, when the registration measure reaches the preset threshold, absolute registration will be performed on the 3D image created based on the preoperative CT image and the intraoperative MRI image, wherein the B-spline control grid is defined in the absolute registration .

In step S230, the spatial position parameter when the registration measure reaches the threshold is searched by the optimization algorithm, which is the spatial positioning parameter of the DRR image in the simulated X-ray.

Step S240, taking the Euler transformation parameter when the similarity between the DRR image and the intraoperative MRI image is the largest as the coordinate conversion parameter for converting the preoperative CT image data into the MRI image.

In this embodiment, when the similarity between the DRR image and the intraoperative MRI image is the largest, it can be considered that the parameters of the Euler transformation correspond to the coordinate transformation matrix between the intraoperative MRI image and the preoperative CT image.

Step S250, performing surgical navigation according to the coordinate transformation parameters combined with the preoperative CT image and the intraoperative MRI image.

In this embodiment, after the three-dimensional image established based on the preoperative CT image and the intraoperative MRI image are fused according to the spatial matching relationship, an image fully displaying bones and soft tissues can be obtained. After obtaining images with comprehensive information, the robotic arm is controlled according to the spatial coordinate relationship to complete the interventional treatment and complete the treatment navigation process.

The above method can provide preoperative planning, teaching training and mechanical arm assistance to complete the treatment.

The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.

Claims (14)

1. A multimodal image registration method, comprising the following steps: Sampling the reference image and the image to be registered according to a first sampling ratio to obtain a reference image and an image to be registered with a first resolution; Using a B-spline surface as a deformation model to register the reference image with the first resolution and the image to be registered, calculate a registration measure, and obtain a first registration parameter; Increase the sampling ratio and repeat the above steps until the registration measure reaches the preset threshold; wherein, each time the registration is performed after increasing the sampling ratio, the previous registration parameter is used as the initial registration parameter. 2. The multimodal image registration method according to claim 1, wherein the step of sampling the reference image and the image to be registered according to the sampling ratio specifically comprises: Normalizing the pixel gray values of the reference image and the image to be registered; 256 gray levels are sampled at a sampling scale. 3. The multimodal image registration method according to claim 1, wherein the steps of sampling and registration are repeated once, wherein the first sampling ratio and the first sampling ratio are set according to the resolution of the reference image and the image to be registered. Resampling ratio. 4. The multimodal image registration method according to any one of claims 1 to 3, characterized in that the sampling method adopts one of the following methods: random sampling, equal interval sampling and pyramid layered sampling. 5. The multimodal image registration method according to claim 3, characterized in that, for the registration optimization algorithm, a gradient descent algorithm is used to set the first sampling and the search step size of repeated sampling, the relaxation factor in the convergence condition, Maximum number of iterations, B-spline surface size. 6. The multimodal image registration method according to claim 1, wherein in the step of calculating the registration measure: The normalized mutual information value is used as the registration measure. 7. The multimodal image registration method according to claim 1, wherein when the registration measure reaches a preset threshold, absolute registration is performed between the reference image and the image to be registered, wherein the absolute registration Define the B-spline control mesh in . 8. A surgical navigation method based on multimodal image registration, used for surgical navigation after registration based on a three-dimensional image established from a preoperative CT image and an intraoperative nuclear magnetic resonance image, comprising the following steps: Digital image reconstruction technology is used to reconstruct preoperative CT images to obtain DRR images; Register the DRR image with the intraoperative MRI image and continuously change the spatial position parameters of the DRR image in the simulated X-ray imaging equipment to obtain the registration measure when the spatial position parameters are different; The spatial position parameter when the registration measure reaches the threshold is searched by the optimization algorithm, which is the spatial positioning parameter of the DRR image in the simulated X-ray; Take the Euler transformation parameter when the similarity between the DRR image and the intraoperative MRI image is the largest as the coordinate conversion parameter for converting the preoperative CT image data to the MRI image; performing surgical navigation in combination with the preoperative CT image and the intraoperative MRI image according to the coordinate transformation parameters; Wherein, the step of registering the DRR image with the MRI image in the operation comprises: Sampling the DRR image and the intraoperative nuclear magnetic resonance image according to a first sampling ratio to obtain the DRR image and the intraoperative nuclear magnetic resonance image with a first resolution; Using a B-spline surface as a deformation model to register the DRR image with the first resolution and the intraoperative MRI image, calculate a registration measure, and obtain a first registration parameter; Increase the sampling ratio and repeat the above registration steps until the registration measure reaches the preset threshold; wherein, each time the registration is performed after increasing the sampling ratio, the previous registration parameter is used as the initial registration parameter. 9. The surgical navigation method based on multimodal image registration according to claim 8, wherein the DRR image and the intraoperative nuclear magnetic resonance image are sampled and acquired according to a first sampling ratio with a first resolution The steps of DRR image and intraoperative MRI image specifically include: Normalize the pixel gray value of the DRR image and the MRI image in the operation; The 256 gray levels are sampled at the first sampling ratio. 10. The surgical navigation method based on multimodal image registration according to claim 8, wherein the steps of sampling and registration are repeated once, wherein the second resolution is set according to the resolution of the reference image and the image to be registered A sampling ratio and a resampling ratio. 11. The surgical navigation method based on multimodal image registration according to claim 10, characterized in that, the gradient descent algorithm is adopted for the registration optimization algorithm, and the search steps and convergence conditions of the first sampling and repeated sampling are set. The relaxation factor, the maximum number of iterations, and the size of the B-spline surface. 12. The surgical navigation method based on multimodal image registration according to claim 8, wherein in the step of calculating the registration measure: The normalized mutual information value is used as the registration measure. 13. The surgical navigation method based on multimodal image registration according to claim 8, wherein when the registration measure reaches a preset threshold, the DRR image and the intraoperative MRI image are absolutely registered , where the B-spline control mesh is defined in absolute registration. 14. The surgical navigation method based on multimodal image registration according to claim 8, wherein the surgical navigation is performed according to the coordinate transformation parameters in combination with the preoperative CT image and the intraoperative MRI image The steps include: The preoperative CT image with the largest similarity and the intraoperative MRI image are fused according to the spatial matching relationship, and a surgical navigation path is obtained according to the fused image.

Images