CN118470222B - Medical ultrasonic image three-dimensional reconstruction method and system based on SDF diffusion - Google Patents

Abstract

The present invention discloses a method and system for three-dimensional reconstruction of medical ultrasound images based on SDF diffusion, and belongs to the field of computer vision technology. First, a visual model pre-trained on medical data is used to extract features of an input ultrasound image. Then, the SDF diffusion process is started, and the reverse diffusion process starts with completely random noise, and gradually denoises to obtain a clean SDF field, wherein the SDF features and ultrasound image features are fused using a state space model and cross attention to achieve the consistency of the SDF field representation of the three-dimensional surface with the ultrasound image. After actual verification, the medical ultrasound image three-dimensional reconstruction method based on SDF diffusion provided by the present invention has the characteristics of high efficiency and high precision.

Description

A method and system for three-dimensional reconstruction of medical ultrasound images based on SDF diffusion

Technical Field

The invention relates to a method and system for three-dimensional reconstruction of medical ultrasonic images based on SDF diffusion, and belongs to the technical field of computer vision.

Background Art

Ultrasound image 3D reconstruction refers to the process of acquiring and processing a series of 2D ultrasound image data to ultimately generate 3D anatomical structure images or volume data, which has important clinical significance and application value. The 3D shape of an object can be expressed in a variety of forms, such as voxels, point clouds, grids, etc. Voxel-based representation methods can use 3D convolutional neural networks (CNNs), which can reconstruct objects with arbitrary topological structures. However, the huge memory requirements and computing time limit most methods to high-resolution reconstruction results, so ultra-high precision reconstruction cannot be achieved. Point cloud representation is relatively simple and highly flexible, but because point clouds are not regular structures, they cannot adapt well to traditional 3D CNN networks. Similarly, the network can represent 3D shapes with high precision, but it is also discrete and cannot be well represented by traditional neural networks.

Diffusion denoising probabilistic model is a class of generative models based on iterative inversion of Markov noise process. In vision, early works formulated the problem as learning variational lower bounds, or framed it as optimizing score-based generative models or discretization of continuous random processes. Many recent works have shown the great potential of diffusion models in content generation tasks, so using diffusion for 3D reconstruction tasks will also give full play to the advantages of the model.

In the field of three-dimensional reconstruction of ultrasound images, traditional methods include position sensor-based methods, which use position sensors (such as magnetic position sensors, optical position sensors, etc.) connected to the probe to track the position and posture of the probe in three-dimensional space in real time, and store the spatial position information of the two-dimensional ultrasound image when it is acquired. Then, based on this position information, multiple two-dimensional images are reconstructed into three-dimensional data. However, this method requires an external position sensor system, which greatly increases the complexity and cost of the system. There are also problems such as cable interference and probe occlusion that affect the accuracy. The method based on free hand scanning uses the motion trajectory of the operator's free movement of the ultrasound probe when scanning the human body surface and the scanning direction of the ultrasound beam to estimate the spatial position relationship of each two-dimensional image, thereby reconstructing three-dimensional data. However, it requires high-precision probe motion trajectory and beam direction estimation, which is not only complex in calculation but also has poor real-time performance. The method based on image registration extracts grayscale, gradient, feature and other information from a series of adjacent two-dimensional ultrasound images, performs registration through similarity measurement, estimates the geometric transformation relationship between images, and finally obtains three-dimensional data. However, it has high requirements on image quality and uniformity of grayscale distribution in the field of view, is easily affected by accuracy, and is complex in calculation.

By analyzing and summarizing the existing ultrasound 3D reconstruction methods, the existing methods have the following shortcomings: (1) Additional equipment or complex algorithms are required to determine the probe motion trajectory and direction, which increases the difficulty and cost of implementation. (2) The reconstruction process requires continuous ultrasound video or a series of continuous ultrasound images, which requires a large amount of data and complex calculations. (3) The reconstruction process is time-consuming and has average accuracy.

Summary of the invention

The purpose of the present invention is to propose a medical ultrasound image three-dimensional reconstruction method based on SDF (Signed Distance Field) diffusion to improve the speed and accuracy of three-dimensional reconstruction of two-dimensional ultrasound images.

SDF is an implicit representation method of the three-dimensional shape of an object. The essence of SDF is to store the shortest distance from each point to the surface of the shape, that is, to draw a surface out of the model. The point values on the outside of the model surface are greater than 0, and the point values on the inside of the model surface are less than 0. SDF can be conveniently used in 3D convolutional networks, and can also be converted into a high-precision grid representation through the marching cube method.

In order to achieve the above-mentioned invention object, the specific technical scheme adopted by the present invention is as follows:

A method for three-dimensional reconstruction of medical ultrasound images based on SDF diffusion comprises the following steps:

S1: Acquire medical ultrasound image data and extract features of the ultrasound image to guide the subsequent diffusion process;

S2: Randomly initialize the SDF field and set the SDF field Initialize randomly according to normal distribution. ;

S3: The diffusion model is used to perform the SDF diffusion process. The diffusion model includes two key processes: forward diffusion process and backward diffusion process. The state space model is first used to fuse the voxel and ultrasound image features to obtain the voxel features after the fusion of ultrasound image features. Then, the cross attention is used to fuse the voxel features and ultrasound image features at a stage with a small voxel resolution.

S4: The mesh is extracted using marching cubes and the triangular mesh model guided by the ultrasound image is finally reconstructed.

Furthermore, in S1, a large amount of medical ultrasound image data is first used to train the visual model MedSAM (Medical Segment Anything Model), and then the trained MedSAM image encoder is used to extract features from the input single medical ultrasound image, and the output is mapped to a feature vector , which is used to guide the SDF to approach the image features in the subsequent diffusion process to achieve SDF 3D reconstruction of ultrasound images.

Furthermore, in S2, the SDF field is a continuous three-dimensional scene representation method, which regards a three-dimensional object or scene as a three-dimensional scalar field composed of distance values. Specifically, for any point in the scene, the SDF value is defined as the signed distance from the point to the surface of the object, that is, if the point is inside the object, the SDF value is the negative distance value from the point to the nearest surface; if the point is outside the object, the SDF value is the positive distance value from the point to the nearest surface; if the point is exactly on the surface of the object, the SDF value is 0.

Furthermore, the S3 specifically includes:

S3-1: The forward diffusion process is a process used during training. It gradually adds Gaussian noise to the data until it is completely destroyed and becomes pure noise with Gaussian distribution. This process can be represented by a Markov chain. A certain amount of Gaussian noise is added at each time step t until the final time T, when the original data becomes pure noise. Given a data sample , forward process It gradually transforms the data samples into pure Gaussian noise:

(1);

in represents the sample at time step t, represents the sample of the previous time step, is the Gaussian transition probability:

(2);

in represents a normal distribution, represents the noise scheduling hyperparameter, which is a learnable coefficient or set to a constant; the forward process allows sampling at any time step t in closed form :

(3);

in , ;therefore, Directly sample through the following formula:

(4);

Among them is a noise randomly sampled from a normal distribution.

S3-2: The reverse diffusion process occurs at generation time and aims to reconstruct the original data from pure noise samples, i.e., the joint distribution , the reverse process is defined as a Markov chain with learned Gaussian transitions:

(5);

(6);

in is the standard normal distribution, Indicated by Parameterized denoising function, is a time step dependent variance, set to ;from Then, we draw the for:

(7);

(8);

in is a A parameterized neural network that starts with a noisy input By repeating this process, we can eventually generate ;

Therefore, the objective function compares the prediction noise With applied noise The difference between:

(9);

In addition, the neural network prediction Rather than noise To modify :

(10);

in, , , Is A parameterized neural network that predicts noise-free data ; In this case, the objective function becomes:

(11);

Specifically, given a clean data sample , according to formula (4) in the forward process, we sample the same shape :

(12);

Then, train a network To predict noise-free data, the MSE objective function is as follows:

(13);

Among them That is, the ultrasound image features extracted in S1;

During inference, the trained neural network is used go through The process of gradually predicting smaller noise samples according to formula (14) To generate new SDF voxels:

(14);

S3-3: The diffusion process uses a U-shaped network, where the voxel feature resolution of each layer is halved (for example: 64→32→16) and the channels are doubled; in order to make the object represented by the generated SDF field consistent with the ultrasound image, it is necessary to fuse the SDF features and the ultrasound image features during the diffusion process, so that the ultrasound image features can guide the SDF diffusion process; at the top of the model, due to the large feature resolution, the voxel features are kept at a large order of magnitude, and the cross-attention cannot be directly used to fuse the voxel and ultrasound image features, so the state space model is used to process them;

The state space model is a mathematical model framework commonly used in time series analysis and control theory. It describes the evolution of a dynamic system as a combination of a state equation and an observation equation. Through a hidden state Map to , this system uses A as the evolution parameter, and B and C as the projection parameters:

(15);

(16);

The Mamba used is a discrete version of the continuous system, which includes a time scale parameter Convert continuous parameters A and B into discrete parameters , ; The commonly used transformation method is zero-order hold (ZOH), which is defined as follows:

(17);

(18);

exist , After discretization, formulas (15) and (16) can be expressed as:

(19);

(20);

Finally, the model calculates the output through global convolution:

(twenty one);

(twenty two);

where M is the length of the input sequence x, is a structured convolution kernel.

In order to apply it to the SDF diffusion process, the voxel features are expanded into one-dimensional features, and the time step t in S3-1 is linearly mapped to a one-dimensional feature of the same size as the voxel feature after sine and cosine transformation and added to the voxel feature. After linear mapping to the same number of channels as the voxel feature, it is connected with the voxel feature to obtain the input x of the SSM.

After the above-mentioned calculation by Mamba, the output y is obtained. After the voxel feature part is taken out, the one-dimensional sequence is reshaped back to the original shape of the voxel feature to obtain the voxel feature after fusion of the ultrasound image feature.

S3-4: After the voxel feature passes through the top of the model, it is downsampled to reduce the resolution of the voxel feature, increase the number of channels, and reduce the overall data volume. At the stage with smaller voxel resolution, cross attention is used to fuse the voxel feature and ultrasound image feature.

The attention function is described as mapping a query and a set of key-value pairs to an output, where the query, key, value, and output are all vectors, and the output is the weighted sum of the values. In practice, the attention function of a set of queries is calculated simultaneously, packaged into a matrix Q, and the keys and values are also packaged into matrices K and V; the output matrix is calculated as:

(twenty three);

Multi-head attention allows the model to jointly focus on information in different representation subspaces at different positions; its formula is as follows:

(twenty four);

(25);

Where concat represents the concatenation operation. are all learnable weight matrices.

For voxels in the grid , project its center onto the image and get the projection coordinates ; Select Close The adjacent image blocks are interactions, because their characteristics are likely to affect Controlled local geometry; neighborhood image patch sets are used Indicates that the selection is as follows: If and The distance between centers is less than the distance threshold , then the patch belong .

Use multi-head self-attention Voxel features at and belongs to Ultrasound image patch feature set of patches in The feature interactions between are modeled as follows:

(26);

(27);

here, It is a standard multi-head attention operation. , , is a learnable matrix, M is the mask for attention computation induced by view projection, using absolute position encoding for voxels and ultrasound image patches.

After cross attention, the ultrasound image features are further fused with the voxel features. At this time, the output voxel features are high-precision SDF fields.

Furthermore, the marching cube algorithm in S4 is a classic algorithm for extracting isosurfaces (isosurfaces) from three-dimensional scalar fields, and is widely used in volume data visualization, geometric reconstruction and other fields. Algorithm steps: Set an isosurface threshold T and traverse all cube cells C in the SDF field:

(1) Calculate the vertex state code index based on the relationship between the 8 vertex scalar values of C and T;

(2) Search the corresponding intersection configuration pattern in the predefined situation table according to the index;

(3) Use linear interpolation to calculate the intersection points with the isosurfaces on the 12 edges of the cube;

(4) Connect the corresponding intersection points according to the pattern to form one or more triangles;

(5) Add triangles to the mesh surface data.

A medical ultrasound image 3D reconstruction system based on SDF diffusion, which consists of an ultrasound image feature extraction module, an SDF diffusion module, and a grid generation module;

The ultrasound image feature extraction module: This module is used to extract the features of the input ultrasound picture, and uses the image encoder of the visual model SAM-Med2D that has been pre-trained with a large amount of medical image data to extract features from the input ultrasound image, providing rich and accurate feature guidance for the subsequent diffusion process.

The SDF diffusion module: This module consists of a U-shaped neural network embedded with a state space model and cross attention, which is used to perform the SDF diffusion process, starting from random noise and gradually denoising to obtain a clean SDF field. During the diffusion process, the state space model and cross attention are used to fuse the voxel features and the ultrasound image features to achieve the purpose of keeping the reconstructed grid consistent with the ultrasound image features.

The mesh generation module: This module executes the marching cubes algorithm to extract a high-precision three-dimensional mesh from the reconstructed SDF field.

Advantages and beneficial effects of the present invention:

The present invention introduces the state space model and cross attention into the SDF diffusion process, fully integrates the ultrasound image features with the SDF features, and realizes an end-to-end fast and accurate ultrasound image 3D reconstruction. To a certain extent, it solves the problems that ultrasound image 3D reconstruction requires additional equipment or complex algorithms to determine the probe motion trajectory and direction, and takes a long time.

Practical verification has shown that the ultrasonic image three-dimensional reconstruction method provided by the present invention has the characteristics of high efficiency and high precision.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an overall flow chart of the present invention.

FIG. 2 is a framework diagram of the present invention.

FIG. 3 is a diagram showing the results of the present invention.

DETAILED DESCRIPTION

The present invention will be further described below in conjunction with Figures 1-3 and specific embodiments.

Embodiment 1:

A method for 3D reconstruction of medical ultrasound images based on SDF diffusion, the overall process is shown in FIG1 , and includes the following steps:

S1: Use the image encoder of SAM-Med2D, a visual model pre-trained with a large amount of medical image data, to extract features from the input single view and output a feature vector , in the subsequent diffusion process, it is fused with the SDF features in the state space model and cross attention to achieve the purpose of guiding 3D reconstruction, so as to realize 3D reconstruction of ultrasound images.

S2: Randomly initialize the voxel SDF field, where SDF stands for Signed Distance Function. The SDF field is a continuous three-dimensional scene representation that treats a three-dimensional object or scene as a three-dimensional scalar field composed of distance values. Specifically, for any point in the scene, the value of SDF is defined as the signed distance from the point to the surface of the object, that is: if the point is inside the object, the SDF value is the negative distance from the point to the nearest surface; if the point is outside the object, the SDF value is the positive distance from the point to the nearest surface; if the point is exactly on the surface of the object, the SDF value is 0. During initialization, the SDF field is initialized. Initialize randomly according to normal distribution. .

S3: Diffusion process is performed on the SDF field. Diffusion Models is an emerging deep generative model that uses a reversible diffusion process to train a neural network to generate the required data from the noise distribution. The generative diffusion model contains two key processes: forward diffusion process and reverse diffusion process. Specifically, it includes:

S3-1: The forward diffusion process is a process used during training. It gradually adds Gaussian noise to the data until it is completely destroyed and becomes pure noise with Gaussian distribution. This process can be represented by a Markov chain. A certain amount of Gaussian noise is added at each time step t until the final time T, when the original data becomes pure noise. Given a data sample , forward process It gradually transforms the data samples into pure Gaussian noise:

(1);

in represents the sample at time step t, represents the sample of the previous time step, is the Gaussian transition probability:

(2);

in represents a normal distribution, represents the noise scheduling hyperparameter, which is a learnable coefficient or set to a constant; the forward process allows sampling at any time step t in closed form :

(3);

in , ;therefore, Directly sample through the following formula:

(4);

Among them is a noise randomly sampled from a normal distribution.

S3-2: The reverse diffusion process occurs at generation time and aims to reconstruct the original data from pure noise samples, i.e., the joint distribution , the reverse process is defined as a Markov chain with learned Gaussian transitions:

(5);

(6);

in is the standard normal distribution, Indicated by Parameterized denoising function, is a time step dependent variance, set to ;from Then, we draw the for:

(7);

(8);

in is a A parameterized neural network that starts with a noisy input By repeating this process, we can eventually generate ;

Therefore, the objective function compares the prediction noise With applied noise The difference between:

(9);

In addition, the neural network prediction Rather than noise To modify :

(10);

in, , , Is A parameterized neural network that predicts noise-free data ; In this case, the objective function becomes:

(11);

Specifically, given a clean data sample , according to formula (4) in the forward process, we sample the same shape :

(12);

Then, train a network To predict noise-free data, the MSE objective function is as follows:

(13);

Among them That is, the ultrasound image features extracted in S1;

During inference, the trained neural network is used go through The process of gradually predicting smaller noise samples according to formula (14) To generate new SDF voxels:

(14);

S3-3: The diffusion process uses a U-shaped network, where the voxel feature resolution of each layer is halved (for example: 64→32→16) and the channels are doubled; in order to make the object represented by the generated SDF field consistent with the ultrasound image, it is necessary to fuse the SDF features and the ultrasound image features during the diffusion process, so that the ultrasound image features can guide the SDF diffusion process; at the top of the model, due to the large feature resolution, the voxel features are kept at a large order of magnitude, and the cross-attention cannot be directly used to fuse the voxel and ultrasound image features, so the state space model is used to process them;

The state space model is a mathematical model framework commonly used in time series analysis and control theory. It describes the evolution of a dynamic system as a combination of a state equation and an observation equation. Through a hidden state Map to , this system uses A as the evolution parameter, and B and C as the projection parameters:

(15);

(16);

The Mamba used is a discrete version of the continuous system, which includes a time scale parameter Convert continuous parameters A and B into discrete parameters , ; The commonly used transformation method is zero-order hold (ZOH), which is defined as follows:

(17);

(18);

exist , After discretization, formulas (15) and (16) can be expressed as:

(19);

(20);

Finally, the model calculates the output through global convolution:

(twenty one);

(twenty two

where M is the length of the input sequence x, is a structured convolution kernel.

In order to apply it to the SDF diffusion process, the voxel features are expanded into one-dimensional features, and the time step t in S3-1 is linearly mapped to a one-dimensional feature of the same size as the voxel feature after sine and cosine transformation and added to the voxel feature. After linear mapping to the same number of channels as the voxel feature, it is connected with the voxel feature to obtain the input x of the SSM.

After the above-mentioned calculation by Mamba, the output y is obtained. After the voxel feature part is taken out, the one-dimensional sequence is reshaped back to the original shape of the voxel feature to obtain the voxel feature after fusion of the ultrasound image feature.

S3-4: After the voxel feature passes through the top of the model, it is downsampled to reduce the resolution of the voxel feature, increase the number of channels, and reduce the overall data volume. At the stage with smaller voxel resolution, cross attention is used to fuse the voxel feature and ultrasound image feature.

The attention function is described as mapping a query and a set of key-value pairs to an output, where the query, key, value, and output are all vectors, and the output is the weighted sum of the values. In practice, the attention function of a set of queries is calculated simultaneously, packaged into a matrix Q, and the keys and values are also packaged into matrices K and V; the output matrix is calculated as:

(twenty three);

Multi-head attention allows the model to jointly focus on information in different representation subspaces at different positions; its formula is as follows:

(twenty four);

(25);

Where concat represents the concatenation operation. are all learnable weight matrices.

For voxels in the grid , project its center onto the image and get the projection coordinates ; Select Close The adjacent image blocks are interactions, because their characteristics are likely to affect Controlled local geometry; neighborhood image patch sets are used Indicates that the selection is as follows: If and The distance between centers is less than the distance threshold , then the patch belong .

Use multi-head self-attention Voxel features at and belongs to Ultrasound image patch feature set of patches in The feature interactions between are modeled as follows:

(26);

(27);

here, It is a standard multi-head attention operation. , , is a learnable matrix, M is the mask for attention computation induced by view projection, using absolute position encoding for voxels and ultrasound image patches.

After cross attention, the ultrasound image features are further fused with the voxel features. At this time, the output voxel features are high-precision SDF fields.

S4: Execute the marching cube algorithm to extract the grid from the SDF field. Algorithm steps: Set an isosurface threshold T and traverse all cube cells C in the SDF field:

(1) Calculate the vertex state code index based on the relationship between the 8 vertex scalar values of C and T;

(2) Search the corresponding intersection configuration pattern in the predefined situation table according to the index;

(3) Use linear interpolation to calculate the intersection points with the isosurfaces on the 12 edges of the cube;

(4) Connect the corresponding intersection points according to the pattern to form one or more triangles;

(5) Add triangles to the mesh surface data;

The final reconstructed high-precision triangular mesh model guided by the ultrasound image is obtained.

Embodiment 2: This embodiment uses embodiment 1 as the basic method to carry out module design.

A medical ultrasound image 3D reconstruction system based on SDF diffusion is composed of an ultrasound image feature extraction module, an SDF diffusion module, and a grid generation module, as shown in FIG2 . The following describes each part in detail:

Ultrasound image feature extraction module: This module is used to extract the features of the input ultrasound image. It uses the image encoder of the visual model SAM-Med2D, which has been pre-trained with a large amount of medical image data, to extract features from the input ultrasound image, providing rich and accurate feature guidance for the subsequent diffusion process.

SDF diffusion module: This module consists of a U-shaped neural network embedded with a state-space model and cross-attention, which is used to perform the SDF diffusion process, starting from random noise and gradually denoising to obtain a clean SDF field. During the diffusion process, the state-space model and cross-attention are used to fuse the voxel features and ultrasound image features to achieve the purpose of keeping the reconstructed grid consistent with the ultrasound image features.

Mesh Generation Module: This module executes the marching cubes algorithm to extract a high-precision 3D mesh from the reconstructed SDF field.

Embodiment 3: This embodiment is verified by example based on the above method and system:

In order to verify the accuracy and efficiency of the three-dimensional reconstruction of ultrasound images of the present invention, experiments were conducted on the USOVA3D dataset.

This embodiment is verified by examples based on the above method and system. In order to verify the accuracy of the SDF three-dimensional reconstruction of ultrasound images proposed by the present invention, the test is conducted on the USOVA3D data set. The result shown in Figure 3 is obtained. The method of the present invention (Unet-mamba) can generate three-dimensional shapes based on the input ultrasound image well, and the output time is short. The model constructed by the present invention has high accuracy and fast speed in the SDF three-dimensional reconstruction of ultrasound images.

The above plans are only implementation methods of the present invention, but the protection scope of the present invention is not limited thereto. All replacements or changes that can be understood and thought of by persons familiar with the technology within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims (4)

1. The medical ultrasonic image three-dimensional reconstruction method based on SDF diffusion is characterized by comprising the following steps of:

s1: acquiring medical ultrasonic image data, extracting characteristics of the ultrasonic image, and guiding a subsequent diffusion process;

s2: randomly initializing SDF field, and randomly initializing SDF field V according to normal distribution

S3: the SDF diffusion process is performed using a diffusion model that contains two key processes: a forward diffusion process and a reverse diffusion process; fusing the voxel and the ultrasonic image features by using a state space model to obtain voxel features fused with the ultrasonic image features, and fusing the voxel features and the ultrasonic image features by using cross attention at a stage with smaller voxel resolution; the method specifically comprises the following steps:

S3-1: the forward diffusion process is represented by a Markov chain, a certain amount of Gaussian noise is added to each time step T until the final moment T, and the original data becomes pure noise; given one data sample X ₀～q(X₀), the forward process q (X ₀:t) gradually converts the data sample into pure gaussian noise:

Where x _t denotes the sample at time step t, x _t-1 denotes the sample at the previous time step, q (x _t∣x_t-1) is the gaussian transition probability:

Wherein the method comprises the steps of Representing normal distribution, beta _t representing noise scheduling super-parameters, which are a learned coefficient or set as a constant; the forward process samples x _t in a closed form at any time step t:

Wherein a _t＝1-β_t is defined as the total number of the components, Thus, x _t is sampled directly by:

where ε is a noise randomly sampled from a normal distribution;

S3-2: the back diffusion process is defined as a markov chain with learned gaussian transitions:

Wherein the method comprises the steps of Is a standard normal distribution, mu _θ(x_t, t) represents a denoising function parameterized by theta,Is a time-step dependent variance set toSampling from p (x _T), then plotting x _t-1～p_θ(x_t-1∣x_t by equation (6) as:

Where ε _θ(x_t, t) is a neural network parameterized by θ that predicts noise from noise input x _T; by repeating this process, X ₀ is finally generated;

Thus, the objective function compares the difference between the predicted noise e _θ(x_t, t) and the applied noise e:

Furthermore, μ _θ(x_t, t) is modified by neural network prediction x ₀ instead of noise ε:

μ_θ(x_t,t)＝γ_tf_θ(x_t,t)+δ_tx_t (10)

Wherein, F _θ(x_t, t) is a neural network parameterized by θ, which predicts noise-free data x ₀; in this case, the objective function becomes:

specifically, a clean data sample is given Sampling the samples having the same shape according to equation (4) in the forward direction

Then, train a networkTo predict noiseless data whose MSE objective function is as follows:

F _p is the ultrasonic image characteristic extracted in S1;

in the reasoning process, a trained neural network is used The process of t=t …,1 predicts progressively smaller noise samples according to equation (14)To generate new SDF voxels:

s3-3: the diffusion process uses a U-shaped network, the voxel feature resolution of each layer is halved, and the channel is doubled; in order to keep the object represented by the generated SDF field consistent with the ultrasonic image, the SDF features and the ultrasonic image features are fused in the diffusion process, so that the ultrasonic image features guide the SDF diffusion process;

the state space model will map a one-dimensional sequence x (t) to y (t) through a hidden state h (t), this system using a as evolution parameter, B, C as projection parameter:

h′(t)＝Ah(t)+Bx(t) (15)

y(t)＝Ch(t) (16)

mamba including a time scale parameter delta converts the continuous parameter A, B into a discrete parameter A common transformation method is zero-order preservation, defined as follows:

At the position of After discretization, equations (15) (16) are expressed as:

y_t＝Ch_t (20)

finally, the model is output through global convolution calculation:

where M is the length of the input sequence x, Is a structured convolution kernel;

In order to fuse the ultrasonic image characteristics and the voxel characteristics, the ultrasonic image characteristics F _p are linearly mapped to the same channel number as the voxel characteristics and then are connected with the voxel characteristics, so that an input x of the SSM is obtained;

Obtaining output y after Mamba calculation, taking out the voxel characteristic part, and remolding the one-dimensional sequence back to the original shape of the voxel characteristic to obtain the voxel characteristic after the ultrasonic image characteristic is fused;

s3-4: after the voxel feature passes through the top of the model, downsampling is carried out to reduce the resolution of the voxel feature, the number of channels is increased, the overall data volume is reduced, and the voxel feature and the ultrasonic image feature are fused by using cross attention at the stage of smaller voxel resolution; simultaneously calculating a group of attention functions of inquiry, packing the attention functions into a matrix Q, and packing keys and values into matrices K and V; the calculated output moment is:

the formula is used as follows:

MultiHead(Q,K,V)＝Concat(head₁,...,head_h)W^O (24)

head_i＝Attention(QW_i ^Q,KW_i ^K,VW_i ^V) (25)

Wherein concat represents a stitching operation, and W ^O、W_i ^Q、W_i ^K、W_i ^V are all learnable weight matrices;

For the voxels v in the grid, projecting the center of the voxel v onto an image to obtain a projection coordinate P; selecting adjacent image blocks close to P to interact with v, wherein a neighborhood image patch set is denoted by N _V, and the selection is as follows: if the distance between P and the center of P _j is less than the distance threshold d _δ, then patch P _j belongs to N _V;

Feature interactions between voxel feature f _V at v and ultrasound image patch feature set f _I belonging to the patch in N _V are modeled using multi-headed self-attention as follows:

Q＝f_VW^Q,K＝f_NW^K,V＝f_NW^V (26)

Here MultiHead (·) is a standard multi-headed attention operation, W ^Q、W^K、W^V is a learnable matrix, M is a mask of attention calculations caused by view projection, absolute position encoding is used for voxels and ultrasound image tiles;

s3-5: further fusing the ultrasonic image features and the voxel features after the cross attention, wherein the output voxel features are high-precision SDF fields;

S4: the meshes are extracted by using the marching cubes, and finally, the triangular mesh model guided by the ultrasonic image is reconstructed.

2. The SDF diffusion-based medical ultrasound image three-dimensional reconstruction method of claim 1, wherein in S1, first training a visual model MedSAM using medical ultrasound image data, then feature extracting an input single medical ultrasound image using a trained MedSAM image encoder, and mapping the output to a feature vectorAnd the method is used for guiding the SDF to approach the picture features in the subsequent diffusion process so as to realize the SDF three-dimensional reconstruction of the ultrasonic image.

3. The SDF diffusion-based medical ultrasound image three-dimensional reconstruction method according to claim 1, wherein the marching cubes algorithm step in S4: setting an isosurface threshold value T, and traversing all cube units C in the SDF field:

(1) Calculating vertex state code index according to the size relation between the 8 vertex scalar values of C and T;

(2) Searching a corresponding intersection point configuration pattern in a predefined condition table according to the index;

(3) Calculating the intersection points with the isosurface on 12 sides of the cube by utilizing linear interpolation;

(4) Connecting corresponding intersection points according to pattern to form one or more triangles;

(5) Triangles are added to the mesh surface data.

4. The system based on the medical ultrasonic image three-dimensional reconstruction method based on SDF diffusion according to claim 1, which is characterized by comprising an ultrasonic image feature extraction module, an SDF diffusion module and a grid generation module;

The ultrasonic image feature extraction module is used for: the module is used for extracting the characteristics of an input ultrasonic image, and the characteristics of the input ultrasonic image are extracted by using a visual model SAM-Med2D image encoder which is pre-trained by a large amount of medical image data;

The SDF diffusion module: the module consists of a U-shaped neural network embedded with a state space model and cross attention, and is used for executing an SDF diffusion process, denoising gradually from random noise to obtain a clean SDF field, and fusing voxel characteristics and ultrasonic picture characteristics by the state space model and the cross attention in the diffusion process to achieve the aim of keeping consistency of a reconstruction grid and the ultrasonic picture characteristics;

The grid generation module: the module executes a marching cubes algorithm to extract a high precision three-dimensional grid from the reconstructed SDF field.