WO2007100263A1

WO2007100263A1 - Method for simulation of ultrasound images

Info

Publication number: WO2007100263A1
Application number: PCT/NO2007/000089
Authority: WO
Inventors: Reidar Brekken
Original assignee: Sinvent As
Priority date: 2006-03-03
Filing date: 2007-03-05
Publication date: 2007-09-07

Abstract

The present invention relates to a method for simulating ultrasound imaging, the simulation being based on virtual or real data and a position indicating a view from which the simulation is performed. According to the present invention the ultrasound-like images are generated by 1) building a database of off-line simulations dependent only on transducer settings and probe configuration, 2) using theoretical ultrasound considerations to calculate which parts of the 3D acoustical model that affects the simulated ultrasound image, and how the image is affected, and 3) combining these images to obtain a final simulated image.

Description

Method for simulation of ultrasound images

The present invention relates to an ultrasound simulator, specifically the generation of ultrasound-like images based on anatomical datasets that may be other than real ultrasound. The simulator can be useful for example for training ultrasound personnel.

Typically a medical ultrasound imaging system consists of a display, a scanner and a probe. The scanner contains hardware and software necessary for generation of ultrasound signals and for transforming the received ultrasound signals into images to be visualized on the display. Usually, the scanner can perform signal processing for extraction of other information from the ultrasound signal, e.g. Doppler-velocity, strain and contrast enhancement.

The probe contains a transducer that converts electrical signals generated by the scanner into high frequency mechanical oscillations (ultrasound) that is transmitted into the body. When the ultrasound wave hits an interface between objects with different acoustical properties, the wave is reflected (backscattered). When the reflected wave hits the transducer, the mechanical oscillations are converted into an electrical signal giving a representation of the received ultrasound signal. The timelag (t) between transmitted and received signal is measured to calculate the distance (r) to the object (r=c*t/2, c=sound velocity in media).

In order to focus and steer the ultrasound beam in different directions, the transducers consist of arrays of several pieco-electric elements. Each element can be activated independently to transmit or receive a signal. The elements are activated according to a certain schedule to ensure that the pulses/waves from each element receive the desired focus point at the same time, i.e. to achieve constructive interference (focusing). This is referred to as beam forming.

Advantages of ultrasound are that it is real-time, scanners are portable, it is relatively inexpensive, and it involves no ionizing radiation. A drawback is that ultrasound acquisition and interpretation is highly user dependent. The level of confidence necessary for clinical decision making is dependent of skills and experi- diagnosis by untrained operators is often considered to be the biggest danger of ultrasound.

The problems may be due to the following points:

• bone and air give a total-reflection of the acoustic energy, thus prohibiting imaging of organs behind ribs and lungs. Also, the image is degraded by inhomogeneous fat layers and bowel gas. The probe has to be placed in a position where the sound waves are transmitted as good as possible into the organ that is being examined ("finding an optimal acoustical window")

• ultrasound does not give a full body scan, only a limited sector oriented according to the position and orientation of the ultrasound probe

• the image quality is varying (noise and artifacts), making it difficult to interpret the images and to separate between normal and pathological cases

These problems are expected to become even more severe as the development of smaller and cheaper ultrasound scanners leads to more users, who may not necessarily have a medical education at the level of the current users. Ultrasound operators therefore need practical training for learning the user interaction necessary for acquiring and interpreting ultrasound images. For some applications, training of operators in clinics may be difficult. This may be due to few pathological cases or few patients, few available instructors or that the situation may be to acute to permit time for training (e.g. trauma). For these situations, a simulator for training ultrasound personnel will be valuable.

Being a result of three-dimensional propagation of sound waves, ultrasound can not be described in detail by means of a one dimensional model. For modeling purposes, the propagation may be approximated as being linear. Results from linear system theory, such as impulse response (or point-spread-functions) or transfer functions can then be used for describing ultrasound. Generally, however, the propagation is non-linear. Accounting for this makes the modeling more complicated. The wave propagation is mainly dependent on two material properties, i.e. the gradients of density and compressibility, lnhomogeneities in human tissue further complicate the modeling of ultrasound wave propagation. Also, the signal is attenuated as it propagates through the body. The absorption of the wave energy increases with increasing frequency. On the other hand, the resolution of the ultrasound signal gets better as the frequency increases. The choice of frequency is therefore a trade off between penetration and resolution.

In general, a simulator could consist of a mannequin (the virtual patient), a virtual ultrasound probe and the necessary hardware and software to generate an ultrasound-like image according to the operator's positioning of the virtual probe relative to the mannequin. The simulator should also contain a case-database with (virtual) patient data and journals, and a user-interface for interaction with the system (virtual scanner). The simulator could also be software based for promoting e-learning, and may include graphics and haptic devices for enhanced "virtual-reality" perception. The student can be trained in choosing the correct probe for the application, setting parameters (e.g. frequency, size of sector, gain), positioning the probe to achieve a good acoustical window and to make diagnosis based on the simulated image.

An ultrasound simulator must generate realistically appearing ultrasound images in real-time (>10-20 images/second). Also, the system cost should be reasonably low.

Generation of simulated ultrasound images could be based upon a table look-up from real ultrasound acquired from actual patients. One disadvantage is that it may be difficult to acquire new cases. Also, an ultrasound image of an organ is dependent of the direction from which the organ is viewed. In order to cover all possible directions, lots of data has to be acquired and put together correctly. Alternatively, some kind of processing must be performed to remove directional dependencies after acquisition, and then add new dependencies during simulation, (ref. US61170781 , Ultrasim). Simulation of wave propagation could be performed by numerically solving physically based equations describing the propagation of sound in an acoustic model of the media. For simplicity, the linear wave propagation equation could be used together with an assumption of homogeneous tissue. The resulting linear system could be analyzed using linear system theory, e.g. impulse-response (point- spread function, PSF) or transfer-functions. More realistic models take into account non-linearity and effects of heterogeneous tissue. However, existing methods for simulation of wave propagation are to computationally expensive to satisfy real-time constraints, and the resulting images are not necessarily resembling real ultrasound closely enough for the present application.

Earlier approaches have avoided simulation of wave propagation by translating from 3D models to ultrasound like-images by 1 D raycasting methods. Loss of energy along the beam is modeled to account for shadows and absorption. The speckle pattern characteristic for ultrasound, which originates from the interference pattern from the backscattered waves from several scatterers, are often modeled as being simply pseudo-random noise that is added to the image. This approach is computationally effective, but not very close to the physics or theories of actual ultrasound imaging. It may therefore suffer from lack of image realism, and it may be difficult to include ultrasound features by using theoretical consideration.

The present invention provides a method for overcoming these and other disadvantages.

In the following, a detailed description of the present invention and non-limiting examples of the present invention are given with reference to the enclosed drawings, where:

Fig. 1 shows a diagram of the present invention,

Fig. 2 shows an example of a pre- (offline) simulated image according to the present invention, Fig. 3a shows a CT slice together with a typical geometry corresponding to the relevant ultrasound sector to be simulated

Fig. 3b shows the CT data from Fig. 3a sampled according to the ultrasound sector,

Fig. 3c shows an image representation of a propagation matrix,

Fig 3d shows a pre-simulated image with the same sampling as the CT-sector shown in Fig. 3b,

Fig. 3e shows the result of element-wise matrix multiplication of Fig. 3c and Fig. 3d,

Fig. 3f is equal to Fig. 3e after scan-conversion.

Fig. 4 shows a demonstrator for the approach according to one embodiment of the present invention,

Fig. 5a shows an example of a simulated ultrasound image according to the present invention, and

Fig. 5b shows a real ultrasound image corresponding to the image simulated and shown in Fig. 5a.

The present invention relates to modeling and simulation of ultrasound images based on a 3D anatomical representation (may be other than ultrasound) of the patient. The parts of the volume relevant for the formation of a simulated ultrasound image can be found by interpolation into the 3D volume based on the positioning of the virtual ultrasound probe relative to the mannequin. According to the present invention, the simulation of ultrasound-like images involves: • Generating an anatomy representing the virtual patient and the acoustical properties related to this anatomy.

• Modeling the ultrasound propagation in a physical body corresponding to the virtual patient.

• Solving the resulting model equations, including estimating the echoes and do the processing necessary for generating the simulated ultrasound image.

• Verification and validation for assuring that the simulator features realistic ultrasound properties that make the student able to draw the correct diagnosis.

Furthermore, the present invention relates to modeling and simulation for generating ultrasound-like images, i.e. to find a transformation of some anatomical data into ultrasound-like images.

According to the present invention, the wave propagation simulation is partly separated from the patient-specific model. Thus it is possible to do parts of the computationally expensive propagation simulation off-line before the training session starts. Next, patient-specific information is added in real-time during the training, taking into account the position of the virtual ultrasound probe, and the theory of ultrasound propagation. Generally, as much as possible of the scanner/transducer specific effects are pre-simulated, while the effects specific of the body being examined and the positioning of the probe, are added in real-time.

This approach is closer to the theory of real ultrasound imaging. It gives more freedom to use theoretical considerations for including properties of ultrasound imaging, while still obtaining real-time performance on a standard laptop PC.

According to the present invention, a method is provided for simulating ultrasound imaging, the simulation being based on virtual or real data, and a position and orientation indicating a view from which the simulation is performed. According to the invention, ultrasound-like images are generated by 1) building a database of off-line simulations dependent only on transducer settings and probe configuration, 2) using theoretical ultrasound considerations to calculate which parts of the 3D acoustical model that affects the simulated ultrasound image, and how the image is affected, and 3) combining these images to obtain a final simulated image.

Furthermore, the present invention provides a method wherein the simulated ultrasound image is systematically compared to real ultrasound for validation, verification and/or calibration of an acoustic model or a simulation model.

The method according to the present invention may be further separated into a pre-training and a real-time part, 1 ) the pre-training part being approximated to be independent of the specific object to be simulated, and dependent on probe configuration and transducer settings, the result being an object independent image, whereas 2) the real-time part includes properties that are dependent of the internal anatomy of the specific object to be simulated, based on values from the representation of the specific object to be simulated, the result being an image (or matrix) describing the intensity weighting of each point in the pre-simulated and object independent image, whereas 3) the final simulated image is obtained by combing the pre-training and real-time part.

The final simulated image may be obtained by combing the pre-training and realtime part by element-wise matrix multiplication.

Also, for the real-time simulation part, a spatial propagation matrix may be calculated on the basis of the spatial impulse response for simulating the ultrasound propagation, wherein the spatial impulse response is used as a weighting matrix for estimating which parts of the 3D data volume that influence each particular point in the simulated ultrasound image and in which degree the image is affected

According to the present invention, the real-time simulation part of the simulator may result in a weigthing matrix, wherein the matrix is used to decide the intensity weight of each point in the pre-simulated ultrasound image, the weighing ma- trix being used for determining high and low intensities in the object independent image.

For the simulation of functional images, the simulation of functional properties may be based on physical modeling of the objects' functional properties combined with results from ultrasound signal processing theory, wherein- the functional properties may comprise Doppler velocity, strain, and/or contrast imaging techniques.

Fig. 1 shows a diagram describing an ultrasound simulator according to the present invention. A virtual patient is created from real-patient data. (The virtual patient may also be represented by a modeled dataset). The simulation can be based on virtual patient data (model) and a position sensor for determining from which view the simulation is performed. Ultrasound-like images are generated by 1) building a database of off-line simulations dependent only on transducer settings and probe configuration, 2) using theoretical ultrasound considerations to calculate which parts of the 3D acoustical model that affects the simulated ultrasound image, and how the image is affected, and 3) combing these images to obtain the final simulated image. By acquiring real ultrasound data from the real patient (with the position of the real probe equal to the virtual probe), the simulated image can be systematically compared to real ultrasound for validation, verification and/or calibration of the acoustic model or the simulation method.

Fig. 2 shows an example of a pre-(offline) simulation image according to the present invention. The image is independent of internal objects of the body subject for the simulated examination, however, the image is dependent of the scanner settings and probe configuration (e.g. speckle pattern, focusing).

Fig. 4 shows a demonstrator for the approach according to one embodiment of the present invention. By pressing the button "load case", patient-specific data is retrieved from a database. In this example a CT/ultrasound compatible phantom is used instead of a real patient. Furthermore, at the left, a 3D visualization of the surface of the body may be shown with sliders for positioning a virtual ultrasound probe relative to the body. The image to the right shows simulated ultrasound image sector which is updated in real-time as the virtual probe is moved over the body surface, i.e. the 3D visualization of the surface of the body mentioned above.

Figs. 5a and 5b show a simulated ultrasound image, and a real ultrasound image, respectively. The anatomic differences between the real and simulated ultrasound images are due to the cross-sections not being exactly the same on the right and left sides. In this example, the simulations are done in real-time on a standard laptop PC. It is understood that the simulations can be done on any other type of suitable computer. It is also possible to provide a device looking similar or exactly like a conventional ultrasound device with the computing power to carry out the simulations, thereby increasing the realism of the training sessions.

According to the present invention, it is further possible to make a software implementation of the simulator, wherein the simulator could be interfaced to any haptics and visualizations device for virtual/augmented reality perception.

The present invention provides a method for making ultrasound-like images from 3D acoustical models. Generally, the method can be described as a simulation in two steps, and a combination of these two steps:

• off-line simulation of properties specific of the image acquisition system (probe/scanner-settings). This part is approximated to being independent of the body (anatomical model) that is being examined.

• Real-time simulation for accounting for the effects specific of the body and the positioning of the probe relative to the body.

According to the present invention it is thus possible to execute a large part of the computationally expensive propagation simulation off-line before the training session starts, whereas patient-specific information is added in real-time. The results of the two steps are combined to obtain the final ultrasound-like image.

To obtain real-time performance on a low-cost system, as much as possible should be simulated off-line. Part of the propagation is made independent (ap- proximation) of the object. This part is realized by simulating propagation in a homogeneous tissue mimicking phantom without any structured internal objects. This phantom consists of a large number of randomly distributed scatterers. The result is an image including e.g. interference patterns (speckle), effects of focusing and/or frequency dependent absorption/attenuation (Fig 2). This will be dependent of the parameters chosen for the probe and scanner (e.g. probe configuration and frequency). Several images are simulated for different resolutions and different random speckle patterns.

Since the first part of the propagation model is simulated off-line before the training starts, time-consume is no issue. Existing (computationally expensive) simulations of wave propagations can therefore be performed. Also, the first part could be produced using real data acquired from a tissue-mimicking phantom containing no structured objects.

The other part includes effects that are dependent on the body being examined and the position and direction of the ultrasound view, e.g. shadows, reflections and different echo intensities. The task that has to be performed in real-time is finding which intensities (voxels) of the 3D volume that affect the ultrasound image formation, and in which way.

One alternative could be to use a 1 D approach (approximating the ultrasound propagation as a ray or beam). This means that the value of each consecutive sample point depends only on one point in the 3D volume and an accumulation of values along each single beam (e.g. for including shadows behind lungs and ribs). In addition to reflections and shadows, reverberations (multiple reflections) can also be modeled by the 1 D approach.

The sound propagation through human tissue is a complicated process, and can not be fully described by a 1 D model. Generally, the tissues are inhomogeneous and the propagation is non-linear. An approximation would be to assume linear wave propagation and homogeneous tissue. Linear system analysis could then be applied e.g. transfer-function or spatial impulse-response (point-spread function, PSF). Instead of considering one point for each sample, we suggest to take into account all points relevant for the image formation by calculating the (3D spatial) point spread function (PSF) and to use the PSF to calculate a weighted sum of all relevant points from the 3D volume (e.g. CT). Thereby, other effects could be included (e.g. sidelobes).

Image realism may be improved by adding artifacts due to non-linear effects, in- homogeneities, and noise. This could be done by distorting the PSF to account for defocusing effects caused by inhomogeneities (aberrations). This distortion . could be different for different parts of the object, thus simulating different fat- layers. The quality could be adapted to allow for simulating more or less difficult cases.

The second part has to be simulated in real-time, and therefore fast methods must be used. A fast simulation method allows for a more complex model which could give more realistic images while still fulfilling the real-time constraint. Computational cost may be significantly reduced by using several processors for par- allelization (e.g. multi-core processors or graphical processing units, GPU).

The result of the second step is used for producing darker and brighter areas in the pre-simulated image, e.g. by generating a weighting matrix (image with values between 0 (black) and 1 (white)) to.be used for element-wise multiplication with the pre-simulated image. Hence, the results from each step are combined to produce the final simulated image to be interpreted by the operator. This image will be dependent on choice of different probes, different settings (e.g. frequency, sector depth and width, gain) and positioning of the probe relative to the (virtual) patient. (Fig 3)

For simulating functional images, such as Doppler or strain, we suggest to first model some dynamic properties. Next we use physical relationships, together with results from ultrasound signal processing, for simulating functional images or properties that could be extracted from real ultrasound signals. An example could be to segment blood vessels from CT images, and to model a given blood velocity profile. By calculating the angle (α) between the simulated ultrasound plane and the modeled blood flow in real time, the simulated velocity could easily be calculated as:

v^_M (^χ> 0 = v_Mode, (x, 0 * cos a + w,

where w is some kind of modeled noise. This could be used for simulating a 2D color-Doppler image, power-Doppler or for displaying a simulated Doppler spectrum.

Example

A software demonstration of the simulator has been implemented. Six slider- values were used for positioning the virtual probe relative to the virtual patient (Fig 4). The demonstrator is used for evaluating the suggested method's realtime and image realism performance. Some examples from a CT/ultrasoύnd compatible phantom are shown in Fig 5. CT-data of the phantom is used in the simulator, and the results can be systematically compared to real ultrasound images acquired from the same phantom. The computations are done in real-time on a standard laptop PC, and the simulated images are very much equal to the real data.

Currently, the anatomy of the virtual patient is created from 3D CT data from real patients. The CT data (describing the intensity of the tissues) is used directly as a description of the acoustical properties of the tissues. One possible improvement is to use segmentation and surface editing for optimizing the realism of features relevant to the diagnosis of the case being examined, or even for generating new cases. Having a segmented model, the acoustical properties could be modeled in greater detail, for example by including compressibility or increasing detail resolution. As an example, perfusion/contrast imaging could be simulated by modeling blood vessels and to assign high intensities to those volume elements modeled as contrast filled vessels. Further efforts should be made for modeling deformations due to pressure applied to skin. It is also possible to simulate the dynamics of the organs by implementing a dynamical model, either by acquiring dynamical data or by a mathematical model

A set of images is generated for different scanner settings and probe configurations. These images are independent of the internal organs of the body subject to the simulated examination.

During the simulation, the positioning of the virtual probe relative to the virtual body is measured for extracting the data (from acoustical/anatomical model) relevant for generating the simulated ultrasound image. Based on theoretical ultrasound considerations, it is decided which parts of this data that is relevant for forming each point in the simulated image, and how these parts influence the result. This information is further used for generating an image describing the intensity weighting in the object independent image. The result is an image dependent on scanner settings and probe-configuration as well as the anatomy of the body subject of the simulated examination. . .

By modeling certain dynamical properties of the body, such as blood velocity patterns of blood vessels, the ultrasound simulator could implement functional imaging (e.g. Doppler) by taking into account the relationship between the model and known results from for example ultrasound signal processing theory.

Claims

1. A method for simulating ultrasound imaging, the simulation being based on virtual or real data and a position indicating a view from which the simulation is performed, characterized in that ultrasound-like images are generated by the steps of:

1) building a database of off-line simulations dependent only on transducer settings and probe configuration,

2) using theoretical ultrasound considerations to calculate which parts of the 3D acoustical model that affects the simulated ultrasound image, and how the image is affected, and

3) combining these images.to obtain a final simulated image.

2. A method according to claim 1 , characterized in that the simulation is separated into a pre-training and a real-time part, the 1) pre-training part being approximated to be independent of the specific object to be simulated, and dependent on probe configuration and transducer settings, the result being an object independent image, whereas 2) the real-time part including properties that are dependent of the internal anatomy of the specific object to be simulated, based on values from the specific object to be simulated, the result being an image describing the intensity weighting of each point in the pre-simulated and object independent image, whereas 3) the final simulated image is obtained by combing the pre-training and real-time part.

3. A method according to claim 4, characterized in that the final simulated image is obtained by combing the pre-training and real-time part by element-wise matrix multiplication.

4. A method according to claim 2, characterized in that a spatial propagation matrix is calculated on the basis of the spatial impulse response for simulating the ultrasound propagation, wherein the spatial impulse response is used for calculating which parts of the 3D acoustical model that affects each pixel in the simulated ultrasound image and how much they affect it, and that the impulse response may be distorted as appropriate for simulating e.g. artifacts due to inhomogeneous tissue.

5. A method according to claim 4, characterized in that the matrix describes which values in the 3D acoustical model that will affect the image formation and in which way they affect the image formation,

6. A method according to any of the previous claims, characterized in that the simulator is arranged to simulate functional properties, and that this is based on physical modeling of objects' functions and results from ultrasound signal processing theory.

7. A method according to claim 6, characterized in that the functional properties comprise Doppler velocity, strain, and/or contrast.

8. A method according to any of the previous claims, characterized in that the method comprises a software implementation of the simulator, wherein the simulator could be interfaced to any haptics and visualizations device for virtual/augmented reality perception.

9. A method according to any of the previous claims, characterized in that the simulated ultrasound image is systematically compared to real ultrasound for validation, verification and/or calibration of an acoustic model or a simulation model.