JP5065740B2 - Image processing method, apparatus, and program - Google Patents

Description

  The present invention relates to an image processing method, apparatus, and program for generating a pseudo 3D image from 3D image data consisting of a group of many 2D images by volume rendering.

  In order to make it easy to grasp the three-dimensional structure of the subject, the computer graphics are used to obtain the three-dimensional image data of the subject consisting of a group of two-dimensional images acquired by a CT apparatus, an MRI apparatus, an ultrasonic diagnostic apparatus, etc. Processing for generating and displaying a pseudo three-dimensional image that is stereoscopically visualized on a two-dimensional plane by using a technique such as an image processing is performed.

  As a method for generating such a pseudo three-dimensional image, for example, as shown in Patent Document 1, opacity (Opacity) and color information of R, G, and B are assigned to each voxel constituting the three-dimensional image. A volume rendering method is known that performs voxel tracking (ray casting) from the viewing and observation side to each pixel on the projection plane, and visualizes not only the boundary surface between the tissues of the subject but also the internal information structure in multiple watermarks. Yes.

In such a volume rendering method, an arbitrary light source is set, and for an arbitrary point in a three-dimensional image illuminated by the light source, the incident direction of light incident on the point from the light source and the point On the basis of the relationship with the normal direction of the surface determined by the gradient of the pixel value at “”, “shading” processing for expressing a stereoscopic effect is performed. Specifically, the calculation of the luminance value b (P _V ) at an arbitrary point P _V in the three-dimensional image is based on color information defined in advance for each pixel value as shown in the following equation (1). Te to the color information c (P _V) which is assigned, and the point P _V normal vector of the surface is determined by the gradient of pixel values in N (P _V), and the unit vector L from the point P _V to the light source This is done by integrating the vector dot product. Here, h is a shading coefficient by diffuse reflection.

For example, as shown in FIG. 1, in the three-dimensional image V illuminated by the light source S, the pixel value of each voxel in the spherical region A is 255, and the pixel value of each voxel in the region B outside the region A When 0 is 0, as shown in FIG. 2, the normal vector determined at each point of the boundary between the region A and the region B is directed outward from the boundary surface. Here, when opacity is given to each voxel so that an area where the pixel value of the voxel is 0 is a transparent area and an area where the pixel value of the voxel is 255 is an opaque area, only the area A is visualized. , illuminated surface is a surface H ₁ of the light source side of the region a as shown in FIG. 3 for the light emitted from the light source S. The pseudo three-dimensional image I ₁ generated by the volume rendering method by ray casting the three-dimensional image V from the viewpoint E is a three-dimensional representation of a spherical subject as shown in FIG.
JP 2006-000338 A

  In the volume rendering method described above, opacity is set for each pixel value, light transmission is calculated, and the subject is visualized. Opacity is a value that indicates the difficulty of transmitting light.When the opacity is low, the information of the object is reflected weakly, and the light reaches the back and the information of the destination object is also reflected. The When the opacity is high, the information of the object is strongly reflected, but the light does not reach the back and the information of the previous object is not reflected. By effectively using this opacity, a translucent state can be created, and the state of the object behind the overlapped portion can also be described.

  Here, the setting of the opacity is not limited to the case where the opacity value is determined in accordance with the magnitude of the pixel value (CT value) as shown in FIG. As shown in FIG. 5, a region where the voxel pixel value is 255 is set to be a transparent region (opacity 0), and a region where the voxel pixel value is 0 is set to be an opaque region (opacity 100). Arbitrary opacity can be set for each pixel value in advance, for example, by visualizing a region B outside the image A of a spherical object, and each voxel can be given opacity for the pixel value of that voxel.

However, in the above-described conventional technique in which the shading process is performed on each point in the three-dimensional image based on the normal direction of the surface defined by the gradient of the pixel value at that point, the normal line determined for each point Since the vector is constant regardless of the opacity setting for the pixel value, as shown in FIG. 5, the region where the voxel pixel value is 255 is transparent, and the region where the voxel pixel value is 0 is opaque. When the opacity is given to each voxel of the three-dimensional image V of FIG. 1 so as to be a region, only the region B is visualized, and the illuminated surface for the light emitted from the light source S is the surface on the light source side of the region B Although the H _2, the normal vectors determined in each point of the boundary between the regions a and B remains in the state shown in FIG. 2, and ray-casting a three-dimensional image V from the viewpoint E, generated pseudo 3-dimensional image I ₂ which is, FIG. As shown in, is depicted dark part which should be illuminated surface, there is a problem of whether the kind of unnatural image light to the portion which should originally become anionic is hitting. Here, the pseudo 3-dimensional image I ₂ shown in FIG. 6, in order to observe the inner surface of the region B, was removed by cutting approximately half region from the observation that the side of the three-dimensional image V (viewpoint E side) Is.

  In view of the above circumstances, an object of the present invention is to provide an image processing method and apparatus for improving the reality of a pseudo three-dimensional image generated by a volume rendering method, and a program therefor.

  According to the image processing method of the present invention, a three-dimensional image is sampled along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on a projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. In an image processing method for generating a pseudo three-dimensional image by a volume rendering method that determines a pixel value of a pixel on a projection plane using a luminance value at each search point, each voxel pixel in each voxel of the three-dimensional image A normal vector at each search point is determined using the opacity assigned according to the value, and a luminance value at the search point is determined using the determined normal vector. (First image processing method).

  Here, the “three-dimensional image” is a virtual three-dimensional image composed of three-dimensional image data composed of a group of a number of two-dimensional images, and the “pseudo three-dimensional image” is a volume rendering method. It means a two-dimensional image that is generated and displayed visually in three dimensions.

  In addition, the image processing method of the present invention is configured to display a three-dimensional image along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on a projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. In an image processing method for generating a pseudo three-dimensional image by a volume rendering method for determining a pixel value of a pixel on a projection plane using a luminance value at each sampled search point, each voxel of the three-dimensional image is assigned to each voxel. Determine the normal vector at each voxel using the opacity assigned according to the pixel value of and determine the luminance value at each exploration point using the normal vector at the neighboring voxel of that exploration point. (Second image processing method).

  The image display apparatus according to the present invention samples a three-dimensional image along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on a projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. In an image processing apparatus for generating a pseudo three-dimensional image by a volume rendering method, comprising a ray casting means for determining a pixel value of a pixel on a projection plane using a luminance value at each search point, each voxel of the three-dimensional image The normal vector determining means for determining the normal vector at each search point using the opacity assigned according to the pixel value of each voxel, and the brightness at the search point using the determined normal vector And a luminance determining means for determining a value (first image processing apparatus).

  In addition, the image display device of the present invention displays a three-dimensional image along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on a projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. In an image processing apparatus for generating a pseudo three-dimensional image by a volume rendering method, comprising a ray casting means for determining a pixel value of a pixel on a projection surface by using the sampled luminance value at each exploration point. Using the opacity assigned to each voxel according to the pixel value of each voxel, normal vector determining means for determining the normal vector in each voxel, and the luminance value at each exploration point, the luminance value at each exploration point And luminance determining means for determining using a normal vector in the neighboring voxel (second image processing apparatus).

  An image display program of the present invention causes a computer to execute the first image processing method (first image processing program).

  An image display program according to the present invention causes a computer to execute the first image processing method (second image processing program).

  According to the image processing method, apparatus, and program of the present invention, when a pseudo three-dimensional image is generated by the volume rendering method, an arbitrary point in the three-dimensional image is subjected to a surface determined by the opacity gradient of the point. Since the shading process is performed based on the normal direction, the natural shading that appears by setting arbitrary opacity to the pixel value can be expressed in the pseudo 3D image generated by volume rendering, A highly realistic pseudo three-dimensional image can be generated.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a hardware configuration diagram showing an overview of a three-dimensional medical image processing system. As shown in the figure, in this system, a modality 1, an image storage server 2, and an image processing workstation 3 are connected via a network 9 in a communicable state.

  The modality 1 acquires a three-dimensional medical image V (three-dimensional image) representing a subject, and specifically includes a CT apparatus, an MRI apparatus, an ultrasonic diagnostic apparatus, and the like.

  The image storage server 2 is a computer that stores and manages a three-dimensional medical image V acquired by the modality 1 and a medical image generated by image processing at the image processing workstation 3 in an image database. And database management software (for example, ORDB (Object Relational Database) management software).

  The image processing workstation 3 is a computer that performs image processing on the three-dimensional medical image V acquired from the modality 1 or the image storage server 2 in response to a request from the interpreter and displays the generated image. In particular, it includes an input device such as a keyboard and a mouse for inputting a request from a radiogram interpreter, a main memory having a capacity capable of storing the acquired three-dimensional medical image V, and a display for displaying the generated image. .

  The storage format of image data and communication between devices via the network 9 are based on a protocol such as DICOM (Digital Imaging and Communications in Medicine).

  FIG. 2 is a block diagram showing portions related to the volume rendering function of the image processing workstation 3. As shown in the figure, the image processing workstation 3 receives an image acquisition means 10 for acquiring a three-dimensional medical image V of a patient to be interpreted from the modality 1 or the image storage server 2 in response to a request from the interpreter. A pixel on the projection plane is obtained by using luminance values and opacity of a plurality of search points obtained by sampling the three-dimensional medical image V at a predetermined interval along a plurality of lines of sight connecting the viewpoint and each pixel on the projection plane. Are determined, and a ray casting unit 60 that generates a volume rendering image (pseudo three-dimensional image) and an image display unit 70 that displays the generated volume rendering image on a display. Comprises a luminance value determining means 40 and an opacity determining means 50 for determining the luminance value and the opacity at each search point, respectively. Further, the luminance value determination means 40 determines a normal vector at each search point using opacity assigned to each voxel of the three-dimensional medical image V according to the pixel value of each voxel. Means 42 and brightness determining means 43 for determining the brightness value at the search point using the determined normal vector.

  Next, the flow of processing for generating a volume rendering image by this medical image processing system, particularly the image processing workstation 3, will be described.

  First, the image acquisition means 10 acquires a three-dimensional medical image V of a patient to be interpreted from the modality 1 or the image storage server 2 in response to a request from an interpreter. This three-dimensional medical image V is obtained by dividing a multi-slice image into voxels and arranging them in a three-dimensional coordinate space. The position of each voxel is the x-axis in the left-right direction of the subject and the y-axis in the front-rear direction. The pixel value of each voxel is associated with the coordinates of the position of the voxel.

Next, the ray casting means 60 obtains pixel values (output pixel values) of pixels constituting the volume rendering image. First, depending on the viewpoint E, light source S, and projection plane F (size, position, number of pixels) set by an initial setting file or input from a keyboard or mouse by an interpreter, for example, as shown in FIG. A three-dimensional medical image V is predetermined along a plurality of lines of sight E _j (j = 1, 2,..., L; L is the number of lines of sight) connecting the viewpoint E and each of the projection pixels on the projection plane F. A plurality of search points P _ji (i = 1, 2,..., N; n is the number of search points on the line of sight E _j ) sampled at intervals are set. Next, along each line of sight E _j , the luminance value b (P _ji ) and opacity α (P _ji ) at each exploration point P _ji are sequentially acquired by the luminance value determining means 40 and the opacity determining means 30 described later. Then, as shown in the following equation (2), when these products are added and the accumulation of the opacity α becomes a predetermined threshold value, or when the ray leaves the target 3D medical image V Then, the process for the line of sight E _j is terminated, and the addition result is determined as the output pixel value C _j of the projection pixel on the projection plane through which the line of sight E _j passes.

  Such processing is performed for each line of sight, output pixel values of all projection pixels on the projection plane are determined, and a volume rendering image is generated. The generated volume rendering image is displayed on the display of the image processing workstation 3 by the image display means 70.

Next, the flow of processing for determining the luminance value b (P _ji ) at each search point P _ji by the luminance value determining means 40 will be described.

The luminance value determining means 40 is composed of a normal vector determining means 42 and a luminance value calculating means 43. First, the normal vector determining means 42 has a normal vector N (P _ji ) at each search point P _ji . To decide. Here, the normal vector of the surface determined by the gradient of opacity at the search point P _ji is determined as the normal vector N (P _ji ) at the search point by the following equation (3).

Here, ∇α (P _ji) is the gradient of the opacity of the search point P _ji, opacity in the six neighboring points adjacent in the x-axis direction, y-axis, or z-axis direction search point P _ji Can be obtained by the following equation (4).

Here, α represents the opacity at that point, and the opacity of each neighboring point is determined by the pixel value at that point based on the opacity previously defined for each pixel value. Note that the pixel value at each neighboring point is obtained by linearly interpolating the pixel values of the eight voxels constituting the lattice including the neighboring point. Here, for simplicity, P _ji = (x, y, z).

  For example, for the three-dimensional image V shown in FIG. 1 where the pixel value of each voxel in the spherical area A is 255 and the pixel value of each voxel in the area B outside the area A is 0, FIG. As shown in FIG. 4, each voxel is not transparent so that an area where the voxel pixel value is 0 is a transparent area (opacity 0) and an area where the voxel pixel value is 255 is an opaque area (opacity 100). When transparency is given, the normal vector determined for each point on the boundary between the region A and the region B is directed outward from the boundary surface, as shown in FIG.

  Further, for this three-dimensional image V, as shown in FIG. 11, an area where the voxel pixel value is 0 is an opaque area (opacity 100), and an area where the voxel pixel value is 255 is a transparent area. When opacity is given to each voxel so that (opacity is 0), the normal vector determined at each point of the boundary between the region A and the region B is the surface of the boundary as shown in FIG. The direction is from the inside to the inside.

Next, the brightness value calculation means 43 calculates the brightness value b (P _ji ) of the search point based on the normal vector at each search point P _ji determined by the normal vector determination means 42, for example: Calculate by (5).

Here, h is a shading coefficient by diffuse reflection. In this embodiment, environmental light and specular reflection light are not considered. Further, L is a unit direction vector from the exploration point P _ji to the light source S, “·” is an inner product of the vectors, and c (P _ji ) is defined in advance for each tissue of the subject (for each pixel value such as a CT value). Assigned color information based on the assigned color information.

Next, processing for determining the opacity α (P _ji ) at each of the search points P _ji by the opacity determination means 50 will be described. The opacity α (P _ji ) at each search point P _ji is determined by the pixel value at the search point P _ji based on the opacity defined in advance for each pixel value. Note that the pixel value of each search point P _ji is obtained by linearly interpolating the pixel values of the eight voxels constituting the grid including the search point.

  Note that the luminance value determination process and the opacity determination process are independent of each other. Therefore, when performing the serial process, either process may be performed first, or the parallel process is performed simultaneously. May be.

Thus, in the three-dimensional medical image processing system as an embodiment of image processing according to the present invention,
When a pseudo three-dimensional image is generated by the volume rendering method, a shadow process is performed on an arbitrary point in the three-dimensional medical image V based on the normal direction of the surface determined by the gradient of the opacity of the point. As a result, natural shadows appearing by setting arbitrary opacity to the pixel value can be expressed in the pseudo 3D image generated by volume rendering, and a highly realistic pseudo 3D image can be generated. it can.

  For example, for the three-dimensional image V shown in FIG. 1 in which the pixel value of each voxel in the spherical area A is 255 and the pixel value of each voxel in the area B outside the area A is 0, FIG. As shown in FIG. 4, each voxel is not transparent so that an area where the voxel pixel value is 0 is a transparent area (opacity 0) and an area where the voxel pixel value is 255 is an opaque area (opacity 100). When transparency is given, as shown in FIG. 10, the normal vectors determined at the points of the boundary between the region A and the region B according to the present invention coincide with those shown in FIG. The generated pseudo three-dimensional image matches that shown in FIG.

Further, for the three-dimensional image V shown in FIG. 1 in which the pixel value of each voxel in the spherical area A is 255 and the pixel value of each voxel in the area B outside the area A is 0, FIG. As shown in FIG. 4, each voxel has an area where the voxel pixel value is 0 as an opaque area (opacity 100 ) and an area where the voxel pixel value is 255 as a transparent area (opacity 0 ). When the transparency is given, as shown in FIG. 11, the normal vector determined at each point of the boundary between the region A and the region B according to the present invention is in the opposite direction to that shown in FIG. Since the direction is inward from the boundary surface, the pseudo three-dimensional image I3 generated by volume rendering is a natural image in which light is applied to the illuminated surface H2, as shown in FIG. Here, in order to observe the inner surface of the region B, the pseudo three-dimensional image I3 shown in FIG. 12 is obtained by cutting and removing a substantially half region from the observation side (viewpoint E side) of the three-dimensional image V. It is.

In the above embodiment, the normal vector determining means 42 uses the opacity assigned to each voxel of the three-dimensional image according to the pixel value of each voxel, and the normal vector at each search point P _ji . determine the N (P _ji), with the determined normal vector N (P _ji), by the brightness determining means 43 determines the luminance value at the search point P _ji, each search point P _ji In the above description, the shading process based on the normal direction of the surface determined by the opacity gradient is performed. However, the normal vector determining unit 42 previously stores a pixel value in each voxel of the three-dimensional image. Based on the opacity defined for each voxel, using the opacity assigned according to the pixel value of that voxel, calculate the opacity gradient at each voxel, and the calculated opacity gradient It is one that determines the normal vector of the surface to be more determined as a normal vector at that voxel, the luminance calculation unit 43, the normal vector N (P _ji) at each search point P _ji, the respective search point A normal vector determined for a neighboring voxel of P _ji is calculated by linear interpolation, and the search point P _{ji is} calculated by using the calculated normal vector N (P _ji ), for example, by the above equation (3). The luminance value b (P _ji ) is calculated using the normal vector determining unit 42 and the luminance calculating unit 43, and the opacity gradient at each search point P _ji is obtained by performing the volume rendering process. The shading process based on the normal direction of the surface determined by the above may be performed.

  In the above embodiment, the image processing workstation 3 performs both image processing and image display. However, an image processing server is separately provided and connected to the network 9, and image processing is performed by the image processing server. You may make it let. As a result, distributed processing is achieved. For example, when images are displayed on a plurality of terminals, it is not necessary to install a plurality of high-performance image processing workstations, which contributes to a reduction in the cost of the entire system.

The figure which shows an example of the three-dimensional image illuminated with the light source The figure which shows an example of the normal vector determined by the conventional image processing method 1 is a diagram illustrating an illuminated surface when an area where the pixel value is 0 in the three-dimensional image of FIG. 1 is set to be a transparent area and an area where the pixel value is 255 is an opaque area. The figure which shows an example of the pseudo | simulation three-dimensional image produced | generated by the conventional image processing method 1 is a diagram illustrating an illuminated surface when an area where the pixel value is 0 in the three-dimensional image in FIG. 1 is set as an opaque area and an area where the pixel value is 255 is set as a transparent area. The figure which shows an example of the pseudo | simulation three-dimensional image produced | generated by the conventional image processing method 1 is a schematic configuration diagram of a three-dimensional medical image processing system according to an embodiment of the present invention. Block diagram showing the volume rendering function of the image processing workstation of FIG. The figure for demonstrating the setting process of the search point by the ray casting means 60 The figure which shows an example of the normal vector determined by the normal vector determination means 42 The figure which shows an example of the normal vector determined by the normal vector determination means 42 The figure which shows an example of the pseudo three-dimensional image produced | generated by the image processing of this invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Modality 2 Image storage server 3 Image processing workstation 9 Network 40 Luminance value determining means 42 Normal vector determining means 43 Luminance value calculating means 50 Opacity determining means 60 Ray casting means S Light source F Projection plane α Opacity E _j Line of sight P _ji exploration point

Claims (6)

The luminance value at each exploration point obtained by sampling the three-dimensional image is used along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on the projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. In the image processing method for generating the pseudo three-dimensional image by the volume rendering method for determining the pixel value of the pixel on the projection plane,
An opacity previously associated with the pixel value of each voxel is assigned to each voxel of the three-dimensional image, and a normal vector of the surface determined by the gradient at each search point of the assigned opacity is obtained. As a normal vector at each exploration point,
An image processing method, wherein a luminance value at the search point is determined using the determined normal vector. The luminance value at each exploration point obtained by sampling the three-dimensional image is used along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on the projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. In the image processing method for generating the pseudo three-dimensional image by the volume rendering method for determining the pixel value of the pixel on the projection plane,
An opacity pre-corresponding to the pixel value of each voxel is assigned to each voxel of the three-dimensional image, and a normal vector of a surface determined by the gradient in each voxel of the assigned opacity is As a normal vector in each voxel,
An image processing method, wherein a luminance value at each search point is determined using a normal vector in a neighboring voxel of each search point. The luminance value at each exploration point obtained by sampling the three-dimensional image is used along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on the projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. An image processing apparatus for generating a pseudo three-dimensional image by a volume rendering method, comprising ray casting means for determining a pixel value of a pixel on the projection plane,
An opacity previously associated with the pixel value of each voxel is assigned to each voxel of the three-dimensional image, and a normal vector of the surface determined by the gradient at each search point of the assigned opacity is obtained. Normal vector determining means for determining a normal vector at each of the search points ;
An image processing apparatus comprising: a luminance determining unit that determines a luminance value at the search point using the determined normal vector. The luminance value at each exploration point obtained by sampling the three-dimensional image is used along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on the projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. An image processing apparatus for generating a pseudo three-dimensional image by a volume rendering method, comprising ray casting means for determining a pixel value of a pixel on the projection plane,
An opacity pre-corresponding to the pixel value of each voxel is assigned to each voxel of the three-dimensional image, and a normal vector of a surface determined by the gradient in each voxel of the assigned opacity is Normal vector determining means for determining as a normal vector in each voxel ;
An image processing apparatus comprising: luminance determining means for determining a luminance value at each search point using a normal vector in a neighboring voxel of each search point. The luminance value at each exploration point obtained by sampling the three-dimensional image is used along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on the projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. In a program for generating a pseudo three-dimensional image by a volume rendering method for determining a pixel value of a pixel on the projection plane,
On the computer,
An opacity previously associated with the pixel value of each voxel is assigned to each voxel of the three-dimensional image, and a normal vector of the surface determined by the gradient at each search point of the assigned opacity is obtained. As a normal vector at each exploration point,
A program for executing determination of a luminance value at the search point using the determined normal vector. The luminance value at each exploration point obtained by sampling the three-dimensional image is used along a plurality of lines of sight connecting an arbitrary viewpoint and each pixel on the projection plane on which the three-dimensional image illuminated by an arbitrary light source is projected. In a program for generating a pseudo three-dimensional image by a volume rendering method for determining a pixel value of a pixel on the projection plane,
On the computer,
An opacity pre-corresponding to the pixel value of each voxel is assigned to each voxel of the three-dimensional image, and a normal vector of a surface determined by the gradient in each voxel of the assigned opacity is As a normal vector in each voxel,
A program for causing a luminance value at each search point to be determined using a normal vector in a neighboring voxel of each search point.