JP7131080B2 - volume rendering device - Google Patents

Description

The present disclosure relates to volume rendering technology for three-dimensional visualization using a plurality of two-dimensional tomographic images.

Conventionally, multiple images are taken continuously at predetermined slice intervals by medical imaging diagnostic equipment (modality) such as CT, MRI, and PET, and stored in a medical information system such as PACS (Picture Archiving and Communication Systems) in DICOM format. Three-dimensional visualization of organs and the like is performed based on the tomographic image.

In the field of 3D computer graphics, a rendering image is generated with limited colors such as 256 colors, and the colors of the rendering image are changed in real time by interactively changing the color map (color lookup table) implemented in hardware. method is used (see Patent Document 1).

A method of rendering with a VIS volume (a voxel having three parameters of a signal value, a shadow luminance value, and an opacity intensity coefficient) that does not depend on a color map has also been proposed (see Patent Document 2).

JP-A-1-321577 Japanese Patent No. 2651787

However, the technique described in Patent Document 1 can only change the RGB values, and does not disclose controlling the α value expressing opacity, which is important in volume rendering. In addition, with the technique described in Patent Document 2, the time required for processing can be reduced by only about 10% compared to the case of re-rendering by applying a color map to signal values.

Therefore, the present disclosure applies a color map to a plurality of tomographic images, suppresses the computational load of the process of generating a rendered image, and improves the speed of the process of generating a rendered image. The task is to provide

The present disclosure includes multiple means for solving the above problems, but if one example is given,
To generate a volume rendering image by referring to a predefined color map based on a plurality of two-dimensional tomographic images in which the object is photographed at predetermined intervals and a signal value is assigned to each pixel. a volume rendering device of
By referring to a color map defined by associating color component values and opacity values with signal values, the signal values of each voxel three-dimensionally arranged in correspondence with each pixel of the plurality of tomographic images are set to be unknown. opacity voxel image creation means for creating an opacity voxel image in which opacity is defined as a voxel value by replacing with transparency;
With reference to the color map, the signal value of each voxel three-dimensionally arranged in correspondence with each pixel of the plurality of tomographic images is replaced with the value of the color component, and the value of the color component is defined as the voxel value. a color component voxel image creating means for creating a color component voxel image;
rendering means for generating a volume rendered image using the opacity voxel image and the color component voxel image;
To provide a volume rendering device characterized by comprising:

Also, the volume rendering device of the present disclosure includes:
Smoothing means for performing a smoothing process to replace the opacity of each voxel constituting the opacity voxel image with the average value of the opacity of the voxel and voxels in the vicinity of the voxel,
The rendering means generates a volume rendering image using a smoothed opacity voxel image.

Also, in this disclosure:
To generate a volume rendering image by referring to a predefined color map based on a plurality of two-dimensional tomographic images in which the object is photographed at predetermined intervals and a signal value is assigned to each pixel. a volume rendering device of
By referring to a color map defined by associating color component values and opacity values with signal values, the signal values of each voxel three-dimensionally arranged in correspondence with each pixel of the plurality of tomographic images are set to be unknown. opacity voxel image creation means for creating an opacity voxel image in which opacity is defined as a voxel value by replacing with transparency;
With reference to the color map, the signal value of each voxel three-dimensionally arranged in association with each pixel of the plurality of tomographic images is replaced with a color component value, and the color whose color component value is defined as the voxel value a color component voxel image creating means for creating a component voxel image;
adding the opacity of the corresponding voxel of the opacity voxel image to each voxel of the color component voxel image to create an opacity-added color component voxel image in which the values of the opacity and the color component are defined as voxel values; Transparency color component voxel image creating means;
rendering means for generating a volume rendering image using the color component voxel image with opacity;
To provide a volume rendering device characterized by comprising:

Further, the volume rendering device of the present disclosure further includes smoothing means for performing a smoothing process of replacing the opacity of each voxel constituting the opacity voxel image with the average value of the opacities of the voxel and voxels in the vicinity of the voxel. prepared,
The color component voxel image with opacity creating means creates the color component voxel image with opacity using a smoothed opacity voxel image.

In addition, the volume rendering device of the present disclosure is the outermost voxel in the color component voxel image, and the opacity of the voxel in the opacity voxel image corresponding to the color component voxel image is 0. a color correcting means for performing a color correction process of replacing the color components of voxels in the color component voxel image corresponding to voxels having non-zero opacity in the vicinity of voxels having opacity of 0 in the opacity voxel image with an average value of color components of the voxels; further prepared,
The color component voxel image with opacity creating means creates the color component voxel image with opacity using a color component voxel image that has undergone color correction processing.

Further, the volume rendering device of the present disclosure calculates the value of the color component of each voxel of the color component voxel image based on the opacity of voxels near the voxel of the opacity voxel image corresponding to the voxel. , calculating a shade value based on the calculated gradient vector and a predetermined light source vector, and replacing the value of the color component with a value obtained by multiplying the shade value by the shade adding means. and

Further, in the volume rendering device of the present disclosure, the shading adding means, in calculating the gradient vector, includes two voxels near each of the voxel of the opacity voxel image in the X-axis direction, the Y-axis direction, and the Z-axis direction. is calculated as the X-direction component, Y-direction component, and Z-direction component of the gradient vector, and a unit vector obtained by correcting the Z-direction component based on a predefined Z-axis direction scaling factor , is calculated as a gradient vector at the voxel.

Further, in the volume rendering device of the present disclosure, the opacity voxel image creating means is arranged in each of the two-dimensional axial directions of the plurality of tomographic images and in each of the three axial directions perpendicular to the tomographic images, every M pixels. The signal value of each voxel arranged three-dimensionally in association with is replaced with opacity, and an opacity voxel image is created in which the opacity is defined as the voxel value,
The color component voxel image creating means is arranged three-dimensionally in association with every M pixels in each of the two-dimensional axial directions of the plurality of tomographic images and each of three axial directions perpendicular to the tomographic images. It is characterized by replacing signal values of voxels with color component values and creating a color component voxel image in which the color component values are defined as voxel values.

Also, the volume rendering device of the present disclosure includes:
When the tomographic image is an image having a 16-bit signal value, the maximum value Dmax and the minimum value Dmin of the signal value are calculated based on all of the plurality of tomographic images or a specific tomographic image. The upper limit Lmax is the value obtained by subtracting the value Dmax - minimum value Dmin) x γ (γ is a real number less than 0.3), and the lower limit is the value obtained by adding (maximum value Dmax - minimum value Dmin) x γ to the minimum value. When the value Lmin is 255 when the signal value exceeds the upper limit Lmax, 0 when the signal value is below the lower limit Lmin, and linearly transforms the range from the lower limit Lmin to the upper limit Lmax from 0 to 255, the signal value is further comprising tomographic image gradation conversion means for creating a gradation reduction image converted to 8 bits;
The voxel image creating means refers to a color map for the tone-reduced image defined by associating the color component value and the opacity with the signal value, and creates each pixel of the plurality of tone-reduced images. Create an opacity voxel image in which the opacity is defined as the voxel value by replacing the signal value of each voxel arranged in three dimensions in correspondence with the opacity,
The color component voxel image creating means refers to the color map for the tone reduction image, and converts the signal values of each voxel three-dimensionally arranged in association with each pixel of the plurality of tone reduction images into color components. is replaced by the value of , and a color component voxel image is created in which the value of the color component is defined as the voxel value.

Further, in the volume rendering device of the present disclosure, the rendering means includes:
Assuming that the coordinate system obtained by projecting and transforming the opacity voxel image to the volume rendering image to be generated is a viewpoint coordinate system, in the viewpoint coordinate system, along the Z-axis direction from each pixel (x, y) of the volume rendering image, While obtaining the opacity from the opacity voxel image by performing the first coordinate transformation for each voxel coordinate (x, y, z) in the viewpoint coordinate system toward the lower limit from the upper limit of the Z axis, A search control mask that searches for opaque voxels whose transparency is not 0, and records the Z coordinate in the viewpoint coordinate system of the opaque voxel whose opacity is not 0 found first for each pixel (x, y) of the volume rendering image. a search control mask creating means to create;
A Z coordinate is obtained from the search control mask for each pixel (x, y) of the volume rendering image, and a predetermined light beam is emitted from the obtained Z coordinate toward the lower limit of the Z axis along the Z axis direction. When irradiating virtual light with intensity, the first coordinate transformation is performed for each voxel coordinate (x, y, z) in the viewpoint coordinate system to obtain the opacity from the opacity voxel image, and the opacity is 0 If a voxel that is not is found, a second coordinate transformation is performed on the coordinates (x, y, z) to obtain a color component composed of (R, G, B) from the color component voxel image, Repeating the process of attenuating the light intensity based on the opacity of the voxel and calculating the cumulative brightness value based on the opacity and color components of the voxel and the attenuated light intensity, and calculating the calculated cumulative brightness value and a ray casting means for providing a pixel value composed of (R, G, B) corresponding to the pixel (x, y) of the volume rendering image based on and

Further, in the volume rendering device of the present disclosure, when the search control mask creation means and the ray casting means perform the first coordinate transformation to acquire the opacity from the opacity voxel image,
Acquiring parameters of the predetermined coordinate transformation including a 3×3 matrix defining rotation in a predetermined three-dimensional coordinate system, offset values in each of the xyz axis directions, enlargement or reduction ratios in the xyz axis directions, and z-axis direction scaling factors. ,
Integer-value coordinates (x, y, z) of voxels in the viewpoint coordinate system are converted into the coordinate system of the opacity voxel image based on the parameters, and real-value coordinates ( X, Y, Z) is calculated,
Identifying a plurality of voxels of the opacity voxel image corresponding to a plurality of integer-valued coordinates (x', y', z') near the calculated real-valued coordinates (X, Y, Z);
The opacity obtained from the opacity voxel image is calculated based on the opacity of the specified plurality of voxels.

Further, in the volume rendering device of the present disclosure, when the ray casting means performs the second coordinate transformation to acquire color components from the color component voxel image,
Acquiring parameters of the predetermined coordinate transformation including a 3×3 matrix defining rotation in a predetermined three-dimensional coordinate system, offset values in each of the xyz axis directions, enlargement or reduction ratios in the xyz axis directions, and z-axis direction scaling factors. ,
Integer-valued coordinates (x, y, z) of voxels in the viewpoint coordinate system are converted into the coordinate system of the color component voxel image based on the parameters, and real-valued coordinates ( X, Y, Z) is calculated,
identifying a plurality of voxels of the color component voxel image corresponding to a plurality of integer coordinates (x', y', z') near the calculated real number coordinates (X, Y, Z);
The color components obtained from the color component voxel image are calculated based on the specified color components of the plurality of voxels.

Also, the volume rendering device of the present disclosure includes:
The rendering means is
3D texture registration means for generating a 3D texture image composed of the color component voxel images;
coordinate transformation means for performing a predetermined coordinate transformation on the 3D texture image to generate a post-transformation 3D texture image;
Lamination in which quadrilaterals on the XY coordinate plane of a three-dimensional space are arranged in the Z-axis direction, and each three-dimensional coordinate of four vertices of each quadrangle is associated with each of four predetermined three-dimensional coordinates of the post-conversion 3D texture image. a laminated quadrangle setting means for setting a quadrangle;
A color component composed of (R, G, B) of a voxel of the converted 3D texture image corresponding to the three-dimensional coordinates on the stacked quadrangle on the line of sight parallel to the Z-axis direction from a predetermined viewpoint is set to the voxel The color components obtained by alpha blending in the order of rectangles farther from the viewpoint based on the opacity of the volume rendering image are given as pixel values composed of (R, G, B) of the volume rendering image. calculating means;
characterized by comprising

Further, in the volume rendering device of the present disclosure, the coordinate transformation means is
A 4x4 matrix defining rotation in a predetermined three-dimensional coordinate system, a predetermined get the parameters of the coordinate transformation of
The 3D texture image is characterized in that the predetermined coordinate transformation using the acquired parameters is performed on the 3D texture image to generate the post-transformation 3D texture image.

Further, the volume rendering apparatus of the present disclosure is characterized in that the coordinate transformation means and the pixel value calculation means are executed using a GPU and frame memory installed in a video card of a general-purpose computer. .

Also, in the present disclosure, the computer is
By referring to a color map defined by associating color component values and opacity values with signal values, the signal values of each voxel three-dimensionally arranged in correspondence with each pixel of the plurality of tomographic images are set to be unknown. Opacity voxel image creation means for creating an opacity voxel image in which opacity is defined as a voxel value by replacing with transparency;
With reference to the color map, the signal value of each voxel three-dimensionally arranged in association with each pixel of the plurality of tomographic images is replaced with a color component value, and the color whose color component value is defined as the voxel value color component voxel image creating means for creating component voxel images;
rendering means for generating a volume rendered image using the opacity voxel image and the color component voxel image;
Provide a program to function as

Also, in the present disclosure, the computer is
By referring to a color map defined by associating color component values and opacity values with signal values, the signal values of each voxel three-dimensionally arranged in correspondence with each pixel of the plurality of tomographic images are set to be unknown. Opacity voxel image creation means for creating an opacity voxel image in which opacity is defined as a voxel value by replacing with transparency;
With reference to the color map, the signal value of each voxel three-dimensionally arranged in association with each pixel of the plurality of tomographic images is replaced with a color component value, and the color whose color component value is defined as the voxel value color component voxel image creating means for creating component voxel images;
opacity-added color component voxel image creating means for creating an opacity-added color component voxel image by adding the opacity of the corresponding voxel of the opacity voxel image to each voxel of the color component voxel image;
rendering means for generating a volume rendering image using the color component voxel image with opacity;
Provide a program to function as

Also, in this disclosure:
A voxel structure composed of voxels determined by referring to a predefined color map, corresponding to each pixel of a plurality of two-dimensional tomographic images of an object photographed at predetermined intervals. voxel data,
The voxel data is a voxel structure including at least an opacity voxel image in which opacity is defined as the value of each voxel and a color component voxel image in which a color component value is defined as the value of each voxel. has a data structure of
The opacity voxel image and the color component voxel image provide voxel data characterized in that they are used to generate a volume rendered image by a rendering means.

Also, in this disclosure:
A voxel structure composed of voxels determined by referring to a predefined color map, corresponding to each pixel of a plurality of two-dimensional tomographic images of an object photographed at predetermined intervals. voxel data,
The voxel data includes at least an opacity voxel image in which opacity is defined as the value of each voxel, and a color component voxel image with opacity in which a color component value and opacity are defined as the value of each voxel. has a data structure of voxel structures, including
The opacity voxel image is used to generate the color component voxel image with opacity, and the color component voxel image with opacity is used to generate a volume rendering image by rendering means. Provides voxel data.

According to the present disclosure, it is possible to apply a color map to a plurality of tomographic images and perform processing for generating rendering images at high speed.

1 is a hardware configuration diagram of a volume rendering device 100 according to an embodiment of the present disclosure; FIG. 1 is a functional block diagram showing the configuration of a volume rendering device according to an embodiment; FIG. 4 is a flowchart showing processing operations of the volume rendering device according to the embodiment; It is a figure which shows the color map used by this embodiment. 1 is a conceptual diagram of a voxel image created by a volume rendering device 10 according to an embodiment of the present disclosure; FIG. 4 is a flowchart showing details of color correction processing for color components of transparent voxels. FIG. 4 is a diagram showing correspondence in 3D texture mapping; FIG. 4 is a diagram showing the configuration of the rendering means 70 when performing 3D texture mapping method processing; 4 is an explanatory diagram showing an example of projection screen setting by rendering means 70 of the present embodiment; FIG. 4 is a flowchart showing details of rendering processing by a 3D texture mapping method; FIG. 4 is a diagram showing the configuration of the rendering means 70 when performing ray-casting processing; 4 is a flowchart showing details of rendering processing by the ray casting method; 4 is a conceptual diagram of coordinate transformation; FIG. 7 is a diagram for explaining a procedure for creating a search control mask by a search control mask creating means 77; FIG. 4 is a flowchart for explaining a procedure for search control mask creating means 77 to search for starting point coordinates in each pixel.

Preferred embodiments of the present disclosure will be described in detail below with reference to the drawings.
<1. Device configuration>
FIG. 1 is a hardware configuration diagram of a volume rendering device 100 according to an embodiment of the present disclosure. A volume rendering apparatus 100 according to the present embodiment can be realized by a general-purpose computer, and as shown in FIG. , a large-capacity storage device 3 such as a hard disk, SSD (Solid State Drive), flash memory, etc. for storing programs and data executed by the CPU 1, and an instruction input I / F (interface) 4 such as a keyboard, mouse, etc. , a data input/output I/F (interface) 5 for data communication with an external device such as a data storage medium, a display unit 6 which is a display device such as a liquid crystal display, and an arithmetic processing unit specializing in graphics. A GPU (Graphics Processing Unit) 7 and a frame memory 8 for holding an image to be displayed on the display unit 6 are provided, and are connected to each other via a bus. Since the calculation result by the GPU 7 is written to the frame memory 8, the GPU 7 and the frame memory 8 are often mounted on a video card having an interface to the display unit 6 and attached to a general-purpose computer via a bus.

FIG. 2 is a functional block diagram showing the configuration of the volume rendering device according to this embodiment. 2, 10 is tomographic image reading means, 15 is color map reading means, 20 is ROI clipping setting means, 30 is voxel image creating means, 40 is smoothing means, 50 is shading means, and 60 is transparent voxel color interpolation means. , 70 is rendering means, and 80 is rendering image output means.

The tomographic image reading means 10 is means for reading a plurality of tomographic images from PACS collected and stored by medical image diagnostic equipment such as CT, MRI, and PET via a storage medium and an input unit. A tomographic image is obtained by photographing an object at predetermined intervals, and is a two-dimensional tomographic image in which a signal value is assigned to each pixel. The color map reading means 15 is a means for reading a color map created in advance by a color map creating means (not shown) from a storage medium or an input section. The ROI clipping setting means 20 is a means for setting an ROI, which is a region of interest, as a voxel image creation target among a plurality of read tomographic images. The voxel image creating means 30 refers to a color map defined by associating color component values and opacities with signal values, and refers to each pixel of a plurality of tomographic images arranged three-dimensionally. It is means for creating an opacity voxel image, a color component voxel image, and a color component voxel image with opacity by giving color component values and opacity to voxels. In this embodiment, the voxel image creating means 30 includes opacity voxel image creating means 31, color component voxel image creating means 32, and color component voxel image creating means 33 with opacity. Means 31 creates an opacity voxel image in which opacity is defined as voxel values, and color component voxel image creating means 32 creates a color component voxel image in which color component values are defined as voxel values. An opacity-added color component voxel image creating means 33 creates an opacity-added color component voxel image in which color component values and opacity are defined as voxel values.

The smoothing means 40 is a means for smoothing the opacity by replacing the opacity of each voxel constituting the opacity voxel image with the average value of the opacity of the voxel and its neighboring voxels. The shading means 50 adds the opacity of voxels near the voxel of the opacity voxel image corresponding to the color component composed of (R, G, B) of each voxel of the color component voxel image. Based on this, the gradient vector in the voxel is calculated, the calculated gradient vector is used to calculate the shade value, and each value of the color components (R, G, B) is multiplied by the shade value and replaced with a value. be.

The color correction means 60 corrects the color component of the voxel (transparent voxel), which is the outermost voxel in the color component voxel image and whose opacity is 0 in the corresponding opacity voxel image, to the color component near the voxel. This is a color correcting means for performing a color correcting process such as replacing with an average value of color components of voxels in a color component voxel image corresponding to voxels whose transparency is not zero. The rendering means 70 is a means for generating a color volume rendering image using the opacity voxel image and the color component voxel image, or the color component voxel image with opacity. The rendering means 70 supports two types of rendering processing, 3D texture mapping method and ray casting method. The rendering image output means 80 is means for outputting the volume rendering image to a predetermined output means in a predetermined manner. Normally, display output is performed on the display unit 6 or the like.

The tomographic image reading means 10 and the color map reading means 15 are realized mainly in the data input/output I/F 5 with the assistance of the CPU 1 . The ROI clipping setting means 20 is realized mainly in the instruction input I/F 4 with the assistance of the CPU 1 . The voxel image creating means 30 , the smoothing means 40 , the shadow adding means 50 and the color correcting means 60 are implemented by the CPU 1 executing programs stored in the storage device 3 . In the case of the 3D texture mapping method, the rendering means 70 is realized mainly by executing a program in the GPU 7 with the assistance of the CPU 1 . Moreover, in the case of the ray casting method, the rendering means 70 is implemented by the CPU 1 executing a program. The rendered image output means 80 is realized mainly in the frame memory 8 and the display section 6 with the assistance of the CPU 1 .

Each component shown in FIG. 2 is actually implemented by installing a dedicated program in hardware such as a computer and its peripherals, as shown in FIG. That is, the computer executes the contents of each means according to a dedicated program. In this specification, a computer means a device that has an arithmetic processing unit such as a CPU or GPU and is capable of data processing. It also includes computers embedded in mobile terminals and various devices.

<2. Processing operation>
Next, processing operations of the volume rendering apparatus shown in FIGS. 1 and 2 will be described. FIG. 3 is a flowchart showing processing operations of the volume rendering apparatus according to this embodiment. First, the tomographic image reading means 10 reads a plurality of tomographic images. After reading a plurality of DICOM-format tomographic images, the ROI clipping setting means 20 sets the ROI clipping (step S10). ROI is an abbreviation for Region of Interest, and means a region of interest. Here, the ranges for which volume rendering images and voxel images are to be created are shown. By setting the ROI, it is possible to generate a volume rendering image in which the subject is cut at an arbitrary position three-dimensionally, which is used to depict organs hidden in the body surface and bones, and the interior of the organs. In the processing after step S20, the ROI is set in step S10 so that the voxel image creation target is limited to a predetermined range.

Xso is the coordinate corresponding to the left end in the X-axis direction of each read tomographic image, Xeo is the coordinate corresponding to the right end in the X-axis direction, Yso is the coordinate corresponding to the upper end in the Y-axis direction, and Yso corresponds to the lower end in the Y-axis direction. Let Yeo be the coordinates, Zso be the coordinates corresponding to the tomographic image of the first slice, and Zeo be the coordinates corresponding to the tomographic image of the final slice. A range defined by a rectangular parallelepiped of [Xs, Xe], [Ys, Ye], and [Zs, Ze] satisfying <Ze≦Zeo is defined as an ROI and specified as a processing target. As a result of the ROI clipping setting in step S10, the number of pixels to be processed is (Xeo−Xso+1)×(Yeo−Yso+1)×(Zeo−Zso+1) pixels, and (Xe−Xs+1)×(Ye−Ys+1)×(Ze− Zs+1) pixels. By performing the ROI clipping setting in step S10 by the ROI clipping setting means 20, in addition to the effect that a volume rendering image in which the organ to be observed is exposed is obtained, the number of pixels to be processed is reduced, thereby suppressing the processing load and improving the responsiveness. Since there is a double merit that it can be improved, it is often set, but it is possible to omit it because it is not an essential process. (A method of creating a three-dimensional mask is also used as a method of depicting the organ to be observed, but the operation is more complicated than setting the ROI and the processing load increases, so the ROI clipping process is almost essential.) At the stage of ROI clipping setting in step S10, only the range to which ROI clipping is applied is defined, and specific clipping processing is executed in each processing after step S20. In each process, the processing load is reduced by performing calculations limited to the ROI range of the voxel image, and a clipped volume rendering image is generated in the rendering process of step S70.

Next, the voxel image creation means 30 performs voxel image creation processing (step S20). A voxel image is a voxel structure in which each pixel of a tomographic image is arranged as a voxel in a three-dimensional space. As a specific process in step S20, a value corresponding to the signal value of each pixel included in the ROI range of the tomographic image after clipping setting is assigned using the color map read from the color map reading unit 15. , to create a voxel image. In this embodiment, three types of voxel images are created: an opacity voxel image, a color component voxel image, and a color component voxel image with opacity.

FIG. 4 is a diagram showing a color map used in this embodiment. In FIG. 4, the signal value is the signal value recorded in each pixel of the tomographic image, R, G, and B are the color components of red, green, and blue, respectively, and α is the degree of display of the voxel. Opacity to specify. FIG. 4 is a color map when the signal values of the tomographic image are recorded in 16 bits and take values from -32768 to 32767. FIG. (CT images are often normalized to a range of -2048 to 2048 with the water component set to 0). As shown in FIG. 4, in the color map, the signal value of each pixel of the tomographic image is associated with each of the R, G, and B color components and the opacity α. By using such a color map, for each voxel of the voxel image corresponding to each pixel of the tomographic image, each of the R, G, and B color components corresponding to the signal value of that pixel and the opacity α are obtained as voxel values. A full-color volume rendering image can be created by giving .

By using a color map such as that shown in FIG. 4, a portion having a predetermined signal value in a tomographic image can be expressed with an arbitrary color component and an arbitrary opacity. In a tomographic image, since signal values differ according to parts such as organs, specific organs can be artificially colored based on differences in signal values by defining a color map in a volume rendering image. or make certain organs transparent to reveal hidden organs behind them. If the opacity α of a specific part is set to the maximum value (α=255), only that part will be clearly displayed, and the parts located behind that part will not be rendered. Conversely, if the opacity α of a specific portion is set to the minimum value (α=0), that portion becomes invisible (transparent), and the portion located behind that portion is exposed. If the opacity α of a specific portion is set to a halftone (0<α<255), that portion is displayed semi-transparently, and the portion located behind it is displayed as a watermark. Moreover, as the color components R, G, and B, arbitrary values can be set so as to facilitate visual recognition of the target portion. Therefore, a color map can be set for each site to be observed, and different volume rendering images can be obtained by using different color maps for the same group of tomographic images. That is, if the color map for lungs is used, a volume rendering image in which the lung portion is conspicuous is obtained, and if the color map for heart is used, a volume rendering image in which the heart portion is conspicuous is obtained. However, since the signal value distribution of each organ may overlap each other, it is often impossible to create a color map that renders only a specific organ. For example, in the color map for the heart, the ribs and sternum are also drawn at the same time, and the heart is hidden by these bones.

In this embodiment, it is possible to change the color map in use or read a different color map by operating via the instruction input I/F 4 . Then, the volume rendering device generates a volume rendering image by referring to the changed color map or the newly loaded color map. Since the volume rendering apparatus of this embodiment can generate a volume rendering image at high speed, it can update the volume rendering image in real time in response to changes in the color map.

In step S20, the voxel image generating means 30 generates signals at each pixel (x, y, z) corresponding to the range of Xs<x<Xe, Ys<y<Ye, and Zs<z<Ze of a plurality of tomographic images. The color map is referred to using the values, and the values of R, G, B, which are the color components corresponding to the signal value, and the opacity α are obtained, and these values are used as the values of the corresponding voxels. For voxel (x,y,z) values in the range x≤Xs or Xe≤x or y≤Ys or Ye≤y or z≤Zs or Ze≤z uniformly R=G=B=α =0. A three-dimensional voxel image corresponding to the plurality of tomographic images is obtained by performing this processing on all pixels in the plurality of tomographic images. As a result, each voxel of the voxel image originally has four values of R, G, B, and α.

In this embodiment, the voxel image creating means 30 has an opacity voxel image creating means 31, a color component voxel image creating means 32, and a color component voxel image creating means 33 with opacity. , an opacity voxel image creating means 31 creates an opacity voxel image of 1 voxel 8-bit format in which only the opacity α is recorded in each voxel, and a color component voxel image creating means 32 records the color components R, G, and B. A 1-voxel 24-bit format color component voxel image recorded in each pixel is created. When volume rendering is performed by the 3D texture mapping method, which will be described later, the opacity-added color component voxel image creation means 33 creates a 1-voxel 32-bit format in which the color components R, G, and B and the opacity α are recorded in each pixel. Create a color component voxel image with transparency. The reason for this is that the voxel image is registered as a 3D texture image in OpenGL (an Open Source Graphics Library based on which the program code for controlling the GPU is automatically generated when rendering processing is described). This is because it must be created in the voxel 32-bit format.

Specifically, the opacity voxel image creating means 31 calculates each pixel (x, y, z) corresponding to the range of Xs<x<Xe, Ys<y<Ye, and Zs<z<Ze of a plurality of tomographic images. The color map is referred to using the signal value corresponding to , the value of α corresponding to the signal value is acquired, and that value is used as the value of the corresponding voxel. A value of 0 is set for voxels (x, y, z) in the range of x≦Xs, Xe≦x, y≦Ys, Ye≦y, z≦Zs, or Ze≦z. By performing this process on all pixels in a plurality of tomographic images, a three-dimensional opacity voxel image corresponding to the plurality of tomographic images can be obtained. The value of each voxel in the opacity voxel image is α only, as described above.

In addition, the color component voxel image creating means 32 or the color component voxel image creating means 33 with opacity has each pixel ( x, y, z) are used to refer to the color map, obtain the values of the R, G, and B color components corresponding to the signal values, and use those values as the values of the corresponding voxels. . R=G=B=0 is set for voxels (x, y, z) in the range of x≦Xs or Xe≦x or y≦Ys or Ye≦y or z≦Zs or Ze≦z. Furthermore, the opacity-added color component voxel image creating means 33 sets the opacity of all voxels of the opacity-added color component voxel image to be created to 0. By performing this processing on all pixels in a plurality of tomographic images, a three-dimensional color component voxel image or opacity-added color component voxel image corresponding to the plurality of tomographic images is obtained. Each voxel of the color component voxel image has three values of R, G, and B as described above. Each voxel of the color component voxel image with opacity has four values of R, G, B, and α as described above (where α is all set to 0 in the initial state). For example, when each component is recorded with 8 bits, 1 voxel of the opacity voxel image is recorded with 8 bits, 1 voxel of the color component voxel image is recorded with 24 bits, and 1 voxel of the color component voxel image with opacity is recorded with 8 bits. Voxels are recorded in 32 bits. As will be described later, the combination of the opacity voxel image in 1-voxel 8-bit format and the color component voxel image in 1-voxel 24-bit format is suitable for the case where the CPU 1 performs rendering processing using the ray casting method. The combination of the opacity voxel image in the format and the color component voxel image with opacity in the 1 voxel 32-bit format is suitable for rendering processing by the 3D texture mapping method using the GPU7.

FIG. 5 is a conceptual diagram of an opacity voxel image and a color component voxel image or a color component voxel image with opacity. 5A shows an opacity voxel image represented by a cube, and FIG. 5B shows a color component voxel image or an opacity-added color component voxel image represented by a cube. In FIG. 5, the number of voxels is set to 6×6×6 for convenience of explanation. As shown in FIG. 5, the opacity voxel image and the color component voxel image or the color component voxel image with opacity have the same number of voxels in each three-dimensional axial direction, and the voxels of the two correspond to each other on a one-to-one basis. there is However, the opacity voxel image can be recorded in 8 bits per voxel, whereas the color component voxel image can be recorded in 24 bits per voxel. 32-bit recording is possible. This difference in recording capacity is expressed as the size of voxels in FIG.
The two voxel images schematically shown in FIG. 5 are actually recorded on a memory such as the RAM 3 for each voxel image. Therefore, as shown in FIG. 5A, small opacity voxel images are collectively recorded in a small area even on the memory, and the access speed by the CPU 1 and GPU 7 is greatly improved.
Voxel data corresponding to the ray casting method corresponds to each pixel of a plurality of two-dimensional tomographic images, is a voxel structure composed of each voxel determined with reference to a color map, and the value of each voxel and a color component voxel image in which the color component value is defined as the value of each voxel.
Voxel data corresponding to the 3D texture mapping method corresponds to each pixel of a plurality of two-dimensional tomographic images, and is a voxel structure composed of each voxel determined with reference to a color map. It has a voxel structure data structure including an opacity voxel image in which opacity is defined as a value and a color component voxel image with opacity in which a color component value and opacity are defined as the value of each voxel. ing.

By creating an opacity voxel image and a color component voxel image or a color component voxel image with opacity as separate voxel images, it is possible to refer to only the opacity voxel image when referring to opacity. Become. Therefore, in the process of creating a volume rendering image, the access time required for frequent opacity references, particularly for opacity references of a plurality of neighboring voxels, is greatly reduced. As a result, it contributes to high-speed creation of volume rendering images and high-speed re-display processing associated with color map changes and the like.

Next, the smoothing means 40 performs smoothing processing (step S30). Specifically, the value of the opacity α of each voxel (x, y, z) in the range of Xs < x < Xe, Ys < y < Ye, and Zs < z < Ze of the opacity voxel image is Smooth using neighboring voxels. As neighboring voxels, for example, 26 voxels adjacent in each direction of the xyz axis can be used. Therefore, voxel (x, y, z) is smoothed using 27 voxels belonging to x−1 to x+1, y−1 to y+1, z−1 to z+1 including itself. Various methods can be used as a specific smoothing method. In this embodiment, the average value of 27 voxels is given as the value of the target voxel.

Smoothing processing is performed on voxels (x, y, z) in the range of Xs < x < Xe, Ys < y < Ye, and Zs < z < Ze of the opacity voxel image, and x ≤ Xs or Xe ≤ x or For voxels (x, y, z) in the range y≦Ys or Ye≦y or z≦Zs or Ze≦z, no changes are made and the original opacity α=0 remains recorded. By performing the smoothing process on substantially the entire opacity voxel image in step S30, it is possible to suppress the occurrence of moire when the volume rendering image is displayed later.

Next, the shadow addition means 50 performs shadow addition processing (step S40). This is a process of adding a shadow corresponding to a predetermined lighting environment to a voxel image. In this process, the color component voxel image in the range of Xs<x<Xe, Ys<y<Ye, and Zs<z<Ze is updated. For this purpose, a certain parallel light source and ambient light components are set, and shadows to be added are calculated. Specifically, first, a light source vector (Lx, Ly, Lz) and an ambient light component Ab are set as unit vectors indicating the directions of the parallel light sources. The numerical values of the light source vector (Lx, Ly, Lz) and the ambient light component Ab can be appropriately set according to the situation of the image to be expressed. .57735, 0.57735) and set Ab=0.2. Lx, Ly, and Lz are real numbers with absolute values less than 1 (including negative values), and Ab is a real number of 0 or more and 1 or less. Then, the gradient vector (Gx, Gy, Gz) of the opacity voxel image is calculated by executing the process according to [Equation 1] below.

[Formula 1]
Gx=(Vα(x+1, y, z)−Vα(x−1, y, z))・(Rxy/Rz)
Gy=(Vα(x,y+1,z)−Vα(x,y−1,z))・(Rxy/Rz)
Gz = (Vα (x, y, z+1) - Vα (x, y, z−1))
G=(Gx ² +Gy ² +Gz ² ) ^1/2

In [Equation 1], Vα (x, y, z) is the opacity at the voxel (x, y, z) of the opacity voxel image, and Rxy is the resolution of the tomographic image (the reciprocal of the pixel interval, the number of pixels per mm , the X direction and the Y direction are the same), Rz is the slice resolution of the tomographic image (reciprocal of the slice interval of the tomographic image, the number of slices of the tomographic image per mm), G is the gradient vector (Gx, Gy, Gz) It's size. Rxy/Rz is defined as the Z-axis direction scaling factor. You may multiply the reciprocal of the magnification. As shown in [Equation 1], the shading means 50 calculates the difference value of the opacity of two voxels in each of the X-axis direction, Y-axis direction, and Z-axis direction of the voxel of the opacity voxel image as a gradient vector. A unit vector calculated as an X-direction component, a Y-direction component, and a Z-direction component, and corrected for the X-direction/Y-direction component or the Z-direction component based on a predefined Z-axis direction scaling factor is used as the gradient at the voxel. Compute as a vector. Further, when G≧1, the processing according to [Equation 2] below is executed to calculate the luminance value S(x, y, z) including only the diffuse reflection component. S(x, y, z) is a real number of 0 or more and 1 or less.

[Formula 2]
S (x, y, z) = (1-Ab) (|Gx · Lx + Gy · Ly + Gz · Lz |) / G + Ab

On the other hand, when the value of G calculated in [Equation 1] is G<1, it is a uniform region (in the background air, lung fields, etc.) where there is no opacity gradient, or it is the outermost region. Therefore, the luminance value S(x, y, z) is set to 0 without adding shading. In [Equation 1], |Gx·Lx+Gy·Ly+Gz·Lz| indicates an absolute value, and even when "Gx·Lx+Gy·Ly+Gz·Lz" is a negative value, S(x, y, z) is a negative value. I try not to.

Then, using the calculated luminance value S (x, y, z), the process according to the following [Formula 3] is executed to obtain each voxel value V of the color component voxel image or the color component voxel image with opacity. (x, y, z, c) is corrected to obtain each corrected voxel value V'(x, y, z, c).

[Formula 3]
V'(x, y, z, c)=S(x, y, z) V(x, y, z, c)

In [Equation 3], c=0, 1 and 2, which correspond to the R, G and B color components, respectively. Therefore, in the case of 8 bits for one color, 24 bits for 3 colors, so in the case of a color component voxel image, it is recorded as it is. In the case of a color component voxel image with opacity, after executing the process according to [Equation 3] for c=0, 1, 2, further, with c=3, according to the following [Equation 4], the corresponding A process for giving the value of the opacity voxel image is performed to obtain each voxel value V'(x, y, z, c) in which information is recorded in one voxel 32 bits.

[Formula 4]
V'(x, y, z, 3) = Vα(x, y, z)

In order to avoid complication of the formula, V'(x, y, z, c) after updating is treated as V(x, y, z, c) below.

The subsequent steps S50 and S60 are performed only when the 3D texture mapping method is used. The processing of steps S50 and S60 is not performed in the case of the ray casting method. Therefore, the color component voxel image with opacity is processed and the color component voxel image is not updated.

In the case of the 3D texture mapping method, when the boundary surface of ROI clipping is rotated by changing the angle and each voxel constituting the boundary surface is shifted stepwise, transparent voxels (R=G=B=α= 0) is interpolated by nearby opaque voxels (α>0) inside the ROI, and the values of R, G, B, and α change to non-zero values. As a result, transparent voxels on the boundary surface are visualized, and moire (interference fringes) are generated on the boundary surface of the volume rendering image with a stair-like period according to the rotation angle. (In the case of the ray casting method, this problem can be avoided by devising a coordinate conversion algorithm.) In order to suppress the occurrence of such moire, the color correction means 60 corrects transparent voxels located on the boundary surface (step S50). Here, a transparent voxel means a voxel whose opacity α value is 0 in the opacity voxel image. In this embodiment, there are two types of voxel images: an opacity voxel image and an opacity-added color component voxel image. have the same number of voxels, and each voxel is associated with each other. Therefore, by referring to the opacity voxel image, opaque voxels in the color component voxel image with opacity can be specified. In step S50, the opacity voxel image is only referenced and not updated. The color component voxel image with opacity is updated by changing the value of the color component of the voxel of the color component voxel image with opacity specified with reference to the opacity voxel image.

FIG. 6 is a flow chart showing the details of the color correction process for the color components of transparent voxels. In the processing shown in FIG. 6, the search for transparent voxels and opaque voxels uses an opacity voxel image in which only 8-bit opacity α is recorded in one voxel. Only the voxel image should be referred to. This greatly reduces the access time required for frequent opacity lookups, especially those of multiple nearby voxels, as described above. In the process shown in FIG. 6, first, a plurality of transparent voxels located on the outermost side of the opacity voxel image are extracted (S51). The outermost transparent voxels of the opacity voxel image are identified by x=Xs, x=Xe, y=Ys, y=Ye, z=Zs, z=Ze. That is, a voxel that satisfies any one of x=Xs, x=Xe, y=Ys, y=Ye, z=Zs, and z=Ze becomes the outermost transparent voxel of the opacity voxel image. Transparent voxels exist in the range of x≦Xs or Xe≦x or y≦Ys or Ye≦y or z≦Zs or Ze≦z by the above-described voxel image creation processing (S20), but x<Xs or In the range of Xe<x or y<Ys or Ye<y or z<Zs or Ze<z, there are no opaque voxels in the neighborhood of 26 and do not contribute to the generation of moire. Color correction processing is limited to transparent voxels located at . As described above, the opacity voxel image and the color component voxel image with opacity have the same number of voxels, and the voxels are associated with each other. Voxels can also be identified. Based on this correspondence relationship, one correction target voxel is extracted from the plurality of extracted voxels (S52). All the values (R, G, B, α) of the outermost voxels are set to 0 in advance by the voxel image creation process (S20) described above.

Next, an opaque voxel near the extracted voxel to be corrected is searched (S53). As the neighboring voxels, for example, 26 voxels adjacent in each of the xyz-axis directions can be used. Therefore, for the voxel (x, y, z) to be corrected, α>0 among the 26 voxels belonging to x−1 to x+1, y−1 to y+1, z−1 to z+1 excluding itself Explore opaque voxels. Specifically, the number of opaque voxels satisfying α>0 among the 26 voxels in the neighborhood is counted. [ 5], the value of the color component of the voxel to be corrected extracted in S52 is replaced with the average value of the color component of C nearby opaque voxels. Update (S55).

[Formula 5]
V (x, y, z, 0) = Σ _{k = -1} , 1 Σ _{j = -1} , 1 Σ _{i = -1, 1; V} α _{(x + i, y + j, z + k) > 0} V(x+i, y+j, z+k, 0)/C
V (x, y, z, 1) = Σ _{k = -1} , 1 Σ _{j = -1} , 1 Σ _{i = -1, 1; V} α _{(x + i, y + j, z + k) > 0} V(x+i, y+j, z+k, 1)/C
V(x,y,z,2)= _{Σk=-1,1Σj =-1,1Σi =-1,1;Vα (x+i,y+j,z+k)>0} V(x+i, y+j, z+k, 2)/C

On the other hand, in the process of step S53, if there is no opaque voxel in the neighborhood, that is, if there are C (C=0) opaque voxels satisfying α>0 in the 26 nearby voxels (S54: NO) , the color component values of the extracted voxels to be corrected are set to 0 (S56).

Subsequently, in step S51, it is determined whether or not processing has been completed for all the extracted transparent voxels (S57). If not completed, the process returns to step S52 to perform processing for extracting the next voxel to be corrected. The processing of steps S52 to S57 is repeatedly executed, and when all the extracted transparent voxels have been processed, the color correction processing of the color components of the transparent voxels in step S50 ends.

After completing the color correction processing of the color components of the transparent voxels in step S50, the corrected color component voxel image with opacity is registered in the 3D texture image (S60). At this time, first, the opacity-added color component voxel image creating means 33 adds the opacity of the corresponding voxel of the opacity voxel image to each voxel of the corrected opacity-added color component voxel image, and Complete the voxel image. That is, in the color component voxel image with opacity, one voxel is in a 32-bit format, and the R, G, and B color components are already recorded in 24 bits. Record α in 8 bits. Note that, as shown in [Formula 4], if each voxel value in which information is recorded in 32 bits per voxel has already been generated in step S40, there is no need to perform this step here. In step S40, if the opacity α of the opacity voxel image is not recorded in the color component voxel image with opacity, in step S60, the process according to [Equation 4] is performed to set the opacity α. Record at 8 bits to complete the color component voxel image with opacity.
Then, the rendering means 70 registers the color component voxel image with opacity as a 3D texture image used when executing the 3D texture mapping method.

FIG. 7 is a diagram showing correspondence in 3D texture mapping. FIG. 7(a) shows a 3D texture map in the texture coordinate system, and FIG. 7(b) shows the stacked rectangles defined in the world coordinate system. The 4 vertices of each quadrangle that makes up the stacked quadrangle are associated with the 4 voxels of the 3D texture image, and each quadrangle is the basis of the texture map when it is not rotated as shown in FIG. There is a one-to-one correspondence with the tomographic image, and the four vertices of each quadrangle correspond to the four corner pixels of the tomographic image.

Specifically, as shown in FIG. 7B, 3D texture mapping is performed on the defined stacked quadrilaterals. In this case, the 3D world coordinates of the four vertices of each quadrangle in the world coordinate system (corresponding to the viewpoint coordinate system) are associated with the 3D texture coordinates of the 3D texture image in the texture coordinate system (corresponding to the voxel coordinate system). As shown in FIG. 7, a quadrilateral consisting of four vertices (-1, -1, z), (-1, 1, z), (1, -1, z), (1, 1, z) , the world coordinates (-1, -1, z) are associated with the texture coordinates (0, 0, r).

As will be described later, when there is an angle change instruction, a converted 3D texture image is generated in which the texture map in FIG. 7A is three-dimensionally rotated in the texture coordinate system. Correspondence breaks down. The color component voxel image with opacity corrected for the color component is registered as a 3D texture image via OpenGL (graphic API).

Subsequently, rendering processing is performed (S70). The rendering process supports two methods, 3D texture mapping method and ray casting method. First, the 3D texture mapping method will be described.

<2.1. 3D Texture Mapping>
FIG. 8 is a diagram showing the configuration of the rendering means 70 when performing processing of the 3D texture mapping method. As shown in FIG. 8, the rendering means 70 has a 3D texture registration means 71, a coordinate conversion means 72, a stacked rectangle setting means 73, and a pixel value calculation means 74. As described above, in the case of the 3D texture mapping method, the rendering means 70 is realized mainly by executing a program in the GPU 7 with the assistance of the CPU 1. However, the coordinate transformation means 72 and the pixel The value calculation means 74 is preferably executed using the GPU and frame memory installed in the video card of the general purpose computer.

In the processing of the 3D texture mapping method, first, a projection screen is set. FIG. 9 is an explanatory diagram showing an example of projection screen setting by the rendering means 70 of this embodiment. The rendering means 70 sets the screen size (number of vertical and horizontal pixels, vertical and horizontal aspect ratio) of the rendered image. The rendering means 70 sets either parallel projection (normal external rendering) or perspective projection (endoscopic mode). The "rendering image" is synonymous with the "volume rendering image", and hereinafter, the "volume rendering image" may be referred to as the "rendering image". When the perspective projection is set, the rendering means 70 sets perspective projection parameters. Perspective projection parameters are composed of, for example, the viewing angle (focal length) of the camera and the viewpoint position (viewpoint and gaze point. The former is the position of the eyes and the latter is the position on the object being viewed, both on the Z axis. In general, the gaze point is fixed at the origin of the world coordinate system), the clipping position (distance on the Z-axis from the viewpoint, two points near and far), and the like. Note that the clipping position may be limited to near objects. As shown in FIG. 9, in the case of parallel projection, all lines of sight from the viewpoint are assumed to be parallel to the Z-axis, and the viewpoint is assumed to be virtually infinitely far away to the left. All stacked rectangles defined in the Z-axis direction are rendered. On the other hand, in the case of perspective projection, the line of sight spreads according to the viewing angle from the viewpoint, and the viewpoint enters the inside of the stacked quadrilateral. far rectangle).

FIG. 10 is a flow chart showing details of rendering processing by the 3D texture mapping method. First, the 3D texture registration means 71 in FIG. 8 registers the color component voxel image with opacity as a 3D texture image in OpenGL. Next, the coordinate transformation means 72 performs predetermined coordinate transformation on the 3D texture image to generate a post-transformation 3D texture image (S71 to S73).

Specifically, the coordinate transformation means 72 includes a 4×4 matrix defining rotation in a predetermined three-dimensional coordinate system, a viewing angle, a viewpoint position, a clipping position, an offset value in each direction of the xyz axes, an enlargement in the xyz axis direction, or Predetermined coordinate transformation parameters including reduction ratio and z-axis direction scaling ratio are obtained, and the 3D texture image is subjected to predetermined coordinate transformation using the obtained parameters to generate a post-transformation 3D texture image.

Describing individual steps, the coordinate transformation means 72 performs scaling and z-direction scaling processing on the 3D texture image (S71). Scaling is carried out by the same magnification Scale in the xyz-axis direction, but z-direction scaling corrects the difference between the resolution Rxy in the xy-axis direction and the resolution Rz in the z-axis direction, so only the z-axis direction is used. is enlarged or reduced by a designated magnification Scz (=Rxy/Rz). Then, the coordinate transformation means 72 performs rotation processing on the 3D texture image using a 4×4 matrix that defines rotation in a predetermined three-dimensional coordinate system (S72).

Then, the coordinate conversion means 72 performs offset processing on the 3D texture image using the offset values (Xoff, Yoff, Zoff) in each of the xyz-axis directions (S73). A converted 3D texture image is generated by these processes (S71 to S73). Next, the stacked quadrangle setting means 73 sets a stacked quadrangle composed of a plurality of quadrilaterals (S74). Specifically, a stacked quadrangle is set by arranging quadrangles on the XY coordinate plane of the three-dimensional space in the Z-axis direction. When setting 3D texture mapping, the stacked quadrangle setting means 73 sets a stacked quadrangle in which the same number of quadrangles as the number of xy plane images are arranged in the Z-axis direction.

The number of squares in the stacked squares can be set arbitrarily, but if the number of squares does not match the number of tomographic images, interpolation processing works in the Z-axis direction, rounding errors in coordinate values due to coordinate conversion accumulate, Striped or grid-like moiré occurs. Then, as shown in FIG. 7, by defining four voxel coordinates in the corresponding post-transformation 3D texture image for the four vertices of each quadrangle, correspondence with the post-transformation 3D texture image is performed ( S75). The voxel coordinates to be defined are not based on the coordinate system of color component voxel images with opacity, but on the texture coordinate system of OpenGL (real values ranging from 0 to 1 in the three axial directions of R, S, and T). be.

Specifically, as shown in FIG. 7B, 3D texture mapping is performed on the defined stacked quadrilaterals. In this case, the three-dimensional world coordinates of the four vertices of each quadrangle in the world coordinate system are associated with the three-dimensional texture coordinates of the 3D texture image in the texture coordinate system. As shown in FIG. 7, a quadrilateral consisting of four vertices (-1, -1, z), (-1, 1, z), (1, -1, z), (1, 1, z) , the world coordinates (-1, -1, z) are associated with the texture coordinates (0, 0, r).

Next, the pixel value calculator 74 performs scan conversion (filling in the inside of the rectangle) in order from the farthest rectangle on the line of sight parallel to the Z-axis direction from a predetermined viewpoint to the nearest rectangle to the viewpoint (S76). At this time, the inside of the rectangle is painted based on the color components (R, G, B) of the voxels of the 3D texture image after conversion attached to each rectangle, and when the same pixel is painted over, the voxels of the 3D texture image after conversion are painted. Also do alpha blending based on opacity. In this way, the color component (R, G, B) composed of (R, G, B) of the voxel of the converted 3D texture image corresponding to the XYZ coordinates of the stacked rectangle is obtained from the viewpoint based on the opacity of the voxel. Color components composed of (R, G, B) obtained by alpha blending in the order of the farthest rectangle are given as pixel values of the volume rendering image. Thereby, a volume rendering image is obtained (S76).

More specifically, all pixel values (RGB values) of the rendering image are initialized in advance with the background color (for example, R=G=B=0). The pixel value calculation means 74 performs scan conversion on the quadrangle furthest from the viewpoint while referring to the attached post-conversion 3D texture image. Parallel projection or perspective projection is performed on the rendered image defined in FIG. Let (i, j) be the coordinates of a pixel to be filled in the rendered image, and u be the interval between pixels in the rendered image (where u is the interval in world coordinates corresponding to one pixel in the rendered image. For example, u= 0.002) for scan conversion. At each scan-converted world coordinate (-1+iu, -1+ju, -1) (i, j are coordinate values of the rendered image, for example, integers 0≤i, j≤511), the corresponding texture coordinates (2iu , 2ju, 0), refer to the converted 3D texture image based on the calculated texture coordinates, obtain the color component values (RGB values) and opacity, and calculate the corresponding rendered image coordinates (i, j ) and the acquired opacity, alpha blending is performed, and the pixel value at the coordinates (i, j) of the corresponding rendering image is updated.

In this way, when the alpha blending process is completed for all corresponding pixels of the rendered image on which the single rectangle furthest from the viewpoint is projected, the Z coordinate is changed to v (the world coordinate where the rectangle is arranged in the Z-axis direction is increased by v. For example, v=2/Sz), and the same process is repeated for the next closest quadrangle (-1+iu, -1+ju, -1+v) to the viewpoint direction to update the rendered image. Rendered image after scan conversion for all quadrilaterals and alpha blending processing for world coordinates (-1+iu, -1+ju, -1+kv) (k is the quadrilateral number and an integer of 0≤k≤Sz-1) is completed. The alpha blending process is performed as a process according to [Formula 6] below.

[Formula 6]
R′=R・α+(1−α)・Rb
G′=G・α+(1−α)・Gb
B′=B・α+(1−α)・Bb

Here, R', G', and B' are the pixel values (RGB values) of the rendering image updated on the projection plane. R, G, B are the color components (RGB values) in the voxel (2iu, 2ju, 2kv) of the 3D texture image after conversion corresponding to the world coordinates (-1+iu, -1+ju, -1+kv) of the rectangle. corresponds to the value of α is also the α value in the transformed 3D texture image voxel (2iu, 2ju, 2kv) corresponding to the quadrangle world coordinates (−1+iu, −1+ju, −1+kv), and controls the overlapping ratio. Rb, Gb, and Bb are pixel values (RGB values) already recorded at the coordinates (i, j) of the rendered image corresponding to the world coordinates (-1+iu, -1+ju, -1+kv). The values of R', G', and B' calculated based on the above [Equation 6] correspond to the quadrangle located on the side opposite to the viewpoint and the quadrangle farthest from the viewpoint. , the background color (for example, Rb=Gb=Bb=0) is given.

<2.2. Ray casting>
Next, the case where the rendering process of step S70 is performed by the ray casting method will be described. FIG. 11 is a diagram showing the configuration of the rendering means 70 for ray casting processing. As shown in FIG. 11, the rendering means 70 has search control mask creating means 77 and ray casting means 78 .

FIG. 12 is a flowchart showing details of rendering processing by the ray casting method. In the processing to be described later, the search control mask creating means 77 and the ray casting means 78 perform coordinate transformation each time an opacity is obtained from an opacity voxel image. Prior to this process, the rendering means 70 sets coordinate transformation parameters that are commonly used when performing coordinate transformation in subsequent steps (step S210). The setting of the coordinate transformation parameters is performed by setting the three-dimensional coordinates in the viewpoint coordinate system based on the viewpoint so that the sequential coordinate transformation processing can be performed smoothly for each voxel when the voxel is irradiated with virtual light. This is a process for converting the value into a three-dimensional coordinate value in the voxel coordinate system in which the voxel image is defined, and obtaining the color component or opacity of the corresponding voxel in the voxel coordinate system. A conceptual diagram of coordinate transformation and specific examples of coordinate transformation parameters will be described later.

Next, the search control mask creation means 77 sets the starting point coordinates of the voxel where the process of integrating the light intensity of the virtual light and the luminance value (RGB value) of the pixel in each pixel of the rendering image is set to all pixels of the rendering image. Compute for pixels. If all voxels through which the virtual light passes are completely transparent (0 opacity), the raycasting process is necessary because the virtual light passes through the space of the voxel image while maintaining its light intensity. do not have. Therefore, in this embodiment, the z-coordinate in the viewpoint coordinate system of the first voxel whose opacity is not 0 when viewed from the viewpoint is searched along the line-of-sight direction. The z-coordinate of the voxel whose opacity is not 0, which is first found by searching, is called the origin coordinate. In step S220, the coordinates of the starting point are searched in the z-axis direction (the negative direction of the z-axis in this embodiment) of the viewpoint coordinate system for each pixel (x, y) of the rendering image. The search result is stored as a search control mask in association with the pixel coordinate values (x, y) of the rendering image. Examples of search control masks are described later.

Each pixel (x, y) of the rendered image is defined in a two-dimensional coordinate system. On the other hand, the voxel coordinate system is defined as a three-dimensional coordinate system, and a rendered image is obtained by projecting and transforming the three-dimensional voxel coordinate system into a two-dimensional coordinate system. However, in calculating the rendered image, the rendered image is placed at a predetermined z-coordinate (in this embodiment, the upper limit in the z-axis direction) of a three-dimensional coordinate system called a viewpoint coordinate system, and each pixel (x, A method of calculating the brightness value of each pixel (x, y) while adding the z-coordinate to y) and back-projecting the three-dimensional viewpoint coordinate system to the three-dimensional voxel coordinate system. The coordinate transformation parameters described above are set on the assumption that projection transformation is performed in the opposite direction, and the coordinate transformation in the ray casting method is assumed to be projection transformation in the opposite direction. (On the other hand, the coordinate transformation in the 3D texture mapping method described above adopts a method of performing projection transformation in the forward direction using a GPU). In the viewpoint coordinate system, it can be assumed that a plurality of voxels are arranged continuously in the line-of-sight direction (z-axis negative direction) with respect to the same coordinates (x, y). In calculating the luminance value of each pixel, virtual light is projected in the z-direction from a predetermined z-coordinate position in the viewpoint coordinate system, and each voxel through which the virtual light passes is subjected to coordinate transformation to obtain three-dimensional voxel coordinates. It will obtain the value of the color component of the voxel of the corresponding color component voxel image in the system and the opacity of the voxel of the opacity voxel image. The predetermined z-coordinate can be obtained from the search control mask calculated in advance by the search control mask creating means 77. The search control mask creating means 77 also performs coordinate conversion to obtain the coordinates for each voxel through which the virtual light passes. Coordinate transformation parameters calculated in step S210 can be used, including the transformation.

In step S220, a voxel whose opacity is not 0 is searched, so there is no need to refer to the color component of the voxel. Therefore, the search control mask creation means 77 refers only to the opacity voxel image and not to the color component voxel image in step S220. Coordinate conversion processing converts a single voxel at coordinates (x, y, z) in the viewpoint coordinate system corresponding to each pixel (x, y) of the rendered image into a three-dimensional voxel image defined as a voxel image. A process of obtaining the color component values (RGB) and opacity of the voxel at the corresponding coordinates (x', y', z') in the coordinate system, where the coordinate values (x', y', z') are Since it is a real number and has a fraction, arithmetic processing for interpolating the color component values (RGB) and opacity of each voxel corresponding to the coordinate values of integers in the vicinity of the coordinate values (x', y', z') is performed. I need. By adopting a method of referring only to the opacity voxel image without referring to the color component voxel image, the calculation load of the coordinate conversion process is reduced to 1/4.

The ray-casting means 78 projects virtual light in the z-negative direction in the viewpoint coordinate system from the z-coordinate of the starting point searched in step S220 at each pixel (x, y) of the rendered image, and the virtual light passes through. Voxel opacity and color component of the corresponding opacity voxel image in the three-dimensional voxel coordinate system for each voxel Light intensity of the virtual light and luminance value (RGB value) of the pixel using the color component of the voxel of the voxel image is obtained by accumulating , and the pixel value of each pixel (x, y) of the rendered image is calculated. Also in step S230, coordinate transformation is performed using the coordinate transformation parameters calculated in step S210, and both the color component value and the opacity of the corresponding voxel in the voxel coordinate system are obtained for each voxel in the viewpoint coordinate system. There is a need. The coordinate conversion process is also performed in step S230 each time a voxel in the viewpoint coordinate system is referred to.

FIG. 13 is a conceptual diagram of coordinate transformation. Since a rendered image is an image viewed from a viewpoint, the viewpoint coordinate system that serves as a reference for each pixel (x, y) of the rendered image does not necessarily match the voxel coordinate system. Therefore, the ray casting means 78 needs to convert the voxel image defined in the voxel coordinate system to the viewpoint coordinate system. However, for example, when a voxel image composed of 256 slice images of 512×512 pixels is converted into the viewpoint coordinate system, x: 512×y: 512×z: It is necessary to create a voxel image transformed into a viewpoint coordinate system of 512 voxels, which requires 512 cubic coordinate transformations. On the other hand, when searching for the coordinates of the starting point, if a voxel whose opacity is not 0 is immediately found, there is no need to refer to subsequent voxels in the z direction. If the light intensity of the virtual light is attenuated below a predetermined value, the integration process can be terminated, and there is no need to refer to subsequent voxels in the z direction. That is, most of the 512 3 voxels that have been transformed into the viewpoint coordinate system are not referred to, so there is no need to perform coordinate transformation and hold all the voxels of the voxel image.

Without generating a voxel image coordinate-transformed to the viewpoint coordinate system as shown in FIG. 13(b), the virtual light indicated by the downward arrow in FIG. Each time, the coordinate transformation process is executed in the opposite direction to the voxel coordinate system in which the voxel image is defined. Since coordinate transformation processing is not executed for voxels that are not referred to, a well-known method of generating all voxel images that have been transformed in advance from the voxel coordinate system to the viewpoint coordinate system and then performing the origin coordinate search and ray casting processing is used. In comparison, memory for holding three-dimensional voxel images is not required, the number of coordinate transformations can be reduced, and the memory access load and computation load can be suppressed.

Various coordinate transformation parameters are used when performing the coordinate transformation. If the positional relationship between the viewpoint coordinate system and the voxel coordinate system is the same, it is considered that the coordinate transformation parameters are also the same. That is, in the search control mask creating means 77 and the ray casting means 78, the same coordinate transformation parameters are used for carrying out coordinate transformation to the voxel coordinate system for all voxels referred to in the viewpoint coordinate system. Therefore, the rendering means 70 calculates coordinate transformation parameters in advance in step S210, and uses the coordinate transformation parameters as common coordinate transformation parameters in subsequent steps. Examples of coordinate transformation parameters include the following.

・3×3 matrix defining rotation in a predetermined three-dimensional coordinate system (the inverse matrix of the 3×3 matrix specified and defined by the instruction input I/F 4 because projection transformation is performed in the opposite direction)
・XYZ axis offset values: Xoff, Yoff, Zoff (unit: voxel)
- Enlargement or reduction ratio: Use the same value for each of the ScaleXYZ axes.
Z-direction scaling factor: Scz=Rxy/Rz, the ratio between the XY-direction resolution Rxy and the Z-direction resolution Rz

FIG. 14 is a diagram for explaining the procedure for the search control mask creating means 77 to create the search control mask in step S220. The search control mask creation means 77 creates a search control mask describing the starting point coordinates for each pixel (x, y) of the rendering image. The coordinates of the starting point refer to the voxel having a non-zero opacity value that is found first by searching the voxel from the viewpoint (z=upper limit) in the negative z-axis direction (z>0) in the viewpoint coordinate system. is the z-coordinate value (positive or 0 value) at . If, as a result of searching, no voxel with an opacity value other than 0 is found, that is, if all voxels are transparent up to the end in the z direction, a negative value (-1) is set as the origin coordinate.

For the coordinates (x, y) of all pixels in the rendered image, it is necessary to first search for the starting point coordinates for projecting the virtual light. A process that needs to refer to the opacity of multiple voxels in a direction and interpolates the opacity values of multiple voxels in the corresponding voxel coordinate system by performing a coordinate transformation process to obtain the opacity of each voxel , the computational load is relatively large. Therefore, by adopting a method of creating a two-dimensional search control mask, it is possible to omit the search process by using the starting point coordinates searched for in the vicinity of the starting point coordinates for some of the coordinates. In this embodiment, a search control mask is created by the following procedure in order to suppress the computational load of search processing for the coordinates of the origin executed for each pixel.

The search control mask creating means 77 first generates even-numbered pixels (2m, 2n) (m=0, 1, . . . , n=0, 1, . Search the origin coordinates for . This corresponds to the shaded portion in FIG. 14(a). Generally, the number of pixels in a rendering image is often set to an even number such as 512×512, and the interpolation processing described later cannot set the origin coordinates of odd-numbered pixels at the end of the image. , the right-end pixel in the x-axis direction and the bottom-end pixel in the y-axis direction are also searched for the starting point coordinates.

The search control mask creation means 77 selects the odd-numbered pixels (2m+1, 2n) in the x-direction among the even-numbered pixels in the y-axis direction of the rendered image, and extracts the previously searched even-numbered pixels in the x-direction. Check if the origin coordinates of the pixels are the same. For example, for the pixel (1,0), it is checked whether the origin coordinates of the pixel (0,0) and the origin coordinates of the pixel (2,0) are the same. However, even if the pixel at the right end in the x direction is odd-numbered, it is excluded from this check processing because the coordinates of the starting point have already been set as shown in FIG. 14(a). If the origin coordinates of both neighbors are the same, it is assumed that the object has a similar shape in the vicinity of the pixel, so the origin coordinates of the intervening pixels are also the same as the origin coordinates of both neighbors. can be regarded as In addition, if the pixel on both sides of the pixel is in the background portion and has a negative starting point coordinate value, the pixel sandwiched between them can also be regarded as being in the background portion. Therefore, the search control mask creating means 77 sets the same starting point coordinates for the pixel (2m+1, 2n) when the starting point coordinates on both sides of the pixel (2m+1, 2n) are the same. The vertically hatched portion in FIG. 14(b) corresponds to this. On the other hand, if the starting point coordinates on both sides are different, the search control mask creating means 77 executes the process of searching for the starting point coordinates and sets the searched starting point coordinates in the same way as the even-numbered pixels on both sides.

The search control mask creating means 77 determines whether or not, for all pixels (x, 2n+1) of odd-numbered pixels in the rendered image in the y-direction, the starting point coordinates of even-numbered pixels on both sides in the y-direction are the same. To check. For example, for the pixel (0,1), it is checked whether or not the starting point coordinates of the pixel (0,0) and the starting point coordinates of the pixel (0,2) are the same. However, even if the pixel in the lower end row in the y direction is odd-numbered, it is excluded from this check processing because the starting point coordinates have already been set as shown in FIG. 14(b). When the origin coordinates of both adjacent pixels are the same, the same origin coordinates are set for the intervening pixels in the same manner as in the x direction. Then, if the starting point coordinates on both sides are different, the search control mask creating means 77 executes the process of searching for the starting point coordinates and sets the searched starting point coordinates in the same way as in the x direction. The horizontally hatched portion in FIG. 14(c) corresponds to this.

FIG. 14 shows a method of searching the starting point coordinates every other pixel in the vertical and horizontal directions, and interpolating one pixel sandwiched between them based on the adjacent pixels on both sides, but this is only an example. 1) It is also possible to search for the coordinates of the starting point every other pixel and interpolate each of the M pixels interposed between them based on the adjacent pixels on both sides. can.

In FIG. 14, the procedure for speeding up the setting of the starting point coordinates by omitting part of the starting point search processing based on the starting point coordinates searched for in the adjacent pixels in the xy direction has been described. Instead of or in combination with this, by speeding up the z-direction starting point search process itself, the calculation load of the starting point search can be suppressed doubly.

The search control mask generator 77 looks up the opacity of each voxel along the z-direction of the line-of-sight coordinate system, starting from the viewpoint. First, the voxel opacity at the viewpoint position (z = upper limit) is checked, and if the opacity is not 0, it is the surface (cut plane) of the subject, so the search is terminated at the origin coordinate z = upper limit, Low computational load. However, when the opacity of voxels at the viewpoint position (z = upper limit) is 0 (many in the background), there is a high possibility that transparent voxels are continuously present in the z direction. If the opacity of all voxels to the end is 0 and the method of sequentially referencing consecutive voxels in the z direction is taken, all voxels It is necessary to refer to the opacity of voxels while performing coordinate transformation for , which entails a huge computational load. Therefore, when the opacity of a voxel at the viewpoint position (z=upper limit) is 0, the opacity is referenced while skipping an arbitrary number of voxels instead of sequentially referencing adjacent voxels. For example, refer to the opacity every 8 voxels in the z direction. If a voxel whose opacity is not 0 is not found even at the end voxel in the z direction, a negative value is set as the coordinates of the starting point, and the search is terminated. On the other hand, if the first voxel with non-zero opacity (not completely transparent) is found, the process moves to accurately determine the origin coordinates.

The search control mask creating means 77 searches for the first voxel whose opacity is 0 by going back in the z-axis positive direction toward the viewpoint from the opacity voxel found first. At this time, the opacity of voxels adjacent in the z-direction of the viewpoint coordinate system is sequentially referred to for each voxel with the skip width as the upper limit. When the first voxel whose opacity is 0 is found, the z coordinate in the viewpoint coordinate system of the previous voxel (negative direction moving away from the viewpoint) is set as the origin coordinate and stored in the search control mask. If the first voxel whose opacity is 0 is not found by tracing back to the upper limit of the skip width, the z coordinate in the viewpoint coordinate system of the first found opaque voxel is stored in the search control mask as the starting coordinate. The above procedure can speed up the search for the starting point in the z direction.

The search control mask creating means 77 refers to the opacity of voxels arranged in the z direction with the viewpoint coordinate system (x, y, z) as a reference. However, what is actually referred to is the opacity of the voxel in the voxel coordinate system, so in order to obtain the actual opacity of the voxel, coordinate conversion from the viewpoint coordinate system to the voxel coordinate system is required. Parameters for coordinate transformation may be calculated in advance in step S210. The coordinate values (x', y', z') of the voxel coordinate system calculated by coordinate conversion are real numbers, but must be converted to integer values in order to refer to voxels in the voxel coordinate system. At this time, it is known that if a single voxel in the voxel coordinate system is referred to by truncating the decimal point to an integer, moiré will occur in the rendered image. By summing the values weighted according to the numerical value after the decimal point of the real number coordinate value for the opacity of voxels of integer coordinate values in the vicinity of the coordinate value (x', y', z'), A method of calculating transparency is desirable. The same applies to the ray-casting process, which will be described later, but in the ray-casting process, it is necessary to refer not only to the voxel opacity but also to the RGB 3-value color component values. can be suppressed to 1/4 of the processing load of coordinate transformation in ray casting processing.

FIG. 15 is a flow chart for explaining the procedure for the search control mask creating means 77 to search for the coordinates of the starting point in each pixel. The search control mask creation means 77 implements this flow chart for each target pixel (x, y). Each step in FIG. 15 will be described below.

The search control mask creating means 77 sets the target pixel (x, y) and the three-dimensional coordinates (x, y, z) for searching the coordinates of the starting point. The initial value of the z coordinate is z=Zs (upper limit). Zs is the z-coordinate of the viewpoint.

The search control mask creating means 77 performs coordinate transformation on the three-dimensional coordinates (x, y, z) to calculate the opacity α of the corresponding voxel of the opacity voxel image. As a calculation procedure, for example, a method of weighted addition of the opacities of adjacent voxels as described above can be used. This coordinate transformation becomes the first coordinate transformation. If α is not 0, the process proceeds to step S305, and if α is 0, the process proceeds to step S303.

In step S303, the search control mask creating means 77 skips a predetermined number of voxels in the z-axis direction. Specifically, m (for example, m=8) is subtracted from the current z-coordinate. As a result of the subtraction, if the z-coordinate is greater than or equal to 0, the process returns to step S302 and repeats the same processing. If the z-coordinate is less than 0, the process proceeds to step S304.

In step S304, the search control mask creation unit 77 creates a value ( For example, a negative value such as -1 is set, and the search processing for the coordinates of the starting point is terminated.

When proceeding to step S305, the search control mask creating means 77 initializes the variable i and then increments it by 1 (S306). If i reaches m, or if i is added to the current z-coordinate and the viewpoint z-coordinate is reached, the process proceeds to step S308, where the current z-coordinate set the value obtained by adding i before incrementing to . If i has not reached m and adding i to the current z-coordinate does not reach the viewpoint z-coordinate, the process proceeds to step S307.

In step S307, the search control mask creation means 77 performs coordinate transformation on the three-dimensional coordinates (x, y, z+i) to calculate the voxel opacity α of the corresponding opacity voxel image. The calculation procedure is the same as in step S302. This coordinate transformation is also the first coordinate transformation. If α is not 0, the process returns to step S306, i is incremented, and the same processing is repeated. If α is 0, the process proceeds to step S308.

In step S308, the search control mask creation means 77 calculates the value M(x, y) of the search control mask corresponding to the current target pixel (x, y) by the z coordinate (z+i) one before step S307. -1) is set to end the origin coordinate search processing.

The ray casting means 78 refers to the starting point coordinates held by the search control mask for the (x, y) coordinates of each pixel of the rendering image to be obtained, and casts virtual light in the z direction from the starting point coordinates in the viewpoint coordinate system. For each voxel (x, y, z) projected and through which the virtual light passes, use the color component value and opacity of the corresponding voxel (x', y', z') in the voxel coordinate system The light intensity of the virtual light and the luminance value (RGB value) of the pixel are integrated for each voxel in the z direction, and until the light intensity of the virtual light attenuates below a predetermined value or reaches the end voxel in the z direction. The process of accumulating is continued, and the pixel value (RGB value) of the pixel (x, y) of the rendering image is calculated and provided based on the finally accumulated luminance value (RGB value) of the pixel.

During the raycasting process, a voxel at a certain z-coordinate may have zero opacity (ie, be transparent). Since the light intensity of the virtual light and the luminance (RGB value) of the pixel are not integrated for the transparent voxel, the transparent voxel is skipped and the next opaque voxel is searched. At this time, the ray casting means 78 starts searching from a transparent voxel in the middle where z>0, which is shifted from the viewpoint in the z direction of the viewpoint coordinate system, and the search control mask creating means 77 searches for the coordinates of the starting point. A similar technique can be used to quickly search for the first voxel with non-zero opacity. In the ray casting process, not only the voxel opacity but also the color component values consisting of three RGB values are required. can be calculated, and the calculation load for coordinate transformation can be reduced to 1/4. When a voxel whose opacity is not 0 is found, the ray-casting process is resumed from that voxel, and the ray-casting process is continued until the light intensity of the virtual light is attenuated below a predetermined value or the voxel at the end of the z direction is reached. . In this way, the ray casting process can be sped up by skipping the process of referring to the color component value and opacity while performing coordinate conversion for transparent voxels existing in the middle of the ray casting process. can.

When the ray casting means 78 refers to color components and opacity, it refers to these values of voxels in the corresponding voxel coordinate system based on the viewpoint coordinate system (x, y, z). Therefore, coordinate conversion from the viewpoint coordinate system to the voxel coordinate system is required. As for the coordinate transformation parameters, those set in the coordinate transformation parameter setting (S210) are used. However, in the ray casting process, not only voxel opacity but also color components consisting of three RGB values are required. Therefore, the decimal point of the real number coordinate value for the color component and the opacity of the voxel corresponding to the eight neighboring integer coordinate values adjacent to the real number coordinate value (x', y', z') in the coordinate-transformed voxel coordinate system A method of obtaining the color component and opacity of a voxel is adopted by summing values weighted according to the following numerical values. This coordinate transformation is the second coordinate transformation. In this case, it is necessary to refer to the opacity voxel image and the color component voxel image.

<3. Decimation of 1/N>
In the volume rendering device, the voxel image creation means 30 creates a voxel image with the same number of voxels as the total number of pixels of the plurality of tomographic images, but the voxel image is created by thinning out the voxels to reduce the number of voxels. You may do so. In that case, the opacity voxel image creation means 31 associates every N pixels in each of the three axial directions of the two-dimensional XY-axis direction of the plurality of tomographic images and the Z-axis direction orthogonal to the tomographic image to create a three-dimensional For each placed voxel, the signal value of that voxel is replaced with the opacity α to create an opacity voxel image in which the opacity is defined as the voxel value. In addition, the color component voxel image creating means 32 or the color component voxel image creating means 33 with opacity is arranged in each of the two-dimensional XY-axis directions of the plurality of tomographic images and the Z-axis direction orthogonal to the tomographic images, For each voxel arranged three-dimensionally in association with every N pixels, the signal value of the voxel is replaced with the value of the color component, and a color component voxel image or opacity-added voxel image in which the color component value is defined as the voxel value Create a color component voxel image. For example, when N=2, each of the X, Y, and Z directions can be thinned out by 1/2, so the number of voxels to be processed is reduced to 1/8, and high-speed processing can be performed. As a result, even when the user continuously switches the color map interactively via the instruction input I/F 4, the volume rendering image with coarse image quality is sequentially displayed, and the color map is updated. A volume rendering image can be displayed while tracking.

<4. Gradation Reduction of Tomographic Image>
In the above embodiment, a tomographic image in which each pixel records a 16-bit signal value is used as it is to create a voxel image. A voxel image may be created later. By converting the tomographic image into 8-bit gradation, the load in processing each pixel can be reduced, contributing to high-speed generation of volume rendering images.

In this case, tomographic image tone conversion means is further provided, and tone conversion is performed each time the tomographic image reading means 10 reads a single tomographic image. Specifically, a series of DICOM format tomographic images Do (x, y, z) (-32768 ≤ Do (x, y, z) ≤ 32767, 0 ≤ x ≤ Sx-1, 0 ≤ y ≤ Sy-1 , 0 ≤ z ≤ Sz-1; resolution: Rxy, Rz), the Sz/2th intermediate tomographic image Do (x, y, z/2) is first read, and all pixels in the same tomographic image Let Dmin be the minimum value of and Dmax be the maximum value of . For Dmin and Dmax, a method of determining the minimum and maximum values from all tomographic images may be used, but if a method of determining only from intermediate tomographic images is used, all slices of the large-capacity 16-bit original tomographic image may be used. Only the gradation-compressed tomographic image D8(x, y, z), which is a gradation-reduced image, can be constructed directly on the memory without reading it and storing it on the memory. Subsequently, a process according to [Equation 7] below is executed to calculate the lower limit value Lmin and the upper limit value Lmax.

[Formula 7]
Lower limit Lmin = (Dmax-Dmin) γ + Dmin
Upper limit value Lmax=(Dmax−Dmin)・(1−γ)+Dmin

In [Equation 7], γ is the contrast adjustment width of the gradation-compressed image, and the closer to 0, the higher the contrast but the lower the luminance. γ is a real value less than 0.3, but is usually set to γ=0.1. Since the luminance contrast of the rendered image can be adjusted by various settings such as a color map, γ may be fixed. Then, the range between the lower limit value Lmin and the upper limit value Lmax is compressed in 256 steps to obtain a gradation-compressed tomographic image D8(x, y, z). Specifically, the gradation compression tomographic image D8 (x, y, z) (0≤D8 (x, y, z)≤255, 0≤x≤Sx-1, 0≤y≤Sy-1, 0≤ When z≦Sz−1; resolution: Rxy, Rz), the process according to the following [Equation 8] is executed to calculate a gradation compression tomographic image D8(x, y, z).

[Formula 8]
D8 (x, y, z) = (Do (x, y, z) - Lmin) 255 / (Lmax - Lmin)
If D8(x,y,z)>255: D8(x,y,z)=255 If D8(x,y,z)<0: D8(x,y,z)=0

As shown in the second formula of [Equation 8], when the signal value exceeds the upper limit value Lmax, it is set to 255, and when it is below the lower limit value Lmin, it is set to 0, and the range from the lower limit value Lmin to the upper limit value Lmax is linearly from 0 to 255. Convert. When creating a voxel image using a gradation-compressed tomographic image, a color map for 8-bit signal values is prepared as a color map for a gradation-reduced image instead of the color map in FIG. . Specifically, a color map in which R, G, B, and α are recorded in association with signal values 0 to 255 is used.

Although the preferred embodiments of the present disclosure have been described above, the present disclosure is not limited to the above embodiments, and various modifications are possible. Modified aspects that do not depart from the gist of the present disclosure are included in the technical scope of the present disclosure.

1 CPU (Central Processing Unit)
2 RAM (random access memory)
3... Storage device 4... Instruction input I/F
5 Data input/output I/F
6... Display unit 7... GPU
8 frame memory 10 tomographic image reading means 15 color map reading means 20 ROI clipping setting means 30 voxel image creating means 40 smoothing means 50 shading addition Means 60 Color Correcting Means 70 Rendering Means 71 3D Texture Registering Means 72 Coordinate Transforming Means 73 Laminated Rectangle Setting Means 74 Pixel Value Calculating Means 77 Searching Control mask creating means 78 Ray casting means 80 Rendering image output means 100 Volume rendering device

Claims (17)

To generate a volume rendering image by referring to a predefined color map based on a plurality of two-dimensional tomographic images in which the object is photographed at predetermined intervals and a signal value is assigned to each pixel. a volume rendering device of
By referring to a color map defined by associating color component values and opacity values with signal values, the signal values of each voxel three-dimensionally arranged in correspondence with each pixel of the plurality of tomographic images are set to be unknown. opacity voxel image creation means for creating an opacity voxel image in which opacity is defined as a voxel value by replacing with transparency;
With reference to the color map, the signal value of each voxel three-dimensionally arranged in association with each pixel of the plurality of tomographic images is replaced with a color component value, and the color whose color component value is defined as the voxel value a color component voxel image creating means for creating a component voxel image;
rendering means for generating a volume rendered image using the opacity voxel image and the color component voxel image;
A volume rendering device comprising: Smoothing means for performing a smoothing process to replace the opacity of each voxel constituting the opacity voxel image with the average value of the opacity of the voxel and voxels in the vicinity of the voxel,
2. The volume rendering apparatus according to claim 1, wherein said rendering means generates a volume rendering image using a smoothed opacity voxel image. To generate a volume rendering image by referring to a predefined color map based on a plurality of two-dimensional tomographic images in which the object is photographed at predetermined intervals and a signal value is assigned to each pixel. a volume rendering device of
By referring to a color map defined by associating color component values and opacity values with signal values, the signal values of each voxel three-dimensionally arranged in correspondence with each pixel of the plurality of tomographic images are set to be unknown. opacity voxel image creation means for creating an opacity voxel image in which opacity is defined as a voxel value by replacing with transparency;
With reference to the color map, the signal value of each voxel three-dimensionally arranged in association with each pixel of the plurality of tomographic images is replaced with a color component value, and the color whose color component value is defined as the voxel value a color component voxel image creating means for creating a component voxel image;
adding the opacity of the corresponding voxel of the opacity voxel image to each voxel of the color component voxel image to create an opacity-added color component voxel image in which the values of the opacity and the color component are defined as voxel values; Transparency color component voxel image creating means;
rendering means for generating a volume rendering image using the color component voxel image with opacity;
A volume rendering device comprising: Smoothing means for performing a smoothing process to replace the opacity of each voxel constituting the opacity voxel image with the average value of the opacity of the voxel and voxels in the vicinity of the voxel,
4. The volume rendering apparatus according to claim 3, wherein said color component voxel image with opacity creating means creates said color component voxel image with opacity using a smoothed opacity voxel image. The value of the color component of the voxel that is the outermost in the color component voxel image and has an opacity of 0 in the opacity voxel image corresponding to the color component voxel image is further comprising color correction means for performing a color correction process of replacing the voxels in the color component voxel image corresponding to the voxels with non-zero opacity near the 0 voxels with the average value of the color components of the voxels;
5. The opacity-added color-component voxel image creation means creates the opacity-added color-component voxel image using a color-component voxel image that has undergone color correction processing. Volume rendering device. The value of the color component of each voxel of the color component voxel image,
Based on the opacity of voxels near the voxel of the opacity voxel image corresponding to the voxel, calculate a gradient vector in the voxel,
calculating a shadow value based on the calculated gradient vector and a predetermined light source vector;
6. The volume rendering apparatus according to any one of claims 1 to 5, further comprising shading adding means for replacing the value of the color component with a value obtained by multiplying the shading value. In calculating the gradient vector, the shading means calculates the difference value of the opacity of two voxels near the voxel of the opacity voxel image in each of the X-axis direction, the Y-axis direction, and the Z-axis direction as the gradient vector. and calculating a unit vector obtained by correcting the Z-direction component based on a predefined Z-axis direction scaling factor as a gradient vector in the voxel. 7. The volume rendering device according to claim 6, characterized by: The opacity voxel image creating means is arranged three-dimensionally in association with every N pixels in each of the two-dimensional axial directions of the plurality of tomographic images and in each of the three axial directions perpendicular to the tomographic images. It replaces the signal value of the voxel with opacity and creates an opacity voxel image in which the opacity is defined as the voxel value,
The color component voxel image creating means is arranged three-dimensionally in association with every N pixels in each of the two-dimensional axial directions of the plurality of tomographic images and each of the three axial directions perpendicular to the tomographic images. 8. A color component voxel image in which the voxel signal values are replaced with color component values to create a color component voxel image in which the color component values are defined as the voxel values. The volume rendering device according to . When the tomographic image is an image having a 16-bit signal value, the maximum value Dmax and the minimum value Dmin of the signal value are calculated based on all of the plurality of tomographic images or a specific tomographic image. The upper limit Lmax is the value obtained by subtracting the value Dmax - minimum value Dmin) x γ (γ is a real number less than 0.3), and the lower limit is the value obtained by adding (maximum value Dmax - minimum value Dmin) x γ to the minimum value. When the value Lmin is 255 when the signal value exceeds the upper limit Lmax, 0 when the signal value is below the lower limit Lmin, and linearly transforms the range from the lower limit Lmin to the upper limit Lmax from 0 to 255, the signal value is further comprising tomographic image gradation conversion means for creating a gradation reduction image converted to 8 bits;
The opacity voxel image creating means refers to a color map for tone-reduced images defined by associating color component values and opacities with signal values, and Replace the signal value of each voxel three-dimensionally arranged in correspondence with the pixel with opacity, create an opacity voxel image in which the opacity is defined as the voxel value,
The color component voxel image creating means refers to the color map for the tone reduction image, and converts the signal values of each voxel three-dimensionally arranged in association with each pixel of the plurality of tone reduction images into color components. 9. The volume rendering apparatus according to claim 1, wherein a color component voxel image in which color component values are defined as voxel values is created. The rendering means is
Assuming that the coordinate system obtained by projecting and transforming the opacity voxel image to the volume rendering image to be generated is a viewpoint coordinate system, in the viewpoint coordinate system, along the Z-axis direction from each pixel (x, y) of the volume rendering image, While obtaining the opacity from the opacity voxel image by performing the first coordinate transformation for each voxel coordinate (x, y, z) in the viewpoint coordinate system toward the lower limit from the upper limit of the Z axis, A search control mask that searches for opaque voxels whose transparency is not 0, and records the Z coordinate in the viewpoint coordinate system of the opaque voxel whose opacity is not 0 found first for each pixel (x, y) of the volume rendering image. a search control mask creating means to create;
A Z coordinate is obtained from the search control mask for each pixel (x, y) of the volume rendering image, and a predetermined light beam is emitted from the obtained Z coordinate toward the lower limit of the Z axis along the Z axis direction. When irradiating virtual light with intensity, the first coordinate transformation is performed for each voxel coordinate (x, y, z) in the viewpoint coordinate system to obtain the opacity from the opacity voxel image, and the opacity is 0 If a voxel that is not is found, a second coordinate transformation is performed on the coordinates (x, y, z) to obtain a color component composed of (R, G, B) from the color component voxel image, Repeating the process of attenuating the light intensity based on the opacity of the voxel and calculating the cumulative brightness value based on the opacity and color components of the voxel and the attenuated light intensity, and calculating the calculated cumulative brightness value and a ray casting means for providing a pixel value composed of (R, G, B) corresponding to the pixel (x, y) of the volume rendering image based on 3. The volume rendering device according to claim 1 or 2, wherein: When the search control mask creation means and the ray casting means perform the first coordinate transformation to obtain the opacity from the opacity voxel image,
Acquiring parameters of the predetermined coordinate transformation including a 3×3 matrix defining rotation in a predetermined three-dimensional coordinate system, offset values in each of the xyz axis directions, enlargement or reduction ratios in the xyz axis directions, and z-axis direction scaling factors. ,
Integer-value coordinates (x, y, z) of voxels in the viewpoint coordinate system are converted into the coordinate system of the opacity voxel image based on the parameters, and real-value coordinates ( X, Y, Z) is calculated,
Identifying a plurality of voxels of the opacity voxel image corresponding to a plurality of integer-valued coordinates (x', y', z') near the calculated real-valued coordinates (X, Y, Z);
11. The volume rendering apparatus according to claim 10, wherein the opacity obtained from the opacity voxel image is calculated based on the opacity of the specified plurality of voxels. When the ray casting means performs the second coordinate transformation to acquire color components from the color component voxel image,
Acquiring parameters of the predetermined coordinate transformation including a 3×3 matrix defining rotation in a predetermined three-dimensional coordinate system, offset values in each of the xyz axis directions, enlargement or reduction ratios in the xyz axis directions, and z-axis direction scaling factors. ,
Integer-valued coordinates (x, y, z) of voxels in the viewpoint coordinate system are converted into the coordinate system of the color component voxel image based on the parameters, and real-valued coordinates ( X, Y, Z) is calculated,
identifying a plurality of voxels of the color component voxel image corresponding to a plurality of integer coordinates (x', y', z') near the calculated real number coordinates (X, Y, Z);
11. The volume rendering apparatus according to claim 10, wherein the color components obtained from the color component voxel image are calculated based on the specified color components of the plurality of voxels. The rendering means is
3D texture registration means for generating a 3D texture image composed of the color component voxel images;
coordinate transformation means for performing a predetermined coordinate transformation on the 3D texture image to generate a post-transformation 3D texture image;
Lamination in which quadrilaterals on the XY coordinate plane of a three-dimensional space are arranged in the Z-axis direction, and each three-dimensional coordinate of four vertices of each quadrangle is associated with each of four predetermined three-dimensional coordinates of the post-conversion 3D texture image. a laminated quadrangle setting means for setting a quadrangle;
A color component composed of (R, G, B) of a voxel of the converted 3D texture image corresponding to the three-dimensional coordinates on the stacked quadrangle on the line of sight parallel to the Z-axis direction from a predetermined viewpoint is Pixel value calculation in which color components obtained by alpha blending in the order of rectangles farther from the viewpoint based on opacity are given as pixel values composed of (R, G, B) of the volume rendering image. means and
6. A volume rendering apparatus according to any one of claims 3 to 5, comprising: The coordinate transformation means is
A 4x4 matrix defining rotation in a predetermined three-dimensional coordinate system, a predetermined get the parameters of the coordinate transformation of
14. The volume rendering apparatus according to claim 13, wherein the 3D texture image is subjected to the predetermined coordinate transformation using the acquired parameters to generate the post-transformation 3D texture image. 15. A volume rendering apparatus according to claim 14, wherein said coordinate transformation means and said pixel value calculation means are executed using a GPU and frame memory installed in a video card of a general-purpose computer. the computer,
With reference to a color map defined by associating color component values and opacity with signal values, a plurality of two - dimensional images are captured at predetermined intervals with respect to the object, and each pixel is given a signal value. Opacity voxel image creation means for creating an opacity voxel image in which opacity is defined as a voxel value by replacing the signal value of each voxel arranged three-dimensionally in correspondence with each pixel of the dimensional tomographic image with opacity;
With reference to the color map, the signal value of each voxel three-dimensionally arranged in association with each pixel of the plurality of tomographic images is replaced with a color component value, and the color whose color component value is defined as the voxel value color component voxel image creating means for creating component voxel images;
rendering means for generating a volume rendered image using the opacity voxel image and the color component voxel image;
A program to function as the computer,
With reference to a color map defined by associating color component values and opacity with signal values, a plurality of two - dimensional images are captured at predetermined intervals with respect to the object, and each pixel is given a signal value. Opacity voxel image creation means for creating an opacity voxel image in which opacity is defined as a voxel value by replacing the signal value of each voxel arranged three-dimensionally in correspondence with each pixel of the dimensional tomographic image with opacity;
With reference to the color map, the signal value of each voxel three-dimensionally arranged in association with each pixel of the plurality of tomographic images is replaced with a color component value, and the color whose color component value is defined as the voxel value color component voxel image creating means for creating component voxel images;
opacity-added color component voxel image creating means for creating an opacity-added color component voxel image by adding the opacity of the corresponding voxel of the opacity voxel image to each voxel of the color component voxel image;
rendering means for generating a volume rendering image using the color component voxel image with opacity;
A program to function as

Images