JP2860048B2 - Nonlinear image conversion apparatus and standard image calculation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a non-linear mapping process between medical images typified by MRI and PET images, a mapping of a distorted ground image taken from a satellite onto a standard map, and a different notation method. The present invention relates to a non-linear image conversion device that performs a non-linear mapping widely required in the field of image processing, such as automatic conversion of the above map, and a standard image calculation method for forming a standard image for a plurality of collected sample images.

[0002]

2. Description of the Related Art The operation of superimposing tomographic images obtained from two brains having different shapes on one another has been generally limited to linear transformation in which a constant magnification is applied vertically and horizontally. Even if such matching can be performed, it has been almost impossible to map a medical image having a different internal tissue distribution for each subject with a different magnification for each corresponding tissue. The superposition of the images required here is a non-linear transformation in which the image drawn on the rubber film is partially expanded and contracted to match another image, but it is a smooth transformation that maintains the neighborhood relationship with the surroundings. It implicitly assumes conversion by a function. Incidentally, recently, Journal of Computer Assisted Tomography (Journal of Computer Assassin)
vol. 15, N
o. 4 (1991) Friston (Karl J.F.)
paper, “Plastic Transformation of P.E.
Tea Images (Plastic Transfo
"Ration of PET Images)", the one-dimensional density distribution function obtained by scanning the tomographic image of the brain is subjected to integral conversion, and under the condition that the distribution of the displacement amount has a smooth relationship. Perform an inverse transformation and obtain two P
Attempts have also been made to align ET images.

[0003]

Since the conventional non-linear image conversion is configured as described above, it is intended to correct the positional displacement between two-dimensional images by one-dimensional conversion, so that a medical image conversion is performed. Each partial image corresponds to each other while preserving the microscopic neighborhood relationship, such as in the case of automatically correcting image differences due to individual differences in tissue distribution in and imaging system defects found in satellite images etc. When it is necessary to realize a smooth non-linear transformation for moving to a position, there has been a problem that satisfactory results are not always obtained.

The inventions described in claims 1 to 3 have been made to solve the above-mentioned problems, and in a two-dimensional image, in a two-dimensional space, in a three-dimensional image. An object of the present invention is to provide a nonlinear image conversion device capable of generating a pixel motion vector while maintaining a phase relationship in a three-dimensional space.

Another object of the present invention is to provide a standard image calculation method capable of easily forming a standard image for a plurality of sample images collected.

[0006]

According to a first aspect of the present invention, there is provided a non-linear image conversion apparatus, comprising: a slide mechanism for sliding a small area of a first image to be deformed on a reference second image; A moving amount calculating mechanism for calculating a moving amount that maximizes the similarity between the image of the small area and the second image, a cumulative calculating mechanism for cumulatively adding the moving amounts to calculate an area moving vector, and each of these area moving vectors It is equipped with a contradiction mitigation mechanism that alleviates the contradiction between them.

Further, according to a second aspect of the present invention, there is provided a nonlinear image conversion apparatus for internally dividing an obtained peripheral area movement vector according to a distance to a pixel to generate a pixel movement vector. A mechanism is provided.

According to a third aspect of the present invention, there is provided a non-linear image conversion device, comprising: a marker providing mechanism for providing a marker to a finite number of points corresponding to each other between the first image and the second image; A marker movement vector generating mechanism for generating a marker movement vector for matching the upper marker with the corresponding marker on the second image is provided.

In the standard image calculation method according to the present invention, each of the collected specimen images is divided into small areas, which are slid on the provisional standard image to set both of them. The amount of movement that maximizes the similarity is obtained, and the true standard image is calculated from the inter-sample average vector between each sample image of the pixel movement vector calculated based on the moving amount and the temporary standard image.

[0010]

The non-linear image conversion apparatus according to the first aspect of the present invention alternately performs microscopic matching for each small area and resolution of inconsistencies between the area movement vectors generated by the matching. Effectively accumulates microscopic information, heuristically grows matching search candidate areas, leads from the part to the entire self-organization, and performs direct non-linear transformation of images in two or three dimensions. Perform effectively.

Further, the internal division interpolation mechanism according to the second aspect of the present invention internally divides the obtained peripheral area motion vector according to the distance to the pixel to generate a pixel motion vector. The amount of change for each pixel is made smooth.

According to a third aspect of the present invention, there is provided a marker moving vector generating mechanism for moving a marker on the first image assigned by the marker attaching mechanism to a corresponding marker on the second image. Generating a vector enables non-linear image conversion including forced association.

According to a fourth aspect of the present invention, there is provided a standard image calculation method, comprising dividing a sample image into small areas, obtaining a pixel movement vector by microscopic matching with a provisional standard image, and calculating the pixel movement vector. By calculating the true standard image from the inter-sample average vector between each sample image of the vector and the temporary standard image, the configuration of the standard image from many collected sample images is facilitated.

[0014]

【Example】

Embodiment 1 FIG. Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing one embodiment of the invention described in claim 1. In the figure, reference numeral 1 denotes a first image layer that holds a first image to be deformed divided into small areas, and 2 denotes a second image layer that holds a second image serving as a reference of the first image. It is. Reference numeral 3 denotes a slide mechanism for sliding the small area of the first image on the second image, and reference numeral 4 denotes a movement for obtaining the movement amount of the small area in which the degree of similarity between the image of the small area and the second image is maximum. It is a quantity calculation mechanism. Reference numeral 5 denotes a cumulative operation mechanism for cumulatively adding the movement amounts obtained in this way to obtain an area movement vector, and 6 denotes an image of the small area by mitigating the contradiction between the obtained area movement vectors of each small area. This is an inconsistency mitigation mechanism that generates an area movement vector that holds the phase relationship between them and feeds it back to the slide mechanism 3.

Next, the operation will be described. First, a small area Dij obtained by dividing the first image held in the first image layer 1 is moved by the slide mechanism 3 to a predetermined range centered on the tip of the area movement vector DVij whose initial value is zero. (For example, within the range of the radius R), the second image layer 2
Is slid on the second image held in. The movement amount dVij when the small area Dij is slid to the top of the most similar second image within the range is obtained by the movement amount calculation mechanism 4. Next, a new movement vector DVij (t + 1) is obtained by accumulatively adding the movement amount dVij according to the following equation by the accumulation operation mechanism 5.

DVij (t + 1) = DVij (t) + dVij ‥‥‥‥‥‥‥‥ (1)

The above operation is performed for all small areas on the first image. In each of the small areas, a movement vector is generated independently. However, in order to keep the phase relation of the image as much as possible, in order to prevent the movement vectors in the neighborhood relation from being inconsistent, the following operation is performed by the inconsistency mitigation mechanism 6, An area movement vector DVij maintaining the phase relationship is obtained.

[0018]

(Equation 1)

However, the above-mentioned area ε1 is obtained by dividing the eight points on the smallest square surrounding the grid point (i, j) on the first image into the area ε1.
2 further represents 16 points on the outer square. Sn is a normalization coefficient represented by Sn = c + 8 + 16b, and coefficients b and c are parameters indicating how much importance is placed on the phase relationship.

The area movement vector from which the contradiction has been removed is fed back to the slide mechanism 3 to provide the center point of the slide range when the small area is slid on the second image in the next sequential process.

The present invention is characterized in that an area movement vector that maximizes the similarity found by a slide within a certain range is calculated for all the small areas, and after the inconsistency is eliminated, the slide range is calculated according to the area movement vector. That is, by repeating the operation of changing the matching search range a plurality of times, a good mapping relationship with good matching is sequentially found.

Next, a specific configuration of the movement amount calculating mechanism 4, which is an important component of the present invention, will be described with reference to FIG. In FIG. 2, reference numeral 10 denotes a correlation unit including a number of squaring elements, addition elements, and a minimum value selector therein, each of which serves to output an optimal movement amount dVij of one small area on the first image. Have. Each of these correlation units 10 summarizes the sum of squares of the difference of each pixel value when a pixel in the small area of the first image is moved on the second image within the slide range for each possible movement amount. The minimum value selector compares the average square errors of the pixel values obtained for all the movement amounts within the slide range, and outputs the movement amount that gives the smallest square error as dVij. As described above, the obtained area movement vector is sent to the slide mechanism 3 again after resolving the contradiction with the surroundings by the contradiction alleviating mechanism 6, and specifies the next matching search area. The value obtained by adding the square error evaluated for each small area for the entire area gives the amount of error in matching of the entire image. When this value falls below a desired set value, the above-described sequential process ends. Is obtained as an output.

Embodiment 2 FIG. Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 3 is a block diagram showing one embodiment of the invention described in claim 2, and the same parts as those in FIG. 1 are denoted by the same reference numerals and description thereof will be omitted. In the figure, 7
Is an internal division interpolation mechanism that internally divides the peripheral area movement vector output from the contradiction mitigation mechanism 6 according to the distance to the pixel and generates a pixel movement vector.

In the first embodiment, the contradiction mitigation mechanism 6
The output obtained is an area movement vector corresponding to the number of small areas. However, in order to actually perform the non-linear conversion of the image smoothly, it is necessary to provide a smooth change amount for each pixel. Therefore, in the second embodiment, the pixel movement vector is obtained by internally dividing the peripheral area movement vector according to the distance to the pixel. As such an internal division method, for example, there is a method of internally dividing an area movement vector of four small areas in which a target pixel is closest in accordance with an image distance to the pixel. FIG. 4C is the same as FIG.
The non-linear image conversion device with the internal interpolation mechanism 7 is used so that the MRI image of the brain shown in (a) matches the artificially created triangular model image shown in FIG. 4 (b). This shows the result of mapping, and a rough feature on the image is captured, and a mapping maintaining the phase relationship is realized.

Embodiment 3 FIG. Next, a third embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a block diagram showing one embodiment of the invention described in claim 3, in which 8 is the first embodiment.
A marker providing mechanism for providing a marker to a finite number of points corresponding to each other between the image and the second image; 9 is a marker movement vector for matching the marker provided on the first image with the corresponding marker on the second image Is generated, and is output to the contradiction mitigation mechanism 6. The other parts are the same as the parts denoted by the same reference numerals in FIG.

In the first and second embodiments, the mapping is realized without explicitly teaching the corresponding points in advance. However, when applied to medical images, the similarity of the images evaluated for each small area does not always satisfy the anatomical requirements with priority. For example, in the standard brain of Talairach, an AC-PC line connecting the anterior commissure and the posterior commissure is used as the coordinate origin, but it is very difficult to automatically recognize the posterior commissure without having anatomical knowledge. Accompanied by Also, from the cerebral physiological point of view, it is necessary to precisely align major ruptures such as the Sylvian fissure and the central groove, which vary greatly between individuals. Therefore, the third embodiment
In the mapping with a marker shown in Fig. 7, the first image displayed on the CRT
A marker is provided to the corresponding point of the image and the second image by the marker providing mechanism 8. Specifically, the marker point is provided by a mouse click or the like. Next, a marker movement vector for the marker point is obtained from the distance between the points on the image by the marker movement vector generation mechanism 9, and the movement vector DVij of the small area including the marker point is fixed to this value during the sequential process. To make a forced correspondence.

FIG. 6 is an explanatory diagram showing an example of such an image conversion with markers, in which markers indicated by white dots are added to ten points of the cerebellum, occipital region and brain stem. In this case, FIG.
The image shown in (a) is converted to be superimposed on the image shown in (b), and the movement vector is shown in (d) and the mapping result is shown in (e). In the vicinity of the occipital region and the cerebellum where the number of marker points is large, a large movement vector is generated to accurately match the contour of each tissue, as compared with the movement vector distribution when no marker is used as shown in FIG. Even when the aspect ratio is numerically evaluated, as shown in Table 1 below, the marker in the process in which the brain image of FIG. 6A approaches the brain image of FIG. The effectiveness of is understood.

[0028]

[Table 1]

Embodiment 4 FIG. Next, a fourth embodiment of the present invention will be described with reference to the drawings. FIG. 7 is a flowchart showing a position embodiment according to the fourth aspect of the present invention. First, step S
At T1, a large number of sample images such as brain MRI images are collected. Next, a provisional standard image for the collected sample image is set in step ST2. The process of setting the temporary standard image is performed, for example, by selecting one of a large number of collected sample images and setting it as a temporary standard image. Next, in step ST3, a pixel movement vector is calculated for each sample image using the temporary reference image. This pixel movement vector divides each of the sample images into small areas, slides the small areas on the temporary standard image, and maximizes the similarity between the image of the small area and the temporary standard image. The amount of movement is calculated and calculated based on the amount of movement. Since the details have already been described in the first to third embodiments, the description is omitted here. Next, in step ST4, an inter-sample average vector that is the average value of the pixel movement vectors between the sample images is obtained. Then, a true standard image is calculated in step ST5 using the inter-sample average vector and the above-mentioned temporary standard image. Specifically, a moving process is performed on each pixel of the set provisional standard image using the inter-sample average vector calculated in step ST4. In this way, a standard image can be constructed from many sample images collected.

[0030]

As described above, according to the first aspect of the present invention, since matching for each small area and elimination of inconsistency from a global perspective are alternately performed, a relatively small number of sequential operations are performed. It is possible to effectively accumulate microscopic information by the number of times and lead from the part to the whole self-organization, and to match the locally distorted image that was impossible with the conventional linear transformation to the standard image Can be efficiently generated, standardization of medical images such as MRI and PET images is facilitated, and physiological functions can be clarified and pathological diagnosis can be accurately performed.

According to the second aspect of the invention, the obtained peripheral area movement vector is internally divided according to the distance to the pixel to generate a pixel movement vector. There is an effect that the amount of change for each pixel can be made smooth.

According to the third aspect of the present invention, a marker is added to a corresponding portion between the first image and the second image, and a marker on the first image matches a corresponding marker on the second image. Since it was configured to generate a marker movement vector,
The non-linear image conversion including the forcible association has an effect of obtaining a more accurate association with the standard image.

According to the fourth aspect of the present invention, an inter-sample average vector of pixel movement vectors obtained by microscopic matching between a small area of each sample image and a provisional standard image is determined. Since the configuration is such that a true standard image is calculated from the average vector and the provisional standard image, it is possible to easily construct a standard image from many collected sample images, and to convert the standard image to an image for which the standard image has not yet been determined. On the other hand, a group average standard image can be easily defined, and the standard image can be used as an image described in a textbook or the like.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a configuration diagram showing a specific configuration of a movement amount calculation mechanism of the embodiment.

FIG. 3 is a block diagram showing a second embodiment of the present invention.

[4] di nonlinear image transformation examples in the above embodiment
5 is a photograph showing a halftone image displayed on a spray .

FIG. 5 is a block diagram showing a third embodiment of the present invention.

FIG. 6 is a photograph showing a halftone image displayed on a display of a non-linear image conversion example with a marker in the embodiment.
It is true and explanatory drawing.

FIG. 7 is a flowchart showing a fourth embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 1st image layer 2 2nd image layer 3 Slide mechanism 4 Movement amount calculation mechanism 5 Accumulation calculation mechanism 6 Conflict reduction mechanism 7 Internal division interpolation mechanism 8 Marker addition mechanism 9 Marker movement vector generation mechanism

──────────────────────────────────────────────────の Continuing on the front page (73) Patent holder 592214818 Tomomitsu Momose 2-5-5 Shimizu, Suginami-ku, Tokyo (73) Patent holder 000006013 Mitsubishi Electric Corporation 2-3-2 Marunouchi, Chiyoda-ku, Tokyo (72 Inventor Yukio Kosugi 1-2-16 Higashitamagawa, Setagaya-ku, Tokyo (72) Inventor Mikiya Sase 1-3-7 Katsuradai, Midori-ku, Yokohama-shi, Kanagawa Pref. 3-7-40 Ichikobuchi, Japan (72) Inventor Junichi Nishikawa 1-6-1, Izumotocho, Komae-shi, Tokyo (72) Inventor Toshimitsu Momose 6-13-12-402, Honkomagome, Bunkyo-ku, Tokyo (72) Inventor Kiyoshi Yoda 8-1-1, Tsukaguchi-Honmachi, Amagasaki City Mitsubishi Electric Corporation Central Research Laboratory (56) References JP-A-4-307682 (JP, A) JP-A-4-76788 (JP, A) JP 63-1829 61 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G06T 1/00 G06T 3/00 G06T 7/00

Claims (4)

(57) [Claims] A first image layer that holds a first image to be transformed divided into small areas and a second image layer that holds a second image that is a reference of the first image; A slide mechanism that slides a small area of an image on the second image, and movement of the small area to a position where the image of the small area that slides on the second image has the highest similarity to the second image A moving amount calculating mechanism for calculating an amount, a cumulative calculating mechanism for calculating an area moving vector by cumulatively adding the obtained moving amounts, and mitigating a contradiction between the obtained area moving vectors of each of the small areas, A non-linear image conversion device comprising: an area movement vector that retains a phase relationship between images of the small area; and a contradiction mitigation mechanism that feeds back to the slide mechanism. 2. An internal division for generating a pixel movement vector by internally dividing an area movement vector around a pixel of interest in the area movement vector output from the contradiction mitigation mechanism in accordance with a distance to the pixel. The nonlinear image conversion device according to claim 1, further comprising an interpolation mechanism. 3. A marker providing mechanism for providing a marker to a finite number of mutually corresponding points in the first image and the second image,
Generating a marker movement vector for matching a marker on the first image with a corresponding marker on the second image;
2. The nonlinear image conversion device according to claim 1, further comprising a marker movement vector generation mechanism for outputting the marker movement vector to the contradiction mitigation mechanism. 4. A standard image calculation method for constructing a standard image from a plurality of collected sample images, wherein a provisional standard image for the collected sample image is set, and each of the sample images is divided into small areas. Then, the small area is slid on the temporary standard image, and a pixel movement vector is calculated based on a movement amount at which the similarity between the image of the small area and the temporary standard image is maximized. A standard characterized by obtaining an inter-sample average vector which is an average value of the pixel movement vector between the respective sample images, and calculating a true standard image using the inter-sample average vector and the temporary standard image. Image calculation method.