JPH07239935A - Nonlinear image converter and standard image calculation method - Google Patents

Abstract

PURPOSE:To obtain a nonlinear image converter in which moving vector of picture element is generated while keeping phase relation in a space and a standard image calculation method in which standard image is easily formed from collected sample image. CONSTITUTION:The converter is provided with a moving quantity calculation mechanism 4 obtaining a moving quantity at which a similarity between a referenced 2nd image and an image of a small area in a 1st image slid on the 2nd image with a slide mechanism 3 is maximized, an accumulation arithmetic mechanism 5 accumulating and adding the moving quantities and a contradiction relax mechanism 6 relaxing the contradiction between area moving vectors, and the method is used to calculate an inter-sample average vector of picture element moving vectors obtained through microscopic matching between an image of a small area of each sample image and a tentative standard image and to calculate a true standard image based on the inter-sample average vector and the tentative standard image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a non-linear mapping process between medical images represented by MRI images and PET images, mapping a distorted ground image captured by a satellite to a standard map, and a different marking method. The present invention relates to a non-linear image conversion device that performs a non-linear mapping that is widely required in the image processing field such as automatic conversion of the above map, and a standard image calculation method that forms a standard image for a plurality of collected sample images.

[0002]

2. Description of the Related Art Conventionally, the operation of superimposing tomographic images obtained from two brains having different shapes on each other has been limited to a linear conversion in which a constant magnification is applied to each of the vertical and horizontal directions. Even if such matching is possible, it was almost impossible to map a medical image having a different internal tissue distribution for each subject with a different magnification for each corresponding tissue. The image superposition required here is, so to speak, a non-linear transformation in which the image drawn on the rubber film is partially expanded and contracted to match the other image, but it is a smooth transformation that maintains the neighborhood relationship with the surroundings. It implicitly assumes conversion by a function. In addition, recently, Journal of Computer Assisted Tomography (Journal of Computer Ass
isted Tomography) Vol. 15, N
o. 4 (1991) in Friston (Karl J.F.
Riston) et al. "Plastic Transformation of PEE
Tea Images (Plastic Transfer)
"Ration of PET Images)", a one-dimensional gray-scale distribution function obtained by scanning a tomographic image of the brain is subjected to integral transformation, and the displacement distribution has a smooth relationship. Inverse conversion is performed and two P
Attempts have also been made to align the ET images.

[0003]

Since the conventional non-linear image conversion is configured as described above, it is intended to correct the positional deviation between two-dimensional images by one-dimensional conversion, so that a medical image is obtained. Each individual image corresponds while maintaining the microscopic neighborhood relationship, as in the case of automatically correcting the image distortion due to the individual difference of the tissue distribution in the image and the defect of the imaging system seen in the satellite image. When it is necessary to realize a smooth nonlinear transformation that moves to a position, there is a problem that a satisfactory result cannot always be obtained.

The inventions set forth in claims 1 to 3 have been made to solve the above-mentioned problems, and in a two-dimensional image, in a two-dimensional space, in a three-dimensional image. Aims to obtain a non-linear image conversion device capable of generating a pixel movement vector while maintaining a phase relationship in a three-dimensional space.

It is another object of the present invention to provide a standard image calculation method capable of easily forming standard images for a plurality of collected sample images.

[0006]

According to a first aspect of the present invention, there is provided a non-linear image conversion apparatus, a slide mechanism for sliding a small area of a first image desired to be deformed on a second image serving as a reference, and a sliding mechanism. A movement amount calculation mechanism that obtains a movement amount that maximizes the similarity between the image of the small area and the second image, a cumulative calculation mechanism that cumulatively adds the movement amounts to calculate an area movement vector, and each area movement vector It is equipped with a contradiction mitigation mechanism that alleviates the contradiction between the two.

According to the second aspect of the present invention, the non-linear image conversion device internally divides the obtained peripheral area movement vector according to the distance to the pixel to generate a pixel movement vector. It has a mechanism.

According to a third aspect of the present invention, there is provided a non-linear image conversion device, wherein a marker applying mechanism for applying a marker to a finite number of points corresponding to each other in the first image and the second image, and the first image. A marker movement vector generation mechanism for generating a marker movement vector for matching the above marker with the corresponding marker on the second image is provided.

Further, in the standard image calculation method according to the invention described in claim 4, each of the collected sample images is divided into small areas, and the sample images are slid on the set temporary standard image and both of them are slid. The true standard image is calculated from the inter-sample average vector between the sample images of the pixel movement vector calculated based on the moving amount that maximizes the similarity and the temporary standard image.

[0010]

According to the non-linear image conversion device of the invention described in claim 1, by alternately performing microscopic matching for each small area and elimination of contradiction between the area movement vectors generated by the matching, Effectively accumulate microscopic information, heuristically grow matching search candidate areas, lead to partial self-organization, and perform direct nonlinear transformation of images in 2D or 3D. Perform effectively.

Further, the internal division interpolating mechanism according to the second aspect of the present invention internally divides the obtained peripheral area movement vector according to the distance to the pixel to generate a pixel movement vector. The amount of change for each pixel is smooth.

Further, the marker movement vector generating mechanism in the third aspect of the invention is a marker movement for matching the marker on the first image given by the marker giving mechanism with the corresponding marker on the second image. By generating a vector, it is possible to perform non-linear image conversion including forced association.

In the standard image calculation method according to the present invention, the sample image is divided into small areas, a pixel movement vector is obtained by microscopic matching with a temporary standard image, and the pixel movement is calculated. By calculating the true standard image from the inter-sample average vector between each sample image of the vector and the provisional standard image, it is easy to construct the standard image from many collected sample images.

[0014]

【Example】

Example 1. Embodiment 1 of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of the invention described in claim 1. In the figure, 1 is a first image layer that holds a first image to be transformed by dividing it into small areas, and 2 is a second image layer that holds a second image that serves as a reference for this first image. Is. Reference numeral 3 is a slide mechanism that slides the small area of the first image on the second image, and 4 is a movement that obtains the movement amount of the small area where the similarity between the image of the slide small area and the second image is maximum. It is a quantity calculation mechanism. Reference numeral 5 is a cumulative calculation mechanism for accumulating the movement amounts thus obtained to obtain the area movement vector, and 6 is for reducing the contradiction between the obtained area movement vectors of the small areas and for the image of the small area. This is a contradiction reducing mechanism that generates an area movement vector that holds the phase relationship between them and feeds it back to the slide mechanism 3.

Next, the operation will be described. First, the small area Dij obtained by dividing the first image held in the first image layer 1 is moved by the slide mechanism 3 to a predetermined range centered on the tip of the area movement vector DVij whose initial value is zero. Within the (for example, radius R) range, the second image layer 2
Slide on the second image held by. The movement amount calculation mechanism 4 obtains the movement amount dVij when the small area Dij is slid onto the most similar second image within the range. Next, a new movement vector DVij (t + 1) is obtained by cumulatively adding the movement amount dVij according to the following formula by the accumulative calculation mechanism 5.

DVij (t + 1) = DVij (t) + dVij ‥‥‥‥‥‥‥‥‥‥ (1)

The above operation is performed for all the small areas on the first image. In each small area, a movement vector is generated independently, but in order to maintain the phase relationship of the image as much as possible, the contradiction mitigation mechanism 6 performs the operation of the following equation in order to prevent contradictions between the movement vectors in the neighborhood relationship. An area movement vector DVij that holds the phase relationship is obtained.

[0018]

[Equation 1]

However, in the area ε1, the eight points on the smallest square surrounding the grid point (i, j) on the first image are defined as the area ε.
2 further represents 16 points on the outer square. Further, Sn is a normalization coefficient represented by Sn = c + 8 + 16b, and coefficients b and c are parameters indicating how important the phase relationship is.

The area movement vector from which the contradiction is removed is fed back to the slide mechanism 3 to give the center point of the slide range when the small area is slid on the second image in the next sequential process.

The present invention is characterized in that an area movement vector that maximizes the degree of similarity found by sliding within a certain range is calculated for all small areas, and after the contradictions are resolved, the slide range is calculated according to the area movement vector. That is, by repeating the operation of changing the matching search range a plurality of times, the mapping relationship with good matching is sequentially found.

Next, the specific construction of the movement amount calculation mechanism 4, which is an important constituent element of the present invention, will be described with reference to FIG. In FIG. 2, reference numeral 10 denotes a correlation unit internally provided with a large number of square elements, addition elements, and a minimum value selector, each of which has a role of outputting the optimum movement amount dVij of one small area on the first image. To have. Each of these correlation units 10 summarizes the sum of squares of the difference between the pixel values when the pixels in the small area of the first image are moved on the second image within the slide range for each possible movement amount. Further, the minimum value selector compares the mean square errors of the pixel values obtained for all the movement amounts within the slide range, and outputs the movement amount giving the smallest square error as dVij. As described above, the obtained area movement vector is sent to the slide mechanism 3 again after the contradiction mitigation mechanism 6 eliminates the contradiction with the surroundings, and specifies the matching search area for the next round. The value obtained by adding the squared error evaluated for each small area for all the areas gives the amount of matching error of the entire image, but when this value falls below the desired set value, the above sequential process ends, and at that time The area movement vector at is obtained as output.

Example 2. Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 3 is a block diagram showing an embodiment of the invention described in claim 2. The same parts as those in FIG. 1 are designated by the same reference numerals and the description thereof will be omitted. In the figure, 7
Is an internal division interpolation mechanism that internally divides the peripheral area movement vector output from the contradiction reduction mechanism 6 according to the distance to the pixel and generates a pixel movement vector.

In the first embodiment described above, the inconsistency reducing mechanism 6
The output obtained is an area movement vector corresponding to the number of small areas, but in order to actually perform the nonlinear conversion of the image smoothly, it is necessary to give a smooth change amount to each pixel. Therefore, in the second embodiment, the pixel movement vector is obtained by internally dividing the peripheral area movement vector according to the distance to the pixel. As such an internal division method, for example, there is a method of internally dividing the area movement vectors of the four small areas that are the closest to the target pixel according to the distance on the image to the pixel. FIG. 4C is the same as FIG.
A non-linear image conversion device with the internal division interpolation mechanism 7 added is used so that the MRI image of the brain shown in (a) matches the artificially created triangular model image shown in FIG. 4 (b). It shows the result of mapping, and the mapping is realized by capturing the rough features on the image and maintaining the phase relationship.

Example 3. Next, a third embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a block diagram showing an embodiment of the invention described in claim 3, in which 8 is a first
A marker applying mechanism for applying a marker to a finite number of points corresponding to each other in the image and the second image, and 9 is a marker movement vector for matching the marker provided on the first image with the corresponding marker on the second image. Is generated and is output to the contradiction mitigation mechanism 6. The other parts are the same as the parts denoted by the same reference numerals in FIG.

In the case of the above-mentioned first and second embodiments, the mapping is realized without explicitly teaching the corresponding points in advance. However, when applied to medical images, the similarity of images evaluated for each small area does not always meet the anatomical requirements preferentially. For example, in the standard brain of Talairach, the AC-PC line connecting the anterior commissure and the posterior commissure is used as the coordinate origin, but it is very difficult to automatically recognize the posterior commissure without anatomical knowledge. Accompanied by. From the viewpoint of cerebral physiology, it is necessary to accurately align the major fractures such as the Sylvian fissure and central groove, which vary greatly among individuals. Therefore, the third embodiment
In the map with the marker shown in, the first image displayed on the CRT is displayed.
A marker is added to the corresponding points of the image and the second image by the marker adding mechanism 8. Specifically, this marker point is given by mouse click or the like. Next, the marker movement vector generation mechanism 9 obtains a marker movement vector for the marker point from the distance between the points on the image, and the movement vector DVij of the small area including this marker point is fixed to this value during the sequential process. The compulsory correspondence of doing is done.

FIG. 6 is an explanatory diagram showing an example of such image conversion with markers, in which markers shown by white dots are added to 10 points of the cerebellum, occipital region, and brain stem. In this case
The image shown in (a) is converted by superimposing it on the image shown in (b), the movement vector is shown in (d), and the mapping result is shown in (e). In the vicinity of the occipital region and the cerebellum where the number of marker points is large, generation of a large movement vector for accurately aligning the contours of each tissue is observed, as compared with the movement vector distribution when no marker is used as shown in FIG. Even when the aspect ratio of A is numerically evaluated, as shown in Table 1 below, the marker in the process in which the brain image of FIG. 6A approaches the brain image of FIG. The effectiveness of is understood.

[0028]

[Table 1]

Example 4. Next, a fourth embodiment of the present invention will be described with reference to the drawings. FIG. 7 is a flow chart showing a position embodiment of the invention described in claim 4. First step S
At T1, a large number of sample images such as MRI images of the brain are collected. Next, a temporary standard image for the acquired sample image is set in step ST2. The temporary standard image setting process is performed by, for example, selecting one of a large number of collected sample images and setting it as a temporary standard image. Next, in step ST3, a pixel movement vector is calculated for each sample image using the temporary reference image. This pixel movement vector divides each sample image into small areas and slides the small areas on a temporary standard image to maximize the similarity between the image of the small area and the temporary standard image. The amount of movement is calculated and calculated based on the amount of movement, which has already been described in detail in Embodiments 1 to 3, and therefore the description thereof is omitted here. Next, in step ST4, an inter-sample average vector, which is an average value of pixel movement vectors between each sample image, is obtained. Then, a true standard image is calculated in step ST5 using this inter-sample average vector and the above-mentioned temporary standard image. Specifically, the moving process using the inter-sample average vector calculated in step ST4 is executed for each pixel of the set provisional standard image. In this way, a standard image can be constructed from many collected sample images.

[0030]

As described above, according to the invention described in claim 1, since the matching for each small area and the resolution of the contradiction seen from a large scale are alternately performed, a relatively small number of successive steps are performed. It becomes possible to effectively accumulate microscopic information by the number of times and lead to self-organization of the whole from the part, and to adjust the locally distorted image, which was impossible with the conventional linear transformation, to the standard image. The moving vector can be efficiently generated, standardization of medical images such as MRI and PET images is facilitated, and physiological functions can be elucidated and pathological conditions can be accurately diagnosed.

According to the second aspect of the invention, the obtained peripheral area movement vector is internally divided according to the distance to the pixel, and the pixel movement vector is generated. There is an effect that the amount of change for each pixel can be made smooth.

According to the third aspect of the invention, a marker is added to the corresponding portion of the first image and the second image so that the marker on the first image matches the corresponding marker on the second image. Since it is configured to generate a marker movement vector,
The nonlinear image conversion including the compulsory association has the effect of obtaining a more accurate association with the standard image.

According to the fourth aspect of the invention, the inter-sample average vector of the pixel movement vectors obtained by microscopic matching of the small area of each sample image and the provisional standard image is obtained, and the inter-sample average is calculated. Since it is configured to calculate the true standard image from the average vector and the provisional standard image, it becomes possible to easily construct the standard image from many collected sample images, and the standard image can be converted into an image for which the standard image has not yet been determined. On the other hand, there is an effect that a set average standard image can be easily defined and can be used as an image described in a textbook.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a configuration diagram showing a specific configuration of a movement amount calculation mechanism of the above embodiment.

FIG. 3 is a block diagram showing a second embodiment of the present invention.

FIG. 4 is an explanatory diagram showing an example of nonlinear image conversion in the above embodiment.

FIG. 5 is a block diagram showing a third embodiment of the present invention.

FIG. 6 is an explanatory diagram showing an example of nonlinear image conversion with a marker in the above-described embodiment.

FIG. 7 is a flow chart showing a fourth embodiment of the present invention.

[Explanation of symbols]

 1 1st image layer 2 2nd image layer 3 Slide mechanism 4 Movement amount calculation mechanism 5 Cumulative calculation mechanism 6 Contradiction mitigation mechanism 7 Internal division interpolation mechanism 8 Marker addition mechanism 9 Marker movement vector generation mechanism

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] April 21, 1995

[Procedure Amendment 2]

[Document name to be amended] Statement

[Name of item to be corrected] Brief description of the drawing

[Correction method] Change

[Correction content]

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a configuration diagram showing a specific configuration of a movement amount calculation mechanism of the above embodiment.

FIG. 3 is a block diagram showing a second embodiment of the present invention.

[4] di nonlinear image transformation examples in the above embodiment
6 is a photograph showing a halftone image displayed on a spray .

FIG. 5 is a block diagram showing a third embodiment of the present invention.

FIG. 6 is a photograph showing a halftone image displayed on a display of a non-linear image conversion example with a marker in the above embodiment.
True and illustration.

FIG. 7 is a flow chart showing a fourth embodiment of the present invention.

[Explanation of Codes] 1 First image layer 2 Second image layer 3 Slide mechanism 4 Movement amount calculation mechanism 5 Cumulative calculation mechanism 6 Contradiction mitigation mechanism 7 Internal division interpolation mechanism 8 Marker addition mechanism 9 Marker movement vector generation mechanism

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication location 9061-5L G06F 15/70 455 A (71) Applicant 592214807 Nishikawa Junichi Izumihoncho, Komae-shi, Tokyo 1- 6-1 (71) Applicant 592214818 Momose Toshimitsu 6-13-12-402 Honkomagome, Bunkyo-ku, Tokyo (71) Applicant 000006013 2-3-3 Marunouchi, Chiyoda-ku, Tokyo Mitsubishi Electric Corporation (72) Inventor Yusio Kosugi 1-2-16 Higashitamagawa, Setagaya-ku, Tokyo (72) Inventor Mikiya Sase 1-3-7 Katsuradai, Midori-ku, Yokohama-shi, Kanagawa Katsuradai Courthouse C101 (72) Inventor Hiroshi Kuwaya 3-Furubuchi, Sagamihara-shi, Kanagawa 7-40 (72) Inventor Junichi Nishikawa 1-6-1 Izumihoncho, Komae City, Tokyo (72) Inventor Toshimitsu Momose 6-13-12-402 (72) Inventor Kiyoshi Yoda 8-1, 1-1 Tsukaguchihonmachi, Amagasaki-shi Central Research Laboratory, Mitsubishi Electric Corporation

Claims (4)

[Claims] 1. A first image layer for holding a first image to be transformed by dividing it into small areas, a second image layer for holding a second image serving as a reference for the first image, and the first image layer. A slide mechanism for sliding a small area of the image on the second image, and a movement of the small area to a position where the image of the small area sliding on the second image shows the maximum similarity to the second image. A movement amount calculation mechanism for obtaining an amount, a cumulative calculation mechanism for cumulatively adding the obtained movement amounts to calculate an area movement vector, and alleviating a contradiction between the obtained area movement vectors of the respective small areas, A non-linear image conversion device comprising: an area movement vector that holds a phase relationship between images of the small area, and a contradiction reduction mechanism that feeds back the area movement vector to the slide mechanism. 2. An internal division for generating a pixel movement vector by internally dividing an area movement vector around a pixel of interest in the area movement vector output from the contradiction mitigation mechanism according to a distance to the pixel. The non-linear image conversion device according to claim 1, further comprising an interpolation mechanism. 3. A marker applying mechanism for applying a marker to a finite number of mutually corresponding points of the first image and the second image,
Generating a marker movement vector for matching a marker on the first image with a corresponding marker on the second image,
The non-linear image conversion device according to claim 1, further comprising a marker movement vector generation mechanism for outputting to the contradiction reduction mechanism. 4. A standard image calculation method for constructing a standard image from a plurality of collected sample images, setting a temporary standard image for the collected sample images, and dividing each of the sample images into small areas. The small area is slid on the temporary standard image, and the pixel movement vector is calculated based on the amount of movement that maximizes the similarity between the small area image and the temporary standard image. A standard characterized by calculating an inter-sample average vector which is an average value of the pixel movement vector among the sample images, and calculating a true standard image using the inter-sample average vector and the temporary standard image. Image calculation method.