JP2006227827A - Image matching method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image matching method for matching corresponding points in two images. <P>SOLUTION: First image matching that determines a first mapping relation from a target image to a reference image using first image correlation potential energy, and second image matching that determines a third mapping relation from the target image to the reference image and a fourth mapping relation from the reference image to the target image using second image correlation potential energy are performed. Points are calculated by mapping using the second mapping relation the points on the target image that are mapped using the first mapping relation. Based on the positional relationship of the points, a first reliability probability is calculated. Points are calculated by mapping using the fourth mapping relation the points on the target image that are mapped using the third mapping relation. Based on the positional relationship of the points, a second reliability probability is calculated. In accordance with the first and second reliability probabilities, the mappings based on the first to fourth mapping relations are combined. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image matching method and apparatus for detecting corresponding points from two images and performing matching.

  In many technical fields such as motion detection, stereo matching, image morphing, image recognition, and moving image coding, image matching technology for obtaining a correspondence relationship from one image to another is a basic problem. According to Non-Patent Document 1, image matching techniques can be broadly classified into four. That is, there are an optical flow method, a block-based method, a gradient method, and a Bayesian method. The optical flow method derives an optical flow equation that “a change in luminance is constant”, and obtains a flow using the optical flow equation as a constraint. The block-based method is a method for obtaining motion by template matching for each block. The gradient method is a method for performing matching in a direction in which the luminance gradient of an image decreases. The Bayesian method is a technique for obtaining matching that is probabilistically plausible.

Patent Document 1 discloses an image matching method using a multi-resolution filter as a technique that does not belong to the above classification. This technique is a highly robust matching technique capable of matching a large motion to a small motion by generating a plurality of multi-resolution image pyramids using a plurality of multi-resolution filters and performing a matching process on the image pyramids in order from the top.
Japanese Patent No. 2927350 A. Murat Tekalp, “Digital Video Processing”, Prentice Hall, 1995

  Many conventional image matching techniques are constructed on the assumption that the brightness of the target object does not change between images. The optical flow method is a representative example, but the block matching algorithm and the Bayes method are the same. For this reason, in the prior art, there is a high possibility that erroneous matching occurs when a change in luminance occurs in the target object.

  An object of the present invention is to provide an image matching method that enables image matching with high accuracy even when a luminance change occurs.

  One aspect of the present invention is an image matching method for obtaining a mapping relationship between a target image and a reference image, wherein a first image correlation potential based on a positional relationship between corresponding points of the target image and the reference image and image information. Using a force due to an energy gradient, an elastic force between each point on the target image or the reference image and its adjacent point, and a frictional force acting on each point on the target image or the reference image, A first image matching step of obtaining a first mapping based on a first mapping relationship from the target image to the reference image and obtaining a second mapping based on a second mapping relationship from the reference image to the target image; , A positional relationship between corresponding points of the target image and the reference image and a force due to a gradient of a second image correlation potential energy based on the image information, and the target image or the reference image And a frictional force acting on each point on the target image or the reference image, and a third mapping relationship from the target image to the reference image using a third mapping relationship from the target image to the reference image. A second image matching step for obtaining a fourth mapping based on a fourth mapping relationship from the reference image to the target image, and a point on the target image mapped by the first mapping relationship. A point obtained by mapping a point by the second mapping relationship is obtained, a first reliability probability is calculated by the positional relationship of the points, and a point obtained by mapping the point on the target image by the third mapping relationship is A reliability calculation step of obtaining points mapped by the fourth mapping relationship and calculating a second reliability probability based on a positional relationship between the points; and the first and second reliability in accordance with the first and second reliability probabilities. Steps for composing the second, third, and fourth maps To provide an image matching method characterized by having a flop.

  According to the present invention, matching can be performed with high accuracy even if a luminance change occurs in the target object. In addition, it is possible to cope with a change in luminance, and it is possible to cope with a scalable movement from a large movement to a small movement.

[First Embodiment]
This embodiment will be described with reference to the block diagram of FIG. In the present embodiment, a target image and a reference image are input, and two image matching units 11 and 12 configured by, for example, a processor are provided. The image matching unit 11 performs image matching 1 assuming that there is no luminance change, and the image matching unit 12 performs image matching 2 that is resistant to luminance changes. The outputs of these image matching units 11 and 12 are input to a reliability calculation unit 13 constituted by, for example, an arithmetic device, and the matching reliability is calculated. When the calculation result of the reliability calculation unit 13 is input to the joining unit 14 configured by, for example, a processor, the joining unit 14 synthesizes the mapping of the image matching units 11 and 12 according to the calculation result of the reliability calculation unit 13. To do.

  In the present embodiment, first, basic image matching 1 will be described.

A continuous image model is expressed as shown in Equation 1.

  This is a real-based continuous image model. Here, since a digital image is considered, the following sampled image model obtained by sampling the above model is used.

This sampling image model is expressed by Equation 2.

At this time, the image matching problem can be formulated as a problem of searching for a motion vector d: R ³ → R ² satisfying the following condition, that is, Expression 3.

However, in this formulation, it is assumed that the image values of the same object do not change with time. Also, since the motion vector d is a real number, it should be noted that the right side uses the notation of a continuous image model (Formula 1). Here, since image matching between the target image, the reference image, and the two images is considered, an equivalent equation 4 is considered.

Also, the motion vector is simplified, and d: R ² → R ² . Since only one motion vector d needs to be determined for the point x, the motion vector d is a unique mapping. Since x + d (x) can be regarded as a mapping to x, it is defined as g (x) = x + d (x). Where g: R ² → R ^{2 is} a unique map. The above image matching problem results in a problem of searching for a mapping g satisfying Equation 5.

A point determined by the mapping g is defined as y = g (x). is a y∈R ^2. Since x = Vn, x uniquely corresponds to a point n on the lattice space. Since map g is a unique map, y also uniquely corresponds to x. Therefore, y uniquely corresponds to n. This is shown in FIG. That is, the space to be handled here is a deformed lattice space corresponding to one-to-one by the point n on the lattice space.

Since y = g (x) = g (Vn) as described above, this is redefined as y _n = g (x) in order to make it easy to understand that there is a one-to-one correspondence with _n . Then Equation 5 of the image matching problem reduces to a problem to find a y _n satisfying Equation 6.

In order to solve Equation 6 of the image matching problem, dynamics is introduced for the point y _n here. In other words, the image matching problem is reduced to the problem of solving the dynamic system about the point y _n . The point y _n moves to a state satisfying Equation 6 while considering the relationship with surrounding points, and converges to an equilibrium state. Assume that image matching is completed by the equilibrium state. This state is expressed as shown in FIG.

A new time axis τ∈R is introduced for the point y, and a function y _n (τ) is defined. Here, it is assumed that the initial value is the same as that of the square lattice x as shown in Equation 7.

Since a new time axis is introduced, the derivative with respect to time can be defined as shown in Equation 8.

Ordinary dynamics are described by the ordinary differential equation 9 as follows.

F∈R ² is the sum of forces. This is also called the equation of motion.

Next, consider the force applied to y _n (τ). First, consider the potential energy that drives the dynamics. This is a force for moving y _n (τ) to a state where Expression 6 is satisfied. When Formula 5 is transformed, Formula 10 is obtained.

Usually, it is difficult to search for a point that exactly satisfies Formula 10 because of noise components included in the image. Therefore, an energy function as shown in Equation 11 is considered.

A point where the energy function _Eu is minimized is searched. Using the principle of the steepest descent method, the local minimum can be reached by descending in the direction of the steepest descent of the energy function _Eu around y _n (τ). Therefore, the gradient in the direction of the steepest descent is defined as a force against y _n (τ). Since the energy function E _u can be considered as image correlation, this force is defined as a force F _u due to image correlation potential energy.

There are various methods for calculating the gradient in the direction of steepest descent. Here, the following method is adopted. As shown in FIG. 4, the gradient in the steepest descent direction is obtained directly by local optimization. Although the image model _Sc is a continuous image model, only the sampled image model _Sp can actually be used. Therefore, local optimization is also performed based on the sampled image model. As shown in FIG. 4, since it is desired to set the sampling point closest to y _n (τ) as the local space center y _c , the following equation 12 is obtained.

When the adjacent space is defined as in Expression 13, the local search space Ω can be defined as in Expression 14.

When local optimization is performed to obtain a vector in the direction, normalized, and multiplied by the magnitude of the gradient, Equation 15 is obtained. This equation shows the force (square error energy) due to the image correlation potential energy.

The force (absolute value difference error energy) by the image correlation potential energy of Expression 17 in which the energy function is defined as Expression 16 can be used for ease of handling in mounting.

Next, consider the ability to describe the relationship with surrounding points. Assume that the matching target image is a two-dimensional projection of a three-dimensional space. If the object in the three-dimensional space is a rigid body, the surface of the rigid body is observed as an object in the two-dimensional image. If an object in the three-dimensional space is observed in the target image and the reference image, there is a high probability that the phase of the object observed in each image will be maintained. As shown in FIG. 5, the positional relationship of the point x on the target image object should be maintained at the point y _n (τ) on the reference image object. This property can be simulated by connecting the points y _n (τ) with a spring. The relationship with the periphery is described by the spring force F _k . First, the lattice point space N _n around the target point is defined as follows. If there are four surrounding points, Equation 18 is obtained.

If the spring constant (elastic constant) is k, the restoring force of the spring applied to the point y _n (τ) is the spring force (elastic force) expressed by Equation 19. The elastic constant is a balance between image correlation energy and elastic energy. If the elastic constant is large, the elastic constant is difficult to be deformed and the result is stable. However, the adaptability to the image is deteriorated. If the elastic constant is small, the image can be easily deformed, so that the adaptability of the image is improved. However, the results are too flexible. So, at present, this parameter is given empirically. Since the behavior is not so sensitive to the value of this parameter, it is basically given a fixed value.

When the four surrounding points are connected, a four-point connection spring model as shown in Equation 20 is obtained.

Finally, consider the power to dissipate the stored energy. If the force applied to y _n (τ) is only F _u and F _k , the energy is conserved and the system is in a steady state where it vibrates. Therefore, the power to dissipate the stored energy is introduced. Frictional force can be used for this. If the speed can be approximated as constant, the frictional force can be described by Equation 21.

Summarizing the above forces, the equation of motion is as shown in Equation 22.
Equation of motion

ODE a force F _u by the image correlation potential energy is not solved analytically 22 can not be solved analytically. Therefore, it is difficult to take the limit of τ → ∞ of the system. Therefore, a sufficiently large time T for the system to converge is considered, and the convergence state of the system is estimated by calculating the t = (0, Τ) interval by numerical analysis.

If the initial value of the ordinary differential equation is determined, a solution can be uniquely obtained by numerical analysis. In general, it is called the initial value problem of ordinary differential equations. There are many numerical solutions to this problem, but famous ones include the Euler method, the Runge-Kutta method, the Birsch store method, the predictor / corrector method, and the hidden Runge-Kutta method. The Runge-Kutta method is the most famous and frequently used. However, since Equation 22 has dimensions corresponding to the size of the image, a complicated numerical solution method is difficult to adapt. Therefore, here we consider applying the Euler method, which is the simplest implementation. Since the Euler method is a numerical solution to the first-order ordinary differential equation, first, Equation 22 is converted to a first-order ordinary differential equation. At this time, variable conversion is performed as shown in Equation 23.

Substituting this variable conversion into Equation 22, Equation 24 is created.
Transformed equations of motion

The Euler scheme for the ODE 25 is expressed by Equation 26.

This advances the solution from t ⁽ⁿ⁾ to t ^{(n + 1)} ≡t ⁽ⁿ⁾ + h. Here, x ⁽ⁿ⁾ indicates n steps, and h is a step width. When the Euler scheme is applied to Formula 24, an update formula based on the Euler method of Formula 27 is obtained.

As described above, image matching is summarized as an algorithm as follows.
Image matching algorithm:

Next, the image matching 2 step will be described.
When one step of image matching is used for an object having a normal translational movement as shown in FIG. 6, a mapping as shown in FIG. 7 is obtained. It can be seen that a matching map is obtained for the object. On the other hand, when one step of image matching is used for an object that performs translational motion in which a luminance change as shown in FIG. 8 occurs, a distorted map as shown in FIG. 9 is generated. This is because the image matching problem is formulated as shown in Equation 6. Formula 6 is formulated under the assumption that the luminance does not change. In order to cope with a change in luminance, it is necessary to redefine the image matching problem as a generalized image matching problem as shown in Equation 28.

  This is based on the correlation function φ of the image, and is a more general expression than Equation 6 assuming that the luminance values of the images are equal. The correlation function φ is defined as the probability that the higher the image correlation is, the higher the correlation is.

The cross-correlation can be expressed by Equation 29 as a correlation function that is robust to the luminance change that can correspond to the luminance change.

Here, B is a calculation target space for calculating cross-correlation. In this case, a local block stretched around the points x and yn is considered. The cross-correlation is obtained by normalizing the covariance of s ₁ and s ₂ when the expected value is 0, and when it is 1, there is a maximum positive correlation. Since it is a dispersion-based correlation function, correct correlation can be calculated even if there is a luminance offset in the entire region.

Similarly, a qualitative ternary representation (QTR) is a statistic that is invariant to the luminance offset. QTR is a technique for comparing large and small patterns of difference values with adjacent pixels, and has a feature that is robust to noise, luminance changes, and the like. In particular, the luminance value shift is invariant. This is expressed by Equation 30 below.

Consider the image correlation potential energy based on equations 28 and 29. An energy function based on Expression 29 is expressed by Expression 31.

Here, B is a local block, and Equation 32 or the like can be used.

If a method similar to that in image step 1 is used, Formula 33 representing a force: cross-correlation model based on image correlation potential energy can be obtained.

  It should be noted that the cross-correlation is a probability density function having a maximum value of 1, so that it is not a minimization problem but a maximization problem.

Similarly, when QTR is formulated as image correlation potential energy, the following equation 34 is obtained.

Here, N is a normalization coefficient.
The cross-correlation model of Equation 33 is used as the force Fu due to the image correlation potential energy, and the other components having the same configuration as the image matching 1 step are set as the image matching 2 step. FIG. 8 shows a translational motion with a luminance change by this image matching two steps, and FIG. 10 shows the result of matching. It can be seen from FIG. 10 that matching is applied without distortion and that it is robust against changes in luminance. However, the image matching 2 step is robust to the luminance change, but the matching performance itself is not as good as the image matching 1 step. Therefore, a configuration in which both are hybridized is required.

  Next, the reliability calculation step will be described.

  I would like to optimally synthesize the mapping by one step of image matching and the mapping by two steps of image matching. However, the reliability using the difference value of luminance cannot be used to cope with the luminance change. Therefore, another degree of reliability is required.

  As shown in FIG. 11, a case is considered where the brightness of the reference image is changed by the light hitting the object of the target image. Consider bidirectional matching from the target image to the reference image and from the reference image to the target image. In the matching from the target image to the reference image, the point of the part where the light of the object hits will be matched toward the point where the surrounding brightness is the same, and the movement will occur in the part that should not have moved originally right. On the other hand, in the matching from the reference image to the target image, it is considered that the point where the light hits stays on the spot because no point of the same luminance exists anywhere in the target image. As described above, in the matching based on the difference value of the luminance, when the luminance change occurs, the bidirectional matching result does not always match. Therefore, the reliability can be defined using the degree of coincidence of the bidirectional matching results.

  From the definition of motion vector d (x, t; lΔt) and mapping to formulate the degree of coincidence of bidirectional mapping, the mapping from the target image to the reference image is g (x, t; lΔt), Assume that the mapping from the reference image to the target image is g (x, t + lΔt; −lΔt). As shown in FIG. 12, a point x on the target image is mapped to a point g (x, t; lΔt) on the reference image, and a point g (x, t; lΔt) on the reference image is a point on the target image. It is mapped to g (g (x, t; lΔt), t + lΔt; −lΔt). When the point x is mapped, it becomes a point g (g (x, t; lΔt), t + lΔt; -lΔt). Therefore, if the two-way mappings match, both points should match. That is, the degree of coincidence can be expressed by the vector g (g (x, t; lΔt), t + lΔt; −lΔt) −x.

Therefore, if the vector norm is used, the reliability using the norm can be defined by Equation 35.

Using the area of the enclosed triangle, the reliability can be defined as Equation 36.

Here, since there are one image matching step and two image matching steps, if the respective mappings are represented by mathematical expressions 37 and 38, the respective reliability can be defined as mathematical expressions 39 and 40.

Finally, the combining step will be described. In the combining step, the mapping in one image matching step and the mapping in two image matching steps are combined. In this case, the combination is performed using the reliability obtained by the reliability calculation unit 13. For example, the reliability p (x | g ₁ (; t; lΔt), g ₁ (; t + lΔt; -lΔt)) and p (x | g ₂ (; t; lΔt), g ₂ (; t + lΔt) Compared with; -lΔt)), a method of selecting a map with high reliability as the map of the point x can be considered.

In addition, we define two mappings for the point x, and the reliability of each mapping is p (x | g ₁ (; t; lΔt), g ₁ (; t + lΔt; -lΔt)) and p (x | g ₂ (; t; lΔt), g ₂ (; t + lΔt; −lΔt)) may be added.

  In the present embodiment, as described above, the target image and the reference image are bi-directionally using the first image correlation potential energy generated by the chromaticity space distance between the corresponding points or the difference in the luminance value between the corresponding points. The target object is configured by performing image matching 1 and performing image matching 2 bidirectionally on the target image and the reference image using the second image correlation potential energy generated by the cross-correlation of the luminance values of the images between corresponding points. Even if a luminance change occurs, matching can be performed with high accuracy.

[Second Embodiment] Hierarchical Structure This embodiment will be described with reference to the block diagram of FIG. In this embodiment, a target image and a reference image are input, and a mapping relationship between these images is obtained.

  In the first embodiment, an image matching technique is provided that enables high-precision matching even if the luminance change occurs in the target object. However, in the present embodiment, in order to be able to deal with a large movement to a small movement in a scalable manner. Introduce a pyramid hierarchy. This will be described in detail below.

First, the basic image matching step is expanded to a pyramid hierarchical structure. The pyramid hierarchical structure of the image is as shown in FIG. 14, and is defined here as the hierarchical structure of the image reduced by down-sampling the original image. Each layer is called a layer. The top layer is layer 0, and the original bottom layer is layer M. For simplicity, assume that the image size is 2 ^M x 2 ^M and the downsampling operator is 1/2 in length and width. The image size and layer number are linked. That is, layer 0 is 2 ⁰ × 2 ⁰ , and the original lowermost layer is layer M 2 ^M × 2 ^M. The hierarchical structure generation step generates these hierarchical structures from the original target image and the reference image. A hierarchical continuous image model is defined by Equation 41.

Further, a hierarchical sampling image model obtained by sampling it is defined by Equation 42.

The motion vector is also hierarchized to be d ^(m) (x ^(m) ), and the mapping g is defined as Equation 43.

If the method of the first embodiment is applied, the equation of motion is expressed by Equation 46.

Each force, that is, the force (square error energy) due to the image correlation potential energy is expressed by Equation 47.

The spring force is expressed by Equation 48.

The frictional force is expressed by Equation 49.

  As described above, it can be seen that the image model and the representation of each point are merely replaced with the hierarchical model.

By solving the equation of motion 44, a map g ^{(m) of the} layer m can be obtained. In order to obtain the mapping g ^(M) in the lowermost layer M, means for propagating the mapping obtained in the upper layer to the lower layer is necessary. For example, it is conceivable that the scale is doubled by applying geometric transformation to the mapping g ^(m) as shown in Equation 50.

  Here, a grid-like mapping value is acquired using an integer operator, but an intermediate value may be interpolated from the grid-like mapping by a bilinear method or the like.

  The hierarchical image matching algorithm will be described later.

The image matching 2 step is similarly expanded. In other words, the force: cross-correlation model by the image correlation potential energy is defined by Equation 51.

  As shown in the block diagram of FIG. 13, a hierarchical structure generation unit 21 to which a target image and a reference image are input is provided. The output of the hierarchical structure generation unit 21 is connected to the image matching units 22 and 23. Image matching units 22 and 23 perform image matching 1 and 2, respectively. The outputs of the image matching units 21 and 22 are input to the reliability calculation unit 24. When the output of the reliability calculation unit 24 is input to the joining unit 25, the joining unit 25 synthesizes the mapping of the image matching units 22 and 23 according to the calculation result of the reliability calculation unit 24. That is, in the present embodiment, reliability calculation and mapping combination are performed for each layer, and the result is fed back to the image matching 1 step and the image matching 2 step.

Consider the reliability calculation step of layer m. Here, since there are one image matching step and two image matching steps, the mappings are expressed by equations 52 and 53, respectively.

In this case, the respective degrees of reliability can be defined by Equations 54 and 55 in the forward direction.

In the case of the reverse direction, it can be defined by Equations 56 and 57 respectively.

In the coupling step in the coupling unit 25, the reliability p (x ^(m) | g ₁ ^(m) (; t; lΔt), g ₁ ^(m) (; t + lΔt; -lΔt)) and p (x ^{( m)} | g ₂ ^(m) (; t; lΔt), g ₂ ^(m) (; t + lΔt; -lΔt))), the forward joint map g _j ^(m) (; t; lΔt). The reliability p (x ^(m) | g ₁ ^(m) (; t + lΔt; -lΔt), g ₁ ^(m) (; t; lΔt)) and p (x ^(m) | g ₂ ^(m) (; t + lΔt; −Δt), g ₂ ^(m) (; t; lΔt)) is used to determine the backward coupled map g _j ^(m) (; t + lΔt; −lΔt) of layer m. For example, a method of selecting a reliable one can be used.

The hierarchical image matching algorithm is as follows.

In the case of forward matching with one step of image matching and two steps of image matching, the forward joint map g _j ^(m) (; t; lΔt) is obtained from the joint step, and in the case of backward matching, the backward direction is obtained. The combined map g _j ^(m) (; t + lΔt; −lΔt) is obtained from the combining step.

  As described above, in the present embodiment, it is possible to cope with a change in luminance, and it is possible to cope with a scalable movement from a large movement to a small movement.

1 is a block diagram of an image matching apparatus that executes an image matching method according to a first embodiment of the present invention. The figure which shows the connection of each space Diagram showing the dynamics for the image matching problem Assumption figure showing force by image correlation potential energy The figure which shows the state with which the phase of the object on a target image and the object of a reference image is maintained Diagram showing normal translational motion Diagram showing mapping in one step of image matching Diagram showing translational motion with brightness change Diagram showing mapping in one step of image matching Diagram showing mapping by two steps of image matching Diagram explaining bidirectional matching Diagram showing bidirectional mapping relationship The block diagram of the image matching apparatus which performs the image matching method by 2nd Embodiment Diagram showing the pyramid hierarchy of images

Explanation of symbols

DESCRIPTION OF SYMBOLS 11, 12 ... Image matching unit, 13 ... Reliability calculation unit, 14 ... Combined unit, 21 ... Hierarchical structure generation unit, 22, 23 ... Image matching unit, 24 ... Reliability calculation unit, 25 ... Combined unit

Claims (9)

In an image matching method for obtaining a mapping relationship between a target image and a reference image,
The positional relationship between corresponding points of the target image and the reference image, the force due to the gradient of the first image correlation potential energy based on the image information, and between each point on the target image or the reference image and its adjacent points A first mapping based on a first mapping relationship from the target image to the reference image is obtained using the elastic force of the target image and the frictional force acting on each point on the target image or the reference image, and the reference A first image matching step for obtaining a second mapping according to a second mapping relationship from an image to the target image;
The positional relationship between corresponding points of the target image and the reference image, the force due to the second image correlation potential energy gradient based on the image information, and between each point on the target image or the reference image and its adjacent points A third mapping based on a third mapping relationship from the target image to the reference image is obtained using the elastic force of the target image and the frictional force acting on each point on the target image or the reference image, and the reference A second image matching step for obtaining a fourth mapping according to a fourth mapping relationship from the image to the target image;
A point obtained by mapping a point on the target image according to the first mapping relationship to a point mapped by the second mapping relationship is obtained, and a first reliability probability is calculated based on the positional relationship between these points. A reliability calculation step of obtaining a point where a point mapped by the third mapping relationship is mapped by the fourth mapping relationship and calculating a second confidence probability by the positional relationship of those points;
Synthesizing the first, second, third and fourth mappings according to the first and second confidence probabilities.   The image matching method according to claim 1, wherein the second image correlation potential energy is generated by cross-correlation of luminance values of images between corresponding points.   The second image correlation potential energy is determined by obtaining a qualitative expression obtained by quantizing a difference in luminance value between adjacent points in the target image and the reference image, and is generated depending on a degree of coincidence of the qualitative expressions between corresponding points. The image matching method according to claim 1, wherein:   3. The image matching method according to claim 1, wherein the first image correlation potential energy is generated by a difference in image luminance value between corresponding points. 4.   The image matching method according to claim 1, wherein the first image correlation potential energy model is generated by a chromaticity space distance of an image between corresponding points.   The reliability calculation step includes a step of obtaining a point C in which a point B obtained by mapping the point A on the target image by the first mapping relationship is mapped by the second mapping relationship, and a distance between the points A and C. A step of calculating a reliability probability that the reliability probability is higher as the distance is closer, and obtaining a point F obtained by mapping a point E on the target image according to the third mapping relationship by the fourth mapping relationship; 6. The image matching method according to claim 1, further comprising a step of calculating a reliability probability that the higher the distance between the points E and F, the higher the reliability probability.   The reliability calculation step includes a step of obtaining a point C in which a point B obtained by mapping the point A on the target image by the first mapping relationship is mapped by the second mapping relationship, and the points A, B, and C A step of calculating a reliability probability that the smaller the area is, the higher the reliability probability is; and a point F obtained by mapping a point E on the target image according to the third mapping relationship according to the fourth mapping relationship. 6. The image matching according to claim 1, further comprising: calculating a reliability probability that a reliability probability is higher as an area of the points D, E, and F is smaller. Method.   The image matching method according to claim 1, wherein each of the target image and the reference image has a pyramid hierarchical structure. In an image matching device for obtaining a mapping relationship between a target image and a reference image,
The positional relationship between corresponding points of the target image and the reference image, the force due to the gradient of the first image correlation potential energy based on the image information, and between each point on the target image or the reference image and its adjacent points A first mapping based on a first mapping relationship from the target image to the reference image is obtained using the elastic force of the target image and the frictional force acting on each point on the target image or the reference image, and the reference First image matching means for obtaining a second mapping according to a second mapping relationship from an image to the target image;
The positional relationship between corresponding points of the target image and the reference image, the force due to the second image correlation potential energy gradient based on the image information, and between each point on the target image or the reference image and its adjacent points A third mapping based on a third mapping relationship from the target image to the reference image is obtained using the elastic force of the target image and the frictional force acting on each point on the target image or the reference image, and the reference Second image matching means for obtaining a fourth mapping according to a fourth mapping relationship from the image to the target image;
A point obtained by mapping a point on the target image according to the first mapping relationship to a point mapped by the second mapping relationship is obtained, and a first reliability probability is calculated based on the positional relationship between these points. A reliability calculation means for obtaining a point obtained by mapping a point of the above-described point according to the third mapping relationship, and calculating a second reliability probability based on the positional relationship between the points;
Combining means for combining the first, second, third and fourth mappings according to the first and second confidence probabilities;
An image matching apparatus comprising:

Images