JP5750884B2 - Image processing apparatus, imaging apparatus, and program - Google Patents

Description

  The present invention relates to an image processing device, an imaging device, and a program.

  Conventionally, various techniques for aligning a plurality of images have been proposed as basic techniques for image composition. As an example, a technique for modeling the movement by the lightness gradient method or the like and performing image alignment is known. In addition, when performing the above-described image alignment, a technique of performing image alignment at multiple resolutions using a reduced image generated from an input image has been studied.

Patent No. 3914973

  When the above-described movement is modeled and image alignment is performed, if the value of an element included in the matrix increases, the calculation load becomes very large.

  In view of the above circumstances, there is provided a technique for suppressing the calculation load for alignment of a plurality of images without greatly affecting the detection accuracy of positional deviation.

  An image processing apparatus according to one aspect includes an acquisition unit that acquires a pair of a first image and a second image, a reduced image generation unit, and a positional deviation calculation unit. The reduced image generation unit generates a pair of reduced images by reducing the first image and the second image with the same image size ratio. The positional deviation calculation unit obtains the positional deviation between the first image and the second image after repeating the alignment of the reduced images in order of increasing image size ratio using a plurality of reduced image pairs. In addition, the reduced image generation unit generates a plurality of reduced image pairs such that each image size ratio changes by a reciprocal of a constant power. Then, the position shift calculation unit determines the interval between the sample points to be calculated in each image so as to change by a multiple of a square root of a constant with respect to the sample point of the reduced image having the minimum image size ratio.

In the one aspect described above, the reduced image generation unit may determine the image size ratio for each reduced image so that the image size ratio for the first image and the second image is 1 / (4 ^k ). . In addition, the position shift calculation unit sets the interval between the sample points in the reduced image having the image size ratio of 1 / (4 ^k ) to a multiple of 2 ^Nk when the minimum image size ratio is 1 / (4 ^N ). You may decide. However, N is an integer greater than or equal to 1, and k is an integer greater than or equal to 1 and less than or equal to N.

  In the one aspect described above, the positional deviation calculation unit includes at least one of a rotationally symmetric position with respect to the origin in the image and a line-symmetrical position with respect to an axis passing through the origin among the sample point candidates. Candidates that satisfy the condition may be grouped. Further, the positional deviation calculation unit may determine whether or not the above candidates are used as sample points for each group.

  An image processing apparatus according to another aspect includes an acquisition unit that acquires a pair of a first image and a second image, a reduced image generation unit, and a positional deviation calculation unit. The reduced image generation unit generates a pair of reduced images by reducing the first image and the second image with the same image size ratio. The positional deviation calculation unit obtains the positional deviation between the first image and the second image after repeating the alignment of the reduced images in order of increasing image size ratio using a plurality of reduced image pairs. In addition, the positional deviation calculation unit is at least one of a rotationally symmetric position with reference to the origin in the image and a line-symmetrical position with respect to an axis passing through the origin among the sample point candidates to be calculated in each image. Group candidates that satisfy the relationship. Then, the positional deviation calculation unit determines whether or not the candidate is used as a sample point for each group.

  The method category includes an imaging device including the above-described image processing device together with an imaging unit, a program that causes a computer to operate as the above-described image processing device, a storage medium that stores the program, and an operation of the above-described image processing device. Any expression is effective as a specific embodiment of the present invention.

  Since the value of the matrix calculation element can be reduced when performing alignment between a plurality of images, the calculation load for image alignment can be suppressed without greatly affecting the detection accuracy of the positional deviation.

The figure which shows the structural example of the image processing apparatus in one Embodiment. 6 is a flowchart illustrating an operation example of an image processing apparatus according to an embodiment. Diagram showing the relationship between the image size ratio of images and the interval between sample points The figure which shows the example of grouping of the sample point candidate in # 106 The figure which shows the outline | summary of the position shift calculation in the multi-resolution in # 107 The figure which shows the structural example of the imaging device in other embodiment.

<Description of One Embodiment>
FIG. 1 is a diagram illustrating a configuration example of an image processing apparatus according to an embodiment. An image processing apparatus according to an embodiment is a personal computer in which an image processing program that executes a composition process of a plurality of images is installed.

  A computer 11 illustrated in FIG. 1 includes a data reading unit 12, a storage device 13, a CPU 14, a memory 15, an input / output I / F 16, and a bus 17. The data reading unit 12, the storage device 13, the CPU 14, the memory 15, and the input / output I / F 16 are connected to each other via a bus 17. Further, an input device 18 (keyboard, pointing device, etc.) and a monitor 19 are connected to the computer 11 via an input / output I / F 16. The input / output I / F 16 receives various inputs from the input device 18 and outputs display data to the monitor 19.

  The data reading unit 12 is used when reading image data or a program from the outside. For example, the data reading unit 12 communicates with a reading device (such as an optical disk, a magnetic disk, or a magneto-optical disk reading device) that acquires data from a removable storage medium, or an external device in accordance with a known communication standard. A communication device to perform (USB interface, LAN module, wireless LAN module, etc.).

  The storage device 13 is configured by a storage medium such as a hard disk or a nonvolatile semiconductor memory. The storage device 13 stores an image processing program. The storage device 13 can also store image data read from the data reading unit 12 and image data synthesized by an image processing program.

  The CPU 14 is a processor that comprehensively controls each unit of the computer 11. The CPU 14 operates as a reduced image generation unit 21, a positional deviation calculation unit 22, and an image processing unit 23 by executing the image processing program (the operations of the reduced image generation unit 21, the positional deviation calculation unit 22, and the image processing unit 23). Will be described later).

  The memory 15 temporarily stores various calculation results in the image processing program. The memory 15 is, for example, a volatile SDRAM.

  Next, an operation example of image composition processing by the image processing apparatus according to the embodiment will be described with reference to the flowchart of FIG. The CPU 14 starts the process of the flowchart of FIG. 2 in response to a program execution instruction from the user.

  Step # 101: The CPU 14 acquires from the data reading unit 12 the data of the first image to be aligned. Then, the CPU 14 records the first image data in the storage device 13.

  Step # 102: The CPU 14 acquires the data of the second image to be aligned from the data reading unit 12. Then, the CPU 14 records the second image data in the storage device 13.

  Here, it is assumed that the first image and the second image have substantially the same positional relationship of the subject between the images. As an example, the first image and the second image are two images extracted from a series of images obtained by moving image shooting or continuous shooting.

  The first image and the second image may be of any type as long as they have the same image size (resolution) and the same format. For example, the first image and the second image may be a RAW image having a Bayer array structure before color interpolation processing, and an image (RGB image) that has undergone color reproduction by performing image processing on the RAW image. , YCbCr image, JPEG image, etc.).

  Step # 103: The reduced image generating unit 21 reduces the first image (# 101) and the second image (# 102) with the same image size ratio, and reduces the first image (first reduced image) and the first image. A pair with two reduced images (second reduced images) is generated. At this time, the reduced image generation unit 21 generates a plurality of pairs of reduced images by changing the value of the image size ratio so that each image size ratio changes with a reciprocal of a power of a constant. Note that the reduced image has a smaller image size (resolution) in proportion to the smaller image size ratio.

Here, the reduced image generation unit 21 in one embodiment determines the image size ratio so that the image size ratio with respect to the input image (first image and second image) can be expressed by the reciprocal of a power of 4. To do. That is, the reduced image generation unit 21 sets the image size ratio so that the minimum image size ratio is “1 / (4 ^N )” and each image size ratio is “1 / (4 ^k )”. (However, in the description of the present specification, N is an integer of 1 or more, and k is an integer of 1 or more and N or less).

For example, when four pairs of reduced images are generated (N = 4), the image size ratios are “1/4 (= 1 / (4 ¹ ))”, “1/16 (= 1/1) in descending order. (4 ² )) ”,“ 1/64 (= 1 / (4 ³ )) ”, and“ 1/256 (= 1 / (4 ⁴ )) ”. In this case, four first reduced images and four second reduced images are generated at four image size ratios of 1/4, 1/16, 1/64, and 1/256, respectively.

  Step # 104: The position shift calculation unit 22 determines the interval between sample points that can be calculated in each image. It is assumed that the sample points in # 104 are evenly arranged in a grid pattern in the image. In one embodiment, the total number of sample points is common among different images.

  Here, the position shift calculation unit 22 at # 104 determines the coordinates of the sample point of the reduced image at which the image size ratio becomes the minimum value. Then, the positional deviation calculation unit 22 determines the interval between the sample points in the other reduced images so as to be a multiple of a square root of a constant with respect to the sample point of the reduced image at which the image size ratio becomes the minimum value. This makes it possible to express the coordinates of the sample point by multiplying the sample point of the reduced image with the lowest resolution in both the first image and second image pair and each reduced image pair. Become.

For example, in the case of a reduced image in which the image size ratio exemplified in # 103 can be expressed by 1 / (4 ^k ), the positional deviation calculation unit 22 in # 104 is determined so that the interval between the sample points is a multiple of 2 ^Nk. That's fine. Note that the interval between the sample points relates to one of the vertical and horizontal sizes, so the constant is “2”. However, since the image size ratio is related to the vertical and horizontal sizes, the constant is “4 (= 2 ² )”. .

As shown in FIG. 3, the interval between sample points in a reduced image with an image size ratio of 1/256 is m. At this time, the interval between the sample points in the reduced image having the image size ratio of 1/64 is “(2 ^4-3 ) · m = 2 m”. It can be seen that since the reduced image having an image size ratio of 1/64 is twice as long as the reduced size image having an image size ratio of 1/256, the relative interval between the sample points is the same. Similarly, the interval between sample points in a reduced image having an image size ratio of 1/4 is “(2 ^4-1 ) · m = 8 m”. It can be seen that the relative interval between the sample points is the same because the reduced image having the image size ratio of 1/4 is 8 times larger in the vertical and horizontal directions than the reduced image having the image size ratio of 1/256.

  Note that the sample points obtained in # 104 are actually sample point candidates, and some of the sample point candidates are excluded from the calculation target by the process of # 106 described later.

  Step # 105: The positional deviation calculator 22 generates differential images using the second image (# 102) and the second reduced image (# 103).

  At this time, the position shift calculation unit 22 obtains a difference between adjacent sample points in the x direction (lateral direction of the image) at each sample point of the second image, and generates a differential image in the x direction. Further, the position shift calculation unit 22 obtains a difference between adjacent sample points in the y direction (vertical direction of the image) at each sample point of the second image, and generates a differential image in the y direction. Then, the positional deviation amount calculation unit 22 repeats the above processing for each second reduced image, and generates differential images in the x direction and the y direction, respectively.

  When four second reduced images are generated with four image size ratios as in the example of # 103, in the process of # 105, two differential images based on the second image and four second reduced images are used. Eight differential images will be generated.

  Step # 106: The positional deviation calculation unit 22 narrows down the sample point candidates (# 104) and determines the sample point to be calculated.

  In general, image alignment methods often perform calculation by extracting characteristic portions (such as edge portions) in an image. Since the example of one embodiment also uses the feature of the edge by the differential image, it is desirable that the position of the sample point is a pixel with an edge. However, if only the part with an edge from the entire image is adopted as the sample point, the image is displayed when the edge part is unevenly distributed in a part of the image or when many sample points are set in a part of the moving object. There is also a possibility that the entire original misalignment amount and the calculation result may deviate. Therefore, in order to obtain a stable calculation result, it is necessary to prevent the distribution of sample points from being biased in the image.

  From the above viewpoint, the position shift calculation unit in # 106 determines a sample point to be calculated by the following processes (a) to (e) (see FIG. 4).

(A) The positional deviation calculation unit 22 groups a sample point candidate to be noticed (a point of interest) and three sample point candidates having the following relationship with respect to the point of interest.
A sample point candidate at a rotationally symmetric position with respect to the point of interest with reference to the origin in the image (for example, the sample point closest to the center of the image or the image center).
A sample point candidate in a line-symmetric position with respect to the point of interest with reference to the x-axis (horizontal axis) passing through the origin in the image.
A sample point candidate located in a line-symmetric position with respect to the point of interest with reference to the y-axis (vertical axis) passing through the origin in the image.

  (B) The positional deviation calculation unit 22 extracts edge amounts from the four sample point candidates grouped in (b). For example, the positional deviation calculation unit 22 may extract the pixel value of the corresponding point from the differential image of the second image obtained in # 105, and may use the edge amount of each sample point candidate.

  (C) The positional deviation calculation unit 22 obtains an average value and a variance value of the edge amounts (obtained in (b) above) of the four grouped sample point candidates.

  (D) The positional deviation calculation unit 22 determines whether or not to adopt four sample point candidates as sample points in units of groups using the average value and the variance value of the edge amounts. For example, the position shift calculation unit 22 may use a function for obtaining an evaluation value from an average value of edge amounts and a variance value and determine that the evaluation value is adopted as a sample point when the evaluation value is equal to or greater than a threshold value. Note that a group of sample point candidates determined not to be used as sample points is excluded from the calculation target in the subsequent calculation process.

  Here, it is assumed that the evaluation value of the above function increases as the average value of the edge amount increases, and increases as the dispersion value of the edge amount decreases. Therefore, in the determination in (d), the higher the edge amount of the sample point candidate and the more the edge amount varies among the sample point candidates, the easier it is to adopt the sample point.

  (E) The positional deviation calculation unit 22 repeats the processes (a) to (d) above, groups all sample point candidates, and determines whether to use the sample point candidates as sample points for each group. To do.

  By performing the processing of # 106, sample points that do not greatly contribute to the calculation of the positional deviation amount can be excluded from the calculation target, so that the calculation load can be suppressed. In addition, the processing of # 106 makes it possible to adopt sample points suitable for the calculation of the positional deviation amount without deviation in the image, and to improve the calculation accuracy of the positional deviation amount.

  Step # 107: The positional deviation calculation unit 22 obtains a positional deviation between the first image and the second image using a geometric transformation model. At this time, the position shift calculation unit 22 uses a plurality of pairs of reduced images having different resolutions and repeats the alignment of the reduced images in order of increasing image size ratio, and then shifts the position of the first image and the second image. Obtained (see FIG. 5).

  First, the operation of the geometric transformation model applied in # 107 will be outlined. Here, for the sake of simplicity, an example in which the first image A and the second image B are subject to calculation will be described.

As an example, the positional deviation calculation unit 22 of one embodiment performs position adjustment between images using a six-axis geometric transformation model shown in Expression (1). Note that Expression (1) expresses the positional deviation amount of the image in a format in which the coordinate (x, y) of the image is coordinate-transformed by a geometric transformation model having six variables (p _{0 to} p ₅ ).

ここで、第１画像Ａと第２画像Ｂとの位置ズレ量の演算は、式（２）の評価値ΔＥを最小化する問題と捉えることができる。

Here, the calculation of the positional deviation amount between the first image A and the second image B can be regarded as a problem of minimizing the evaluation value ΔE of the equation (2).

上記の式（２）では、一方の画像（第１画像Ａ）の座標（ｘ，ｙ）を幾何変換して得た画素値と、他方の画像（第２画像Ｂ）の座標（ｘ，ｙ）の画素値との差の二乗和によって、評価値ΔＥを求めている。上記の評価値ΔＥが小さいほど、２画像間の合致度は高くなる。なお、上記の「ｎ」は画像Ａ，Ｂの差を求める標本点の数を示す。

In the above equation (2), the pixel value obtained by geometric transformation of the coordinates (x, y) of one image (first image A) and the coordinates (x, y) of the other image (second image B). ) To obtain the evaluation value ΔE. The smaller the evaluation value ΔE, the higher the degree of match between the two images. The above “n” indicates the number of sample points for obtaining the difference between the images A and B.

本明細書において、「ｄｘ」は第２画像Ｂのｘ方向の微分画像を示し、「ｄｙ」は第２画像Ｂのｙ方向の微分画像を示す。また、下付の添字ｎは、各画像における標本点ｎの座標であることを示している。

In this specification, “dx” indicates a differential image of the second image B in the x direction, and “dy” indicates a differential image of the second image B in the y direction. Further, the subscript n indicates the coordinates of the sample point n in each image.

図５に示すように、多重解像度で位置ズレ演算を行うことで、低解像度の画像で求めた位置ズレ量の演算結果を、高解像度の画像を用いた位置ズレ量の演算の初期値とすることができ、処理全体での演算量を削減できる。また、低解像度の画像では画像の平滑化が行われることからノイズの影響を低減できる。

As illustrated in FIG. 5, by performing the positional shift calculation at multiple resolutions, the calculation result of the positional shift amount obtained from the low resolution image is used as the initial value of the positional shift amount calculation using the high resolution image. It is possible to reduce the amount of calculation in the entire process. Further, since the image is smoothed in the low resolution image, the influence of noise can be reduced.

  In the calculation of the positional deviation amount in one embodiment, it is assumed that the positional deviation amount is very small and that the positional deviation itself can be expressed by a geometric transformation model. For this reason, the initial value of the calculation and how to take the sample points of the calculation are important factors in the calculation performance.

  Here, since the elements of the matrix shown in Equation (4) are the accumulated values of n, it can be seen that the value of the elements of the matrix can be reduced if the number of sample points is reduced by the method of # 106.

上記の手法による場合、位置ズレ演算部２２は、行列の各要素を求めるときに低解像度の画像の座標値で演算を行える。低解像度の画像は、そもそも画像が小さいので座標値も相対的に小さな値となる。そのため、上記の手法による場合には、行列演算の各要素の値をより小さな値にできるので、位置ズレの演算において精度を保ちつつ演算規模を抑制することができる。

In the case of the above method, the positional deviation calculation unit 22 can perform calculation using the coordinate values of the low-resolution image when obtaining each element of the matrix. Since the low resolution image is small in the first place, the coordinate value is relatively small. Therefore, in the case of the above method, the value of each element of the matrix calculation can be made smaller, so that the calculation scale can be suppressed while maintaining accuracy in the calculation of the positional deviation.

  Step # 108: The image processing unit 23 synthesizes the first image and the second image using the positional deviation amount obtained in # 107. For example, the image processing unit 23 can generate a combined image in which camera shake is corrected by combining the first image and the second image, which are continuously shot with an exposure period shorter than the appropriate exposure. . Further, the image processing unit 23 can generate a composite image having a wide dynamic range by combining the first image and the second image acquired by changing the exposure conditions. Then, the CPU 14 records the composite image data in the storage device 13. Above, description of the flowchart of FIG. 2 is complete | finished.

<Description of other embodiments>
FIG. 6 is a diagram illustrating a configuration example of an imaging apparatus according to another embodiment. The other embodiment is an example in which the image processing apparatus of one embodiment is mounted on the electronic camera 31, and the image processing apparatus acquires the first image and the second image from the imaging unit 33 of the electronic camera 31.

  The electronic camera 31 includes a photographic lens 32, an imaging unit 33, an image processing engine 34, a first memory 35 and a second memory 36, a recording I / F 37, and an operation unit 38. Here, the imaging unit 33, the first memory 35, the second memory 36, the recording I / F 37, and the operation unit 38 are each connected to the image processing engine 34.

  The imaging unit 33 is a module that captures (captures) an image of a subject formed by the photographing lens 32. For example, the imaging unit 33 includes an imaging device that performs photoelectric conversion, an analog front-end circuit that performs analog signal processing, and a digital front-end circuit that performs A / D conversion and digital signal processing.

  The image processing engine 34 is a processor that comprehensively controls the operation of the electronic camera 31. For example, the image processing engine 34 causes the imaging unit 33 to continuously shoot the first image and the second image in response to the user's shooting instruction input in the shooting mode. Further, the image processing engine 34 can operate in the same manner as the CPU 14 (reduced image generation unit 21, position shift calculation unit 22, image processing unit 23) of the embodiment by executing a program.

  The first memory 35 is a memory that temporarily stores data of the first image and the second image, and is configured by, for example, an SDRAM that is a volatile storage medium. The second memory 36 is a memory for storing a program executed by the image processing engine 34, and is configured by a non-volatile memory such as a flash memory, for example.

  The recording I / F 37 has a connector for connecting a nonvolatile storage medium. The recording I / F 37 writes / reads image data to / from the storage medium 39 connected to the connector. The storage medium 39 is, for example, a hard disk or a memory card incorporating a semiconductor memory. In FIG. 6, a memory card is illustrated as an example of the storage medium.

  The operation unit 38 has a plurality of switches that accept user operations. The operation unit 38 includes, for example, a release button for receiving a recording still image shooting instruction.

  Hereinafter, an operation example of the electronic camera 31 in another aspect will be briefly described. In the electronic camera 31 according to another aspect, the imaging unit 33 continuously captures the first image and the second image in response to the user's capturing instruction input. Thereby, the acquisition of the first image and the second image corresponding to the processes of # 101 and # 102 in FIG. 2 is performed. Note that the image to be aligned may be a partial frame of a moving image (including a so-called through image for composition confirmation).

  Then, the image processing engine 34 uses the first image and the second image acquired by the imaging unit 33 to detect the positional deviation amount of the image by the same method as # 103 to # 107 in FIG. After that, the image processing engine 34 performs image composition processing using the amount of positional deviation similarly to # 108 in FIG. 2, and generates a composite image with corrected camera shake and a composite image with a wide dynamic range. Thereafter, the composite image is recorded on the storage medium 39 via the recording I / F 37. The image processing engine 34 may display a composite image on a display unit (not shown). In such other embodiments, substantially the same effects as those of the first embodiment can be obtained.

<Supplementary items of the embodiment>
(A) In place of the six variables (p _{0 to} p ₅ ) described in the above embodiment, the calculation method of the present invention can be similarly applied to a geometric transformation model that implements projective transformation using eight variables.

  (B) The positional deviation calculation unit 22 of # 106 of the above embodiment is any of the sample point candidate that is rotationally symmetric with the target point, the sample point candidate that is symmetrical with the target point and the x axis, and the sample point candidate that is symmetrical with the target point and the y axis. One of them may be grouped with the attention point.

  (C) In the above embodiment, the example in which each function of the image processing apparatus is realized by software by a program has been described. However, in the present invention, each function of the reduced image generation unit 21, the position shift calculation unit 22, and the image processing unit 23 may be realized by hardware using an ASIC.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 11 ... Computer, 12 ... Data reading part, 13 ... Memory | storage device, 14 ... CPU, 21 ... Reduced image generation part, 22 ... Position shift calculation part, 23 ... Image processing part, 31 ... Electronic camera, 33 ... Imaging part, 34 ... Image processing engine

Claims (6)

An acquisition unit for acquiring a pair of a first image and a second image;
A reduced image generating unit that generates a pair of reduced images by reducing the first image and the second image at the same image size ratio;
A positional deviation calculation unit for obtaining a positional deviation between the first image and the second image after repeating the alignment of the reduced image in order of increasing value of the image size ratio using a plurality of pairs of reduced images; Prepared,
The reduced image generation unit generates a plurality of reduced image pairs such that each of the image size ratios changes by a reciprocal of a power of a constant,
The positional deviation calculation unit determines the interval between the sample points to be calculated in each image so as to change with a multiple of the square root of the constant with respect to the sample points of the reduced image where the image size ratio is the minimum value ,
The positional deviation calculation unit satisfies the relationship of at least one of a rotationally symmetric position with respect to an origin in an image and a line symmetric position with respect to an axis passing through the origin, among the sample point candidates. An image processing apparatus for determining whether or not the candidate is set as the sample point for each group. The image processing apparatus according to claim 1.
The reduced image generation unit determines an image size ratio for each reduced image so that an image size ratio with respect to the first image and the second image is 1 / (4 ^k ),
The positional shift calculation unit, when the image size ratio of the minimum value of 1 / a (4 ^N), the image size ratio at intervals of two multiple of ^Nk of the sample points of the reduced image of 1 / (4 ^k) An image processing apparatus to be determined. (However, N is an integer of 1 or more, and k is an integer of 1 or more and N or less.) An acquisition unit for acquiring a pair of a first image and a second image;
A reduced image generating unit that generates a pair of reduced images by reducing the first image and the second image at the same image size ratio;
A positional deviation calculation unit for obtaining a positional deviation between the first image and the second image after repeating the alignment of the reduced image in order of increasing value of the image size ratio using a plurality of pairs of reduced images; Prepared,
The positional deviation calculation unit includes at least one of a rotationally symmetric position with respect to the origin in the image and a line symmetric position with respect to an axis passing through the origin, among the sample point candidates to be calculated in each image. An image processing apparatus that groups the candidates that satisfy the above relationship and determines whether or not the candidates are used as the sample points for each group. An imaging unit for imaging the first image and the second image;
The image processing apparatus according to any one of claims 1 to 3 ,
An imaging apparatus comprising: On the computer,
An acquisition process for acquiring a pair of a first image and a second image;
Reduced image generation processing for generating a pair of reduced images by reducing the first image and the second image at the same image size ratio;
A positional deviation calculation process for obtaining a positional deviation between the first image and the second image after repeating the alignment of the reduced images in order of increasing value of the image size ratio using a plurality of pairs of reduced images; Let it run
In the reduced image generation process, a plurality of pairs of reduced images are generated such that each of the image size ratios changes with a reciprocal of a power of a constant,
In the positional deviation calculation process, the interval between the sample points to be calculated in each image is determined so as to change with a multiple of the square root of the constant with respect to the sample point of the reduced image whose image size ratio is the minimum value ,
In the positional deviation calculation process, among the candidate sample points, the candidate satisfying at least one of a rotationally symmetric position with respect to an origin in an image and an axisymmetric position with respect to an axis passing through the origin. A program for determining whether or not the candidate is set as the sample point for each group . On the computer,
An acquisition process for acquiring a pair of a first image and a second image;
Reduced image generation processing for generating a pair of reduced images by reducing the first image and the second image at the same image size ratio;
A positional deviation calculation process for obtaining a positional deviation between the first image and the second image after repeating the alignment of the reduced images in order of increasing value of the image size ratio using a plurality of pairs of reduced images; Let it run
In the positional deviation calculation processing, at least one of a rotationally symmetric position with reference to the origin in the image and a line-symmetrical position with respect to an axis passing through the origin among the sample point candidates to be calculated in each image A program for grouping the candidates that satisfy the above relationship and determining whether or not the candidate is used as the sample point for each group.

Images