JP4821704B2 - Image distortion correction method - Google Patents

Description

  The present invention relates to an image distortion correction method for correcting image distortion caused by distortion aberration of an imaging optical system including a lens.

  For example, in a TV camera used for an intercom system with a TV, it is necessary to capture a wide range as far as possible from a subject (visitor) and the TV camera. Since a lens is used, there is a problem that a captured image is distorted due to distortion aberration or the like of an imaging optical system (super wide-angle lens). Therefore, the present applicant has already proposed a method of correcting image distortion by performing coordinate conversion in which the position (coordinates, address) of a pixel is moved in the horizontal direction (or horizontal and vertical directions) (for example, patents). References 1 and 2).

  An image distortion correction method (hereinafter referred to as “conventional example 1”) described in Patent Document 1 formulates characteristics of image distortion caused by a lens of a television camera or the like, and converts coordinates calculated in advance using the formula. And a correction coefficient for correcting the pixel value after the coordinate conversion is stored in a memory table, and the coordinate conversion and the pixel value correction are performed with reference to the memory table for the input image captured from the TV camera. Is.

Also, the image distortion correction method described in Patent Document 2 (hereinafter referred to as “conventional example 2”) uses arbitrary coordinates of an input image as (x ′, y ′), and coordinates after image distortion correction as ( x ′ = {1−k (y−y ₀ ) ² } x + k (y−) where x, y), the coordinates of the image center are (x ₀ , y ₀ ), and the parameter representing the distortion rate of the image is k. y ₀ ) The input image coordinates are converted using a coordinate conversion formula expressed by ² x ₀ , y ′ = y, and the pixel values of the converted input image coordinates are linearly approximated with neighboring pixel values. Is to be calculated.
Japanese Patent No. 3451644 Japanese Patent No. 3047681

By the way, in the above-described conventional example 2, coordinate conversion is performed according to the distance from the coordinates (x ₀ , y ₀ ) of the image center. When the deviation generated between the center and the center becomes large, the image distortion cannot be corrected appropriately. Further, the conventional example 1 has a drawback that the memory capacity is greatly increased because the conversion coordinates and the correction coefficient corresponding to the above-described center deviation are stored in the memory table. That is, in the conventional examples 1 and 2, there is a problem that it is difficult to perform correction according to the characteristics of the image distortion such as a shift amount generated between the optical axis center of the imaging optical system and the image center.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image distortion correction method capable of easily performing correction according to image distortion characteristics.

In order to achieve the above object, according to a first aspect of the present invention, in an image distortion correction method for correcting image distortion due to distortion of an optical system, arbitrary coordinates of an input image are set to (x ′, y ′), and after distortion correction. Is the coordinate of (x, y), the coordinate of the center of the image is (x ₀ , y ₀ ), and n is a positive integer,

The coordinates of the input image are obtained using the coordinate conversion formula expressed by the following equation, and the pixel value at the coordinates of the input image is obtained by linear approximation with the pixel values of the pixels existing in the vicinity of the coordinates.

According to the first aspect of the present invention, the correction according to the characteristics of the image distortion can be easily performed by appropriately setting the coefficient a _ijkl and the coordinates of the image center in the coordinate conversion formula.

The invention of claim 2 is characterized in that, in the invention of claim 1, at least one of the coefficient a _ijkl and the coordinates x ₀ and y ₀ of the image center in the coordinate conversion formula is changed.

According to the second aspect of the present invention, the image distortion can be more appropriately _achieved by changing at least one of the coefficient a _ijkl and the coordinates x ₀ and y ₀ of the image center in the coordinate conversion equation according to the image distortion characteristics. Can be corrected.

According to a third aspect of the present invention, in the second aspect of the invention, a plurality of combinations of the coefficient a _ijkl and the coordinates of the image center in the coordinate conversion formula are prepared in advance, and any one of the plurality of combinations is prepared. One of the combinations is selected and coordinate transformation is performed.

According to the invention of claim 3, it is possible to easily change to the coefficient a _{ijkl and} the coordinates x ₀ and y _{0 of the} image center necessary for performing appropriate image distortion correction.

The invention of claim 4 is the invention of claim 2 or 3, compensation before the image after correction to the input image of the coefficient a _ijkl and image center in accordance with the horizontal magnification of coordinates x _0, y ₀ At least one of them is changed .
According to the invention of claim 4, it is possible to correct distortion appropriately according to the horizontal magnification of the corrected image.

The invention of claim 5 is the invention of any one of claims 2-4, compensation prior to the coordinates of the coefficients a _ijkl and image center in accordance with the position movement amount in the horizontal direction of the image after correction to the input image It is characterized in that at least one of x ₀ and y ₀ is changed .
According to the fifth aspect of the invention, it is possible to correct distortion appropriately according to the amount of horizontal position movement of the corrected image.

The invention of claim 6 is the invention of any one of claims 2 to 5, the coefficient according to the ratio of the horizontal magnification of the top and bottom of the image after correction for compensation prior to input image a _ijkl In addition, at least one of the coordinates x ₀ and y ₀ of the image center is changed .
According to the sixth aspect of the present invention, it is possible to perform appropriate distortion correction according to the ratio of the magnification in the horizontal direction at the upper end and the lower end of the corrected image with respect to the input image before correction.

The invention according to claim 7 is the same as that according to claim 1 , wherein the pixel value of the coordinate is the same as the coordinate when the coordinate of the input image obtained by the coordinate conversion formula does not exist within the range of the actual input image . The pixel value of the coordinate which exists in the nearest position on a horizontal line and exists in the range of an actual input image is employ | adopted.
According to the seventh aspect of the present invention, it is possible to make inconspicuous a problem occurring in the corrected image by referring to coordinates that do not exist within the range of the input image.

The invention of claim 8 is the invention of claim 1 , wherein when the coordinates of the input image obtained by the coordinate conversion formula do not exist within the range of the actual input image , a value representing an achromatic color is used as the pixel value of the coordinates. It is characterized by adopting.

The invention of claim 9 is the invention of claim 8 , wherein a value representing black is adopted as the value representing the achromatic color .

According to a tenth aspect of the present invention, in the first aspect of the present invention, when the coordinates of the input image obtained by the coordinate conversion formula do not exist within the range of the actual input image, the coefficient a is set so that the coordinates are outside the display range. _The image is enlarged by adjusting _ijkl .

According to the tenth aspect of the present invention, it is possible to make inconspicuous a problem that occurs in the corrected image by referring to coordinates that do not exist within the range of the input image.

According to an eleventh aspect of the present invention, in the first aspect of the present invention, when the coordinates of the input image obtained by the coordinate conversion formula do not exist within the range of the actual input image, the input image is enlarged to display the coordinates. The coefficient aijkl is adjusted so as to be outside.

According to the eleventh aspect of the present invention, it is possible to make inconspicuous a problem that occurs in the corrected image by referring to coordinates that do not exist within the range of the input image.

According to the present invention, the correction according to the characteristics of the image distortion can be easily performed by appropriately setting the coefficient a _ijkl and the coordinates of the image center in the coordinate conversion formula.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, a hardware configuration for realizing an image distortion correction method according to the present invention, that is, an image distortion correction apparatus will be described with reference to FIG.

  As shown in FIG. 1, a digital image output from the television camera CAM is taken into the image distortion correction apparatus 1 as an input image, and the image after the image distortion is corrected in the image distortion correction apparatus 1 is displayed like a liquid crystal display. Output to the device DSP and displayed. However, when the TV camera CAM and the display device DSP support only analog signals, an A / D converter that converts an analog input image into a digital image and a D that converts a digital output image into an analog image The / A converter needs to be mounted on the image distortion correction apparatus 1.

The image distortion correction apparatus 1 includes a line buffer memory 2 that stores image data (pixel values) for one horizontal line of an input image, and arbitrary coordinates (addresses) of the input image (x ′, y ′). When the later coordinates are (x, y), the coordinates of the image center are (x ₀ , y ₀ ), and n is a positive integer, the coordinates (x ', Y') to obtain a linear approximation of the pixel values at the coordinates (x ', y') of the input image obtained by the coordinate transformation means 3 from the pixel values of the pixels existing in the vicinity of the coordinates. And a pixel value correction means 4 to be obtained. The coordinate conversion means 3 and the pixel value correction means 4 are realized by a microcomputer executing a predetermined image distortion correction processing program.

  As shown in FIG. 2A, the input image input from the TV camera CAM to the image distortion correction apparatus 1 includes distortion aberration in an imaging optical system such as a lens provided in the TV camera CAM (in the illustrated example, barrel distortion). ) Due to image distortion. The second term on the right-hand side of the above coordinate conversion formula is that the curve a shown in FIG. 2B, that is, the vertical straight line (straight line parallel to the y-axis) that should pass through arbitrary coordinates (x, y) Approximate the curve distorted by the above, and substitute the arbitrary coordinates (x, y) into the above coordinate conversion formula, the coordinates (x ′, y ′) of the input image, that is, the coordinates (x, y) The image distortion can be corrected by obtaining the coordinate moved by the image distortion and obtaining the pixel value of the coordinate (x, y) from the pixel value of the coordinate (x ′, y ′) of the input image. However, the processing for obtaining the pixel value of the coordinate (x, y) from the pixel value of the coordinate (x ′, y ′) of the input image, that is, the linear approximation performed by the pixel value correcting means 4 will be described in Conventional Example 2. Therefore, since it is a well-known technique, detailed description is abbreviate | omitted.

Here, also in Conventional Example 2, coordinate conversion using the coordinate conversion formula x ′ = {1−k (y−y ₀ ) ² } x + k (y−y ₀ ) ² x ₀ , y ′ = y is performed. Although the image distortion is corrected, as can be seen from the expression x ′ = x−k (x−x ₀ ) (y−y ₀ ) ^{2 obtained} by modifying the coordinate conversion expression of the conventional example 2, it corresponds to the characteristics of the image distortion. Therefore, since there are only three parameters that can be changed, the parameter k representing the distortion rate of the image and the coordinate x ₀ or y ₀ of the image center, it is difficult to perform fine distortion correction. In contrast, in the image distortion correction method according to the present invention, a total of four or more parameters including a plurality of coefficients a _ijkl and image center coordinates x ₀ or y ₀ , corrected coordinates (x, y), and image center Since the absolute values | x−x ₀ | and | y−y ₀ | of the difference between the coordinates (x ₀ , y ₀ ) of the image are included in the coordinate conversion equation, the image distortion characteristics are compared with those of the conventional example 2. Therefore, it is possible to easily perform fine distortion correction. In addition, since the coordinate conversion formula includes only multiplication and addition, the calculation time is simple, so the processing time is short. As a result, it is necessary to store the calculation result in advance in the data table as in Conventional Example 1. Nor.

  For example, an image of a square pattern wall surface captured by a television camera CAM and displayed on the display device DSP without performing image distortion correction is shown in FIG. As shown in FIG. 3B, if the straight line in the vertical direction and the straight line in the horizontal (horizontal) direction are both distorted as described above, the same image is corrected using the image distortion correction method of Conventional Example 1. The straight line distortion in the direction can be considerably corrected. Here, paying attention to the vertical straight line L1 near the left end of the screen in FIG. 3B, it can be seen that the distortion correction is insufficient at portions near the upper end and the lower end of the screen.

On the other hand, when the image distortion correction is performed by the image distortion correction apparatus 1 of the present embodiment, for example, the coefficient a _{1020 in} the term (x−x ₀ ) (y−y ₀ ) ^{2 in} the above coordinate conversion formula, and (x− x ₀ ) | y−y ₀ | ³ term coefficient a ₁₀₀₃ and (x−x ₀ ) | x−x ₀ || y−y ₀ | ³ term coefficient a ₁₁₀₂ in the television camera CAM. If an appropriate value is set according to the image distortion characteristic of the optical system, the image distortion at the upper end and the lower end of the same straight line L1 as shown in FIG. It can be corrected. In other words, since only one type (k) of the image distortion rate parameter is set in Conventional Example 1, the image distortion correction is insufficient as the distance from the image center increases (approaching the upper and lower ends of the image). However, according to the image distortion correction method of the present invention, coordinate conversion is performed using a plurality of coefficients (for example, the three coefficients a ₁₀₂₀ , a ₁₀₀₃ , and a _{1102 in} the above example). Thus, sufficient distortion correction can be performed even in a portion close to the upper end and the lower end of the image.

Further, in the television camera CAM, when the optical axis center of the optical system and the center (image center) of the image sensor do not coincide with each other (actually, the deviation amount between them is so large that it cannot be ignored in actual use), for example, As described above, a barrel-shaped distortion as shown in FIG. 4 (a) is generated in an image when a square-shaped wall surface is captured by the television camera CAM and displayed on the display device DSP without performing image distortion correction. When the image distortion is corrected by assuming that the optical axis center O of the optical system and the image center O ′ coincide with each other (actually, the deviation amount between them is small enough to be ignored in actual use) As shown in FIG. 4B, the correction amount becomes larger than necessary as the distance from the image center O ′ increases. On the other hand, if the optical axis center O of the optical system and the image center O ′ are not coincident, the coordinates (x ₀ , y ₀ ) of the image center are changed so as to coincide with the optical axis center O. As shown in FIG. 4C, the image distortion can be appropriately corrected according to the characteristics of the image distortion (deviation between the optical axis center O and the image center O ′).

Here, the data of the coefficient a _{ijkl and} the coordinates (x ₀ , y ₀ ) of the image center need to be stored in advance in a rewritable nonvolatile memory included in the image distortion correction apparatus 1. For example, it is alternatively selected from a plurality of types of values (data) or a combination of a plurality of values according to a setting state by a mechanical switch (for example, a dip switch) provided in the image distortion correction apparatus 1 It is also possible to change it. If the data of the coefficient a _{ijkl and} the coordinates (x ₀ , y ₀ ) of the image center can be changed in this way, the image distortion can be corrected more appropriately. As shown in FIG. 5A, when a three-dimensional object (for example, a person H) is imaged by the television camera CAM, image distortion in the television camera CAM is completely corrected by the image distortion correction apparatus 1. The image of the subject (person H) may be displayed distorted (see FIG. 5B). Even in such a case, changing the value of the coefficient a _ijkl in the coordinate conversion formula to adjust the correction degree of the image distortion can alleviate the display of the three-dimensional subject being distorted as described above. it can. Similarly, there is also an effect of making the center deviation between the optical axis center of the optical system and the image sensor inconspicuous, and not by _reducing the amount of displacement by ideal correction, but by optimizing the value of each coefficient a _ijkl Can be adjusted so that there is no sense of incongruity.

In addition, it is possible to intentionally change the corrected image (image displayed on the display device DSP) by manipulating the coefficient a _ijkl in the coordinate conversion formula. For example, with respect to an image after normal correction as shown in FIG. 6A, the image is enlarged in the x-axis direction (lateral direction) by changing the coefficient a ₁₀₀₀ of the first-order term of x. (See FIG. 6B), and when a part of the image is lost in the optical system of the television camera CAM, the image is intentionally enlarged so that the injury is not displayed. it can. Further, the image can be moved in the horizontal direction by changing the constant term a ₀₀₀₀ (see FIG. 6C). Alternatively, by changing the coefficient a ₀₀₁₀ of the first-order term of y, the perspective can be adjusted by changing the horizontal magnification at the upper and lower ends of the image (see FIG. 6D).

By the way, when the coordinates of the input image are obtained using the coordinate conversion formula, if the coordinates are converted into coordinates that do not actually exist, for example, coordinates in the region S in FIG. There is a possibility that a problem arises that it is not possible to obtain the above. Therefore, the coordinate conversion means 3 verifies whether or not the coordinates (x ′, y ′) of the input image obtained from the coordinate conversion formula correspond to coordinates that do not exist (coordinates in the region S), and correspond to coordinates that do not exist. If the pixel value correction unit 4 sets the pixel value of the coordinate to a predetermined value, for example, a value (data) representing an achromatic color such as black, the coordinate (region) that does not actually exist in the corrected image Since S ′) is displayed in an achromatic color, it can be made inconspicuous (see FIG. 7B). Or you may set the pixel value of the coordinate which does not exist to the same value as the pixel value in the nearest coordinate among the actual coordinates which exist on the same horizontal line as the said coordinate. Further, by changing the coefficient a ₁₀₀₀ of the first-order term of x and enlarging the image in the horizontal direction so that the nonexistent coordinates (region S ′) are out of the display range in the corrected image, the above-described case. Thus, there is an advantage that the image quality can be improved as compared with the case where a part of the screen is displayed in an achromatic color (see FIG. 8).

1 is a block diagram of an image distortion correction apparatus for realizing an image distortion correction method of the present invention. (A), (b) is explanatory drawing in the case of correct | amending the barrel distortion of an input image in a horizontal direction. (A) is an image before correcting image distortion, (b) is an image after performing image distortion correction of a conventional example, and (c) is an image after correcting by the image distortion correcting method of the present invention. FIG. It is explanatory drawing when an optical axis and the image center have shifted | deviated, (a) is the image before correction | amendment, (b) is the image after correction | amending assuming that the optical axis and image center correspond, ( (c) is an explanatory view showing an image after being corrected by the image distortion correction method of the present invention. (A), (b) is explanatory drawing in the case of correct | amending distortion of the image which imaged the three-dimensional photographic subject (person). (A)-(d) is explanatory drawing of the image after changing and correct _{| amending} coefficient _aijkl in a coordinate transformation type _| formula. (A), (b) is explanatory drawing of the image distortion correction method of this invention in case the coordinate calculated | required by coordinate transformation does not actually exist in an input image. It is explanatory drawing of the image distortion correction method of this invention in the case where the coordinate calculated | required by coordinate transformation does not exist in an input image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image distortion correction apparatus 2 Line buffer memory 3 Coordinate conversion means 4 Pixel value correction means CAM TV camera DSP Display apparatus

Claims (11)

In an image distortion correction method for correcting image distortion due to distortion of an optical system,
When arbitrary coordinates of the input image are (x ′, y ′), coordinates after distortion correction are (x, y), coordinates of the image center are (x ₀ , y ₀ ), and n is a positive integer,

An image characterized in that the coordinates of the input image are obtained using a coordinate transformation expression represented by the following, and the pixel value at the coordinates of the input image is obtained by linear approximation with the pixel values of the pixels existing in the vicinity of the coordinates. Distortion correction method. The image distortion correction method according to claim 1, wherein at least one of a coefficient a _ijkl and an image center coordinate x ₀ , y _{0 in} the coordinate conversion formula is changed. A plurality of combinations of the coefficient a _ijkl and the coordinates of the center of the image in the coordinate conversion formula are prepared in advance, and coordinate conversion is performed by selecting any one combination from the plurality of combinations. The image distortion correction method according to claim 2, wherein: The coefficient a _ijkl and at least one of the image center coordinates x ₀ and y ₀ are changed according to the horizontal magnification of the corrected image with respect to the input image before correction. 3. The image distortion correction method according to 3. Claims, characterized in that to change at least one of complement factor according to the position movement amount in the horizontal direction of the image after correction to the positive input image before a _ijkl and coordinate x ₀ of the image center, y ₀ The image distortion correction method according to any one of 2 to 4 . Altering at least one of coefficients a _ijkl and coordinate x ₀ of the image center, y ₀ in accordance with the ratio between the horizontal magnification of the top and bottom of the image after correction for compensation prior to input image The image distortion correction method according to claim 2, wherein the image distortion correction method is one of the following. When the coordinates of the input image obtained by the coordinate conversion formula do not exist within the range of the actual input image, the pixel value of the coordinate is present at the closest position on the same horizontal line as the coordinate and the actual input image distortion correction method of claim 1, wherein employing the pixel values of the coordinates exist within the range of the image. Image distortion according to claim 1, characterized by using a value representing an achromatic color when not within range of the coordinate conversion formula in the obtained input image coordinates actual input image as a pixel value of the coordinates Correction method. The image distortion correction method according to claim 8, wherein a value representing black is adopted as the value representing the achromatic color . When the coordinates of the input image obtained by the coordinate conversion formula do not exist within the range of the actual input image, the image is enlarged by adjusting the coefficient a _ijkl so that the coordinates are outside the display range. The image distortion correction method according to claim 1 . When the coordinates of the input image obtained by the coordinate conversion formula do not exist within the range of the actual input image, the coefficient a _ijkl is adjusted so that the coordinates are outside the display range by enlarging the input image. image distortion correction method of claim 1, wherein.

Images