JPS63182961A - System for detecting distortion of color image - Google Patents

Abstract

PURPOSE:To reproduce beautiful image display with fidelity, by finding the aberration of a partial image by a correlative operation based on the similarity of the partial image, and correcting geometric distortion from aberration distribution at the representative point of a document image. CONSTITUTION:A red color image is selected as an image set as reference, and the partial image which becomes the reference is segmented from the red color image at a reference part image extracting part 105. Also, a green color image and a blue color image which become the target to be detected of distortion are selected by a selector 104, and their partial images are segmented being shifted at a targeted partial image extracting part 106. A correlative operation part 107 finds the similarity at every picture element of a reference partial image and a targeted partial image, A correction quantity at every picture element is calculated at a correction quantity calculating part 110 by using an interpolation method proportional to a distance, and the correction of a targeted image is performed at an image correction part 110. In such a way, the image in which the targeted image is corrected is stored by selecting a selector 111 at an after-correction green color image storage part 112 or an aftercorrection blue color image storage part 113.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to distortion detection and correction of color images, and in particular, a geometric method based on correlation of partial images suitable for detecting distortion of color document images containing characters. This paper relates to a distortion detection method.

[Conventional technology]

Traditionally, color images taken with a color television camera have been developed by Nagata and Teruo Kawai (Hitachi).

NHK): “Television Camera Measurement” Journal of the Television Society, Vol. 38. No, 8. pp, 762-77
0 (1982), a method for measuring distortion has been established, and its correction has been carried out. However, since high resolution is required as a document image input device, a television camera is not used, and an image scanner using a line sensor is mainly used. Regarding the correction of color document images input from this image scanner, please refer to Hidekazu Sekizawa, Naofumi Yamamoto, and Haruko Yogami (Toshiba Research Institute) ``Color variation correction of color scanner light source''
1st Annual National Conference of the Institute of Electronics and Communication Engineers 1286 (Showa 6)
Hiroo Wakabayashi, Hisao Ota (Nippon Telegraph and Telephone Public Corporation Yokosuka Telecommunications Laboratory): “Study of color correction for color contact sensors” (March 1, 2013) As discussed at the 1985 IEICE General Conference 1328 (March 1985), methods related to color correction have been proposed, such as one that corrects the emission wavelengths of three-color light sources.

In addition, for correction of monochrome document images, see Naoki Kobayashi and Minoru Kanzaki (NTT Telecommunications Laboratories): “Distortion correction method for simple shadow image input method” 1985 IEICE General Conference 1314 (March 1985) (Mon.), Minoru Kanzaki, Naoki Kobayashi (NTT Telecommunications Laboratories): ``Distortion correction method for simple dual-image input method (Part 2)'' 1985 IEICE General Conference 1285 (March 1985) )
As discussed in , correction of distortion in hand-scanner handheld document input devices has been proposed. However, no consideration has been given to detecting geometric distortion of color document images.

[Problem that the invention seeks to solve]

The above-mentioned conventional technology does not take into account the detection and correction method of geometric distortion related to the pixel arrangement of color images, and when this color image is displayed, color shift is noticeable in the characters in the color document image. , there was a problem that it was unsightly.

Furthermore, since the arrangement of pixels in the three primary color images is shifted, there is a problem in that the color image cannot be processed normally.

An object of the present invention is to detect and correct geometric distortion between images of three primary colors in a color document image.

[Means for solving problems]

The above object is achieved by determining the positional deviation of the partial images by correlation calculation based on the similarity of the partial images, and correcting the geometric distortion from the positional deviation distribution at the representative points of the document image.

[Effect]

In a color document image, characters, two figures, and a photograph are mixed, but these parts have a correlation among the three primary color images. In particular, black text on a white background is the most commonly seen partial image in document images.

The partial image including this character portion is originally a monochrome image, and the red component, green component, and blue component have approximately the same value, so there is an extremely significant correlation in the three primary color images.

Therefore, among the images of the three primary colors, partial images of each color including the black character portion on the white background are extracted from the same position, and the relative positional shift of these partial images is determined. This positional shift corresponds to geometric distortion. To find the relative positional shift, it is sufficient to extract a partial image of a certain size from the target image and find the similarity for each pixel while shifting the position. The similarity is
Detection is performed by a correlation calculation that calculates the difference in gray values between pixels, and among them, the position with the highest similarity is determined.

〔Example〕

An embodiment of the present invention will be described below with reference to FIG. A surface image of a color document is collected by a color image input device 100 such as a color image scanner, and images of three primary colors, that is, a red image is stored in a red image storage section 101, a green image is stored in a green image storage section 102, and a blue image is stored in a green image storage section 102. are respectively stored in the blue image storage unit 103. In order to detect geometric distortion, for example, a red image is selected as a reference image, and a reference partial image is extracted from this red image in the reference partial image extraction unit 105. One of the green and blue images is selected in advance by the selector 104 via the control unit 120, and similarly, in the target partial image extraction unit 106,
The partial images are cut out while being shifted. The correlation calculation unit 107 calculates pixel-by-pixel similarity between the standard partial image that is the reference and the target partial image that is the object of distortion detection.

An error in pixel data, which will be described later, is used as an index of this similarity. Then, the minimum value detection unit 108 determines the minimum value of the mismatch error, that is, determines the shifted position of the target partial image that most closely matches the standard partial image. Based on this shifted position, the correction amount calculation unit 109 calculates the correction amount for each pixel using an interpolation method proportional to the distance, which will be described later.The image correction unit 110 corrects the target image. An image obtained by correcting the target image is stored in the edge color image storage section 112 or the corrected blue image storage section 113 according to the selection of the selector 111. Note that the original image address control section 114 and the corrected image address control section 115 are as follows:
In the above series of processing.

Generates address signals and control signals such as initialization.

In detecting geometric distortion based on the correlation of partial images, first, one of the three primary color images is selected as a standard image. Here, the red image is selected as the standard image. In addition,
Since geometric distortion is expressed by the relative positional deviation between the three primary color images, the remaining color images (green image and blue image) may be selected as the standard image. A partial image containing black characters on a white background is cut out from the image as a standard partial image.

Then, the cutting position of the standard partial image is set as the standard position. In this embodiment, it is assumed that the standard position is given in advance. Now, a case will be described in which geometric distortion of a blue image is detected using a standard red image as a reference.

In the blue image, a partial image is cut out from the target image while shifting the cutout position around the corresponding standard position, and a correlation calculation with the aforementioned standard partial image is performed to find the most matching shifted position. If the determined position deviates from the standard position, geometric distortion exists. Therefore, the displacement between the standard position and the shifted position is determined, and this is taken as the amount of geometric distortion.

Regarding the green image, a correlation calculation is similarly performed between the partial image cut out from the green image and the standard partial image of the red image, and the displacement from the standard position is determined for the most matching shifted position. In this way.

Geometric distortions of the blue and green images in a certain partial area are determined based on the red image.

To find the distribution of geometric distortion in the entire color document image, standard partial images can be cut out from a number of different positions in the document image, and the geometric distortion can be calculated based on the correlation of each partial image as described above. .

Here, we will show a correlation calculation method for determining the similarity of partial images. Now, as shown in Figure 2, the target document image 2
The coordinate system 203 of 00 is (xty), and the gray value of one pixel is expressed as f(x, y), and the standard partial image 20
2 (referred to as a standard partial image) is g (io J).

The size of this partial image is assumed to be ■ in the horizontal direction and J in the vertical direction. In the correlation calculation, a partial image 201 (referred to as a target partial image) having the same size as the standard partial image is cut out from the target document image while shifting its position.
The starting point 204 of the shifted position at this time is (xotyo). In addition, the coordinates for expressing the amount of shift in the X direction and the ≦S-1.
It is assumed that the longitudinal force direction is 0≦t≦T−1. In order to find the similarity of the cut out target partial image with respect to the standard partial image, here we will calculate the sum of the differences between pixels (degree of mismatch). Within the shift range 205 (lateral force direction 0≦S≦
S-1. The minimum mismatch degree err in the vertical force direction (0≦t≦T-1) is expressed by the following equation.

The shift position when having this minimum mismatch degree err is C3
'lt'), this shifted position (s'
+t') corresponds to the cutout position from the target image of the partial image that most matches the standard image g (io j).

The process of correlation calculation of partial images is shown in FIG. 3, Step 3
00, the position for cutting out a standard partial image is set. Next, in step 301, a standard partial image is cut out from the reference color image.

Let this partial image be g He J), step 302
and 303, the vertical shift coordinate t is 0≦t! T-1
It will be gradually increased until. Furthermore, the horizontal shift coordinates are also increased sequentially until 0≦S≦S−1. Then, in step 304, a partial image is cut out from the color image to be subjected to distortion detection. This cut out partial image is f(x
, )+s+i, Yo+t+j). In addition,(
xo+yo) is the starting point of the shift, as described above. In step 305, an accumulation variable sum for counting the degree of mismatch is initialized, that is, sum←0.

Then, steps 308 and 309 are performed while sequentially updating the horizontal coordinates of one partial image and then the vertical coordinates in steps 306 and 307. In step 308, the difference between the pixel values of the two partial images is calculated. And step 3
In step 09, the differences are accumulated, and the differences are sequentially added to the accumulation variable sum. When steps 306 and 307 are completed, in step 310, the minimum value err of the sum of differences sum is determined.
is determined, and further, in step 311, the minimum shift coordinate is registered.

FIG. 4 shows the configuration of the correlation calculation section 107. The difference between the pixel data sent out from the standard partial image extraction unit 105 and the pixel data sent out from the target partial image extraction unit 106 is
The subtraction unit 400 calculates the difference, and the absolute value of the difference result is calculated by the absolute value converting unit 401.

The adder 402 sequentially adds the absolute values of the differences, stores them in the cumulative error register 403, and sends them to the minimum value detector 108.

FIG. 5 shows the configuration of the minimum value detection section 108.

The accumulated error sent from the correlation calculation section 107 is compared in magnitude with the contents of the minimum value register 502 in the magnitude comparison section 500, and if the accumulated error concerned is smaller,
This accumulated error is stored in the minimum value register 502 via the selector 501. Similarly, if the cumulative error is smaller, the shift coordinates sent from the original image address control unit 114 are stored in the shift coordinate (horizontal direction) storage register 503. The shifted coordinates (vertical direction) are stored in the storage register 504, and these shifted coordinates are sent to the correction amount calculation unit 109.

FIG. 6 shows a procedure for detecting geometric shape distortion using correlation of partial images. First, in step 600, the position of the partial image to be cut out is determined and registered as a standard position. It is necessary to set the position of this partial image so that it includes a black character part on a white background.

As shown in step 601, the following process is performed until all of the many standard positions are exhausted. That is, in step 602, a partial image corresponding to the standard position is cut out from the red image, and this is taken as the standard partial image. Then, in step 603, a correlation calculation is performed on the green image with the previous red standard partial image while shifting the cutting position around the part corresponding to the standard position, and in step 604, the coordinates of the most matching point are determined. As shown in step 605, the difference between these coordinates and the coordinates of the standard position becomes the amount of geometric distortion of the green image when the red image is taken as a reference. Furthermore, similarly for the blue image, in step 606, a correlation calculation is performed with the previous red standard partial image while shifting the cropping position around the part corresponding to the standard position,
In step 607, the coordinates of the most matching point are determined.

As shown in step 608, the difference between these coordinates and the coordinates of the standard position is the amount of geometric distortion in the blue image.

The amount of geometric distortion obtained in this way is saved for each corresponding standard position. The above-described series of processes is then repeated until all of the many standard positions are exhausted.

The image is divided into rectangular regions in advance, and the vertical and horizontal pixel correction amounts are set for each divided region.

Here, the reason why a typical correction amount is set for each divided area is to reduce the amount of processing for calculating the correction amount. This correction amount is calculated based on the amount of displacement with respect to the standard image representing the aforementioned geometric distortion. Note that if a black character portion with a white background does not exist in a divided area, the amount of correction is set in advance by a simple interpolation method using a difference in coordinate values based on the correction amount in an adjacent divided area. Further, different correction amounts are given to each color image of the color image.

The position of each pixel is corrected according to this correction amount, but since the correction amount differs depending on the divided area, with a simple method, the correction amount may differ greatly at the boundary of the divided area, and in this case, the image will not be smooth. The disconnections will be noticeable. Therefore, interpolation is applied to the correction amount so that the correction amount changes continuously little by little even near the boundary of one divided area. A method of interpolating the amount of correction will be described below. Now, as shown in FIG. 7, in the target image 700, the horizontal number of the divided area 701 is m, and the vertical number is n (1≦+n
:aM, 1≦n≦N), and each divided area is represented by (m, n). The correction f702 in the divided area (m, n) is Δx
(m, n), Δy(m.

n). For the sake of simplicity, it is assumed here that this correction amount is assigned to the center of the divided area. Four adjacent divided areas (m, n), (m+1.n), (m, n+1)
, (m+1.n+1) center point 703.704,705
, 706, the correction amount δx (a, b), δy (a, b) for the pixel of interest 707 in the rectangular area surrounding
It is assumed that it is proportional to the difference between the coordinates of the pixel of interest and the center coordinates of the divided area, and interpolation is performed using the following equation.

δx (a, b) = Δx (m, n) (1-a) (1-b)
+Δx(m+l,n)a(1-b)+Δx(m,n+1
)(1-a)b+Δx(m+1.n+1)abδy(a
,b)=Δy(m,n)(1-a)(1-b)+Δy(
I11+1. n) a(1-b)+Δy(IIl, n+1
) (1-a)b+Δy(+++1.n+1)ab However, the variable a is the area (m, n) when the horizontal length of the rectangular area surrounding the center point of the four adjacent divided areas is 1.
It is the ratio of the horizontal length from the center point to the pixel of interest.

Similarly, the variable S is the area (m
, n) from the center point to the pixel of interest in the vertical direction. Note that, for example, to calculate the correction amount for a pixel of interest in a divided area at the edge, such as a divided area in the column m = 1 or m = M, a virtual divided area is set outside the edge, Similarly, calculate using the above formula. At this time, it is assumed that the amount of correction for the virtual area is the same as the amount of correction for the adjacent divided area.

The above-described interpolation method provides, for each pixel position, a correction amount for moving pixels of other color images with respect to the red pixel. The geometric distortion of one pixel is corrected based on this correction amount.

In this correction procedure, as shown in FIG. 8, first, in steps 800 and 801, horizontal and vertical coordinates of a two-dimensional array for storing the corrected green image are sequentially generated along the scan. Corresponding to these coordinates, the amount of correction for the green image is stored, so according to this coordinate generation, the amount of correction is read out in step 802, and based on the amount of correction read out in step 803, the original In this process of extracting pixel data from the green image, the coordinates of the original image are obtained by adding the readout correction amounts to the horizontal and vertical coordinates that are generated sequentially, and the coordinates are calculated in step 804 according to these coordinate values. Extract pixel data from the original image. This series of processing is performed on the entire green image. Similarly, for the blue image, step 8
At 06° 807, 808, 809, pixel data is read out from the original blue image according to the amount of correction for the blue image and stored in a two-dimensional array that stores the corrected image.

[Effects of the Invention] According to the present invention, it is possible to detect geometric distortion in character parts that are commonly included in color document images, so color shift in character parts can be corrected and beautiful, faithful screen display can be achieved. It has the effect of reproducing.

[Brief explanation of the drawing]

Fig. 1 is a configuration diagram of an embodiment of the present invention, Fig. 2 is an explanatory diagram of a coordinate system for correlation calculation, Fig. 3 is a diagram showing a correlation calculation process of a partial image, and Fig. 4 is a correlation calculation unit. FIG. 5 is a block diagram of the minimum value detection section, a calculation section, 108...minimum value detection section, 105... standard partial image extraction section, 106...
Target partial image extraction unit, 205...shift range, 202
...Standard partial image, 308...Calculation of pixel value difference,
403... Cumulative error register, 500... Size comparison section 6 7th eye way U - 4 main eye A [7' 昃昃Z目弾＊ゆう橡f(χ,ρ Shoulder Is 11 minutes ν1(Mu, 19 ↓ Member elementary school I direct spear ni dankocho eTl 2nd country 7th ward

Claims (1)

[Claims] 1. In an image distortion detection method having means for inputting a color image of three primary colors and means for processing the inputted image, one of the three primary color images is selected as a reference image, and a text portion is extracted from the primary color image. A first means of cutting out the included partial image as a partial image at a reference position, and selecting a primary color image other than the above as a target image for detecting distortion, and shifting the position around the reference position, a second means for cutting out a partial image; a third means for determining a correlation between the two partial images and a shift position for cutting out a partial image that minimizes the degree of mismatch; and a fourth means for calculating a distortion correction amount.