JP2001186533A - Image processing unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing unit that corrects a chromatic aberration of an input image signal with a simple and inexpensive configuration and corrects the chromatic aberration through interpolation processing of a missing color signal. SOLUTION: The image processing unit is provided with an image information storage section 31, that stores a color signal of image information obtained from an image pickup element 1, a chromatic aberration information storage section 33 that stores information of a chromatic aberration produced in an image pickup optical system 2, chromatic aberration correction information generating sections 34 and 35 that read chromatic aberration information from the chromatic aberration information storage section 33, on the basis of the pixel position of the color signal of the image information and the information with respect to the image pickup optical system to generate chromatic aberration correction information, an interpolation coefficient calculation section 36, that calculates an interpolation coefficient on the basis of the pixel position of the color signal of the image information and the chromatic aberration correction information and the interpolation processing section 36, that uses the color signal of the image information and the interpolation coefficient to conduct interpolation processing with respect to the color signal of the image information and the color signal of the missing image information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for correcting magnification chromatic aberration of an optical system generated in an image pickup apparatus by signal processing, and more particularly, to correcting magnification chromatic aberration when a missing color signal is interpolated by signal processing. And an image processing apparatus.

[0002]

2. Description of the Related Art One of the aberrations generated in an optical system used in an image pickup apparatus and the like is chromatic aberration of magnification. Chromatic aberration is caused by a difference in the refractive index depending on the wavelength. In particular, chromatic aberration of magnification is a phenomenon in which the position (magnification) of an image on an image plane is shifted for each color because the focal length differs depending on the color. In the present specification, lateral chromatic aberration is hereinafter simply referred to as chromatic aberration. This chromatic aberration can be corrected to some extent by the material of the lens or the like, but in this case, the cost of manufacturing the lens is increased due to the expensive material.

Therefore, several methods for correcting chromatic aberration by signal processing have conventionally been proposed. For example, in Japanese Patent Publication No. 6-5959, correction data for performing an interpolation process from an error between a reference color and another color and a nearby color signal is stored in advance, and the stored data is read out during an imaging operation. There is disclosed a technique in which correction is performed by interpolating data of another color with neighboring data.

Japanese Unexamined Patent Publication No. Hei 8-205181 discloses that chromatic aberration information (shift information of other colors with respect to a reference color) corresponding to a lens state such as a zoom position and a screen position is stored in a memory in advance, and the corresponding chromatic aberration information is stored. Is disclosed in which image data in which chromatic aberration with respect to a reference color has been corrected by using is interpolated. Further, in this publication, as a device for reducing the memory of chromatic aberration information, chromatic aberration is determined according to the distance from the center of the screen, or chromatic aberration information is partially stored, and a portion where chromatic aberration information does not exist is interpolated. It is also disclosed that the chromatic aberration information is obtained by the following.

[0005]

However, in the above-described conventional example, chromatic aberration correction is basically performed at a stage where all the colors of each pixel are present. An image pickup device having three image pickup devices (three-plate image pickup device) is required, and the image pickup device itself becomes large and the cost increases due to the large number of image pickup devices.

On the other hand, when a single-chip or two-chip image sensor is used (the number of image sensors is small for color signals necessary for image reproduction), the image pickup device itself is compared with an image pickup device using a three-chip image sensor. Can be made smaller and less expensive, but in this case, it is necessary to interpolate the missing color signal for each pixel in order to reproduce it as an image. However, the color signal to be interpolated in this case is affected by the chromatic aberration described above.
An image of a portion different from the image of the reference pixel will be generated.

This will be described in more detail with reference to FIG. FIG. 10 shows a state of interpolation of a missing color signal in the single-chip image sensor. Here, a G signal exists at the pixel of interest "G {i, j}", and it is necessary to interpolate the R signal and the B signal to reproduce the image. With respect to the G signal image formed at this position, ideally, the interpolated R signal and B signal are formed at the same position. However, actually, due to the chromatic aberration of the optical system, the image formation positions of the R signal and the B signal regarding the G signal image “R {i, j}”,
"B {i, j}" is imaged at a position different from the position of the G signal as shown in FIG.

Here, when the interpolation processing of the missing color signal of the R signal and the B signal is performed without performing the chromatic aberration correction, the missing color signal of the image which is displaced from the image of the referenced G signal is obtained. Interpolation is performed, and the color may be reproduced as a color different from the actual color of the subject. In the case of FIG.
Even if an attempt is made to interpolate the R signal and B signal of the image at {i, j} "as they are, the R
Since the signal and the B signal are not "G {i, j}", appropriate interpolation processing cannot be performed as a result.

From the above, it is necessary to correct the chromatic aberration when performing the interpolation processing of the missing color signal. Here, as a method of correcting chromatic aberration by signal processing, a look-up table (LUT) in which position correction information indicating a position shift correction amount of a color signal is stored for each pixel based on chromatic aberration information of an imaging optical system is stored in a memory in advance. In the missing color signal interpolation process, using the position of the pixel as an address in L of the memory.
A method of reading the position correction information from the UT, correcting the position based on the position correction information, and performing an interpolation process is conceivable.

However, in this case, since the LUT of the position correction information is referred to each pixel before the interpolation processing, an increase in calculation time at the time of reference and an increase in the circuit scale of a memory and the like are caused, and cost is reduced. It was difficult to correct chromatic aberration with a cheap imaging device.

Accordingly, an object of the present invention made in view of the above point is to provide an image processing apparatus capable of correcting chromatic aberration of an input image signal with a simple and inexpensive configuration and correcting chromatic aberration by interpolation of a missing color signal. It is something to offer.

[0012]

According to a first aspect of the present invention, there is provided an image processing apparatus, comprising: an image pickup device that detects missing image information for each pixel based on a color signal of the image information obtained from an image pickup device via an image pickup optical system; In an image processing apparatus for restoring a color signal by interpolation processing, an image information storage unit that stores a color signal of image information obtained from the imaging device, and a chromatic aberration that stores information of chromatic aberration generated in the imaging optical system as chromatic aberration information An information storage unit that reads chromatic aberration information from the chromatic aberration information storage unit and corrects chromatic aberration based on a pixel position of a color signal of image information stored in the image information storage unit and information on the imaging optical system. Correction information generating unit for generating chromatic aberration correction information for the image information, a pixel position of a color signal of image information stored in the image information storage unit, and the chromatic aberration correction information An interpolation coefficient calculation unit that calculates an interpolation coefficient for an interpolation process based on the chromatic aberration correction information generated by the component unit; a color signal of image information stored in the image information storage unit; and the interpolation coefficient calculation unit And an interpolation processing unit that performs an interpolation process on the color signal of the image information stored in the image information storage unit and the color signal of the missing image information using the interpolation coefficient calculated in It is a feature.

According to the present invention, the chromatic aberration correction information is generated from the chromatic aberration information based on the pixel position of the color signal of the image information and the state of the imaging optical system. Chromatic aberration correction information suitable for each pixel can be obtained, and an interpolation coefficient is generated from the chromatic aberration correction information according to the pixel position of the color signal of the image information, and the color signal of the input image information and the missing Interpolation processing is performed on the color signal of the image information that is present, so interpolation coefficients suitable for chromatic aberration correction are obtained for each pixel, and the chromatic aberration of the input color signal and the missing color signal can be corrected for each pixel by interpolation processing As a result, it is possible to generate a high-quality image in which chromatic aberration is corrected for all color signals for all pixels by signal processing while minimizing the cost of the imaging optical system and the imaging device. To become.

According to a second aspect of the present invention, there is provided an image processing apparatus for restoring a missing color signal of image information for each pixel by interpolation based on a color signal of image information obtained from an image sensor through an imaging optical system. An image information storage unit that stores a color signal of image information obtained from the image sensor; and a chromatic aberration that generates information of chromatic aberration generated in the imaging optical system based on the color signal of image information obtained from the image sensor. An information generation unit; a chromatic aberration information storage unit that stores chromatic aberration information generated by the chromatic aberration information generation unit as chromatic aberration information; a pixel position of a color signal of image information stored in the image information storage unit; A chromatic aberration correction information generator that reads chromatic aberration information from the chromatic aberration information storage unit based on the information about the optical system and generates chromatic aberration correction information for correcting chromatic aberration; And an interpolation unit that calculates an interpolation coefficient for an interpolation process based on a pixel position of a color signal of image information stored in the image information storage unit and chromatic aberration correction information generated by the chromatic aberration correction information generation unit. A coefficient calculation unit, and a color of the image information stored in the image information storage unit using a color signal of the image information stored in the image information storage unit and the interpolation coefficient calculated by the interpolation coefficient calculation unit. And an interpolation processing unit that performs interpolation processing on the signal and the color signal of the missing image information.

According to the second aspect of the present invention, since chromatic aberration information is generated from the color signal of the image information, it is possible to obtain chromatic aberration information suitable for an actual imaging optical system for obtaining the image information, and to obtain the chromatic aberration information. Is used to generate chromatic aberration correction information based on the pixel position of the color signal of the image information and the state of the imaging optical system, so that chromatic aberration correction information suitable for each pixel is obtained according to a change in the state of the imaging optical system. It becomes possible. Further, since an interpolation coefficient is generated from the chromatic aberration correction information according to the pixel position of the color signal of the image information, an interpolation coefficient suitable for chromatic aberration correction is obtained for each pixel, and an image input based on the interpolation coefficient is obtained. Since the interpolation process is performed on the color signal of the information and the color signal of the missing image information, it is possible to correct the chromatic aberration of the input color signal and the missing color signal on a pixel-by-pixel basis. It is possible to generate a high-quality image in which chromatic aberration is corrected for all color signals for all pixels by signal processing while minimizing the cost of the image sensor and the image sensor.

The invention according to claim 3 is the invention according to claim 1 or 2
Wherein the chromatic aberration information storage unit stores, as the chromatic aberration information, a characteristic that depends on a pixel position of a color signal of the image information and information on the imaging optical system. is there.

According to the third aspect of the present invention, it is not necessary to store the chromatic aberration information for each pixel, so that the memory capacity of the chromatic aberration information storage unit can be reduced, and the chromatic aberration information storage unit and its surroundings can be reduced. Can be simplified.

The invention according to claim 4 is the invention according to claim 1 or 2
Wherein the chromatic aberration correction information generating unit is configured to determine the chromatic aberration information read from the chromatic aberration information storage unit based on a pixel position of a color signal of the image information and information on the imaging optical system. And the chromatic aberration correction information is generated.

According to the fourth aspect of the present invention, since it is not necessary to hold information for generating chromatic aberration correction information for each pixel, the memory capacity can be reduced, and the chromatic aberration correction information generation unit And the peripheral circuit configuration can be simplified.

According to a fifth aspect of the present invention, in the image processing apparatus according to any one of the first to fourth aspects, the information on the imaging optical system is a focal length of the imaging optical system. Is what you do.

According to the fifth aspect of the invention, even if the focal length changes due to a zoom operation or the like at the time of photographing, it is possible to obtain chromatic aberration information and chromatic aberration correction information corresponding thereto and appropriately correct chromatic aberration. As a result, it is possible to always perform appropriate chromatic aberration correction even in various photographing states.

According to a sixth aspect of the present invention, in the image processing apparatus according to any one of the first to fourth aspects, the information on the imaging optical system is obtained by dividing an image forming plane of the imaging optical system into a plurality of blocks. The information is characterized by being divided and defined.

According to the sixth aspect of the present invention, even for an image pickup optical system in which chromatic aberration occurs in a complicated manner, it is divided into a plurality of blocks and partially corresponding chromatic aberration information and chromatic aberration correction information are obtained to partially obtain the chromatic aberration information. Since chromatic aberration can be appropriately corrected, chromatic aberration can be appropriately corrected on the entire image plane, that is, on the entire image.

According to a seventh aspect of the present invention, in the image processing apparatus according to the second aspect, the chromatic aberration information generating section stores pattern information for correcting chromatic aberration as pattern reference information. A pattern image information storage unit that stores, as pattern image information, a color signal of image information obtained by capturing the pattern information, and a pattern that performs position correction on the pattern image information stored in the pattern image information storage unit An image information position correcting unit, a pattern information color shift comparing unit that generates color shift information by comparing the pattern image information position-corrected by the pattern image information position correcting unit with the pattern reference information, and A chromatic aberration information calculation unit that calculates the chromatic aberration information based on the color shift information generated by the information color shift comparison unit. It is characterized in.

According to the seventh aspect of the present invention, chromatic aberration information is generated by preparing pattern information for correcting chromatic aberration and comparing image information obtained by photographing the obtained pattern information with reference information stored in advance. Even if an error occurs in the design specifications of the imaging optical system, chromatic aberration information suitable for each device can be obtained, and as a result, chromatic aberration correction can be appropriately performed in any device.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image processing apparatus according to the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a basic configuration of an electronic still camera according to a first embodiment of the present invention. This electronic still camera uses a single-chip color CCD imaging device 1 having an electronic shutter function.
A subject image is formed on the CCD imaging device 1 via a lens group 2 and an aperture / shutter mechanism 3, and the subject image is photoelectrically converted and output as an image signal. The image signal of the subject image from the CCD image sensor 1 is subjected to noise components removed by a correlated double sampling circuit or the like (not shown).
The signal is amplified by the amplifier 4, further converted into a digital signal by the A / D converter 5 and supplied to the camera signal processing circuit 6, where it is processed as image data.

The output of the A / D converter 5 is AF, A
Also supplied to the E and AWB detection circuit 7, where the AF for automatically controlling the focus prior to the original photographing is performed.
AF detection processing for extracting information, AE detection processing for extracting AE information for automatically controlling exposure, and AWB detection processing for extracting AWB information for automatically setting white balance are performed. This AF, AE, AW
The AF information, AE information, and AWB information from the B detection circuit 7 are supplied to the lens group 2, the aperture / shutter mechanism 3, and the camera signal processing circuit 6 via the CPU 8, respectively.

The camera signal processing circuit 6 and the CPU 8
The bus line 9 is connected to a DRAM 11 used as a working memory when performing color processing of image data and the like via a memory controller 10 and a camera signal processing circuit 6. A compression circuit (JPEG) 12 for compressing image data from the camera is connected. The bus line 9 includes a compression circuit 12
A memory card I / F 14 is connected to record the image data compressed in step 3 on the memory card 13, and the image data recorded on the memory card 13 is read out and displayed, and the shooting state is displayed. For this purpose, a liquid crystal display (LCD) 16 is connected via a display circuit 15. Further, the memory card 13 is connected to the bus line 9.
Is connected to a personal computer (PC) 17 for transferring image data recorded in the personal computer (PC) 17.

The CPU 8 includes AF, AE, AWB
A strobe device 19 controlled based on AE information from the detection circuit 7 and an input key 20 for setting various photographing modes and driving a trigger switch are connected to each other. Note that the CCD image pickup device 1 is driven by a timing pulse from a timing generator (TG) 21 under the control of the CPU 8.

Further, the lens group 2 is driven by a motor 22 in response to a zoom operation from an input key 20, and its focal length information is taken in by the CPU 8 via the motor 22 and is transmitted along with AF information, AE information and AWB information. It is supplied to the camera signal processing circuit 6.

FIG. 2 shows a lens group 2 used in this embodiment.
1 shows an example of the configuration. The lens group 2 has a characteristic of rotational symmetry of an optical axis with respect to aberration and the like by a zoom lens, and can be set to a telephoto (telephoto) or wide-angle (wide) state by driving the motor 22 to change the focal length. It has a configuration.

FIG. 3 is a block diagram showing an example of the internal configuration of the camera signal processing circuit 6 in the present embodiment.
The camera signal processing circuit 6 includes a frame memory 31, a white balance setting unit 32, an imaging optical system information memory 3,
3, a chromatic aberration correction information characteristic calculation unit 34, a chromatic aberration correction information calculation unit 35, a chromatic aberration correction / missing color interpolation processing unit 36, a color enhancement matrix conversion unit 37, and a γ correction unit 38. In advance, information on the imaging optical system including chromatic aberration information is stored in advance.

In the camera signal processing circuit 6, an image signal output as a digital signal from the A / D converter 5 is stored in a frame memory 31, and white balance setting is performed on the image signal stored in the frame memory 31. Part 3
2, the AWB information and AE output from the CPU 8
The white balance is adjusted using the information and the like, and the white balance adjusted image signal is supplied to the chromatic aberration correction information calculation unit 35 and the chromatic aberration correction / missing color interpolation processing unit 36, respectively.

On the other hand, the chromatic aberration correction information characteristic calculating section 34 converts information about the imaging optical system such as chromatic aberration information stored in advance in the imaging optical system information memory 33 and focal length information and AF information output from the CPU 8. A characteristic for generating chromatic aberration correction information expressed as a function depending on an image plane, that is, a position of a pixel on the CCD image sensor 1, is calculated based on the calculated chromatic aberration correction information. It is supplied to the calculation unit 35.

The chromatic aberration correction information calculating section 35 is based on the image signal subjected to white balance adjustment by the white balance setting section 32 and the characteristic of the chromatic aberration correction information calculated by the chromatic aberration correction information characteristic calculating section 34, and is based on each pixel. The chromatic aberration correction information is calculated from the characteristics of the chromatic aberration correction information with respect to the position of the reference pixel with reference to the image signal, and the calculated chromatic aberration correction information is supplied to the chromatic aberration correction / missing color interpolation processing unit. In the present embodiment, it is assumed that the chromatic aberration correction information indicates the positional shift of the image in the reference pixel for each color as the coordinate value of the image plane at the sub-pixel level.

In the chromatic aberration correction / missing color interpolation processing unit 36,
Based on the image signal white-balanced adjusted by the white balance setting unit 32 and the chromatic aberration correction information calculated by the chromatic aberration correction information calculation unit 35, chromatic aberration correction processing for the image signal input as described later, An interpolation process for correcting chromatic aberration is performed on the color signal being processed.

The image signal which has been subjected to the chromatic aberration correction and the missing color interpolation processing by the chromatic aberration correction / missing color interpolation processing section 36 is subjected to color enhancement processing by a color enhancement matrix conversion section 37, and then subjected to γ correction by a γ correction section 38. The signal is transmitted to the bus line 9.

FIG. 4 is a flowchart showing the flow of processing in the chromatic aberration correction / missing color interpolation processing section 36 described above.
The image signal obtained by photographing with the electronic still camera shown in FIG. 1 has a chromatic aberration in itself, that is, a different position shift for each color in each pixel, and a portion shifted from a position to be originally referred to as an image. Therefore, missing color signals cannot be properly interpolated unless the chromatic aberration is corrected.

The chromatic aberration correction / missing color interpolation processing unit 3
In step S6, first, in step S1, pixel values, image plane coordinate values, and chromatic aberration correction information of the corresponding color are read for pixels near a reference pixel position having a color signal to be subjected to chromatic aberration correction and missing color interpolation. However, in the chromatic aberration correction of the reference pixel, information of the reference pixel itself is also read. Note that the number of neighboring pixels required here can be, for example, about the same as that of general interpolation that is usually performed.

Next, in step S2, the read image plane coordinate values of each pixel are corrected based on the chromatic aberration correction information.
This is chromatic aberration correction for each pixel, and as a result, the position of the image corresponding to the pixel value of each pixel is an appropriate position.

FIG. 5A shows an example. Here, the reference pixel p0 {i, j} is set to a G signal (white point).
And 4 adjacent to the diagonal of p0 {i, j} as neighboring pixels
Pixels are used. Here, the reference pixel p0 {i, j}
The chromatic aberration correction information of G at {GFx (i, j), G
Fy (i, j)}, the position where the reference pixel p0 {i, j} is originally imaged is shifted by the chromatic aberration correction information due to the chromatic aberration correction in step S2, and p0 in FIG.
{i + GFx (i, j), j + GFy (i, j)}. Similarly, the chromatic aberration correction also shifts the original imaging position of the neighboring pixels by the amount corresponding to the chromatic aberration correction information.

After that, in step S3, a weighting coefficient (interpolation coefficient) used for the interpolation processing is calculated based on the distance between the position of each pixel corrected in chromatic aberration in step S2 and the reference pixel position. Here, the calculated interpolation coefficient takes into account the correction of chromatic aberration. For each pixel whose characteristic has been corrected for chromatic aberration, when the distance from the reference pixel position is small (close), the interpolation coefficient is increased. When the distance from the pixel position is large (far), the interpolation coefficient is reduced.

FIG. 5B shows an example of this, and shows a state after the chromatic aberration has been corrected from FIG. 5A.
In FIG. 5B, when the pixel P {i, j} at the reference position is interpolated by the five chromatic aberration correction pixels P0 to P4 located in the vicinity, the pixel P {i, j} and the pixel Pn (n = 0,
Assuming that the distance to (1, 2, 3, 4) is Dn, the interpolation coefficient Wn corresponding to the pixel Pn is as shown in equation (1).

[0045]

Wn = (S−Dn) / (4 × S) (1) where S = ΣDn (n = 0, 1, 2, 3, 4)

As is apparent from equation (1), the interpolation coefficient W
n is determined based on the ratio of the distance Dn to the total sum S of the distances. That is, the distance Dn
Is smaller, the interpolation coefficient Wn is larger, so that the calculation ratio of the pixel value in the interpolation is larger. If the distance Dn is larger, the interpolation coefficient Wn is smaller, so that the calculation ratio of the pixel value in the interpolation is smaller. It is clear that the sum of the interpolation coefficients Wn is 1. Further, the expression (1) is generalized, and the interpolation coefficient Wn when the interpolation process is performed using N chromatic aberration correction pixels located in the vicinity is as shown in the expression (2).

[0047]

Wn = (S−Dn) / {(N−1) × S} (2) where S =｝ Dn (n = 0, 1,..., N−1)

Next, in step S4, the pixel value of the corresponding color signal at the reference pixel position is calculated by interpolation using the calculated interpolation coefficient and the corresponding pixel value of each pixel subjected to chromatic aberration correction. . Here, assuming that the pixel value of the N chromatic aberration correction pixels Pn corresponding to the N interpolation coefficients Wn in the equation (2) is PVn, the pixel P at the reference position
The interpolated pixel value PV {i, j} of {i, j} is as shown in equation (3).

[0049]

## EQU3 ## PV {i, j} = Σ (PVn × Wn) (3) where n = 0, 1,..., N−1

In steps S1 to S4 described above, the pixel value at the reference position is calculated by interpolation by reflecting the positional relationship of the chromatic aberration corrected pixel value in the interpolation coefficient.
It is possible to obtain an appropriate pixel value in which chromatic aberration is corrected for the reference position. Further, the hearing aid method for correcting the chromatic aberration described with reference to FIG. 4 can be realized by the same method for both the input image signal and the missing color signal, so that the chromatic aberration correction of the input signal and the missing color signal There is no need to separate the interpolation and the processing, and the processing system itself can be simply configured.

As described above, according to the present embodiment, the chromatic aberration correction information is calculated as a function depending on the position of the pixel based on the chromatic aberration information of the imaging optical system. Thus, a large-capacity memory is not required, and the processing circuit scale can be reduced. Also,
Even if the state of the imaging optical system changes due to zooming or the like, chromatic aberration correction information corresponding thereto is obtained, so that appropriate chromatic aberration correction information can always be calculated even in various situations such as wide angle and telephoto. Further, regardless of the input color signal and the missing color signal, the interpolation processing for correcting the chromatic aberration can be performed by the same processing for each pixel, so that the processing circuit scale can be reduced and the processing speed can be increased. As a result, a color image in which chromatic aberration is appropriately corrected can be reproduced.

In the second embodiment of the present invention, in the electronic still camera shown in FIG. 1, the motor 22 is omitted, and the lens group 2 is constituted by an eccentric free-form surface lens. Other basic configurations are the same as those in FIG.

FIG. 6 shows a lens group 2 used in the present embodiment.
1 shows an example of the configuration. This lens group 2 is a non-rotationally symmetric decentered free-form surface lens that can obtain a very bright and high-resolution image signal, can be reduced in size and weight,
Etc.

FIG. 7 is a block diagram showing an example of the internal configuration of the camera signal processing circuit 6 shown in FIG. 1 in the present embodiment. The camera signal processing circuit 6 includes a frame memory 41, a white balance setting unit 42, a first switch 43, a chromatic aberration information generation unit 44, a chromatic aberration information memory 45,
A second switch 46, an imaging optical system information memory 47, a chromatic aberration correction information characteristic calculating unit 48, a chromatic aberration correction information calculating unit 49,
It has a chromatic aberration correction / missing color interpolation processing section 50, a color enhancement matrix conversion section 51, and a γ correction section 52. Here, the frame memory 41, the white balance setting unit 4
2. The chromatic aberration correction / missing color interpolation processing unit 50, the color enhancement matrix conversion unit 51, and the γ correction unit 52 are each configured as shown in FIG.
Since the functions are the same as those of the frame memory 31, the white balance setting unit 32, the chromatic aberration correction / missing color interpolation processing unit 36, the color enhancement matrix conversion unit 37, and the γ correction unit 38 shown in FIG. The internal processing in the chromatic aberration correction / missing color hearing aid processing unit 50 is the same as that in FIG.

In this embodiment, a reference pattern for generating chromatic aberration information is prepared in advance, and the chromatic aberration information is stored in a chromatic aberration information memory 45 by photographing the reference pattern with an electronic still camera. The chromatic aberration correction information is calculated based on the imaging optical system information stored in the system information memory 47 in advance.

For this purpose, first, the CPU 8 controls the first switch 43 and the second switch 46 based on the photographing mode signal of the general photographing mode or the reference pattern photographing mode selected by the state of the input key 20 of FIG.

Here, when the photographing mode signal indicates the reference pattern photographing mode, the first switch 43 is driven to transmit the image signal after white balance adjustment to the chromatic aberration information generating section 44, and the second switch 46 is driven. Is driven to the open state so that the information from the chromatic aberration information memory 45 is not transmitted to the chromatic aberration correction information characteristic calculation unit 48. Accordingly, in the reference pattern photographing mode, the image signal of the photographed reference pattern is supplied to the chromatic aberration information generation unit 44, and chromatic aberration information is generated by a method described later, and the generated chromatic aberration information is stored in the chromatic aberration information memory 45. The information is stored and used for chromatic aberration correction of an image signal shot in the general shooting mode.

On the other hand, when the photographing mode indicates the general photographing mode, the first switch 43 sends the image signal after white balance adjustment to the chromatic aberration correction information calculating section 49 and the chromatic aberration correction / missing color interpolation processing section 50. The second switch 46 drives the chromatic aberration correction information characteristic calculation unit 4
8 is driven to the closed state so that information from the chromatic aberration information memory 45 is transmitted. Thus, in the general shooting mode,
The chromatic aberration correction information characteristic calculation unit 48 includes the chromatic aberration information memory 4
The chromatic aberration information in 5 is read via the second switch 46 and the imaging optical system information in the imaging optical system information memory 47 is read.

Here, the imaging optical system information stored in advance in the imaging optical system information memory 47 is information defined by dividing the entire image plane into a plurality of partial blocks as shown in FIG. Is information that determines to which block the pixel belongs in accordance with the position of.

The chromatic aberration correction information characteristic calculation section 48 calculates a characteristic for generating chromatic aberration correction information based on the chromatic aberration information for each block of the imaging optical system information. As described above, calculating the characteristics for individually generating the chromatic aberration correction information corresponding to each block requires a non-rotationally symmetric shape like a decentered free-form surface lens, and a chromatic aberration partially different and complicated. This is a very effective method when an imaging optical system is generated. The characteristic for generating the chromatic aberration correction information is expressed as a function depending on the position of the pixel on the CCD image sensor 1, as in the first embodiment.

The chromatic aberration correction information calculating section 49 reads the image signal after white balance adjustment and the characteristics of the chromatic aberration correction information calculated by the chromatic aberration correction information characteristic calculating section 48, respectively, and calculates chromatic aberration correction information. The processing in the chromatic aberration correction information calculating section 49 is basically the same as the processing in the chromatic aberration correction information calculating section 35 in the first embodiment shown in FIG. 3, but in the present embodiment, the image plane is changed. Since the characteristics of the chromatic aberration correction information are different for each divided block, the characteristics of the chromatic aberration correction information for the block corresponding to the position of the reference pixel are selected, and based on the characteristics of the selected chromatic aberration correction information, the image position shift in the reference pixel is performed. Is calculated as coordinate values of the image plane. Subsequent processing is the same as in the first embodiment.

FIG. 9 shows the chromatic aberration information generator 4 shown in FIG.
FIG. 4 is a block diagram illustrating an example of the configuration of the inside of FIG. The chromatic aberration information generation unit 44 stores the reference pattern image signal memory 6
1, a reference pattern reference signal memory 62, an image signal position correction unit 63, an image signal / reference signal color shift comparison unit 64, and a chromatic aberration information calculation unit 65.

In the present embodiment, the chromatic aberration information of the image pickup optical system based on the optical design values and the like is not stored in advance as in the first embodiment, but based on an image signal obtained by photographing a reference pattern. Generate chromatic aberration information. For this reason,
In generating the chromatic aberration information, first, the image signal of the captured reference pattern transmitted via the first switch 43 is read and stored in the reference pattern image signal memory 61. Here, the image signal of the reference pattern stored in the reference pattern image signal memory 61 is a single-plate image signal after white balance adjustment, and a missing color signal exists for each pixel.

Separately from this, reference information of a reference pattern having no chromatic aberration is stored in the reference pattern reference signal memory 62 in advance. The reference information of the reference pattern stored in the reference pattern reference signal memory 62 is handled as single-plate reference information by a single-color filter similar to that of the imaging apparatus. The reference pattern used here is preferably a pattern in which a plurality of colors are mixed, such as a color chart, from the viewpoint of generating chromatic aberration information, and reference information without chromatic aberration is easily obtained. As described above, it is preferable to use artificial data such as CG in terms of handling.

Next, in the image signal position correcting section 63,
After performing processing such as alignment of the reference pattern with respect to the image signal, for example, alignment of the center of the reference pattern with the center of the entire image, the image signal / reference signal color misregistration comparison unit 64 compares the image signal with the image signal at the same pixel position. The color misregistration is calculated by comparing with the signal.

The method of calculating the color shift in the color shift comparing section 64 between the image signal and the reference signal also depends on the reference pattern.
For example, in which direction and how much the color of the reference pixel is deviated from the pixel being referred to and the pixel value in the vicinity thereof, it is calculated as color deviation information by performing this in the entire image.

Next, the chromatic aberration information calculating section 65 calculates chromatic aberration information based on the color shift information from the color shift comparing section 64 between the image signal and the reference signal. Here, the chromatic aberration information is not calculated for each pixel, and for example, the average of the color shift information of the corresponding color in the block is set as the chromatic aberration information of the block in a form corresponding to the partial block defined in the imaging optical system information. Chromatic aberration characteristics that are differently calculated for each block are calculated. The chromatic aberration information generated by the chromatic aberration information calculation unit 65 is transmitted to the chromatic aberration information memory 45.

In the above configuration, it is possible to update the chromatic aberration information by photographing the reference pattern by switching the input key 20 of the image pickup apparatus. It is also possible to store in advance by the method described above, and to generally prevent the chromatic aberration information from being updated.

As described above, according to the present embodiment, the entire image plane is divided into a plurality of blocks, and chromatic aberration correction information as a function depending on the position of a pixel is calculated in a state corresponding to each block. Since the interpolation processing is performed, it is possible to calculate chromatic aberration correction information appropriate for the reference pixel even for a complicated imaging optical system having partially different chromatic aberration characteristics. That is, appropriate chromatic aberration correction can be performed for any part, and as a result, a chromatic aberration correction effect can be obtained for the entire image. Further, since the chromatic aberration information is generated by photographing the reference pattern, for example, even if an error occurs due to manufacturing or the like with respect to the design specification of the imaging optical system,
Optimal chromatic aberration information can be obtained in each device, and as a result, an image in which chromatic aberration has been corrected can be reproduced. Further, as in the case of the first embodiment, the interpolation processing for correcting the chromatic aberration can be performed by the same processing for each pixel regardless of the input color signal and the missing color signal. The scale can be reduced and the processing speed can be increased, and as a result, a color image in which chromatic aberration is appropriately corrected can be reproduced.

[0070]

As is apparent from the above description, according to the present invention, the chromatic aberration of the input image signal can be corrected with a simple and inexpensive configuration, and the chromatic aberration can be corrected by the interpolation processing of the missing color signal. A processing device can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of an electronic still camera according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration of an example of a lens group illustrated in FIG. 1;

FIG. 3 is a block diagram showing a configuration of an example of the inside of the camera signal processing circuit shown in FIG. 1;

FIG. 4 is a flowchart showing a flow of processing in a chromatic aberration correction / missing color interpolation processing unit shown in FIG. 3;

FIG. 5 is a diagram showing an example of chromatic aberration correction and missing color interpolation.

FIG. 6 is a diagram illustrating a configuration of an example of a lens group used in a second embodiment of the present invention.

FIG. 7 is a block diagram illustrating a configuration of an example of the inside of the camera signal processing circuit illustrated in FIG. 1 according to the second embodiment;

FIG. 8 is a diagram for explaining imaging optical system information stored in advance in an imaging optical system information memory shown in FIG. 7;

9 is a block diagram illustrating a configuration example of an inside of the chromatic aberration information generation unit illustrated in FIG. 7;

FIG. 10 is a diagram illustrating a state of interpolation of a missing color signal in a single-chip image sensor when chromatic aberration correction is not performed.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Color CCD imaging element 2 Lens group 3 Aperture / shutter mechanism 4 Amplifier 5 A / D converter 6 Camera signal processing circuit 7 AF, AE, AWB detection circuit 8 CPU 9 Bus line 10 Memory controller 11 DRAM 12 Compression circuit (JPEG) 13 Memory Card 14 Memory Card I / F 15 Display Circuit 16 Liquid Crystal Display (LCD) 17 Personal Computer (PC) 18 PCI / F 19 Strobe Device 20 Input Key 21 Timing Generator (TG) 22 Motor 31 Frame Memory 32 White Balance Setting Unit 33 imaging optical system information memory 34 chromatic aberration correction information characteristic calculation unit 35 chromatic aberration correction information calculation unit 36 chromatic aberration correction / missing color interpolation processing unit 37 color enhancement matrix conversion unit 38 γ correction unit 41 frame memory 42 white balance setting unit 43 1 switch 44 chromatic aberration information generation unit 45 chromatic aberration information memory 46 second switch 47 imaging optical system information memory 48 chromatic aberration correction information characteristic calculation unit 49 chromatic aberration correction information calculation unit 50 chromatic aberration correction / missing color interpolation processing unit 51 color enhancement matrix conversion unit 52 γ correction unit 61 reference pattern image signal memory 62 reference pattern reference signal memory 63 image signal position correction unit 64 color shift comparison unit between image signal and reference signal 65 chromatic aberration information calculation unit

Claims (7)

[Claims] 1. An image processing apparatus for restoring a color signal of missing image information for each pixel by interpolation based on a color signal of image information obtained from an image sensor via an image pickup optical system, comprising: An image information storage unit that stores a color signal of the obtained image information; a chromatic aberration information storage unit that stores chromatic aberration information generated in the imaging optical system as chromatic aberration information; and image information stored in the image information storage unit. A chromatic aberration correction information generating unit that reads chromatic aberration information from the chromatic aberration information storage unit and generates chromatic aberration correction information for correcting chromatic aberration based on the pixel position of the color signal and information on the imaging optical system; Interpolation processing based on the pixel position of the color signal of the image information stored in the information storage unit and the chromatic aberration correction information generated by the chromatic aberration correction information generation unit An interpolation coefficient calculating unit for calculating an interpolation coefficient for the image information storage unit using the color signal of the image information stored in the image information storage unit and the interpolation coefficient calculated by the interpolation coefficient calculation unit. An image processing apparatus comprising: an interpolation processing unit that performs an interpolation process on a color signal of stored image information and a color signal of the missing image information. 2. An image processing apparatus for restoring a missing color signal of image information for each pixel by interpolation processing based on a color signal of image information obtained from an imaging device via an imaging optical system, comprising: An image information storage unit that stores a color signal of the obtained image information; a chromatic aberration information generation unit that generates information of chromatic aberration generated in the imaging optical system based on the color signal of the image information obtained from the imaging device; A chromatic aberration information storage unit that stores information of the chromatic aberration generated by the chromatic aberration information generation unit as chromatic aberration information; and a pixel position of a color signal of image information stored in the image information storage unit and information on the imaging optical system. A chromatic aberration correction information generation unit that reads chromatic aberration information from the chromatic aberration information storage unit based on the chromatic aberration information and generates chromatic aberration correction information for correcting chromatic aberration; An interpolation coefficient calculation unit that calculates an interpolation coefficient for an interpolation process based on a pixel position of a color signal of image information stored in a storage unit and chromatic aberration correction information generated by the chromatic aberration correction information generation unit; Using the color signal of the image information stored in the image information storage unit and the interpolation coefficient calculated by the interpolation coefficient calculation unit, the color signal of the image information stored in the image information storage unit and the missing An interpolation processing unit that performs an interpolation process on a color signal of existing image information. 3. The chromatic aberration information storage unit stores a characteristic depending on a pixel position of a color signal of the image information and information on the imaging optical system as the chromatic aberration information. An image processing apparatus according to claim 1. 4. The chromatic aberration correction information generation unit converts the chromatic aberration information read from the chromatic aberration information storage unit into a characteristic dependent on a pixel position of a color signal of the image information and information on the imaging optical system. The image processing apparatus according to claim 1, wherein the chromatic aberration correction information is generated. 5. The information on the imaging optical system is a focal length of the imaging optical system.
5. The image processing apparatus according to claim 4. 6. The information according to claim 1, wherein the information on the imaging optical system is information that defines an image forming plane of the imaging optical system by dividing the imaging surface into a plurality of blocks. The image processing apparatus according to any one of the preceding claims. 7. A pattern reference information storage unit for storing pattern information for correcting chromatic aberration as pattern reference information, a chromatic aberration information generation unit, and a color signal of image information obtained by capturing the pattern information as pattern image information. A pattern image information storage unit that stores the pattern image information stored in the pattern image information storage unit; a pattern image information position correction unit that performs position correction on the pattern image information stored in the pattern image information storage unit; Comparing the obtained pattern image information and the pattern reference information to generate color shift information; a color shift between pattern information comparing unit; and a color shift information generated by the color shift between pattern information comparing unit based on the color shift information. The image processing apparatus according to claim 2, further comprising: a chromatic aberration information calculation unit configured to calculate chromatic aberration information.