JP4479527B2 - Image processing method, image processing apparatus, image processing program, and electronic camera - Google Patents

Description

  The present invention relates to a technique for correcting a change in saturation of an image caused by signal processing such as gradation conversion.

When shooting with a digital still camera or the like, the background may be bright and the main subject such as a person may appear dark. In addition, when shooting on a clear day, the shadow of the building appears darker than it looks in the captured image.
This is because human vision has a very wide dynamic range and the sensitivity is adjusted for each viewing area, while the digital still camera has a narrow dynamic range, and the appropriate exposure is adjusted over the entire screen. Because it is.

Tone correction is performed on such an image to make a more preferable image, and the brightness is adjusted for each area so that both bright and dark areas have appropriate brightness. There is a technique.
For example, in the method described in Patent Document 1 below, a luminance signal is generated from an RGB signal, the luminance signal is converted into a predetermined gradation characteristic, and the ratio of the luminance signal before and after the gradation conversion is converted into an RGB signal. By multiplying, a tone-corrected RGB signal is obtained. In this method, the ratios of RGB color components are kept constant before and after correction. In this case, at the brightly corrected portion, the difference between the color components is enlarged, and the color of the image becomes darker, resulting in an image with enhanced saturation.

As a technique for suppressing such saturation change, a method described in Patent Document 2 is known. In the method described in Patent Document 2, an RGB signal having a constant ratio of color components before and after correction and an RGB signal having a constant difference between color components are respectively obtained, and the luminance signal value before gamma correction is obtained. Accordingly, weighted synthesis of both signals is performed. By this weighted synthesis, it is possible to suppress an increase in the difference between the color components and suppress the above-described change in saturation.
JP-A-4-150171 JP-A-5-76036

  An object of the present invention is to provide an image processing technique for correcting a change in saturation associated with image signal processing by processing different from the method described in Patent Document 2.

[1]
The image processing method of the present invention includes the following steps.
A luminance signal is acquired or generated from a color image signal, signal processing predetermined for the luminance signal is performed, and the luminance signal after the signal processing is divided by the luminance signal before the signal processing is performed Find the change ratio.
Based on “each color component included in the color image signal” and “maximum value of each color component”, a correction coefficient is obtained in units of pixels by the following equation.
[(Each color component) / (maximum value of each color component)] ^P
However, P is -1 <= P <0.
By multiplying the correction coefficient change ratio, calculated an adjusted rate of change is performed in units of pixels a signal processing color image signals based on the adjusted pre change ratio.

[2]
Another image processing method of the present invention includes the following steps.
A luminance signal is obtained from a color image signal, and a luminance signal V in the HSV color space before signal processing is performed after signal processing is performed on the luminance signal after predetermined signal processing is performed. V′−V which is a difference from “is obtained.
A correction coefficient is obtained for each pixel by the following formula using “each color component included in the color image signal” and “maximum value of each color component”.
[(Each color component) / (maximum value of each color component)] ^{P + 1}
By adding the color components in terms of the difference of the luminance signal is multiplied by the correction coefficient, it performs signal processing of the color image signal in units of pixels.

[3]
The image processing apparatus of the present invention includes the following change generation unit, correction coefficient generation unit, and signal processing unit.
The change generation unit acquires or generates a luminance signal from the color image signal, performs predetermined signal processing on the luminance signal, and converts the luminance signal after the signal processing to the luminance signal before performing the signal processing. Find the change ratio consisting of the value divided by .
The correction coefficient generation unit obtains a correction coefficient in units of pixels using the following equation based on “each color component included in the color image signal” and “maximum value of each color component”.
[(Each color component) / (maximum value of each color component)] ^P
However, P is -1 <= P <0.
The signal processing unit, by multiplying the correction coefficient change ratio, calculated an adjusted rate of change is performed in units of pixels signal processing color image signals on the basis of the adjusted variation ratio.

[4]
Another image processing apparatus of the present invention includes the following change generation unit, correction coefficient generation unit, and signal processing unit.
The change generation unit acquires or generates a luminance signal from the color image signal, performs signal processing with the luminance signal V in the HSV color space after performing signal processing predetermined on the luminance signal and before performing signal processing. V′−V which is a difference from the luminance signal V ′ after the calculation is obtained.
The correction coefficient generation unit obtains a correction coefficient in units of pixels from the following formula using “each color component included in the color image signal” and “maximum value of each color component”.
[(Each color component) / (maximum value of each color component)] ^{P + 1}
The signal processing unit multiplies the luminance signal difference by a correction coefficient and adds the result to the color component, thereby performing signal processing of the color image signal in units of pixels .

[5]
An image processing program according to the present invention is an image processing program for causing a computer to function as the change generation unit, the correction coefficient generation unit, and the signal processing unit according to claim 3 or claim 4.

[6]
An electronic camera according to the present invention includes the image processing device according to the above [3] or [4], and an imaging unit that images a subject and generates a color image signal. Furthermore, this electronic camera performs signal processing on the color components included in the color image signal generated by the imaging unit by the image processing apparatus.

  In the present invention, a correction coefficient corresponding to (color component of image signal) / (maximum value of color component) is generated for each color component, and this correction coefficient is multiplied by the image signal, whereby saturation associated with gradation conversion is generated. Suppress degree change.

<< First Embodiment >>
FIG. 1 is a block diagram illustrating a configuration of the image processing apparatus 11.
In FIG. 1, a color image signal is input to the image processing apparatus 11. The change generation unit 12 extracts a luminance signal from the color image signal, and obtains a change that occurs in the luminance signal when predetermined signal processing is performed. The change generation unit 12 detects a change ratio and a change occurring in the color component (change equivalent) from the change before and after the signal processing of the luminance signal.

On the other hand, the correction coefficient generation unit 13 generates a correction coefficient corresponding to (color component such as RGB) / (maximum value among color components such as RGB) for the color image signal. This correction coefficient is a coefficient value correlated with the saturation.
The signal processing unit 14 performs signal processing of the color image signal based on the output (change ratio or change equivalent) of the change generation unit 12 and the correction coefficient of the correction coefficient generation unit.

FIG. 2 is a diagram illustrating a data flow of the image processing apparatus 11 according to the first embodiment.
Hereinafter, the operation of the image processing apparatus 11 will be described with reference to FIG.
First, an RGB signal is input (S1 in FIG. 2). The change generation unit 12 adds a predetermined coefficient to the RGB signal and generates a luminance signal in units of pixels (S2 in FIG. 2). The change generation unit 12 performs signal processing such as gamma correction on the luminance signal to generate a corrected luminance signal (S3 in FIG. 2). Note that a well-known Retinex process may be performed on the luminance signal to generate a corrected luminance signal.

The luminance signal handled by the change generation unit 12 may be V in the HSV color space, Y in the YCbCr space, L in the Lab color space, or the like. In general, this luminance signal is preferably a signal reflecting the brightness and color density of an image.
Next, the change generation unit 12 obtains the ratio between the luminance signal V before correction and the luminance signal V ′ after correction, and sets it as a gain K (S4 in FIG. 2).

  On the other hand, the correction coefficient generator 13 obtains max (R, G, B) in units of pixels from the input RGB signal (S6 in FIG. 2). Next, the correction coefficient generation unit 13 obtains the RGB color components and the ratio of max (R, G, B) in units of color components and in units of pixels (S7 in FIG. 2). This ratio is a value reflecting the density (saturation) of each color component in each pixel. The correction coefficient generator 13 obtains a correction coefficient by substituting this ratio into the function f. If a monotone decreasing function or a function similar thereto is used for the function f, it is possible to suppress a change in saturation after signal processing by suppressing an adjusted change ratio, which will be described later, at a higher saturation point. On the contrary, if a monotonically increasing function or a function similar thereto is used for this function f, it is possible to emphasize the saturation change after signal processing at a higher saturation point.

  The signal processing unit 14 calculates the adjusted change ratio in units of color components and in units of pixels by multiplying the above-described change ratio and the correction coefficient. The signal processing unit 14 multiplies the RGB color component by this adjusted change ratio for each pixel to perform signal processing in consideration of the saturation change on the input RGB signal.

By such processing, the RGB signal shown in the following equation is output (S8 in FIG. 2).
As K = V ′ / V,
R ′ = K · f (R / max (R, G, B)) · R
G ′ = K · f (G / max (R, G, B)) · G
B ′ = K · f (R / max (R, G, B)) · B

For example, if the correction coefficient f (x) = x ^ p (where ^ is a power operator),
R ′ = K · (R / max (R, G, B)) ^ p · R
G ′ = K · (G / max (R, G, B)) ^ p · G
B ′ = K · (R / max (R, G, B)) ^ p · B
It becomes. In this case, −1 ≦ p <0 is preferable.

Here, in particular, f (1) = 1 is set as the function f for obtaining the correction coefficient. In this case, since the correction coefficient value is always set to “1” for the maximum color component of the RGB signal, signal processing without correction is performed for the maximum color component.
On the other hand, for the color components other than the maximum, if p <0, the correction coefficient becomes larger than 1, so that the color component is largely corrected, and saturation enhancement accompanying gradation conversion is suppressed. The degree of this suppression can be adjusted by the value of p.

If −1 ≦ p, a reversal phenomenon in which a color component other than the maximum exceeds the maximum color component does not occur.
The correction coefficient f (x) is not limited to a power function, and is preferably a monotonously decreasing function. For example, the degree of saturation reduction can be controlled with a high degree of freedom by using an appropriate function or function table.

Further, when the brightness V of the HSV space is used as the luminance signal, V = max (R, G, B).
R ′ = V ′ · f (R / V) · R
G ′ = V ′ · f (G / V) · G
B ′ = V ′ · f (B / V) · B
It becomes.

In this case, if f (x) = x ^ p,
R ′ = V ′ · (R / V) ^ (p + 1)
G ′ = V ′ · (G / V) ^ (p + 1)
B ′ = V ′ · (B / V) ^ (p + 1)
It becomes. In addition, according to the subjective evaluation experiment of image quality, about p = −0.4 is preferable.
Next, another embodiment will be described.

<< Second Embodiment >>
Since the image processing apparatus according to the second embodiment has the same configuration as that of the first embodiment (FIG. 1), description of the configuration is omitted.
FIG. 3 is a diagram illustrating a data flow of the image processing apparatus 11 according to the second embodiment.
Since the processing of S5, S8, and S9 is different from the first embodiment, the processing will be described below.

In the above-described processing, saturation correction is also applied to a portion where the luminance K does not change with the gain K = 1.
Therefore, the saturation adjustment amount is changed according to the gain (S5 in FIG. 3), and saturation correction is not performed in the vicinity of the gain K = 1. Therefore, by using an adjustment function g () in which y = 0 and g (y) = 0,
R '= R + g (K-1) .f (R / max (R, G, B)). R
G '= G + g (K-1) .f (G / max (R, G, B)). G
B '= B + g (K-1) .f (B / max (R, G, B)). B
(S8, S9 in FIG. 3).

As such an adjustment function g (), a monotonically increasing function is preferable. For example, if g (y) = y,
R '= R + (K-1) .f (R / max (R, G, B)). R
G '= G + (K-1) .f (G / max (R, G, B)). G
B '= B + (K-1) .f (B / max (R, G, B)). B
It becomes.

Further, assuming that the correction coefficient f (x) = x ^ p and the brightness V of the HSV space as the luminance signal,
R '= R + (V'-V). (R / V) ^ (p + 1)
G ′ = G + (V′−V) · (G / V) ^ (p + 1)
B '= B + (V'-V). (B / V) ^ (p + 1)
It becomes. In addition, according to the subjective evaluation experiment of image quality, it is preferable that p + 1 = 0.6.
Next, another embodiment will be described.

<< Third Embodiment >>
The third embodiment is an embodiment of an electronic camera.
FIG. 4 is a block diagram showing the configuration of the present embodiment.
In FIG. 4, a photographing lens 12 is attached to the electronic camera 11. In the image space of the photographic lens 12, the light receiving surface of the image sensor 13 is arranged. The operation of the image sensor 13 is controlled by the output pulse of the timing generator 22b.

An image generated by the image sensor 13 is temporarily stored in the buffer memory 17 via the A / D converter 15 and the signal processor 16.
The buffer memory 17 is connected to the bus 18. An image processing unit 19, a card interface 20, a microprocessor 22, a compression / decompression unit 23, and an image display unit 24 are connected to the bus 18. Of these, the card interface 20 reads / writes data from / to the removable memory card 21. Further, a user operation signal is input to the microprocessor 22 from the switch group 22 a of the electronic camera 11. Further, the image display unit 24 displays an image on a monitor screen 25 provided on the back surface of the electronic camera 11.
The electronic camera 11 having such a configuration executes the signal processing of the first and second embodiments in the microprocessor 22 and the image processing unit 19.
Such signal processing may be performed on the image data at the time of imaging, or may be performed later on the image data stored in the memory card 21.

<< Additional items of embodiment >>
Note that the gradation conversion processing of the above-described embodiment can also be realized on a computer. In this case, the image processing program may be used to cause the computer to function as the change generation unit 12, the correction coefficient generation unit 13, and the signal processing unit 14 in the first and second embodiments.

  In the above-described embodiment, the RAW data may be subjected to signal processing using the input image signal as RAW data (color array data such as Bayer array). In this case, since luminance signals and color components are not arranged for each pixel, it is not possible to generate a change ratio or a correction coefficient for each pixel.

  In this case, processing for generating a luminance signal and a color component for each pixel is performed on the RAW data using the color components of the peripheral pixels, and a change ratio and a correction coefficient are obtained for the obtained luminance signal and color components. Also good.

  Further, the RAW data may be divided into pixel block units necessary for generating the luminance signal and all the color components, and the change ratio and the correction coefficient may be determined from the luminance signal and color components obtained for each pixel block.

  Note that signal processing may be performed directly on the color array data in the RAW data. Further, it may be stored in image processing information in RAW data so that signal processing can be performed later.

  Further, in the image processing server or image album server on the Internet or the like, the image processing method of the above-described embodiment may be provided as a service for image data transmitted from a user. Note that a print server that acquires image data via a recording medium or a communication medium and provides a print service may implement the image processing method of the above-described embodiment as pre-print processing.

  In the above-described embodiment, the RGB color components have been described. However, the present invention is not limited to this. In general, color components of an arbitrary color system can be used.

  As described above, the present invention is a technique that can be used for image processing and the like.

2 is a block diagram illustrating a configuration of an image processing apparatus 11. FIG. It is a figure which shows the data flow of the image processing apparatus 11 in 1st Embodiment. It is a figure which shows the data flow of the image processing apparatus 11 in 2nd Embodiment. It is a figure which shows the structure of an electronic camera.

Explanation of symbols

V luminance signal 11 image processing device 12 change generation unit 13 correction coefficient generation unit 14 signal processing unit

Claims (6)

A luminance signal is obtained or generated from a color image signal, predetermined signal processing is performed on the luminance signal, and the luminance signal after the signal processing is divided by the luminance signal before the signal processing is performed Find the change ratio consisting of
A correction coefficient is obtained in units of pixels by the following formula using “each color component included in the color image signal” and “maximum value of each color component”.
[(Each color component) / (maximum value of each color component)] ^P
However, P is -1 <= P <0.
An image processing method, wherein an adjusted change ratio is obtained by multiplying the change ratio by the correction coefficient, and signal processing of the color image signal is performed on a pixel basis based on the adjusted change ratio. Luminance signal V in the HSV color space after acquiring or generating a luminance signal from a color image signal and performing signal processing predetermined on the luminance signal and before performing the signal processing and luminance after performing the signal processing V′−V which is a difference from the signal V ′ is obtained,
A correction coefficient is obtained in units of pixels by the following formula using "each color component included in the color image signal" and "maximum value of each color component"
[(Each color component) / (maximum value of each color component)] ^{P + 1}
An image processing method characterized in that signal processing of the color image signal is performed in units of pixels by multiplying the difference between the luminance signals by the correction coefficient and adding the result to the color component. A luminance signal is obtained or generated from a color image signal, predetermined signal processing is performed on the luminance signal, and the luminance signal after the signal processing is divided by the luminance signal before the signal processing is performed A change generation unit for obtaining a change ratio composed of the obtained values ;
A correction coefficient generation unit that obtains a correction coefficient in units of pixels according to the following equation based on “each color component included in the color image signal” and “maximum value of each color component”;
[(Each color component) / (maximum value of each color component)] ^P
However, P is -1 <= P <0.
A signal processing unit that obtains an adjusted change ratio by multiplying the change ratio by the correction coefficient, and performs signal processing of the color image signal in units of pixels based on the adjusted change ratio; An image processing apparatus. Luminance signal V in the HSV color space after acquiring or generating a luminance signal from a color image signal and performing signal processing predetermined on the luminance signal and before performing the signal processing and luminance after performing the signal processing A change generator for obtaining V′−V which is a difference from the signal V ′;
A correction coefficient generation unit that obtains a correction coefficient in units of pixels according to the following equation using “each color component included in the color image signal” and “maximum value of each color component”;
[(Each color component) / (maximum value of each color component)] ^{P + 1}
An image processing apparatus comprising: a signal processing unit that performs signal processing of the color image signal in units of pixels by multiplying the difference between the luminance signals by the correction coefficient and adding the result to the color component. .   An image processing program for causing a computer to function as the change generation unit, the correction coefficient generation unit, and the signal processing unit according to claim 3 or 4. An image processing apparatus according to claim 3 or 4,
An imaging unit that images a subject and generates a color image signal;
An electronic camera, wherein a color component included in the color image signal generated by the imaging unit is signal-processed by the image processing device.

Images