JP2016010013A - Image processing apparatus and method for the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve highly accurate light source conversion processing without requiring a color conversion matrix in each light source.SOLUTION: A white point information acquisition unit 302 acquires white point information of a photographic scene of imaging data. A reverse γ processing unit 301 and a reverse processing unit 304 apply reverse processing of image processing applied to the imaging data by an imaging apparatus to the imaging data. A color value conversion unit 305 converts the imaging data processed by reverse processing into imaging data of a colorimetric color value. A white point conversion unit 307 applies adaptation conversion for converting white points of the imaging data into white points in an output color space to the imaging data of the colorimetric color value. An output value conversion unit 308 converts the imaging data processed by the adaptation conversion into imaging data of the output color space.

Description

  The present invention relates to image data color conversion processing.

  Various methods have been proposed as methods for adjusting the color balance of image data taken by an imaging device such as a digital camera or a digital video camera. Patent Document 1 discloses a process for color-converting image data shot under a shooting light source different from a standard light source into image data under a standard light source by applying a color conversion matrix prepared for each light source and then adjusting the gain ( Hereinafter, the light source conversion process) will be described.

  However, in the above method, it is necessary to prepare a profile including a color conversion matrix for each light source, and in order to cope with various light sources, profiles corresponding to the number of light sources are required. In addition, if the type of light source and the spectral distribution are different at the time of shooting and when the profile is created, there is a problem that the color conversion accuracy deteriorates.

Japanese Patent No. 4767699

  An object of the present invention is to perform a highly accurate light source conversion process without requiring a color conversion matrix for each light source.

  The present invention has the following configuration as one means for achieving the above object.

  In the image processing according to the present invention, white point information of a shooting scene of imaging data is acquired, the imaging process is performed on the imaging data by the imaging device, and the imaging data after the inverse processing is processed. The image data is converted into colorimetric color values, the image data having the colorimetric color values is subjected to white adaptation conversion based on the white point information and the white point of the output color space, and the adaptation conversion is performed. The captured image data is converted into image data in the output color space.

  According to the present invention, highly accurate light source conversion processing can be performed without requiring a color conversion matrix for each light source.

FIG. 3 is a block diagram illustrating a relationship between an imaging apparatus and an image processing apparatus according to an embodiment. 1 is a block diagram illustrating a configuration example of an information processing apparatus that functions as an image processing apparatus. The block diagram which shows the structural example of the light source conversion process of the imaging data in an image processing apparatus. The flowchart explaining a light source conversion process. FIG. 6 is a block diagram illustrating a configuration example of light source conversion processing of imaging data in the image processing apparatus according to the second embodiment. 9 is a flowchart for explaining light source conversion processing according to the second embodiment.

  Hereinafter, an image processing apparatus and an image processing method according to embodiments of the present invention will be described in detail with reference to the drawings. In addition, an Example does not limit this invention concerning a claim, and all the combinations of the structure demonstrated in an Example are not necessarily essential for the solution means of this invention.

[Device configuration]
The relationship between the image pickup apparatus and the image processing apparatus according to the embodiment will be described with reference to the block diagram of FIG. The imaging device 101 is a digital video camera or the like, and includes a recording unit 102 that records imaging data. The recording unit 102 is a recording device such as a hard disk drive (HDD) or a solid state drive (SSD), or a reader / writer of a recording medium such as a memory card.

  The image processing apparatus 103 is connected to the imaging apparatus 101 via an interface 204 including a serial bus such as USB or a wireless network, and can acquire imaging data from the imaging apparatus 101. The recording unit 102 may be an external recording device that can be connected to the imaging device 101 and the image processing device 103 via a serial bus or a wireless network, in addition to being built in the imaging device 101.

  An example of the configuration of an information processing apparatus that functions as the image processing apparatus 103 is shown in the block diagram of FIG. The CPU 201 executes an operating system (OS) or a program stored in the ROM 209 or the storage unit 203 using the main memory 202 such as a RAM as a work memory, and controls a configuration to be described later via the system bus 206. The storage unit 203 is an HDD, an SSD, or the like, and stores programs and various data for realizing a light source conversion process described later.

  Connected to the interface 204 are an operation unit 207 such as a keyboard and a mouse, an imaging device 101, a recording medium 210 such as a memory card and a USB memory, and the like. A monitor 211 is connected to the video card (VC) 205. The CPU 201 displays information indicating a user interface (UI), processing progress, processing results, and the like on the monitor 211.

  For example, the CPU 201 loads an application program (AP) stored in the storage unit 203 or the recording medium 210 into a predetermined area of the main memory 202 in accordance with a user instruction input via the operation unit 207. Then, the AP is executed, and the UI is displayed on the monitor 211 according to the AP.

  Next, the CPU 201 inputs various data stored in the storage unit 203 and the recording medium 210 and imaging data from the imaging device 101 according to the UI operation by the user, and loads them into a predetermined area of the main memory 202. Then, predetermined arithmetic processing is performed on various data loaded into the main memory 202 in accordance with the AP. Then, the CPU 201 displays the calculation processing result on the monitor 205 or stores it in the storage unit 203 or the recording medium 210 in accordance with the UI operation by the user.

  Note that the CPU 201 can also exchange programs, data, and arithmetic processing results with a server device on a wired or wireless network via the interface 204. In addition, a tablet computer can be used as the information processing apparatus. In this case, a touch panel superimposed on the screen of the monitor 211 is the operation unit 207.

[Light source conversion processing]
A block diagram of FIG. 3 shows a configuration example of light source conversion processing of imaging data in the image processing apparatus 103. Note that the processing configuration shown in FIG. 3 is realized by the CPU 201 executing a light source conversion processing program. Details of the processing of each unit will be described later.

  The input unit 300 inputs imaging data from the imaging device 101. When the recording unit 102 exists independently from the imaging device 101, the input unit 300 inputs imaging data from the recording unit 102. The inverse γ processing unit 301 performs inverse gamma processing on the input imaging data. The white point information acquisition unit 302 acquires white point information at the time of imaging.

  The correction amount calculation unit 303 calculates a white balance correction amount (hereinafter, WB correction amount) based on the white point information and the parameters stored in the WB setting storage unit 309. The reverse processing unit 304 performs reverse processing (hereinafter referred to as WB reverse processing) based on the WB correction amount on the imaging data subjected to the white balance processing.

  The color value conversion unit 305 converts, for example, RGB values of the reverse-processed imaging data output from the reverse processing unit 304 into, for example, tristimulus values XYZ. In the following description, it is assumed that the RGB value of the imaging data is converted to the tristimulus value XYZ. However, the conversion destination is not limited to the tristimulus value XYZ, and any colorimetric color value may be used. , Luv value, Jab value, etc.

  The white point correction amount calculation unit 306 calculates a correction amount (hereinafter, white point correction amount) for converting from white at the time of imaging to white in the output color space. The output color space is a destination color space that is a standard color space such as a color space defined by an output device (for example, the monitor 211) to which imaging data specified by the user is output or an sRGB space.

  The white point conversion unit 307 performs white point conversion processing based on the white point correction amount on the tristimulus values XYZ of the imaging data output from the color value conversion unit 305. The output value conversion unit 308 converts the tristimulus values XYZ of the imaging data output from the white point conversion unit 307 into RGB values in the output color space.

  The output unit 310 outputs the imaging data output by the output value conversion unit 308 to an output destination designated by the user. Output destinations include various devices including the imaging device 101, the recording unit 102, the recording medium 210, and the server device connected to the interface 204. Further, the imaging data may be output to the VC 205 and a video or an image may be reproduced by the monitor 211, or the imaging data may be stored in the storage unit 203.

  The light source conversion process will be described with reference to the flowchart of FIG. The CPU 201 starts the light source conversion process shown in FIG. 4 according to the user instruction, and inputs imaging data to be processed by the input unit 300 (S400). The inverse γ processing unit 301 performs gamma processing with characteristics opposite to those during imaging on the input imaging data (S401). That is, the inverse γ processing unit 301 performs an inverse operation on the imaging data with respect to the gamma operation performed inside the imaging apparatus 101 during imaging, and outputs imaging data having a linear value with respect to the luminance of the subject. .

Next, the white point information acquisition unit 302 acquires color temperature information at the time of white balance processing as white point information at the time of imaging from, for example, metadata of the shooting data (S402). The WB correction amount calculation unit 303 calculates a WB correction amount for performing WB reverse processing based on the color temperature information acquired by the white point information acquisition unit 302 (S403). The WB setting storage unit 309 stores, for each color temperature, gain amounts Rgain, Ggain, and Bgain that are multiplied by the RGB value of the imaging data in the white balance processing of the imaging apparatus 101. The WB correction amount calculation unit 303 reads the gain amount corresponding to the color temperature information from the WB setting storage unit 309 and calculates the WB correction amount by the following equation.
k _R = 1 / Rgain (t);
k _G = 1 / Ggain (t);
k _B = 1 / Bgain (t);… (1)
Here, Rgain (t), Ggain (t), and Bgain (t) are gain amounts corresponding to the color temperature t.

The WB correction amount k _R k _G k _B is the reciprocal of the gain multiplied by the RGB value in the white balance processing at the time of shooting, and is equivalent to the white RGB balance amount of the shooting scene.

Next, as shown in the following formula, the inverse processing unit 304 multiplies the imaging data output by the inverse γ processing unit 301 by the WB correction amount k _R k _G k _B to perform the reverse process of the white balance process ( S404).
R ′ = R × k _R ;
G ′ = G × k _G ;
B '= B × k _B ;… (2)

  In addition, although the example in which the reverse processing unit 304 performs the reverse process of the white balance process of the imaging device 101 has been described, not only the white balance process but also other color conversion processing such as matrix calculation is performed inside the imaging device 101. If so, the reverse processing unit 304 performs the reverse processing.

Next, the color value conversion unit 305 converts the R′G′B ′ value of the imaging data output from the inverse processing unit 304 into tristimulus values XYZ by matrix calculation processing of the following equation (S405).
┌ ┐ ┌ ┐
│X│ │R'│
│Y│ = M│G'│… (3)
│Z│ │B'│
└ ┘ └ ┘
┌ ┐
│a11 a12 a13│
M = │a21 a22 a23│… (4)
│a31 a32 a33│
└ ┘

  The matrix M (coefficients a11 to a33) shown in Expression (4) is created in advance by an arbitrary creation method. As a method for creating the matrix M, for example, photographing with a plurality of color patches is performed by the imaging apparatus 101 and colorimetry by a measuring instrument. Then, the R'G'B 'value obtained by performing the WB inverse process on the pixel value RGB of the imaging data is converted into a colorimetric value (tristimulus value XYZ) by Equation (3) (in other words, the conversion accuracy is The coefficients a11-a33 are determined by the method of least squares (to be the maximum). Note that the coefficients a11 to a33 of the matrix M are conversion coefficients for the pixel value R′G′B ′ subjected to the WB inverse process, and are coefficients that do not depend on the light source.

  Next, the white point correction amount calculation unit 306 calculates a correction amount for adaptation conversion for converting the white point of the imaging data into a white point defined in the output color space (S406). In the present embodiment, Bladford transformation is adopted as adaptation transformation.

The white point correction amount calculation unit 306 converts the WB correction amount calculated in step S403 into white tristimulus values XwsYwsZws of the shooting scene using the conversion matrix M (Equation (4)) as the white point information of the shooting scene. (S406a). Although an example in which the WB correction amount is used as the white point information of the shooting scene will be described, white color measurement data measured in the actual shooting scene may be used as the white point information.
┌ ┐ ┌ ┐
│Xws│ │k _R │
│Yws│ = M│k _G │… (5)
│Zws│ │k _B │
└ ┘ └ ┘

Subsequently, the white point correction amount calculation unit 306 converts XwsYwsZws into a response of the LMS cone using the Bladford conversion formula (S406b).
┌ ┐ ┌ ┐
│Ls│ │Xws│
│Ms│ = Mbf│Yws│… (6)
│Ss│ │Zws│
└ ┘ └ ┘
┌ ┐
│ 0.8951 0.2664 -0.1614│
Mbf = │-0.7502 1.71315 0.0367│… (7)
│ 0.0389 -0.0685 1.0296│
└ ┘

Subsequently, the white point correction amount calculation unit 306 converts the tristimulus value XwdYwdZwd of the specified white point in the output color space into a response of the LMS cone by the same calculation as in Equation (6) (S406c).
┌ ┐ ┌ ┐
│Ld│ │Xwd│
│Md│ = Mbf│Ywd│… (8)
│Sd│ │Zwd│
└ ┘ └ ┘

Subsequently, the white point correction amount calculation unit 306 calculates the correction amounts ρ, γ, and β for adaptation conversion from the white LMS cone response value of the shooting scene and the white LMS cone response value of the output color space. (S406d), the process of step S406 is terminated.
ρ = Xwd / Xws;
γ = Ywd / Yws;
β = Zwd / Zws;… (9)

Next, the white point conversion unit 307 performs white adaptation conversion on the XYZ values output from the color value conversion unit 305 according to the following formula to convert the values into X′Y′Z ′ values (S407).
┌ ┐ ┌ ┐ ┌ ┐
│X'│ │ρ 0 0│ │X│
│Y'│ = Mbf ^-1 │ 0 β 0│Mbf│Y│… (10)
│Z'│ │ 0 0 γ│ │Z│
└ ┘ └ ┘ └ ┘

Next, the output value conversion unit 308 converts the X′Y′Z ′ value into an RGB value in the color space (output color space) of the output device (S408). For example, when the output color space is the sRGB space, the X′Y′Z ′ value is converted into the R′G′B ′ value by the following formula.
┌ ┐ ┌ ┐
│R'│ │X'│
│G'│ = Msrgb│Y'│… (11)
│B'│ │Z'│
└ ┘ └ ┘
┌ ┐
│ 3.240970 -1.537383 -0.498611│
Msrgb = │-0.969244 1.875968 0.041555│… (12)
│ 0.055630 -0.203977 1.056972│
└ ┘

Subsequently, the output value conversion unit 308 converts the R′G′B ′ value into an sRGB value by gamma conversion of the following equation.
if (R '≤ 0.0031308)
R = 12.92R ';
else
R = 10.55R ^{1 /} 2.4-0.055;
if (G '≤ 0.0031308)
G = 12.92G ';
else
G = 10.55G ^{1 /} 2.4-0.055;
if (B '≤ 0.0031308)
B = 12.92B ';
else
B = 10.55B ^{1 /} 2.4-0.055;… (13)

  Next, the output unit 310 outputs the imaging data (RGB values of the output color space) to the output destination (S409). By outputting the imaging data, for example, an image or an image of imaging data under a light source defined in the output color space is reproduced on the monitor 211 or reproduced on the monitor or viewfinder of the imaging apparatus 101.

  In the above, the sRGB space is shown as an example of the output color space. However, any color space such as AdobeRGB space or ACESRGB space is applicable. In addition, although an example in which the Bladford conversion is used as the white adaptation conversion method has been described, any method such as CAT02 can be used.

  In the present embodiment, the color filter spectral characteristics of the image sensor of the image capturing apparatus 101 are desirably characteristics (router conditions) that can be linearly converted with respect to the color matching function xyz defined in CIE1931. For example, the colorimetric quality factor (q-factor) representing the degree of linearity is desirably 0.9 or more for all RGB.

  In this way, the imaging process 101 performs reverse processing of the image processing performed on the imaging data on the imaging data to obtain imaging data independent of the light source at the time of imaging. Then, the imaging data independent of the light source is converted into imaging data having a colorimetric color value, and white adaptation conversion is performed to convert the light source into imaging data corresponding to the white point of the output color space. At that time, the light source conversion processing only needs to include the white point information of the shooting scene and the white point information of the output color space, and the light source conversion processing is performed with high accuracy without providing a color conversion matrix or profile for each light source. be able to.

  Hereinafter, an image processing apparatus and an image processing method according to a second embodiment of the present invention will be described. Note that the same reference numerals in the second embodiment denote the same parts as in the first embodiment, and a detailed description thereof will be omitted.

  In the first embodiment, an example in which reverse correction is performed using information on white balance processing at the time of imaging has been described. However, the internal processing of the imaging apparatus 101 such as gamma processing varies depending on the ISO sensitivity and shooting mode settings, for example. Therefore, in the second embodiment, image processing for performing desired light source conversion processing on imaging data regardless of settings at the time of shooting will be described.

  A block diagram of FIG. 5 shows a configuration example of light source conversion processing of imaging data in the image processing apparatus 103 of the second embodiment. The processing configuration shown in FIG. 5 differs from the configuration shown in FIG. 3 in the following points. That is, the light source conversion processing of the second embodiment includes a shooting condition acquisition unit 501 that acquires the shooting conditions of the imaging apparatus 101 at the time of shooting, a gamma processing setting acquisition unit 502 that acquires gamma processing settings according to the shooting conditions, and each shooting condition. Γ setting storage unit 503 in which the gamma processing settings are stored.

  The light source conversion process according to the second embodiment will be described with reference to the flowchart of FIG. The CPU 201 starts the light source conversion process shown in FIG. 6 according to the user instruction, and inputs the imaging data to be processed by the input unit 300 as in the first embodiment (S400).

  Next, the shooting condition acquisition unit 501 acquires shooting conditions from, for example, metadata of shooting data (S411). The shooting conditions include information such as ISO sensitivity, aperture value, shutter speed, and shooting mode. Here, assuming that the gamma processing at the time of shooting differs for each ISO sensitivity, information indicating the ISO sensitivity is acquired.

  Next, the gamma processing setting acquisition unit 502 acquires the gamma processing setting corresponding to the shooting condition from the γ setting storage unit 503 (S412). The γ setting storage unit 503 stores, for example, gamma coefficients and gamma correction formulas as gamma processing settings for each ISO sensitivity. The gamma processing setting acquisition unit 502 acquires the gamma processing setting corresponding to the ISO sensitivity, and outputs the gamma processing setting to the inverse γ processing unit 301.

  The inverse γ processing unit 301 performs gamma processing having characteristics opposite to the input gamma processing setting on the input imaging data (S401). That is, the inverse γ processing unit 301 performs an inverse operation on the imaging data with respect to the gamma operation performed inside the imaging apparatus 101 during imaging, and outputs imaging data having a linear value with respect to the luminance of the subject. . The processing after step S402 is the same as the processing in the first embodiment shown in FIG.

  As described above, since the gamma processing having the reverse characteristics is performed on the imaging data based on the imaging conditions of the imaging apparatus 101, the light source conversion processing can be performed with high accuracy regardless of the imaging conditions.

[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program It can also be realized by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  301 ... Inverse γ processing unit, 302 ... White point information acquisition unit, 304 ... Inverse processing unit, 305 ... Color value conversion unit, 307 ... White point conversion unit, 308 ... Output value conversion unit

Claims (12)

Acquisition means for acquiring white point information of a shooting scene of imaging data;
Reverse processing means for performing reverse processing of the image processing performed on the imaging data by the imaging device on the imaging data;
First conversion means for converting the imaging data after the reverse processing into imaging data of a colorimetric color value;
Second conversion means for performing adaptation conversion for converting a white point of the imaging data into a white point of an output color space to the imaging data of the colorimetric color value;
An image processing apparatus comprising: third conversion means for converting the imaging data subjected to the adaptation conversion into imaging data in the output color space.   2. The image processing apparatus according to claim 1, wherein the inverse processing unit performs inverse processing of gamma processing performed by the imaging apparatus. Furthermore, it has a means to acquire the imaging condition of the imaging data,
2. The image processing apparatus according to claim 1, wherein the reverse processing means performs reverse processing of gamma processing corresponding to the photographing condition.   4. The image processing device according to claim 1, wherein the reverse processing unit performs reverse processing of white balance processing performed by the imaging device based on the white point information. Furthermore, it has a memory | storage means to store the gain amount of the white balance process which the said imaging device performs for every color temperature,
5. The reverse processing unit according to claim 4, wherein the reverse processing unit acquires a gain amount corresponding to a color temperature indicated by the white point information from the storage unit, and performs the reverse process of the white balance processing using an inverse number of the gain amount. Image processing device.   6. The second conversion unit according to claim 5, wherein the second conversion means calculates a correction amount used for the adaptation conversion from the white point of the shooting scene calculated from the reciprocal of the gain amount and the white point of the output color space. Image processing device.   6. The image processing apparatus according to claim 5, wherein the second conversion unit calculates a correction amount used for the adaptation conversion from a white point measured in the shooting scene and a white point of the output color space.   8. The image processing apparatus according to claim 1, further comprising an input unit that inputs the imaging data from the imaging apparatus.   8. The image processing apparatus according to claim 1, further comprising an input unit that inputs the imaging data from a recording medium. Obtain the white point information of the shooting scene of the imaging data,
The imaging device performs reverse processing of the image processing performed on the imaging data on the imaging data,
The imaging data after the reverse processing is converted into imaging data having a colorimetric color value,
Based on the white point information and the white point of the output color space, the image data of the colorimetric color value is subjected to white adaptation conversion,
An image processing method for converting imaging data subjected to the adaptation conversion into imaging data in the output color space.   10. A program for causing a computer to function as each unit of the image processing apparatus according to claim 1.   A computer-readable recording medium on which the program according to claim 11 is recorded.

Images