JP2001008224A - Image storage device, image reproducing device, image storage method, image reproducing method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image, resulting from revising a lighting environment, on a photographing object. SOLUTION: Image data are acquired (step ST11), and a spectrum distribution of a lighting light at photographing is acquired from a multi-band sensor as lighting component data (step ST12). Then object color component data equivalent to spectral reflectance of the photographing object are obtained by using the image data and the lighting component data (step ST13). Then the obtained lighting component data and the obtained object color component data are stored in such a way that they are extracted respectively independently (step ST14). Thus, data denoting the effect of a lighting environment on an image and data denoting a component of eliminating the effect of the lighting environment from the image are individually stored. As a result, arbitrary lighting component data can be combined with the object color component data at reproduction, so as to obtain an image with the lighting environment onto the photographing object revised.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for converting an image obtained by a digital camera, a scanner, or the like into an image having a changed lighting environment.

[0002]

2. Description of the Related Art Conventionally, image processing for correcting the color tone and atmosphere of an image obtained as digital data by an image input device such as a digital camera or a scanner has been performed. A representative example of such processing is color correction based on white balance. The white balance based correction corrects the image so that white objects appear white based on the overall color balance of the image, thereby removing some of the color effects of the illumination light on the imaged object from the image, The image is corrected to fit the human vision.

[0003]

The correction of the color of an image, which has been conventionally performed, is aimed at correcting the color to match human vision and fine correction of the color. Therefore, the data of the original image and the data relating to the color correction are not handled separately. For example, the corrected image is integrally stored as RGB luminance information.

[0004] On the other hand, it may be desirable to modify the hue of an image in order to include the sensation that one image gives an observer to another image. That is, there is a case where it is desired to capture the atmosphere produced by the lighting environment when a certain image is shot into an image shot in another lighting environment. Note that the illumination environment refers to an environment relating to illumination in consideration of not only the characteristics of the light source but also the surrounding conditions of the imaging target.

However, conventionally, since image data is handled as integrated data, it has been impossible to capture the atmosphere produced by the lighting environment in one image into another image. Further, even if an attempt is made to derive an atmosphere produced by a specific lighting environment by correcting the color of the image, simply changing the color of the entire image will result in an unnatural image.

Accordingly, the present invention has been made in view of the above-described problems, and has as its object to appropriately obtain an image in which the illumination environment for a photographing object is changed.

[0007]

According to the first aspect of the present invention, there is provided means for acquiring first data indicating an influence of a lighting environment on an image, and a second data for removing a component of the image from which the influence of the lighting environment has been removed. And a means for storing the second data in an independently retrievable state.

According to a second aspect of the present invention, in the image storage device according to the first aspect, the second data is data including a plurality of basis functions and a plurality of corresponding weighting coefficients.

A third aspect of the present invention is the image storage device according to the first or second aspect, further comprising means for storing the first data.

According to a fourth aspect of the present invention, there is provided the image storage device according to any one of the first to third aspects, wherein the means for acquiring the first data receives the first data from the object when the image is taken. Means for acquiring the intensities of a plurality of wavelength bands of the light to be emitted.

According to a fifth aspect of the present invention, in the image storage device according to any one of the first to third aspects, the means for acquiring the first data includes a plurality of filters corresponding to a plurality of wavelength bands. Means for acquiring a plurality of preliminary images as the image by performing photographing via the CPU, and means for obtaining the first data from the plurality of preliminary images.

[0012] According to a sixth aspect of the present invention, the first data indicating the influence of a predetermined lighting environment on an image and the second data stored by the image storage device according to any one of the first to fifth aspects are provided. To play the image.

According to a seventh aspect of the present invention, there is provided the image reproducing apparatus according to the sixth aspect, wherein each of a plurality of candidate data indicating an influence of a different illumination environment on the image is observed by the influence of the illumination environment. Means for displaying a list of sensibility information given to the plurality of candidate data, accepting selection of sensibility information from the list, and Means for determining corresponding candidate data as the first data.

[0014] The invention according to claim 8 is a step of obtaining first data indicating an influence of a lighting environment on an image, and a step of obtaining a component of the image from which the influence of the lighting environment has been removed as second data. Storing the second data in an independently retrievable state.

According to a ninth aspect of the present invention, in the image storing method according to the eighth aspect, the second data is data including a plurality of basis functions and a plurality of corresponding weighting coefficients.

According to a tenth aspect of the present invention, there is provided the image storing method according to the eighth or ninth aspect, further comprising the step of storing the first data.

The invention according to claim 11 is the invention according to claims 8 to 10
Wherein the step of acquiring the first data includes the step of acquiring intensities of a plurality of wavelength bands of light incident from an imaging target when the image is captured. .

The invention of claim 12 is the invention of claims 8 to 10
The image storing method according to any one of the preceding claims, wherein the step of acquiring the first data includes performing a plurality of preliminary images as the image by photographing through each of a plurality of filters corresponding to a plurality of wavelength bands. And a step of obtaining the first data from the plurality of preliminary images.

According to a thirteenth aspect of the present invention, there is provided a step of preparing first data indicating an influence of a predetermined illumination environment on an image, and the first data stored by the image storing method according to any one of the eighth to twelfth aspects. And a step of preparing an image using the first data and the second data.

According to a fourteenth aspect of the present invention, there is provided the image reproducing method according to the thirteenth aspect, wherein each of a plurality of candidate data indicating an influence of a different lighting environment on the image is observed by the influence of the lighting environment. Sentiment information indicating a feeling given to the user is provided in advance, and the step of preparing the first data includes the step of displaying a list of sentiment information added to the plurality of candidate data; Accepting the selection and determining candidate data corresponding to the selected sensibility information as the first data.

According to a fifteenth aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for processing an image, wherein the program comprises: first data indicating an influence of a predetermined lighting environment on an image; Item 6. The computer causes the computer to execute a process of reproducing an image using the second data stored by the image storage device according to any one of Items 1 to 5.

[0022]

DESCRIPTION OF THE PREFERRED EMBODIMENTS <1. First Embodiment> FIG.
Is a digital camera 1 according to the first embodiment of the present invention.
It is a perspective view which shows the whole of FIG. The digital camera 1 includes a lens unit 11 for photographing, and a lens unit 1
And a main unit 12 that processes the image acquired as digital data in 1.

The lens unit 11 has a lens system 111 having a plurality of lenses, and a CCD 112 for acquiring an image of an object to be imaged through the lens system 111. The image signal output from the CCD 112 is transmitted to the main unit 12.
Sent to. The lens unit 11 includes a finder 113 for the operator to capture the subject, and a multi-band sensor 114 used for image processing described later.
Is provided.

Note that other sensors such as a distance measuring sensor are also arranged in the window where the multiband sensor 114 is provided.

The main body 12 is provided with a flash 121 and a shutter button 122. An operator captures an object through the finder 113 and operates the shutter button 122, so that an image is electrically acquired by the CCD 112. You. At this time, if necessary, the flash 121
Emits light.

The image signal from the CCD 112 is transmitted to the main unit 12.
The processing described below is performed internally, and the main unit 1
2 is stored in an external memory 123 (a so-called memory card) mounted on the memory card 2. The external memory 123 is the main unit 1
2 When the cover on the lower surface is opened and the take-out button 124 is operated, it is taken out from the main body 12. The data stored in the external memory 123 as a recording medium can be transferred to another device using a separately provided computer or the like. Conversely, the digital camera 1 can read data stored in the external memory 123 by another device.

A liquid crystal display, operation buttons, and the like are provided on the back of the main body 12, and a desired input operation is performed by an operator operating the buttons while looking at the screen.

FIG. 2 shows the configuration of the digital camera 1.
FIG. 2 is a block diagram schematically showing a configuration mainly for executing processing according to the present invention.

In the configuration shown in FIG.
CCD 112, A / D converter 115, shutter button 1
The CPU 22, the ROM 22, and the RAM 23 realize a function of acquiring an image. That is, the lens system 111
When the shutter button 122 is pressed, the image signal from the CCD 112 is digitally converted by the A / D converter 115.
The digital image signal converted by the A / D converter 115 is stored as image data 231 in the RAM 23 of the main unit 12. The CPU 21 controls these processes by the RO.
This is performed by operating according to the program 221 stored in the M22.

The CPU 2 provided in the main body 12
1. ROM 22 and RAM 23 implement the function of processing images. Specifically, according to the program 221 stored in the ROM 22, the CPU 21 performs image processing on the acquired image while using the RAM 23 as a work area. At this time, information from the multi-band sensor 114 is used.

The illumination component data 232 and the object color component data 233 stored (ie, stored) in the RAM 23
Is the output of the multi-band sensor 114 and the image data 23
1, which will be described later in detail. These data are appropriately stored in the external memory 123.

The display 24 is a means for displaying images and displaying information to the operator.
5 is a means for receiving an input operation from the operator.

FIG. 3 mainly shows a CPU 21 and a ROM 22.
FIG. 4 is a block diagram showing a configuration of functions realized by the RAM 23 and other configurations, and FIG. 4 is a flowchart showing a flow of image processing. In the configuration shown in FIG. 3, the illumination component data generator 201 and the object color component data generator 202
And the image reproducing unit 203 includes the CPU 21, the ROM 22,
This is a function realized by the RAM 23 and the like. Hereinafter, the operation of the digital camera 1 will be described with reference to these drawings.

First, the CCD 112 is operated by the photographing operation described above.
Is transmitted from the A / D converter 115 to the RAM 23, and the image data 23
It is stored as 1 (step ST11).

Next, the illumination component data generation unit 201 generates illumination component data 232 based on the output from the multi-band sensor 114, and stores it in the RAM 23 (step ST12). The illumination component data 232 is data indicating the effect of the illumination environment on the imaging target in the obtained image.

The multi-band sensor 114 is a sensor that obtains the intensity of incident light in a plurality of wavelength bands. The intensities of a plurality of wavelength bands obtained by the multi-band sensor 114 are combined by the illumination component data generation unit 201 into data indicating a spectral distribution, and are stored in the RAM 23 as illumination component data 232.

That is, in the digital camera 1, the spectral distribution of light incident on the digital camera 1 from an object to be photographed is treated as data indicating the influence of the illumination environment on the acquired image. . Generally, in an image of a shooting target illuminated with white light, the average of colors becomes gray. Therefore, the spectral distribution obtained by the multiband sensor 114 can be used as data indicating the effect of the illumination environment.

As the multi-band sensor 114, a sensor having a plurality of light intensity detectors each provided with a filter that transmits only light in each wavelength band can be used. For example, a wavelength band in the visible region is divided into a plurality of wavelength bands, and a filter that transmits only light in each wavelength band is provided in each light intensity detector. The plurality of wavelength bands is preferably four or more, but it is possible to estimate the approximate spectral distribution of light incident on the digital camera 1 even with three (for example, three color filters of RGB). In this case, C
The CD 112 can also serve as the multiband sensor 114.

As the small and high-resolution multi-band sensor 114, “Spectral Colorimeter CM-100” by Nobukazu Kawagoe and two others, Minolta Techno Report No.5 1988
(Pages 97 to 105), there is also known a CCD in which metal film interference filters having different thicknesses are provided on a CCD stepwise. In this multi-band sensor, CC
The thickness of the metal film interference filter is changed for each area D, and it is realized that the intensity of light in a predetermined wavelength band is obtained for each area of the CCD.

After the illumination component data generation unit 201 generates and stores the illumination component data 232, the object color component data generation unit 202 illuminates the image data 231 using the image data 231 and the illumination component data 232. Object color component data 233 is obtained as a component from which the influence of the environment has been removed (step ST13). The object color component data 233 is data substantially corresponding to the spectral reflectance of the imaging target. Hereinafter, the object color component data 233
The principle of obtaining the spectral reflectance of the object to be photographed corresponding to the above will be described.

Assuming that the number of input wavelength bands (number of input bands) of the CCD 112 is 3 (3 bands of RGB),
RGB pixel value ρ of each pixel which is the output value of CCD 112
r, ρg, ρb are λ for the wavelength in the visible region, Rr (λ), Rg (λ), Rb (λ) for the spectral sensitivity of each color of the CCD 112, When the reflectance (hereinafter, referred to as “target spectral reflectance”) is S (λ), and the spectral distribution of illumination light (including indirect illumination light) (that is, illumination component data 232) is E (λ). ,

[0042]

(Equation 1)

Is satisfied.

Here, the spectral sensitivity R of each color of the CCD 112 is
r (λ), Rg (λ), and Rb (λ) are characteristics that can be measured from the CCD 112, and the spectral distribution E (λ) of the illumination light is a value obtained from the output of the multiband sensor 114. Of course, the pixel value ρ which is the output value of the CCD 112
r, ρg, ρb are also known.

Therefore, the target spectral reflectance S (λ) is defined as a weighted sum of _three basis functions S ₁ (λ), S ₂ (λ), and S ₃ (λ).

[0046]

(Equation 2)

Are expressed as follows. The weighting coefficients b ₁ , b ₂ and b ₃ are characteristic coefficients of the target spectral reflectance S (λ). Thus, the pixel values ρr, ρg, ρb are

[0048]

(Equation 3)

Is transformed. That is, the pixel values ρr, ρ
g and ρb are

[0050]

(Equation 4)

Is represented as

From the above, Equation 4 is obtained by adding weighting coefficients b ₁ ,
By solving for b ₂ and b ₃ , the target spectral reflectance S
(λ) can be obtained.

The object color component data generation unit 202 obtains weighting coefficients b ₁ , b ₂ , and b ₃ for each pixel by the above method. Of course, in the actual calculation, the wavelength λ is treated as a discrete variable. Then, three basis functions S ₁ (λ), S
₂ (λ), S ₃ (λ) and three weighting coefficients b ₁ ,
b ₂ and b ₃ are stored in the RAM 23 as object color component data 233. As a result, data corresponding to the spectral reflectance of the imaging target is stored as the object color component data 233. Note that the basis function may be appropriately set according to the pixel value, or may be a function that does not depend on the pixel. When the basis function is used in common for all pixels, the only data to be stored for each pixel is a weighting coefficient.

When the illumination component data 232 and the object color component data 233 are obtained, these data are transferred and stored in the external memory 123 which is detachable from the digital camera 1 as needed according to the instruction of the operator (step S).
T14). That is, only one of the illumination component data 232 and the object color component data 233 may be stored in the external memory 123, or both may be stored in the external memory 123. Will be saved.

Next, the RAM 23 or the external memory 123
The usage of the illumination component data 232 and the object color component data 233 stored in the.

FIG. 5 is a flowchart showing a flow of an operation of reproducing an image in the digital camera 1 using the illumination component data 232 and the object color component data 233. In addition,
The external memory 123 stores a plurality of object color component data 233
And a plurality of illumination component data 232 are stored in advance.

First, when the operator gives an instruction through the operation buttons 25 while watching the display on the display 24, the desired object color component data 233 is read into the RAM 23 (step ST21). Thereby, the target of the image to be reproduced (that is, the shooting target at the time of shooting) is determined.

Next, desired illumination component data 232 is read into the RAM 23 in accordance with an instruction from the operator (step ST22). That is, when it is desired to reproduce an image that is the basis of the already read object color component data 233, the illumination component data 232 obtained when the object color component data 233 is obtained is read into the RAM 23,
When it is desired to reproduce the image under the lighting environment when the other image is captured, the illumination component data 232 obtained when the other image is captured is read into the RAM 23.

Thereafter, the illumination component data 232 and the object color component data 233 are combined by the image reproducing unit 203 shown in FIG. 3 to generate target image data (step ST23). At the time of reproduction, the target spectral reflectance S (λ) of each pixel is obtained from the object color component data 233 by the calculation shown in Expression 2, and the RG of each pixel is calculated using Expression 1.
The pixel values ρr, ρg, ρb of B are obtained.

Then, the obtained image data is displayed on the display 24 provided on the back of the digital camera 1 (step S24).

As described above, in the digital camera 1, the illumination component data 232 is acquired as data indicating the effect of the illumination environment at the time of photographing on the image using the multiband sensor 114, and the illumination component data 232 is used. Then, object color component data 233 is obtained as data indicating a component obtained by removing the influence of the lighting environment from the image. Further, these data are stored in a state where they can be separately and independently taken out.

Therefore, in the digital camera 1, by appropriately combining the illumination component data 232 with the object color component data 233, it is possible to reproduce an image of a desired photographing object under a desired illumination environment. That is, when an image is to be reproduced in the atmosphere at the time of shooting, the illumination component data 232 and the object color component data 233 generated in pairs with each other may be combined, and the influence of the illumination environment of other images is taken in. To reproduce the object color component data 23
3 is combined with the illumination component data 232 obtained in another illumination environment to reproduce an image in which the illumination environment has been changed.

In the digital camera 1, the illumination component data 232 is easily obtained using the multi-band sensor 114.

In the above description, the number of input bands of the digital camera 1 is three, but the number of input bands of the digital camera 1 may be four or more. That is, the CCD 112 may output the pixel value of each pixel corresponding to four or more wavelength bands. In this case, since the number of equations shown in Equation 1 is 4 or more in accordance with the number of input bands, the basis functions shown in Equation 2 can be increased to the same number as the number of input bands. As a result, the target spectral reflectance S (λ) is more accurately obtained, and the quality of the reproduced image is improved.

<2. Second Embodiment> FIG. 6 is a diagram showing a configuration of a digital imaging system 3 according to a second embodiment of the present invention. The digital imaging system 3 includes a monochrome digital camera 31 for acquiring an image to be captured, a rotary filter 32 having a plurality of filters, and a computer 33 for processing an image obtained by the digital camera 31. The rotary filter 32 has a plurality of filters that transmit only light in a predetermined wavelength band, and limits light incident on the digital camera 31 to a specific wavelength band.

FIG. 7 is a block diagram showing a main configuration of the digital imaging system 3. As shown in FIG. In the digital camera 1 according to the first embodiment, as shown in FIG. 2, the illumination component data 232 is obtained using the multiband sensor 114,
Although image processing is performed inside the digital camera 1, in the digital imaging system 3 according to the second embodiment, the illumination component data 232 is obtained from a plurality of images captured by the digital camera 31. The first embodiment differs from the first embodiment in that the image processing is performed by the computer 33.

As shown in FIG. 7, the digital camera 31 includes a lens system 311, a CCD 312, and an A / D converter 31.
3 and a function of acquiring an image to be captured as digital data.

The computer 33 is a digital camera 31
CPU 4 has a function of processing an image acquired by
1, an HDD (fixed disk) 42, a RAM 43, and a disk drive 44. The HDD 42 stores a program 421 for processing an image.
AM 43 includes preliminary image data 431 and illumination component data 4
32 and an object color component data 433 are stored in the area and an image processing operation is performed in the area.

The RAM 43 is capable of transferring data to and from a recording medium such as a fixed disk, a portable magnetic disk, or a magneto-optical disk. As in the first embodiment, the illumination component data 432 and the object color The component data 433 is exchanged with the recording medium as needed. In addition,
FIGS. 6 and 7 illustrate the recording medium 9 in which data is transferred to the RAM 43 and the like via the disk drive 44 as the recording medium.

Further, a program for image processing is stored in the recording medium 9, and the program is read into the RAM 43.
The computer 33 may function as an image processing device by the program in the D 42 and the program in the RAM 43.

The computer 33 further has a display 4 for displaying images and displaying information to the operator.
And an input unit 46 such as a keyboard and a mouse for receiving an input from the operator.

Digital camera 31 and computer 3
3 is similar to the configuration of the digital camera 1 according to the first embodiment as described above, but in this digital imaging system 3, the illumination component data 432 and the object color This embodiment differs from the first embodiment in that component data 433 is obtained.

The rotary filter 32 has a configuration in which a disk-shaped rotating body is provided with a plurality of filters that transmit only light in a specific wavelength band, and the digital camera 31 has different wavelengths by rotating the rotating body. Only the light in the band enters while being switched. Each filter corresponds to each wavelength band obtained by dividing the wavelength band of visible light into a plurality.

The computer 33 can recognize which type of filter is located in front of the lens system 311 of the digital camera 31 based on a signal from an encoder provided in the rotary filter 32.

FIG. 8 shows the CPU 41, HDD 42, RAM
FIG. 8 is a block diagram illustrating a functional configuration and other configurations realized by the component data generation unit 43 and the like; a component data generation unit 401 and an image reproduction unit 403 include a CPU 41 illustrated in FIG.
42 and a work area of the RAM 43.

FIG. 9 is a flowchart showing the flow of the operation of the digital imaging system 3. Hereinafter, the operation of the digital imaging system 3 will be described with reference to FIGS.

First, a predetermined filter is used for the digital camera 3.
1 lens system 311 (step ST3).
1) Photographing is performed under the control of the computer 33 (step ST32). As a result, only light in the wavelength band transmitted by the filter is incident on the digital camera 31, and preliminary image data 431 corresponding to one preliminary image is transferred from the A / D converter 313 of the digital camera 31 to the computer 33 and is transmitted to the RAM 43. Is stored.

Next, the filter for rotating the rotary filter 32 and transmitting only light of another wavelength band is formed by the lens system 311.
Is located in front of the camera and performs photographing (steps ST31 and S31).
T32). Thereby, the preliminary image data 431 corresponding to another preliminary image is stored in the RAM 43.

Thereafter, by changing the filters and photographing repeatedly (step ST33), the preliminary image data 431 corresponding to each filter is stored in the RAM 43. The preliminary image data 431 includes the rotary filter 3
Information about the filter from No. 2 is given. Note that capturing a plurality of preliminary images is equivalent to capturing a color image to be captured.

When the preliminary image data 431 corresponding to a plurality of preliminary images is stored in the RAM 43, the component data generating section 401 generates illumination component data 432 and object color component data 433 from the preliminary image data 431. (Step ST34). That is, if the CPU 41
The arithmetic processing described below is performed according to the program 421 in D42.

Illumination component data 432 from a plurality of preliminary images
As a method for obtaining the object color component data 433, any method may be used as long as appropriate data is generated. For example, Shoji Tominaga, "Camera system and algorithm for achieving color constancy", IEICE Technical Report PRU95-11 (19
95-05) (pages 77 to 84) may be employed. In this method, the spectral distribution of the illumination light and the spectral reflectance of the imaging target are obtained as follows.

First, the spectral distribution C (λ) of light incident from a position on a photographing target (hereinafter referred to as “target position”) has a wavelength λ, a spectral reflectance at the target position S (λ), and illumination. Assuming that the spectral distribution of light is E (λ),

[0083]

(Equation 5)

Are represented as Here, α and β are scale factors, αS (λ) E (λ) is a diffuse reflection component, and β
E (λ) corresponds to a specular reflection component.

The pixel value of the pixel corresponding to the target position in the image captured by the k-th filter (ie,
The output value of the CCD 312) ρk is defined as Rk (λ), where Sk is the spectral sensitivity of the CCD 312 corresponding to the wavelength band of the filter.

[0086]

(Equation 6)

## EQU10 ##

Here, by using a dimensional linear model that expresses the spectral distribution E (λ) of the illumination light and the spectral reflectance S (λ) at the target position as a weighted sum of a plurality of basis functions,
Finally,

[0089]

(Equation 7)

This is transformed. Here, ρ is a vector having a pixel value ρk as an element, and Λ is the spectral reflectance S (λ) at the target position.
Is a matrix representing the response of the sensor to the basis function of the dimensional linear model, σ is the vector of the weighting coefficients of the dimensional linear model of the spectral reflectance S (λ), and H is the dimensional of the spectral distribution E (λ) of the illumination light. A matrix representing the response of the sensor to the basis function of the linear model, and ε is a vector of weighting coefficients of the dimensional linear model of the spectral distribution E (λ).

Then, under the same illumination, σ and ε can be obtained from the above equations by using an appropriately simplified calculation method while utilizing the rule that the direction of the vector Hε is constant for a plurality of target positions. . Thus, the coefficients of the two dimensional linear models are obtained, and the spectral distribution E (λ) of the illumination light and the spectral reflectance S (λ) at the target position are obtained.

Using the method exemplified above, the component data generation unit 401 obtains the spectral distribution E (λ) of the illumination light and the spectral reflectance S (λ) at the target position. Note that the illumination component data 4
32, the basis function and the weighting coefficient vector of the dimensional linear model of the spectral distribution E (λ) of the illumination light are stored. As the object color component data 433, the basis function of the dimensional linear model of the spectral reflectance S (λ). And a weighting coefficient vector for each pixel is stored.

In the above description, the scale factors α, β
Has been described as being appropriately set in advance,
In this case, the scale factors α and β may be stored as part of the object color component data 433. Further, in order to simplify the processing, the calculation may be performed with the scale factor β set to 0.

The illumination component data 432 and the object color component data 433 are obtained by the component data generator 401.
When stored in the RAM 43, one or both of them are appropriately stored in the recording medium 9 according to the instruction of the operator (step ST36). Thus, the illumination component data 432
The object color component data 433 is stored in a state where it can be taken out independently.

At this time, before the illumination component data 432 is stored, information indicating a feeling (hereinafter referred to as “kansei information”) is included in the illumination component data 432 based on the input of the operator through the input unit 46. (Step ST3)
5). That is, as shown in FIG.
Is added to the illumination component data 432, or FIG.
Are associated as shown in FIG. As described above, the illumination component data 432 is data indicating the effect of the illumination environment in the image, and an image reproduced using the illumination component data 432 gives the observer a certain kind of stimulus. Therefore,
In the digital imaging system 3, the illumination component data 432 is stored in an identifiable state using information indicating the sensation (sensitivity) received by the observer.

More specifically, words indicating a sense of season such as "spring-like" or "midsummer", words indicating a feeling received from a time such as "sunset", "early morning", "afternoon", and "cold" Information such as words indicating a temperature sensation such as "" is hot and hot "and words indicating a sensation received from the weather such as" misty "or" cloudy "are added to the illumination component data 432 and saved. .

FIGS. 12 to 14 show illumination component data 4.
FIG. 3 is a diagram illustrating an example of a spectral distribution of illumination light represented by 32. As the sensibility information, for example, words such as "sunset" in FIG. 12, "cold" in FIG. 13, and "afternoon" in FIG. 14 are given. Of course, it is not necessary to add information indicating a feeling to all the illumination component data 432, and a phrase such as "fluorescent light" may be added.

Note that the sensibility information is not limited to words, but may be color, sound, or the like.

Next, the illumination component data 432 and the object color component data 4 stored in the recording medium 9 as described above.
The operation of the digital imaging system 3 when an image is reproduced by using 33 will be described with reference to FIG.

When an image is reproduced, as in the first embodiment, first, desired object color component data 433 is read from the recording medium 9 into the RAM 43 in accordance with an instruction from the operator (step ST41). Thereby, the shooting target of the displayed image is determined.

Next, the image reproducing section 403 reads out the sentiment information 432a given to each illumination component data 432 stored in the recording medium 9, and displays a list of these on the display 45 (step ST42). ). As a result, the illumination component data 432, which is a selection candidate in the recording medium 9, can be selected based on the sensitivity information 432a. 16 and 17 are diagrams showing examples of the display screen of the display 45 when the light source is shown and when the light source is not shown.

The operator designates one of them through the input section 46 while viewing the list of the kansei information 432a. When receiving the designation from the operator, the computer 33 receives the illumination component data 4 to which the designated sensitivity information 432a is added.
32 is determined as the illumination component data 232 to be used for reproduction (step ST43).
Is read from the recording medium 9 to the RAM 43 (step S
T44).

Thereafter, as in the first embodiment, the image reproducing unit 403 combines the illumination component data 432 and the object color component data 433 stored in the RAM 43 by performing the operation shown in Expression 5, and synthesizes the image data. Is generated (step ST45). The generated image data is displayed on the display 45.
(Step ST46).

As described above, the digital imaging system 3 acquires a plurality of preliminary images using a plurality of filters, and generates the illumination component data 432 and the object color component data 433 based on these preliminary images. Since these data are stored in such a manner that they can be independently taken out, it is possible to appropriately reproduce an image that would be obtained from a photographing target under a desired lighting environment as in the first embodiment. it can.

The illumination component data 432 shown in FIG.
By providing a standard light (such as D65) as shown in the example, accurate color reproduction of the photographing target can be performed from an image obtained in an arbitrary lighting environment. An image suitable as an image to be used can be generated.

Further, in the digital imaging system 3, the illumination component data 432 is given sensitivity information 432a so that the operator can select the illumination component data 432 based on the sensitivity information 432a. Thus, it is possible to enjoy image reproduction based on sensitivity.

<3. Modifications> While the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-discussed preferred embodiments, but allows various modifications.

For example, the illumination component data in the above embodiment may be any data as long as the data indicates the effect of the illumination environment on the image. It does not need to be strictly required. Also, the object color component data may be any data as long as the data shows the component from which the influence of the lighting environment has been removed from the image. There is no.

In the above embodiment, the illumination component data is stored as a plurality of basis functions and a plurality of weighting coefficients. However, the storage format of the illumination component data may be another format. For example, the illumination component data may be stored as a characteristic curve of the spectral distribution.

The illumination component data need not always be obtained using the digital camera 1 or the digital imaging system 3. For example, illumination component data intended for virtual illumination may be separately generated. Further, the illumination component data used when the image is reproduced is stored in the external memory 1.
23 or a plurality of illumination component data stored in the recording medium 9.

When the illumination component data and the object color component data are stored separately and independently (for example, as different files having the same file name but different extensions) in the RAM, the external memory 123, the recording medium 9, and the like. Although described, the illumination component data and the object color component data may be stored in any form as long as the data can be extracted independently.
For example, object color component data may be stored in the first half of one file, and illumination component data may be stored in the second half, and the object color component data and the illumination component data for each pixel are sequentially arranged in one file. It may be arranged.

In the second embodiment, the rotary filter 32 is arranged in front of the digital camera 31. However, a mechanism for switching a plurality of filters may be provided inside the digital camera 31. Further, a digital camera 31 having a plurality of (preferably four or more) input bands may be used without providing a filter switching mechanism, so that an image may be obtained for each of a plurality of wavelength bands. . Further, the digital camera 31 and the rotary filter 32 do not need to be connected to the computer 33 by a transmission line, and the preliminary image data 431 and the like may be taken into the computer 33 via a recording medium.

Further, in the second embodiment, the illumination component data 432 and the object color component data 433 are generated by the component data generation unit 401. However, the illumination component data 432 is Then, the object color component data 433 may be obtained by the calculation shown in Expression 4 in the same manner as in the first embodiment.
This simplifies the operation. As described above, various methods can be adopted as a calculation method for obtaining the illumination component data and the object color component data.

Further, the present invention is not limited to the case where an image is obtained using a digital camera, but can be used for an image obtained with another image input device such as a scanner.

[0115]

According to the first to fifth and eighth to twelfth aspects of the present invention, the second data indicating the component from which the influence of the illumination environment has been removed from the image is stored in such a manner that it can be independently extracted. Therefore, when reproducing an image, the first data and the second data generated independently of each other can be used. This makes it possible to reproduce an image in which the lighting environment has been changed.

According to the fourth and eleventh aspects of the present invention, the first data can be easily obtained.

According to the sixth and seventh aspects and the thirteenth to fifteenth aspects of the present invention, it is possible to reproduce an image in which the illumination environment for the photographing target is changed.

Further, according to the inventions set forth in claims 7 and 14, the influence of the lighting environment on the reproduced image can be selected based on sensitivity.

[Brief description of the drawings]

FIG. 1 is a perspective view showing an entire digital camera according to the present invention.

FIG. 2 is a block diagram showing a configuration for executing processing according to the present invention in the digital camera shown in FIG. 1;

FIG. 3 is a block diagram showing the configuration shown in FIG. 2 divided into functions.

FIG. 4 is a flowchart showing an operation flow when saving an image in the digital camera shown in FIG. 1;

5 is a flowchart showing a flow of an operation when reproducing an image in the digital camera shown in FIG. 1.

FIG. 6 is a diagram showing a configuration of a digital imaging system according to the present invention.

FIG. 7 is a block diagram showing a configuration for executing processing according to the present invention in the digital imaging system shown in FIG. 6;

8 is a block diagram showing the configuration shown in FIG. 7 divided into functions.

FIG. 9 is a flowchart showing a flow of operation when saving an image in the digital imaging system shown in FIG. 6;

FIG. 10 is a schematic diagram showing a relationship between illumination component data and sensibility information.

FIG. 11 is a schematic diagram illustrating a relationship between illumination component data and sensibility information.

FIG. 12 is a diagram illustrating an example of a spectral distribution of illumination light represented by illumination component data.

FIG. 13 is a diagram illustrating an example of a spectral distribution of illumination light represented by illumination component data.

FIG. 14 is a diagram illustrating an example of a spectral distribution of illumination light represented by illumination component data.

FIG. 15 is a flowchart showing a flow of an operation when reproducing an image in the digital imaging system shown in FIG. 6;

FIG. 16 is a diagram showing an example of a screen on which a list of sensibility information is displayed.

FIG. 17 is a diagram illustrating an example of a screen on which a list of sensitivity information is displayed.

[Explanation of symbols]

Reference Signs List 1 digital camera 3 digital imaging system 9 recording medium 21, 41 CPU 22 ROM 23, 43 RAM 31 digital camera 32 rotary filter 33 computer 42 HDD 44 disk drive 45 display 46 input unit 114 multi-band sensor 201 illumination component data generation unit 202 Object color component data generation units 203, 403 Image reproduction unit 231 Image data 232, 432 Illumination component data 233, 433 Object color component data 401 Component data generation unit 421 Program 431 Preliminary image data 432a Sensitivity information ST12 to ST14, ST21 to ST24, ST31-
ST34, ST36, ST41 to ST46 Step

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat 参考 (Reference) H04N 5/225 H04N 9/04 Z 5C065 5/235 G06F 15/66 310 5C066 5/91 H04N 1/40 D 5C077 9/04 1/46 A 5C079 5/91 J 9/79 9/79 GF term (reference) 2H002 CC00 DB17 HA04 JA07 5B057 AA11 AA20 BA02 CA01 CA08 CA12 CB01 CB08 CB12 CC01 CE17 CH11 DB02 DB06 DB09 DC25 5C022 AA13 AB AB15 AC03 AC42 AC54 AC55 5C053 FA08 FA27 HA33 HA40 JA26 KA01 KA24 KA25 LA02 5C055 AA03 AA06 BA03 DA00 EA05 HA37 5C065 AA03 BB02 BB41 CC01 CC09 DD02 EE15 GG18 GG26 GG32 GG44 5C066 KA01 KA01 KA01 KA01 GM01 5C077 LL01 LL19 MM03 MM20 MP08 PP31 PP32 PP37 PP43 PP71 PQ18 PQ22 SS03 TT09 5C079 JA16 JA25 LA01 LA02 LB01 MA01 MA11 MA17 NA21 PA00

Claims (15)

[Claims] 1. A first image showing an influence of an illumination environment on an image.
Means for acquiring data, means for acquiring a component of the image from which the influence of the lighting environment has been removed as second data, and means for storing the second data in a state that can be independently retrieved. An image storage device characterized by the above-mentioned. 2. The image storage device according to claim 1, wherein the second data is data including a plurality of basis functions and a corresponding plurality of weight coefficients. 3. The image storage device according to claim 1, further comprising: a unit configured to store the first data. 4. The image storage device according to claim 1, wherein the means for acquiring the first data includes a plurality of light beams incident from a photographing target when the image is photographed. An image storage device, comprising: means for acquiring intensity in a wavelength band. 5. The image storage device according to claim 1, wherein the means for acquiring the first data captures an image via each of a plurality of filters corresponding to a plurality of wavelength bands. An image storage device comprising: means for acquiring a plurality of preliminary images as the image by performing the operation; and means for obtaining the first data from the plurality of preliminary images. 6. An image is reproduced using first data indicating an influence of a predetermined lighting environment on an image, and the second data stored by the image storage device according to claim 1. An image reproducing apparatus, comprising: 7. The image reproducing apparatus according to claim 6, wherein each of the plurality of candidate data indicating an influence of a different lighting environment on the image gives a sense of an image to an observer due to the influence of the lighting environment. A means for displaying a list of sentiment information assigned to the plurality of candidate data, accepting selection of sentiment information from the list, and selecting candidate data corresponding to the selected sentiment information. Means for determining the first data. 8. A first image showing an influence of an illumination environment on an image.
A step of acquiring data; a step of acquiring, as second data, a component of the image from which the influence of the illumination environment has been removed; and a step of storing the second data in a state that can be independently retrieved. An image storage method characterized by the following. 9. The image storage method according to claim 8, wherein the second data is data including a plurality of basis functions and a corresponding plurality of weighting coefficients. 10. The image storage method according to claim 8, further comprising the step of storing the first data. 11. The image storage method according to claim 8, wherein the step of obtaining the first data includes: a step of obtaining a plurality of light beams incident from a shooting target when the image is shot. Acquiring an intensity in a wavelength band. 12. The image storage method according to claim 8, wherein the step of acquiring the first data includes capturing an image via each of a plurality of filters corresponding to a plurality of wavelength bands. An image storing method, comprising: acquiring a plurality of preliminary images as the images by performing the image processing; and obtaining the first data from the plurality of preliminary images. 13. A step of preparing first data indicating an influence of a predetermined lighting environment on an image, and preparing the second data stored by the image storing method according to claim 8. And a step of reproducing an image using the first data and the second data. 14. The image reproducing method according to claim 13, wherein each of a plurality of candidate data indicating an influence of a different lighting environment on an image gives a sense of an image to an observer due to the influence of the lighting environment. The sentiment information to be given is given in advance, and the step of preparing the first data includes the step of displaying a list of sentiment information given to the plurality of candidate data; Deciding candidate data corresponding to the sentiment information as the first data. 15. A computer-readable recording medium having recorded thereon a program for processing an image, wherein the program comprises: first data indicating an influence of a predetermined lighting environment on an image; A recording medium characterized by causing a computer to execute a process of reproducing an image using the second data stored by the image storage device according to any one of the above.