JP5942356B2 - Spectral information acquisition apparatus, spectral information acquisition method, and spectral information acquisition program - Google Patents

Description

  The present invention relates to a spectral information acquisition apparatus, a spectral information acquisition method, and a spectral information acquisition method for acquiring spectral information of a subject to be photographed using an imaging means that forms an imaging system by combining an imaging device such as a digital camera and an optical filter such as an interference filter. The present invention relates to a spectral information acquisition program.

  Spectral information is an important physical characteristic for determining the color of a shooting target. In particular, acquiring spectral reflectance of a shooting target as image information performs color reproduction of the shooting target in an arbitrary illumination environment. It will be very useful above.

  When acquiring spectral information of an imaging target using an imaging device such as a digital camera, information is insufficient for a monochrome camera or an RGB three-channel camera. In many cases, a multiband camera that acquires information is used.

  As an example of a method for realizing a simple multiband camera, an imaging system is configured by combining an RGB digital camera and an optical filter installed between the camera and an imaging target (subject). When the optical filter shown is not attached and when the optical filter shown in the lower part of FIG. 6 is attached, photographing by the imaging system is sequentially repeated to obtain two images having different spectral sensitivities of the imaging system, and obtaining color information for a total of six channels (For example, Non-Patent Document 1 and Non-Patent Document 2). The middle part of FIG. 6 shows the spectral transmittance characteristics of the optical filter installed between the RGB digital camera and the imaging target.

  Such a multi-band realization method has a relatively small number of channels, ie, six channels. However, many spectral reflectances of natural objects and color materials can be expressed by a linear sum of a small number of basis functions of about six at most. By using as a learning sample a sample group having the same kind of spectral reflectance as that of a subject that is known and measured in advance, the spectral reflectance of the subject can be obtained with high accuracy even in 6 channels. It can be acquired (for example, Non-Patent Document 3).

  As an optical filter used in the above-described method, there is a method using a color glass filter (for example, Non-Patent Document 1), but the color glass filter has a desired spectral sensitivity characteristic because the change in the spectral transmittance waveform is gradual. Is difficult to achieve, and the accuracy of spectral reflectance estimation is poor. In addition, since the spectral transmittance of the entire color glass filter is low, there is a problem that the spectral sensitivity of the imaging system when the filter is mounted is reduced and the S / N of the obtained image is deteriorated.

  In addition, as an optical filter used in the above method, there is a method using an interference filter (for example, Non-Patent Document 2). Since this interference filter can control the spectral transmittance to some extent, it is easy to obtain a desired spectral sensitivity characteristic, Further, since a high spectral transmittance can be realized, the spectral sensitivity of the imaging system when the filter is attached is increased, and there is an advantage that a high S / N can be maintained.

  However, the interference filter has an optical characteristic that the transmission wavelength band shifts to the short wavelength side when the incident angle of light to the filter is not perpendicular to the filter surface. As a result, when the spectral reflectance is acquired using an imaging system that is multibanded using an interference filter, there is a problem that the accuracy of spectral reflectance estimation decreases from the center of the image toward the periphery.

  Therefore, as a means for overcoming the above problems, an imaging system is configured by using a liquid crystal tunable filter or a plurality of interference filters having a narrow transmission wavelength band and a monochrome camera, and obtained by this imaging system. When estimating the spectral reflectance from the captured image, the wavelength shift amount of the filter spectral transmittance for each pixel position is identified in advance, and when the spectral reflectance is estimated from the captured image as described above, the estimated spectral reflectance is estimated. There is a method of correcting reflectance data by a wavelength shift amount (for example, Patent Document 1 and Patent Document 2).

  However, this method of correcting only the wavelength shift amount can be achieved only by using a narrow band filter and using the center wavelength of the transmission wavelength band as the spectral reflectance sample point. It cannot be applied to a method using a filter having a rate waveform.

JP-A-10-142053 JP 2002-335538 A

Roy S. Berns, Lawrence A. Taplin, Mahdi Nezamabadi, Mahnaz Mohammadi and Yonghui Zhao, “Spectral imaging using a commercial color-filter array digital camera”, in proc. 14th Triennal ICOM-CC meeting, pp.743-750 (2005 ) Masaru Hashimoto, Junko Kishimoto, “Color reproduction with a 2-shot 6-band still image shooting system using a commercial digital camera”, Color Forum JAPAN 2007 Proceedings pp.113-116 (2007) Edited by Yoichi Miyake, Introduction to spectral image processing Chapter 4, University of Tokyo Press, 2006

  Therefore, when acquiring spectral information of a subject to be photographed using an imaging system that combines an interference filter having a wide transmission wavelength band and a digital camera as described above, it depends on the incident angle of light incident on the interference filter. Therefore, there is a problem that the estimation accuracy of the spectral reflectance decreases as the pixel position moves toward the peripheral portion.

  Accordingly, the present invention has been made to solve the above-described problems, and a simple camera multiband using an optical filter having an incident angle dependency of spectral transmittance and an imaging device such as a digital camera. And a spectral information acquisition apparatus, a spectral information acquisition method, and a spectral information acquisition program that acquire spectral information of a subject to be imaged without reducing accuracy even at the periphery of the image.

In order to solve the above problems, the invention corresponding to claim 1 is a spectral information acquisition device for acquiring spectral information to be imaged, an imaging device for photographing the photographing target, the imaging device and the imaging target an imaging means comprising an optical filter to be installed perpendicular to the optical axis of Oite the imaging device between said light to be imaged at the position of the image sensor corresponding to each pixel of the image captured by the imaging means calculating the angle of incidence on the optical filter, previously measured for each angle of incidence and stored using said spectral transmittance data of the optical filter, to calculate the spectral transmittance data corresponding to the incident angle of the previous SL light, wherein Means for calculating a spectral information estimation matrix for each pixel; and means for calculating spectral information of a position on the object to be imaged from the spectral information estimation matrix calculated by the calculation means and the pixel value of the pixel. Preparation A configuration, the incident angle to the optical filter, and the coordinate values of the image center, the coordinate values on the image of each pixel, the ratio the number of pixels of the image with respect to the length of the image sensor in a predetermined direction And the focal length of the photographing device.

  According to a second aspect of the present invention, the imaging means described in the first aspect of the present invention includes the imaging device such as a digital camera and the optical filter installed between the imaging device and the subject to be photographed. The imaging system is configured to repeat imaging while sequentially attaching or detaching one or a plurality of optical filters, and the plurality of the captured images are used as a set of image inputs.

  According to a third aspect of the present invention, the imaging means described in the first aspect of the present invention includes the imaging device such as a digital camera and the optical filter installed between the imaging device and the subject to be photographed. A plurality of imaging systems each having different spectral characteristics of the entire imaging system, and the images captured by the plurality of imaging systems are used as a set of image inputs. To do.

Further, the invention includes an imaging device for photographing a photographic subject, the imaging means comprising an optical filter to be installed perpendicular to the optical axis of Oite the imaging device during the image pickup device and the imaging object corresponding to claim 4 And a first step of acquiring the image to be imaged, and an incident angle of the light imaged at the position of the image sensor corresponding to each pixel of the image acquired in the first step to the optical filter. Using the spectral transmittance data of the optical filter calculated and stored in advance for each incident angle, spectral transmittance data corresponding to the incident angle of the light is calculated, and a spectral information estimation matrix for each pixel is calculated. A second step of calculating, and a third step of calculating spectral information of the position on the object to be imaged from the spectral information estimation matrix calculated in the second step and the pixel value of the pixel; By providing a spectral information acquisition method for acquiring spectral information of the imaging object, the incident angle to the optical filter, and the coordinate values of the image center, the coordinate values on the image of each pixel in a predetermined direction Is calculated by the ratio of the number of pixels of the image to the length of the image sensor and the focal length of the photographing device.

  In the invention corresponding to claim 5, as the first step described in the invention corresponding to claim 4, the optical device installed between the imaging device such as a digital camera and the imaging device and the imaging object is used. An imaging system configured with a filter, wherein one or a plurality of optical filters are repeatedly attached and detached between the imaging device and the subject to be imaged, and a set of these captured images is taken as a set. It is characterized by image input.

  In the invention corresponding to claim 6, as the first step described in the invention corresponding to claim 4, the optical device installed between the imaging device such as a digital camera and the imaging device and the imaging target An imaging system composed of a filter and having a plurality of imaging systems each having different spectral characteristics of the entire imaging system, and a set of image inputs with images captured by the plurality of imaging systems It is characterized by.

Furthermore, invention corresponding to claim 7 is configured by an imaging device for photographing a photographic subject, and the imaging target and the optical filter to be installed vertically to the optical axis of Oite the imaging device between the imaging device An imaging system that captures the imaging target in a computer in which spectral transmittance data of the optical filter for each incident angle of light is stored in advance and the spectral information of the imaging target is acquired and processed. An image acquisition procedure for acquiring an image of the imaging target by an imaging system including the imaging device and an optical filter installed between the imaging target, and an image sensor corresponding to each pixel of the image acquired by the image acquisition procedure wherein calculating the angle of incidence on the optical filter, previously measured every angle of incidence of the light using the stored spectral transmittance data of the optical filter was of the imaged position light Calculating a spectral transmittance data corresponding to the incident angle of the previous SL light, wherein the spectral information estimation matrix calculation procedure for calculating the spectral information estimation matrix at each pixel, the spectral information estimation calculated in the spectral information estimation matrix calculation procedure and a pixel value of the matrix and the pixel is spectral information acquisition program order to execute the spectral information acquisition processing procedure for calculating the spectral information of the position on the imaging target to be imaged, incident to said optical filter The angle depends on the coordinate value of the center of the image, the coordinate value on the image of each pixel, the ratio of the number of pixels of the image to the length of the image sensor in a predetermined direction, and the focal length of the imaging device. Calculated.

  In the invention corresponding to claim 8, the image acquisition procedure described in the invention corresponding to claim 7 includes the optical filter installed between the imaging device such as a digital camera and the imaging device and the imaging target. The imaging system is configured to repeat imaging while attaching or detaching one or a plurality of the optical filters between the imaging device and the imaging target, and a set of images including the plurality of the captured images. It is characterized by being input.

  In the invention corresponding to claim 9, the image acquisition procedure described in the invention corresponding to claim 7 includes the optical filter installed between the imaging device such as a digital camera and the imaging device and the imaging object. A plurality of imaging systems each having different spectral characteristics of the entire imaging system, and images taken by the plurality of imaging systems are used as a set of image inputs. Features.

  According to the present invention, it is possible to realize a simple multiband camera using an optical filter having an incident angle dependency of spectral transmittance and an imaging device such as a digital camera, and to reduce the accuracy in the peripheral portion of the image. It is possible to acquire the spectral information of the object to be imaged.

The block diagram which shows one Embodiment of the spectral information acquisition apparatus which concerns on this invention. The figure explaining that the incident angle to an interference filter changes with pixel positions. The figure explaining the geometric relationship of the imaging system which combined the interference filter and the digital camera. The figure explaining the relationship between a pixel position and the position imaged on the actual image sensor. The flowchart explaining the process sequence of the spectral information acquisition method and spectral information acquisition program which concern on this invention. The figure explaining the principle of the modulation | alteration of the spectral sensitivity characteristic of the imaging system by an interference filter.

  Embodiments of the present invention will be described below with reference to the drawings. In this embodiment, an example in which spectral reflectance is acquired as spectral information of an imaging target acquired by the device of the present invention will be described.

  FIG. 1 is a block diagram showing an embodiment of a spectral information acquisition apparatus according to the present invention. In FIG. 1, reference numeral 1 denotes a subject that is a subject to be obtained as a spectral reflectance acquisition target, reference numeral 2 denotes a photographing light source that emits light toward the subject 1, and reference numeral 10 denotes a spectral reflectance acquisition device that acquires the spectral reflectance of the subject 1. It is.

  The spectral reflectance acquisition device 10 includes an interference filter 11, a filter attaching / detaching device 12, an RGB digital camera 13, and a computer 14.

  The interference filter 11 is arranged between the subject 1 and the RGB digital camera 13, and constitutes an imaging system in combination with the RGB digital camera 13, while having a function of modulating the spectral sensitivity of the imaging system. Yes. The spectral transmittance of the interference filter 11 is measured for each incident angle, and the measured spectral transmittance data is stored in the filter transmittance data table 151 of the external storage device 15 constituting a part of the computer 14.

  As the external storage device 15, in addition to the filter transmittance data table 151, the spectral reflectance / noise storage unit 152 that stores the measured spectral reflectance / noise correlation matrix data, and the spectral sensitivity that is known in advance of the RGB digital camera 13. A camera spectral sensitivity data storage unit 153 that stores data, a light source spectral data storage unit 154 that stores spectral distribution data of the imaging light source 2 that is known in advance, and a shared data storage unit 155 that stores other necessary data are provided. .

  The filter attaching / detaching device 12 is a device that detachably arranges the interference filter 11 in front of the RGB digital camera 13 by moving the interference filter 11 forward and backward.

  The RGB digital camera 13 captures and acquires an image to be captured. The spectral sensitivity of each channel of the camera 13 is known, and is stored in the camera spectral sensitivity data storage unit 153 in advance as described above. Yes.

  The computer 14 controls the overall processing operation of the spectral reflectance acquisition apparatus 10 and executes desired data processing, and is constituted by a personal computer or the like.

  In addition to the external storage device 15 described above, the computer 14 includes a program storage unit 16 as a main storage device including a ROM that stores a spectral information acquisition program, and general image processing of captured images, as well as spectral information. Consists of an image processing unit 17 configured by a CPU that executes spectral information acquisition processing by the acquisition program, and a CPU that controls the photographing operation of the filter attaching / detaching device 12 and the RGB digital camera 13 according to the spectral information acquisition program. And a device control unit 18.

  Next, the principle or acquisition operation of spectral information in the spectral information acquisition apparatus configured as described above will be described. Here, an example will be described in which the spectral reflectance of the subject 1 that is the spectral reflectance acquisition target is acquired in units of pixels using the spectral reflectance acquisition device 10.

  For simplicity of explanation, it is assumed that the response characteristics of each channel of the RGB digital camera 13 are linear. Further, it is assumed that the interference filter 11 is arranged perpendicular to the optical axis of the RGB digital camera 13. Further, it is assumed that the subject 1 and the spectral reflectance acquisition apparatus 10 are sufficiently separated from each other. Hereinafter, the spectral information is handled as discretized data, and is represented by a vector or a matrix.

Now, the spectral reflectance of a point on the object 1 r _i, E the spectral distribution of the imaging light source 2, the spectral sensitivity S _n of the RGB digital camera 13, when the spectral transmittance of the interference filter 11 is T, lower, respectively It can be expressed by a formula.

Here, the superscript T on the matrix represents transposition. Subscript i represents the pixel number, and N represents the number of wavelength samplings. For example, when spectroscopic information is sampled at intervals of 380 nm to 730 nm and 10 nm, N = 36. In the spectral distribution E of the photographic light source 2, diagonal components e ₁ , e ₂ ,..., E _N represent the spectral intensities of the respective wavelengths, and the other components are zero. Each column of spectral sensitivity S _n of the RGB digital camera 13 represents R, G, the spectral sensitivity of each channel of the B. With respect to the spectral transmittance T of the interference filter 11, diagonal components t ₁ , t ₂ ,..., T _N represent the spectral transmittance of each wavelength, and the other components are zero.

When configuring the interference filter 11 and the imaging system by combining the RGB digital camera 13, the spectral sensitivity S _t of the imaging system during the filter mounting it can be calculated by (5) below.

Here, when a total of 6 channels of captured images are obtained by repeating the imaging twice with and without the filter, the spectral sensitivities of the entire imaging system for 6 channels are summarized as follows (6 ) can be expressed by the formula of S _c.
S _c = [S _n S _t ] (6)
In the case of a multiband imaging system in which the interference filter 11 and the RGB digital camera 13 are combined, the pixel value obtained with the image number i can be modeled by the following equation (7).
g _{i =} S _c ^T E r _i (7)
In the above equation, g _i represents a pixel value, and the pixel value g _i can be represented.
g _{i =} [g _i1 g _i2 g _i3 g _i4 g _i5 g _i6] ^T (8)
In the equation (8), the first, second and third components are the pixel values of the R, G and B channels when the filter is not attached, and the fourth, fifth and sixth components are R and G when the filter is attached. , B represents the pixel value of each channel.

Thus, the estimation of the spectral reflectance corresponds to solving the inverse problem of the model of Equation (7) and obtaining the unknown r _i from g _{i. When} solving this by linear calculation, the following equation (9) is obtained. An estimated spectral reflectance can be obtained.

  Here, a conversion matrix W (hereinafter referred to as a spectral reflectance estimation matrix) from pixel values to spectral reflectance has a (6 × 36) dimension, and a solution that satisfies this is not unique.

However, since it is known that many spectral reflectances in the natural world can be represented by a linear sum of low-dimensional (about 3-9 dimensional) basis functions, a spectral reflectance estimation matrix is utilized using this fact. Various methods for deriving W have been proposed. Here, when the Wiener estimation method is used as a method for obtaining the spectral reflectance estimation matrix W, the spectral reflectance estimation matrix W can be obtained by the following equation (10).

Here, in equation (10), R _r is the correlation matrix of the spectral reflectance to be estimated, the R _n is the correlation matrix of the image noise of the imaging system. R _r can be obtained in advance by collecting a large number of actually measured samples of spectral reflectance data belonging to the category to be estimated and calculating with a correlation matrix. Noise R _n can be measured in advance by imaging a uniform white object and regarding variations in pixel values as noise.

  Accordingly, when the spectral reflectance is estimated based on the inverse problem of the camera imaging model of equation (7) as described above, the spectral sensitivity S of the RGB digital camera 13, the spectral distribution E of the imaging light source 2, and the interference filter 11 The spectral transmittance T needs to be known. It is also known that the measurement accuracy affects spectral reflectance estimation.

Incidentally, the spectral transmittance T of the interference filter 11 has a characteristic that it depends on the incident angle of light. That is, the wavelength shift amount λ (θ) of the spectral transmittance T when light is incident at an angle of θ [degree] from the optical axis can be obtained by an approximate expression of the expression (11) (where the incident angle is 10 degrees). In the following cases):

Here, n _eff is a characteristic determined by the configuration of the thin film of the interference filter 11 called the effective refractive index.

On the other hand, when the interference filter 11 is installed in front of the RGB digital camera 13 and picked up, each pixel of the digital camera 13 passes through the interference filter 11 for light that forms an image at the position of the image sensor corresponding to the pixel. The incident angle is different. As shown in FIG. 2, when the interference filter 11 is installed perpendicular to the optical axis of the RGB digital camera 13, the incident angles θ ₁ , θ ₂ , and θ ₃ to the interference filter 11 increase concentrically from the image center. It becomes. Here, it is assumed that the distance between the RGB digital camera 13 and the subject 1 is sufficiently long and the incident angle can be represented by light passing through the lens center.

  Therefore, due to the characteristics of the interference filter 11 as described above, the spectral reflectance T is estimated from the multiband image based on the equation (10) using only one of the spectral transmittances T at the time of 0 degree incidence. In this case, since a modeling error due to the wavelength shift of the interference filter 11 occurs, there arises a problem that the estimation accuracy of the spectral reflectance decreases from the center of the image toward the peripheral portion as described above.

  Therefore, in the device of the present invention, by creating a spectral reflectance estimation matrix by changing the filter spectral transmittance T used for each pixel, a modeling error is reduced, and a high spectral reflectance is obtained even in the peripheral portion of the image. The estimation accuracy can be realized and will be described with reference to FIGS.

  FIG. 3 is a schematic diagram showing the relationship between the pixel position and the incident angle θ to the interference filter 11 when the subject 1 is imaged by the digital camera 13, and FIG. 4 shows the pixel position and the position imaged on the actual image sensor. It is a figure explaining the relationship.

  First, the incident angle θ of light to the interference filter 11 corresponding to a certain pixel is determined. Here, it is assumed that the interference filter 11 is installed perpendicular to the optical axis of the RGB digital camera 13.

Now, assuming that the coordinates of the pixel i on the image are (Xi, Yi), there is a one-to-one correspondence with the position (x _i , y _i ) of the camera 13 on the image sensor. When the coordinate value of the image center is (X _c , Y _c ), the radius r _i from the sensor center on the image sensor corresponding to the radius Ri from the image center can be expressed by the following equation (12).

In the above equation, W is the horizontal length of the image sensor, and W is the number of horizontal pixels of the image. Here, W and W may be the vertical length of the image sensor and the number of vertical pixels of the image, respectively. At this time, the incident angle θ to the interference filter 11 can be calculated by the following equation (13).
_{θ = arctan (r i / f} ) ...... (13)
In the above equation, f indicates the focal length of the lens of the RGB digital camera 13. If the incident angle θ to the interference filter 11 is known, the spectral reflectance of the pixel is calculated by calculating the spectral transmittance of the interference filter 11 corresponding to θ from the spectral transmittance data for each incident angle measured in advance. The matrix W can be derived based on the equation (10).

  Next, an embodiment of a spectral information acquisition method and a spectral information acquisition program according to the present invention will be described with reference to FIG.

  FIG. 5 is a flowchart showing a series of processing procedures for obtaining the spectral reflectance (spectral information). The spectral information acquisition program is always stored in the program storage unit 16, and the apparatus control unit 18 or the image processing unit 17 is activated during execution of a series of processes. The spectral information acquisition program is read from the program storage unit 16 and loaded. The program is stored in, for example, the shared data storage unit 155 of the external storage device 15 that is a memory, and the spectral information acquisition program is executed. Note that when executing the spectral information acquisition program, the device control unit 18 and the image processing unit 17 cooperate with each other and proceed with a series of processes shown in FIG.

  First, the computer 14 arranges the interference filter 11 between the subject 1 to be imaged and the digital camera 13 as an imaging device, and uses an imaging system including the interference filter 11 and the RGB digital camera 13. Then, an image acquisition procedure for acquiring an image from the subject 1 is executed (S1, S2).

  As this image acquisition procedure, for example, after irradiating the subject 1 with light from the photographing light source 2, the device control unit 1 controls the filter attaching / detaching device 12 in a predetermined order. This is a procedure for issuing a shooting instruction to the camera 13 and imaging the subject 1.

  That is, the device control unit 18 issues a control instruction for attaching and not attaching the interference filter 11 to the filter attaching / detaching device 12, and sends a photographing instruction to the RGB digital camera 13 after the interference filter 11 is attached and detached. By repeating, so-called multiband imaging is performed, in which two images are obtained when the filter is attached and when the filter is not attached (S1).

The RGB digital camera 13 transfers a set of two multiband images captured as described above to the image processing unit 17 of the computer 14 (S2).
Here, the image processing unit 17 repeats the same processing as many times as the number of pixels to be estimated of the image received from the RGB digital camera 13 (S3-S8), thereby performing the spectral information estimation matrix calculation processing procedure (S3) for the pixels. -S6) and spectral information acquisition processing procedure (S7) are executed.

  The image processing unit 17 calculates the incident angle θ of the light incident on the target pixel i to the interference filter 11 based on the equations (12) and (13) (S4), and then the spectral transmittance of the interference filter 11 T is calculated (S5).

  That is, since the known spectral transmittance data for each incident angle of the interference filter 11 is stored in the filter transmittance data table 151 in advance, the image processing unit 17 is based on the incident angle θ calculated in step S4. With reference to the filter transmittance data table 151, spectral transmittance data of the interference filter 11 at the calculated incident angle θ is calculated (S5). For example, since the spectral transmittance data measured from the incident angle θ = 0 of the interference filter 11 in increments of 1 degree is stored in the filter transmittance data table 151, linear interpolation between the filter transmittance data tables 151 or the like is performed. Spectral transmittance data corresponding to the corresponding incident angle is calculated by the processing.

  Subsequently, the image processing unit 17 uses the spectral transmittance data calculated in step S5 to calculate the spectral reflectance estimation matrix W using the equation (10) (S6).

Further, the image processing unit 17 executes a spectral information acquisition processing procedure (S7). In this spectral information acquisition processing procedure, the spectral reflectance r _i is estimated from the spectral reflectance estimation matrix W calculated in step S6 and the pixel value g _{i of the} corresponding pixel according to the equation (9).

  Thereafter, the above-described series of processing steps (S3 to S7) are executed for all the estimation target pixels (S8), and the process ends.

  Therefore, according to the embodiment as described above, when the interference filter 11 and the RGB digital camera 13 are combined to make a multi-band and the spectral reflectance of the subject 1 is acquired, the spectral transmittance of the interference filter 11 is obtained. In consideration of the incident angle dependence, the spectral reflectance is calculated from the spectral reflectance estimation matrix and the pixel value of the corresponding pixel, so the accuracy of the spectral reflectance estimation is reduced without degrading toward the periphery of the image. Spectral information about the subject can be acquired.

(Other embodiments)
The above-described embodiment shows an example of the present invention, and the present invention is not limited to this, and appropriate changes or modifications are made without departing from the scope of the present invention. Also good.

(1) For example, in the above-described embodiment, the spectral reflectance of the subject is used as the acquired spectral information. However, the present invention is not limited to this, and other spectral information, for example, the spectral intensity distribution of the shooting scene may be used. For example, if the spectral distribution of the photographic light source 2 is known, the spectral intensity distribution of the photographic scene is obtained by multiplying the spectral reflectance obtained by the equation (9) by the spectral intensity distribution E of the photographic light source 2. Can do. On the other hand, if the spectral distribution of the imaging light source 2 is unknown, the correlation matrix used for the spectral information estimation matrix (here, the spectral intensity distribution estimation matrix) based on the Wiener estimation derived by the above equation (4) is used as the spectral of various light sources. The distribution may be sampled and calculated.

(2) In the above-described embodiment, an example in which the Wiener estimation method is used as the method for deriving the spectral information estimation matrix has been described. However, the present invention is not limited to this. For example, a method based on principal component analysis, etc. Even if the formula is changed to another estimation matrix calculation formula, the spectral information estimation matrix may be calculated in consideration of the incident angle dependency of the optical filter for each pixel.

(3) In the embodiment described above, the spectral transmittance of the interference filter 11 is stored in the filter transmittance data table 151, and the spectral transmittance is calculated based on this. Based on the above, the shift amount of the filter spectral transmittance for each pixel may be calculated to obtain spectral transmittance data.

(4) In the above-described embodiment, the spectral information estimation matrix is recalculated for each pixel. However, the present invention is not limited to this. For each filter incident angle, (10) Spectral reflectance estimation matrix is calculated based on the formula, and this is stored in advance in the external storage device 15 constituting the computer 14 as cache data, and the cache data already stored is referred to when estimating for each pixel. By doing so, the spectral reflectance estimation matrix may be obtained.

(5) Further, in the above-described embodiment, the number of the imaging system including the interference filter 11 and the digital camera 13 is one. However, the present invention is not limited to this, and the spectral characteristics of the entire imaging system are different. A plurality of such imaging systems may be provided, and multibanding may be performed using a plurality of images obtained by imaging from these to obtain the spectral reflectance of the subject 1. In addition, when there is no plural imaging system, and it is necessary to obtain the spectral reflectance of a material capable of estimating the spectral reflectance from the color information of the three channels without attaching / detaching the filter 11 in the imaging system. However, this method can be applied when an optical filter such as the interference filter 11 whose transmittance depends on the incident angle is incorporated in the imaging system.

(6) Furthermore, in the above-described embodiment, the interference filter 11 is installed perpendicular to the optical axis of the RGB digital camera 13, but the present invention is not limited to this. If there is a means capable of identifying the angle formed with the interference filter 11, the spectral transmittance of the interference filter 11 corresponding to the angle may be calculated to obtain a spectral reflectance estimation matrix.

(7) Furthermore, in the above-described embodiment, the interference filter 11 is used as the optical filter. However, the present invention is not limited to this, and there is an incident angle dependency of transmittance such as a liquid crystal tunable filter. Applicable when general filter is used.

(8) Further, in the above-described embodiment, the subject 1 is sufficiently far and only the angle formed by the light beam passing through the center of the lens and the interference filter 11 is used. However, the present invention is not limited to this. Is close to the spectral reflectance acquisition device 10, and even when light of various filter incident angles is collected on the pixel by the lens, the expected value of the filter incident angle of the collected light is determined from the relationship between the aperture and the subject distance. The incident angle may be represented by it.

(9) Furthermore, in the above-described embodiment, the response characteristics of the RGB digital camera 13 are linear for each channel. However, the present invention is not limited to this, and the response characteristics of the RGB digital camera 13 are identified in advance. In addition, if this inverse function is linearized by applying it to the acquired pixel value, this method can be applied.

  DESCRIPTION OF SYMBOLS 1 ... Subject, 2 ... Imaging light source, 10 ... Spectral reflectance acquisition device, 11 ... Interference filter, 12 ... Interference filter attaching / detaching device, 13 ... RGB digital camera, 14 ... Computer, 15 ... External storage device, 16 ... Program storage part , 17: Image processing unit, 18: Device control unit, 151: Filter transmittance data table, 152: Spectral reflectance / noise storage unit, 153: Camera spectral sensitivity data storage unit, 154: Light source spectral data storage unit.

Claims (9)

A spectral information acquisition device that acquires spectral information of an imaging target,
An imaging device for photographing the photographing target, an imaging device consisting of an optical filter to be installed perpendicular to the optical axis of Oite the imaging device during the image acquisition device and the imaging target,
The incident angle to the optical filter of the light imaged at the position of the image sensor corresponding to each pixel of the image picked up by the image pickup means is calculated, and measured and stored in advance for each incident angle of the optical filter. Means for calculating spectral transmittance data corresponding to the incident angle of the light using the spectral transmittance data, and calculating a spectral information estimation matrix in each pixel;
And a pixel value of the spectral information estimation matrix and pixel calculated by the calculation means, a spectral information acquisition device Ru and means for calculating the spectral information of the position on the imaging target to be imaged,
The incident angle to the optical filter includes the coordinate value of the center of the image, the coordinate value on the image of each pixel, the ratio of the number of pixels of the image to the length of the image sensor in a predetermined direction, and the photographing device. And a focal length of the spectral information acquisition device.   The imaging means is an imaging system including the imaging device such as a digital camera and the optical filter installed between the imaging device and the subject to be imaged, and one or a plurality of the optical filters are included. 2. The spectral information acquisition apparatus according to claim 1, wherein imaging is repeated while sequentially attaching and detaching, and the plurality of captured images are used as a set of image inputs.   The image pickup means is an image pickup system including the image pickup device such as a digital camera and the optical filter installed between the image pickup device and the object to be photographed, and the spectral characteristics of the entire image pickup system are different. The spectral information acquisition apparatus according to claim 1, further comprising: a plurality of imaging systems, wherein the images captured by the plurality of imaging systems are used as a set of image inputs. An imaging device for photographing a photographic subject, the first acquiring an image of the imaging target by the imaging means comprising an optical filter to be installed perpendicular to the optical axis of Oite the imaging device during the image pickup device and the imaging target And the steps
The incident angle to the optical filter of the light imaged at the position of the image sensor corresponding to each pixel of the image acquired in the first step is calculated, and the optical measured and stored in advance for each incident angle A second step of calculating spectral transmittance data corresponding to an incident angle of the light using spectral transmittance data of the filter, and calculating a spectral information estimation matrix in each pixel;
By providing the third step of calculating spectral information of the position on the imaging target to be imaged from the spectral information estimation matrix calculated in the second step and the pixel value of the pixel, A spectral information acquisition method for acquiring spectral information,
The incident angle to the optical filter includes the coordinate value of the center of the image, the coordinate value on the image of each pixel, the ratio of the number of pixels of the image to the length of the image sensor in a predetermined direction, and the photographing device. And a focal length of the spectral information.   The first step is an imaging system including the imaging device such as a digital camera and the optical filter installed between the imaging device and the imaging target, and one or a plurality of the optical devices 5. The spectral information acquisition method according to claim 4, wherein imaging is repeated while attaching and detaching a filter between the imaging device and the imaging target, and the plurality of captured images are used as a set of image inputs.   The first step is an imaging system including the imaging device such as a digital camera and the optical filter installed between the imaging device and the imaging target. 5. The spectral information acquisition method according to claim 4, wherein a plurality of different image pickup systems are provided, and images captured by the plurality of image pickup systems are used as a set of image inputs. An imaging device for photographing a photographic subject, an imaging system is provided comprised of said shooting target and the optical filter to be installed vertically to the optical axis of Oite the imaging device between the imaging device, and advance the light Spectral transmittance data of the optical filter for each incident angle is stored, and the computer for acquiring and processing the spectral information of the imaging target,
An image acquisition procedure for acquiring an image of the imaging target by an imaging system including an imaging device that captures the imaging target and an optical filter installed between the imaging device and the imaging target;
The incident angle to the optical filter of the light imaged at the position of the image sensor corresponding to each pixel of the image acquired by this image acquisition procedure is calculated, and measured and stored in advance for each incident angle of the light. Spectral transmittance data corresponding to the incident angle of the light is calculated using spectral transmittance data of the optical filter, and a spectral information estimation matrix calculation procedure for calculating a spectral information estimation matrix in each pixel;
And a pixel value of the spectral information estimation matrix and pixel calculated by the spectral information estimation matrix calculation procedure, the order to execute the spectral information acquisition processing procedure for calculating the spectral information of the position on the imaging subject in the imaging A program for obtaining spectral information,
The incident angle to the optical filter includes the coordinate value of the center of the image, the coordinate value on the image of each pixel, the ratio of the number of pixels of the image to the length of the image sensor in a predetermined direction, and the photographing device. Spectral information acquisition program calculated by the focal length of .   The image acquisition procedure is an imaging system including the imaging device such as a digital camera and the optical filter installed between the imaging device and the imaging target, and one or a plurality of the optical filters 8. The spectral information acquisition program according to claim 7, wherein imaging is repeated while attaching and detaching between the imaging device and the imaging target, and the plurality of captured images are used as a set of image inputs. The image acquisition procedure includes an imaging system including the imaging device such as a digital camera and the optical filter installed between the imaging device and the imaging target, and each has a different spectral characteristic. The spectral information acquisition program according to claim 7, further comprising: a plurality of such imaging systems, wherein the images captured by the plurality of imaging systems are used as a set of image inputs.

Images