JP2016090291A - Imaging apparatus, spectroscopy system and spectroscopy method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent occurrence of coma aberration and deterioration in resolution.SOLUTION: A spectroscopy element P is constituted of at least two prism P1 and P2 which are bonded so that the light incident surface and light emission surface are generally parallel with each other. The two prisms P1 and P2 have the same refraction index with respect to a beam of light having a specific wavelength but have the Abbe number different from each other with respect to a beam of light having a specific wavelength.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an imaging apparatus, a spectroscopic system, and a spectroscopic method using the same for acquiring a spectroscopic image.

  By utilizing spectral information of a large number of bands (for example, several tens of bands or more) each having a narrow band, it is possible to grasp detailed physical properties of an observation object that is impossible with a conventional RGB image. A camera that acquires multi-wavelength information is called a “hyperspectral camera”. Hyperspectral cameras are used in various fields such as food inspection, biological inspection, drug development, and component analysis of minerals.

  As an example of using an image acquired by limiting the wavelength of an observation target to a narrow band, Patent Document 1 discloses an apparatus that discriminates between a tumor site and a non-tumor site of a subject. This apparatus detects that protoporphyrin IX accumulated in cancer cells emits fluorescence of 635 nm and photo-protoporphyrin emits fluorescence of 675 nm by irradiation of excitation light. Thereby, a tumor site | part and a non-tumor site | part are identified.

  Patent Document 2 discloses a method of determining the freshness of a fresh food that decreases with the passage of time by acquiring information on the reflectance characteristics of continuous multi-wavelength light.

Hyperspectral cameras that can measure multi-wavelength images or reflectivity can be broadly classified into the following four methods.
(A) Line sensor method (b) Electronic filter method (c) Fourier transform method (d) Interference filter method

  (A) In the line sensor method, one-dimensional information of an object is acquired using a member having a line-shaped slit. The light that has passed through the slit is separated according to the wavelength by a spectroscopic element such as a diffraction grating or a prism. The separated light for each wavelength is detected by an image sensor (image sensor) having a plurality of pixels arranged two-dimensionally. In this method, since only one-dimensional information of the measurement object can be obtained at a time, two-dimensional spectrum information is obtained by scanning the entire camera or the measurement object perpendicularly to the slit direction. The line sensor method has an advantage that a high-resolution multi-wavelength image can be obtained. Patent Document 3 discloses an example of a hyperspectral camera of a line sensor system.

  (B) The electronic filter method includes a method using a liquid crystal tunable filter (Liquid Crystal Tunable Filter: LCTF) and a method using an acousto-optic tunable filter (AOTF). The liquid crystal tunable filter is an element in which a linear polarizer, a birefringent filter, and liquid crystal cells are arranged in multiple stages. Only light of an arbitrary specific wavelength can be extracted by removing light of an unnecessary wavelength only by voltage control. The acoustooptic element is composed of an acoustooptic crystal to which a piezoelectric element is bonded. When an electrical signal is applied to the acousto-optic crystal, ultrasonic waves are generated, and a dense standing wave is formed in the crystal. Only light of any specific wavelength can be extracted by the diffraction effect thereby. This method has an advantage that high-resolution moving image data can be acquired although the wavelength is limited.

  (C) The Fourier transform method uses the principle of a two-beam interferometer. The light beams from the measurement object are branched by a beam splitter, and the respective light beams are reflected by a fixed mirror and a moving mirror, recombined, and then observed by a detector. By changing the position of the moving mirror with time, it is possible to acquire data indicating the intensity change of interference depending on the wavelength of light. Spectral information is obtained by Fourier transforming the obtained data. The advantage of the Fourier transform method is that multi-wavelength information can be acquired simultaneously.

  (D) The interference filter method is a method using the principle of a Fabry-Perot interferometer. A configuration is used in which an optical element having two highly reflective surfaces separated by a predetermined distance is arranged on the sensor. The distance between the two surfaces of the optical element is different for each region, and is determined so as to match the interference condition of light of a desired wavelength. The interference filter method has an advantage that multi-wavelength information can be acquired simultaneously and in a moving image.

  In addition to these methods, there is also a method using compressed sensing as disclosed in Patent Document 4, for example. The apparatus disclosed in Patent Document 4 separates light from a measurement target with a first spectroscopic element such as a prism, marks it with an encoding mask, and further returns the path of the light beam with the second spectroscopic element. Thereby, an image encoded and multiplexed with respect to the wavelength axis is acquired by the sensor. A plurality of images with multiple wavelengths can be reconstructed from the multiplexed images by applying compressed sensing.

  Compressed sensing is a technique for restoring more data from acquired data with a small number of samples. If the two-dimensional coordinates of the measurement target are (x, y) and the wavelength is λ, the data f to be obtained is three-dimensional data of x, y, λ. On the other hand, the image data g obtained by the sensor is two-dimensional data compressed and multiplexed in the λ axis direction. The problem of obtaining data f having a relatively large amount of data from acquired image g having a relatively small amount of data is a so-called defect setting problem and cannot be solved as it is. However, in general, natural image data has redundancy, and this defect setting problem can be converted into a good setting problem by skillfully using it. An example of a technique for reducing the amount of data by utilizing image redundancy is jpeg compression. In the jpeg compression, a method is used in which image information is converted into frequency components, and non-essential portions of data, for example, components with low visual recognition are removed. In compressed sensing, such a technique is incorporated into arithmetic processing, and a data space to be obtained is converted into a space represented by redundancy, thereby reducing unknowns and obtaining a solution. For this conversion, for example, discrete cosine transform (DCT), wavelet transform, Fourier transform, total variation (TV) or the like is used.

International Publication No. 13/002350 JP 2007-108124 A JP 2011-89895 A US Pat. No. 7,283,231

  In a conventional hyperspectral camera using compressed sensing, a spectral element such as a prism is inserted in the optical path. For this reason, the coma aberration generate | occur | produced and the subject that the resolution fell occurred.

  The present disclosure provides a new imaging technique that can suppress the occurrence of coma aberration and the accompanying decrease in resolution.

  A spectroscopic element according to an aspect of the present disclosure is a spectroscopic element configured by joining at least two prisms such that a light incident surface and a light output surface are substantially parallel to each other, and the two prisms are specified. The refractive indexes of the light beams having the same wavelengths are substantially the same, and the Abbe numbers of the light beams having the specific wavelengths are different.

  An imaging device according to another aspect of the present disclosure is disposed on an optical path of light incident from an object, has a plurality of regions having a first light transmittance, and a second lower than the first light transmittance. A plurality of regions having a plurality of light transmittances, a spectroscopic element that is disposed on an optical path of light that has passed through the encoding element, and disperses the light according to a wavelength, and passes through the spectroscopic element And an image sensor that acquires an image on which components of a plurality of wavelength bands of light dispersed by the spectroscopic element are superimposed. The spectroscopic element is configured by joining at least two prisms such that a light incident surface and a light exit surface are substantially parallel, and the two prisms have substantially the same refractive index of light of a specific wavelength, And the Abbe number about the said specific wavelength light differs, and has a substantially parallel light-incidence surface and light-projection surface.

  According to the present disclosure, it is possible to suppress the occurrence of coma aberration and the accompanying decrease in resolution.

It is a perspective view showing typically spectroscopic element P of Embodiment 1 of this indication. It is a figure which shows the structural example of 3 steps | paragraphs which bonded together 3 prisms P1-P3. It is a figure which shows the structural example of 4 steps | paragraphs which bonded together 4 prisms P1-P4. It is a schematic diagram which shows the imaging device in Embodiment 2 of this indication. It is a front view when coding element C in Embodiment 2 of this indication is seen from the subject side, (a) shows a coding pattern with binary distribution of transmission and reflection, and (b). The encoding pattern which has the transmittance | permeability distribution of a gray scale is shown. 12 is a flowchart illustrating an overview of a spectroscopic method according to a second embodiment of the present disclosure. It is a schematic diagram which shows the imaging device in Embodiment 3 of this indication. It is a figure which shows an example of the imaging state on the image pick-up surface of the image pick-up element S in Example 1 of this indication. (A) is a schematic cross-sectional view showing an example of a light beam that passes through the spectroscopic element P and reaches the image sensor S, (b) is a lateral aberration diagram, and (c) is a spot diagram. It is a figure which shows an example of the image formation state on the imaging surface of the image pick-up element S in the comparative example 1 of this indication. (A) is a schematic cross-sectional view showing an example of a light beam that passes through the spectroscopic element P and reaches the image sensor S, (b) is a lateral aberration diagram, and (c) is a spot diagram. It is a figure which shows MSE (mean square error) between the spectral separation image produced | generated in each of Example 1 and four comparative examples of this indication, and a correct image.

  Prior to describing the embodiments of the present disclosure, knowledge found by the present inventors will be described.

  According to the study of the present inventor, the conventional hyperspectral camera described above has the following problems. (A) The line sensor method needs to scan a camera in order to obtain a two-dimensional image, and is not suitable for taking a moving image of a measurement target. (C) The Fourier transform method is also not suitable for moving image shooting because it is necessary to move the reflecting mirror. (B) Since the electronic filter method acquires images one wavelength at a time, multi-wavelength images cannot be acquired simultaneously. (D) In the interference filter system, the number of wavelength bands from which an image can be acquired and the spatial resolution are traded off, and thus, when acquiring a multi-wavelength image, the spatial resolution is sacrificed. Thus, there is no existing hyperspectral camera that satisfies the three requirements of high resolution, multiple wavelengths, and moving image shooting (one-shot shooting) at the same time.

  At first glance, the configuration using compressed sensing seems to be able to satisfy high resolution, multiple wavelengths, and video shooting at the same time. However, since the image is originally reconstructed based on estimation from a small amount of data, the spatial resolution of the acquired image tends to be lower than that of the original image. In particular, the higher the compression rate of acquired data, the more pronounced the effect. Furthermore, since a spectroscopic element such as a prism is inserted in the optical path, there is a problem that coma aberration occurs and resolution is lowered.

  The present inventor has found the above-mentioned problems and studied a configuration for solving these problems. The inventor has paid attention to the fact that coma aberration can be suppressed and the resolution can be improved by configuring a spectroscopic element by appropriately combining a plurality of different materials. According to an embodiment of the present disclosure, three requirements of high resolution, multiple wavelengths, and moving image shooting (one-shot shooting) can be satisfied at the same time. In the embodiment of the present disclosure, the information in the wavelength direction is compressed among the three-dimensional information in the x direction, the y direction, and the wavelength direction. Therefore, it is only necessary to hold two-dimensional data, and the amount of data can be suppressed. For this reason, the embodiment of the present disclosure is effective for long-term data acquisition.

  The present disclosure includes an imaging device, a system, and a method described in the following items.

（Φ（ｆ）は正則化項、τは重み係数）
の式に基づいて推定されるベクトルｆ’を、前記波長帯域ごとの複数の画像として生成する、
項目７から９のいずれかに記載の撮像装置。
［項目１１］
前記信号処理回路は、前記波長帯域ごとの複数の画像を動画像として生成する、項目７から１０のいずれかに記載の撮像装置。
［項目１２］
項目２から６のいずれかに記載の撮像装置と、
前記画像と、前記符号化素子における光透過率の空間分布とに基づいて、前記少なくとも１つの分光素子を通過した光の波長帯域ごとの複数の画像を生成する信号処理装置と、
を備える分光システム。
［項目１３］
対象物から入射する光の光路上に配置され、第１の光透過率を有する複数の領域と、前記第１の光透過率よりも低い第２の光透過率を有する複数の領域とを有する符号化素子を用いて入射光の強度を変調させるステップと、
前記符号化素子を通過した光を、分光素子を用いて波長に応じて分散させ、撮像素子の撮像面上に像を形成するステップであって、前記分光素子は、光入射面および光出射面が略平行になるように少なくとも２つのプリズムを接合することによって構成され、前記２つのプリズムは、特定の波長の光の屈折率が略一致し、かつ、前記特定の波長の光についてのアッベ数が異なっている、ステップと、
前記分光素子を通過した光の複数の波長帯域の成分が重畳した画像を取得するステップと、
前記画像と、前記符号化素子における光透過率の空間分布とに基づいて、前記分光素子を通過した光の波長帯域ごとの複数の画像を生成するステップと、
を含む分光方法。 [Item 1]
A spectroscopic element configured by joining at least two prisms such that a light incident surface and a light output surface are substantially parallel,
The two prisms have substantially the same refractive index of light of a specific wavelength and different Abbe numbers for the light of the specific wavelength.
Spectroscopic element.
[Item 2]
Item 2. The spectroscopic element according to Item 1, wherein the joint surface of the two prisms is planar.
[Item 3]
A plurality of regions disposed on an optical path of light incident from the object and having a first light transmittance; and a plurality of regions having a second light transmittance lower than the first light transmittance. An encoding element;
A spectroscopic element that is disposed on an optical path of light that has passed through the encoding element and disperses the light according to a wavelength;
An image sensor that is arranged on an optical path of light that has passed through the spectroscopic element and acquires an image in which components of a plurality of wavelength bands of light dispersed by the spectroscopic element are superimposed;
With
The spectroscopic element is configured by joining at least two prisms such that a light incident surface and a light exit surface are substantially parallel, and the two prisms have substantially the same refractive index of light of a specific wavelength, And the Abbe number for the light of the specific wavelength is different, and has a substantially parallel light incident surface and light exit surface,
Imaging device.
[Item 4]
Item 4. The imaging device according to Item 3, wherein the joint surface of the two prisms is planar.
[Item 5]
A first optical system disposed between the object and the encoding element and focusing light from the object on a surface of the encoding element;
A second optical system that is disposed between the encoding element and the spectroscopic element and focuses light that has passed through the encoding element on an imaging surface of the imaging element;
Item 5. The imaging device according to Item 3 or 4, further comprising:
[Item 6]
5. The imaging apparatus according to item 3 or 4, further comprising an optical system that is disposed between the encoding element and the spectroscopic element and focuses light that has passed through the encoding element on an imaging surface of the imaging element.
[Item 7]
Items 3 to 6, further comprising a signal processing circuit that generates a plurality of images for each wavelength band of light that has passed through the spectroscopic element based on the image and a spatial distribution of light transmittance in the encoding element. The imaging device according to any one of the above.
[Item 8]
8. The imaging device according to item 7, wherein the signal processing circuit generates a plurality of images for each wavelength by a statistical method.
[Item 9]
The imaging device according to item 7 or 8, wherein the number of data in the plurality of images for each wavelength band of light is larger than the number of data in the image acquired by the imaging device.
[Item 10]
The signal processing circuit is determined based on a vector g having elements of signal values of a plurality of pixels in the image acquired by the imaging element, a spatial distribution of light transmittance in the encoding element, and spectral characteristics of the spectral element. Using the matrix H

(Φ (f) is a regularization term, τ is a weighting factor)
A vector f ′ estimated based on the equation is generated as a plurality of images for each wavelength band.
The imaging device according to any one of items 7 to 9.
[Item 11]
The imaging device according to any one of items 7 to 10, wherein the signal processing circuit generates a plurality of images for each wavelength band as moving images.
[Item 12]
The imaging device according to any one of items 2 to 6,
A signal processing device that generates a plurality of images for each wavelength band of light that has passed through the at least one spectroscopic element based on the image and a spatial distribution of light transmittance in the encoding element;
A spectroscopic system comprising:
[Item 13]
A plurality of regions disposed on an optical path of light incident from the object and having a first light transmittance; and a plurality of regions having a second light transmittance lower than the first light transmittance. Modulating the intensity of incident light using an encoding element;
The light passing through the encoding element is dispersed according to the wavelength using a spectroscopic element to form an image on the imaging surface of the image sensor, the spectroscopic element comprising a light incident surface and a light output surface Are joined to each other so that the refractive indexes of the two wavelengths are substantially the same, and the Abbe number for the light of the specific wavelength is substantially the same. Are different, steps,
Obtaining an image in which components of a plurality of wavelength bands of light passing through the spectroscopic element are superimposed;
Generating a plurality of images for each wavelength band of light that has passed through the spectroscopic element based on the image and a spatial distribution of light transmittance in the encoding element;
A spectroscopic method comprising:

  Hereinafter, more specific embodiments of the present disclosure will be described with reference to the drawings. In the following description, a signal indicating an image (a set of signals indicating pixel values of each pixel) may be simply referred to as “image”. In the following description, the xyz coordinates shown in the figure are used.

(Embodiment 1)
FIG. 1 is a perspective view schematically showing the spectroscopic element P of the first embodiment. The spectroscopic element P is configured by a multistage prism in which two prisms P1 and P2 are combined. In the spectroscopic element P, a light incident surface (hereinafter sometimes referred to as a “light incident surface”) and a surface from which the light is emitted (hereinafter also referred to as a “light emergence surface”) are approximately. It is comprised so that it may become parallel. The two materials constituting each of the prisms P1 and P2 are selected from materials that have substantially the same refractive index for light of a specific wavelength and have different (preferably different) Abbe numbers for light of that wavelength. The specific wavelength can be set, for example, to a representative wavelength (main wavelength) within a desired spectral wavelength range. The dominant wavelength can be, for example, the center wavelength in the wavelength range to be measured or the important wavelength. Here, the “absence” of the Abbe number means that the difference between the Abbe numbers of the two materials is 10 or more. The difference between the Abbe numbers of the two materials is preferably 15 or more, and more preferably 20 or more. The fact that the refractive indexes of the two materials are substantially the same means that the difference between the refractive indexes of the two materials is 0.05 or less, and preferably 0.02 or less.

ここで、ｎａ、ｎｂ、ｎｃは、それぞれ、波長λａ、λｂ、λｃにおける屈折率を表す。λａ、λｃは任意の波長で良いが、例えば、λｂよりも使用波長帯域の始端、終端に近い波長を選択すると良い。 The Abbe number in this specification is not limited to the Abbe number related to the wavelength of the Fraunhofer line that is generally used, but may be defined for any wavelength. In the present disclosure, for any wavelengths λa, λb, and λc that satisfy λa <λb <λc, the Abbe number νb can be defined as the following (Equation 1).

Here, na, nb, and nc represent refractive indexes at wavelengths λa, λb, and λc, respectively. λa and λc may be arbitrary wavelengths. For example, it is preferable to select a wavelength closer to the start and end of the used wavelength band than λb.

  In the present embodiment, the occurrence of coma aberration can be suppressed by making the light incident surface of the spectroscopic element P substantially parallel to the surface from which the light is emitted. Here, “substantially parallel” is not limited to being strictly parallel, but includes a case where the angle formed by two surfaces is 3 ° or less. This angle is preferably set to 1 ° or less, and more preferably 0.5 ° or less. By using the spectroscopic element P as shown in FIG. 1 for an imaging apparatus described later, it is possible to reduce degradation of the resolution of an image for each generated wavelength band. This effect will be described later with reference to the results of Example 1 and Comparative Example 1.

  In the present embodiment, the joint surface of the two prisms is planar. When the joint surface has a curved surface shape, it may be possible to correct the aberration for a light beam having a certain range of incident angles, but not for the light beam having a different range of incident angles. By making the joint surface of the two prisms planar, aberration can be corrected regardless of the incident angle of the light beam. The joining surface does not need to be strictly planar, and some curvature or unevenness is allowed.

  The configuration of the spectroscopic element P is not limited to a configuration in which two prisms are joined. For example, a multi-stage prism may be used in which three or more prisms are bonded together so that the light incident surface and the light output surface are substantially parallel to each other.

  FIG. 2A shows a three-stage configuration example in which three prisms P1 to P3 are bonded together. FIG. 2B shows a four-stage configuration example in which four prisms P1 to P4 are bonded together. It is not limited to these examples, The structure which joined 5 or more prisms may be sufficient. Even in such a configuration, the prism P can be formed of a plurality of materials in which the refractive indexes of light of a specific wavelength are approximately the same and the Abbe numbers for the light of a specific wavelength are different from each other. Further, the light incident surface and the light emitting surface are configured to be substantially parallel. The material is not limited to two types, and three or more types of materials may be used. When three or more kinds of materials are used, it is only necessary that the refractive indices at specific wavelengths are substantially the same and the Abbe numbers are different for at least two kinds of materials.

Table 1 shows an example of a combination of materials for the spectroscopic element P when the d-line is the dominant wavelength. In Table 1, the wavelength region is assumed to be a visible light region, and the Abbe number ν _d of d-line is defined by the following (Equation 2).

Here, n _d , n _F , and n _C represent the refractive indexes of the d line, the F line, and the C line, respectively. The d-line is 587.56 nm, the F-line is 486.1 nm, and the C-line is 656.3 nm.

  In Table 1, although the name of the glass material made from HOYA Corporation is described as a material name, it is not limited to this. Other glass manufacturer materials may be used. The combination of materials is not limited to the example shown in Table 1, and a combination of the same types of materials may be used. Each material may be arbitrarily selected as long as the refractive indices of the main wavelengths are substantially the same and the Abbe numbers of the main wavelengths are different from each other. The at least two materials constituting the spectroscopic element P may be a combination of a glass material and a resin material or a combination of resin materials as in the combination of number 125 in Table 1. Further, a composite material in which inorganic particles (for example, zirconia) are dispersed in a resin material (for example, acrylate resin) may be used as in the combination of number 126 in Table 1.

  The wavelength region of light in the present disclosure is not limited to the visible light region, and may be ultraviolet light, near infrared, mid infrared, far infrared, and other wavelength regions. Further, the wavelength used for calculating the Abbe number may be arbitrarily set.

ここで、ｓ線は８５２．１１ｎｍ、ｒ線は７０６．５２ｎｍ、ｔ線は１０１３．９８ｎｍの波長の光線である。ｎｓはｓ線の屈折率、ｎｒはｒ線の屈折率、ｎｔはｔ線の屈折率を表す。 Table 2 below shows an example of a combination of two materials constituting the spectroscopic element P when the s-line is the main wavelength. The wavelength region in this example assumes a near infrared light region. The Abbe number ν _s of the s line is defined as shown in the following (Equation 3).

Here, the s-line is a light beam having a wavelength of 852.11 nm, the r-line is 706.52 nm, and the t-line is 101.98 nm. ns represents the refractive index of the s line, nr represents the refractive index of the r line, and nt represents the refractive index of the t line.

  The greater the inclination angle of the joint surface of the spectroscopic element P with respect to the plane perpendicular to the optical axis, the greater the degree of spectroscopy (hereinafter also referred to as “spectral quantity”). Further, as the distance between the spectroscopic element P and the image sensor increases, the spectroscopic amount increases. Therefore, the spectral amount can be adjusted by changing the inclination angle of the joint surface of the spectroscopic element P and / or the distance between the spectroscopic element P and the imaging element. When the inclination angle of the joint surface is increased, the associated aberration increases. On the other hand, when the distance between the spectroscopic element P and the imaging element is increased, the change in aberration is small. Therefore, when there is a margin in the flange back of the optical system, the inclination angle of the joint surface of the spectroscopic element P is designed to be small, and the distance between the spectroscopic element P and the imaging element is increased to ensure the spectral amount. be able to. The inclination angle of the joint surface of the spectroscopic element P with respect to the plane perpendicular to the optical axis is set to, for example, an angle of 5 degrees or more and 30 degrees or less, and in an example, may be set to an angle of 10 degrees or 20 degrees. When there is no margin in the flange back of the optical system, it is necessary to adjust the spectral amount by changing the inclination angle of the joint surface of the spectroscopic element P. The inclination angle may be set to an arbitrary angle.

  By using the spectral element P as described above, the occurrence of coma aberration can be suppressed as will be described later. Thereby, it becomes possible to acquire a plurality of images for each wavelength band in which a decrease in resolution is suppressed.

(Embodiment 2)
FIG. 3 is a schematic diagram illustrating the imaging device D1 according to the second embodiment. The imaging device D1 of the present embodiment includes imaging optical systems L1 and L2, an encoding element C, a spectroscopic element P, and an imaging element S. FIG. 3 also shows a signal processing circuit Pr for processing the image signal output from the image sensor S. The signal processing circuit Pr may be incorporated in the imaging device D1, or may be a component of the signal processing device that is electrically connected to the imaging device D1 in a wired or wireless manner. The signal processing circuit Pr is based on the image G acquired by the image sensor S, and a plurality of images separated for each wavelength band of light from the measurement object O (hereinafter referred to as “spectral separation images” or “multiwavelength images”). )) F is estimated.

  Each of the imaging optical systems L1 and L2 includes at least one imaging lens. In FIG. 3, each lens is depicted as one lens, but these may be constituted by a combination of a plurality of lenses.

  The encoding element C is disposed on the imaging surface of the imaging optical system L1. The encoding element C is a mask having a spatial distribution of light transmittance. The encoding element C has at least a plurality of regions having a first light transmittance and a plurality of regions having a second light transmittance lower than the first light transmittance. The encoding element C modulates the intensity of incident light that has passed through the light beam separating element B and allows it to pass therethrough.

  FIG. 4 is a diagram illustrating an example of a two-dimensional distribution of light transmittance of the encoding element C. FIG. 4A shows the transmittance distribution of the encoding element C in the present embodiment. In FIG. 4A, the black portion indicates a region that hardly transmits light (referred to as “light-shielding region”), and the white portion indicates a region that transmits light (referred to as “translucent region”). In this example, the light transmittance of the white portion is approximately 100%, and the light transmittance of the black portion is approximately 0%. The encoding element C is divided into a plurality of rectangular areas, and each rectangular area is a light transmitting area or a light shielding area. Therefore, in the example shown in FIG. 4A, the encoding element C includes a plurality of rectangular regions having a first light transmittance of 100% and a plurality of rectangular regions having a second light transmittance of approximately 0%. Have. The two-dimensional distribution of the light transmitting region and the light shielding region in the encoding element C can be, for example, a random distribution or a quasi-random distribution.

  The concept of random distribution and quasi-random distribution is as follows. First, each rectangular area in the encoding element C can be regarded as a vector element having a value of 1 or 0, for example, depending on the light transmittance. In other words, a set of rectangular regions arranged in a line can be regarded as a multidimensional vector having a value of 1 or 0. Therefore, the encoding element C includes a plurality of multidimensional vectors in the row direction. At this time, the random distribution means that any two multidimensional vectors are independent (not parallel). The quasi-random distribution means that a configuration that is not independent among some multidimensional vectors is included.

  It can be said that the encoding process by the encoding element C is a process of performing marking for distinguishing images by light of each wavelength dispersed by the subsequent spectroscopic element P. As long as such marking is possible, the transmittance distribution may be arbitrarily set. In the example shown in FIG. 4A, the ratio of the number of black portions to the number of white portions is 1: 1, but is not limited to such a ratio. For example, the distribution may be biased on one side, such as the number of white portions: the number of black portions = 1: 9. Further, as shown in FIG. 4B, a mask having a gray scale transmittance distribution may be used. In this case, the encoding element C has a plurality of rectangular regions having a third light transmittance different from the first and second light transmittances. Information on the transmittance distribution of the encoding element C is acquired in advance by design data or actual calibration.

  The spectroscopic element P is an element that disperses an incident light beam according to a wavelength. The spectroscopic element P is the same as the spectroscopic element P in the first embodiment. The spectroscopic element P is configured by a combination of prisms made of two materials whose refractive indices of the main wavelength are approximately equal to each other and whose Abbe numbers of the main wavelength are different from each other. The surface on which the light beam is incident and the surface on which the light beam is emitted are configured to be substantially parallel. As described above, the main wavelength is a representative wavelength in a desired spectral wavelength range, and may be a center wavelength or an important wavelength. With such a configuration, the spectroscopic element P disperses the incident light flux according to the wavelength. Generation of coma aberration can be suppressed by making the light incident surface and light exit surface substantially parallel. Thereby, degradation of the resolution of the spectrally separated image F generated by the process described later can be reduced.

  The light encoded by the encoding element C passes through the imaging optical system L2 and enters the spectroscopic element P. The spectroscopic element P in the present embodiment splits the light image formed on the image pickup surface of the image pickup element S1 so as to shift in the y direction (vertical direction of the image) according to the wavelength. The degree of displacement in the y direction on the imaging surface (sometimes referred to as “spectral amount”) is the refractive index of each material constituting the spectroscopic element P, the Abbe number, the angle of inclination of the joint surface, and the spectroscopic element P and the imaging It is determined by the distance between the element S. When the spectroscopic element P is a diffractive optical element, it can be adjusted by changing the pitch of the diffraction grating. In the present embodiment, the spectroscopic element P splits in the y direction, but may split in the x direction (the horizontal direction of the image) or other directions. Further, the spectral amount or the spectral direction may be changed according to the measurement object O. For example, the spectroscopic element P may be configured to shift in the x direction instead of the y direction for the measurement object O having a high-frequency texture (horizontal stripes or the like) in the y direction.

  The shift amount of the image on the imaging surface of the imaging element S by the spectroscopic element P is calculated in advance from the design specification by calculation or by actual measurement calibration. The spectral shift by the spectral element P is not a discrete shift for each wavelength band but a continuous shift. On the other hand, in the signal processing circuit described later, the spectrally separated image F is reconstructed for each wavelength band of a predetermined width. Therefore, strictly speaking, the image of each wavelength is shifted on the image sensor S even within the wavelength band of each reconstructed image. In order to improve the accuracy of reconstruction of the spectrally separated image F, it is desirable to correct the shift of the image within the wavelength band. This correction may be performed by computer calculation, but is preferably performed by actual calibration in consideration of the effects of aberrations of the optical system and mounting errors. For example, calibration can be performed by placing a white plate as a subject at the position of the measurement object O and forming an image of the encoding element C on the imaging element S through a bandpass filter of each desired wavelength band. The band pass filter may be exchanged for each band to acquire all desired band data, but some bands are selected and measured, and other bands are calculated by interpolation of the measured data May be. By this method, the spectral shift amount by the spectroscopic element P can be calculated, and the transmittance information of the encoding element C for each wavelength band can also be acquired. Based on the calibration data calculated here, the elements of the matrix H in (Equation 4) described later are determined.

  The imaging device S is a monochrome type imaging device having a plurality of photodetection cells (also referred to as “pixels” in this specification) arranged in a two-dimensional manner. The imaging device S can be, for example, a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) sensor. The light detection cell may be constituted by a photodiode, for example. The image sensor S is not necessarily a monochrome type image sensor. For example, a color type image sensor having a filter of R / G / B, R / G / B / IR, or R / G / B / W may be used. By using a color type image sensor, the amount of information regarding the wavelength can be increased, and the accuracy of reconstruction of the spectrally separated image F can be improved. However, when a color type image sensor is used, the information amount in the spatial direction (x, y direction) decreases, so the information amount relating to the wavelength and the resolution are in a trade-off relationship. The wavelength range to be acquired may be arbitrarily determined, and is not limited to the visible wavelength range, but may be a wavelength range of ultraviolet, near infrared, mid-infrared, or far infrared.

  The signal processing circuit Pr is a circuit that processes the image signal output from the image sensor S. The signal processing circuit Pr is, for example, a digital logic processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), or a central processing unit (CPU) / image processing arithmetic processor (GPU) and a computer program It can be realized by a combination. Such a computer program is stored in a recording medium such as a memory, and the arithmetic processing described later can be executed by the CPU executing the program. As described above, the signal processing circuit Pr may be an element outside the imaging device D1. In such a configuration, a signal processing device Pr such as a personal computer (PC) electrically connected to the imaging device D1 or a cloud server on the Internet has the signal processing circuit Pr. In this specification, a system including such a signal processing device and an imaging device is referred to as a “spectral system”.

  Hereinafter, the operation of the imaging device D1 in the present embodiment will be described.

  FIG. 5 is a flowchart showing an outline of the spectroscopic method in the present embodiment. First, in step S101, the intensity of incident light is spatially modulated using the encoding element C. Next, in step S102, the light beam encoded by the encoding element C is separated according to the wavelength by using the spectroscopic element P. In subsequent step S <b> 103, an image on which the components of the light separated by the spectroscopic element P are superimposed is acquired by the imaging element S. In step S104, a plurality of images for each wavelength band are generated based on the image acquired by the imaging device S and the transmittance distribution of the encoding device C.

  Next, a process of acquiring the captured image G by the imaging device D1 of the present embodiment will be described.

  The light beam from the measuring object O is focused by the imaging optical system L1, and the image is encoded by the encoding element C. In other words, the intensity of light passing through the encoding element C is modulated according to the spatial distribution of the transmittance of the encoding element C. The encoded light beam is refocused and imaged by the imaging optical system L2. In the process, the luminous flux is dispersed according to the wavelength by the spectroscopic element P. At this time, the encoded information is also affected by the spectroscopic element P. As a result, the dispersed images having the encoded information are formed on the imaging surface of the imaging device S as multiple images that overlap each other. A plurality of black dots included in the image G illustrated in FIG. 3 schematically represents a low-luminance portion generated by encoding. Note that the number and arrangement of black dots shown in FIG. 3 do not reflect the actual number and arrangement. In practice, there may be more low-brightness parts than the number shown in FIG. Multiple image information is converted into a plurality of electrical signals (pixel signals) by a plurality of light detection cells in the image sensor S, and a captured image G is generated. In FIG. 3, the shift in the y direction of the multiple images in the photographed image G is drawn larger than the actual for easy understanding. Actually, the deviation between the images in the two adjacent wavelength bands can be as small as, for example, about one pixel to several tens of pixels.

  The imaging device D1 may further include a band pass filter that transmits only a part of the wavelength band component of the incident light beam. Thereby, a measurement wavelength band can be limited. By limiting the measurement wavelength band, a spectrally separated image F with high separation accuracy limited to a desired wavelength can be obtained.

  Next, a method of reconstructing the multi-wavelength spectral separation image F based on the captured image G, the spatial distribution characteristics of the transmittance of the encoding element C, and the spectral characteristics of the spectral element P will be described. Here, the multi-wavelength means a wavelength band that is larger than the wavelength bands of three colors (R, G, and B) acquired by a normal color camera, for example. The number of wavelength bands (hereinafter sometimes referred to as “spectral band number”) may be, for example, about 4 to 100. Depending on the application, the number of spectral bands may exceed 100.

The data to be obtained is a spectrally separated image F, and the data is represented as f. If the number of spectral bands (number of bands) is w, f is data obtained by integrating image data f1, f2,. Each of the image data f1, f2,..., Fw is a collection of two-dimensional data of n × m pixels, where n is the number of pixels in the x direction of the image data to be obtained and m is the number of pixels in the y direction. . Therefore, the data f is three-dimensional data having the number of elements n × m × w. If the spectroscopic element P shifts the spectroscopic image by one pixel in the y direction for each spectral band to be obtained, the number of elements of the data g of the acquired captured image G is n × (m + w−1). Data g in the present embodiment can be expressed by the following (Equation 4).

  Here, since f1, f2,..., Fw are data having n × m elements, the vector on the right side is strictly a one-dimensional vector of n × m × w rows and 1 column. The vector g is expressed by being converted into a one-dimensional vector of n × (m + w−1) rows and one column and calculated. The matrix H represents a transformation in which the vector f is intensity-modulated by encoding, the components f1, f2,... Fw are shifted one pixel at a time in the y direction and added. Therefore, H is a matrix of n (m + w−1) rows n × m × w columns.

  Here, since it is assumed that the image of each wavelength band is shifted by one pixel, the number of elements of g is n × (m + w−1), but it is not always necessary to shift each image by one pixel. The number of pixels to be shifted may be two pixels or more. The number of pixels to be shifted depends on how the spectral band and the number of spectral bands in the reconstructed spectral separated image F are designed. The number of elements of g changes according to the number of pixels to be shifted. Further, the spectral direction is not limited to the y direction, and may be shifted in the x direction. In general, when ky and kx are arbitrary natural numbers and the image is shifted by ky pixels in the y direction and kx pixels in the x direction, the number of elements of the data g is {n + kx · (w−1)} × { m + ky · (w−1)}.

Now, given a vector g and a matrix H, it is likely that f can be calculated by solving the inverse problem of (Equation 4). However, since the number of elements n × m × w of the obtained data f is larger than the number of elements n (m + w−1) of the acquired data g, this problem becomes a defect setting problem and cannot be solved as it is. Therefore, the signal processing circuit Pr according to the present embodiment uses the redundancy of the image included in the data f and obtains a solution using a compression sensing technique. Specifically, the data f to be obtained is estimated by solving the following equation (Equation 5).

  Here, f ′ represents the estimated data of f. The first term in parentheses in the above equation represents a deviation amount between the estimation result Hf and the acquired data g, a so-called residual term. Here, the sum of squares is used as a residual term, but an absolute value or a square sum of squares or the like may be used as a residual term. The second term in parentheses is a regularization term (or stabilization term) described later. (Equation 5) means obtaining f that minimizes the sum of the first term and the second term. The signal processing circuit Pr can converge the solution by recursive iterative calculation and calculate the final solution f ′.

  The first term in parentheses in (Expression 5) means an operation for obtaining the sum of squares of the difference between the acquired data g and Hf obtained by system-transforming f in the estimation process by the matrix H. Φ (f) in the second term is a constraint condition for regularization of f, and is a function reflecting the sparse information of the estimated data. As a function, there is an effect of smoothing or stabilizing the estimated data. The regularization term may be represented by, for example, discrete cosine transform (DCT), wavelet transform, Fourier transform, total variation (TV), or the like of f. For example, when the total variation is used, it is possible to acquire stable estimated data that suppresses the influence of noise in the observation data g. The sparsity of the measurement object O in each regularization term space varies depending on the texture of the measurement object O. A regularization term in which the texture of the measurement object O becomes sparser in the regularization term space may be selected. Alternatively, a plurality of regularization terms may be included in the calculation. τ is a weighting factor, and the larger the value, the larger the amount of redundant data reduction (the higher the compression ratio), and the smaller the value, the weaker the convergence to the solution. The weighting factor τ is set to an appropriate value that f converges to some extent and does not cause overcompression.

  In addition, although the calculation example using the compression sensing shown in (Formula 5) was shown here, you may solve using another method. For example, other statistical methods such as maximum likelihood estimation and Bayesian estimation can be used. Further, the number of spectrally separated images F is arbitrary, and each wavelength band may be arbitrarily set.

  As described above, in the present embodiment, the spectroscopic element P is composed of a plurality of materials having substantially the same refractive index for a specific wavelength and different Abbe numbers, and the light incident surface and the light output surface are substantially parallel. It is configured to be. Thereby, as shown in Example 1 to be described later, it is possible to suppress the occurrence of coma aberration and to suppress the decrease in resolution due to the compression sensing. According to this embodiment, the three requirements of high resolution, multiple wavelengths, and moving image shooting (one-shot shooting) can be satisfied simultaneously. In addition, since it is sufficient to hold two-dimensional data at the time of imaging, it is effective for long-term data acquisition. In addition, although the image pick-up element and signal processing circuit in this embodiment acquire a moving image, you may be comprised so that only a still image may be acquired.

(Embodiment 3)
The third embodiment is different from the second embodiment in that a multi-wavelength image is reconstructed using a blurred state on the image plane of the coding pattern. Hereinafter, detailed description of the same contents as those in the second embodiment will be omitted.

  FIG. 6 is a schematic diagram showing the imaging device D2 of the present embodiment. In the imaging device D2, unlike the imaging device D1, the encoding element C is disposed between the object O and the imaging optical system L. The imaging optical system L focuses the light beam that has passed through the encoding element C on the imaging surface of the imaging element S. In the present embodiment, since it is not necessary to arrange the encoding element C on the image forming surface of the image forming optical system, an optical system (relay optical system) other than the image forming optical system L is not required, and the entire optical system. Can be reduced in size.

  The image encoded by the encoding element C is acquired in a blurred state on the imaging element S. Therefore, this blur information is held in advance and reflected in the system matrix H of (Equation 4). Here, the blur information is represented by a point spread function (PSF). PSF is a function that defines the degree of spread of a point image to surrounding pixels. For example, when a point image corresponding to one pixel on an image spreads over a region of k × k pixels around the pixel due to blur, the PSF is a group of coefficients indicating the influence on the luminance of each pixel in the region ( Matrix). The spectral separation image F can be reconstructed by reflecting in the system matrix H the influence of the blur of the coding pattern due to the PSF.

  The position where the encoding element C is disposed is arbitrary, but it is necessary to prevent the encoding pattern of the encoding element C from being excessively diffused and lost. For this purpose, for example, in the imaging optical system L, it is preferable to arrange in the vicinity of the lens closest to the measuring object O or in the vicinity of the measuring object O, and it is preferable to dispose it away from the stop. In particular, since the focal length is short in an optical system having a wide angle of view, the overlapping of light fluxes at each angle of view is small at these positions, and the encoded pattern on the image sensor S is less likely to remain blurred. In addition, even when the encoding element C is arranged at a position closer to the imaging element S, the encoding pattern is likely to remain. However, also in this case, it is necessary to arrange the encoding element C closer to the object O than the spectroscopic element P.

Example 1
Next, examples of the present disclosure will be described.

  FIG. 7 is a diagram illustrating an example of an imaging state on the imaging surface of the imaging element S in the embodiment using the spectral element P of the present disclosure. In this embodiment, a spectroscopic element P in which two prisms as shown in FIG. 1 are joined is used. The two types of materials constituting the spectroscopic element P were selected from those having a d-line refractive index nd close to each other but a d-line Abbe number νd deviating. Specifically, the combination of the glass material M-TAC60 (nd: 1.75501, νd: 51.16) of No. 24 shown in Table 1 and FF8 (nd: 1.75211, νd: 25.05) is used. It was. FIG. 7A is a schematic cross-sectional view showing an example of light rays that pass through the spectroscopic element P and reach the image pickup element S. FIG. As shown in FIG. 7A, the light incident surface and the light exit surface of the spectroscopic element P are parallel to each other, and the inclination angle of the joint surface is 20 ° with respect to the incident surface. The thickness of the spectroscopic element P is 8 mm, and the distance from the rear surface of the spectroscopic element P to the imaging element S is 33 mm.

  FIG. 7B is a lateral aberration diagram in this example. In each figure, the horizontal axis represents the coordinates in the y direction on the entrance pupil plane, and the vertical axis represents the amount of lateral aberration. FIG. 7C is a spot diagram in this embodiment. FIGS. 7B and 7C show examples in which light of each wavelength of 450, 550, and 650 nm is imaged at positions with image heights of 0 mm, 2.5 mm, and 5 mm. From these aberration diagrams and spot diagrams, it can be seen that although any wavelength is well dispersed, the occurrence of aberration is suppressed.

(Comparative Example 1)
As Comparative Example 1, the same analysis was performed on an example in which the light emitting surface of the spectroscopic element P was inclined by 5 ° in Example 1. FIG. 8A is a diagram illustrating an example of an imaging state on the imaging surface in the first comparative example. The two types of materials for the spectroscopic element P are the same as those in the first embodiment.

  FIG. 8A is a schematic cross-sectional view illustrating an example of light rays that pass through the spectroscopic element P and reach the imaging element S in Comparative Example 1. FIG. FIG. 8B is an aberration diagram in Comparative Example 1. FIG. 8C is a spot diagram in the first comparative example. Compared to FIG. 7B, it can be seen that the aberration curve of each wavelength is curved in FIG. 8B. This represents coma aberration that occurs because the thickness of the spectroscopic element P is not uniform. Due to the influence of coma, the spot size shown in FIG. 8C is larger than the spot size shown in FIG. This leads to resolution degradation.

  From the above, in the first embodiment, compared with the first comparative example, it is possible to suppress the deterioration of the resolution. From this result, the effectiveness at the time of using the spectroscopic element in this indication was checked.

  The imaging device according to the present disclosure is useful for a camera or a measurement device that acquires a multi-wavelength two-dimensional image. It can also be applied to sensing for living bodies, medical care, and beauty, food foreign matter / residual pesticide inspection system, remote sensing system and in-vehicle sensing system.

O Measurement object G Captured image L, L1, L2 Imaging optical system C Encoding element P, P1, P2 Spectroscopic element Pr Signal processing circuit S Imaging element F, F1, F2, F3, Fi Spectral separation image D1, D2 Imaging apparatus

Claims (13)

A spectroscopic element configured by joining at least two prisms such that a light incident surface and a light output surface are substantially parallel,
The two prisms have substantially the same refractive index of light of a specific wavelength and different Abbe numbers for the light of the specific wavelength.
Spectroscopic element.   The spectroscopic element according to claim 1, wherein a joint surface of the two prisms is planar. A plurality of regions disposed on an optical path of light incident from the object and having a first light transmittance; and a plurality of regions having a second light transmittance lower than the first light transmittance. An encoding element;
A spectroscopic element that is disposed on an optical path of light that has passed through the encoding element and disperses the light according to a wavelength;
An image sensor that is arranged on an optical path of light that has passed through the spectroscopic element and acquires an image in which components of a plurality of wavelength bands of light dispersed by the spectroscopic element are superimposed;
With
The spectroscopic element is configured by joining at least two prisms such that a light incident surface and a light exit surface are substantially parallel, and the two prisms have substantially the same refractive index of light of a specific wavelength, And the Abbe number for the light of the specific wavelength is different, and has a substantially parallel light incident surface and light exit surface,
Imaging device.   The imaging apparatus according to claim 3, wherein a joint surface of the two prisms is planar. A first optical system disposed between the object and the encoding element and focusing light from the object on a surface of the encoding element;
A second optical system that is disposed between the encoding element and the spectroscopic element and focuses light that has passed through the encoding element on an imaging surface of the imaging element;
The imaging device according to claim 3 or 4, further comprising:   The imaging apparatus according to claim 3, further comprising an optical system that is disposed between the encoding element and the spectroscopic element and focuses light that has passed through the encoding element onto an imaging surface of the imaging element.   The signal processing circuit which produces | generates the several image for every wavelength band of the light which passed the said spectroscopic element based on the said image and the spatial distribution of the light transmittance in the said encoding element is further provided. The imaging device according to any one of the above.   The imaging apparatus according to claim 7, wherein the signal processing circuit generates a plurality of images for each wavelength by a statistical method.   The imaging device according to claim 7 or 8, wherein the number of data in the plurality of images for each wavelength band of light is larger than the number of data in the image acquired by the imaging device. The signal processing circuit is determined based on a vector g having elements of signal values of a plurality of pixels in the image acquired by the imaging element, a spatial distribution of light transmittance in the encoding element, and spectral characteristics of the spectral element. Using the matrix H

(Φ (f) is a regularization term, τ is a weighting factor)
A vector f ′ estimated based on the equation is generated as a plurality of images for each wavelength band.
The imaging device according to claim 7.   The imaging device according to claim 7, wherein the signal processing circuit generates a plurality of images for each wavelength band as moving images. An imaging device according to any one of claims 2 to 6,
A signal processing device that generates a plurality of images for each wavelength band of light that has passed through the at least one spectroscopic element based on the image and a spatial distribution of light transmittance in the encoding element;
A spectroscopic system comprising: A plurality of regions disposed on an optical path of light incident from the object and having a first light transmittance; and a plurality of regions having a second light transmittance lower than the first light transmittance. Modulating the intensity of incident light using an encoding element;
The light passing through the encoding element is dispersed according to the wavelength using a spectroscopic element to form an image on the imaging surface of the image sensor, the spectroscopic element comprising a light incident surface and a light output surface Are joined to each other so that the refractive indexes of the two wavelengths are substantially the same, and the Abbe number for the light of the specific wavelength is substantially the same. Are different, steps,
Obtaining an image in which components of a plurality of wavelength bands of light passing through the spectroscopic element are superimposed;
Generating a plurality of images for each wavelength band of light that has passed through the spectroscopic element based on the image and a spatial distribution of light transmittance in the encoding element;
A spectroscopic method comprising:

Images