JP5708036B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus such as a digital camera, a digital video camera, and a surveillance camera.

  Imaging devices such as digital cameras, digital video cameras, and surveillance cameras are deployed in various types depending on applications. For example, an imaging device corresponding to light in the visible light wavelength band such as a general camera and an imaging device corresponding to a far infrared wavelength band such as a camera for night monitoring are known.

  At this time, each imaging device needs to change the material of the lens according to the wavelength band of the reflected light from the subject. That is, in an imaging apparatus that supports light in the visible light wavelength band, for example, a transparent glass material lens may be used. The transparent glass material has a characteristic of transmitting light in the visible light wavelength band but not transmitting light in the far infrared wavelength band. As a raw material for the transparent glass material, for example, a high-purity material with few impurities such as silica, boric acid, lanthanum oxide, gadolinium oxide, niobium oxide, zirconium oxide, barium, potassium is used. On the other hand, in an imaging device corresponding to light in the far-infrared wavelength band, it is necessary to use a special material such as germanium (element symbol: Ge). Germanium has a characteristic of transmitting light in the far-infrared wavelength band but not transmitting light in the visible light wavelength band.

  As described above, the lens corresponding to the wavelength band of the reflected light from the subject is provided, and the imaging device for the visible light wavelength band and the imaging device for the far infrared light wavelength band are separately configured. And the imaging device for visible light wavelength bands and the imaging device for far-infrared light wavelength bands have been provided to consumers separately according to applications.

  On the other hand, in recent years, for example, surveillance cameras that operate day and night, when the imaging environment is bright, the subject is imaged in the visible light wavelength band, and when the imaging environment is dark or the field of view is poor due to smoke, fog, etc. There is a need for an imaging device that can image a subject in the far-infrared wavelength band. In this case, for example, two imaging devices for the visible light wavelength band and the far infrared light wavelength band may be prepared and arranged in the same place.

  Patent Document 1 discloses an imaging device having sensitivity from a visible light region to a far infrared region as a reference technique of the present invention. Patent Document 2 discloses an imaging device configured to be switched between visible light and infrared light.

JP 2008-204978 A JP 2009-145541 A

  However, when two imaging devices for the visible light wavelength band and the far infrared light wavelength band are prepared, it is necessary to provide a lens and an imaging element for each imaging device. In addition, various adjustments have been required when combining images captured by the respective imaging devices.

  The present invention has been made in view of such circumstances, and provides a technique capable of providing an easy-to-see image regardless of day and night or an imaging environment with a small and simple configuration.

An imaging device according to the present invention receives a light in a visible light wavelength band and a far-infrared light wavelength band, and receives the light in the visible light wavelength band that passes through the lens, and converts the light into a first image signal. An imaging unit for visible light, a far-infrared ray imaging unit that receives light in the far-infrared ray wavelength band that passes through the lens, and converts the light into a second image signal, and the first image signal. An edge comparison unit that compares an edge portion of image data obtained from the second image signal and an edge portion of image data obtained from the second image signal, and the first image signal based on a comparison result by the edge comparison unit. And the second image signal , or both, and the output image data from either or both of the determined first image signal and second image signal. Output image data generation It is equipped with a door.

  According to the technology of the present invention, it is possible to provide an easy-to-view image with a small and simple configuration regardless of day or night or an imaging environment.

It is a figure which shows the structure of the imaging device concerning embodiment of this invention. It is a figure which shows an example of the transmittance | permeability of a lens. It is a figure which shows an example of a structure of an imaging part. It is a figure which shows the relationship between the internal structure of the AFE part for visible rays, and the AFE part for far-infrared rays, and a surrounding component. It is a figure which shows the operation | movement flow of the imaging device concerning embodiment of this invention.

<Embodiment>
FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. As shown in FIG. 1, the imaging apparatus 1000 includes a lens 100, an imaging unit 200, a visible light AFE unit 300, a far infrared light AFE unit 400, a signal processing unit 500, and an output unit 600. It is comprised including.

  The lens 100 is formed to transmit at least light in the visible light wavelength band and the far infrared light wavelength band. The lens 100 forms an image of the reflected light of the subject of the imaging apparatus 1000. FIG. 2 is a diagram illustrating an example of the transmittance of the lens 100. Here, an example in which barium fluoride (BaF2) is used as the material of the lens 100 is illustrated. As shown in FIG. 2, at least visible light wavelength band of 0.36 μm to 0.83 μm (based on Japanese standard JIS Z8120) and far infrared light wavelength band of about 7 μm to about 16 μm The transmittance exceeds 85% in the 10 μm band. In addition to the barium fluoride (BaF2), the material of the lens 100 may be calcium fluoride (CaF2), potassium bromide (KBr), sodium chloride (NaCl), or the like. Further, polyethylene, zinc sulfide [zinc selenium] AR (Anti Reflection) coat (ZnSe), or the like may be used.

  Returning to FIG. 1, the imaging unit 200 includes a visible light imaging unit 210 and a far-infrared light imaging unit 220. The visible light imaging unit 210 receives light in the visible light wavelength band that passes through the lens 100 and photoelectrically converts this light into a first image signal. As an example of the visible light imaging unit 210, a CCD (Charge Coupled Device) can be used. When converting an image into an image signal, the CCD uses a circuit element called a charge-coupled element to transfer the light-receiving element so as to read out the electric charge generated from the light. A CCD is a type of MOS structure semiconductor device. A large number of electrodes are provided on an oxide film on the surface of a silicon substrate, and a potential well is created by applying different voltages to each electrode next to each other. It is something that can be done. In the CCD, by appropriately controlling the voltage applied to each electrode, the charge of each element is transferred to the adjacent element at the same time. As a result, the charge of each pixel held by each element can be sequentially taken out to the outside in a bucket relay manner. Utilizing this property, if the charge input from the end of one row is taken out from the opposite end with a delay corresponding to the number of transfers corresponding to the number of elements, it can be operated as a delay line (delay line).

  The far-infrared ray imaging unit 220 receives light in the far-infrared ray wavelength band that passes through the lens 100 and photoelectrically converts the light into a second image signal. As an example of the far-infrared ray imaging unit 220, a microbolometer can be used. This microbolometer is composed of a special infrared detecting element called a microbolometer, and far infrared rays are detected using this element.

  FIG. 3 is a diagram illustrating an example of the configuration of the imaging unit 200. 3A is a plan view of the imaging unit 200, and FIG. 3B is an enlarged plan view of the A portion. As shown in FIG. 3B, the visible light imaging unit 210 and the far-infrared light imaging unit 220 are alternately arranged in a matrix. Thus, by arranging the visible light imaging unit 210 and the far-infrared light imaging unit 220 uniformly on the condensing surface of the imaging unit 200, the visible light image captured by the visible light imaging unit 210 and The far-infrared light image captured by the far-infrared ray imaging unit 220 can be acquired at the same angle of view. As a result, the visible light image and the far-infrared light image can be easily combined. Note that the example illustrated in FIG. 3 is an example of the imaging unit 200 and is not limited thereto. That is, in the example of FIG. 3, as the most preferable mode, the visible light imaging unit 210 and the far infrared light imaging unit 220 are arranged in a matrix, but in the present invention, the incident light from the lens 100 is imaged. The visible light imaging unit 210 and the far-infrared light imaging unit 220 are not limited to the form in which the visible light imaging unit 210 and the far-infrared light imaging unit 220 are arranged in a matrix.

  A visible light AFE (Analog Front End) unit 300 converts the first image signal generated by the visible light imaging unit 210 from an analog signal to a digital signal. In addition, the far-infrared ray AFE unit 400 converts the second image signal generated by the far-infrared ray imaging unit 220 from an analog signal to a digital signal.

  Here, the configurations of the visible light AFE unit 300 and the far-infrared light AFE unit 400 will be described in detail. FIG. 4 is a diagram illustrating a relationship between the internal configuration of the visible light AFE unit 300 and the far infrared ray AFE unit 400 and peripheral components.

  As shown in FIG. 4, the visible light AFE unit 300 includes a CDS (Correlated Double Sampling) unit 310, an AGC (Automatic Gain Control) unit 320, and an A / D (Analog to Digital) conversion unit 330. doing. Similarly, the far-infrared ray AFE unit 400 includes a CDS unit 410, an AGC unit 420, and an A / D conversion unit 430.

  The CDS unit 310 is a circuit for removing noise of the first image signal input from the visible light imaging unit 210. Similarly, this is a circuit for removing noise of the second image signal input from the far-infrared ray imaging unit 220. The CDS units 310 and 410 are configured based on a correlated double sampling method.

  Specifically, the CDS unit 310 has two crank pulses, the first crank pulse clamps the field-through period of the first image signal, and the second crank path The signal period of the image signal is clamped. Then, the CDS unit 310 takes out the difference between the voltage clamped by the first clamp pulse and the voltage clamped by the second crank pulse, and removes unnecessary noise in the first image signal. Similarly, the CDS unit 410 also has two crank pulses. The first crank pulse clamps the field-through period of the second image signal, and the second crank pulse Clamp the signal period of the image signal. The CDS unit 410 takes out the difference between the voltage clamped by the first clamp pulse and the voltage clamped by the second crank path, and removes unnecessary noise in the second image signal.

  The AGC units 320 and 420 are controlled by control of a control unit (not shown) of the imaging apparatus 1000, and change the gains of the first image signal and the second image signal in a programmable manner. Specifically, the AGC units 320 and 420 adjust the gain of each image signal component processed by the CDS units 310 and 410. Thereby, it is possible to adjust the luminance of the entire subject image captured in two dimensions. For example, if the gain is decreased, a dark pattern image is obtained, and if the gain is increased, a bright pattern image is obtained. The AGC units 320 and 420 adjust the gain in a programmable manner to balance the luminance of the entire image, and generate an image with an easy-to-see pattern.

  A / D converters 330 and 430 convert each image signal gain-adjusted by AGC units 320 and 420 into a digital signal. At this time, the A / D conversion units 330 and 430 convert each image signal from analog to digital in accordance with the interface of the signal processing unit 500. By digitizing each image signal output from the imaging unit 200, signal processing in the signal processing unit 500 can be facilitated.

  3, the timing generator unit 700 includes a visible light imaging unit 210, a far infrared light imaging unit 220, CDS units 310 and 410, AGC units 320 and 420, and an A / D conversion unit 330. 430 and the signal processing unit 500. The timing generator unit 700 outputs a processing timing signal of each unit and a reference signal for driving the imaging unit 200. The timing generator unit 700 also outputs crank pulses from the CDS units 310 and 410.

  Returning to FIG. 1, the signal processing unit 500 receives an image signal (a two-dimensional image digital signal) input from the visible light AFE unit 300 and the far-infrared light AFE unit 400, and The edge portion is determined, and output image data is generated using the first image signal and the second image signal based on these determination results and the like. The signal processing unit 500 includes a visible light density conversion unit 510, a far infrared light density conversion unit 520, a visible light feature extraction filter unit 530, a far infrared light feature extraction filter unit 540, and an edge comparison. A unit 550 and an output image data generation unit 560 are included. Below, each structure in the signal processing part 500 is demonstrated.

  The visible light density conversion unit 510 performs density conversion on the first image signal output from the visible light AFE unit 300 as preprocessing for image analysis. Further, the visible light density conversion unit 510 outputs the first image signal after the density conversion to the visible light feature extraction filter unit 530 and the output image data generation unit 560. Similarly, the far-infrared ray density conversion unit 520 performs density conversion on the second image signal output from the far-infrared ray AFE unit 400 as preprocessing for image analysis. Further, the far-infrared ray density conversion unit 520 outputs the second image signal after density conversion to the far-infrared ray feature extraction filter unit 540 and the output image data generation unit 560.

  Here, density conversion is to improve contrast, and to improve bias in density distribution and signals concentrated in a certain range. That is, the visible light density conversion unit 510 and the far-infrared light density conversion unit 520 perform, for example, a process of brightening an image that is too dark and a process of darkening an image that is too bright. In the density conversion process, pixels having the same luminance value are extracted to identify how the luminance distribution of the image is. Further, for example, when the dynamic range of the imaging apparatus 1000 is 246 gradations, but the image obtained by the first or second image signal has only 128 gradations, the dynamic range of the imaging apparatus 1000 is reduced. Correction processing is performed on the first or second image signal so as to correspond to an image of 246 gradations so that it can be effectively utilized to the maximum extent. The visible light density conversion unit 510 and the far-infrared light density conversion unit 520 can perform density conversion regardless of linearity or non-linearity.

  The visible light feature extraction filter unit 530 extracts an edge portion (boundary) of image data obtained from the first image signal subjected to the density conversion processing by the visible light density conversion unit 510. Similarly, the far-infrared ray feature extraction filter unit 540 extracts an edge portion (boundary) of image data obtained from the second image signal subjected to the density conversion processing by the far-infrared ray density conversion unit 520. To do.

  Here, a portion in the image having similar characteristics such as a luminance value, a color, and a pattern (texture) is defined as one image region. At this time, the edge portion (boundary) refers to a boundary between a certain image region and an image region adjacent thereto. The visible light feature extraction filter unit 530 and the far infrared ray feature extraction filter unit 540, for example, extract an edge portion by differentiating an input image signal.

  In this differentiation processing, processing for deleting low frequency components (regions) is performed using a differentiation filter. The differential filter includes a primary differential filter and a secondary differential filter. As the primary differential filter, for example, a Roberts filter, a Prwitt filter, a Sobel filter, a line detection filter, or the like can be used. For example, a Laplacian filter can be used as the secondary differential filter.

  In such differentiation processing, the visible light feature extraction filter unit 530 calculates a first edge luminance differential value that is a differential value of luminance at the edge portion of the image data obtained from the first image signal. Similarly, the far-infrared ray feature extraction filter unit 540 calculates a second edge luminance differential value that is a differential value of the luminance of the edge portion of the image data obtained from the second image signal.

  Further, the visible light feature extraction filter unit 530 calculates a first edge number that is the number of edge parts of the image data obtained from the first image signal. Similarly, the far-infrared ray feature extraction filter unit 540 calculates the second number of edges, which is the number of edge portions of the image data obtained from the second image signal.

  A difference value between the first or second edge luminance differential value calculated by the visible light feature extraction filter unit 530 or the far-infrared light feature extraction filter unit 540 and the luminance differential value of the peripheral edge region is obtained. The larger the size, the clearer the edge (boundary) exists. In addition, it can be said that the more the number of locations where the difference value is larger, the more edge portions and the fine details are reproduced. An image signal generated based on an image signal with clear edge portions or an image signal with a large number of edge portions is an easy-to-view image with high resolution. In the calculation of the number of edge portions, for example, a threshold value is provided for the difference value between the first or second edge luminance differential value and the luminance differential value of the peripheral edge region, and the difference value is set to a predetermined value. When it is equal to or greater than the threshold value, it is counted as an edge portion.

  The edge comparison unit 550 compares the information input by the visible light feature extraction filter unit 530 with the information input by the far infrared light feature extraction filter unit 540. Specifically, the edge comparison unit 550 compares the edge portion of the image data obtained from the first image signal with the edge portion of the image data obtained from the second image signal. That is, the edge comparison unit 550 compares the first edge luminance differential value and the second edge luminance differential value, and outputs the comparison result to the output image data generation unit 560. At the same time, the edge comparison unit 550 compares the first edge number with the second edge number and outputs the comparison result to the output image data generation unit 560.

  The output image data generation unit 560 generates output image data from the first image signal and the second image signal based on the comparison result by the edge comparison unit 550. The first and second image signals used here are those whose density has been converted by the visible light density converter 510 and the far-infrared light density converter 520.

  When the edge comparison unit 550 determines that the first edge luminance differential value is larger than the second edge luminance differential value, the output image data generation unit 560 uses the first image signal to generate the second image. Output image data is generated without using a signal. On the other hand, when the edge comparison unit 550 determines that the first edge luminance differential value is smaller than the second edge luminance differential value, the output image data generation unit 560 uses the second image signal to generate the first image. Output image data is generated without using a signal. As described above, the output image data generation unit 560 generates the output image data using the image signal having the larger edge luminance differential value among the images obtained from the first or second image signal. For the user of the apparatus 1000, it is possible to provide a visually easy-to-view image with high resolution.

  Furthermore, for example, when the difference in edge luminance differential value between the first image signal and the second image signal is small, the output image data generation unit 560 may output the first image signal or the second image. Rather than selecting one of the signals, both image signals are combined to generate output image data. That is, when the edge comparison unit 550 determines that the difference value between the first edge luminance differential value and the second edge luminance differential value is equal to or less than a predetermined value, both the first image signal and the second image signal are detected. Is used to generate output image data synthesized. In this way, by making it possible to freely set the threshold value of the difference value between the first edge luminance differential value and the second edge luminance differential value, output image data that matches the characteristics of the subject is generated. Can do. At this time, the first image signal or the second image signal can also be weighted.

  Further, when the edge comparison unit 550 has the first edge number larger than the second edge number, the output image data generation unit 560 uses the first image signal and does not use the second image signal. Output image data may be generated. Conversely, when the edge comparison unit 550 has the second edge number larger than the first edge number, the output image data generation unit 560 uses the second image signal and does not use the first image signal. The output image data may be generated. As described above, the output image data generation unit 560 generates the output image data using the image signal having the larger number of edge portions among the images obtained from the first or second image signal. An image that is visually easy to see can be provided to the user of the apparatus 1000.

  Further, when the edge comparison unit 550 determines that the difference between the first edge number and the second edge number is equal to or less than a predetermined value, the output is performed using both the first image signal and the second image signal. Image data may be generated. In this way, by making it possible to freely set the threshold value of the difference value between the first edge number and the second edge number, it is possible to generate output image data that matches the characteristics of the subject. At this time, the first image signal or the second image signal can also be weighted.

  Returning to FIG. 1, the output unit 600 outputs an image to a display device (not shown) such as a monitor based on the output image data generated by the output image data generation unit 560. Specifically, the output unit 600 performs resolution conversion on the output image data, and then outputs the image to the display unit. The output signal of the output unit 600 is an analog output for NTSC (National Television System Committee) or RGB output for an analog display device, and DVI (Digital Visual Interface:)-I output and digital output for HDMI (High-Definition Multimedia Interface). In addition, if a codec is installed in the output unit 600, a digital signal can be encoded and compressed, and a visible light image and a far-infrared light image can be individually output, and further, these images are synthesized. An image can be output. In addition, a maximum of three outputs, which are each output of a visible light image, a far-infrared light image, and a composite image thereof, can be converted into a protocol that can be transmitted over an IP network and output.

  Next, the operation of the imaging apparatus according to the embodiment of the present invention will be described. FIG. 5 is a diagram illustrating an operation flow of the imaging apparatus according to the embodiment of the present invention.

  As shown in FIG. 5, when imaging of the subject is started by the imaging apparatus 1000, first, reflected light of the subject passes through the lens 100 and enters the imaging unit 200 (step (hereinafter referred to as S) 501. ). At this time, the lens 100 transmits at least light in the visible light wavelength band and light in the far infrared light wavelength band.

  The imaging unit 200 receives light that passes through the lens 100 and converts this light into a first image signal and a second image signal (S502). Specifically, the visible light imaging unit 210 receives light in the visible light wavelength band that passes through the lens 100 and photoelectrically converts this light into a first image signal. The far-infrared ray imaging unit 220 receives light in the far-infrared ray wavelength band that passes through the lens 100 and photoelectrically converts this light into a second image signal.

  Next, the visible light AFE unit 300 and the far-infrared light AFE unit 400 perform analog-to-digital conversion on the first image signal and the second image signal input from the imaging unit 200, and digitally convert each image signal. Conversion to a signal (S503). Here, the CDS units 310 and 410 in the visible light AFE unit 300 and the far-infrared light AFE unit 400 remove noise of each image signal. Also, each AGC unit 320, 420 adjusts the gain of each image signal. Then, the A / D conversion units 330 and 430 convert the gain-adjusted image signals into digital signals and output them to the signal processing unit 500.

  The signal processing unit 500 receives image signals input from the visible light AFE unit 300 and the far-infrared light AFE unit 400, determines the strength and edge of the signal, and based on the determination results and the like. The output image data is generated using the first image signal and the second image signal.

  The visible light density conversion unit 510 performs density conversion on the first image signal, and at the same time, the far-infrared light density conversion unit 520 performs density conversion on the second image signal (S504). The visible light density conversion unit 510 outputs the first image signal after the density conversion to the visible light feature extraction filter unit 530 and the output image data generation unit 560. Similarly, the far-infrared ray density conversion unit 520 outputs the second image signal after density conversion to the far-infrared ray feature extraction filter unit 540 and the output image data generation unit 560.

  Next, the feature extraction filter unit 530 for visible light extracts the edge portion of the image data obtained from the first image signal after density conversion, and the feature extraction filter unit 540 for far-infrared light extracts the post-density conversion. Edge portions of image data obtained from the second image signal are extracted (S505). Specifically, the visible light feature extraction filter unit 530 calculates a first edge luminance differential value that is a differential value of the luminance of the edge portion of the image data obtained from the first image signal. Further, the far-infrared ray feature extraction filter unit 540 calculates a second edge luminance differential value which is a differential value of the luminance of the edge portion of the image data obtained from the second image signal. Further, the visible light feature extraction filter unit 530 calculates a first edge number that is the number of edge parts of the image data obtained from the first image signal. Similarly, the far-infrared ray feature extraction filter unit 540 calculates the second number of edges, which is the number of edge portions of the image data obtained from the second image signal. The edge extraction results by the visible light feature extraction filter unit 530 and the far infrared light feature extraction filter unit 540 are output to the edge comparison unit 550.

  The edge comparison unit 550 compares the edge portion of the image data obtained from the first image signal with the edge portion of the image data obtained from the second image signal (S506). Specifically, the edge comparison unit 550 compares the first edge luminance differential value with the second edge luminance differential value, and outputs the comparison result to the output image data generation unit 560. At the same time, the edge comparison unit 550 compares the first edge number with the second edge number and outputs the comparison result to the output image data generation unit 560.

  Then, the output image data generation unit 560 generates output image data based on the comparison result of the edge comparison unit 550 (S507).

  Specifically, when the edge comparison unit 550 determines that the first edge luminance differential value is larger than the second edge luminance differential value, the output image data generation unit 560 uses the first image signal, Output image data is generated without using the second image signal. On the other hand, when the edge comparison unit 550 determines that the first edge luminance differential value is smaller than the second edge luminance differential value, the output image data generation unit 560 uses the second image signal to generate the first image. Output image data is generated without using a signal. Further, when the edge comparison unit 550 determines that the difference value between the first edge luminance differential value and the second edge luminance differential value is equal to or less than a predetermined value, both the first image signal and the second image signal are detected. Is used to generate output image data synthesized.

  Further, when the edge comparison unit 550 has the first edge number larger than the second edge number, the output image data generation unit 560 uses the first image signal and does not use the second image signal. Output image data may be generated. Conversely, when the edge comparison unit 550 has the second edge number larger than the first edge number, the output image data generation unit 560 uses the second image signal and does not use the first image signal. The output image data may be generated. Further, when the edge comparison unit 550 determines that the difference between the first edge number and the second edge number is equal to or less than a predetermined value, the output is performed using both the first image signal and the second image signal. Image data may be generated.

  Finally, the output unit 600 outputs an image to a display device (not shown) based on the image data generated by the output image data generation unit 560 (S508).

  As described above, the imaging apparatus 1000 according to the embodiment of the present invention includes the lens 100, the visible light imaging unit 210, the far infrared light imaging unit 220, the edge comparison unit 550, and the output image data generation. Part 560 as a minimum configuration. The lens 100 transmits light in a visible light wavelength band and a far infrared light wavelength band. The visible light imaging unit 210 receives light in the visible light wavelength band that passes through the lens 100 and converts it into a first image signal. The far-infrared ray imaging unit 220 receives light in the far-infrared ray wavelength band that passes through the lens 100 and converts it into a second image signal. The edge comparison unit 550 compares the edge portion of the image data obtained from the first image signal with the edge portion of the image data obtained from the second image signal. Then, the output image data generation unit 560 generates output image data from the first image signal and the second image signal based on the comparison result by the edge comparison unit 550.

  Thus, in the present invention, the lens 100 that transmits light in the visible light wavelength band and the far infrared light wavelength band is provided, and the visible light wavelength band light and the far infrared light wavelength band that transmit through the lens 100 are provided. Light is converted into a first image signal and a second image signal. Thereby, an image signal corresponding to the visible light image and an image signal corresponding to the far-infrared light image are obtained. In addition, by comparing the edge portions of the image data obtained from these image signals by the edge comparison unit 550, it is possible to identify visually easy-to-see images with high resolution among visible light images or far-infrared light images. can do. Since the output image data generation unit 560 generates output image data based on the comparison result by the edge comparison unit 550, two imaging devices for the visible light wavelength band and the far infrared light wavelength band are used. Compared with the case where it is prepared, it is possible to provide an easy-to-view image with a small and simple configuration regardless of day or night or an imaging environment.

  In the imaging apparatus 1000 according to the embodiment of the present invention, the edge comparison unit 550 includes a first edge luminance differential value that is a differential value of the luminance of the edge portion of the image data obtained from the first image signal, and The second edge luminance differential value, which is the luminance differential value of the edge portion of the image data obtained from the second image signal, is compared. Thereby, it is possible to compare the degree of change in luminance at the edge portion of the visible light image with the degree of change in luminance at the edge portion of the far-infrared light image.

  In the imaging apparatus 1000 according to the embodiment of the present invention, the output image data generation unit 560 determines that the first edge luminance differential value is larger than the second edge luminance differential value when the edge comparison unit 550 determines that the first edge luminance differential value is larger. The output image data is generated using the image signal 1 and not using the second image signal, and the edge comparison unit 550 determines that the first edge luminance differential value is smaller than the second edge luminance differential value. When the second image signal is used, the output image data is generated without using the first image signal. As a result of comparing the degree of change in luminance at the edge part of the visible light image with the degree of change in luminance at the edge part of the far-infrared light image, output using the image signal having the larger degree of change. Image data can be generated. As a result, it is possible to provide an image that is easier to see out of the visible light image or the far-infrared light image.

  For example, in a visible light image, when smoke or fog is generated between a subject that is an imaging target, the outer shape of the subject becomes unclear, and it is difficult to detect the subject. Therefore, the image captured in this state is low in resolution and looks as if the subject was seen through the frosted glass. On the other hand, in the far-infrared light image, since heat is detected to generate image data, it is not easily affected by smoke or fog, and even when the subject cannot be detected by the visible light imaging unit 210, the heat source The subject can be detected by detecting the above. When the heat source of the subject is uniform, in the far-infrared light image, the outer shape of the subject becomes unclear with a low resolution, and it is difficult to detect the subject. On the other hand, in the visible light image, since visible light is sensed, it can be imaged without being affected by the temperature of the heat source. In this way, by generating the output image data by making use of the characteristics of both the visible light imaging unit 210 and the far-infrared light imaging unit 220, it is possible to provide an image that makes it easier to see the subject image.

  In the imaging apparatus 1000 according to the embodiment of the present invention, when the edge comparison unit 550 determines that the difference value between the first edge luminance differential value and the second edge luminance differential value is equal to or less than a predetermined value, the first Output image data is generated using both the image signal and the second image signal. As a result of comparing the degree of change in luminance at the edge part of the visible light image with the degree of change in luminance at the edge part of the far-infrared light image, the visible light can be obtained even when there is no significant difference in the degree of change. An easy-to-see image can be provided by a composite image of the image and the far-infrared light image.

  In the imaging apparatus 1000 according to the embodiment of the present invention, the edge comparison unit 550 obtains the first edge number that is the number of edge portions of image data obtained from the first image signal and the second image signal. The second edge number which is the number of edge portions of the image data to be compared is compared. Thereby, the number of edge portions of the visible light image can be compared with the number of edge portions of the far-infrared light image.

  In the imaging apparatus 1000 according to the embodiment of the present invention, the output image data generation unit 560 determines that the first image signal when the edge comparison unit 550 determines that the first edge number is greater than the second edge number. , The output image data is generated without using the second image signal, and the edge comparison unit 550 determines that the first edge number is smaller than the second edge number. The output image data is generated without using the first image signal. Thus, as a result of comparing the number of edge portions of the visible light image and the number of edge portions of the far-infrared light image, output image data can be generated using the image signal having the larger number of edge portions. it can. As a result, it is possible to provide an image that is easier to see out of the visible light image or the far-infrared light image.

  In the imaging apparatus 1000 according to the embodiment of the present invention, when the edge comparison unit 550 determines that the difference value between the first edge number and the second edge number is equal to or less than a predetermined value, the first image signal and Output image data is generated using both of the second image signals. As a result, when the number of edge portions of the visible light image and the number of edge portions of the far-infrared light image are compared, the composite image of the visible light image and the far-infrared light image is obtained even if the number is not significantly different Thus, an easy-to-see image can be provided.

  In the imaging apparatus 1000 according to the embodiment of the present invention, the visible light imaging unit 210 and the far infrared light imaging unit 220 are preferably formed on the same substrate. Thereby, compared with the case where the imaging part for visible rays and a far-infrared ray are manufactured separately, the imaging device 1000 can be manufactured more simply.

  Preferably, the visible light imaging unit and the far-infrared light imaging unit are alternately arranged in a matrix. As described above, the visible light imaging unit 210 and the far infrared light imaging unit 220 are alternately arranged in a matrix so as to be uniformly arranged in the imaging unit 200, whereby the visible light imaging unit 210 is arranged. And the far-infrared light image captured by the far-infrared ray imaging unit 220 can be acquired at the same angle of view. As a result, the visible light image and the far-infrared light image can be easily combined.

  In particular, various problems arise when it is desired to synthesize a visible light image captured by an imaging device for the visible light wavelength band and a far infrared light image captured by an imaging device for the far infrared light wavelength band. It was. In order to synthesize a visible light image and a far-infrared light image, an optical matching method and an electrical matching method are known.

  However, according to the optical matching method, it is necessary to match the angle of view (zoom / focus) of the lens of each imaging device. For this reason, the data output of the subject image captured by each imaging device is detected by an external image processing device, and the result is fed back by an external optical control device, so that it is necessary to coordinate between a plurality of external devices. As a result, there is a problem that the control becomes complicated.

  Further, according to the method of electrically matching, it is necessary to provide an external image processing device in order to cut out the image of the overlapping area within the angle of view of both lenses from the image output from each imaging device. There are problems such as complicated processing by an external image processing apparatus and difficulty in sequential processing.

  In the present invention, the visible light image and the far-infrared light image are acquired at the same angle of view by devising the arrangement method of the visible-light image capturing unit 210 and the far-infrared light image capturing unit 220. For this reason, the visible light image and the far-infrared light image can be easily combined without causing the problem of the control and processing of the external optical control device as described above.

  The present invention has been described above based on the embodiment. The embodiment is an exemplification, and various modifications, increases / decreases, and combinations may be added to the above-described embodiments without departing from the gist of the present invention. It will be understood by those skilled in the art that modifications to which these changes, increases / decreases, and combinations are also within the scope of the present invention.

DESCRIPTION OF SYMBOLS 100 Lens 200 Imaging part 210 Imaging part for visible rays 220 Imaging part for far infrared rays 300 AFE part for visible rays 400 AFE part for far infrared rays 500 Signal processing part 510 Density conversion part for visible rays 520 For far infrared rays Density conversion unit 530 Visible light feature extraction filter unit 540 Far infrared light feature extraction filter unit 550 Edge comparison unit 560 Output image data generation unit 600 Output unit 1000 Imaging device

Claims (9)

A lens that transmits light in the visible light wavelength band and the far-infrared light wavelength band;
A visible light imaging unit that receives light in the visible light wavelength band that passes through the lens and converts the light into a first image signal;
A far-infrared ray imaging unit that receives light in the far-infrared ray wavelength band that passes through the lens and converts the light into a second image signal;
An edge comparison unit that compares an edge portion of image data obtained from the first image signal and an edge portion of image data obtained from the second image signal;
Based on the comparison result by the edge comparison unit, it is determined whether to use the first image signal or the second image signal , or both, and the determined first image signal and the determined image signal An imaging apparatus comprising: an output image data generation unit configured to generate the output image data from one or both of the second image signals .   The edge comparison unit includes a first edge luminance differential value that is a differential value of luminance of an edge portion of image data obtained from the first image signal, and an edge portion of image data obtained from the second image signal. The imaging apparatus according to claim 1, wherein a second edge luminance differential value that is a differential value of luminance of the second edge luminance is compared. The output image data generation unit
When the edge comparison unit determines that the first edge luminance differential value is larger than the second edge luminance differential value, the first image signal is used, and the second image signal is not used. Generating the output image data;
When the edge comparison unit determines that the first edge luminance differential value is smaller than the second edge luminance differential value, the second image signal is used, and the first image signal is not used. The imaging apparatus according to claim 2, wherein the output image data is generated. When the edge comparison unit determines that a difference value between the first edge luminance differential value and the second edge luminance differential value is a predetermined value or less, the first image signal and the second image signal The imaging apparatus according to claim 2 , wherein the output image data is generated using both of the two .   The edge comparison unit is a first edge number that is the number of edge portions of image data obtained from the first image signal and a first edge number that is the number of edge portions of image data obtained from the second image signal. The imaging device according to claim 1, which compares the number of edges of two. The output image data generation unit
When the edge comparison unit determines that the number of first edges is larger than the number of second edges, the output image data is used without using the second image signal and using the first image signal. Produces
When the edge comparison unit determines that the number of first edges is smaller than the number of second edges, the output image data is used without using the first image signal and using the second image signal. The imaging device according to claim 5, which generates When the edge comparison unit determines that the difference value between the first edge number and the second edge number is equal to or less than a predetermined value, both the first image signal and the second image signal are used. The imaging apparatus according to claim 5 , wherein the output image data is generated.   The imaging apparatus according to claim 1, wherein the visible light imaging unit and the far infrared light imaging unit are formed on the same substrate.   The imaging device according to claim 8, wherein the visible light imaging unit and the far infrared light imaging unit are alternately arranged in a matrix.

Images