WO2005091042A1

WO2005091042A1 - Method and device for capturing images with large lighting dynamics

Info

Publication number: WO2005091042A1
Application number: PCT/FR2004/002578
Authority: WO
Inventors: Jean-Claude Robin
Original assignee: Lheritier
Priority date: 2004-02-19
Filing date: 2004-10-12
Publication date: 2005-09-29
Also published as: EP1716442A1; FR2866714A1; FR2866714B1; IL177573A0

Abstract

The invention relates to a method for capturing images with large lighting dynamics by means of at least two different sensors, one (C2) of said sensors having a high sensitivity and a low resolution, and the other (C1) having a low sensitivity and a high resolution. According to the inventive method, the image to be captured is projected to the entrance of an optical separator in such a way as to divide the projected beam into two complementary fractions determined according to the sensitivity of the sensors; the two fractions of the beam are respectively applied to the two sensors by means of two separate optical paths; a parameter relating to the luminosity of the image to be captured is detected; the signal emitted by one of the sensors is selected according to the detected parameter; and the signal selected is used as a signal of the captured image.

Description

METHOD AND DEVICE FOR CAPTURING LARGE DYNAMIC IMAGES OF A CLEAR LEVEL.

The present invention relates to a method and a device for capturing images with high dynamic range of illumination level.

This method applies in particular to the capture of images in the field of visible and near infrared spectral bands, which capture requires sensors of a different nature, adapted to very different levels of illumination.

It also applies to the association of sensors whose spectral bands are distant and whose nature of the radiation is different.

Generally, we know that optronics uses a spectral perimeter ranging from 0.2 μm to 20 μm; indeed, the ultraviolet band is between 0.2 μm and 0.45 μm; the visible band is between 0.45 μm and 0.75 μm; the infrared band is between 0.75 μm and 20 μm.

Furthermore, in the visible band, the level of illumination can be very different, requiring the use of sensors either based on photoconduction for high levels of illumination, or based on photo-emission for low levels of d illuminance.

Thus capturing images at very different levels of illumination, or even in different bands, such as those of infrared and visible, requires the use of different sensors and consequently the installation of an optic and an image acquisition processing for each of the sensors.

Charge transfer devices (DTCs) are analog shift registers that can be used as delay lines, analog memories, and analog signal processing components.

In the image detectors, the charge transfer devices serve to put in series the information read by each pixel, and thus make it possible to carry out a multiplexing.

More precisely, radiation or any other form of energy induces electric charges in a semiconductor detector material; these charges are stored in an insulating element forming a capacitor, then, under the action of an electric field, are transferred into another neighboring identical element.

The information, put in the form of electric charges contained in a mosaic produced from such capacitive elements, is collected at the end of the chain, after a succession of transfers. The simultaneous transfers of the contents of each capacitor, into the neighboring capacitor, are effected by the set of external voltages applied to these capacitors.

The collection of all the charges contained in the network is done on a single output electrode. The device behaves like a phased array detector, associated with a system for reading and multiplexing information in the form of a video signal.

Charge transfer devices (DTCs) are made up of CMOS ("Complementary Metal Oxide Semiconductor") cells, or "CCD: Charge-Coupled Device" cells. In the visible area with standard illumination level (daylighting), imaging with a retina of DTC elements can be treated in two different ways:

In one case, the matrix of DTC elements subjected to radiation is read by transferring the charges induced along each line to an output register, constituted by a column of DTC elements used as a multiplexer. The charges from each column are transferred simultaneously to the neighboring column. At each step, a column fills the output register which is immediately read by transfer to the video amplifier. The operating frequency of the device is high due to the speed of the transfer.

In the other case, the content of the DTC elements subjected to radiation is transferred into a buffer memory, itself made up of DTC elements. This configuration allows harmonization between the integration time of the radiation corresponding to a frame, the very short transfer time to the buffer memory and the memory read time corresponding to the acquisition time of a new frame. The operating frequency is thus lowered.

In both cases, the semiconductor element subjected to the radiation is silicon.

In the infrared domain, two technologies, called "monolithic" or "hybrid", coexist.

Monolithic technology whose detector elements and reading circuits have similar substrates, generally silicon; the silicon-platinum detector elements use a Schottky barrier diode between the platinum silicide and the P-type silicon. The spectral sensitivity is located in the extended band between the visible and 5 μm, with a strong decrease from 3 μm.

As for the hybrid structure, the semiconductor element used is a function of the spectral band used; the main semiconductor elements used are lead selenide, indium antimonide and cadmium-mercury telluride. The structures are complex because they integrate on the one hand the photosensitive elements, on the other hand the registers for reading, multiplexing, delay, and sometimes the preamplification modules. Two-dimensional mosaics are superimposed and connected via indium microbeads.

The spectral sensitivity of indium antimonide is between 1 μm and 6 μm, with a maximum at 5 μm.

As for the spectral sensitivity of cadmium-mercury telluride, it is a function of the relative composition of the ternary mixture; thus, it can be maximum in the band 3 μm to 5 μm, or maximum in the band 8 μm to

12 μm.

In the visible domain with low illumination (night illumination), imaging by a retina of DTC elements is preceded by an intensifier of light images. This essentially consists of a photocathode comprising a layer of a material which, placed in a vacuum, emits electrons when it receives light. In the presence of an electric field, these so-called primary electrons are accelerated and then multiplied in a wafer of microchannels under the effect of secondary emission during their journey through the microchannels. The so-called secondary electrons hit a screen, which is brought to the acceleration potential. A block of optical fibers, capturing the image obtained, then makes it possible to reverse the image and then to reduce the dimensions of the retina; thus the block of optical fiber degreasers peπnet to optimize the resolution and to cover the entire field of the mosaic of DTC elements. These devices are called intensified DTC sensors. Traditionally, devices for capturing images with high lighting dynamics consist of: • either two sensors: a sensor with DTC elements for daytime imaging and a sensor with intensified DTC elements or a sensor for imaging infrared, • either from a sensor, generally of the type with intensified DTC elements.

In the second category, there are essentially four types of devices: • the sensor with intensified DTC elements associated with an electronic shutter controlled by the average value of the illumination, • the sensor with intensified DTC elements associated with a controlled iris and a neutral filter with strong attenuation in the central area, close to the optical axis, • the intensified DTC element sensor associated with variable optical attenuators, • the DTC element sensor for monochrome daytime imaging associated with an electronic chain with great gain.

The first category of these devices has the disadvantage of requiring two different optics associated with the two sensors which differ dimensionally.

The second category (apart from the fourth) has the drawback of using a sensor with fragile intensified DTC elements with high intensity illuminations, of using (in two cases) a fragile mechanical device, even bulky, in a severe mechanical environment, and does not allow color to be transmitted; as for the latter case, the electronic processing is limited by the signal to noise ratio at low levels of illumination and, moreover, the processing, allowing noise reduction, causes a dragging effect associated with moving objects. In general, these devices do not allow significant dynamics to be obtained; in fact, DTC element sensors for monochrome daytime imaging have a threshold with a low level of illumination of 10 ^"3 to 10 ^{" 4} lux; the sensors with intensified DTC elements have a threshold of 10 ^"6 lux; as for the admissible level at high level of illumination, it is 10 ⁴ lux for the sensors with DTC elements for daytime imaging. Thus, the dynamics currently obtained is close to 10 ⁶ .

A dynamic of 10 ⁹ , that is to say between 10 ^"5 lux and 10 ⁴ lux, is therefore not possible with conventional devices.

The broadening of the level of illumination dynamics and / or of the spectral sensitivity requires compromises between the different parameters specific to the sensors used, described above, which compromises do not meet the objectives sought.

The invention therefore more particularly aims to eliminate these drawbacks.

It proposes a method for capturing images with high dynamic range of illumination level using at least two sensors of different nature, one of which has a high sensitivity and a low resolution, while the other has a low sensitivity and a high resolution, consisting in: - projecting the image that one wishes to capture at the input of an optical splitter so as to divide the projected beam into two complementary fractions determined according to the sensitivity of the sensors, - to apply the two beam fractions to the two sensors respectively via two separate optical channels, - to detect a parameter relating to the brightness of the image to be captured, - to select, depending on the detected parameter, the signal emitted by one or other of the sensors,

- to use the selected signal as the signal of the captured image.

More specifically,

• the separation into at least two optical channels consists in transmitting a small fraction of the light energy incident on the DTC element sensor for daytime imaging, and a high fraction of the light energy incident on the DTC element sensor intensified for night imaging, or on the infrared sensor, • switching the video signals from the sensors consists of: - under high incident light energy, taking the useful video signal from the DTC element sensor for day imaging and to protect the sensor with intensified DTC elements for night imaging or the infrared sensor, - under low incident light energy, with taking the useful video signal coming from the sensor with intensified DTC elements for night imaging or of the infrared sensor , • said switching, a function of the level of incident light energy: - is determined by the DTC element sensor for daytime imaging, - pr takes into account the awareness time of the sensor with intensified DTC elements for night imaging or of the infrared sensor, - introduces a hysteresis effect around the switch point from one sensor to the other sensor, - updates the '' stop the unused sensor.

The above-mentioned device for separating into at least two optical channels will consist of a separating blade or prism and a set of lenses making it possible to produce light beams adapted in intensity and incidence to the sensors, and to adapt said light beams to the size of the sensors. Advantageously, the aforementioned separation device will only comprise a blade or a separation prism if the sensors have the same size, the set of lenses being no longer necessary.

As for the switching device, it will include a threshold detector associated with the video signal of the sensor with DTC elements for daytime imaging, means of command and control of the supply of each sensor which take into account in particular the time of sensitization, introduce a hysteresis effect around the tipping point, and limit the consumption of the unused sensor.

The invention further aims to take advantage of the compatibility in the processing of DTC elements of different sensors and thus to have a single pixel reading chain and amplification of the video signal.

This can in particular be applied to the combination of a sensor with DTC elements for monochrome daytime imaging, and a sensor with intensified DTC elements for nighttime imaging or with an infrared sensor.

Therefore, it will be possible to view an image whose level of illumination is high, followed by an image whose level of illumination has suddenly dropped, and this without interruption during the generation of successive images.

Since the incident light energy will be largely attenuated by the separation into two optical channels, before reaching the DTC element sensor for day imaging, the dynamics of the assembly made up of the two sensors of different nature will be close to the sum of the respective dynamics of said sensors. Furthermore, the separation into several optical channels and the switching of the video signals will make it possible to avoid an abnormal exposure of the sensors and thus to avoid an irreversible deterioration of the performances of said sensors.

Finally, the presence of a set of lenses associated with each of the sensors, will make it possible to obtain almost identical shooting fields, the limitations to the equality of the fields are mainly due to the inaccuracies on the focal lengths of said sets of lenses.

An embodiment of the invention will be described below, by way of nonlimiting example, with reference to the appended drawings in which:

Figure 1 is a block diagram of a device according to the invention, Figure 2 is a block diagram of the device comprising a first improvement, Figure 3 is a block diagram of the device comprising a second improvement, Figure 4 is a block diagram of the device comprising a third improvement, FIG. 5 is a block diagram of the switching system.

In the example shown in FIG. 1, the device according to the invention comprises: • a head optic OT making it possible to focus the incident light beam on a sensor C ₂ , • a semi-reflecting mirror MSR, arranged upstream of the sensor C ₂ , followed by a reflective mirror MR, • an optic Oi followed by an optic C making it possible to focus the light beam coming from the reflective mirror MR, on a sensor Ci, • an electronic circuit Ei, associated with the sensor Ci, delivering the video signal (arrow Vj), • an electronic circuit E ₂ , associated with the sensor C ₂ , delivering the video signal (arrow V ₂ ), • a switching block BC, receiving the video signals (arrows Ni, V ₂ ), delivering the output video signal (arrow SV), and driving the electronic circuits Ei, E ₂ , respectively by the commands (arrows Cdi, Cd ₂ ).

The semi-reflecting mirror MSR, located between the head optic OT and the sensor C ₂ , transmits light energy greater than that reflected; the ratio between the transmitted light energy and the incident light energy is between 95/100 and 80/100; more precisely, approximately 90% of the incident light energy will be transmitted to the sensor C ₂ , and approximately 10% of the light energy will be transmitted to the sensor Ci. Advantageously, the function of separation into two light beams may be provided by a separating prism.

The Ci sensor is a DTC element sensor for daytime imaging, including color; the sensor C ₂ may be either an infrared sensor for infrared imaging, or a sensor with intensified DTC elements for night imaging.

The video signals (arrows V ₁₅ V ₂ ), respectively from the electronic circuits Ei, E ₂ , associated with the sensors Ci, C ₂ , are directed to the switching block BC. This switching block BC has the essential function of controlling the two sensors Ci, C ₂ as a function of the level of illumination and of managing the transition phase during the switching from one of the two sensors to the other.

Thus, when viewing images under high illumination, the sensor Ci will be active and the sensor C ₂ will be inactive and therefore protected; the signal output video (arrow SV) will be that delivered by the sensor Ci, or the video signal (arrow Ni).

When the illumination drops sharply, the sensor C ₂ is activated, the sensor Ci remaining active during the transition phase; the sensor C ₂ then delivers the video signal (arrow V ₂ ); the sensor Ci will then be deactivated; the output video signal (arrow SV) will be that delivered by the sensor C ₂ , or the video signal (arrow V ₂ ).

Note that despite the latency specific to the sensor C ₂ , the observer will not notice any interruption during the generation of successive images.

It will also be noted that the dynamics of the device for very high lighting levels will be increased since the sensor C1 receives only a fraction of the incident light energy, ie 10% in the present case.

In the example shown in FIG. 2, the device according to the invention further comprises a wafer of optical fibers GFO, of small thickness, situated in the image plane of the beam reflected by the semi-reflecting mirror MSR, upstream of the mirror reflective MR. Said wafer of GFO optical fibers makes it possible to overcome the non-uniformity of the image which appears in particular when the focal length of the head optics is short, that is to say when the field is large.

In the example shown in Figure 3, the device according to the invention further comprises an optical fiber reducer RFO, positioned on the sensor Ci; this configuration makes it possible to adapt the fields of vision between the two channels associated with the two sensors Ci, C ₂ , and thus to remove the reflecting mirror MR and the two optics Oi, O _r . The horizontal reversal of the image, introduced by the semi-reflecting mirror MSR, will be compensated either by an adequate electronic processing associated with the electronic circuit El, or by an RFO reducer with twisted optical fibers; this second solution makes it possible to dispense with an additional consumption of electrical energy linked to the processing of the inverted image.

In the example shown in FIG. 4, the device according to the invention further comprises an infrared rejection filter FR, located upstream of the optical fiber reducer RFO, making it possible to restore the colorimetry of the color image of a faithfully, in the case where the sensor C1 is a sensor with DTC elements for day color imaging.

In the example shown in FIG. 5, the switching system BC comprises: • a CONT controller responsible for managing and controlling the operating parameters of the activated channel, and activating, if necessary, the switching of the channels, • a COM switch responsible for transmitting the activated channel from information received from the CONT controller, • a REF reference means defining, for the CONT controller, the conditions for switching from the active channel to the inactive channel, • two D detectors _1? D ₂ , detecting the video signals (arrows Vi, V ₂ ), respectively from the electronic circuits Ei, E ₂ associated with the sensors Ci, C ₂ .

Thus, the switching system BC receives the video signal (arrows Vi or V ₂ ) from the activated channel and transmits it at the output of the device according to the invention (arrow SV). The channel activated at a given time is used, via the Di or D ₂ detectors, as a measurement means making it possible to decide whether to switch to the other channel.

In general, when the device according to the invention is started, the channel activated by default will be that from the sensor Ci dedicated to daytime imaging.

The CONT controller, using information from the REF reference element and the DD ₂ video detectors, activates the inactive channel, causes the active channel to be stopped or put on standby (Cdi arrows, Cd ₂ ), and triggers the switching via the COM switch.

Furthermore, a hysteresis type means, which is an integral part of the CONT controller, makes it possible to avoid the risks of inadvertent switching around the tipping point.

Advantageously, at intermediate lighting levels, that is to say compatible simultaneously with the two sensors Ci, C ₂ , the video signals (arrows Vi, V ₂ ) will be beforehand sent to a multiplexing system (not shown) located upstream of the switching system BC, which multiplexing system will make it possible to superimpose the images from the two sensors CC ₂ and to transmit them in a sequential manner.

Thus, this method for capturing images with a high dynamic level of illumination according to the invention meets the objectives sought, namely: - an operating dynamic both at very high levels of illumination and at very low levels of illumination; it should be noted that the performance at low level of illumination of the sensor with intensified DTC elements does not constitute a limit to the process, - the capture of day images in black and white or in color, - the protection of the sensor with high sensitivity under high level of illumination, - the quality of the images captured under intermediate illumination, - the superposition of images from the two sensors and / or to transmit them by a time multiplexing under intermediate lighting.

In the same way, the device implementing the method according to the invention meets the desired objectives, namely: - the compactness and lightness of the assembly, - the behavior in severe mechanical environment taking into account the absence of moving mechanical parts, - the absence of a drag effect in the presence of moving objects.

Claims

claims

1. Method for capturing images with a high dynamic level of illumination using at least two sensors of different nature, one of which (C ₂ ) has a high sensitivity and a low resolution, while the other (Ci) has a low sensitivity and a high resolution, characterized in that it consists in: - projecting the image that one wishes to capture at the input of an optical splitter so as to split the beam projected in two complementary fractions determined according to the sensitivity of the sensors, the above optical separator transmits a small fraction of the light energy of the image to be captured on the detector (Ci), and a high fraction of the light energy of the image to be captured on the sensor (C _2), - applying respectively to the two fractions of beam on the two sensors of the two separate optical paths intennédiaire, - detecting a parameter relating to the brightness of the image capture, - to SELECTER, depending on the parameter detected, the signal emitted by one or other of the sensors, - selects to use the signal as a signal of the captured image.

2. Method according to claim 1, characterized in that the aforesaid sensor (C ₂ ) having a high sensitivity and a low resolution is a sensor with intensified DTC elements for night imaging or an infrared sensor for infrared imaging.

3. Method according to claim 1, characterized in that the aforesaid sensor (Ci) having a low sensitivity and a high resolution is a sensor with DTC elements for day imaging.

4. Method according to claim 1, characterized in that the optical separator consists of a semi-reflecting mirror (MSR), a reflecting mirror (MR), a head optic (OT) and optics (Oi, Or) associated with the sensor (Ci).

5. Method according to claim 4, characterized in that a wafer of optical fibers (GFO) is located in the image plane of the beam downstream of said semi-reflecting mirror (MSR), and upstream of said reflecting mirror (MR).

6. Method according to claim 1, characterized in that the optical separator consists of a semi-reflecting mirror (MSR) and a fiber optic reducer (RFO).

7. Method according to claim 6, characterized in that the optical fiber reducer (RFO) is a twisted optical fiber reducer.

8. Method according to claim 1, characterized in that the optical separator consists of a semi-reflecting mirror (MSR), a fiber optic reducer (RFO), and a rejector filter (FR).

9. Method according to claim 1, characterized in that the detection of the parameter relating to the brightness of the image to be captured and the selection, as a function of the detected parameter, of the signal emitted by one or other of the sensors ( Ci, C ₂ ), are performed by a switching block (BC).

10. Method according to claim 9, characterized in that the aforesaid switching block (BC): • activates by default the sensor (Ci) dedicated to daytime imaging, • takes into account the sensor awareness time (C ₂ ), • introduces a hysteresis effect around the switch point from one sensor to the other sensor, • puts the unused sensor to sleep or standby.

11. Method according to claim 9, characterized in that the above switching block (BC) is preceded by a time multiplexing system.

12. Device for implementing the method according to claim 1, intended for capturing images with high dynamic range of illumination level using at least two sensors of different nature, one of which (C ₂ ) has a high sensitivity and a low resolution, while the other () has a low sensitivity and a high resolution, characterized in that it comprises: - means for projecting the image that one wants to capture , - means for separating the projected beam into two complementary fractions determined according to the sensitivity of the sensors, the aforementioned means for separating the projected beam transmit a small fraction of the light energy of the image to be captured on the detector (Ci ), and a high fraction of the light energy of the image to be captured on the detector (C ₂ ), - means for transmitting the two beam fractions to the two sensors via two optical channels. are separated, - means for detecting a parameter relating to the brightness of the image to be captured, - means for selecting, as a function of the detected parameter, the signal emitted by one or other of the sensors.

13. Device according to claim 12, characterized in that the aforesaid sensor (C ₂ ) having a high sensitivity and a low resolution is a sensor with intensified DTC elements for night imaging or an infrared sensor for infrared imaging.

14. Device according to claim 12, characterized in that the aforesaid sensor (Ci) having a low sensitivity and a high resolution is a sensor with DTC elements for daytime imaging.

15. Device according to claim 12, characterized in that the means for separating the projected beam consist of a semi-reflecting mirror (MSR), a reflecting mirror (MR), a head optic (OT) and optics (Oi, Gold) associated with the sensor

16. Device according to claim 15, characterized in that a wafer of optical fibers (GFO) is located in the image plane of the beam downstream of said semi-reflecting mirror (MSR), and upstream of said reflecting mirror (MR).

17. Device according to claim 12, characterized in that the means for separating the projected beam consist of a semi-reflecting mirror (MSR) and a fiber optic reducer (RFO).

18. Device according to claim 17, characterized in that the optical fiber reducer (RFO) is a twisted optical fiber reducer.

19. Device according to claim 12, characterized in that the optical separator consists of a semi-reflecting mirror (MSR), a fiber optic reducer (RFO), and a rejector filter (FR).

20. Device according to claim 12, characterized in that the means for detecting the parameter relating to the brightness of the image to be captured and the means for selecting, as a function of the detected parameter, the signal emitted by one or the other of the sensors (CC ₂ ), constitute a switching block (BC).

21. Device according to claim 20, characterized in that the aforesaid switching block (BC): • activates by default the sensor (Ci) dedicated to daytime imaging, • takes into account the awareness time of the sensor (C ₂ ), • introduces a hysteresis effect around the switch point from one sensor to the other sensor, • switches the unused sensor off or on standby.

22. Device according to claim 20, characterized in that the above switching block (BC) is preceded by a time division multiplexing system.