WO2022253676A1

WO2022253676A1 - Time-of-flight camera having a spad light time-of-flight sensor

Info

Publication number: WO2022253676A1
Application number: PCT/EP2022/064239
Authority: WO
Inventors: Stefan Bölsker
Original assignee: Ifm Electronic Gmbh; pmdtechnologies ag
Priority date: 2021-06-02
Filing date: 2022-05-25
Publication date: 2022-12-08
Also published as: DE102022113233A1

Abstract

The invention relates to a time-of-flight camera, comprising an illumination means (10) for emitting light pulses and a SPAD light time-of-flight sensor (20) having at least one SPAD pixel, wherein: the at least one SPAD pixel is free of distortion; in order to determine distance, light events occurring at the SPAD pixel are counted with temporal resolution in predefined time intervals; light events are assigned to an object if the number of light events exceeds an evaluation threshold value (AS); an object distance is determined using a time difference between an emitted light pulse and a received light pulse; the SPAD pixel has a first and a second memory; the first memory is used for storing, with temporal resolution, the number of light events, which are detected in order to form a histogram for determining distance; the second memory is used to store a number of background-light events; the histogram events are detected in a histogram interval and the background-light events are detected in a background-light interval; for a distance measurement, both intervals are run through multiple times during a predefined measurement duration; the time-of-flight camera is designed such that the counting of the light events in a light time-of-flight pixel is stopped if either the predefined measurement duration is reached or the number of detected background-light events exceeds a preset background-light threshold value (HS).

Description

Time-of-flight camera with a SPAD time-of-flight sensor

The invention relates to a time-of-flight camera with a SPAD time-of-flight sensor which has at least one distortion-free SPAD time-of-flight pixel according to the species of the independent claim.

In the presence of strong background light, a non-linear distortion of the measurement results in the histogram occurs. In particular, the background signal observed in the histogram may not be constant due to the distortion. Various methods with which this distortion can already be avoided during the recording of the histogram have been described in the scientific literature. See for example A Gupta et al (2019). "Asynchronous Single-Photon 3D Imaging", IEEE/CVF International Conference on Computer Vision 7908 et seq.

In particular, the "photon-driven acquisition" method, also referred to as "free-running", and the "asynchronous single-photon 3D imaging" method are suitable here for forming a SPAD pixel "distortion-free" in this sense.

For the invention to function correctly, it is necessary to use one of these methods or a comparable method so that the background signal/background light signal is not distorted during the measurement.

For example, DE 102017 113675 A1 discloses an optoelectronic sensor with at least one avalanche photodiode operated in a Geiger mode, with an evaluation unit that determines a common measured value for the distance from a large number of individual light propagation times. Since the avalanche photodiode does not work without distortion, it is intended to take into account the inherently existing exponentially decreasing frequency of detectable background events and to estimate how many individual light propagation times are to be expected in a time interval due to background events and to determine the common measured value from a time interval in which significantly more Single light propagation times are than expected.

The object of the invention is to improve the dynamics of a SPAD time-of-flight camera The object is advantageously achieved by the time-of-flight camera according to the invention according to the species of the independent claim.

A time-of-flight camera is advantageously provided, with lighting for emitting a light pulse and a SPAD light-time-of-flight sensor with at least one SPAD pixel, the at least one SPAD pixel being configured free of distortion, light events occurring at the SPAD pixel being counted in a time-resolved manner at predetermined time intervals to determine the distance be, wherein light events are assigned to an object when the number of light events exceeds an evaluation threshold, wherein an object distance is determined based on a time difference between emitted and received light, wherein the SPAD pixel has a first and second memory, the first memory for storing, in a time-resolved manner, the number of light events collected to form a histogram for determining distance, and wherein the second memory is for storing a number of background light events, the histogram events in a em histogram interval and the background light events are recorded in a background light interval, with both intervals being run through several times during a predetermined measurement period for a distance measurement, with the time-of-flight camera being designed in such a way that the counting of the light events in a light-time-of-flight pixel is stopped when either the specified measurement duration has been reached or when the number of detected background light events exceeds a preset background light threshold value.

This procedure has the advantage that the histogram for the distance measurement does not become saturated.

The time-of-flight camera advantageously has an evaluation unit which is designed in such a way that the evaluation threshold value is determined based on the number of background light events recorded in the second memory.

It is thus possible to determine a valid evaluation threshold value without analyzing the noise of the histogram stored in the first memory.

In particular, it is advantageous to define the background light threshold value in such a way that an evaluation threshold value calculated on the basis of the background light threshold value remains below the maximum bit value of the first memory.

In a further embodiment, in a time-of-flight camera that has a time-of-flight sensor with a plurality of SPAD pixels, a termination period is recorded for each pixel in which the background light events have reached or exceeded the background light threshold value and based on the termination periods recorded for each pixel calculate a gray value image.

The invention is explained in more detail below using exemplary embodiments with reference to the drawings.

Show it:

FIG. 1 schematically shows a SPAD time-of-flight camera,

FIG. 2 shows an exemplary embodiment of a camera system,

FIG. 3 shows a structure according to FIG. 2 with an anamorphic lens system,

FIG. 4 shows a typical SPAD histogram; FIG. 5 shows an evaluation according to the invention.

In the following description of the preferred embodiments, the same reference symbols designate the same or comparable components.

FIG. 1 shows a measurement situation for an optical distance measurement with a time-of-flight camera. The time-of-flight camera or time-of-flight camera system 1 comprises a transmitter unit or an illumination module 10 with illumination 12 and associated beam-shaping optics 15, as well as a receiving unit or time-of-flight camera 20 with receiving optics 25 and a time-of-flight sensor 22.

Infrared light-emitting diodes, VCSELs or laser diodes are preferably suitable as the illumination source or light source 12 . Of course there are others too Radiation sources in other frequency ranges are conceivable, in particular light sources in the visible frequency range are also possible.

A sensor or pixel according to the SPAD principle (single photon avalanche diode) is particularly suitable as the light propagation time measuring sensor.

FIG. 2 schematically shows a time-of-flight camera system in which the illumination 12 is designed in such a way that an essentially strip-shaped illumination light is emitted. This illumination light, which is emitted and reflected by an object 40, naturally also impinges on the time-of-flight sensor 20 in the form of strips.

Illumination 12 and time-of-flight sensor 20 are arranged next to one another and inclined towards one another, so that the field of view FOV of the time-of-flight sensor 20 can be adequately illuminated by the illumination area FOI of the illumination 12 .

Depending on the application, the lighting 12 can optionally also cover a larger lighting area vertically and is not necessarily set to be in the form of strips. The illumination is preferably designed as a VCSEL array, with LEDs or other light sources also being conceivable in principle.

In the case of objects that are far away or are weakly reflective, in particular, there is a risk that the object will not be correctly detected due to the low light energy hitting the sensor. Therefore, the highest possible light sensitivity of the time-of-flight camera system is desirable. This improves the properties of the time-of-flight camera system in the following respects:

-The probability of detecting an object is increased

-Detection is also made possible under less favorable conditions. Unfavorable conditions are in particular a greater distance to the object, lower reflectivity of the object and a higher intensity of the background lighting.

-Reduction of measurement uncertainty of the time-of-flight measurement

-Reduction of the duration of a time-of-flight measurement or increase of the measurement rate A lens system with different focal lengths in the x and y directions can advantageously be proposed to increase the light sensitivity. FIG. 3 shows an example of an arrangement with an anamorphic lens system.

The different focal length can be generated by different methods, for example:

- Use of one or more cylindrical lenses or toric lenses to change the focal length of the lens in one direction,

- Use of special acylindrical or atoric lenses for targeted, direction-specific influencing of the focal length,

- Use of a system of cylindrical and/or toric lenses for targeted, direction-specific influencing of the focal length.

In particular, it is advantageous to minimize the field of view FOVy in the vertical y-direction and maximize the field of view FOVx in the horizontal x-direction:

FOVy < FOVx

By specifically influencing the focal length, the number of pixels that should be sensitive at each operating point can be designed in a targeted manner. In particular, a targeted adjustment of the vertical field of view can be carried out independently of the sensor format.

FIG. 4 shows a typical acquisition of a histogram for a distance determination with, for example, a time-to-digital converter TDC or counter. After the triggering of a transmission pulse, the events occurring at the SPAD pixel are recorded in the usual manner in a predetermined time interval/histogram interval and stored in a memory in a time-resolved manner. This procedure is repeated until a specified number of light pulses is triggered or a specified measurement duration is reached. After the memory has been read out, an evaluation threshold value AS is established on the basis of the histogram, in particular on the basis of the noise component in the histogram. An object distance is then determined on the basis of the bins in which the number of events exceeds this evaluation threshold value AS.

FIG. 5 shows a procedure according to the invention, with which a SPAD sensor remains functional despite a limited memory depth of the histogram memory, even in the case of strong extraneous light. The core idea is to use a pixel-specific exposure time control of the SPAD sensor used as a receiver.

The chip architecture provides that, on the one hand, the distance-dependent events per pixel are recorded in a histogram or histogram interval (e.g. 8 bit per bin) in a first memory and, in parallel, the extraneous light as SPAD events separately per pixel in a second memory (e.g. 16 bits) can be counted in a backlight interval.

The extraneous light / background light is recorded together with the histogram measurement in a common measurement interval. As soon as the number of background light events counted separately for each pixel exceeds a background light threshold HS, the extraneous light counter and the histogram for the respective pixel are frozen, i.e. the measurement is stopped and the associated pixel selection no longer reacts to further SPAD events. Neighboring pixels continue to integrate until they reach the background light threshold or a globally specified measurement duration.

The advantage is that it is impossible for the histogram memory to overflow due to extraneous light or background light. Furthermore, the dynamics of the entire pixel array is increased, since the pixels can be adjusted to different reflectivities and extraneous light intensities of the observed objects through their individual exposure time control.

As outlined in FIG. 8, a number of the backlight events are measured during the backlight interval in which, for example, no light is emitted. In the case shown, for example, x bins are provided for measuring the background light. In the simplest case, only a single bin can be provided. It is also possible to make the bin length longer in the backlight interval than in the histogram interval.

In the example outlined, the number of background light events recorded individually for each pixel exceeds the previously specified background light threshold value HS. As previously described, the further counting of events both in the background light interval and in the histogram interval would now be stopped for each pixel. The second memory for detecting the background light is designed according to the invention in such a way that the memory can be read multiple times during a distance measurement, so that it is also possible to check the number of events in relation to the background light threshold value HS.

In the example shown, the second memory for the background light and the first memory for the histogram each have a bit depth of 8 bits. In principle, however, the second memory for the background light can also have a different bit depth.

As can be seen in the histogram, the object-relevant events have reached saturation. This is not critical, however, because the procedure according to the invention ensures that the background light is at a sufficiently large distance from the saturation limit (max. bit value) so that the bins that are in saturation here can be identified as the object distance.

Based on the number of events in the second memory (backlight memory), an evaluation threshold value is determined, for example using a look-up table. In this case, the threshold value should preferably be set so high that statistically the background light only triggers a false-positive object detection with a negligibly low probability.

The measurement is then carried out with this threshold value and the first point in time at which the histogram exceeds the set threshold value is output as the result of the TOF measurement. In addition, additional information can be output in order to enable subsequent interpolations and to increase the resolution.

Reference List

1 time-of-flight camera

10 lighting module 12 lighting

20 receiver, time-of-flight camera 22 time-of-flight sensor

40 object d object distance q charge

Claims

patent claims

1. Time-of-flight camera with lighting (10) for emitting light pulses and a SPAD time-of-flight sensor (20) with at least one SPAD pixel, wherein the at least one SPAD pixel is formed without distortion, with distance determination occurring at the SPAD pixel

Light events are counted in a time-resolved manner at predetermined time intervals, with light events being assigned to an object if the number of

Light events exceeds an evaluation threshold value (AS), an object distance being determined on the basis of a time difference between an emitted and received light pulse, the SPAD pixel having a first and second memory, the first memory being used for time-resolved storage of the number of

Light events used to form a histogram

Distance determination are detected, is used and the second memory for storing a number of

Background light events is used, with the histogram events being recorded in a histogram interval and the background light events being recorded in a background light interval, with both intervals being run through several times during a predetermined measurement period for a distance measurement, the time-of-flight camera being designed in such a way that the counting of light events in a time-of-flight pixel is stopped when either the predetermined measurement duration is reached or when the number of detected background light events exceeds a preset background light threshold value (HS).

2. The time-of-flight camera as claimed in claim 1, having an evaluation unit which is designed in such a way that the evaluation threshold value (AS) is defined on the basis of the number of background light events recorded in the second memory.

3. Time-of-flight camera according to claim 2, in which the background light threshold value (HS) is defined such that an evaluation threshold value (AS) calculated from the background light threshold value (HS) remains below the maximum bit value of the first memory.

4. Time-of-flight camera according to one of the preceding claims, in which the time-of-flight camera has a time-of-flight sensor with a plurality of SPAD pixels, with a termination period being recorded individually for each pixel, in which the background light events have reached or exceeded the background light threshold value (HS).

5. The time-of-flight camera as claimed in claim 4, in which a gray value image is calculated on the basis of the pixel-individually recorded break-off periods.