JP2017133930A - Distance image generation device and method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device and a method which can generate a highly accurate distance image by a TOF system.SOLUTION: A distance image generation device 10 includes: a first light source 11 for radiating modulation light to an object space; a second light source 12 for radiating a beam-like or a sheet-like laser light which crosses the object space; and an imaging element arranged with a plurality of photosensitive parts in a matrix. The distance image generation device generates an original distance image generated from a phase difference between the modulation light emitted from the first light source and the reflection light from the object space of the modulation light, and includes: an imaging part 13 for generating a lightness image including a reflection light from the object space of the laser light emitted from the second light source; and a correction part 18 for generating a corrected distance image by correcting distance information of the original distance image based on a position of a pixel that records the reflection light of the laser light in the lightness image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a range image generation apparatus based on a time of flight (TOF) method.

  In the field of robot vision and the like, the importance of a distance image sensor that acquires space three-dimensional information in real time is increasing. One such sensor is a distance image sensor (hereinafter simply referred to as a “TOF sensor”) using the TOF method. This sensor generates a distance image by irradiating a target space with modulated light and detecting a phase delay of reflected light received by a planar imaging element for each pixel. For example, Patent Documents 1 and 2 disclose TOF sensors.

Many methods for detecting the phase delay have been proposed. A known example will be described with reference to FIG. As shown in FIG. 5A, the amount of reflected light received every T / 2 period is A0, A2, and T / 2 is delayed from the start point by T / 4 period as shown in FIG. 5B. When the amount of reflected light received by period and A1, A3, phase shift phi _d of the reflected light,
φ _d = arctan {(A1-A3) / (A0-A2)}
Is required. The time t required for reciprocation to the reflecting surface reflecting the light is obtained by t = Tφ _d / 2π, and the distance z to the reflecting surface is obtained by z = ct / 2. Here, c is the speed of light.

  In many TOF sensors, the electric charge generated in the photodiode (PD) according to the amount of light is electronically distributed to a plurality of accumulating units formed on the side of the PD, and the electric charges generated by a plurality of times of light reception are collected together. Read each as a signal. Thereby, since components other than a sensor can be reduced, an apparatus can be reduced in size. In addition, since the phase delay can be calculated simply by taking the difference and ratio of a plurality of signals, the calculation load is light and high-speed measurement is possible.

JP 2003-051988 A JP 2004-294420 A

  The problem with the current TOF sensor is to improve the accuracy of distance information. Since the TOF sensor obtains distance information by electronic processing, it is affected by various electronic errors. The influence of random errors can be reduced by increasing the number of times of light reception under the same conditions and averaging. However, systematic errors cannot be removed by averaging.

  The present invention has been made in consideration of the above, and an object of the present invention is to provide an apparatus and a method capable of generating a distance image with higher accuracy by the TOF method.

  For the above purpose, the present invention corrects the original distance image by the TOF sensor with distance information based on the principle of triangulation.

  The distance image generation apparatus of the present invention includes a first light source that irradiates modulated light to a target space, a second light source that irradiates a beam-like or sheet-like laser light that crosses the target space, an imaging unit, and a correction unit. Have The imaging unit includes an imaging element in which a plurality of photosensitive units are arranged in a matrix, and is based on a phase difference between the modulated light emitted from the first light source and the reflected light from the target space of the modulated light. A distance image is generated, and a brightness image including reflected light from the target space of the laser light emitted from the second light source is generated. The correction unit corrects the distance information of the original distance image based on the position of the pixel where the reflected light of the laser light is recorded in the brightness image, and generates a corrected distance image.

  The distance image generation method of the present invention irradiates the target space with modulated light, receives the reflected light of the modulated light with an imaging device having a plurality of photosensitive portions arranged in a matrix, and reduces the phase delay of the reflected light. A step of detecting each pixel to generate an original distance image, and irradiating a laser beam in a beam shape or a sheet shape so as to cross the target space, and a brightness image including reflected light of the laser beam is emitted by the imaging device. And generating the corrected distance image by correcting the distance information of the original distance image based on the position of the pixel where the reflected light of the laser beam is recorded in the brightness image.

  Preferably, in the distance image generation method, the beam-shaped or sheet-shaped laser light is emitted from a second light source having a fixed position and angle with respect to the imaging element.

  According to the distance image generation apparatus or the distance image generation method of the present invention, it is possible to correct the original distance image by the TOF method by obtaining the distance from the pixel position where the reflected light of the laser light is recorded to the object that reflects the laser light. . Distance measurement using reflected laser light utilizes the principle of triangulation and is not affected by the electronic error of the TOF method, so that a more accurate distance image can be obtained.

It is a figure which shows the structure of the distance image generation apparatus which is one Embodiment of this invention. It is a figure for demonstrating the effect | action of a beam-shaped laser. It is a figure for demonstrating the effect | action of a sheet-like laser. It is a figure which shows the flow of the distance image generation method which is one Embodiment of this invention. It is a figure for demonstrating the well-known method of the distance measurement by a TOF system.

  An embodiment of a distance image generating apparatus of the present invention will be described with reference to FIGS.

  In FIG. 1, the distance image generation device 10 of the present embodiment includes a first light source 11, a second light source 12, an imaging unit 13, a control unit 16, and a correction unit 18.

  The first light source 11 irradiates the entire target space to be measured with modulated light. The wavelength of the light to be modulated is not particularly limited as long as it can be sensed by the image sensor 14 to be described later, but near infrared is preferably used so that the irradiated light is not noticeable. The modulated light is intensity-modulated at a high frequency into a pulse shape or a sine wave shape. The higher the modulation frequency, the better the distance resolution. The modulation frequency is typically suppressed to about several MHz to 10 MHz due to limitations of manufacturing technology and the like. As the light to be modulated, LED light or laser light that can be modulated at such a high frequency is used.

  The imaging unit 13 includes an imaging element 14 and an image generation unit 15. Light from the target space is condensed on the image sensor by a lens or the like. Preferably, a band-pass filter that allows the modulated light emitted from the first light source to pass is disposed in front of the image sensor. Thereby, the influence of environmental light can be suppressed.

  In the image sensor 14, photodiodes (PDs) that are photosensitive portions are arranged in a matrix, and each PD corresponds to each pixel of the distance image. Charges generated in the PD are distributed to a plurality of storage units formed beside the PD, and charges generated by a plurality of light receptions are stored in the storage unit. The accumulated charge is transferred as a signal to the image generation unit for each accumulation unit of each PD.

  The image generation unit 15 performs arithmetic processing on the signal transferred from the PD, and has distance information for each pixel from the phase difference between the modulated light emitted from the first light source and the reflected light received by the imaging device. An original distance image is generated. In addition, the image generation unit generates a brightness image composed of the received light intensity for each pixel.

  The second light source 12 irradiates a beam-like or sheet-like laser beam so as to cross the target space, that is, to cross the field of view of the imaging unit 13. In order to irradiate the sheet-like laser light, it is preferable to use a line laser instead of scanning a beam-like laser. The line laser spreads the laser beam into a sheet shape by passing it through a cylindrical lens. This eliminates the time for scanning. The wavelength of the laser light is not particularly limited as long as it can be sensed by the image sensor 14. When a band-pass filter is arranged in front of the image sensor, the wavelength passes through the filter. The wavelength of the laser light can be set to the same wavelength as the modulated light emitted from the first light source 11, for example.

  Here, the position and the angle of the second light source 12 with respect to the imaging device 14 may be adjustable according to the distance and spread of the target space, but do not change at least during a series of measurements on the same target space. It is fixed.

  The control unit 16 controls the entire distance image generation device 10. For example, the start and end of measurement, the on / off of the first light source and the second light source, the modulation of the first light source, the synchronization of the first light source and the imaging unit, the charge distribution of the image sensor and the signal reading from the storage unit are controlled.

  The correction unit 18 receives the original distance image and the brightness image from the imaging unit 13. In the brightness image generated during the irradiation of the laser beam from the second light source, the reflected light from the target space of the laser beam is reflected. Referring to FIG. 2, when irradiating the beam-shaped laser light 31 from the second light source 12, the reflected light 33 from the surface of the object (target object) 22 in the target space 21 is received by the image sensor 14, and the brightness It appears as a dot on the image. With reference to FIG. 3, when the sheet-like laser light 41 is irradiated from the second light source 12, the reflected light appears linearly on the brightness image. The correction unit obtains the distance from the pixel position where the reflected light is recorded to the object surface that reflects the laser light. The correction unit corrects the original distance image based on the distance thus obtained. The corrected distance image is output to the outside in an appropriate format according to the Ethernet (registered trademark) standard or the like.

  Note that the above-described parts do not necessarily have to be physically separated. For example, the correction unit may be configured integrally with the image generation unit and incorporated in the imaging unit.

  Next, an embodiment of the distance image generation method of the present invention will be described with reference to FIG.

  In FIG. 4, the distance image generation method of this embodiment is implemented using the distance image generation apparatus, and includes an original distance image generation step S1, a brightness image generation step S2, and an original distance image correction step S3. Either the original distance image generation step S1 or the lightness image generation step S2 may be performed first.

  The same method as the known TOF sensor can be used for the original distance image generation step S1. For example: First, modulated light is irradiated from the first light source to the entire target space. While irradiating modulated light, according to the control of the control unit, the imaging unit synchronizes with the modulation period of the modulated light and opens and closes the electronic shutter of the image sensor, and converts the amount of charge accumulated at different timing for each pixel into a signal Then, the image is transferred to the image generation unit. The image generation unit calculates a plurality of signals from the image sensor and generates an original distance image. In the original distance image generation step, preferably, the second light source is turned off. This is to reduce background noise when generating the original distance image.

  In the brightness image generation step S2, laser light is first irradiated from the second light source so as to cross the target space. Next, while irradiating laser light, light from the target space is received by the image sensor at an appropriate time interval to generate a brightness image. The image generation unit may generate a brightness image using a signal based on the charge amount accumulated at any timing in the original distance image generation step S1. This is because the laser light emitted from the second light source is not modulated and is irrelevant to the modulation period of the first light source. In the lightness image generation step, preferably, the first light source is turned off. This is to reduce background noise when generating a brightness image.

  The laser light emitted from the second light source may be a beam or a sheet. By irradiating a beam of laser light, energy consumption can be reduced. On the other hand, there is a possibility that the reflected light cannot be clearly discriminated on the brightness image because the beam hits the corner or step portion of the object surface. When the sheet-like laser light is irradiated, the reflected light appears in a line shape on the brightness image, so that the distance information of a plurality of points on the line can be used.

  The second light source may be configured to be separable from the image sensor, and may be set at an appropriate position every time the brightness image generation step S2 is performed. However, in that case, it is necessary to calibrate the position of the second light source and the emission direction of the laser light each time. Preferably, the two light sources are integrated so that the position and angle of the second light source with respect to the image sensor are constant. In order to integrate the two, they may be housed in a common housing or may be mechanically coupled after being housed in separate housings. This eliminates the need for complicated calibration work.

In the original distance image correction step S3, the original distance image is corrected based on the brightness image. According to the brightness image, the distance from the position of the pixel where the reflected light of the laser beam is recorded to the object surface reflected in the pixel is uniquely determined. The original distance image correction method is not particularly limited. For example, assuming that the distance obtained from the brightness image is z _L and the distance information of the same pixel of the original distance image is z _T , z _L −z _T is added to the distance information of all the pixels of the original distance image, or z _L / z _T can be multiplied. This method is particularly effective for removing systematic errors caused by electronic factors included in the original distance image. In addition, since the original distance image and the brightness image are generated using the same image sensor, there is no parallax between the two images, and there is no possibility of miscorresponding between the two images unlike the stereo viewing method.

  In the above method, other images may be used as appropriate in order to improve the SN ratio between the original distance image and the brightness image. For example, a distance image of only an object may be extracted using a background image obtained by imaging the target space in a state where there is no object whose distance information is to be measured.

  The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the technical idea.

  For example, a plurality of second light sources may be used. For example, two line lasers may be arranged and used so that the sheet-shaped lights intersect each other. In this case, the distance information based on the brightness image can be used in a wider range by sequentially switching the two second light sources to generate the brightness image.

DESCRIPTION OF SYMBOLS 10 Distance image generation apparatus 11 1st light source 12 2nd light source 13 Image pick-up part 14 Image pick-up element 15 Image generation part 16 Control part 17 Input part 18 Correction part 19 Output part 21 Target space 22 Object (object)
31 Beam-shaped laser beam 33 Reflected beam-shaped laser beam 41 Sheet-shaped laser beam

Claims (3)

A first light source that irradiates the target space with modulated light;
A second light source for irradiating a beam-like or sheet-like laser beam across the target space;
A plurality of photosensitive units includes an image sensor arranged in a matrix, and generates an original distance image from a phase difference between modulated light emitted from the first light source and reflected light from the target space of the modulated light, An imaging unit that generates a brightness image including reflected light from the target space of the laser light emitted from the second light source;
A correction unit that corrects the distance information of the original distance image based on the position of the pixel that records the reflected light of the laser light in the brightness image, and generates a corrected distance image;
A distance image generating apparatus having: The target space is irradiated with modulated light, the reflected light of the modulated light is received by an imaging device in which a plurality of photosensitive portions are arranged in a matrix, and the phase delay of the reflected light is detected for each pixel, and the original distance image A step of generating
Irradiating a beam-like or sheet-like laser beam so as to cross the target space, and generating a brightness image including the reflected light of the laser beam by the imaging device;
Correcting the distance information of the original distance image based on the position of the pixel that recorded the reflected light of the laser light in the brightness image, and generating a corrected distance image;
A distance image generation method comprising: The beam-like or sheet-like laser light is emitted from a second light source having a fixed position and angle with respect to the image sensor.
The distance image generation method according to claim 2.

Images