KR102420954B1 - Pattern laser based 3d information sensor - Google Patents

Abstract

The present invention relates to a laser source irradiating a laser beam generated by a point light source, a line generator that transmits the point light source at a predetermined distance from the tip of the laser source and changes it into line light, and a line generator disposed at the tip of the line generator to generate the line light A CCTV camera module and TOF (Time of Flight) A receiver consisting of a ToF camera module having a sensor and a TOF image image of a 2D pattern light received from the receiver and a CCTV image image are synchronously mapped and the proximity distance of each area of a 3D depth-based object is detected. It relates to a pattern laser-based 3D information sensor configured to include a control unit that performs a function.

Description

PATTERN LASER BASED 3D INFORMATION SENSOR

The present invention relates to a pattern laser-based 3D information sensor, and more particularly, to a pattern laser-based 3D information sensor capable of extracting three-dimensional information through a spatial scan based on a pattern laser light source.

Recently, object detection technology using a LIDAR (Light Detection and Ranging) sensor is attracting attention as the core of the front and rear obstacle detection systems of moving objects (autonomous driving vehicles, drones, electric wheelchairs, etc.). In addition, as the installation of vehicle obstacle warning devices such as ultrasonic sensors, infrared sensors, and video cameras becomes legally mandatory, there is an increasing demand for rear and surround detection systems that provide various information in a complex and integrated manner by recognizing the surrounding environment of the vehicle.

Therefore, unlike the rotation method, a non-rotating lidar sensor that covers only a limited viewing angle, such as a low-cost, lightweight lidar (Flash LIDAR) with a simple structure, is being commercialized, and it can be said that it is a suitable form to be applied to a short-distance sensing system. .

Such a non-rotating lidar sensor needs to provide a light emitting unit using a more advanced laser beam that detects 3D depth information for reliable obstacle detection in various types of outdoor environments (night, rain, fog, etc.) have.

On the other hand, in order to form a laser beam, a lens system in which a plurality of lenses having different functions and shapes are combined is generally used. In the case of forming a laser beam using such a lens system, since the plurality of lenses partially absorb the laser beam emitted from the laser source, there is a problem in that the energy of the laser beam becomes weaker as it passes through the plurality of lenses.

In addition, the laser beam formed by using the lens system is focused to the center, and therefore the homogeneity of the laser beam is greatly different at the center and the edge of the linear or planar beam, so a uniform linear or planar beam is formed. It is difficult to do this, the apparatus is complicated, it is difficult to align a plurality of lenses, and there are problems in that the manufacturing cost increases due to the use of expensive lenses.

As a conventional technique for solving the above problems, a plurality of cameras capable of configuring a stereo camera for acquiring a structured light pattern from an image from an object in Korean Patent Application Laid-Open No. 10-2013-0055088 (2013.05.28.) ; and a first depth value for each pixel between the structured light source unit and one camera among the plurality of cameras, and a second depth value for each pixel from a binocular image obtained from a stereo camera configured with a plurality of cameras for each pixel area of the object. A cost function that is estimated using disparity values of matching pixels, and using a first reliability value for the estimated first depth value for each pixel and a second reliability value for the estimated second depth value for each pixel in the depth value estimation process A structured light pattern-based 3D image information estimation apparatus including a depth map estimator that calculates using and methods are disclosed.

The present invention was derived to solve the problems of the prior art, and an object of the present invention is to convert a laser beam generated as a point light source into line light and generate a line generator based on a grating structure. An object of the present invention is to provide a patterned laser-based 3D information sensor that forms 2D pattern light in a constant radiation direction with a wide radiation angle by grating diffraction.

In addition, another object of the present invention is to form a line beam by diffracting the beam at regular intervals using a laser light source, and a 2D-pattern light source for surface light that forms a constant 2D pattern beam by locating the lines of the formed grating pattern at right angles to each other It is intended to provide a module.

In addition, an object of the present invention is to provide a light source module that provides a large-area radiation light source having a near-radiation characteristic of 120 degrees or more with a grating-based light diffraction 2D pattern laser.

Another object of the present invention is to provide a 3D information sensor as a module in which a light emitting unit, a light receiving unit, and a control (processing) processing unit are integrated.

A pattern laser-based 3D information sensor according to an aspect of the present invention for solving the above technical problem is a laser source that irradiates a laser beam generated by a point light source, and a line light by passing the point light source from the tip of the laser source at a predetermined distance. A line generator that converts into a line generator and a uniform grating that is disposed at the tip of the line generator and diffracts the line light at a predetermined angle. Synchronization mapping of the TOF image image of the 2D pattern light received by the receiver and the CCTV image (Mapping) and has a feature configured to include a control unit that performs a function of detecting a proximity distance for each area of a 3D depth-based object.

In addition, the uniform lattice body of the light source unit of the present invention has a lattice structure consisting of protrusions in the form of a triangular line pattern having a predetermined pitch and a predetermined inclination angle on a transparent film made of polycarbonate (PC) or PET (polyethylene terephthalate) material ( It is characterized by the formation of gratings.

In addition, the pitch interval of the triangular line pattern of the uniform lattice body of the present invention is 100 μm, and the inclination angle is 35 degrees to 40 degrees.

In addition, the uniform lattice body of the present invention is formed in a double structure superimposed on each other so that the directions of the triangular line lattice pattern are orthogonal to each other to form a 2D pattern light that is uniformly emitted in vertical and horizontal directions.

In addition, the receiver of the present invention is characterized in that it further includes a phase meter for measuring and correcting a phase difference between the received 2D pattern light and the 2D pattern light emitted from the light source unit.

In addition, when calculating the average of depth values in the video image of the received 2D pattern light, the control unit of the present invention is characterized in that the weight reduction method is applied as the distance from the center point of the object region is increased.

In addition, the control unit of the present invention merges and expands the peripheral area when the depth value is within a certain range for a certain area of the video image of the received 2D pattern light, and when the difference in the depth value is greater than a certain range, There is a feature of splitting a detailed area.

According to the pattern laser-based 3D information sensor of the present invention described above, unlike the existing obstacle detection methods (ultrasound, infrared, image camera), it provides a laser-type high-wide-angle high-power light source unit that is robust to the external environment to provide a variety of moving objects (self-driving vehicles, It has the advantage of being applicable to the front and rear obstacle detection systems of drones, electric wheelchairs, etc.).

In addition, the present invention detects 3D depth information based on 2D pattern light, thereby providing a reliable obstacle detection function applied in various types of outdoor environments (night, rain, fog, etc.), driver convenience, and road traffic It has the effect of providing surround sensing (Lidar, image sensor, real-time image processing) information that contributes to safety and life protection.

In addition, the present invention has the advantage of being able to replace the existing front and rear cameras and obstacle sensors as a detection system that provides integrated three-dimensional (large area and distance) information with high accuracy.

1 is a block diagram showing the configuration of a pattern laser-based 3D information sensor according to an embodiment of the present invention.
2 is an exemplary diagram illustrating a 2D patterned light generation path of a patterned laser-based 3D information sensor according to an embodiment of the present invention.
3 is a perspective view showing a uniform grid according to an embodiment of the present invention and an exemplary view showing details.
Figure 4 is an example image of the TOF camera module for receiving the light source irradiated by the 2D pattern light source according to the present invention.
5 is an exemplary view showing a demonstration image of proximity object detection based on 3D depth data according to the present invention.

The terms or words used in the present specification and claims should not be construed as being limited to their ordinary or dictionary meanings, and the inventor may properly define the concept of the term in order to best describe his invention. Based on the principle that there is, it should be interpreted as meaning and concept consistent with the technical idea of the present invention.

Therefore, the configuration shown in the embodiments and drawings described in the present specification is only the most preferred embodiment of the present invention and does not represent all the technical spirit of the present invention, so various equivalents that can replace them at the time of the present application It should be understood that there may be water and variations.

In addition, in the description of the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary diagram showing the configuration of a pattern laser-based 3D information sensor according to an embodiment of the present invention, Figure 2 is a 2D pattern light generation path of the pattern laser-based 3D information sensor according to an embodiment of the present invention 3 (a) is a perspective view showing a uniform grid according to an embodiment of the present invention, and FIG. 3 (b) is an exemplary view showing details of the uniform grid.

Referring to the illustration, the pattern laser-based 3D information sensor of the present invention is configured to include a light source unit 100 , a receiver unit 200 , and a control unit 300 .

The light source unit 100 includes a laser source 110 for generating a laser beam, a line generator 120 for changing a point light source generated from the laser source 110 into line light, and diffracting the generated line light at a predetermined angle. For this purpose, at least one uniform grating body 130 is formed in a uniform grating structure to generate a 2D pattern laser beam.

The laser source 110 generates a laser beam serving as a light source, and may be composed of either a light emitting diode (LED) or a laser diode (LD), and in particular, a high-power laser diode (LD). ), it is more preferable if it is a light source that can form a condensed light source.

In this case, a focal point condensing lens may be provided on the front surface of the laser source 110 for condensing the laser light, and the focused point light source may be irradiated therethrough.

The line generator 120 serves to convert the focused point light into line light and generate it, and transmits the point light output from the tip of the laser source 110 at a predetermined distance to change it into line light.

In this case, the line generator 120 may be applied using either a cylinder lens (CL) type or a grating structure type.

A cylinder lens (CL) is a lens used to change the aspect ratio of an image by focusing incident light on one line or by magnifying an image only on a single axis. That is, the cylinder lens can be said to be a lens suitable for generating a line in which incident light is focused in a single line in a cylindrical shape and is elongated.

In addition, using the same structure as the uniform grating 130 of the present invention, the focused point light may be transmitted in a plurality of line forms, and an optical filter capable of transmitting the point light in a plurality of line forms may be used. . Accordingly, the line generator 120 of the present invention can be changed and applied as long as it can transmit the focused point light and change it into a plurality of line lights.

Accordingly, the point light from the laser source 110 is transmitted through the line generator 120 to form line light.

The uniform grating 130 is a diffraction grating that is disposed at the front end of the line generator 120 at a predetermined distance and can diffract the incident laser light into beams of different directions, and the uniform grating of the present invention ( 130 applies a structure formed by applying a grating made of protrusions in the form of a triangular line pattern having a predetermined pitch and a predetermined inclination angle as shown in the figure.

In addition, according to implementation, the uniform grid body 130 may include a bottom portion substantially parallel to the main surface by a predetermined interval between adjacent protrusions in the shape of a triangular line.

In this case, the pitch interval of the triangular-line grid structure is about 100 μm, and the brazing inclination angle may be formed at 35 degrees to 40 degrees.

Accordingly, according to a more preferred embodiment as shown in (b) of FIG. 3, the pitch interval of the triangular line grid structure may be formed in a triangular line pattern having a brazing inclination angle of 100.21 μm and 38 degrees. Meanwhile, the triangular-line grid structure may have a height of 33.40um and a thickness of 0.20mm.

The material of the uniform grid body 130 is a triangular line grid structure on a transparent film made of a uniform polycarbonate (PC) or PET (polyethylene terephthalate) material of 0.2 mm to 1 mm.

The uniform grid body 130 of the present invention is superimposed on each other in a pair so that the line directions of the triangular line grid pattern of the uniform grid body 130 are orthogonal to each other at the tip of the line generator 120 are combined or double , the light source can form a 2D pattern light that is emitted in a constant direction of vertical and horizontal, and can be used for scanning.

Accordingly, the line light incident from the line generator 120 may be converted into a planar light source formed as a uniform pattern beam diffracted at a predetermined angle by the uniform grating body 130 .

The light source unit 100 having the above configuration transmits a laser beam having a wavelength of 850 nm (or 850 nm ± 10 nm) to a radiation angle (Field of View, FOV) in a range of about 120 degrees horizontally and about 70 degrees vertically, and the radiation distance is the maximum. It is irradiated with light in a 2D pattern that can form constant coordinates around 20m.

In addition, the receiving unit 200 of the present invention is provided with a frame rate of a certain rate or higher in order to receive the two-dimensional pattern light emitted from the light source unit 100, and CCD ( A CCTV camera module 210 having a high-performance color image sensor using either Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) and TOF (Time of Flight) for stable outdoor operation among various methods of 3D depth camera It may include a 3D Time of Flight (ToF) camera module 220 having a sensor.

Accordingly, the receiver 200 of the present invention is designed to go through a synchronization process to operate regardless of an external light source or signal. The synchronization process includes a circuit that receives only the signal according to the light source unit 100 and removes other signals caused by the external environment.

Accordingly, the receiver 200 receives the laser light reflected by the two-dimensional pixel matrix, measures the phase difference between the transmitted light and the reflected light in analog form, converts it into a digital value through A/D conversion (Converter), and the converted It is made by including a circuit for calculating a two-dimensional depth (depth) as a digital value.

That is, the receiver 200 corrects the phase difference by using a phase meter that corrects the phase difference between the phase of the light transmitted from the light source 100 and the received reflected light, thereby removing other signals caused by the external environment. (100) is made to calculate the 3D ToF (Time of Flight) of the emitted signal.

In addition, the control unit 300 synchronizes the TOF image image of the 2D pattern light received from the receiving unit 200 and the CCTV image image, and clusters the 3D depth data, and It divides the space and performs the function of detecting the proximity distance for each area of the object.

To this end, the control unit 300 detects an effective information display module through visible light image and 3D object area synchronization (Sync), a space division module through 3D depth area division and synthesis (split & merge), and average distance information for each 3D object. And various image processing functions including an image synthesis module, a phase synchronization control module according to TOF (Time of Flight) of the light source and receiver, an image processing module through synchronization mapping between TOF cameras and CCTV cameras, and functional modules such as a monitor interface, etc. Modules may be included.

At this time, the spatial division module through region division and synthesis by 3D depth performs a function of merging and expanding detailed regions when the depth values are similar, and splitting the detailed regions when the depth values are different. do.

In addition, the average distance information detection and image synchronization (Sync) module for each 3D object calculates the average of the 3D depth of the detailed area included in the object, and may be performed by applying a weight reduction method as the distance from the center point of the object area increases.

The control unit 300 may be implemented as at least one device selected from a logic circuit, a programming logic controller, a microcomputer, a microprocessor, and the like. A communication module may be provided to interwork with an external device, or may be applied by being coupled to the communication module.

4 is an image example diagram of a TOF camera module that receives the light source irradiated by the 2D pattern light source unit according to the present invention.

As shown in FIG. 4 , it can be seen that the TOF camera module 220 can acquire an image for extracting 3D distance information of an object (trash can) with respect to a 2D pattern light source at a distance of 2 m from the external environment.

In addition, FIG. 5 is an exemplary view showing a demonstration image of 3D depth data-based proximity object detection according to the present invention. In detecting a proximity object by the TOF camera module 220 and the CCTV camera module 210, the TOF camera module ( 220) and CCTV camera module 210, it is shown that it is possible to detect a nearby object through spatial division through region division and synthesis by 3D depth and average distance information detection and image synchronization process for each 3D object. are giving

By using the structure of the uniform grating 130 as in the present invention described above, the focal length generated by a general lens is eliminated, and a uniform light distribution with a long measurement distance is maintained due to the directional radiation of light according to a deep depth. It is possible to provide a pattern laser-based 3D information sensor that can improve recognition in indoor and outdoor environments due to less influence from external light sources, and achieve stability of the human body due to energy dissipation.

As described above, in the detailed description of the present invention, preferred embodiments have been described, but those of ordinary skill in the art will not depart from the spirit and scope of the present invention as set forth in the following claims. It will be understood that various modifications and variations of the present invention may be made.

100: light source unit 110: laser source
120: line generator 130: uniform grid
200: receiver 210: CCTV camera
220: ToF camera 300: control unit

Claims (7)

A laser source that irradiates a laser beam generated by a point light source, a line generator that transmits the point light source from the tip of the laser source at a predetermined distance to change it into line light, and a line generator that is disposed at the tip of the line generator to direct the line light at a predetermined angle Consists of a uniform grating that diffracts, and a light source that converts and emits two-dimensional pattern light;
A receiver comprising a CCTV camera module using either a CCD or CMOS sensor for receiving the two-dimensional pattern light emitted from the light source and a ToF camera module having a TOF (Time of Flight) sensor;
And a control unit that performs a function of synchronizing the TOF image and the CCTV image image of the 2D pattern light received by the receiver and performing a function of detecting a proximity distance for each area of a 3D depth-based object,
The uniform lattice body,
It is formed in a double structure superimposed on each other so that the directions of the triangular line grid pattern are mutually orthogonal to form a 2D pattern light that is constantly radiated in vertical and horizontal directions,
The control unit is
The 3D depth data is clustered (clustered) and the space of the video image is divided to detect the proximity distance for each area of the object,
The control unit is
Effective information display module through visible light image and 3D object area synchronization (Sync), space division module through 3D depth area division and synthesis (split & merge), average distance information detection and image synthesis module for each 3D object, light source and receiver A pattern laser-based 3D information sensor, characterized in that it comprises a phase synchronization control module according to the TOF (Time of Flight), and an image processing module through synchronization mapping between the TOF camera and the CCTV camera.
The method according to claim 1,
The uniform grating body of the light source part has a grating structure composed of protrusions in the form of a triangular line pattern having a predetermined pitch and a predetermined inclination angle on a transparent film made of a polycarbonate (PC) or PET (polyethylene terephthalate) material. Pattern laser-based 3D information sensor, characterized in that made. 3. The method according to claim 2,
A pattern laser-based 3D information sensor, characterized in that the pitch interval of the triangular line pattern of the uniform grid is 100 μm, and the inclination angle is 35 to 40 degrees.
delete The method according to claim 1,
The receiving unit is a pattern laser-based 3D information sensor, characterized in that it further comprises a phase meter (Phase Meter) for measuring and correcting the phase difference between the received 2D pattern light and the 2D pattern light emitted from the light source unit.
The method according to claim 1,
The control unit is a pattern laser-based 3D information sensor, characterized in that when calculating the average of the depth values in the video image of the received 2D pattern light, the pattern laser-based 3D information sensor, characterized in that it applies a weight reduction method as the distance from the center point of the object region.
The method according to claim 1,
The control unit merges and expands the peripheral region when the depth value is within a certain range for a certain area of the video image of the received 2D pattern light, and divides the detailed area when the difference in the depth value is greater than a certain range ( pattern laser-based 3D information sensor, characterized in that split).

Images