KR20180134280A - Apparatus and method of face recognition verifying liveness based on 3d depth information and ir information - Google Patents

Abstract

The present invention relates to a face recognizing device for confirming liveness based on 3D depth information and infrared information comprising: an extracting device, a 3D verifying module, and a face matching module. The extracting device includes a depth information processing chip which extracts and outputs a depth map by receiving simultaneous data of an RGB image (visible rays) and an IR image (infrared rays) through a single lens from a dual aperture single lens 4-color sensor camera. When a value calculated from depth information of the depth map exceeds a 3D reference value, the 3D verifying module generates a 3D reference passing signal. After the 3D reference passing signal is received, the RGB image is compared with an RGB template of a previously stored face template. If the RGB image and the RGB template match within error limits, the face matching module generates a face recognition passing signal. The present invention enhances security.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face recognition apparatus and method for verifying whether a living body is based on three-dimensional depth information and infrared information,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a face recognition apparatus and method for verifying whether a living body is based on three-dimensional depth information and infrared information.

In general, a security system based on face recognition is known. This security system can play a role of allowing a person who is verified through face recognition to perform a specific operation, such as accessing a door, accessing an information processing device, and the like.

In the face recognition method, a security system including a camera for inputting an image and a control device for comparing the input image with a face template stored in advance is known. According to this, after the face template is created in advance from the images of people allowed to use the security system, at the application time of the actual security, the real-time input image is captured by the camera with respect to the person requesting the use, It is judged whether or not the minutiae are coincident with each other in comparison with the template to determine whether or not the use is permitted.

As described above, in the conventional face recognition technology, when the pattern of the face image matches only within the limits of the error, when a sophisticated photograph (two-dimensional) or a face copied model (three-dimensional) The user is exposed to a security risk of failing to make a false pass for a facial pattern.

That is, when an input image is generated, an attempt is made to disable the security determination by causing the camera to take a picture of a person who is allowed to use the security. That is, an input image of a sophisticated 2D photograph can be extremely similar to a stored face template, so that the match degree of the feature point can be determined to be high. Therefore, there is a problem that the security system can make a false judgment.

To overcome this problem, a technique for performing additional verification of information that is not a photograph is proposed. That is, by detecting flickering of the eyes, movement of the face and hair, it is verified that it is not a simple photograph.

However, flickering of the eyes and movement of the face can disable the security judgment by, for example, causing the camera to capture the face moving image of the person who is allowed to use the security when the input image is generated. That is, the moving image can blink, the face, and the hair can also move, so that the input image by the moving image can be judged to pass the additional verification as a living body. Therefore, there is a problem that the security system can also make a false judgment.

On the other hand, in the following Patent Document, there is disclosed a device for automatically detecting a face of a person in a video captured by a security system, means for finding an eye region after the normalization process, means for detecting infrared reflection occurring in the pupil of the eye by infrared illumination And a technique for determining whether a fake image is hacked to improve the performance of a security system related to face recognition and to improve the usability.

Patent Publication 10-2010-0097866

In the technique of the patent document, it is necessary to find a face from an input image through image processing and to search for eyes again. Therefore, high-speed processing hardware and software required for processing are required.

* In order to detect the infrared reflection of the pupil of the eye, means for judging processing of the light emission, the light reception, and the infrared reflection of the infrared illumination are required. These means are composed of hardware and software separately provided in addition to the camera.

For people wearing glasses, sunglasses, or contact lenses, infrared reflection may not occur even by infrared light emission. In this case, the technique becomes obsolete.

The present invention has been made to solve the problems of the related art, and it is an object of the present invention to provide a face recognition apparatus and a face recognition apparatus which combines a three-dimensional depth information and an infrared information extracting apparatus of a dual aperture single lens It is intended to provide a face recognition apparatus and method for confirming whether or not a living body is verified so as to enhance security by mounting a biometric verification function in two stages without equipment.

It is another object of the present invention to provide a face recognition apparatus and method for confirming whether or not a living body is present without searching for a face from an input image through an image processing and searching eyes again.

It is another object of the present invention to provide a face recognition apparatus and method for confirming whether or not a living body is present, which is capable of minimizing the size and cost of a device without requiring a means for judging processing of infrared ray illumination and infrared ray reflection.

And to provide a face recognition apparatus and method for verifying whether or not a living body is applicable, which can be applied to a person wearing glasses, a sunglass, or a contact lens.

According to another aspect of the present invention, there is provided a face recognition apparatus for verifying whether a subject is living or not based on three-dimensional depth information and infrared information. The face recognition apparatus includes a dual Aperture single lens An extraction device including a depth information processing chip for receiving simultaneous data of an RGB image (visible light) and an IR image (infrared) from a single lens 4-color sensor camera through a single lens and extracting and outputting a depth map; , A stereoscopic verification module that generates a stereoscopic reference pass signal when the value calculated from the depth information from the depth map is equal to or more than a stereoscopic reference value, And a face matching module for generating a face recognition passing signal when the difference is within an error limit.

Here, the body temperature verification module may generate a body temperature reference passage signal when the intensity of the wavelength corresponding to the body temperature of the IR image is equal to or higher than the body temperature reference value, in place of or in addition to the stereoscopic verification module. .

Here, the face matching module is preferably configured to further compare the IR image with a previously stored IR template.

Alternatively, the face matching module is preferably configured to further compare depth information from the depth map with a previously stored depth information template.

And the filter used in the double diaphragm of the barrel of the double-aperture single-lens four-color sensor camera has a shallow depth of field (R, G, B) of relatively small first F- (IR) having a wavelength of 8 탆 or more and less than 14 탆, and having a spectral characteristic by being implemented by a filter having a relatively large second F value and having a deep depth, The color sensor for visible light is fabricated so as to receive a signal from the filter having the depth of the first F value in accordance with the spectroscopic characteristic, It is preferable to be constructed so as to receive a signal from a filter of a depth value.

On the other hand, if the body temperature verification module exceeds the predetermined body temperature reference value based on the RGB value (0 to 255) when the Bayer pattern is distributed as a video stream, It is preferable that it is determined that the IR signal of the subject is strong.

Meanwhile, the method of the present invention is a face recognition method for confirming whether or not a living body is based on three-dimensional depth information and infrared information. The method includes the steps of: extracting a single lens (single lens) An extracting step of extracting and outputting a depth map by receiving simultaneous data of an RGB image (visible ray) and an IR image (infrared ray) through the depth map; compared with this is the three-dimensional reference value or more, three-dimensional and three-dimensional verification step of generating a pass signal based on, after receiving the three-dimensional reference pass signal, RGB template of the face templates stored in advance on the RGB images, if they match within a margin of error facial recognition And a face matching step of generating a pass signal.

Here, the body temperature verification step may be performed in place of or in addition to the stereoscopic verification step to generate a body temperature reference passage signal when the intensity of the wavelength corresponding to the living body temperature in the wavelength of the IR image is equal to or higher than the body temperature reference value .

According to the present invention, the conventional face recognition technology is combined with a three-dimensional depth information and an infrared information extracting device of a dual aperture single lens (single lens) method, and a two-step biometric confirmation function There is provided a face recognition apparatus and method for confirming whether or not a living body is present so as to enhance the security of the face.

There is provided a facial recognition apparatus and method for confirming whether a living body is living, without requiring a process of finding a face from an input image through image processing and searching eyes again.

There is provided a face recognition apparatus and method for confirming whether or not a living body is present, which is capable of minimizing the size and cost of the device without requiring a means for processing the determination of infrared light emission and infrared reflection.

There is also provided a face recognition apparatus and method for verifying whether or not a living body can be applied to a person wearing glasses, a sunglass, or a contact lens.

FIG. 1 is a block diagram of a face recognition apparatus for confirming whether a living body is based on three-dimensional depth information and infrared information according to an embodiment of the present invention.
2 is a graph showing infrared wavelengths corresponding to wavelengths of visible light and infrared light and body temperature of the human body.
3 is a block diagram of a face recognition apparatus for confirming whether a living body is living based on three-dimensional depth information and infrared information according to another embodiment of the present invention.
Fig. 4 is an exemplary photograph of a case where the body temperature determination is performed in a bi-directional manner according to a reference value.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, for elements having the same function by the same structure, the same reference numerals are retained even if the drawings are different, and detailed description thereof may be omitted. Also, to be connected includes the concept of passing through the medium in the middle. And a certain functional block may be configured to be absorbed by other functional blocks or may be divided into a plurality of different blocks and implemented.

≪ First Embodiment-Stereoscopic Verification >

The face recognition apparatus for confirming whether or not living bodies are living bodies is based on three-dimensional depth information and infrared information. As shown in FIGS. 1 and 3, the face recognition apparatus includes an extraction device 11 , a stereoscopic verification module 15, and a face matching module 13, 13a .

The extraction device 11 includes an RGB image (visible light) 11a and an IR image (infrared light) 11b (visible light) through a single lens from a Dual Aperture Single Lens 4-color sensor camera 10, And a depth information processing chip SoC for extracting and outputting the depth map 11c.

The double-aperture single-lens four-color sensor camera 10 separates an image received through one lens into visible light color light (RGB) and infrared monochrome light (IR) while separating the light through a double diaphragm, Color sensor composed of four light receiving portions (R, G, B, IR) constituting each pixel unit of the device. Thus, according to the camera 10, simultaneously generated RGB images (color images) 11a and IR images (monochrome images) 11b having the same viewpoint passing through one lens are outputted. The RGB image 11a and the IR image 11b are transmitted to the depth information processing chip SoC.

The depth information processing chip SoC is a semiconductor chip for extracting depth information of a double-iris single-lens system, for example, model name DR1152 of eWBM Co., Ltd., Depth information extraction and infrared image extraction. As described above, the technique of extracting the depth by the double-aperture single-lens four-color sensor camera 10 can be already used by the applicant of the system-on-a-chip (SoC) and the depth information system in which the solution is developed.

From the depth information processing chip SoC, as shown in Figs. 1 and 3, the depths calculated from the distribution of the depth information through the well-known processing process as well as the RGB image 11a and the IR image 11b, A map (Depth Map) 11c is output. The present invention enhances security processing using these RGB image 11a, IR image 11b, and depth map 11c. In particular, the IR image 11b can be used simultaneously for confirmation of body temperature characteristics, image pickup at low illumination, and the like.

The stereoscopic verification module 15 is a module for generating a stereoscopic reference pass signal when the value calculated from the depth information from the depth map 11c is equal to or larger than the stereoscopic reference value. As described above, the present invention utilizes depth information, which is the output of a conventional double-iris single-lens depth information extraction solution technique, as three-dimensional (3D) object identification data.

The face matching module 13 or 13a compares the RGB image 11a with the RGB template 12a of the stored face template 12 after receiving the stereoscopic reference passage signal And generates a face recognition pass signal. Here, the face matching module 13 of FIG. 1 is a conventional basic RGB image-based two-dimensional (2D, flat) face recognition algorithm that simply compares the captured RGB image 11a with a previously stored RGB template 12a to be.

<Operation>

In the present invention, the operation of the simple 2D face matching module 13 is controlled to operate by using the stereoscopic reference pass signal of the stereoscopic verification module 15 as a trigger, thereby preventing an error in security judgment. That is, face matching is performed only when the object is three-dimensional, and security is passed when the face matching is successful.

<Effect>

The present invention relates to an image processing apparatus and a method of processing an image, comprising: a double-aperture single-lens camera having a four-color sensor; a processing chip for extracting depth maps by receiving RGB and IR signals; And the like are integrated as one. As described above, when separate technologies are configured as a single module for extracting depth information and implemented as a face recognition device and simultaneously forged with a two-dimensional image (photograph or video) or a three-dimensional model, It is possible to implement an efficient system that can do this.

With this technology, a face recognition device with a high level of security can be achieved with a small form factor and low cost.

&Lt; Second Embodiment-Body Temperature Verification >

If the intensity of the wavelength corresponding to the biological body temperature in the wavelength of the IR image 11b is equal to or higher than the body temperature reference value in place of or in conjunction with the stereoscopic verification module 15 or the stereoscopic verification module 15, And a body temperature verifying module 14 for generating a body temperature .

Using the infrared waveform characterization from the human body, it can be confirmed whether the body temperature is sensed. The characteristic of the body temperature shows a local maximum waveform at a wavelength of infrared (IR) near 8 to 14 탆, particularly around 9.5 탆. In the present invention, the body temperature can be analyzed and sensed only by infrared image analysis from the camera module 10 of the double diaphragm system.

The body temperature verification can be implemented by mounting the body temperature detection algorithm through the infrared analysis on the AP or the improved depth information processing chip (SoC).

Of course, IR-FPA (Infrared Focal Plane Array) sensors or thermal imaging cameras should be used to use existing technologies for precise body temperature measurement. However, the present invention does not measure such a precise body temperature, but merely confirms the temperature of a living body. Therefore, when the intensity of the infrared image from the camera of the double-aperture type using the 4-color sensor exceeds a calibrated intensity threshold, it can be determined that the image sensor is a living body. That is, instead of using a general IR-FPA sensor, body temperature sensing may be performed by reading the IR image output from the 4-color sensor of FIG. 2 as a body temperature if the intensity of the IR image exceeds a predetermined threshold.

<Effect>

Body temperature, particularly a human body temperature, appears in an infrared (IR) region having a wavelength of about 9.5 탆 and a wavelength of about 8 탆 or more and less than 14 탆 as peak values, as shown in Fig. And no matter how sophisticated, body temperature with this pattern can not be detected from photographs, moving images, or moving mannequin models.

Of course, thermal energy can be detected from an electronic device that reproduces a photograph or a moving image. However, since the wavelength pattern of the thermal energy exhibits a different pattern from that of the living body temperature, the intensity of the living body body temperature It is possible to distinguish the electronic device from the living body. However, when the intensity pattern in the other wavelength region is stronger than the reference value, a negative judgment routine may be added so as to suspend passage of the body temperature verification in order to avoid judgment error by the electronic device or the like.

Therefore, more reliable verification of the living body is possible by body temperature verification.

&Lt; Third Embodiment-Face Matching Further Including IR Matching >

The face matching module 13a may be configured to further compare the IR image 11b with the previously stored IR template 12b. 3, in place of the conventional two-dimensional face recognition algorithm 13, not only the RGB data 11a but also the IR data 11b already provided in the solution of the double-iris method are used , An algorithm 13a in which the recognition rate is increased may be used.

Using this, it is possible to compare feature points that can not be obtained only by the comparison of the RGB image 11a by comparing the IR image 11b, which is an all-focused image for all depths. It is possible.

<Fourth Embodiment-3D Matching by Face Matching Further Including Depth Information Matching>

The face matching module 13a may be configured to further compare the depth information from the depth map 11c with the previously stored depth information template 12c. That is, instead of the conventional two-dimensional face recognition algorithm, as shown in FIG. 3, an algorithm with a higher recognition rate can be used by using not only RGB but also depth information provided in a solution of a double-iris method, as shown in FIG.

Using this information, it is possible to compare the three-dimensional feature points of the face such as a person with a high nose or a person with a protruding cheekbone, for example, so that the recognition rate can be drastically improved.

<Example of Optimization of Infrared Area for Body Temperature Verification>

The filter used in the double diaphragm of the mirror barrel of the double-aperture single-lens four-color sensor camera 10 has a relatively small first F value (for example, for a visible light (R, G, B) (IR) having a wavelength of 8 占 퐉 or more and less than 14 占 퐉, for example, by a filter having a relatively large depth of second F value .

In this case, the 4-color sensor for detecting signals of respective wavelengths of the image signal may be configured such that a color sensor for visible light is manufactured to receive a signal from a filter having a depth of the first F value in accordance with the spectral characteristic, May be fabricated to receive a signal from a filter having a depth of a second F-number.

Here, the general optical system for extracting the double iris depth information is often used to detect near infrared rays as an IR input when implementing the depth information extracting function, but there is a side effect that the visible ray image is reddishly contaminated by near infrared rays near to visible light. In the present invention, instead of the near-infrared rays, a dual aperture can be formed by implementing far-infrared rays in a region of 8 to 14 占 퐉, which is a typical infrared wavelength emitted from the human body, with a filter having a large F value (deep depth). Thus, it is possible to confirm the human body temperature without deteriorating the original depth information extracting function and to have the effect of eliminating the side effect that the visible ray image is contaminated with red light by using the far infrared rays which are not in contact with the visible light ray. And can be easily implemented without greatly increasing the hardware form factor.

<Specific Embodiment of Body Temperature Verification>

When the body temperature verification module 14 exceeds the previously set body temperature reference value based on the RGB values (0 to 255) when the Bayer pattern is distributed as a video stream, It may be determined that the IR signal of the band is strong and is a living body.

Thus, by determining whether the body temperature verification is simply based on whether the intensity of the IR signal exceeds the reference value, for example, as shown in Fig. 4, the body temperature can be discriminated clearly and quickly.

<Method>

The present invention may be embodied as a method invention, which corresponds to the invention of the apparatus. The face recognition method of the present invention for confirming the living body is a face recognition method for confirming the living body based on the three dimensional depth information and the infrared information. The face recognition method includes an extraction step (11) , a stereo verification step (15) And a face matching step (13, 13a) .

The extraction step 11 is a step of extracting an RGB image (visible light) 11a and an IR image (infrared light) 11b (visible light) through a single lens from a dual aperture single- ), Extracts the depth map 11c, and outputs the depth map 11c.

The stereoscopic verification step (15) is a step of generating a stereoscopic reference pass signal when the value calculated from the depth information from the depth map (11c) is equal to or larger than the stereoscopic reference value.

The face matching step 13 and 13a compare the RGB image 11a with the RGB template 12a of the stored face template 12 after receiving the stereoscopic reference pass signal and if they match within the error limits And generating a face recognition pass signal.

If the intensity of the wavelength corresponding to the biological body temperature in the wavelength of the IR image 11b is equal to or higher than the body temperature reference value in place of the stereoscopic verification step 15 or with the stereoscopic verification step 15, And a body temperature verification step (14) for generating a body temperature .

Although the present invention has been described with reference to specific embodiments, the present invention is not limited thereto. Modifications, modifications and variations made by those skilled in the art within the scope of the claims are to be construed as being within the scope of the present invention.

For example, the IR image 11b may be utilized when acquiring facial image data or performing face matching (13, 13a) in a dark environment with low illumination, which is indicated by the dotted arrows in Figs.

INDUSTRIAL APPLICABILITY The present invention can be used in the industry of a face recognition apparatus and method for confirming whether or not a living body is based on three-dimensional depth information and infrared information.

10: Dual Aperture Single Lens 4-Color Sensor Camera
11: Extraction device including a depth information processing chip (SoC)
11a: RGB image
11b: IR image
11c: Depth Map
12: Face template
13: Face matching module
14: Body temperature verification module
15: stereoscopic verification module
16: Reference strength
17: Pass signal

Claims (6)

A face recognition apparatus for confirming whether a living body is based on three-dimensional depth information and infrared information,
Dual Aperture Single Lens 4-color sensor Depth information for extracting and outputting a depth map by receiving simultaneous data of RGB image (visible light) and IR image (infrared) from a camera through a single lens An extraction device including a processing chip ,
A three-dimensional verification module for generating a three- dimensional reference passage signal when the value calculated from the depth information from the depth map is not less than a stereoscopic reference value,
A face matching module for receiving the stereoscopic reference pass signal and for comparing the RGB image with an RGB template of a previously stored face template and generating a face recognition pass signal when the RGB images match within an error limit;
Instead of or in combination with the stereoscopic verification module,
A body temperature verification module for generating a body temperature reference passage signal when the intensity of a wavelength corresponding to a living body temperature in the wavelength of the IR image is equal to or higher than a body temperature reference value,
And a face recognition unit for recognizing whether or not a living body is present. The method according to claim 1,
The face matching module is configured to further compare the IR image with a pre-stored IR template.
And a face recognition unit for recognizing the presence or absence of a living body. The method according to claim 1 or 2,
The face matching module is configured to further compare depth information from the depth map with a previously stored depth information template
And a face recognition unit for recognizing the presence or absence of a living body. The method according to claim 1 or 2,
The filter used in the double diaphragm of the lens barrel of the double-aperture single-lens four-color sensor camera is a filter having a relatively shallow depth of field F (R, G, B) And has a spectroscopic characteristic which is realized by a filter having a deep depth of second F value for an infrared ray (IR) having a wavelength of 8 占 퐉 or more and less than 14 占 퐉,
Wherein the color sensor for visible light is fabricated to receive a signal from a filter having a depth of the first F value in accordance with the spectral characteristic, The sensor is designed to receive a signal from the filter of depth of the second F value
And a face recognition unit for recognizing the presence or absence of a living body. The method according to claim 1,
When the body temperature verification value exceeds the predetermined body temperature reference value based on the RGB value (0 to 255) when the Bayer pattern is distributed as a video stream, the body temperature verification module sets the IR It is determined that the signal is strong and is a living body
And a face recognition unit for recognizing the presence or absence of a living body. A method of recognizing a living body based on three-dimensional depth information and infrared information,
Dual Aperture Single Lens 4-color sensor Depth information for extracting and outputting a depth map by receiving simultaneous data of RGB image (visible light) and IR image (infrared) from a camera through a single lens An extraction step including a processing step ,
A stereoscopic verification step of generating a stereoscopic reference pass-through signal when the value calculated from the depth information from the depth map is equal to or more than a stereoscopic reference value,
A face matching step of receiving the stereoscopic reference pass signal and comparing the RGB image with an RGB template of a stored face template and generating a face recognition pass signal when the RGB images match within an error limit;
In place of or in combination with the stereoscopic verification step,
A body temperature verification step of generating a body temperature reference passage signal when the intensity of a wavelength corresponding to a living body temperature in the wavelength of the IR image is equal to or higher than a body temperature reference value
And a face recognizing unit for recognizing whether or not a living body is recognized.

Images