CN110163078B - Living body detection method, living body detection device and service system applying living body detection method - Google Patents

Abstract

The embodiment of the invention discloses a living body detection method, which comprises the following steps: acquiring an image of an object to be detected, which is shot under a binocular camera, wherein the image comprises an infrared image and a visible light image; extracting image physical information from a biological feature area in the image to obtain image physical information of the biological feature area in the image, wherein the biological feature area is used for indicating the position of the biological feature of the object to be detected in the image; if the image physical information of the biological feature area in the image indicates that the object to be detected is a living body, carrying out deep semantic feature extraction on the biological feature area in the image based on a machine learning model to obtain deep semantic features of the biological feature area in the image; and carrying out living body judgment of the object to be detected according to the deep semantic features of the biological feature area in the image. The invention solves the problem of poor defensive performance of living body detection on the attack of the prosthesis in the prior art.

Description

Liveness detection method, device and service system using liveness detection method

Technical Field

The present invention relates to the field of computers, and in particular to a living body detection method, a device, and a service system using the living body detection method.

Background technique

With the development of biometric recognition technology, biometric recognition has been widely used, for example, face payment, face recognition in video surveillance, fingerprint recognition and iris recognition in access control authorization, etc. Biometric recognition also faces various threats, such as attackers using forged faces, fingerprints, irises, etc. for biometric recognition.

To this end, existing liveness detection solutions are mainly based on human-computer interaction solutions, which require the object to be detected to cooperate and make corresponding actions, such as nodding, blinking, smiling, etc., in order to analyze the actions of the object to be detected for liveness detection.

The inventors have discovered that the above solution not only has high requirements on the detection object, which easily leads to a poor user experience, but also has the problem of poor defense against prosthesis attacks.

Summary of the invention

The embodiments of the present invention provide a liveness detection method, an apparatus, an access control system, a payment system, a service system, an electronic device and a storage medium using the liveness detection method, which can solve the problem that liveness detection has poor defense against prosthetic attacks.

Wherein, the technical solution adopted by the present invention is:

According to one aspect of an embodiment of the present invention, a liveness detection method includes: obtaining an image of an object to be detected captured by a binocular camera, the image including an infrared image and a visible light image; extracting image physical information of a biometric region in the image to obtain image physical information of the biometric region in the image, the biometric region being used to indicate a position of a biometric of the object to be detected in the image; if the image physical information of the biometric region in the image indicates that the object to be detected is alive, extracting deep semantic features of the biometric region in the image based on a machine learning model to obtain deep semantic features of the biometric region in the image; and determining the liveness of the object to be detected based on the deep semantic features of the biometric region in the image.

In an exemplary embodiment, regional position matching is performed between the biometric region in the infrared image and the biometric region in the visible light image, including: performing regional position detection on the biometric region in the infrared image and the biometric region in the visible light image, respectively, to obtain a first regional position corresponding to the biometric region in the infrared image, and a second regional position corresponding to the biometric region in the visible light image; calculating a correlation coefficient between the first regional position and the second regional position; and determining that the regional positions match if the correlation coefficient exceeds a set correlation threshold.

In an exemplary embodiment, the area position matching is performed between the biometric area in the infrared image and the biometric area in the visible light image, including: determining a first horizontal distance between the object to be detected and the vertical plane where the binocular camera is located; based on the infrared camera and the visible light camera in the binocular camera, obtaining a second horizontal distance between the infrared camera and the visible light camera; according to the first horizontal distance and the second horizontal distance, obtaining a horizontal distance difference between the biometric area in the infrared image and the biometric area in the visible light image; if the horizontal distance difference exceeds a set distance threshold, determining that the area position does not match.

In an exemplary embodiment, the biometric area is a face area; after acquiring the image of the object to be detected captured by a binocular camera, the method further includes: performing face detection on the infrared image and the visible light image respectively; if it is detected that the infrared image does not contain a face area, and/or that the visible light image does not contain a face area, it is determined that the object to be detected is a prosthesis.

According to one aspect of an embodiment of the present invention, a liveness detection device includes: an image acquisition module, which is used to acquire an image of an object to be detected captured by a binocular camera, wherein the image includes an infrared image and a visible light image; an image physical information extraction module, which is used to extract image physical information of a biometric region in the image to obtain image physical information of the biometric region in the image, wherein the biometric region is used to indicate a position of a biometric feature of the object to be detected in the image; a deep semantic feature extraction module, which is used to extract deep semantic features of the biometric region in the image based on a machine learning model if the image physical information of the biometric region in the image indicates that the object to be detected is alive, to obtain deep semantic features of the biometric region in the image; and an object liveness determination module, which is used to determine the liveness of the object to be detected based on the deep semantic features of the biometric region in the image.

According to one aspect of an embodiment of the present invention, a door access control system using a liveness detection method includes a reception device, an identification electronic device and an access control device, wherein the reception device is used to collect images of objects entering and exiting using a binocular camera, and the images include infrared images and visible light images; the identification electronic device includes a liveness detection device, which is used to extract image physical information and deep semantic features from the biometric feature areas in the images of the objects entering and exiting, and judge whether the objects entering and exiting are alive based on the extracted image physical information and deep semantic features; when the objects entering and exiting are alive, the identification electronic device performs identity recognition on the objects entering and exiting, so that the access control device configures access rights for the objects entering and exiting that have successfully completed identity recognition, so that the objects entering and exiting control the access gate of the designated area according to the configured access rights to perform a release action.

According to one aspect of an embodiment of the present invention, a payment system using a liveness detection method includes a payment terminal and a payment electronic device, wherein the payment terminal is used to collect an image of a payment user using a binocular camera, the image including an infrared image and a visible light image; the payment terminal includes a liveness detection device, which is used to extract image physical information and deep semantic features from a biometric feature area in the image of the payment user, respectively, and judge whether the payment user is alive based on the extracted image physical information and deep semantic features; when the payment user is alive, the payment terminal authenticates the payment user, so as to initiate a payment request to the payment electronic device when the payment user passes the identity authentication.

According to one aspect of an embodiment of the present invention, a service system applying a liveness detection method includes a service terminal and an authentication electronic device, wherein the service terminal is used to collect images of service personnel using a binocular camera, and the images include infrared images and visible light images; the service terminal includes a liveness detection device, which is used to extract image physical information and deep semantic features from biometric feature areas in the images of the service personnel, respectively, and judge whether the service personnel is alive based on the extracted image physical information and deep semantic features; when the service personnel is alive, the service terminal requests the authentication electronic device to authenticate the service personnel, and distributes service business instructions to the service personnel who pass the identity authentication.

According to one aspect of an embodiment of the present invention, an electronic device includes a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the living body detection method as described above is implemented.

According to one aspect of an embodiment of the present invention, a storage medium stores a computer program, and when the computer program is executed by a processor, the living body detection method as described above is implemented.

In the above technical solution, based on the infrared images taken by the binocular camera and combined with the physical information and deep semantic features of the image, liveness detection is performed on the object to be detected, which can effectively filter out various types of prosthetic attack behaviors of the attacker and does not rely on the cooperation of the object to be detected.

Specifically, an image of the object to be detected captured by a binocular camera is obtained, and image physical information of the biometric feature area in the image is extracted. When the image physical information of the biometric feature area in the image is only that the object to be detected is alive, deep semantic features are extracted from the biometric feature area in the image based on a machine learning model, so as to determine the liveness of the object to be detected according to the extracted deep semantic features. Thus, the attacker's prosthetic attack behavior on video playback is filtered out based on the infrared image captured by the binocular camera, the prosthetic attack behavior on black and white photos is filtered out based on the image physical feature information, and the prosthetic attack behavior on color photos and hole masks is filtered out based on the deep semantic feature information. At the same time, the object to be detected is allowed to perform liveness detection in a non-cooperative free state, thereby solving the problem of poor defense of liveness detection against prosthetic attacks in the prior art.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

FIG. 1 is a schematic diagram of an implementation environment of an application scenario in which a liveness detection method is applied.

Fig. 2 is a hardware structure block diagram of an electronic device according to an exemplary embodiment.

Fig. 3 is a flow chart showing a living body detection method according to an exemplary embodiment.

FIG. 4 is a flow chart of step 330 in the embodiment corresponding to FIG. 3 in one embodiment.

FIG. 5 is a flow chart of step 333 in the embodiment corresponding to FIG. 4 in one embodiment.

FIG. 6 is a schematic diagram showing the difference between a color histogram of a color image and a color histogram of a black-and-white image involved in the embodiment corresponding to FIG. 5 .

FIG. 7 is a flow chart of step 333 in the embodiment corresponding to FIG. 4 in another embodiment.

FIG8 is a flow chart of step 3332 in the embodiment corresponding to FIG7 in one embodiment.

FIG. 9 is a schematic diagram of an HSV model involved in the embodiment corresponding to FIG. 8 .

FIG. 10 is a flowchart of step 3332 in the embodiment corresponding to FIG. 7 in another embodiment.

FIG. 11 is a flow chart of step 350 in the embodiment corresponding to FIG. 3 in one embodiment.

FIG. 12 is a flow chart of step 370 in the embodiment corresponding to FIG. 3 in one embodiment.

FIG. 13 is a flow chart showing step 320 in one embodiment according to an exemplary embodiment.

FIG. 14 is a flow chart showing step 320 in another embodiment according to an exemplary embodiment.

FIG. 15 is a schematic diagram of the horizontal distance difference involved in the embodiment corresponding to FIG. 14 .

Fig. 16 is a schematic diagram showing key points of a human face according to an exemplary embodiment.

FIG. 17 is a schematic diagram of a specific implementation of a liveness detection method in an application scenario.

Fig. 18 is a block diagram showing a living body detection device according to an exemplary embodiment.

Fig. 19 is a block diagram of an electronic device according to an exemplary embodiment.

The above-mentioned drawings have shown clear embodiments of the present invention, which will be described in more detail hereinafter. These drawings and textual descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments.

Detailed ways

Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are only examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

FIG. 1 is a schematic diagram of an implementation environment of an application scenario in which a liveness detection method is applied.

As shown in FIG. 1( a ), the implementation environment includes a payment user 510 , a smart phone 530 , and a payment server 550 .

If the paying user 510 is determined to be alive through the liveness detection method, the paying user 510 can use the smart phone 530 to perform identity authentication, and after passing the identity authentication, request the payment server 550 to complete the payment process of the order to be paid.

As shown in FIG. 1( b ), the implementation environment includes a reception device 610 , an identification server 630 , and an access control device 650 .

If the object 670 is determined to be alive through the liveness detection method, the object 670 can be identified through the recognition server 630 after the reception device 610 captures the image. After the identification of the object 670 is completed, the access control barrier of the relevant area can be released by relying on the access control authority configured by the access control device 650.

As shown in FIG. 1( c ), the implementation environment includes a service personnel 710 , a service terminal 730 , and an authentication server 750 .

If the service personnel 710 is determined to be alive through the liveness detection method, the service terminal 730 can collect images of the service personnel 710, and the authentication server 750 can perform identity authentication based on the image. After passing the identity authentication, the service terminal 730 distributes service business instructions to perform related services.

In the above three application scenarios, only the objects to be detected, such as the paying user 510, the entry and exit objects 670, and the service personnel 710, can continue the subsequent identity authentication or identity recognition through liveness detection, thereby effectively reducing the work pressure and traffic pressure of identity authentication or identity recognition, so as to better complete various identity authentication or identity recognition tasks.

Please refer to Fig. 2, which is a hardware structure block diagram of an electronic device according to an exemplary embodiment. The electronic device is suitable for the smart phone 530, the identification server 630, and the authentication server 750 in the implementation environment shown in Fig. 1.

It should be noted that this electronic device is only an example adapted to the present invention and cannot be considered to provide any limitation on the scope of use of the present invention. This electronic device cannot be interpreted as needing to rely on or necessarily having one or more components in the exemplary electronic device 200 shown in FIG. 2.

The hardware structure of the electronic device 200 may vary greatly due to different configurations or performances. As shown in FIG. 2 , the electronic device 200 includes: a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .

Specifically, the power supply 210 is used to provide operating voltage for each hardware device on the electronic device 200 .

The interface 230 includes at least one wired or wireless network interface for interacting with external devices.

Of course, in other examples adapted by the present invention, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, as shown in FIG. 2, which is not specifically limited here.

The memory 250 is a carrier for storing resources, which may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 251, an application 253 and data 255, etc. The storage method may be temporary storage or permanent storage.

Among them, the operating system 251 is used to manage and control various hardware devices and application programs 253 on the electronic device 200 to enable the central processor 270 to calculate and process the massive data 255 in the memory 250. It can be WindowsServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

The application 253 is a computer program that performs at least one specific task based on the operating system 251, and may include at least one module (not shown in FIG. 2 ), each of which may include a series of computer-readable instructions for the electronic device 200. For example, a liveness detection device may be regarded as an application 253 deployed on the electronic device.

The data 255 may be photos, pictures, etc. stored in a disk, stored in the memory 250 .

The central processor 270 may include one or more processors and is configured to communicate with the memory 250 through at least one communication bus to read the computer-readable instructions stored in the memory 250, thereby realizing the operation and processing of the mass data 255 in the memory 250. For example, the liveness detection method is completed by the central processor 270 reading a series of computer-readable instructions stored in the memory 250.

It is understood that the structure shown in Fig. 2 is only for illustration, and the electronic device 100 may also include more or fewer components than those shown in Fig. 2, or have components different from those shown in Fig. 1. Each component shown in Fig. 2 may be implemented by hardware, software, or a combination thereof.

Please refer to FIG. 3 . In an exemplary embodiment, a living body detection method is applicable to an electronic device. The hardware structure of the electronic device may be as shown in FIG. 2 .

The living body detection method can be performed by an electronic device and may include the following steps:

Step 310: Acquire an image of the object to be detected captured by a binocular camera, wherein the image includes an infrared image and a visible light image.

First of all, the object to be detected can be a paying user of an order to be paid, a bank card user to deposit/withdraw money, an entry and exit object to pass through a access control, or a service personnel to receive service business. This embodiment does not specifically limit the object to be detected.

Correspondingly, different objects to be detected can correspond to different application scenarios. For example, a paying user who is about to pay an order corresponds to the order payment application scenario, a bank card user who is about to deposit/withdraw money corresponds to the bank card application scenario, an entry and exit object who is about to pass through the access control corresponds to the access control application scenario, and a service personnel who is about to receive service business corresponds to the passenger service application scenario.

It can be understood that in any of the above application scenarios, there may be fake attack behaviors of attackers. For example, criminals may replace service personnel and use fake attack behaviors to pass identity authentication in order to carry passengers. Therefore, the liveness detection method provided in this embodiment can be applied to different application scenarios depending on the different objects to be detected.

Secondly, for the acquisition of the image of the object to be detected, it can be the image of the object to be detected captured and collected by the binocular camera in real time, or it can be the image of the object to be detected pre-stored in the electronic device, that is, by reading the image captured and collected by the binocular camera in a historical time period in the cache area of the electronic device. This embodiment does not limit this.

It is to be supplemented here that the image can be a video or a number of photos, and therefore, the subsequent liveness detection is performed in units of image frames.

Finally, the binocular camera includes an infrared camera for generating infrared images, and a visible light camera for generating visible light images.

The binocular camera can be installed on a camera, a video recorder, or other electronic devices with image acquisition function, such as a smart phone.

It is understandable that prosthetic attacks based on video playback often require attackers to use the screen configured on the electronic device to play back the video. At this time, if the infrared camera projects infrared light onto the screen, reflections will occur, making it impossible for the generated infrared image to contain the biometric feature area.

This is conducive to achieving liveness determination of the object to be detected based on the infrared image captured by the binocular camera, that is, filtering out fake attack behaviors related to video playback.

Step 330 , extracting image physical information of the biometric feature region in the image to obtain image physical information of the biometric feature region in the image.

First, the biometric features of the object to be detected, such as face, eyes, mouth, hands, feet, fingerprints, irises, etc. Correspondingly, the position of the biometric features of the object to be detected in the image constitutes the biometric feature area in the image of the object to be detected, which can also be understood as the biometric feature area is used to indicate the position of the biometric features of the object to be detected in the image.

Secondly, the physical information of the image reflects the texture and color of the image, including but not limited to: the texture information and color information of the image.

It should be noted that in terms of image texture, there are significant differences between the texture details presented in the image by the biological features of a living body and the texture details presented in the image by the biological features of a prosthesis, while in terms of image color, the visible light images captured and collected by a living body are usually color images.

Therefore, after the physical information of the biological feature area in the image is extracted, the liveness detection can be performed on the object to be detected according to the above-mentioned difference between the living body and the prosthesis.

If the image physical information of the biometric feature area in the image indicates that the object to be detected is a living body, the process jumps to step 350 to continue to perform liveness detection on the object to be detected.

On the contrary, if the image physical information of the biological feature area in the image indicates that the object to be detected is a prosthesis, the liveness detection of the object to be detected is stopped, so as to improve the efficiency of liveness detection.

Step 350: Based on the machine learning model, deep semantic features are extracted for the biometric feature area in the image to obtain deep semantic features of the biometric feature area in the image.

Since the texture and color of the image reflected by the physical information of the image are not only prone to significant changes due to changes in the shooting angle, which in turn leads to errors in liveness determination, but also the ability to defend against prosthetic attack behaviors is limited and is only applicable to relatively simple photo attacks. For this reason, in this embodiment, based on the machine learning model, the deep semantic features of the biometric feature area in the image are extracted to improve the defense capability of liveness detection against prosthetic attack behaviors, while achieving the adaptation of the shooting angle.

The machine learning model is trained based on a large number of positive and negative samples to achieve liveness determination of the object to be detected.

In order to allow the subject to be detected to be detected in a non-cooperative free state (such as nodding, turning, shaking the head) for liveness detection, the positive samples and negative samples are images obtained by taking pictures of the living body and the prosthesis at different angles using a binocular camera.

Through model training, positive samples and negative samples are used as training inputs, and the living body corresponding to the positive samples and the prosthesis corresponding to the negative samples are used as training true values, so as to construct a machine learning model for liveness determination of the object to be detected.

Specifically, model training iteratively optimizes the parameters of a specified mathematical model through positive and negative samples so that the specified algorithm function constructed by these parameters meets the convergence conditions.

The specified mathematical model includes but is not limited to: logistic regression, support vector machine, random forest, neural network, etc.

Specify algorithm functions, including but not limited to: maximum expectation function, loss function, etc.

For example, the parameters of the specified mathematical model are randomly initialized, and the loss value of the loss function constructed by the randomly initialized parameters is calculated based on the current sample.

If the loss value of the loss function does not reach the minimum, the parameters of the specified mathematical model are updated, and the loss value of the loss function constructed by the updated parameters is calculated based on the next sample.

The iteration cycle is repeated until the loss value of the loss function reaches the minimum, which means that the loss function converges. At this time, the specified mathematical model also converges and meets the preset accuracy requirements, and the iteration is stopped.

Otherwise, the parameters of the specified mathematical model are iteratively updated, and the loss value of the loss function constructed by the updated parameters is iteratively calculated based on the remaining samples until the loss function converges.

It is worth mentioning that if the number of iterations reaches the iteration threshold before the loss function converges, the iteration will be stopped to ensure the efficiency of model training.

When the specified mathematical model converges and meets the preset accuracy requirements, it means that the model training has been completed. The machine learning model that has completed the model training will have the ability to extract deep semantic features of the biological feature areas in the image.

Then, based on the machine learning model, deep semantic features can be extracted from the biological feature areas in the image, and used to determine the liveness of the object to be detected.

Optionally, the machine learning model includes but is not limited to: a convolutional neural network model, a deep neural network model, a residual neural network model, etc.

Among them, deep semantic features include color features and texture features of the image. Compared with the color information and texture information in the physical information of the image, they can better improve the ability to distinguish between living objects and prostheses, so as to improve the ability of living object detection to defend against prosthesis attack behaviors.

Step 370: Perform a liveness determination on the object to be detected based on the deep semantic features of the biological feature area in the image.

Through the process described above, it is possible to effectively resist various types of prosthetic attack behaviors such as video playback, black and white photos, color photos, hole masks, etc., while allowing the object to be detected to perform liveness detection in a non-cooperative free state, thereby improving the accuracy of liveness detection while improving the user experience, and fully ensuring the security of liveness detection.

Referring to FIG. 4 , in an exemplary embodiment, step 330 may include the following steps:

Step 331, acquiring a visible light image including the biometric feature area according to the acquired image.

It should be understood that the infrared image captured and collected by the living body based on the infrared camera in the binocular camera is actually a grayscale image, while the visible light image captured and collected by the living body based on the visible light camera in the binocular camera is a color image.

Based on this, in order to filter out fake attack behaviors such as black and white photos, the basis for determining the liveness of the object to be detected is the visible light image.

Therefore, after obtaining the infrared image and visible light image of the object to be detected taken by the binocular camera, it is necessary to first obtain the visible light image containing the biological feature area, so that the living body of the object to be detected can be determined based on whether the visible light image is a color image.

Step 333: extracting image physical information of the biometric feature area in the visible light image from the biometric feature area in the visible light image.

If the physical information of the image indicates that the visible light image is a color image, it can be determined that the object to be detected is a living body. Conversely, if the physical information of the image indicates that the visible light image is a black and white image, it can be determined that the object to be detected is a prosthesis.

Optionally, the image physical information includes, but is not limited to: color information defined by a color histogram, and texture information defined by a LBP (Local Binary Patterns)/LPQ (Local Phase Quantization) histogram.

The process of extracting physical information from images is explained below.

Please refer to FIG. 5 , in an exemplary embodiment, the image physical information is color information.

Accordingly, step 333 may include the following steps:

Step 3331, based on the biometric feature area in the visible light image, calculate the color histogram of the biometric feature area in the visible light image.

Step 3333: Use the calculated color histogram as color information of the biometric feature area in the visible light image.

As shown in Figure 6, there is a relatively obvious distribution difference between the color histogram (a) of the color image and the color histogram (b) of the black and white image. Therefore, based on the color histogram of the biometric feature area in the visible light image, the liveness determination of the object to be detected can be performed to filter out the fake attack behavior on the black and white photo.

With the help of the above-mentioned embodiments, the extraction of color information is realized, thereby making it possible to perform living body determination of the object to be detected based on the color information.

Please refer to FIG. 7 , in another exemplary embodiment, the image physical information is texture information.

Accordingly, step 333 may include the following steps:

Step 3332: Create a color space corresponding to the biometric region in the visible light image based on the biometric region in the visible light image.

The color space corresponding to the biometric feature region in the visible light image is essentially a color set that describes the biometric feature region in the visible light image in a mathematical way.

Optionally, the color space can be constructed based on HSV parameters or YCbCr parameters. The HSV parameters include hue (H), saturation (S) and brightness (V), and the YCbCr parameters include color brightness and concentration (Y), blue concentration offset (Cb) and red concentration offset (Cr).

Step 3334: for the color space, extract local binary pattern features in the spatial domain, and/or extract local phase quantization features in the frequency domain.

Among them, the Local Binary Patterns (LBP) feature is an accurate description of the texture details of the biological feature area in the visible light image based on the image pixels themselves, thereby reflecting the grayscale changes of the visible light image.

The local phase quantization (LPQ) feature is based on the transformation coefficient of the image in the transform domain, which accurately describes the texture details of the biological feature area in the visible light image, thereby reflecting the gradient distribution of the visible light image.

In other words, whether it is the local binary pattern feature or the local phase quantization feature, the essence is to analyze the texture details of the biological feature area in the visible light image, so as to define the texture information of the biological feature area in the visible light image.

Step 3336: Generate an LBP/LPQ histogram based on the local binary pattern feature and/or the local phase quantization feature as texture information of the biological feature area in the visible light image.

Therefore, the LBP/LPQ histogram is generated based on the local binary pattern features and/or the local phase quantization features. Through the mutual combination and mutual complementation of LBP/LPQ, the texture details of the biological feature area in the visible light image can be more accurately described, thereby fully ensuring the accuracy of liveness detection.

With the effects of the above-mentioned embodiments, the extraction of texture information is realized, thereby making it possible to perform a liveness determination of the object to be detected based on the texture information.

Furthermore, the process of creating the color space is described as follows.

Please refer to FIG. 8 , in an exemplary embodiment, step 3332 may include the following steps:

Step 3332a, based on the biometric feature area in the visible light image, obtain HSV parameters corresponding to the biometric feature area in the visible light image, and the HSV parameters include hue (H), saturation (S) and brightness (V).

Step 3332c, constructing an HSV model according to the acquired HSV parameters as the color space corresponding to the biological feature area in the visible light image.

As shown in FIG9 , the HSV model is essentially a hexagonal pyramid. Accordingly, the construction process of the HSV model includes: constructing the boundary of the hexagonal pyramid through hue (H), constructing the horizontal axis of the hexagonal pyramid through saturation (S), and constructing the vertical axis of the hexagonal pyramid through brightness (V).

Thus, the construction of the color space based on HSV parameters is completed.

Please refer to FIG. 10 , in another exemplary embodiment, step 3332 may include the following steps:

Step 3332b, based on the biometric area in the visible light image, obtain YCbCr parameters corresponding to the biometric area in the visible light image, wherein the YCbCr parameters include color brightness and concentration (Y), blue concentration offset (Cb) and red concentration offset (Cr).

Step 3332d: construct a color space corresponding to the biometric feature area in the visible light image based on the acquired YCbCr parameters.

Specifically, the acquired YCbCr parameters are converted into RGB parameters, and then the RGB color channels represented by the RGB parameters are used to construct an RGB picture, that is, the color space of the biometric region in the visible light image. The RGB color channels include a red channel R representing red, a green channel G representing green, and a blue channel B representing blue.

Thus, the construction of the color space based on the YCbCr parameters is completed.

Through the cooperation of the above embodiments, the creation of a color space is achieved, thereby enabling the extraction of texture information based on the color space to be realized.

In an exemplary embodiment, after step 330, the method described above may further include the following steps:

The liveness determination of the object to be detected is performed based on the image physical information of the biological feature area in the visible light image.

Specifically, the image physical information of the biometric feature area in the visible light image is input into a support vector machine classifier, and the color category of the visible light image is predicted to obtain the color category of the visible light image.

First, the support vector machine classifier is generated by training the color categories of visible light images based on a large number of learning samples, wherein the learning samples include visible light images belonging to black and white images and visible light images belonging to color images.

Secondly, the color category includes the color image category and the black and white image category.

Then, if the predicted color category of the visible light image is a black and white image category, that is, the color category of the visible light image indicates that the visible light image is a black and white image, then it is determined that the object to be detected is a prosthesis.

On the contrary, if the predicted color category of the visible light image is a color image category, that is, the color category of the visible light image indicates that the visible light image is a color image, then it is determined that the object to be detected is a living body.

With the help of the above-mentioned embodiments, the liveness determination of the object to be detected based on the visible light image is realized, that is, the fake attack behavior on the black and white photos is filtered out.

In an exemplary embodiment, the machine learning model is a deep neural network model, and the deep neural network model includes an input layer, a convolution layer, a connection layer and an output layer. The convolution layer is used for feature extraction, and the connection layer is used for feature fusion.

Optionally, the deep neural network model may further include an activation layer and a pooling layer, wherein the activation layer is used to increase the convergence speed of the deep neural network model, and the pooling layer is used to reduce the complexity of feature connections.

Optionally, the convolutional layer is configured with multiple channels, each channel being capable of being used for inputting images with different channel information in the same image, thereby improving the accuracy of feature extraction.

For example, the image is a color image. Assuming that the convolution layer is configured with three channels A1, A2, and A3, then the color image can be input into the three channels configured in the convolution layer in the GRB color channel manner, that is, the color image part corresponding to the red channel R is input into the A1 channel, the color image part corresponding to the green channel G is input into the A2 channel, and the color image part corresponding to the blue channel B is input into the A3 channel.

As shown in FIG. 11 , step 350 may include the following steps:

Step 351, input the image into the convolutional layer from the input layer in the deep neural network model.

Step 353, using the convolutional layer to perform feature extraction, obtain shallow semantic features of the biometric feature area in the image, and input them into the connection layer.

Step 355, using the connection layer to perform feature fusion, obtain the deep semantic features of the biological feature area in the image, and input them to the output layer.

Among them, shallow semantic features include image shape features and spatial relationship features, while deep semantic features include image color features and texture features.

In other words, shallow semantic features are obtained through feature extraction in the convolutional layer, and deep semantic features are obtained through feature fusion in the connection layer. This means that features of different resolutions and scales in the deep neural network model can be correlated with each other rather than isolated, which can effectively improve the accuracy of liveness detection.

Referring to FIG. 12 , in an exemplary embodiment, step 370 may include the following steps:

Step 371, using the activation function classifier in the output layer to perform classification prediction on the image.

Step 373: determine whether the object to be detected is a living body according to the category predicted by the image.

The activation function classifier, that is, the softmax classifier, is used to calculate the probability that an image belongs to different categories.

In this embodiment, the categories of images include: living body category and prosthesis category.

For example, for an image, based on the activation function classifier in the output layer, it is calculated that the probability that the image belongs to the living body category is P1, and the probability that the image belongs to the prosthesis category is P2.

If P1>P2, it means that the image belongs to the living body category, and the object to be detected is determined to be a living body.

On the contrary, if P1<P2, it means that the image belongs to the prosthesis category, and the object to be detected is determined to be a prosthesis.

With the help of the above-mentioned embodiments, liveness determination of the object to be detected based on deep semantic features is realized, that is, prosthetic attack behaviors regarding color photos and hole masks are filtered out, and it does not rely on the cooperation of the object to be detected with liveness detection.

In an exemplary embodiment, after step 310, the method described above may further include the following steps:

Step 320 , performing region position matching between the biometric region in the infrared image and the biometric region in the visible light image.

It can be understood that for the infrared camera and the visible light camera in the binocular camera, if the infrared camera and the visible light camera are shooting the free state (such as nodding) of the same object to be detected at the same time, then the regional position of the biometric area in the infrared image captured thereby and the regional position of the biometric area in the visible light image have a large correlation.

Therefore, in this embodiment, whether there is a large correlation between the two is determined by regional position matching, and then whether the object to be detected is a living body is determined.

If the area positions do not match, that is, the area position of the biometric area in the infrared image has little correlation with the area position of the biometric area in the visible light image, which means that the infrared camera and the visible light camera do not capture the same individual, then the object to be detected is determined to be a prosthesis.

On the contrary, if the area positions match, that is, the area position of the biometric area in the infrared image is highly correlated with the area position of the biometric area in the visible light image, that is, the infrared camera and the visible light camera take pictures of the same individual, then the object to be detected is determined to be a living body.

It is worth mentioning that step 320 can be set before any one of step 330 and step 350, and this embodiment does not limit this.

The matching process of regional locations is described below.

Referring to FIG. 13 , in an exemplary embodiment, step 320 may include the following steps:

Step 321, performing area position detection on the biometric area in the infrared image and the biometric area in the visible light image, respectively, to obtain a first area position corresponding to the biometric area in the infrared image and a second area position corresponding to the biometric area in the visible light image.

Among them, area position detection can be achieved based on the projective geometry method of computer vision.

Step 323: Calculate the correlation coefficient between the first area position and the second area position.

If the correlation coefficient exceeds the set correlation threshold, the region positions are determined to be matched, and then the object to be detected is determined to be a living body, and the subsequent steps of living body detection can be continued.

On the contrary, if the correlation coefficient is less than the set correlation threshold, it is determined that the area positions do not match, and then it is determined that the object to be detected is a prosthesis. At this time, the subsequent steps of liveness detection are stopped, thereby improving the efficiency of liveness detection.

Please refer to FIG. 14 , in another exemplary embodiment, step 320 may include the following steps:

Step 322: determine a first horizontal distance between the object to be detected and the vertical plane where the binocular camera is located.

Step 324: Based on the infrared camera and the visible light camera in the binocular camera, a second horizontal distance between the infrared camera and the visible light camera is obtained.

Step 326: Obtain a horizontal distance difference between the biometric feature region in the infrared image and the biometric feature region in the visible light image according to the first horizontal distance and the second horizontal distance.

As shown in Figure 15, A represents the object to be detected, B1 represents the infrared camera in the binocular camera, and B2 represents the visible light camera in the binocular camera. X1 represents the vertical plane where the object to be detected A is located, and X2 represents the vertical plane where the binocular camera is located.

Then, the formula for obtaining the horizontal distance difference D is as follows:

D＝L/Z.

Wherein, D represents the horizontal distance difference, that is, the difference between the horizontal coordinates of the infrared image and the visible light image on the horizontal plane. Z represents the first horizontal distance, and L represents the second horizontal distance.

If the horizontal distance difference D is less than the set distance threshold, it is determined that the area positions match, and then it is determined that the object to be detected is a living body, and the subsequent living body detection steps can be continued.

On the contrary, if the horizontal distance difference D exceeds the set distance threshold, it is determined that the area positions do not match, and then it is determined that the object to be detected is a prosthesis. At this time, the subsequent steps of liveness detection are stopped, thereby improving the efficiency of liveness detection.

Through the above process, the liveness determination of the object to be detected based on the regional position is realized, and the images belonging to the prosthesis are effectively filtered, which is conducive to improving the accuracy of liveness detection.

In an exemplary embodiment, the biometric region is a face region.

Accordingly, after step 310, the method described above may further include the following steps:

Face detection is performed on the infrared image and the visible light image respectively.

As shown in FIG16 , there are 68 key points of facial features in the image, including: 6 key points 43 to 48 of the eyes in the image, 20 key points 49 to 68 of the mouth in the image, etc. The above key points are uniquely represented by different coordinates (x, y) in the image.

Based on this, in this embodiment, face detection is achieved through a face key point model.

The facial key point model essentially constructs an index relationship for the facial features in the image, so that the key points of specific facial features can be located from the image through the constructed index relationship.

Specifically, for the image of the object to be detected, that is, the infrared image or the visible light image, after it is input into the facial key point model, the key points of the facial features in the image are indexed and marked. As shown in Figure 16, the indexes marked by the six key points of the eyes in the image are 43 to 48, and the indexes marked by the twenty key points of the mouth in the image are 49 to 68.

At the same time, the coordinates of the key points marked with indexes in the image are stored accordingly, thereby constructing an index relationship between the index corresponding to the image and the coordinates for the facial features of the object to be detected.

Then, based on the index relationship, the coordinates of the key points of the facial features of the object to be detected in the image can be obtained from the index, and then the position of the facial features of the object to be detected in the image, that is, the facial feature area in the image, can be determined.

If it is detected that the infrared image does not contain a human face region, and/or the visible light image does not contain a human face region, the object to be detected is determined to be a prosthesis, thereby stopping subsequent liveness detection to improve the efficiency of liveness detection.

On the contrary, if it is detected that the infrared image contains a human face region and the visible light image contains a human face region, it is determined that the object to be detected is a living body, and the process jumps to step 330 .

Therefore, based on the above process, the liveness determination of the object to be detected based on the infrared image captured by the binocular camera is achieved, that is, the prosthetic attack behavior on the video playback is filtered out.

In addition, face detection based on the face key point model has good accuracy and stability for facial feature recognition of different facial expressions, which fully guarantees the accuracy of liveness detection.

In an exemplary embodiment, after step 370, the method described above may further include the following steps:

If the object to be detected is a living body, a face recognition model is called to perform face recognition on the image of the object to be detected.

The face recognition process is explained below in conjunction with specific application scenarios.

Fig. 1(a) is a schematic diagram of an implementation environment of an order payment application scenario. As shown in Fig. 1(a), in this application scenario, the implementation environment includes a payment user 510, a smart phone 530 and a payment server 550.

For a certain order to be paid, the paying user 510 scans his face through the binocular camera configured by the smart phone 530, so that the smart phone 530 obtains the image of the paying user 510, and then uses the face recognition model to perform face recognition on the image.

The face recognition model extracts the user features of the image to calculate the similarity between the user features and the specified user features. If the similarity is greater than the similarity threshold, the paying user 510 passes the identity authentication. The specified user features are pre-extracted for the paying user 510 by the smart phone 530 through the face recognition model.

After the paying user 510 passes the identity authentication, the smart phone 530 initiates an order payment request to the payment server 550 for the order to be paid, thereby completing the payment process of the order to be paid.

FIG1( b ) is a schematic diagram of an implementation environment of a door access control application scenario. As shown in FIG1( b ), the implementation environment includes a reception device 610 , an identification server 630 , and a door access control device 650 .

The reception device 610 is equipped with a binocular camera to take a face photo of the in-and-out object 670, and send the obtained image of the in-and-out object 670 to the recognition server 630 for face recognition. In this application scenario, the in-and-out object 670 includes staff and visitors.

The recognition server 630 extracts the person feature of the image through the face recognition model to calculate the similarity between the person feature and multiple designated person features, obtains the designated person feature with the greatest similarity, and then identifies the person identity associated with the designated person feature with the greatest similarity as the identity of the entry and exit object 670, thereby completing the identity recognition of the entry and exit object 670. The designated person feature is pre-extracted by the recognition server 630 for the entry and exit object 670 through the face recognition model.

After the identity recognition of the object 670 is completed, the identification server 630 sends a access authorization instruction to the access control device 650 for the object 670, so that the access control device 650 configures the corresponding access authority for the object 670 according to the access authorization instruction, and then the object 670 controls the access gate of the designated work area to perform the release action with the access authority.

Of course, in different application scenarios, flexible deployment can be performed according to actual application requirements. For example, the identification server 630 and the access control device 650 can be deployed as the same server, or the reception device 610 and the access control device 650 can be deployed on the same server. This application scenario does not limit this.

Fig. 1(c) is a schematic diagram of the implementation environment of the passenger service application scenario. As shown in Fig. 1(c), in this application scenario, the implementation environment includes a service personnel 710, a service terminal 730 and an authentication server 750. In this application scenario, the service personnel 710 is a passenger driver.

The service terminal 730 disposed in the vehicle is equipped with a binocular camera to take a photo of the service personnel 710 and send the obtained image of the service personnel 710 to the authentication server 750 for face recognition.

The authentication server 750 extracts the person features of the image through the face recognition model to calculate the similarity between the person features and the specified person features. If the similarity is greater than the similarity threshold, the service personnel 710 passes the identity authentication. The specified person features are pre-extracted for the service personnel 710 by the service terminal 730 through the face recognition model.

After the service personnel 710 passes the identity authentication, the service terminal 730 can distribute service business instructions to the service personnel 710, so that the service personnel - the passenger driver can reach the destination and pick up passengers according to the instructions of the service business instructions.

In the above three application scenarios, the liveness detection device can be used as a precursor module for face recognition.

As shown in FIG. 17 , by executing steps 801 to 804 , liveness determination of the object to be detected is performed multiple times based on face detection, region matching detection, image physical information detection, and deep semantic feature detection, respectively.

Therefore, the liveness detection device can accurately determine whether the object to be detected is alive, and then achieve defense against various types of prosthetic attacks. It can not only fully guarantee the security of identity authentication/identity recognition, but also effectively reduce the workload and traffic pressure of subsequent face recognition, thereby better facilitating various face recognition tasks.

The following is an embodiment of the device of the present invention, which can be used to perform the liveness detection method involved in the present invention. For details not disclosed in the embodiment of the device of the present invention, please refer to the method embodiment of the liveness detection method involved in the present invention.

Please refer to FIG. 18 . In an exemplary embodiment, a living body detection device 900 includes but is not limited to: an image acquisition module 910 , an image physical information extraction module 930 , a deep semantic feature extraction module 950 , and an object living body determination module 970 .

The image acquisition module 910 is used to acquire an image of the object to be detected captured by a binocular camera, wherein the image includes an infrared image and a visible light image.

The image physical information extraction module 930 is used to extract image physical information from the biometric feature area in the image to obtain image physical information of the biometric feature area in the image, where the biometric feature area is used to indicate the position of the biometric feature of the object to be detected in the image.

The deep semantic feature extraction module 950 is used to extract deep semantic features of the biological feature area in the image based on a machine learning model if the image physical information of the biological feature area in the image indicates that the object to be detected is a living body, so as to obtain the deep semantic features of the biological feature area in the image.

The object liveness determination module 970 is used to perform liveness determination of the object to be detected based on the deep semantic features of the biological feature area in the image.

In an exemplary embodiment, the image physical information extraction module 930 includes but is not limited to: a visible light image acquisition unit and an image physical information extraction unit.

The visible light image acquisition unit is used to acquire a visible light image containing the biometric feature area based on the acquired image.

The image physical information extraction unit is used to extract the image physical information of the biometric feature area in the visible light image from the biometric feature area in the visible light image.

In an exemplary embodiment, the physical image information is color information.

Accordingly, the image physical information extraction unit includes but is not limited to: a color histogram calculation subunit and a color information definition subunit.

Wherein, the color histogram calculation subunit is used to calculate the color histogram of the biometric feature area in the visible light image based on the biometric feature area in the visible light image.

The color information definition subunit is used to use the calculated color histogram as the color information of the biological feature area in the visible light image.

In an exemplary embodiment, the image physical information is texture information.

Accordingly, the image physical information extraction unit includes but is not limited to: a color space creation subunit, a local feature extraction subunit and a texture information definition subunit.

The color space creation subunit is used to create a color space corresponding to the biometric feature area in the visible light image based on the biometric feature area in the visible light image.

The local feature extraction subunit is used to extract local binary pattern features in the spatial domain and/or extract local phase quantization features in the frequency domain for the color space.

The texture information definition subunit is used to generate an LBP/LPQ histogram as texture information of the biological feature area in the visible light image according to the local binary pattern feature and/or the local phase quantization feature.

In an exemplary embodiment, the color space creation subunit includes but is not limited to: a first parameter acquisition subunit and a first construction subunit.

Among them, the first parameter acquisition subunit is used to acquire HSV parameters corresponding to the biometric feature area in the visible light image based on the biometric feature area in the visible light image, and the HSV parameters include hue (H), saturation (S) and brightness (V).

The first construction subunit is used to construct an HSV model according to the acquired HSV parameters as a color space corresponding to the biological feature area in the visible light image.

In an exemplary embodiment, the color space creation subunit includes but is not limited to: a second parameter acquisition subunit and a second construction subunit.

Among them, the second parameter acquisition subunit is used to obtain YCbCr parameters corresponding to the biometric feature area in the visible light image based on the biometric feature area in the visible light image, and the YCbCr parameters include color brightness and concentration (Y), blue concentration offset (Cb) and red concentration offset (Cr).

The second construction subunit is used to construct a color space corresponding to the biological feature area in the visible light image according to the acquired YCbCr parameters.

In an exemplary embodiment, the device 900 further includes but is not limited to: a second object living body determination module.

The second object liveness determination module is used to perform liveness determination on the object to be detected based on the image physical information of the biological feature area in the visible light image.

In an exemplary embodiment, the second object living body determination module includes, but is not limited to: a color category prediction unit, a first object living body determination unit, and a second object living body determination unit.

The color category prediction unit is used to input the image physical information of the biometric feature area in the visible light image into the support vector machine classifier, perform color category prediction on the visible light image, and obtain the color category of the visible light image.

The first object living body determination unit is configured to determine that the object to be detected is a prosthesis if the color category of the visible light image indicates that the visible light image is a black and white image.

The second object living body determination unit is configured to determine that the object to be detected is a living body if the color category of the visible light image indicates that the visible light image is a color image.

In an exemplary embodiment, the machine learning model is a deep neural network model, which includes an input layer, a convolutional layer, a connection layer and an output layer.

Accordingly, the deep semantic feature extraction module 950 includes but is not limited to: an image input unit, a feature extraction unit and a feature fusion unit.

Among them, the image input unit is used to input the image into the convolution layer from the input layer in the deep neural network model.

The feature extraction unit is used to perform feature extraction using the convolution layer to obtain shallow semantic features of the biological feature area in the image and input the shallow semantic features into the connection layer.

The feature fusion unit is used to perform feature fusion using the connection layer to obtain deep semantic features of the biological feature area in the image and input the deep semantic features into the output layer.

In an exemplary embodiment, the object living body determination module 970 includes but is not limited to: a classification prediction unit and a third object living body determination unit.

The classification prediction unit is used to perform classification prediction on the image using the activation function classifier in the output layer.

The third object living body determination unit is used to determine whether the object to be detected is a living body according to the category predicted by the image.

In an exemplary embodiment, the device 900 further includes but is not limited to: a region position matching module and a fourth object living body determination module.

The region position matching module is used to perform region position matching between the biometric feature region in the infrared image and the biometric feature region in the visible light image.

The fourth object living body determination module is used to determine that the object to be detected is a prosthesis if the area positions do not match.

In an exemplary embodiment, the area position matching module includes but is not limited to: an area position detection unit, a correlation coefficient calculation unit and a fifth object living body determination unit.

Among them, the area position detection unit is used to perform area position detection on the biometric area in the infrared image and the biometric area in the visible light image, respectively, to obtain a first area position corresponding to the biometric area in the infrared image and a second area position corresponding to the biometric area in the visible light image.

A correlation coefficient calculation unit is used to calculate the correlation coefficient between the first area position and the second area position.

The fifth object living body determination unit is configured to determine that the area positions match if the correlation coefficient exceeds a set correlation threshold.

In an exemplary embodiment, the area position matching module includes but is not limited to: a first horizontal distance determining unit, a second horizontal distance determining unit, a horizontal distance difference acquiring unit and a sixth object living body determining unit.

Wherein, the first horizontal distance determination unit is used to determine a first horizontal distance between the object to be detected and the vertical plane where the binocular camera is located.

The second horizontal distance determining unit is used to obtain a second horizontal distance between the infrared camera and the visible light camera based on the infrared camera and the visible light camera in the binocular camera.

A horizontal distance difference acquisition unit is used to acquire a horizontal distance difference between the biometric feature area in the infrared image and the biometric feature area in the visible light image according to the first horizontal distance and the second horizontal distance.

The sixth object living body determination unit is configured to determine that the area positions do not match if the horizontal distance difference exceeds a set distance threshold.

In an exemplary embodiment, the biometric region is a face region.

Accordingly, the device 900 further includes but is not limited to: a face detection module and a seventh object living body determination module.

Wherein, the face detection module is used to perform face detection on the infrared image and the visible light image respectively.

The seventh object living body determination module is used to determine that the object to be detected is a prosthesis if it is detected that the infrared image does not contain a human face area and/or the visible light image does not contain a human face area.

It should be noted that, when the liveness detection device provided in the above embodiment performs liveness detection processing, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the liveness detection device will be divided into different functional modules to complete all or part of the functions described above.

In addition, the embodiments of the liveness detection device and the liveness detection method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment, which will not be repeated here.

In an exemplary embodiment, a door access control system using a liveness detection method includes a reception device, an identification server, and a door access control device.

Wherein, the reception equipment is used to collect images of objects entering and leaving using a binocular camera, and the images include infrared images and visible light images.

The recognition server includes a liveness detection device for extracting image physical information and deep semantic features from the biometric feature area in the image of the entering and exiting object, and judging whether the entering and exiting object is alive based on the extracted image physical information and deep semantic features.

When the object entering and exiting is a living body, the recognition server performs identity recognition on the object entering and exiting, so that the access control device configures access control permissions for the object entering and exiting that has successfully completed identity recognition, so that the object entering and exiting controls the access control barrier of the designated area to perform a release action according to the configured access control permissions.

In an exemplary embodiment, a payment system using a liveness detection method includes a payment terminal and a payment server.

Wherein, the payment terminal is used to collect images of paying users using a binocular camera, and the images include infrared images and visible light images.

The payment terminal includes a liveness detection device for extracting image physical information and deep semantic features from the biometric feature area in the image of the payment user, and judging whether the payment user is alive based on the extracted image physical information and deep semantic features.

When the payment user is alive, the payment terminal performs identity authentication on the payment user, and initiates a payment request to the payment server when the payment user passes the identity authentication.

In an exemplary embodiment, a service system applying a liveness detection method includes a service terminal and an authentication server.

Wherein, the service terminal is used to collect images of service personnel using a binocular camera, and the images include infrared images and visible light images.

The service terminal includes a liveness detection device for extracting image physical information and deep semantic features from the biometric feature area in the image of the service personnel, and judging whether the service personnel is alive based on the extracted image physical information and deep semantic features.

When the service personnel are alive, the service terminal requests the authentication server to authenticate the identity of the service personnel and distributes service instructions to the service personnel who pass the identity authentication.

Please refer to FIG. 19 . In an exemplary embodiment, an electronic device 1000 includes at least one processor 1001 , at least one memory 1002 , and at least one communication bus 1003 .

The memory 1002 stores computer-readable instructions, and the processor 1001 reads the computer-readable instructions stored in the memory 1002 through the communication bus 1003 .

When the computer-readable instructions are executed by the processor 1001, the living body detection method in the above-mentioned embodiments is implemented.

In an exemplary embodiment, a storage medium stores a computer program, which, when executed by a processor, implements the living body detection method in the above-mentioned embodiments.

The above contents are only preferred exemplary embodiments of the present invention and are not intended to limit the implementation scheme of the present invention. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope required by the claims.

Claims (8)

1. A living body detecting method, characterized by comprising:

acquiring an image of an object to be detected, which is shot under a binocular camera, wherein the image comprises an infrared image and a visible light image;

Acquiring a visible light image containing a biological feature area according to the acquired image;

calculating a color histogram of a biological feature region in the visible light image based on the biological feature region in the visible light image;

taking the calculated color histogram as color information of a biological feature area in the visible light image, wherein the biological feature area is used for indicating the position of the biological feature of the object to be detected in the image;

Inputting color information of a biological feature area in the visible light image into a support vector machine classifier, and performing color category prediction on the visible light image to obtain a color category of the visible light image;

If the color type of the visible light image indicates that the visible light image is a black-and-white image, judging that the object to be detected is a prosthesis;

If the color type of the visible light image indicates that the visible light image is a color image, judging that the object to be detected is a living body;

If the color information of the biological feature area in the image indicates that the object to be detected is a living body, carrying out deep semantic feature extraction on the biological feature area in the image based on a machine learning model to obtain deep semantic features of the biological feature area in the image;

And carrying out living body judgment of the object to be detected according to the deep semantic features of the biological feature area in the image.

2. The method of claim 1, wherein the machine learning model is a deep neural network model comprising an input layer, a convolutional layer, a connection layer, and an output layer;

The machine learning model-based deep semantic feature extraction is performed on the biological feature area in the image to obtain deep semantic features of the biological feature area in the image, and the machine learning model-based deep semantic feature extraction comprises the following steps:

inputting the image from an input layer in the deep neural network model to the convolution layer;

Extracting features by using the convolution layer to obtain shallow semantic features of a biological feature area in the image, and inputting the shallow semantic features into the connection layer;

and carrying out feature fusion by utilizing the connecting layer to obtain deep semantic features of the biological feature area in the image, and inputting the deep semantic features into the output layer.

3. The method according to claim 2, wherein the performing the living body determination of the object to be detected based on the deep semantic features of the biometric region in the image comprises:

using an activation function classifier in the output layer to classify and predict the image;

and judging whether the object to be detected is a living body or not according to the type predicted by the image.

4. The method of claim 1, wherein after the capturing the image of the object to be detected taken under the binocular camera, the method further comprises:

Performing region position matching between a biological feature region in the infrared image and a biological feature region in the visible light image;

and if the region positions are not matched, judging that the object to be detected is a prosthesis.

5. A living body detecting device, characterized by comprising:

the image acquisition module is used for acquiring an image of an object to be detected, which is shot under the binocular camera, wherein the image comprises an infrared image and a visible light image;

The image physical information extraction module is used for extracting image physical information of a biological feature area in the image to obtain image physical information of the biological feature area in the image, wherein the biological feature area is used for indicating the position of the biological feature of the object to be detected in the image;

The deep semantic feature extraction module is used for extracting deep semantic features of the biological feature area in the image based on a machine learning model if the image physical information of the biological feature area in the image indicates that the object to be detected is a living body, so as to obtain the deep semantic features of the biological feature area in the image;

the object living body judging module is used for judging the living body of the object to be detected according to the deep semantic features of the biological feature area in the image;

wherein, the image physical information is color information, the image physical information extraction module includes:

A visible light image acquisition unit configured to acquire a visible light image including the biometric region from the acquired image;

A color histogram calculation subunit configured to calculate a color histogram of a biological feature region in the visible light image based on the biological feature region in the visible light image;

A color information definition subunit, configured to use the calculated color histogram as color information of a biometric area in the visible light image;

the apparatus further comprises:

The color type prediction unit is used for inputting the color information of the biological feature area in the visible light image into a support vector machine classifier, and performing color type prediction on the visible light image to obtain the color type of the visible light image;

A first object living body determination unit configured to determine that the object to be detected is a prosthesis if a color class of the visible light image indicates that the visible light image is a black-and-white image;

A second object living body determination unit configured to determine that the object to be detected is a living body if the color class of the visible light image indicates that the visible light image is a color image.

6. A service system to which a living body detection method is applied, characterized in that the service system includes a service terminal and an authentication server, wherein,

The service terminal is used for acquiring images of service personnel by using the binocular camera, wherein the images comprise infrared images and visible light images;

The service terminal includes the living body detecting device according to claim 5 for judging whether the service person is living or not based on the image of the service person;

when the service personnel is a living body, the service terminal requests the authentication server to carry out identity authentication on the service personnel, and distributes service business instructions to the service personnel passing the identity authentication.

7. An electronic device, comprising:

A processor; and

A memory having stored thereon computer readable instructions which, when executed by the processor, implement the in-vivo detection method of any one of claims 1 to 4.

8. A storage medium having stored thereon a computer program, which when executed by a processor implements the living body detection method according to any one of claims 1 to 4.