CN112307973B - Living body detection method, living body detection system, electronic device, and storage medium - Google Patents

Abstract

The embodiment of the invention relates to the field of face recognition, and discloses a method, a system, electronic equipment and a storage medium for detecting a living body. The invention is applied to electronic equipment with at least two image acquisition devices, wherein the at least two image acquisition devices acquire images at the same position, and the method comprises the following steps: simultaneously acquiring at least two pictures through the image acquisition device; and respectively sending the pictures to at least one main neural network and at least one auxiliary neural network, wherein the output result of the main neural network is fused with the output result of the auxiliary neural network. Outputting the living body characteristics through the main neural network and the auxiliary neural network, and sending the living body characteristics to a full connection layer; outputting the category of the living body characteristic as the result of the living body detection through the full connection layer and the softmax layer. The living body detection through the multi-input flow neural network architecture has higher precision and speed.

Description

Liveness detection method, system, electronic device and storage medium

technical field

Embodiments of the present invention relate to the field of face recognition, and in particular to a living body detection method, system, electronic device and storage medium.

Background technique

Face recognition is widely used in face recognition payment, face access control, face attendance, witness verification and other scenarios. Usually face recognition systems include face photo collection, face detection, face feature extraction, face search/ Compared with other functional modules. When the collected photos are printed, electronic screen output or face images wearing masks, the system can easily be invaded by forged faces by outsiders. In order to avoid fake face fraud, adding a live face detection module to the face recognition system can improve the security of the system. The main function of live face detection is to judge whether the face to be recognized is a real person and give is the confidence information of the real person.

However, related technologies usually use a single-input stream deep neural network architecture for live detection. The input image is one of RGB images, depth images, and near-infrared images. The network structure is relatively simple, but since the input is a single type of image information , it is difficult to improve the accuracy of liveness detection after reaching a certain value.

Contents of the invention

The purpose of the embodiment of the present invention is to provide a living body detection method, system, electronic device and storage medium, and the precision and speed of the living body detection through the multi-input stream neural network architecture are higher.

In order to solve the above technical problems, an embodiment of the present invention provides a living body detection method, comprising the following steps:

Obtain at least two pictures simultaneously through the image acquisition device;

The pictures are respectively sent to at least one main neural network and at least one auxiliary neural network, wherein the output result of the main neural network is fused with the output result of the auxiliary neural network.

Output living body features through the main neural network and the auxiliary neural network, and send to the fully connected layer;

The category of the living body feature is output through the fully connected layer and the softmax layer as the result of the living body detection.

Embodiments of the present invention also provide a living body detection system, including:

An image acquisition module, configured to simultaneously acquire at least two images through the image acquisition device;

A living body detection module, configured to send the pictures to at least one main neural network and at least one auxiliary neural network, wherein the output of the main neural network is fused with the output of the auxiliary neural network; through the main The neural network and the auxiliary neural network output the living body features and send them to the fully connected layer;

The result output module is used to output the category of the living body feature through the fully connected layer and the softmax layer.

Embodiments of the present invention also provide an electronic device, including:

at least one processor; and,

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the living body detection method.

The embodiment of the present invention also provides a computer storage medium, including: a computer program is stored, and it is characterized in that, when the computer program is executed by a processor, a living body detection method is realized.

Compared with the prior art, the embodiment of the present invention adopts multi-device to collect pictures and multi-network to detect pictures, which has higher accuracy than the previous one which only uses a single picture to detect, and at the same time, different neural networks can be set as the main and auxiliary according to the needs. The neural network, especially the network model with higher accuracy and faster speed can be set in the main and auxiliary neural network respectively, so that the living body detection has the advantages of both accuracy and speed, and improves the accuracy and speed. And multiple neural networks can be set as needed to adapt to various application scenarios.

In addition, the living body detection method provided by the present invention, after simultaneously acquiring at least two pictures by the image acquisition device, includes: detecting whether there is a human face image in the picture; if the picture has the human face image, then Send the face image to the main neural network. Before the liveness detection, the images are first screened, and only the pictures with portraits are subjected to further liveness detection, which reduces the number of detections and speeds up the processing.

In addition, in the living body detection method provided by the present invention, when the output result of the main neural network is fused with the output result of the auxiliary neural network, both the main neural network and the auxiliary neural network include a plurality of Blocks; the method The method includes: fusing the output result of each block in the main neural network with the output result of the same block in the auxiliary neural network; sending the output result to the next Block in the main neural network. The output of each Block in the main neural network is fused with the output of the auxiliary neural network, so that the output results have both the advantages of the network model applied by the main neural network and the auxiliary neural network, making the detection of living body faster and more efficient .

In addition, in the living body detection method provided by the present invention, the live body features are output through the main neural network and the auxiliary neural network and sent to the fully connected layer, including: combining the features output by the main neural network with the auxiliary neural network The features output by the neural network are fused; and the fused result is sent to the fully connected layer. The features output by the main and auxiliary networks are fused, and finally the entire network model for liveness detection is obtained to obtain a final result, so that the final result can be compared with the preset category.

Description of drawings

One or more embodiments are exemplified by the pictures in the corresponding drawings, and these exemplifications do not constitute a limitation to the embodiments. Elements with the same reference numerals in the drawings represent similar elements. Unless otherwise stated, the drawings in the drawings are not limited to scale.

Fig. 1 is a flow chart 1 of the living body detection method provided by the first embodiment of the present invention;

Fig. 2 is a flow chart 2 of the living body detection method provided by the first embodiment of the present invention;

Fig. 3 is a flowchart of the training of the living body detection model provided by the second embodiment of the present invention;

Fig. 4 is a flow chart of the living body detection system provided by the third embodiment of the present invention;

FIG. 5 is a schematic structural diagram of an electronic device provided by a fourth embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, various implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings. However, those of ordinary skill in the art can understand that, in each implementation manner of the present invention, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following implementation modes, the technical solution claimed in this application can also be realized.

The division of the following embodiments is for the convenience of description, and should not constitute any limitation to the specific implementation of the present invention, and the various embodiments can be combined and referred to each other on the premise of no contradiction.

The first embodiment of the present invention relates to a living body detection method. The specific process is shown in Figure 1.

Step 101, simultaneously acquire at least two pictures through the image acquisition device.

In this embodiment, at least two image acquisition devices are provided on the living body detection device, and the image acquisition devices may be cameras, or other devices that can be used for image acquisition. The image acquisition equipment is relatively fixed and can simultaneously acquire pictures.

Specifically, the acquired picture may be a color (RGB) picture, a near-infrared picture, a depth map, etc. In practice, at least one RGB picture is generally used, but there is no limitation on the type of picture to be acquired here.

Furthermore, it is necessary to perform face detection on the picture before entering the main and auxiliary neural networks. If there is no face in the picture, it is not necessary to perform subsequent liveness detection. The specific process is shown in Figure 2:

Step 201, sending the first picture to the face detection module.

Specifically, the face detection module uses existing deep neural networks for face detection, such as MTCNN, YOLO, RetinaNet, etc.

Step 202, the face detection module sends the face image in the first picture to the main neural network.

In step 203, the face detection module sends the position coordinates of the face in the first picture to the O-Net module.

Specifically, the O-Net module includes an O-Net architecture and other logic functions, and is an independent logic module for processing the second picture, that is, returning the face image in the second picture.

In step 204, the O-Net module combines the position coordinates to return the face image in the second picture.

Step 205, sending the face image in the second picture to the auxiliary neural network.

Specifically, the main neural network receives the first picture processed by the face detection module, and the auxiliary neural network receives the second picture processed by the O-Net module.

It should be noted that in practice, the main neural network is generally set to a higher-precision neural network, and the auxiliary neural network is set to a faster neural network. The reason is that the results of Block in the main neural network need to be fused with the auxiliary neural network. As a result of the Block in the network, the fused auxiliary neural network cannot generally be slower than the main neural network, otherwise it will slow down the work of the main neural network and the speed of the entire living body detection. Due to this setting in practice, the O-Net module with a relatively fast regression speed is used in the auxiliary neural network to speed up the processing speed of the auxiliary neural network.

Step 102, sending the pictures to at least one main neural network and at least one auxiliary neural network respectively, wherein the output result of the main neural network is fused with the output result of the auxiliary neural network.

In this embodiment, the main neural network generally adopts a neural network architecture with higher precision, and the auxiliary neural network generally adopts a neural network architecture with faster processing speed, but it can be used differently according to different application scenarios of the living body detection method described in the present invention. The neural network architecture of , which is not specifically limited here.

For example, the main neural network adopts the BlockA structure based on ResNet (Residual Network), and the auxiliary neural network adopts the BlockB structure based on MobileNet:

Specifically, 4 convolutional layers are generally used in BlockA, and residual is added to the output of the 2nd and 4th convolutions, and 6 convolutional layers are generally used in BlockB, and a residual is added after the 3rd convolutional layer residual, this method effectively solves the problem of network degradation.

Furthermore, the convolution of BlockB uses the separation structure of depthwise (dw) and pointwise (pw) to extract features. Compared with conventional convolution operations, the number of parameters and computational cost are relatively low, which greatly reduces the amount of calculation.

In this embodiment, the output of each BlockA structure in the main neural network will be fused with the output result of BlockB at the same level, and the fused result will be sent to the next BlockA structure, for example, the output result of BlockA_1 will be fused with the output result of BlockB_1 , and send the fused result to BlockA_2.

Specifically, the fusion method can be concatenation of the channel dimension or element-wise add, and other feature fusion methods in the convolutional neural network can be used, which are not specifically limited here.

Step 103, output the living body features through the main neural network and the auxiliary neural network, and send to the fully connected layer.

In this embodiment, the main neural network and the auxiliary neural network respectively output features and fuse the features, and the fusion method may be concatenation.

Step 104, output the category of the living body feature through the fully connected layer and the softmax layer as the result of the living body detection.

In this embodiment, the neural network is cascaded with a fully connected layer and a softmax layer to process the features output by the neural network.

Specifically, the fully connected layer is used to reduce the influence of the feature position on the classification, and the softmax layer is used to output the category of the feature.

The step division of the above various methods is only for the sake of clarity of description. During implementation, it can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. ; Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.

The second embodiment of the present invention relates to the training of a living body detection model, as shown in FIG. 3 , including:

Step 301, acquire a picture through an image acquisition device.

In this embodiment, the pictures acquired include living body pictures and non-living body pictures. The living body pictures are pictures of real people, and non-living body pictures are not limited to printed face photos, face photos displayed on electronic screens, faces wearing masks, cartoons, etc. The face of a person, etc. Since the purpose of this embodiment is to train the living body detection model, the more training samples are, the better the training effect will be, and the higher the accuracy of the living body detection will be.

Step 302, classifying and labeling the acquired pictures.

In this embodiment, the captured pictures are marked as corresponding categories according to the captured objects. For example, the categories are divided into real people, printed photos, photos of electronic screens, photos wearing masks, and other non-living objects.

Specifically, the labeled label can be a one-hot vector, that is, the labeled value is 1 for the category corresponding to the picture, and 0 for others.

Step 303, pre-processing the pictures to construct a training set.

In this embodiment, the manner of pre-processing is shown in step 201 and steps 203-204, which will not be repeated here.

The first picture processed in step 201 and the second picture processed in step 204 are added to the training set and the category of the group of pictures is marked.

Step 304, train the network model through the training set.

In this embodiment, the structure of the network model is as shown in step 102, which will not be repeated here.

Specifically, the features output by the main and auxiliary neural networks are fused, the feature categories obtained by the network model are output through the fully connected layer and the softmax layer, and the loss function is output with the pre-marked category, as shown in formula (1):

Among them, y _i is the category of the input picture pair, and O _i is the output category of softmax.

Further, the network model is trained by the stochastic gradient descent method (SGD), and the network model and optimal parameters are saved.

The third embodiment of the present invention relates to a living body detection system, as shown in FIG. 4 , including:

The picture acquisition module 401 is configured to simultaneously acquire at least two pictures through the image acquisition device.

In this embodiment, the picture acquisition module 401 includes a picture acquisition module and a face recognition module.

Specifically, the image acquisition module includes at least two image acquisition devices, and the image acquisition devices are relatively fixed and acquire images of the same position.

The face recognition module includes a face detection module and an O-Net module, wherein the face detection module processes the first picture and sends it to the main neural network, and the O-Net module processes the second picture and sends it to the auxiliary neural network.

The living body detection module 402 is configured to send the pictures to at least one main neural network and at least one auxiliary neural network, wherein the output result of the main neural network is fused with the output result of the auxiliary neural network; through the The main neural network and the auxiliary neural network output the living body features and send them to the fully connected layer;

Specifically, the living body detection module includes at least one main neural network and at least one auxiliary neural network, and a fully connected layer and a softmax layer cascaded with the neural network, wherein each layer is connected in a cascaded manner.

The result output module 403 is configured to output the category of the living body feature as the result of the living body detection through the fully connected layer and the softmax layer.

Specifically, the result output module will output the living body detection results according to the preset categories.

It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in cooperation with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here in order to reduce repetition. Correspondingly, the relevant technical details mentioned in this implementation manner can also be applied in the first implementation manner.

It is worth mentioning that all the modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, or a part of a physical unit, or multiple physical units. Combination of units. In addition, in order to highlight the innovative part of the present invention, units that are not closely related to solving the technical problems proposed by the present invention are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.

The fifth embodiment of the present invention relates to an electronic device, as shown in FIG. 5 , including:

At least one processor 501; and, a memory 502 communicatively connected to the at least one processor 501; wherein, the memory stores instructions executable by the at least one processor, and the instructions are processed by the at least one processor implemented by a processor, so that the at least one processor can execute any one of the living body detection methods.

Wherein, the memory and the processor are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus links one or more processors and various circuits of the memory together. The bus may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein. The bus interface provides an interface between the bus and the transceivers. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing means for communicating with various other devices over a transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, further, the antenna also receives the data and transmits the data to the processor.

The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Instead, memory can be used to store data that the processor uses when performing operations.

Those skilled in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program. The program is stored in a storage medium and includes several instructions to make a device (which can be a single-chip , chip, etc.) or a processor (processor) executes all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-OnlyMemory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes.

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific examples for realizing the present invention, and in practical applications, various changes can be made to it in form and details without departing from the spirit and spirit of the present invention. scope.

Claims (9)

1. A method for detecting a living body, applied to an electronic device having at least two image capturing devices, wherein the at least two image capturing devices capture images of a same position, comprises:

simultaneously acquiring at least two pictures through the image acquisition device;

respectively sending the pictures to at least one main neural network and at least one auxiliary neural network, wherein the output result of the main neural network is fused with the output result of the auxiliary neural network;

outputting the living body characteristics through the main neural network and the auxiliary neural network, and sending the living body characteristics to a full connection layer;

outputting the category of the living body characteristic as a result of the living body detection through the full connection layer and the softmax layer;

in the process of fusing the output result of the main neural network with the output result of the auxiliary neural network, the main neural network and the auxiliary neural network both comprise a plurality of blocks; the method comprises the following steps:

fusing the output result of each Block in the primary neural network with the output result of the Block in the same stage in the auxiliary neural network;

and sending the output result to the next Block in the master neural network.

2. The in-vivo detection method according to claim 1, comprising, after said simultaneously acquiring at least two pictures by said image acquisition device:

detecting whether the first picture has a face image;

and if the first picture has the face image, the face image is sent to the main neural network.

3. The in-vivo detection method according to claim 2, wherein the first picture having the face image sends the face image to the autonomic neural network, further comprising:

and sending the position coordinates of the human face.

4. The in-vivo detection method according to claim 3, after the sending of the coordinates of the position of the face, comprising:

regressing the face image in the second picture by combining the position coordinates;

and sending the face image in the second picture to the auxiliary neural network.

5. The in-vivo detection method according to claim 1, wherein said sending the pictures to at least one primary neural network and at least one secondary neural network respectively comprises:

and training the main neural network and the auxiliary neural network according to preset classification, wherein the classification at least comprises a live human living body class.

6. The in-vivo detection method according to claim 1, wherein the outputting of the in-vivo characteristics through the main neural network and the auxiliary neural network and the sending to a full connection layer comprises:

fusing the characteristics output by the main neural network and the characteristics output by the auxiliary neural network;

and sending the fusion result to the full connection layer.

7. A living body detection system, comprising:

the image acquisition module is used for acquiring at least two images simultaneously through the image acquisition device;

the living body detection module is used for respectively sending the pictures to at least one main neural network and at least one auxiliary neural network, wherein the output result of the main neural network is fused with the output result of the auxiliary neural network; outputting the living body characteristics through the main neural network and the auxiliary neural network, and sending the living body characteristics to a full connection layer;

a result output module for outputting the category of the living body feature as a result of the living body detection through the full connection layer and the softmax layer;

wherein, the in vivo detection module is specifically configured to: in the process of fusing the output result of the main neural network with the output result of the auxiliary neural network, the main neural network and the auxiliary neural network both comprise a plurality of blocks;

Fusing the output result of each Block in the main neural network with the output result of the Block in the same level in the auxiliary neural network;

and sending the output result to the next Block in the master neural network.

8. An electronic device, comprising:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the liveness detection method of any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the in-vivo detection method according to any one of claims 1 to 6.

Images