CN112464690A - Living body identification method, living body identification device, electronic equipment and readable storage medium - Google Patents
Living body identification method, living body identification device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN112464690A CN112464690A CN201910841123.0A CN201910841123A CN112464690A CN 112464690 A CN112464690 A CN 112464690A CN 201910841123 A CN201910841123 A CN 201910841123A CN 112464690 A CN112464690 A CN 112464690A
- Authority
- CN
- China
- Prior art keywords
- image
- living body
- face
- feature
- body identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 230000004927 fusion Effects 0.000 claims abstract description 55
- 238000012549 training Methods 0.000 claims description 30
- 230000011218 segmentation Effects 0.000 claims description 16
- 238000011176 pooling Methods 0.000 claims description 10
- 238000013527 convolutional neural network Methods 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 238000013528 artificial neural network Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 abstract description 22
- 230000000694 effects Effects 0.000 abstract description 9
- 238000012795 verification Methods 0.000 abstract description 9
- 230000008859 change Effects 0.000 abstract description 7
- 230000000875 corresponding effect Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 13
- 238000001514 detection method Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 238000002372 labelling Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 239000008280 blood Substances 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 210000004204 blood vessel Anatomy 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001727 in vivo Methods 0.000 description 2
- 230000010349 pulsation Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005242 forging Methods 0.000 description 1
- 230000001121 heart beat frequency Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/162—Detection; Localisation; Normalisation using pixel segmentation or colour matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/40—Spoof detection, e.g. liveness detection
- G06V40/45—Detection of the body part being alive
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application provides a living body recognition method, a living body recognition device, electronic equipment and a readable storage medium. Therefore, on the basis of utilizing the depth image characteristics in the space dimension, the multi-frame RGB image input fusion mode in the time domain dimension is combined, and the image characteristics in the frequency domain dimension are considered, so that the influence of the change of the environment on the living body identification effect can be reduced when the living body identification is carried out, the generalization capability of the living body identification effect is improved, the user does not need to cooperate intentionally, and the verification process period is shortened. In addition, the deployment is convenient, and a plurality of sensors are not required to be arranged, so that the method is suitable for wide terminal equipment scenes.
Description
Technical Field
The present application relates to the field of image recognition technologies, and in particular, to a method and an apparatus for living body recognition, an electronic device, and a readable storage medium.
Background
With the development of social economy and technology, identity verification of real people is generally required to be performed through face recognition in many business scenes, and the method is convenient for users to use. Although the human face features are features which have strong discrimination and are easy to collect, the human face features are also easy to be attacked by fake data of non-real persons, and the safety of identity verification is influenced. Therefore, it is necessary to perform living body recognition on a human face.
Disclosure of Invention
In view of the above, an object of the present application is to provide a method, an apparatus, an electronic device and a readable storage medium for identifying a living body, so as to solve or improve the above problems.
According to an aspect of the present application, there is provided a living body identification method applied to an electronic device, the method including:
continuously collecting multiple frames of RGB images aiming at a face to be recognized in a preset time period, and respectively obtaining a face image of the face to be recognized from each frame of RGB image;
extracting fusion depth image features and fusion frequency domain image features corresponding to the face images;
and performing living body identification according to the fusion depth image characteristics and the fusion frequency domain image characteristics.
According to another aspect of the present application, there is provided a living body identification apparatus applied to an electronic device, the apparatus including:
the image acquisition module is used for continuously acquiring a plurality of frames of RGB images aiming at the face to be recognized in a preset time period and respectively obtaining the face image of the face to be recognized from each frame of RGB image;
the feature extraction module is used for extracting fusion depth image features and fusion frequency domain image features corresponding to the face images;
and the living body identification module is used for carrying out living body identification according to the fusion depth image characteristics and the fusion frequency domain image characteristics.
According to another aspect of the present application, there is provided an electronic device comprising one or more storage media and one or more processors in communication with the storage media, the one or more storage media storing processor-executable machine-executable instructions that, when the electronic device is run, are executed by the processors to perform the aforementioned living body identification method.
According to another aspect of the present application, there is provided a readable storage medium having stored thereon machine executable instructions which, when executed, implement the aforementioned living body identification method.
Based on any one of the aspects, the method comprises the steps of continuously collecting multiple frames of RGB images aiming at a face to be recognized in a preset time period, respectively obtaining face images of the face to be recognized from the frames of RGB images, then extracting fusion depth image features and fusion frequency domain image features corresponding to the face images, and carrying out living body recognition according to the fusion depth image features and the fusion frequency domain image features. Therefore, on the basis of utilizing the depth image characteristics in the space dimension, the multi-frame RGB image input fusion mode in the time domain dimension is combined, and the image characteristics in the frequency domain dimension are considered, so that the influence of the change of the environment on the living body identification effect can be reduced when the living body identification is carried out, the generalization capability of the living body identification effect is improved, the user does not need to cooperate intentionally, and the verification process period is shortened. In addition, the deployment is convenient, and a plurality of sensors are not required to be arranged, so that the method is suitable for wide terminal equipment scenes.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a schematic flow chart illustrating a living body identification method according to an embodiment of the present application;
FIG. 2 shows a sub-flow diagram of step S110 shown in FIG. 1;
FIG. 3 is a schematic flow chart illustrating a living body identification method according to an embodiment of the present disclosure;
FIG. 4 shows a sub-flow diagram of step S130 shown in FIG. 1;
FIG. 5 is a schematic diagram illustrating functional modules of a living body identification device provided by an embodiment of the present application;
fig. 6 shows a block diagram illustrating a structure of an electronic device for implementing the living body identification method according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some of the embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
Based on the technical problems known in the foregoing background art, the inventor of the present application finds that, in the existing face recognition scheme, the living body detection scheme is generally divided into two types, one type is an adaptive detection scheme in which a user completes a corresponding action according to a specific command (for example, a command such as blinking eyes, stretching a tongue, shaking a number, and pointing a head), and the other type is an uncooperative scheme in which the user does not need to make a special action, and only needs to acquire one or more frames of images of the user and then perform living body identification.
In the matching type living body detection scheme, the user is required to complete various actions, so that the verification process period of the face recognition process is long, and the actual use experience of the user is poor.
In non-cooperative in vivo sensing schemes, multiple sensors are typically employed as the acquisition devices for the input data sources. For example, it is common to use an RGB image, a near infrared image, and a depth sensor image as input data sources for living body recognition. Although the use of multiple sensors can increase the amount of information from the data sources and improve the accuracy of in vivo detection. However, multiple sensors necessarily add additional hardware cost and place certain requirements on the electronics during deployment. For example, for a mobile terminal, a capture device of an input data source thereof is generally configured with only a camera capturing RGB images, thus making it difficult to apply a living body discrimination scheme using a multi-sensor in many scenarios.
In the living body detection scheme for performing face recognition by using RGB images, the discrimination accuracy is generally reduced, and the dependence on the illumination condition of the environment where the living body detection is performed, the imaging quality of the camera, and the like is strong, so that the generalization performance of the living body detection is not robust enough when the environment where the living body detection is performed changes. In addition, in the current scheme of using only RGB images for live body detection, the change condition of the face image in the time dimension cannot be mined well, resulting in low accuracy of live body detection. Once false detection occurs, a security risk may be created.
For this reason, based on the findings of the above technical problems, the inventors propose the following technical solutions to solve or improve the above problems. It should be noted that the above prior art solutions have shortcomings which are the results of practical and careful study of the inventor, therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present application in the following description should be the contribution of the inventor to the present application in the course of the invention creation process, and should not be understood as technical contents known by those skilled in the art.
Fig. 1 shows a schematic flow chart of the living body identification method provided in the embodiment of the present application, and it should be understood that, in other embodiments, the order of some steps in the living body identification method of the present embodiment may be interchanged according to actual needs, or some steps may be omitted or deleted. The steps of the living body identification method are described in detail below.
Step S110, continuously collecting multiple frames of RGB images aiming at the face to be recognized in a preset time period, and respectively obtaining the face image of the face to be recognized from each frame of RGB image.
And step S120, extracting fusion depth image characteristics and fusion frequency domain image characteristics corresponding to each face image.
And step S130, performing living body identification according to the fusion depth image characteristics and the fusion frequency domain image characteristics.
In this embodiment, in a scene related to face recognition, for example, in an attendance verification, an access control verification, a payment verification, and the like, in order to prevent an illegal intruder from impersonating a real face by forging a face image, a face video, a face mold, and the like, it is necessary to perform living body detection in a face recognition process. Accordingly, in step S110, a continuous multi-frame RGB image of the face to be recognized may be acquired within a preset time period under the current visual field of the camera through the internal RGB camera or through the external RGB camera connected through external communication. For example, consecutive 10 frames of RGB images of a face to be recognized may be acquired within 1 second.
Wherein each frame of RGB image is an image to be subjected to living body recognition. Each frame of RGB image may be an image frame obtained by image acquisition of a living human face, or an image frame obtained by copying an existing image including a human face. It can be understood that, this embodiment is a scheme for detecting whether the face to be recognized is a live face, because each frame of RGB image may include a live face image and may also include a non-live face image.
According to the living body identification method provided by the embodiment, on the basis of utilizing the depth image characteristics on the space dimension, the multi-frame RGB image input fusion mode on the time domain dimension is combined, and meanwhile, the image characteristics on the frequency domain dimension are considered, so that the influence of the change of the environment on the living body identification effect can be reduced when the living body identification is carried out, the generalization capability of the living body identification effect is improved, the user does not need to cooperate intentionally, and the verification process period is shortened. In addition, the deployment is convenient, and a plurality of sensors are not required to be arranged, so that the method is suitable for wide terminal equipment scenes.
Since the acquired multi-frame RGB images usually include other background areas besides the face area of the face to be recognized, if the other background areas also participate in the living body recognition process, the accuracy of the living body recognition may be affected, and an additional calculation amount is added. In view of this, in one possible implementation, please refer to fig. 2, regarding step S110, it may include sub-step S111 and sub-step S112, which are described in detail below.
And a substep S111, determining a face region in each frame of RGB image through a pre-trained face segmentation model aiming at each frame of RGB image.
And a substep S112, intercepting a face image of the face to be recognized from the frame RGB image according to the face region.
In this embodiment, the face segmentation model may be obtained by a first training sample set through deep learning network training. The first training sample set comprises a plurality of training image samples marked with face marking information.
For example, in an alternative embodiment, a face segmentation model is initialized according to a deep neural network, that is, an initial face segmentation model is obtained according to the deep neural network through random initialization, then a plurality of training image samples labeled with face labeling information are respectively input into the initial face segmentation model, and a face image obtained through analysis is output, at this time, the obtained face image is not necessarily a face image in the face labeling information, so that it is also necessary to compare the output face image of each training image sample with the face image in the face labeling information, determine a loss function value between the output face image and the face image in the face labeling information, then update the face segmentation model according to the loss function value, and through repeated training for a plurality of times, the accuracy of the image obtained through output analysis of the face segmentation model can be continuously improved, the face segmentation model is trained by repeating the above process through a large number of training image samples, and the face segmentation model capable of identifying the face region and the face segmentation information can be obtained. The face labeling information may include face key points or face segmentation information.
Therefore, the face region in each frame of RGB image can be determined through the face segmentation model obtained through training, and then the face image of the face to be recognized is intercepted from the frame of RGB image according to the face region, so that the influence on the accuracy of living body recognition caused by the participation of the background region in the RGB image in the subsequent living body recognition process is avoided, and meanwhile, the calculated amount is reduced.
In addition, in step S120, a detailed description will be given below of the process of acquiring the fused depth image feature corresponding to each face image.
In one possible embodiment, please refer to fig. 3 in combination, first, the depth image features of each face image can be extracted through the depth neural network. The depth image features may be used to represent depth information such as texture, contour, depth abstract information, and the like of the face image, that is, the depth image features may represent a feature map of depth information of the RGB image.
On the basis, the depth image features can be input into a shallow convolutional neural network to obtain fused depth image features corresponding to the face images. The convolutional Neural network cnn (convolutional Neural network) includes a feature extractor composed of a convolutional layer and a sub-sampling layer. In the convolutional layer of the convolutional neural network, one neuron is connected to only part of the neighbor neurons. In a convolutional layer of a convolutional neural network, several Feature planes (Feature maps) are usually included, each Feature plane may be composed of some neurons arranged in a rectangle, and the neurons of the same Feature plane share a weight, where the shared weight may be understood as a convolutional kernel.
For example, for RGB pictures, each color threshold is a feature card, and then the feature cards of red, green and blue are included, and the feature cards of each color are arranged in a rectangle, thereby forming a feature plane. The convolution kernel is generally initialized in the form of a random decimal matrix, and the convolution kernel learns to obtain a reasonable weight in the training process of the network. Sharing the weights (i.e., convolution kernels) can reduce the connections between layers of the convolutional neural network while reducing the risk of over-fitting. Sub-sampling, also called Pooling (Pooling), typically takes the form of Mean sub-sampling (Mean Pooling, averaging over small rectangular coverage) and maximum sub-sampling (Max Pooling, taking the maximum over small rectangular coverage). Sub-sampling can be viewed as a special convolution process. Convolution and sub-sampling greatly simplify the complexity of the model and reduce the parameters of the model.
Therefore, the shallow convolutional neural network corresponding to the above can be used in the embodiment, so that the training speed is increased, the use of the mobile terminal is facilitated, and the real-time training can be performed to continuously perfect the model parameters.
The fusion depth image features can be understood as the depth image features of a plurality of face images fused on the time domain dimension, so that the feature information amount is increased, and the generalization capability of the subsequent living body identification effect is improved.
Meanwhile, referring to fig. 3, the following proceeds to describe in detail the process of acquiring the fused frequency domain image features corresponding to each face image.
In one possible implementation, each face image may first be separately fourier transformed to convert each face image from the spatial domain to the frequency domain, resulting in a frequency domain image of each face image. In particular, in a living body, the pulsation of the heart causes a change in the blood volume in the blood vessel of the skin tissue, and the pulsation of the heart causes a periodic change in the blood volume in the blood vessel of the skin tissue. Therefore, when light irradiates the surface of the skin of a human face, the reflected or transmitted light beam of the light also changes periodically, and therefore the change of the reflected light intensity can represent the beating of the heart.
Therefore, the present embodiment can convert the time domain signal in each face image, which is originally difficult to process, into a frequency domain signal that is easier to recognize a living body by performing fourier transform on each face image. For example, if only the pixel value of each face image is used, it is difficult to analyze the heartbeat information contained therein, and after fourier transform is performed to convert each face image from a spatial domain to a frequency domain, the frequency domain image of each face image can be easily obtained, and the obtained frequency corresponding to the highest-amplitude point in the frequency domain image can be used to represent the characteristic information of the heartbeat frequency of the face to be recognized.
On the basis, the frequency domain image of each face image can be input into a shallow pooling network layer for pooling processing, and the fused frequency domain image characteristics corresponding to each face image are obtained.
The fusion of the frequency domain image features can be understood as the fusion of the frequency domain image features of a plurality of face images in a time domain dimension, so that the feature information amount is increased, and the generalization capability of the subsequent living body identification effect is improved.
Based on the foregoing process, in this embodiment, by performing living body identification according to the fusion depth image feature and the fusion frequency domain image feature, on the basis of using the depth image feature in the spatial dimension, the feature information amount can be increased by combining the multi-frame RGB image input fusion mode in the time domain dimension and considering the image feature in the frequency domain dimension. In the living body identification process, the inventor researches and discovers that if the living body identification is carried out by directly using the fusion depth image characteristic and the fusion frequency domain image characteristic, due to different image characteristics, the comparison needs to be carried out separately in the identification process, so that the comparison calculation amount is increased, and the accuracy of the living body identification is influenced.
In order to solve the above problem, in one possible implementation, please refer to fig. 4, which specifically includes sub-step S131 and sub-step S132 for step S130, and is described in detail below.
And the substep S131, cascading the fusion depth image characteristic and the fusion frequency domain image characteristic to obtain a cascading characteristic image.
In the substep S132, living body recognition is performed based on the cascade feature image.
In the embodiment, after the fusion depth image features and the fusion frequency domain image features are cascaded, originally different image features are cascaded into one cascaded feature image, so that when living body identification is carried out according to the cascaded feature image, individual comparison is not needed, the comparison calculation amount can be reduced, and the accuracy of the living body identification is improved.
In detail, dimensions of different features need to be considered in the cascading process, for example, in an alternative example, for the sub-step S131, a multi-dimensional sub-feature map of the fused depth image feature and a multi-dimensional sub-feature map of the fused frequency-domain image feature may be specifically cascaded to obtain a multi-dimensional cascaded sub-feature map. For example, for a 128-dimensional sub-feature map of a fused depth image feature, the sub-feature map and a 128-dimensional sub-feature map corresponding to a fused frequency-domain image feature may be cascaded, and so on, so as to obtain a cascaded sub-feature map with multiple dimensions. And then, fusing the cascade sub-feature images with multiple dimensions to obtain a cascade feature image.
After the cascade characteristic image is obtained, living body identification can be carried out according to the cascade characteristic image. For example, the living body recognition may be performed by a deep learning manner. For example, in an alternative example, for the sub-step S132, in particular, the living body recognition features of the cascade feature image may be extracted through a pre-trained living body recognition model, and then the face to be recognized is subjected to label classification according to the living body recognition features, so as to obtain a classification label of the face to be recognized. Wherein the classification label is a living body face label or a non-living body face label.
In detail, in one possible implementation, the living body recognition model may be obtained by training a second training sample set with a deep learning network, where the second training sample set includes a plurality of cascaded feature images labeled with classification labels, and the classification labels are living body face labels or non-living body face labels.
For example, a living body recognition model can be initialized according to a deep learning network, that is, an initial living body recognition model is obtained according to the deep learning network through random initialization, then a plurality of cascade feature images labeled with classification labels are respectively input into the initial living body recognition model, classification labels obtained through analysis are output, the obtained classification labels are not necessarily labeled classification labels, therefore, the output classification labels and the labeled classification labels of each cascade feature image need to be compared, loss function values between the output classification labels and the labeled classification labels are determined, then the living body recognition model is updated according to the loss function values, and through repeated training for a plurality of times, the accuracy of the classification labels obtained through output analysis of the living body recognition model can be continuously improved, and the living body recognition model is trained by repeating the processes through a large number of cascade feature images labeled with classification labels, a living body recognition model that can recognize the classification label in the cascade feature image can be obtained. That is, the obtained living body recognition model may have the capability of recognizing a certain cascade feature image as a living body or a non-living body.
Based on the same inventive concept, please refer to fig. 5, which shows a functional module schematic diagram of the living body identification device 200 provided in the embodiment of the present application, and the embodiment can perform functional module division on the living body identification device 200 according to the above method embodiment. For example, the functional blocks may be divided for the respective functions, or two or more functions may be integrated into one processing block. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation. For example, in the case of dividing each function module in correspondence with each function, the living body identification device 200 shown in fig. 5 is only a device diagram. The living body identification device 200 may include an image acquisition module 210, a feature extraction module 220, and a living body identification module 230, and the functions of the functional modules of the living body identification device 200 are described in detail below.
The image acquisition module 210 is configured to continuously acquire multiple frames of RGB images for a face to be recognized within a preset time period, and obtain a face image of the face to be recognized from each frame of RGB image. It is understood that the image capturing module 210 can be used to perform the step S110, and for the detailed implementation of the image capturing module 210, reference can be made to the above description related to the step S110.
And the feature extraction module 220 is configured to extract a fusion depth image feature and a fusion frequency domain image feature corresponding to each face image. It is understood that the feature extraction module 220 can be used to perform the step S120, and for the detailed implementation of the feature extraction module 220, reference can be made to the above description about the step S120.
And the living body identification module 230 is used for carrying out living body identification according to the fusion depth image characteristics and the fusion frequency domain image characteristics. It is understood that the living body identification module 230 can be used to execute the above step S130, and for the detailed implementation of the living body identification module 230, reference can be made to the above description of step S130.
In a possible implementation manner, the image acquisition module 210 may specifically obtain a face image of a face to be recognized from each frame of RGB image by the following method:
determining a face region in each frame of RGB image through a pre-trained face segmentation model aiming at each frame of RGB image, wherein the face segmentation model is obtained by training a first training sample set through a deep learning network, and the first training sample set comprises a plurality of training image samples marked with face marking information;
and intercepting a face image of the face to be recognized from the frame RGB image according to the face area.
In a possible implementation manner, the feature extraction module 220 may specifically extract the fused depth image feature and the fused frequency domain image feature corresponding to each face image by the following method:
respectively extracting the depth image characteristics of each face image through a depth neural network;
and inputting the features of each depth image into a shallow convolutional neural network to obtain the corresponding fusion depth image features of each face image.
In a possible implementation manner, the feature extraction module 220 may specifically extract the fused depth image feature and the fused frequency domain image feature corresponding to each face image by the following method:
respectively carrying out Fourier transform on each face image to convert each face image from a space domain to a frequency domain to obtain a frequency domain image of each face image;
and inputting the frequency domain image of each face image into a shallow pooling network layer for pooling to obtain the fusion frequency domain image characteristics corresponding to each face image.
In one possible embodiment, the living body identification module 230 may specifically perform living body identification by:
cascading the fusion depth image features and the fusion frequency domain image features to obtain a cascading feature image;
and performing living body identification according to the cascade characteristic image.
In one possible implementation, the living body identification module 230 may obtain the cascade feature image by:
cascading the sub-feature maps with multiple dimensions of the fusion depth image features and the sub-feature maps with the dimensions corresponding to the fusion frequency domain image features to obtain cascaded sub-feature maps with multiple dimensions;
and fusing the cascade sub-feature images with multiple dimensions to obtain a cascade feature image.
In one possible embodiment, the living body identification module 230 may specifically perform living body identification by:
extracting living body identification characteristics of the cascade characteristic images through a pre-trained living body identification model;
and performing label classification on the face to be recognized according to the living body recognition characteristics to obtain a classification label of the face to be recognized, wherein the classification label is a living body face label or a non-living body face label.
In a possible implementation manner, the living body recognition model is obtained by training through a deep learning network through a second training sample set, wherein the second training sample set comprises a plurality of cascade feature images marked with classification labels, and the classification labels are living body face labels or non-living body face labels.
Based on the same inventive concept, please refer to fig. 6, which shows a schematic block diagram of a structure of an electronic device 100 for executing the living body identification method, provided by an embodiment of the present application, and the electronic device 100 may include a machine-readable storage medium 120 and a processor 130.
In this embodiment, the machine-readable storage medium 120 and the processor 130 are both located in the electronic device 100 and are separately located. However, it should be understood that the machine-readable storage medium 120 may also be separate from the electronic device 100 and accessible by the processor 130 through a bus interface. Alternatively, the machine-readable storage medium 120 may be integrated into the processor 130, e.g., may be a cache and/or general purpose registers.
The processor 130 is a control center of the electronic device 100, connects various parts of the entire electronic device 100 using various interfaces and lines, performs various functions of the electronic device 100 and processes data by running or executing software programs and/or modules stored in the machine-readable storage medium 120 and calling data stored in the machine-readable storage medium 120, thereby performing overall monitoring of the electronic device 100. Alternatively, processor 130 may include one or more processing cores; for example, the processor 130 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor.
The processor 130 may be a general-purpose Central Processing Unit (CPU), a microprocessor, an Application-Specific Integrated Circuit (ASIC), or one or more Integrated circuits for controlling the execution of programs of the living body identification method provided in the method embodiments described below.
The machine-readable storage medium 120 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an Electrically Erasable programmable Read-Only MEMory (EEPROM), a compact disc Read-Only MEMory (CD-ROM) or other optical disk storage, optical disk storage (including compact disc, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The machine-readable storage medium 120 may be self-contained and coupled to the processor 130 via a communication bus. The machine-readable storage medium 120 may also be integrated with the processor. The machine-readable storage medium 120 is used for storing machine-executable instructions for performing aspects of the present application. The processor 130 is configured to execute machine executable instructions stored in the machine readable storage medium 120 to implement the living body identification method provided by the foregoing method embodiment.
Since the electronic device 100 provided in the embodiment of the present application is another implementation form of the method embodiment executed by the electronic device 100, and the electronic device 100 can be used to execute the living body identification method provided in the method embodiment, the technical effects obtained by the method embodiment can refer to the method embodiment, and are not described herein again.
Further, the present application also provides a readable storage medium containing computer executable instructions, and when executed, the computer executable instructions can be used for the living body identification method provided by the method embodiment.
Of course, the storage medium provided in the embodiments of the present application contains computer-executable instructions, and the computer-executable instructions are not limited to the above method operations, and may also perform related operations in the living body identification method provided in any embodiments of the present application.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the present application has been described in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a review of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
The above description is only for various embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and all such changes or substitutions are included in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (11)
1. A living body identification method applied to an electronic device, the method comprising:
continuously collecting multiple frames of RGB images aiming at a face to be recognized in a preset time period, and respectively obtaining a face image of the face to be recognized from each frame of RGB image;
extracting fusion depth image features and fusion frequency domain image features corresponding to the face images;
and performing living body identification according to the fusion depth image characteristics and the fusion frequency domain image characteristics.
2. The living body recognition method according to claim 1, wherein the step of obtaining the face image of the face to be recognized from each frame of RGB image respectively comprises:
determining a face region in each frame of RGB image through a pre-trained face segmentation model, wherein the face segmentation model is obtained by training a first training sample set through a deep learning network, and the first training sample set comprises a plurality of training image samples marked with face marking information;
and intercepting the face image of the face to be recognized from the frame of RGB image according to the face area.
3. The living body recognition method according to claim 1, wherein the step of extracting the fused depth image feature and the fused frequency domain image feature corresponding to each face image includes:
respectively extracting the depth image characteristics of each face image through a depth neural network;
and inputting the features of each depth image into a shallow convolutional neural network to obtain the fusion depth image features corresponding to each face image.
4. The living body recognition method according to claim 1, wherein the step of extracting the fused depth image feature and the fused frequency domain image feature corresponding to each face image includes:
respectively carrying out Fourier transform on each face image to convert each face image from a space domain to a frequency domain to obtain a frequency domain image of each face image;
and inputting the frequency domain image of each face image into a shallow pooling network layer for pooling to obtain the fusion frequency domain image characteristics corresponding to each face image.
5. The living body identification method according to any one of claims 1 to 4, wherein the step of performing living body identification based on the fused depth image feature and the fused frequency domain image feature comprises:
cascading the fusion depth image features and the fusion frequency domain image features to obtain a cascading feature image;
and performing living body identification according to the cascade characteristic image.
6. The living body identification method according to claim 5, wherein the step of concatenating the fused depth image feature and the fused frequency domain image feature to obtain a concatenated feature image comprises:
cascading the sub-feature maps with multiple dimensions of the fusion depth image feature with the sub-feature map with the dimensions corresponding to the fusion frequency domain image feature to obtain cascaded sub-feature maps with multiple dimensions;
and fusing the cascade sub-feature images of the multiple dimensions to obtain a cascade feature image.
7. The living body identification method according to claim 5, wherein the step of performing living body identification from the cascade feature image includes:
extracting living body identification characteristics of the cascade characteristic image through a pre-trained living body identification model;
and performing label classification on the face to be recognized according to the living body recognition characteristics to obtain a classification label of the face to be recognized, wherein the classification label is a living body face label or a non-living body face label.
8. The living body recognition method according to claim 7, wherein the living body recognition model is obtained by training a second training sample set by using a deep learning network, the second training sample set comprises a plurality of cascade feature images marked with classification labels, and the classification labels are living body face labels or non-living body face labels.
9. A living body recognition apparatus applied to an electronic device, the apparatus comprising:
the image acquisition module is used for continuously acquiring a plurality of frames of RGB images aiming at the face to be recognized in a preset time period and respectively obtaining the face image of the face to be recognized from each frame of RGB image;
the feature extraction module is used for extracting fusion depth image features and fusion frequency domain image features corresponding to the face images;
and the living body identification module is used for carrying out living body identification according to the fusion depth image characteristics and the fusion frequency domain image characteristics.
10. An electronic device comprising one or more storage media and one or more processors in communication with the storage media, the one or more storage media storing processor-executable machine-executable instructions that, when executed by the electronic device, are executed by the processors to perform the living body identification method of any one of claims 1-8.
11. A readable storage medium storing machine executable instructions which when executed perform the living body identification method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910841123.0A CN112464690A (en) | 2019-09-06 | 2019-09-06 | Living body identification method, living body identification device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910841123.0A CN112464690A (en) | 2019-09-06 | 2019-09-06 | Living body identification method, living body identification device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112464690A true CN112464690A (en) | 2021-03-09 |
Family
ID=74807697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910841123.0A Pending CN112464690A (en) | 2019-09-06 | 2019-09-06 | Living body identification method, living body identification device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112464690A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113569707A (en) * | 2021-07-23 | 2021-10-29 | 北京百度网讯科技有限公司 | Living body detection method, living body detection device, electronic apparatus, and storage medium |
| CN114333078A (en) * | 2021-12-01 | 2022-04-12 | 马上消费金融股份有限公司 | Living body detection method, living body detection device, electronic apparatus, and storage medium |
| CN114495328A (en) * | 2022-01-19 | 2022-05-13 | 深圳市凯迪仕智能科技有限公司 | Door lock device and control method thereof |
| CN114626462A (en) * | 2022-03-16 | 2022-06-14 | 小米汽车科技有限公司 | Pavement mark recognition method, device, equipment and storage medium |
| CN115147705A (en) * | 2022-09-06 | 2022-10-04 | 平安银行股份有限公司 | Face copying detection method and device, electronic equipment and storage medium |
| WO2022206319A1 (en) * | 2021-04-02 | 2022-10-06 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, and device, storage medium and computer program product |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107992842A (en) * | 2017-12-13 | 2018-05-04 | 深圳云天励飞技术有限公司 | Biopsy method, computer installation and computer-readable recording medium |
| CN108830308A (en) * | 2018-05-31 | 2018-11-16 | 西安电子科技大学 | A kind of Modulation Identification method that traditional characteristic signal-based is merged with depth characteristic |
| CN109190528A (en) * | 2018-08-21 | 2019-01-11 | 厦门美图之家科技有限公司 | Biopsy method and device |
-
2019
- 2019-09-06 CN CN201910841123.0A patent/CN112464690A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107992842A (en) * | 2017-12-13 | 2018-05-04 | 深圳云天励飞技术有限公司 | Biopsy method, computer installation and computer-readable recording medium |
| CN108830308A (en) * | 2018-05-31 | 2018-11-16 | 西安电子科技大学 | A kind of Modulation Identification method that traditional characteristic signal-based is merged with depth characteristic |
| CN109190528A (en) * | 2018-08-21 | 2019-01-11 | 厦门美图之家科技有限公司 | Biopsy method and device |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022206319A1 (en) * | 2021-04-02 | 2022-10-06 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, and device, storage medium and computer program product |
| CN113569707A (en) * | 2021-07-23 | 2021-10-29 | 北京百度网讯科技有限公司 | Living body detection method, living body detection device, electronic apparatus, and storage medium |
| CN114333078A (en) * | 2021-12-01 | 2022-04-12 | 马上消费金融股份有限公司 | Living body detection method, living body detection device, electronic apparatus, and storage medium |
| CN114495328A (en) * | 2022-01-19 | 2022-05-13 | 深圳市凯迪仕智能科技有限公司 | Door lock device and control method thereof |
| CN114495328B (en) * | 2022-01-19 | 2024-02-20 | 深圳市凯迪仕智能科技股份有限公司 | Door lock device and control method thereof |
| CN114626462A (en) * | 2022-03-16 | 2022-06-14 | 小米汽车科技有限公司 | Pavement mark recognition method, device, equipment and storage medium |
| CN114626462B (en) * | 2022-03-16 | 2023-03-24 | 小米汽车科技有限公司 | Pavement mark recognition method, device, equipment and storage medium |
| CN115147705A (en) * | 2022-09-06 | 2022-10-04 | 平安银行股份有限公司 | Face copying detection method and device, electronic equipment and storage medium |
| CN115147705B (en) * | 2022-09-06 | 2023-02-03 | 平安银行股份有限公司 | Face copying detection method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11783639B2 (en) | Liveness test method and apparatus | |
| CN112215180B (en) | Living body detection method and device | |
| CN112464690A (en) | Living body identification method, living body identification device, electronic equipment and readable storage medium | |
| RU2691195C1 (en) | Image and attribute quality, image enhancement and identification of features for identification by vessels and individuals, and combining information on eye vessels with information on faces and/or parts of faces for biometric systems | |
| US12062251B2 (en) | Device and method with image matching | |
| CN105518709B (en) | The method, system and computer program product of face for identification | |
| WO2019134536A1 (en) | Neural network model-based human face living body detection | |
| CN110889312B (en) | Living body detection method and apparatus, electronic device, computer-readable storage medium | |
| RU2431190C2 (en) | Facial prominence recognition method and device | |
| JP2021101384A (en) | Image processing apparatus, image processing method and program | |
| CN106570489A (en) | Living body determination method and apparatus, and identity authentication method and device | |
| CN108416291B (en) | Face detection and recognition method, device and system | |
| CN111222380B (en) | Living body detection method and device and recognition model training method thereof | |
| CN105426843A (en) | Single-lens lower palm vein and palm print image acquisition device and image enhancement and segmentation method | |
| CN112101195A (en) | Crowd density estimation method and device, computer equipment and storage medium | |
| CN113033519A (en) | Living body detection method, estimation network processing method, device and computer equipment | |
| CN115147936A (en) | Living body detection method, electronic device, storage medium, and program product | |
| Das et al. | Human face detection in color images using HSV color histogram and WLD | |
| CN111126283A (en) | Rapid in-vivo detection method and system for automatically filtering fuzzy human face | |
| Cheng et al. | [Retracted] DTFA‐Net: Dynamic and Texture Features Fusion Attention Network for Face Antispoofing | |
| CN114118203A (en) | Image feature extraction and matching method and device and electronic equipment | |
| RU2735629C1 (en) | Method of recognizing twins and immediate family members for mobile devices and mobile device realizing thereof | |
| Hu et al. | Dynamic Texture Model for Eye Blinking Re-identification under Partial Occlusion | |
| CN115797980A (en) | Palm print identification method, palm print identification system, electronic equipment and storage medium | |
| Mohammed et al. | Performance Evolution Ear Biometrics Based on Features from Accelerated Segment Test |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |