CN104361328B - A kind of facial image normalization method based on adaptive multiple row depth model - Google Patents
A kind of facial image normalization method based on adaptive multiple row depth model Download PDFInfo
- Publication number
- CN104361328B CN104361328B CN201410677837.XA CN201410677837A CN104361328B CN 104361328 B CN104361328 B CN 104361328B CN 201410677837 A CN201410677837 A CN 201410677837A CN 104361328 B CN104361328 B CN 104361328B
- Authority
- CN
- China
- Prior art keywords
- ssda
- image
- training
- layer
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010606 normalization Methods 0.000 title claims abstract description 22
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 12
- 230000001815 facial effect Effects 0.000 title claims abstract 18
- 238000012549 training Methods 0.000 claims abstract description 65
- 238000000034 method Methods 0.000 claims abstract description 50
- 230000008859 change Effects 0.000 claims abstract description 28
- 238000012937 correction Methods 0.000 claims abstract description 16
- 238000005457 optimization Methods 0.000 claims abstract description 9
- 238000005286 illumination Methods 0.000 claims description 46
- 238000013528 artificial neural network Methods 0.000 claims description 23
- 239000013598 vector Substances 0.000 claims description 14
- 230000004913 activation Effects 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 230000006698 induction Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims description 2
- 238000011176 pooling Methods 0.000 claims 1
- 238000013459 approach Methods 0.000 abstract description 2
- 230000006872 improvement Effects 0.000 abstract description 2
- 238000003062 neural network model Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 5
- 238000012360 testing method Methods 0.000 description 4
- 230000000875 corresponding effect Effects 0.000 description 3
- 230000002411 adverse Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013179 statistical model Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 208000027697 autoimmune lymphoproliferative syndrome due to CTLA4 haploinsuffiency Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to a kind of facial image normalization method based on adaptive multiple row depth model, this approach includes the following steps S1：Establish adaptive multiple row depth model；S2：The training of adaptive multiple row depth model；S3：Target facial image normalization.A kind of face normalization method based on the adaptive depth model of multiple row provided by the invention, by the way that the progress linear combination of multiple row depth model is realized that the joint of a variety of unfavorable factors for influencing recognition of face is corrected, the best initial weights of each row depth model are adaptively calculated using nonlinear optimization method simultaneously, i.e., adaptively adjust the improvement factor of each factor according to input picture.Compared with traditional method for carrying out face correction using single deep neural network model, the face normalization method provided by the invention based on the adaptive depth model of multiple row is stronger to the robustness of various change factor.
Description
Technical Field
The invention belongs to the technical field of face recognition, and relates to a face image normalization method based on a self-adaptive multi-column depth model.
Background
As a typical biological feature recognition technology, the face recognition is favored by people with the advantages of high naturalness and acceptability, convenience in concealment and the like, and has wide application prospects in the aspects of national public safety, military safety, financial safety, human-computer interaction and the like. However, the best face recognition system in the world can basically meet the requirements of general applications only under the conditions of relatively good cooperation of users and relatively ideal acquisition conditions. In an unconstrained environment (user mismatch and non-ideal acquisition conditions), face recognition is difficult due to poor stability of face features, large influence of various external conditions (such as different illumination conditions, shielding and other factors), and the like. The normalization of the face image refers to the correction of the face image affected by illumination change, angle change, expression change, shielding and other factors to the front face under standard conditions, and the above difficult problems of face recognition can be effectively solved. The existing face image normalization method usually only removes and corrects one or two factors, and can be divided into face posture correction and face illumination correction.
Posture correction can be generally divided into two categories: 2D-based posture correction and 3D-based posture correction. The 2D-based method performs angle rectification by 2D image matching or a method of encoding a test image using basis functions or samples. For example, documents c.d. castillo and d.w. jacobs, "Wide-base stereo for face registration with large position variation," in IEEE Conference on Computer Vision and Pattern Registration (CVPR)2011, and pp.537-544 calculate the similarity between two faces using stereo matching. Document a.li, s.shan, and w.gao, "Coupled bias-variance tradoff for cross-face recognition," IEEE Transactions on Image Processing vol.21, pp.305-315,2012 represents a face Image to be tested by a linear combination of training images, and performs face recognition using linear regression coefficients as features. The 3D method usually first obtains 3D face data or estimates a 3D model from the 2D image and then matches it with a 2D test face image. Documents s.li, x.liu, x.chai, h.zhang, s.lao, and s.shann, "morphology field based image matching for face recognition access," incecv: Springer,2012, pp.102-115 first generate a virtual angle for the test face image through a series of 3D displacement fields (displacement fields) sampled from a 3D face database, and then match the synthesized face with the registered face. Similarly, documents a.astana, t.k.marks, m.j.jones, k.h.tieu, and m.rohith, "full automatic position-innovative surface registration via 3D position registration," in IEEE International Conference on Computer Vision (ICCV)2011, and pp.937-944 use an Active angle-based Appearance Model (AAM) to achieve matching of the 3D Model to the 2D image.
The face illumination preprocessing method can be divided into three types: an illumination normalization method, a method for modeling illumination variation, and a method for extracting illumination invariant features. The illumination normalization is to pre-process the face image by adopting an image processing technology to obtain the face image under the uniform illumination condition. Histogram Equalization (HE), Gamma correction, and Logarithmic Transformation (LT) are the most common methods of illumination normalization. Methods of modeling illumination variations typically assume the effects of illumination on the face image and then use these assumptions to model or remove the illumination effects. The illumination change modeling method can be divided into two types, namely a statistical model-based method and a physical model-based method. The statistical model-based method obtains a linear subspace with approximate illumination change by learning images formed by each person under different illumination conditions, and the linear subspace method is commonly known as Eigenface, Fisherface, segmented linear subspace method and the like. An assumption (such as a Lambertian surface assumption) is made on the illumination reflection property of the surface of an object by a physical model-based method, and a face image model under different illumination conditions is obtained according to the assumption, wherein the face image model typically has an illumination cone and a spherical harmonic function. One drawback of the model-based approach is that a large number of pictures of the face under different lighting conditions and different 3D model information are required for training. Moreover, the human face is not a perfect Lambertian surface. The above disadvantages limit the application of the illumination variation modeling method. The method for extracting the illumination invariant features is based on the simple idea of finding a face expression mode with invariant illumination. The method for extracting the illumination invariant feature comprises a gradient feature method, a Quotient Image (QI) method and a Retinex model-based method. Typical gradient features are edge maps, image gradients, and Local Binary Patterns (LBP). The method does not depend on a physical model, is simple to implement, has a limited improvement on the identification effect, and has an obvious effect only under the condition that the illumination change is not severe. The quotient image is the ratio of the test image and the composite image (linear combination of 3 images not under the same illumination condition), and can be regarded as an illumination invariant since the ratio in the Lambertian reflectance model is only related to the reflectance of the object. The Retinex model is a simplified version of the Lambertian model. Compared with the quotient image method, the Retinex-based method has the advantages that only one image is needed to extract the illumination invariants of the image, and the image does not need to be aligned. The illumination preprocessing method based on Retinex includes Single Scale Retinex (SSR), Multi-Scale Retinex (MSR), Self-Quotient Image (SQI), Morphological Quotient Image (MQI) and the like.
The face image normalization method has certain limitations. For example, acquiring 3D data requires additional computational effort and resources; and deriving 3D models from 2D data is an ill-posed problem; the statistical illumination model is usually obtained from a constraint environment and cannot be well popularized to practical application. More importantly, the above method can only remove the influence of a few one or two adverse factors. In practical situations, various influencing factors such as illumination, angle, expression, and occlusion are usually interacted, and the action of one changing factor is usually correlated with other changing factors, in which case, when one or two changing factors are processed separately, it is difficult to achieve a real practical effect due to the influence of other factors. Therefore, a more effective human face image normalization method is needed.
Disclosure of Invention
In view of the above, the present invention provides a face image normalization method based on an adaptive multi-column depth model,
in order to achieve the purpose, the invention provides the following technical scheme:
a face image normalization method based on a self-adaptive multi-column depth model comprises the following steps:
s1: establishing a self-adaptive multi-column depth model;
s2: training a self-adaptive multi-column depth model; the training includes S21: training a deep neural network; s22: prediction of training data weights; s23: training a weight prediction module;
s3: and normalizing the target face image.
Further, the training of the deep neural network in S21 includes S211: training a stack-type sparse denoising self-encoder SSDA; s212: and (5) training a Deep Convolutional Neural Network (DCNN).
Further, the training of the SSDA comprises the steps of:
s2111: the SSDA consists of K DAs, the training of the SSDA is the training of each DA, and the DA can be trained by optimizing the following sparse regularization reconstruction loss function:
where y denotes a clean training image, x denotes an image in which y is contaminated with noise, and N pieces of training data D { (x)1,y1),...,(xN,yN)}；is the output of DA, and the reconstruction of x is the approximation of y, theta is the parameter to be optimized, lambda, beta and rho are hyper-parameters, J is the number of hidden nodes,is the average output value vector of hidden layer nodes, sparse induction termIs the target activation value rho and the average activation value of the jth hidden nodeBy selecting a smaller target activation value p such thatThe component of (a) is as small as possible;
s2112: after the first DA is trained, the hidden layer outputs of the clean image and the polluted image are respectively used as the clean image and the polluted image to train a second DA, and the process is repeated until the training of K DAs is completed;
s2113: the parameters of the entire network are fine tuned using standard back propagation algorithms by minimizing the loss function:
wherein W(l)Parameters for layer l in SSDA are indicated.
Further, the training of the DCNN comprises the steps of:
s2121: the initialization of the parameters is carried out in a layer-by-layer mode:
wherein Y is a target image group route set corresponding to the input image set X, X1、X2、X3Outputs of three local connection layers respectively; o, P and Q are all a fixed binary matrix consisting of W1X、W2X1And W3X2Adding the pixel values of the same position in the generated feature map for OW1X、PW2X1And QW3X2The same size as Y;
s2121: updating parameters by minimizing the following reconstruction error loss function by adopting a random gradient descent method:
wherein W ═ { W ═ W1,W2,W3,W4}。
Further, the prediction of the training data weight in S22 is the optimal weight vector S ═ S for solving the training data D1,...,sc]TBy solving a quadratic optimization problem mins Is obtained in whichRepresenting a combination of C DNN output images.
Further, the weight prediction module in the training of the S23 weight prediction module is a Radial Basis Function (RBF) neural network, and the parameters of the RBF are trained by adopting a standard neural network parameter optimization method by taking the characteristic vector phi as input and S as output; after step S22, a new training sample set (Φ, S) is generated, where Φ ═ f1,...,fC]The feature vectors are obtained by connecting output values of SSDA hidden layer nodes or DCNN feature extraction layers; f. ofiA hidden layer (SSDA) of the ith column DNN or a vector of output values of a feature extraction layer (DCNN), i 1.., C;
further, the S3 method for normalizing the target face image includes AMC-SSDA, AMC-DCNN and AMC-SSDA + DCNN.
The invention has the beneficial effects that: the invention provides a face normalization method based on a multi-column self-adaptive depth model, which realizes the combined correction of various adverse factors influencing face recognition by linearly combining the multi-column depth model, and meanwhile, the optimal weight of each column of depth model is self-adaptively calculated by utilizing a nonlinear optimization method, namely, the correction factor of each factor is self-adaptively adjusted according to an input image. Compared with the traditional method for correcting the face by using a single depth neural network model, the method based on the self-adaptive multi-column depth model has stronger robustness on various change factors.
Drawings
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings, in which:
FIG. 1 is a diagram of an adaptive multi-column depth model architecture;
FIG. 2 is a view of the SSDA architecture;
FIG. 3 is a diagram of the structure of DCNN.
Detailed Description
Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
The invention provides a face image normalization method based on a self-adaptive multi-column depth model, which comprises the following steps of:
step one, establishing a self-adaptive multi-column depth model. The adaptive multi-column depth model is formed by linearly combining a plurality of Deep Neural Networks (DNNs), as shown in fig. 1, each DNN (one column in fig. 1) is used for removing the influence of a certain type of factors (one or a combination of several of factors such as illumination, angle, occlusion, expression) and the parameters of the DNN are also trained by using data influenced by the factors. An uncorrected image is input, each DNN corrects the image by a certain factor and outputs a restored image, the weight prediction module predicts an optimal combined weight (namely, the weight is determined by the input image in a self-adaptive mode) according to the output (characteristic) of each DNN hidden layer node, and the final restored image is a weighted average of each DNN output image. The deep neural network DNN may select a Stacked Sparse Denoising Autoencoder (SSDA), a Deep Convolutional Neural Network (DCNN), or a combination thereof. The SSDA method takes the face picture affected by various factors as a noisy image, removes various affected factors in the face picture by a denoising method, and recovers the face with normal illumination, front, no shielding and natural expression; and the DCNN firstly extracts the characteristics with invariance through a deep convolution network, and then carries out face reconstruction to directly reconstruct the face with normal illumination, front, no shielding and natural expression.
And step two, training the model. The training process of the adaptive multi-column depth model comprises the training of a deep neural network (SSDA, DCNN), the prediction of the weight of training data and the training of a weight prediction module. The method comprises the following steps:
and step 21, training the deep neural network. The training of the deep neural network includes training of the SSDA and training of the DCNN.
Step 211, training of SSDA. The SSDA is composed of a plurality of Denoise Autoencoders (DA). The DA is a two-layer neural network (an input layer, a hidden layer, and an output layer) consisting of 1 encoder and 1 decoder. An SSDA formed by K DAs is 1 multi-layer neural network and comprises an input layer, 2K-1 hidden layers and an output layer. Let the kth DA comprise an encoderAnd a decoderThen the SSDA may be divided into code portionsToAnd a decoding part fromToAs shown in fig. 2.
Since the SSDA is composed of a plurality of DAs, training of the SSDA is training of each DA. Let y denote a clean training image, x denote an image with y contaminated with noise, and given N training data D { (x)1,y1),...,(xN,yN) The DA can be trained by optimizing a sparse regularized reconstruction loss function as follows.
is the output of DA (reconstruction of x) is an approximation to y, Θ ═ W, b, W ', b' } are parameters to be optimized λ, β and ρ are hyperparameters, J is the number of hidden nodes,is the average output value vector of hidden layer nodes, sparse induction termIs the target activation value rho and the average activation value of the jth hidden nodeBy selecting a smaller target activation value p such thatThe component (c) is as small as possible, so that the output of the hidden layer in the optimization process is 0 most of the time to achieve sparsity. After training the first DA, the hidden layer outputs of the clean image and the contaminated image are respectively used as the clean image and the contaminated image to train the second DA, and the process is repeated until the training of the K DA is completed. Finally, the parameters of the entire network are fine tuned using standard back propagation algorithms by minimizing the following loss function:
wherein W(l)Parameters for layer l in SSDA are indicated.
Step 212, training of the DCNN. The DCNN is composed of a feature extraction layer and a reconstruction layer, as shown in fig. 3. The feature extraction layer is composed of 3 local connection layers and 2 posing layers and is used for extracting invariant features. Wherein the local connection layer passes through a sparse weight matrix W1、W2、W3Filtering the input image (or the feature map output by the previous layer) to generate a feature map (or a new feature map), and passing the posing layer through a matrix V1And V2The feature map (feature map) output by the local connectivity layer is downsampled to reduce the number of parameters that need to be learned while preserving more robust features. The reconstruction layer is a full connection layer and passes through a weight matrix W4The extracted features are transformed into a frontal face image under standard conditions.
The training of the DCNN is mainly divided into two processes of initialization and updating of parameters. The initialization process of the parameters can be performed in a layer-by-layer manner as shown in the following formula:
wherein Y is a set of target images (ground route) corresponding to the set of input images X, X1、X2、X3Outputs of three local connection layers respectively; o, P and Q are all a fixed binary matrix consisting of W1X、W2X1And W3X2Adding the pixel values of the same position in the generated feature map for OW1X、PW2X1And QW3X2The same size as Y. According to the above formula, the input picture X is first passed through a linear transformation W without posing1To approximate Y. Once W is1After initialization, W can be used1Calculating X1Then, the second expression above can be used to pair W2The initialization is performed and repeated until all the parameter matrices are initialized.
After the initialization of the parameters is completed, the parameters are updated by minimizing the following reconstruction error loss function by adopting a random gradient descent method:
wherein W ═ { W ═ W1,W2,W3,W4}。
Step 22, prediction of the weights of the training data. Prediction problem of training data weight, i.e. solving optimal weight vector s ═ s of training data D1,...,sc]TCan be optimized by solving a quadratic optimization problem mins Is obtained in whichRepresenting a combination of C DNN output images.
And step 23, training a weight prediction module. After the training of DNN (SSDA or DCNN) and the weight prediction of the training data are completed, a new training sample set (Φ, s) can be generated, where Φ ═ f1,...,fC]Is a feature vector obtained by connecting the output values of all SSDA hidden layer nodes (or DCNN feature extraction layers), fiC is a hidden layer (SSDA) of the ith column (DNN) or a feature extraction layer (DCNN) output value vector. The weight prediction module can be trained by the training sample. The weight prediction module is a radial basis function RBF neural network, and parameters of the RBF can be trained by adopting a standard neural network parameter optimization method by taking the characteristic vector phi as input and s as output.
the method comprises the steps of normalizing a target face image, wherein three target face image normalization schemes can be provided based on the use and combination of neural networks with different depths, AMC-SSDA, the basic idea of face correction based on adaptive multi-column stacked sparse denoising self-encoders (AMC-SSDA) is to consider a face image affected by illumination change, expression change, posture change, shielding and the like as a noise polluted image, use a front face image under a standard condition as a clean image, train a plurality of stacked sparse denoising self-encoders (SSDA) to denoise, use each SSDA corresponding to one influence factor or a combination of different influence factors to remove the influence of one or more factors, use all linear combinations of the SSDA to form the AMC-SSDA to perform joint denoising, use illumination change, expression change, posture change, shielding and the like of the face to remove various noises, realize the face correction, use the SSDA in the AMC-SSDA to perform joint denoising, use a DCNN-SSDA combined reconstruction method to reconstruct a multi-column linear combination of the face image under a DCNN-SSDA and a standard illumination change, use a DCNN-SSDA to reconstruct a multi-SSN image, use a DCNN-SSDA to reconstruct a normal illumination image, and a multi-SSDA convolution image, and a plurality of images under a standard illumination change, and a standard illumination change, wherein the illumination change of the SSDA is formed by a DCNN.
The deep neural network selected by the model in the embodiment of the present invention is SSDA or DCNN, but in practical cases, the model is not limited to SSDA or DCNN.
Finally, it is noted that the above-mentioned preferred embodiments illustrate rather than limit the invention, and that, although the invention has been described in detail with reference to the above-mentioned preferred embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the invention as defined by the appended claims.
Claims (1)
1. A face image normalization method based on a self-adaptive multi-column depth model is characterized by comprising the following steps: the method comprises the following steps:
s1: establishing a self-adaptive multi-column depth model; the self-adaptive multi-column depth model is formed by linearly combining a plurality of Deep Neural Networks (DNNs), each DNN is used for removing the influence of certain specific type factors, including one or more of illumination change, expression change, posture change and shielding factors, and parameters of the DNN are trained by adopting data influenced by the factors; inputting an uncorrected image, performing correction of certain factors on the uncorrected image by each DNN and outputting a restored image, predicting the optimal combination weight by a weight prediction module according to the output of each DNN hidden layer node, wherein the weight is determined by the input image in a self-adaptive manner, and the final restored image is the weighted average of each DNN output image; wherein, the deep neural network DNN selects a Stacked Sparse Denoising Autoencoder (SSDA), a Deep Convolutional Neural Network (DCNN), or a combination of the two; the SSDA takes the face image affected by various factors as a noisy image, removes various affected factors in the face image by a denoising method, and recovers the face with normal illumination, front, no shielding and natural expression; the DCNN firstly extracts the characteristics with invariance through a deep convolution network, and then carries out face reconstruction to directly reconstruct the face with normal illumination, front, no shielding and natural expression;
s2: training a self-adaptive multi-column depth model;
the training includes S21: training a deep neural network DNN; s22: prediction of training data weights; s23: training a weight prediction module;
s3: normalizing the target face image;
the training of the deep neural network in S21 includes S211: training a stack-type sparse denoising self-encoder SSDA; s212: training a Deep Convolutional Neural Network (DCNN);
the training of the SSDA comprises the following steps:
s2111: the SSDA is composed of a plurality of Denoising Autoencoders (DA); the DA is a neural network with two layers, an input layer, a hidden layer and an output layer, and consists of 1 encoder and 1 decoder; an SSDA formed by K DAs is 1 multilayer neural network and comprises an input layer, 2K-1 hidden layers and an output layer; let the kth DA comprise an encoderAnd a decoderThen the SSDA is divided into coded portionsToAnd a decoding part fromToThe SSDA consists of K denoising autocoders DA, training of the SSDA is training of each DA, and the DA can be trained by optimizing a sparse regularization reconstruction loss function as follows:
wherein X ═ { X ═ X1,x2,...xNDenotes a set of input images affected by various factors, Y ═ Y1,y2,...yNRepresenting a target image group treth set corresponding to the input image set; given N training data D { (x)1,y1),...,(xN,yN)}；Is the output of the DA; Θ ═ WS,b,WS'and b' are parameters to be optimized, lambda, β and rho are hyper-parameters, J is the number of hidden nodes,is the average output value vector of hidden layer nodes, sparse induction termIs the target activation value rho and the average activation value of the jth hidden nodeBy selecting a smaller target activation value p such thatThe component of (a) is as small as possible;
s2112: after the first DA is trained, the hidden layer outputs of the clean image and the polluted image are respectively used as the clean image and the polluted image to train a second DA, and the process is repeated until the training of K DAs is completed;
s2113: the parameters of the entire network are fine tuned using standard back propagation algorithms by minimizing the loss function:
whereinA parameter representing layer I in SSDA;
s2121: the DCNN comprises a feature extraction layer and a reconstruction layer, wherein the feature extraction layer comprises 3 local connection layers and 2 posing layers and is used for extracting invariant features; wherein the local connection layer passes through the sparse weight matrix Filtering the input image or the feature map output by the previous layer to generate a feature map or a new feature map, and downsampling the feature map featuremap output by the local connection layer through matrixes V1 and V2 by the pooling layer to reduce the number of parameters needing to be learned and simultaneously retain more robust features; the reconstruction layer is a full connection layer and passes through a weight matrixWill liftThe obtained features are converted into a front face image under a standard condition; the initialization of the parameters is carried out in a layer-by-layer mode:
wherein, X1、X2、X3Outputs of three local connection layers respectively; o, P and Q are all a fixed binary matrix consisting ofAndadding the pixel values of the same position in the generated feature map for makingAndthe same size as Y;
s2122: updating parameters by minimizing the following reconstruction error loss function by adopting a random gradient descent method:
wherein,the prediction of the training data weight of S22 is to solve the optimal weight vector S ═ S of the training data D1,...,sc]TBy solving a quadratic optimization problemIs obtained in whichA combination of output images representing C DNNs; the weight prediction module in the training of the S23 weight prediction module is a Radial Basis Function (RBF) neural network, and the parameters of the RBF are trained by adopting a neural network parameter optimization method by taking the characteristic vector phi as input and S as output; after step S22, a new training sample set (Φ, S) is generated, where Φ ═ f1,...,fC]The feature vectors are obtained by connecting output values of SSDA hidden layer nodes or DCNN feature extraction layers;
the S3 target face image normalization method comprises AMC-SSDA, AMC-DCNN and AMC-SSDA + DCNN, AMC-SSDA is based on the basic idea of face correction of an adaptive multi-column stacked sparse denoising self-encoder (AMC-SSDA) that a face image affected by illumination change, expression change, posture change and shielding factors is regarded as a noise pollution image, a front face image under a standard condition is used as a clean image, a plurality of stacked sparse denoising self-encoders (SSDA) are trained to denoise, each SSDA corresponds to one or a combination of different influencing factors, the influence of one or more factors is removed, then the AMC-SSDA is formed by linear combination of all SSDAs to form the AMC-SSDA to perform combined denoising, illumination change, expression change, posture change and shielding removal of various noises are completed, face correction is realized, the SSDA in the AMC-SSDA is replaced by the SSDA in the AMC-SSDA, namely, the AMC-SSDA is formed, different from the SSDA through correction, the DCNN is trained for normalization of the illumination change of the face image, the SSDA, the face image is reconstructed by a multi-SSDA combined reconstruction of the DCNN, the normal AMC-SSDA, the facial image is used for realizing the reconstruction of the facial image, the facial image reconstruction of the facial image under the facial image, the facial image reconstruction of the facial image, the facial reconstruction of facial image, the facial reconstruction of the facial image under the facial image, the facial image under the facial image, the facial image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410677837.XA CN104361328B (en) | 2014-11-21 | 2014-11-21 | A kind of facial image normalization method based on adaptive multiple row depth model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410677837.XA CN104361328B (en) | 2014-11-21 | 2014-11-21 | A kind of facial image normalization method based on adaptive multiple row depth model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104361328A CN104361328A (en) | 2015-02-18 |
| CN104361328B true CN104361328B (en) | 2018-11-02 |
Family
ID=52528586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410677837.XA Active CN104361328B (en) | 2014-11-21 | 2014-11-21 | A kind of facial image normalization method based on adaptive multiple row depth model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104361328B (en) |
Families Citing this family (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104778448B (en) * | 2015-03-24 | 2017-12-15 | 孙建德 | A kind of face identification method based on structure adaptive convolutional neural networks |
| CN104850735A (en) * | 2015-04-28 | 2015-08-19 | 浙江大学 | Activity recognition method based on stack own coding |
| US9633306B2 (en) * | 2015-05-07 | 2017-04-25 | Siemens Healthcare Gmbh | Method and system for approximating deep neural networks for anatomical object detection |
| CN105654028A (en) * | 2015-09-29 | 2016-06-08 | 厦门中控生物识别信息技术有限公司 | True and false face identification method and apparatus thereof |
| US9904874B2 (en) * | 2015-11-05 | 2018-02-27 | Microsoft Technology Licensing, Llc | Hardware-efficient deep convolutional neural networks |
| CN105426860B (en) * | 2015-12-01 | 2019-09-27 | 北京眼神智能科技有限公司 | The method and apparatus of recognition of face |
| US10460231B2 (en) * | 2015-12-29 | 2019-10-29 | Samsung Electronics Co., Ltd. | Method and apparatus of neural network based image signal processor |
| CN105657402B (en) * | 2016-01-18 | 2017-09-29 | 深圳市未来媒体技术研究院 | A kind of depth map restoration methods |
| US10325351B2 (en) * | 2016-03-11 | 2019-06-18 | Qualcomm Technologies, Inc. | Systems and methods for normalizing an image |
| CN106157307B (en) | 2016-06-27 | 2018-09-11 | 浙江工商大学 | A kind of monocular image depth estimation method based on multiple dimensioned CNN and continuous CRF |
| CN106529589A (en) * | 2016-11-03 | 2017-03-22 | 温州大学 | Visual object detection method employing de-noising stacked automatic encoder network |
| CN106780662B (en) * | 2016-11-16 | 2020-09-18 | 北京旷视科技有限公司 | Face image generation method, device and equipment |
| CN106780658B (en) | 2016-11-16 | 2021-03-09 | 北京旷视科技有限公司 | Face feature adding method, device and equipment |
| CN106599878A (en) * | 2016-12-28 | 2017-04-26 | 深圳市捷顺科技实业股份有限公司 | Face reconstruction correction method and device based on deep learning |
| US10424045B2 (en) * | 2017-06-21 | 2019-09-24 | International Business Machines Corporation | Machine learning model for automatic image registration quality assessment and correction |
| CN107680036A (en) * | 2017-08-15 | 2018-02-09 | 湖北工业大学 | The joint sparse Vector Parallel method for reconstructing of network is stacked based on convolution depth |
| CN107886062B (en) * | 2017-11-03 | 2019-05-10 | 北京达佳互联信息技术有限公司 | Image processing method, system and server |
| CN109934062A (en) * | 2017-12-18 | 2019-06-25 | 比亚迪股份有限公司 | Training method, face identification method, device and the equipment of eyeglasses removal model |
| CN108829683B (en) * | 2018-06-29 | 2022-06-10 | 北京百度网讯科技有限公司 | Hybrid label learning neural network model and training method and device thereof |
| CN111191655B (en) | 2018-11-14 | 2024-04-16 | 佳能株式会社 | Object identification method and device |
| CN109934116B (en) * | 2019-02-19 | 2020-11-24 | 华南理工大学 | Standard face generation method based on confrontation generation mechanism and attention generation mechanism |
| CN109919864A (en) * | 2019-02-20 | 2019-06-21 | 重庆邮电大学 | A kind of compression of images cognitive method based on sparse denoising autoencoder network |
| CN109872291B (en) * | 2019-02-21 | 2021-04-23 | 中国科学技术大学 | Regularization method and system for resisting convergent noise in ANN |
| CN110363099A (en) * | 2019-06-24 | 2019-10-22 | 昆明理工大学 | A kind of expression recognition method based on local parallel deep neural network |
| US11443137B2 (en) | 2019-07-31 | 2022-09-13 | Rohde & Schwarz Gmbh & Co. Kg | Method and apparatus for detecting signal features |
| CN113569598A (en) * | 2020-04-29 | 2021-10-29 | 华为技术有限公司 | Image processing method and image processing apparatus |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104112263A (en) * | 2014-06-28 | 2014-10-22 | 南京理工大学 | Method for fusing full-color image and multispectral image based on deep neural network |
-
2014
- 2014-11-21 CN CN201410677837.XA patent/CN104361328B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104112263A (en) * | 2014-06-28 | 2014-10-22 | 南京理工大学 | Method for fusing full-color image and multispectral image based on deep neural network |
Non-Patent Citations (4)
| Title |
|---|
| adaptive multi-column deep neural networks with application to robust image denoising;forest agostinelli et al;《advances in neural information processing systems 2013》;20140224;第1、3-4页 * |
| image denoising and inpainting with deep neural networks;junyuan xie et al;《advances in neural information processing systems 2012》;20130325;第1、3-6页 * |
| multi-column deep neural networks for image classification;dan ciregan et al;《2012 ieee conference on computer vision and pattern recognition》;20121231;第3642-3649页 * |
| 改进的深度卷积网络及在碎纸片拼接中的应用;段宝彬，韩立新;《计算机工程与应用》;20140501;第50卷(第9期);第177-179页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104361328A (en) | 2015-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104361328B (en) | A kind of facial image normalization method based on adaptive multiple row depth model | |
| Dong et al. | Deep spatial–spectral representation learning for hyperspectral image denoising | |
| Divakar et al. | Image denoising via CNNs: An adversarial approach | |
| Bai et al. | Deep learning methods for solving linear inverse problems: Research directions and paradigms | |
| US11645835B2 (en) | Hypercomplex deep learning methods, architectures, and apparatus for multimodal small, medium, and large-scale data representation, analysis, and applications | |
| Dong et al. | Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization | |
| Liew et al. | Gender classification: a convolutional neural network approach | |
| Gai et al. | New image denoising algorithm via improved deep convolutional neural network with perceptive loss | |
| Ahn et al. | Block-matching convolutional neural network for image denoising | |
| Zaied et al. | A novel approach for face recognition based on fast learning algorithm and wavelet network theory | |
| CN106548159A (en) | Reticulate pattern facial image recognition method and device based on full convolutional neural networks | |
| Fu et al. | Adaptive spatial-spectral dictionary learning for hyperspectral image denoising | |
| Liu et al. | The classification and denoising of image noise based on deep neural networks | |
| Singh et al. | ResDNN: deep residual learning for natural image denoising | |
| Zhao et al. | A novel deep learning algorithm for incomplete face recognition: Low-rank-recovery network | |
| CN110321777A (en) | A kind of face identification method based on the sparse denoising self-encoding encoder of stack convolution | |
| Routray et al. | An efficient image denoising method based on principal component analysis with learned patch groups | |
| CN112950505B (en) | Image processing method, system and medium based on generation countermeasure network | |
| Meng et al. | A novel gray image denoising method using convolutional neural network | |
| Yang et al. | Blind image quality assessment of natural distorted image based on generative adversarial networks | |
| Kang et al. | Stacked denoising autoencoders for face pose normalization | |
| Zhou et al. | Sparse representation with enhanced nonlocal self-similarity for image denoising | |
| CN117196963A (en) | Point cloud denoising method based on noise reduction self-encoder | |
| Ahn et al. | Block-matching convolutional neural network (BMCNN): improving CNN-based denoising by block-matched inputs | |
| Hu et al. | Patch-based low-rank minimization for image denoising |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20151222 Address after: 400714 No. 256 Fangzheng Road, Beibei District, Chongqing Applicant after: CHONGQING DELING TECHNOLOGY CO., LTD. Address before: 400714 Chongqing Road, Beibei District, No. 266 Applicant before: Chongqing Institute of Green and Intelligent Technology of the Chinese Academy of Sciences |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20160126 Address after: 401122 Chongqing hi tech park in northern New Mount Huangshan Central Avenue No. 5 mercury technology office building 6 Applicant after: CHONGQING ZHONGKE YUNCONG TECHNOLOGY CO., LTD. Address before: 400714 No. 256 Fangzheng Road, Beibei District, Chongqing Applicant before: CHONGQING DELING TECHNOLOGY CO., LTD. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 401120 5 stories, Block 106, West Jinkai Avenue, Yubei District, Chongqing Patentee after: Chongqing Zhongke Yuncong Technology Co., Ltd. Address before: 401122 Central Office Building, Mercury Science and Technology Building, No. 5 Huangshan Avenue, High-tech Park, North New District, Chongqing Patentee before: CHONGQING ZHONGKE YUNCONG TECHNOLOGY CO., LTD. |
|
| CP03 | Change of name, title or address |