CN111274915B - Deep local aggregation descriptor extraction method and system for finger vein image - Google Patents

Abstract

The invention discloses a deep local aggregation descriptor extraction method and a system for a finger vein image, wherein the method comprises the following steps: constructing a basic network module; constructing a VLAD coding module; setting K clustering center vectors as trainable parameters of the network; training the network in batches, wherein the training steps are as follows: preprocessing the finger vein image; the finger vein image obtains a multichannel characteristic diagram through a basic network module; combining the multi-channel feature map with a clustering center vector in a VLAD coding module to finish VLAD coding; mining the negative samples difficult to divide to obtain triples, calculating a loss function and reversely transmitting and updating a network weight coefficient; and extracting the local aggregation descriptor of the finger vein image to be detected by adopting a trained network. According to the method, the descriptors which are fixed in dimension and irrelevant to the distribution sequence of the original image blocks are obtained, the problem of matching failure between finger vein images with different sizes and finger vein images with the same category due to the difference of finger gestures is solved, and the depth local aggregation descriptors with better characterization force are obtained.

Description

A deep local aggregation descriptor extraction method and system for finger vein images

technical field

The invention relates to the technical field of finger vein feature extraction, in particular to a method and system for extracting deep local aggregation descriptors of finger vein images.

Background technique

Finger vein recognition technology is a new generation of biometric technology. Compared with traditional biometric technology, finger vein recognition has the advantages of non-contact collection, live detection, and low equipment cost. Finger vein recognition acquires finger images through infrared CCD cameras, and extracts finger vein-related features for identity authentication and identification. The collected finger vein images often have noise interference. How to extract robust features of finger vein images is a research focus in vein recognition technology. Traditional feature descriptors such as LBP (Local Binary Pattern) and LDC (Local Directional Code) The representation ability of such as is greatly affected by the image quality, and the feature map that retains the spatial information requires complex template matching for recognition.

In recent years, a variety of deep learning-based solutions have been proposed for finger vein recognition technology. For the finger vein verification problem, the pixel-level classification of finger vein images is based on the idea of image segmentation. This method is slow in classification and difficult to apply in practice. The scene is used, and the method based on the convolutional network uses the existing image classification model, and the model size is large. In addition, the extracted features are sensitive to finger poses, and complex preprocessing or template matching schemes are required for recognition of finger vein images with different finger poses. Therefore, it is more advantageous in practical applications to study lightweight convolutional network models and extract feature descriptors that are more robust to finger pose changes in finger vein images.

Contents of the invention

In order to overcome the defects and deficiencies existing in the prior art, the present invention provides a method and system for extracting depth local aggregated descriptors of finger vein images. The central vectors are concatenated to obtain descriptors with fixed dimensions and independent of the distribution order of the original image blocks, which solves the problem of The matching problem between finger vein images of different sizes and the matching failure problem caused by finger pose differences between finger vein images of the same category are solved. On this basis, the descriptor is further encoded by VLAD to obtain a more expressive depth The local aggregation descriptor has excellent performance in finger vein recognition and verification tasks. The size of the network model of the present invention is only 1.1M, which can better meet the demand for lightweight in engineering applications.

In order to achieve the above object, the present invention adopts the following technical solutions:

The present invention provides a method for extracting depth local aggregation descriptors of finger vein images, comprising the following steps:

Build a basic network module for extracting local features of finger vein images;

Construct a VLAD encoding module, which is used to perform VLAD encoding on the feature map obtained by the basic network module;

Set the K cluster center vectors as the trainable parameters of the network;

Input finger vein images in batches to train the network, the training steps include:

Preprocessing the finger vein image;

Finger vein image samples get multi-channel feature maps through the basic network module;

The multi-channel feature map is combined with the cluster center vector in the VLAD encoding module to complete the VLAD encoding;

Mining difficult negative samples to obtain triplets, calculating the loss function and backpropagating to update the network weight coefficients until the end of the iterative training;

The trained network is used to extract the local aggregation descriptor of the finger vein image to be tested.

As a preferred technical solution, said preprocessing the finger vein image, the specific steps include:

Extraction of the region of interest: extract the region of interest of the finger vein training image, and complete the finger tilt correction through affine transformation;

Standardize the region of interest to obtain the final finger vein training sample image;

Adjust the size of finger vein training sample images according to the receptive field and the original scale of the image.

As a preferred technical solution, the extraction of the region of interest, the specific steps include:

Two Sobel operators Mask _u and Mask _d are used to detect the upper and lower edges of the finger vein training image respectively, and the midline of the finger is fitted by the method of linear regression, and the angle formed by the midline and the horizontal direction is calculated, and the finger vein is adjusted by affine transformation. The vein training image is rotated, and the tilt correction is completed. Finally, the region of interest is obtained by intercepting the circumscribed rectangle according to the edge of the finger. The two Sobel operators are expressed as:

Among them, Mask _u and Mask _d represent two Sobel operators extended to 3×9.

As a preferred technical solution, the multi-channel feature map is combined with the clustering center vector in the VLAD encoding module to complete the VLAD encoding, and the specific steps include:

The multi-channel feature map is converted into w _out ×h _out local descriptors { _xi ,i=1,2,...,w _out ×h _out } that describe the original image with dimension C _out , and input to the VLAD encoding module To encode, calculate the matrix V of K rows and C _out columns, and the elements at (k, j) positions are:

in, and represent the jth component of the i-th descriptor x _i and the j-th component of the k-th cluster center c _k respectively, a _k (xi ₎ represents the probability that the descriptor x _i belongs to the k-th cluster, c _k' represents other cluster center vectors except the kth cluster center vector;

Flatten the matrix V into a one-dimensional vector and perform _L2 normalization to obtain a local aggregation descriptor with a length of K×C _out .

As a preferred technical solution, the mining of indistinguishable negative samples to obtain triplets, the specific steps include:

Select two local aggregation descriptors and and The samples belong to the same category, Constitute a pair of positive samples;

For each positive sample pair Pick a negative sample from another class form a triplet negative sample make Min, where marg represents the set threshold parameter.

As a preferred technical solution, the calculation of the loss function and backpropagation updates the network weight coefficients, and the calculation formula for calculating the loss function for triples of the same batch is:

Among them, m represents the category of the same batch of images, and n represents the number of samples in each category.

The present invention also provides a deep local aggregation descriptor extraction system for finger vein images, including: a basic network module construction unit, a VLAD encoding module construction unit, a cluster center vector construction unit, a training unit and an extraction unit;

The basic network module construction unit is used to build a basic network module, and the basic network module is used to extract local features of finger vein images;

The VLAD encoding module construction unit is used to construct a VLAD encoding module, and the VLAD encoding module is used to perform VLAD encoding on the feature map obtained by the basic network module;

The cluster center vector construction unit is used to set K cluster center vectors as network trainable parameters;

The training unit is used to input finger vein images to train the network in batches, and the training unit includes: an image preprocessing module, a multi-channel feature map acquisition module, a combination encoding module, a triplet construction module and an iterative update module;

The image preprocessing module is used to preprocess the finger vein image;

The multi-channel feature map acquisition module is used to obtain the multi-channel feature map through the basic network module through the finger vein image sample;

The combination encoding module is used to combine the multi-channel feature map in the VLAD encoding module to complete the VLAD encoding in combination with the clustering center vector;

The triplet building block is used to mine difficult negative samples to obtain triplets;

The iterative update module is used to calculate the loss function and backpropagate to update the network weight coefficients until the end of the iterative training;

The extraction unit is used to extract the local aggregation descriptor of the finger vein image to be tested by using the trained network.

As a preferred technical solution, the structure of the basic network module adopts 6 concatenated convolution modules, respectively represented as Conv_i, i={1,2,3,4,5,6}, each convolution module contains 3 ×3 Conv2d layer, BN layer and Relu activation layer, the number of convolution kernels of each convolution module is 32, 32, 64, 64, 128, 128, the padding of all convolution layers is set to 1, Conv_3 and The convolution step of Conv_5 is set to 2, and Conv_1, Conv_2, Conv_4 and Conv_6 are all set to 1.

As a preferred technical solution, all convolutional layers are initialized with an orthogonal matrix, the bias is fixed at 0, and the weight and bias of the BN layer are fixed at 1 and 0, respectively.

As a preferred technical solution, the VLAD encoding module sets a 1×1 convolutional layer on the network structure.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

(1) The present invention obtains descriptors through CNN network end-to-end learning. The size of the network model is only 1.1M. The extracted descriptors can be further used for tasks such as finger vein verification and recognition, which are flexible and widely used.

(2) For a finger vein image of any size, the present invention obtains K cluster centers through network automatic learning, and connects the cluster center vectors in series to form a descriptor vector representing the features of the finger vein image, thereby solving the problem of different sizes of finger vein images. Since the descriptor vector is composed of the cluster center vector rather than the feature vector of the local block of the image, it can still be matched correctly even if there is a difference in the spatial position of the two finger vein images, thus solving the problem of the same category index. Matching failure problem caused by difference in finger pose between vein images.

(3) The present invention performs VLAD encoding on the features of the finger vein image, and only introduces an additional 1×1 convolution parameter, fully utilizes the information of the feature map, and obtains a more expressive finger vein image descriptor .

(4) The present invention constructs triplet samples during network training, reduces the demand on the number of finger vein training images, and ensures that the number of positive and negative samples used for training is equal; when constructing samples, it adopts a difficult-to-separate negative sample mining strategy, Speed up the network convergence; the triplet loss function is used to train the network, so that the network focuses on learning the differences between different finger vein images rather than label information, and improves the generalization performance of the method.

Description of drawings

Fig. 1 is the image example diagram of the finger vein data set of the present embodiment;

Fig. 2 is the schematic diagram of the division mode of the vein data set of the present embodiment;

Fig. 3 is the schematic flow chart of batch training of the network model of the present embodiment;

FIG. 4 is an example diagram of training and test images obtained by extracting a region of interest in this embodiment;

Fig. 5 is a schematic flow chart of the network model test of the present embodiment;

FIG. 6 is a schematic structural diagram of a deep local aggregation descriptor extraction system for finger vein images in this embodiment;

FIG. 7 is a schematic structural diagram of the basic network module of this embodiment;

FIG. 8 is a schematic structural diagram of a VLAD encoding module in this embodiment.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Example

In this example, training and testing are performed on three finger vein data sets of SDUMLA, FV-USM, and MMCBNU_6000. The SDUMLA data set comes from Shandong University, and the finger vein images come from 636 fingers of 106 subjects. Six gray-scale BMP images were collected for each ring finger, and the resolution of the images was 320×240; the FV-USM dataset was from the University of Malaysia, which consisted of vein images of the left and right index fingers and middle fingers of 123 subjects, and the images of the database came from two There are 12 images for each finger in different experiments; the MMCBNU_6000 dataset comes from Chonbuk National University in South Korea, which consists of finger vein images from 100 volunteers. Each finger image is collected 10 times, with a total of 6000 images.

As shown in Figure 1, the image example of the above-mentioned disclosed finger vein data set is given in the figure, as shown in Figure 2, the finger images of 100 identities of the MMCBNU_6000 finger vein data set are divided into 600 according to the finger category in this embodiment Classes, each category contains 10 sample images, and images of 300 categories are randomly selected as the training set, then the training set has a total of 3000 samples, and the rest are used as the test set. In this embodiment, the test set is further divided into registered templates Library and sample set to be tested;

This embodiment is mainly implemented based on the deep learning framework Pytorch. The graphics card used in the experiment is GTX1080Ti, and the finger vein descriptor extracted from the test image is used for identification and verification tasks.

This embodiment provides a method for extracting a deep local aggregation descriptor of a finger vein image, comprising the following steps:

Build a basic network module for extracting local features of finger vein images;

Construct a VLAD encoding module, which is used to perform VLAD encoding on the feature map obtained by the basic network module;

Set K clustering center vectors as network trainable parameters: In this embodiment, K clustering center vectors {c _k ,k=1,2,...,K} whose dimension is C _out are set and uniformly distributed randomly Initialization, c _k is determined through network learning. For finger vein images of any size, this embodiment obtains K cluster centers through network automatic learning, and the cluster center vectors are concatenated to form a descriptor vector representing finger vein image features, whose length It is fixed to 128×K, thus solving the matching problem between finger vein images of different sizes. In particular, since the descriptor vector is composed of the cluster center vector rather than the feature vector of the local block of the image, it can still be correctly matched when there is a difference in the spatial position of the two finger vein images, thereby solving the problem of the same category of finger vein images. The problem of matching failures caused by differences in finger postures between . ~15, K is preferably set to 10 in this embodiment.

As shown in Figure 3, input finger vein images to train the network in batches, and the training steps include:

The finger vein image is preprocessed, and the region of interest used for training is obtained by segmenting the finger area from the background and intercepting the circumscribed rectangular area of the finger edge, thus removing the background noise while retaining as much as possible of the original image. information;

In this embodiment, the specific steps of preprocessing the finger vein image include:

Extraction of the region of interest: use two Sobel operators Mask _u and Mask _d to detect the upper and lower edges of the finger vein training image respectively, fit the midline of the finger through linear regression, and calculate the angle formed by the midline and the horizontal direction, The finger vein training image is rotated by affine transformation to complete the tilt correction, and finally the circumscribed rectangle of the entire finger is intercepted according to the outermost edge point of the finger to obtain the region of interest, as shown in Figure 4, training and testing are obtained according to the region of interest image;

In this embodiment, the two Sobel operators are represented as:

Among them, Mask _u and Mask _d represent two Sobel operators extended to 3×9;

Adjust the size of the finger vein training image according to the receptive field and the original ratio of the image, and calculate the average aspect ratio of all training images to be 2. According to the structural parameters of the basic network module, the receptive field of each feature point in the output feature map can be calculated as For a local area of 23×23, the height of the input image is adjusted to h=64, and the width is adjusted to w=128. At this time, the height of the receptive field of each feature point in the feature map is about 1/3 of the height of the input image. Can reflect richer finger vein information;

For each training image, the standardization process of subtracting the mean value and dividing the variance is performed to reduce the influence of uneven illumination and obtain the final finger vein training sample image;

Build a training batch sampler: load training images in batches, randomly select m categories in each batch, and load n samples in each category, that is, a total of m×n samples. In this embodiment, m=16, n = 6;

Initialize the trainable parameters in the network structure, initialize the parameters of the basic network module, where the weight of the convolutional layer is initialized with an orthogonal matrix, the bias is fixed to 0, and the weight and bias of the BN layer are fixed to 1 and 0; the initialization class Center vector, the number of descriptor clusters K is set to 10, the descriptor cluster centers c _k ,k=1,2,…,K, randomly initialized with uniform distribution; the trainable parameters of the VLAD coding module are initialized, and 1× 1 The parameters of the convolution are initialized according to the cluster center, where the weight of the convolution kernel is initialized to w _k =2c _k , and the bias is initialized to b _k =-‖c _k ‖ ² ;

Set the hyperparameter marg of the triplet loss to 1;

Set the number of model iterations to 200, the learning rate is fixed at 0.01, the optimization method is Adam (Adaptive moment estimation, adaptive moment estimation), initialize the minimum verification loss and the model saving path, and use it to save the model with the smallest verification loss later, set the initial model The number of iterations and the initial sample batch are 0, and the setting of the batch sampler determines that the batch size of the algorithm training is 16×6=96;

Increase the number of model training iterations by 1, continue the training of the model, increase the batch of training samples by 1, start loading a batch of samples or continue to load the next batch of samples;

Load 96 preprocessed training images from the training set according to the settings of the batch sampler;

After the finger vein image training sample image passes through the 6-layer 3×3 convolution module of the basic network module, because there are two layers of convolution with a step size of 2, the effect of pooling is realized, and finally the 128-channel size is 16× 32-pixel feature map;

The multi-channel feature map is combined with the cluster center vector in the VLAD encoding module to complete the VLAD encoding, and K cluster center vectors {c _k ,k=1,2,…,K} whose dimension is C _out are set and uniformed The distribution is randomly initialized, and c _k is determined through network learning;

The specific steps of VLAD encoding include:

Transform the C _out channel feature map with resolution w _out ×h _out into w _out ×h _out local descriptors describing the original image with dimension C _out { _xi ,i=1,2,...,w _out × h _out }, this embodiment is specifically expressed as: 16×32 128-dimensional local descriptors, represented by { _xi ,i=1,2,...,512}, and a 1280-dimensional local aggregation is obtained after passing through the VLAD coding module descriptor;

And input the VLAD encoding module for encoding, that is, calculate the matrix V of K rows and C _out columns, and the elements at (k, j) positions are:

in, and represent the jth component of the i-th descriptor x _i and the j-th component of the k-th cluster center c _k respectively, a _k (xi ₎ represents the probability that the descriptor x _i belongs to the k-th cluster, c _k' represents other cluster center vectors except the kth cluster center vector;

Flatten the matrix V into a one-dimensional vector and perform _L2 normalization to obtain a local aggregation descriptor with a length of K×C _out ;

The same batch of training images is processed by the basic network module and the VLAD encoding module, and a total of m×n local aggregation descriptors are obtained in Indicates the local aggregation descriptor of the i-th training sample in the batch. A batch of 96 training samples in this embodiment obtains 96 descriptors through the network to form a matrix in Represents the descriptor obtained after each training image is processed by the basic network module and the VLAD encoding module, and the element of the (i,j) position of the Euclidean distance matrix between two descriptors is obtained through matrix calculation The size of the matrix is 96×96, and the elements of the diagonal are 0;

Mining difficult negative samples to obtain triplets, select two local aggregation descriptors and and The samples belong to the same category, Constitute a pair of positive samples;

For each positive sample pair Pick a negative sample from another class form a triplet negative sample make Minimum, where marg represents the set threshold parameter. In this embodiment, the threshold marg is set to 1 when calculating the distance difference between similar descriptors and heterogeneous descriptors, which can better distinguish similar and heterogeneous descriptors;

Calculate the loss function and backpropagate to update the network weight coefficients to determine whether a training has been completed for all samples. If a training is completed, enter the verification loss step, otherwise return to continue training;

Judging whether the current verification loss is less than the minimum loss, if so, save the model or update the saved model, and update the value of the minimum loss, otherwise enter to judge whether to complete the set number of iterations;

Determine whether 200 iterations have been completed, if so, end the training, otherwise go to the model training iteration step, that is, increase the number of model training iterations by 1, continue the model training, increase the training sample batch by 1, and start loading a batch of samples or Continue loading the next batch of samples;

In this embodiment, the calculation formula for calculating the loss function for triplets in the same batch is:

Among them, m represents the category of the same batch of images, and n represents the number of samples in each category; this embodiment constructs triplet samples during network training, reducing the demand on the number of finger vein training images and ensuring that they are used for training The number of positive and negative samples is equal; when constructing samples, the indistinguishable negative sample mining strategy is used to speed up network convergence; the triple loss function is used to train the network, so that the network focuses on learning the differences between different finger vein images rather than label information, improving generalization performance of the method.

As shown in Figure 5, the trained network is used to extract the local aggregation descriptor of the finger vein image to be tested. The network structure of the test phase is the same as that of the training phase. The specific steps are as follows:

The finger vein images to be tested are subjected to the same image preprocessing steps as in network training, including region of interest extraction and standardization, and size adjustment. After image preprocessing, all images in the test set are resized to 64×128, and the preprocessed Input the finger vein image into the trained network to obtain its deep local aggregation descriptor, which can be further used for finger vein recognition or verification.

In this embodiment, finger vein recognition and verification tasks are tested respectively;

For the finger vein recognition task:

Input all images of the registered template library into the network to obtain feature descriptors;

For each image in the sample set to be tested, the descriptor is obtained through the network, and the Euclidean distance between the descriptor and the feature descriptors of all registered templates is calculated;

Sort the Euclidean distance, and identify the current sample to be tested as the registered template with the smallest Euclidean distance;

For the finger vein verification task:

By using the registered template library for testing, select an appropriate binary classification threshold of 1;

For each sample to be tested and other samples of the same category from the test set to form a positive sample pair, randomly select the same amount of heterogeneous samples to form a negative sample pair, forming a total of 300×5×5×2=15000 pairs of positive and negative samples ;

Get the descriptors by passing the sample pairs one by one through the network, and calculate the Euclidean distance between any pair of descriptors;

If the Euclidean distance between the two descriptors is lower than 1, it indicates that the two samples belong to the same category, and the verification is successful, otherwise the verification fails;

Finally, the test results are saved. In this embodiment, deep local aggregation descriptors are used, and no special processing is required for finger vein images of different finger poses. Matching and recognition can be realized directly by calculating simple measurement methods such as Euclidean distance and cosine similarity.

In this embodiment, the testing methods for the other two public databases are basically the same as the above steps. As shown in Table 1 below, the evaluation index involved in the experimental results of the 1:1 verification of the method of this embodiment is EER (EqualError Rate), that is, the FAR when FAR (False Accept Rate) is equal to FRR (False Reject Rate) In the experiment, the FAR value when |FAR-FRR|<0.0001 is taken as EER.

Table 1 Test finger vein 1:1 verification EER result table

the SDUMLA FV-USM MMCBNU_6000 EER 0.95% 0.38% 0.10%

As shown in Table 2 below, this embodiment refers to the experimental results of vein 1:N recognition, and the evaluation index involved is IR(k):

Among them, B represents the set of all samples to be tested, b is a certain sample to be tested, rank(b) is the ranking of the similarity between the samples to be tested and the samples of the same category in the registered template library, U _B represents the rank of the sample set to be tested quantity. IR(1) represents the ratio of the sample whose similarity ranks first to the samples of the same category in the template library in the test sample to all the test samples.

Table 2 Test finger vein 1:N recognition IR(k) result table

the SDUMLA FV-USM MMCBNU_6000 IR(k) 99.50% 99.87% 100%

As shown in Table 3 below, the parameters of the model in this embodiment and the correlation time of finger vein recognition tested on the CPU.

Table 3 Model size and time consumption table

model size feature extraction time Euclidean distance calculation time 1.1M 0.0144s 0.00037s

It can be seen from the above table 1-table 3 that the network proposed in this embodiment is effective in both finger vein recognition and verification tasks, and the network model is only 1.1M, and the feature extraction time is short. In this embodiment When the description sub-dimension is 1280, the calculation of the similarity takes a short time; this embodiment performs VLAD encoding on the features of the finger vein image, and makes full use of the features when only an additional 1×1 convolution parameter is introduced. Figure information, get a more expressive finger vein image descriptor.

As shown in Figure 6, this embodiment also provides a system for extracting deep local aggregated descriptors of finger vein images, including: a basic network module construction unit, a VLAD encoding module construction unit, a cluster center vector construction unit, a training unit and an extraction unit. unit;

In this embodiment, the basic network module construction unit is used to construct the basic network module, and the basic network module is used to extract the local features of the finger vein image; the VLAD encoding module construction unit is used to construct the VLAD encoding module, and the VLAD encoding module is used for Perform VLAD encoding on the feature map obtained by the basic network module; the cluster center vector construction unit is used to set K cluster center vectors as network trainable parameters; the training unit is used to input finger vein images to train the network in batches, extract The unit is used to extract the local aggregation descriptor of the finger vein image to be tested by using the trained network;

As shown in Figure 7, the structure of the basic network module adopts 6 convolution modules connected in series, denoted as Conv_i, i={1,2,3,4,5,6}, each convolution module contains 3×3 The Conv2d layer, BN layer, and Relu activation layer of the Conv2d layer, the number of convolution kernels of each convolution module are 32, 32, 64, 64, 128, and 128, and the padding of all convolutional layers is set to 1. Conv_3 and Conv_5 The convolution step size is set to 2, Conv_1, Conv_2, Conv_4 and Conv_6 are all set to 1, all convolutional layers are initialized with an orthogonal matrix, the bias is fixed at 0, and the weight and bias of the BN layer are not updated, the BN layer The weight and bias of are fixed at 1 and 0 respectively, which can reduce model training parameters and have little effect on the results;

As shown in Figure 8, the VLAD encoding module sets a 1×1 convolutional layer on the network structure, where the weight w _k =2c _k and the bias b _k =-‖c _k ‖ ² are used to represent the simplified a _k ( _xi ):

In this embodiment, the training unit includes: an image preprocessing module, a multi-channel feature map acquisition module, a combined encoding module, a triplet construction module and an iterative update module;

In this embodiment, the image preprocessing module is used to preprocess the finger vein image; the multi-channel feature map acquisition module is used to pass the finger vein image sample through the basic network module to obtain a multi-channel feature map; the combined coding module is used to combine the multi-channel feature map The channel feature map is combined with the clustering center vector in the VLAD encoding module to complete the VLAD encoding; the triplet construction module is used to mine difficult negative samples to obtain triplets; the iterative update module is used to calculate the loss function and backpropagate to update the network weight coefficients , until the end of the iterative training.

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (9)

1. a depth local aggregation descriptor extraction method of finger vein image, is characterized in that, comprises the following steps: Build a basic network module for extracting local features of finger vein images; Construct a VLAD encoding module, which is used to perform VLAD encoding on the feature map obtained by the basic network module; The multi-channel feature map is combined with the cluster center vector in the VLAD encoding module to complete the VLAD encoding. The specific steps include: The multi-channel feature map is converted into w _out × h _out local descriptors { _xi , i=1, 2, ..., w _out × h _out } that describe the original image with dimension C _out , and input into VLAD The encoding module encodes and calculates the matrix V of K rows and C _out columns, and the elements at (k, j) positions are: in, and represent the jth component of the i-th descriptor x _i and the j-th component of the k-th cluster center c _k respectively, a _k (xi ₎ represents the probability that the descriptor x _i belongs to the k-th cluster, c _k' represents other cluster center vectors except the kth cluster center vector; Flatten the matrix V into a one-dimensional vector and perform _L2 normalization to obtain a local aggregation descriptor with a length of K×C _out ; Set the K cluster center vectors as the trainable parameters of the network; Input finger vein images in batches to train the network, the training steps include: Preprocessing the finger vein image; The preprocessed finger vein image obtains a multi-channel feature map through the basic network module; The multi-channel feature map is combined with the cluster center vector in the VLAD encoding module to complete the VLAD encoding; Mining difficult negative samples to obtain triplets, calculating the loss function and backpropagating to update the network weight coefficients until the end of the iterative training; The trained network is used to extract the local aggregation descriptor of the finger vein image to be tested. 2. the depth local aggregation descriptor extracting method of finger vein image according to claim 1, is characterized in that, described finger vein image is preprocessed, and concrete steps comprise: Extraction of the region of interest: extract the region of interest of the finger vein training image, and complete the finger tilt correction through affine transformation; Standardize the region of interest to obtain the final finger vein training sample image; Adjust the size of finger vein training sample images according to the receptive field and the original scale of the image. 3. the depth local aggregation descriptor extraction method of finger vein image according to claim 2, is characterized in that, the extraction of described region of interest, concrete steps comprise: Two Sobel operators Mask _u and Mask _d are used to detect the upper and lower edges of the finger vein training image respectively, and the midline of the finger is fitted by the method of linear regression, and the angle formed by the midline and the horizontal direction is calculated, and the finger vein is adjusted by affine transformation. The vein training image is rotated, and the tilt correction is completed. Finally, the region of interest is obtained by intercepting the circumscribed rectangle according to the edge of the finger. The two Sobel operators are expressed as: Among them, Mask _u and Mask _d represent two Sobel operators extended to 3×9. 4. the depth local aggregation descriptor extraction method of finger vein image according to claim 1, is characterized in that, described digging indistinguishable negative sample obtains triplet, and concrete steps comprise: Select two local aggregation descriptors and and The samples belong to the same category, Constitute a pair of positive samples; For each positive sample pair Pick a negative sample from another class form a triplet negative sample make Minimum, where marg represents the set threshold parameter. 5. The method for extracting depth local aggregation descriptors of finger vein images according to claim 4, wherein the calculation loss function is backpropagated to update the network weight coefficients, and the loss function is calculated for triplets of the same batch The calculation formula is: Among them, m represents the category of the same batch of images, and n represents the number of samples in each category. 6. A system for extracting depth local aggregation descriptors of finger vein images, comprising: a basic network module construction unit, a VLAD coding module construction unit, a cluster center vector construction unit, a training unit and an extraction unit; The basic network module construction unit is used to build a basic network module, and the basic network module is used to extract local features of finger vein images; The VLAD encoding module construction unit is used to construct a VLAD encoding module, and the VLAD encoding module is used to perform VLAD encoding on the feature map obtained by the basic network module; The multi-channel feature map is combined with the cluster center vector in the VLAD encoding module to complete the VLAD encoding. The specific steps include: The multi-channel feature map is converted into w _out × h _out local descriptors { _xi , i=1, 2, ..., w _out × h _out } that describe the original image with dimension C _out , and input into VLAD The encoding module encodes and calculates the matrix V of K rows and C _out columns, and the elements at (k, j) positions are: in, and represent the jth component of the i-th descriptor x _i and the j-th component of the k-th cluster center c _k respectively, a _k (xi ₎ represents the probability that the descriptor x _i belongs to the k-th cluster, c _k' represents other cluster center vectors except the kth cluster center vector; Flatten the matrix V into a one-dimensional vector and perform _L2 normalization to obtain a local aggregation descriptor with a length of K×C _out ; The cluster center vector construction unit is used to set K cluster center vectors as network trainable parameters; The training unit is used to input finger vein images to train the network in batches, and the training unit includes: an image preprocessing module, a multi-channel feature map acquisition module, a combination encoding module, a triplet construction module and an iterative update module; The image preprocessing module is used to preprocess the finger vein image; The multi-channel feature map acquisition module is used to obtain the multi-channel feature map through the basic network module through the finger vein image sample; The combination encoding module is used to combine the multi-channel feature map in the VLAD encoding module to complete the VLAD encoding in combination with the clustering center vector; The triplet building block is used to mine difficult negative samples to obtain triplets; The iterative update module is used to calculate the loss function and backpropagate to update the network weight coefficients until the end of the iterative training; The extraction unit is used to extract the local aggregation descriptor of the finger vein image to be tested by using the trained network. 7. the deep local aggregation descriptor extraction system of finger vein image according to claim 6, is characterized in that, the structure of described basic network module adopts 6 convolution modules connected in series, is expressed as Conv_i respectively, i={1 , 2, 3, 4, 5, 6}, each convolution module contains 3×3 Conv2d layer, BN layer and Relu activation layer, and the number of convolution kernels of each convolution module is 32, 32, 64, 64, 128, 128, the padding of all convolutional layers is set to 1, the convolution stride of Conv_3 and Conv_5 is set to 2, and Conv_1, Conv_2, Conv_4 and Conv_6 are all set to 1. 8. the depth local aggregation descriptor extraction system of finger vein image according to claim 7, is characterized in that, all convolution layers adopt orthogonal matrix initialization, bias is fixed to 0, and the weight of BN layer and bias are respectively fixed for 1 and 0. 9. The deep local aggregation descriptor extraction system of finger vein images according to claim 6 or 7, wherein the VLAD encoding module is provided with a 1×1 convolutional layer on the network structure.

Images