CN109447153A - Divergence-excitation self-encoding encoder and its classification method for lack of balance data classification - Google Patents

Abstract

The invention discloses a divergence-excited self-encoder used for unbalanced data classification and a classification method thereof, which comprises sequentially connecting a data input layer, a coding layer unit, a bottleneck layer, a decoding layer unit and a data output layer; The layer is connected to the data output layer; the loss function in the bottleneck layer is used to calculate the divergence loss of the data input to the bottleneck layer, which is used to explicitly motivate and distinguish samples with different labels; the loss function in the data output layer unit is used to calculate The reconstruction loss for the output data of the decoding layer unit. The unbalanced data classification method provided by the present invention is superior to the method based on convolutional neural network in terms of F1 value and convergence speed; secondly, the autoencoder needs fewer parameters to be manually adjusted, unlike the loss-sensitive volume The hyperparameters need to be optimized as in the neural network; finally, when the number of minority samples is small, the autoencoder model proposed in the present invention performs well.

Description

Divergence-excited autoencoders for classification of unbalanced data and their classification methods

technical field

The invention belongs to the technical field of unbalanced data classification and processing, and in particular relates to a divergence-excitation autoencoder used for unbalanced data classification and a classification method thereof.

Background technique

Unbalanced data exists in large numbers in actual data classification problems, which means that the sample size of each category in the classification varies greatly. Commonly used classification algorithms are usually based on the assumption that the data of different categories are roughly evenly distributed. However, in practice, there are often a large number of Extremely skewed data, i.e. samples in one class occupy the majority of the dataset. Therefore, when encountering such data, the performance of the algorithm usually degrades significantly. For many years, reducing the problem caused by the imbalanced data has been a challenge. Most of the previous studies have proposed from the perspective of data resampling and algorithm improvement. solution.

Traditionally, for datasets with imbalanced samples, the solution mainly employs two strategies: data resampling before classification, or classification using a loss function that is sensitive to imbalanced samples.

Data resampling methods aim to oversample the minority class to increase its sample size, or undersample the majority class to reduce its sample size; while this technique can produce a balanced dataset, it results in loss of information (undersampling). sampling) or repeated representation (oversampling). The sample imbalance sensitive function is carried out by focusing on the minority samples that are usually ignored, so that the classification can pay more attention to these minority samples. However, these methods need to manually adjust the hyperparameters, and the effect is often unsatisfactory; in addition, there are some The algorithm aims to learn the features of the minority class centrally, however, this will lose the rich information from the majority class, which is not feasible when the absolute number of the minority class is small (e.g. 20 samples).

SUMMARY OF THE INVENTION

In view of the above deficiencies in the prior art, the divergence-excited autoencoder and its classification method for unbalanced data classification provided by the present invention solve the problem of slow convergence and inaccurate classification results when classifying unbalanced data. The problem.

In order to achieve the above purpose of the invention, the technical solution adopted in the present invention is: a divergence-excited autoencoder for unbalanced data classification, comprising sequentially connecting a data input layer, an encoding layer unit, a bottleneck layer, a decoding layer unit and data output layer;

The data input layer is connected with the data output layer;

Both the bottleneck layer and the data output layer units have loss functions;

The loss function in the bottleneck layer is used to calculate the divergence loss of the data input to the bottleneck layer, and then motivates to distinguish samples with different labels;

The loss function in the data output layer unit is used to calculate the reconstruction loss of the output data of the decoding layer unit.

Further, the data input layer is used to input the data to be classified;

The coding layer unit includes several convolutional layers connected in sequence for coding the data to be classified inputted by the data input layer;

The bottleneck layer includes a feature layer and a function loss layer; wherein, the feature layer is used to derive the output data of the coding layer unit, and is divided into minority samples and majority samples according to their label values, and the function loss layer is used to calculate minority samples and majority samples. The divergence loss and gradient of ;

The decoding layer unit includes several sequentially connected deconvolution layers for decoding the data output by the bottleneck layer and reconstructing the data to be classified;

The data output layer calculates the reconstruction loss and gradient of the output data according to the reconstructed data to be classified and the data to be classified, and outputs the classified data.

Further, in the data output layer:

When the output data of the decoding layer unit is standardized data, the decoding layer unit is connected with the data output layer through the sigmoid function, and the reconstruction loss calculated by the loss function of the data output layer is the cross entropy loss;

When the output data of the decoding layer unit is not standardized data, the decoding layer unit is directly connected to the data output layer, and the reconstruction loss calculated by the loss function of the data output layer is the least square loss;

Among them, when the output data of the decoding layer unit is standardized data, the sigmoid function σ( ) is:

in, is the pixel value of the reconstructed data to be classified at coordinates (i, j);

The cross-entropy loss L _r ( ) of the data output layer is:

Wherein, X _i,j is the pixel value of the input data to be classified at the coordinates (i, j);

X is the input data to be classified;

is the reconstructed data to be classified;

The gradient corresponding to the cross-entropy loss for:

Among them, * is the pairwise multiplication operator;

When the output data of the decoding layer unit is not normalized data, the least square loss L _r ( ) of the data output layer is:

Among them, the superscript T is the transpose operator;

The gradient corresponding to the least squares loss for:

where m is the number of samples input to the data output layer.

Further, the divergence loss L _d (·) in the bottleneck layer is:

in,

Z ^N* is the feature element value of most samples in the bottleneck layer;

Z ^K is the feature element value of a few samples in the bottleneck layer;

σ( ) is the sigmoid function;

v is a function that copies m rows of data so that it has the same size as the subtracted matrix;

m ₁ is the number of samples in the majority sample class;

is the bottleneck layer data of the i-th majority sample class data sample;

is the bottleneck layer data of the i-th minority sample class data sample;

is the jth dimension data of the bottleneck layer data of the ith majority sample class data sample;

is the jth dimension data of the bottleneck layer data of the ith minority sample class data sample;

is the output of the sigmoid function of the j-th dimension of the i-th minority sample class data sample;

The gradient corresponding to the divergence loss includes the gradient of Z ^N* and the gradient of Z ^K ;

where, the gradient of Z ^N* for:

Among them, the gradient of Z ^K for:

A classification method for divergence-excited autoencoders for classification of unbalanced data, comprising the following steps:

S1. Build a divergence-excited autoencoder;

S2, determine the training set input to the training of the divergence-excitation encoder, and input the data input layer;

S3. Take minimizing the reconstruction loss and optimizing the divergence loss as the training objectives in turn, and complete the training of the divergence-excitation encoder;

S4. Input the data in the training set and the test data into the trained divergence-excitation autoencoder, and use the cosine distance classifier to classify the test data.

Further, the data in the training set in the step S2 includes the training data X, the labels of the data _yn of the majority sample class in the training data X and the labels of the data y _k of the minority class of the training data X.

Further, the step S3 is specifically:

S31, input the training data X in the training set, the label of the data y _n of the majority sample class in the training data X, and the label of the data y _k of the minority sample class in the training data X into the divergence-excitation autoencoder in turn;

S32, respectively determining the label index of the data y _k of the minority sample class and the label index of the data y _n of the majority sample class;

Among them, the label index of the data y _k of the minority sample class is

The label index of the data y _n of the majority sample class is

S32. Perform the input excitation-autoencoder on the label index of the data y _k of the minority sample class and the label index of the data y _n of the majority sample class Calculate the gradient corresponding to the reconstruction loss for each time, and back-propagate the gradient corresponding to the reconstruction loss for each calculation to the data output layer to form a training subset with balanced labels;

S34, according to the training subset with balanced labels, calculate The gradient corresponding to the sub-divergence loss is back-propagated to the bottleneck layer;

S35 , respectively evaluate the reconstruction loss and the divergence loss of the current divergence-excitation autoencoder, and complete the training of the divergence-excitation autoencoder when the reconstruction loss and the divergence loss have no significant gradients.

Further, the step S4 is specifically:

S41. Input the test data X ^* into the trained divergence-excited autoencoder;

S42, obtain the characteristic representation z ^* of each sample x ^* in the test data X ^* according to the trained divergence-excitation autoencoder;

S43, input the training data X into the trained divergence-excited autoencoder, and obtain the data X ^K of the minority sample class with the classification label and the data X ^N of the majority sample class;

S44. Combine the feature representation z ^* of each sample x ^* in the test data X ^* , the samples in the data ^XK of the minority sample class and the samples in the data ^XN of the majority sample class, and determine the test in turn according to the cosine distance classifier The category of each sample x ^* in the data X ^* , completes the classification of the test data X ^* .

Further, the classification method of a sample x ^* in the step S44 is specifically:

S441, calculate the cosine distance D _N between each sample ^zi and z ^* in the data X ^N of most sample classes;

S442, calculate the cosine distance D _K between each sample z ^j and z ^* in the data X ^K of the minority sample class;

S443. Determine the median values of D _N and D _K respectively, and compare their sizes;

S444. Classify the sample x ^* corresponding to z ^* and the data with the data label into one category according to the smaller median value.

Further, it is characterized in that;

The cosine distance D _N in the step S431 is:

The cosine distance D _K in the described step S432 is:

Among them, |||| is the operator for taking the norm.

The beneficial effects of the present invention are as follows: the divergence-excited autoencoder and its classification method for dividing and equalizing data classification proposed by the present invention have the following advantages: First, in terms of F1 value and convergence speed, the present invention proposes The autoencoder-based method is superior to the convolutional neural network-based method; secondly, the autoencoder requires fewer parameters for manual tuning, unlike loss-sensitive convolutional neural networks that require optimization of hyperparameters; finally, When the number of minority samples is small, the autoencoder model proposed in the present invention performs well.

Description of drawings

FIG. 1 is a structural diagram of a divergence-excited autoencoder used for unbalanced data classification in an embodiment provided by the present invention.

Fig. 2 is a flowchart of a classification method of a divergence-excited autoencoder for unbalanced data classification in an embodiment provided by the present invention.

FIG. 3 is a schematic diagram of an inverse divergence loss function in an embodiment provided by the present invention.

FIG. 4 is a flowchart of a training method for a divergence-excited autoencoder in an embodiment provided by the present invention.

FIG. 5 is a schematic diagram of gradient curves of two variables in the embodiment provided by the present invention.

FIG. 6 is a flowchart of a data classification method for a divergence-excited autoencoder in an embodiment provided by the present invention.

FIG. 7 is a comparison diagram of the classification effects of the three methods in the embodiment provided by the present invention when there are only 30 samples in the minority sample class.

FIG. 8 is a comparison diagram of the classification effects of the three methods in the embodiment provided by the present invention when there are only 10 samples in the minority sample class.

FIG. 9 is a schematic diagram showing the comparison of the classification effects of the three methods in the embodiment provided by the present invention with respect to the sample variation of the minority sample class.

FIG. 10 is a schematic diagram of performance comparison between 5 keywords and 5000 majority background samples using the method of the present invention in the embodiment provided by the present invention.

FIG. 11 is a schematic diagram of performance comparison between 10 keyword samples and 5000 majority backgrounds using the method of the present invention in the embodiment provided by the present invention.

Detailed ways

The specific embodiments of the present invention are described below to facilitate those skilled in the art to understand the present invention, but it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes Such changes are obvious within the spirit and scope of the present invention as defined and determined by the appended claims, and all inventions and creations utilizing the inventive concept are within the scope of protection.

As shown in Figure 1, a divergence-excited autoencoder for unbalanced data classification includes sequentially connecting a data input layer, an encoding layer unit, a bottleneck layer, a decoding layer unit and a data output layer;

The data input layer is connected with the data output layer;

Both the bottleneck layer and the data output layer units have loss functions;

The loss function in the bottleneck layer is used to calculate the divergence loss of the data input to the bottleneck layer, and is used to explicitly motivate and distinguish samples with different labels;

The loss function in the data output layer unit is used to calculate the reconstruction loss of the output data of the decoding layer unit.

Wherein, the data input layer is used to input the data to be classified;

The coding layer unit includes several convolutional layers connected in sequence for coding the data to be classified inputted by the data input layer;

The bottleneck layer includes a feature layer and a function loss layer; wherein, the feature layer is used to derive the output data of the coding layer unit, and is divided into minority samples and majority samples according to their label values, and the function loss layer is used to calculate minority samples and majority samples. The divergence loss and gradient of ;

The decoding layer unit includes several sequentially connected deconvolution layers for decoding the data output by the bottleneck layer and reconstructing the data to be classified;

The data output layer calculates the reconstruction loss and gradient of the output data according to the reconstructed data to be classified and the data to be classified, and outputs the classified data.

The loss function in the self-encoder proposed by the present invention includes reconstruction loss and divergence loss, wherein the reconstruction loss can be cross entropy loss or least square loss;

When the output data of the decoding layer unit is standardized data, the decoding layer unit is connected with the data output layer through the sigmoid function, and the reconstruction loss calculated by the loss function of the data output layer is the cross entropy loss;

When the output data of the decoding layer unit is not standardized data, the decoding layer unit is directly connected to the data output layer, and the reconstruction loss calculated by the loss function of the data output layer is the least square loss;

Among them, when the output data of the decoding layer unit is standardized data, the sigmoid function σ( ) is:

in, In order to reconstruct the pixel value of the data to be classified at the coordinates (i, j);

The cross-entropy loss L _r ( ) of the data output layer is:

Wherein, X _i,j is the pixel value of the input data to be classified at the coordinates (i, j);

X is the input data to be classified;

is the reconstructed data to be classified;

The output gradient corresponding to the cross-entropy loss for:

When represented by a matrix, the above gradient expression can be written as:

Among them, * is the pairwise multiplication operator;

When the output data of the decoding layer unit is not normalized data, the least square loss L _r ( ) of the data output layer is:

Among them, the superscript T is the transpose operator;

The gradient corresponding to the least squares loss for:

where m is the number of samples input to the data output layer.

For the above cross-entropy loss and least squares loss, the l ₂ norm w ^Tw can be added as a penalty.

In addition, the divergence loss exists in the bottleneck layer, and its purpose is to explicitly motivate to distinguish samples with different labels;

For the bottleneck layer, it is assumed that there are m ₁ samples in the data Z ^K of the minority sample class, m ₂ samples in the data Z ^N of the majority sample class, and m ₂ >>m ₁ . However, we can sample m1 samples from ^ZN and obtain ZN ^★ to equalize the number _of samples and apply the sigmoid function to ^{ZN ★} and ZK.

Losses can be applied:

It is worth noting that the above divergence loss is not exactly KL-divergence, since KL-divergence is for distributions rather than data. However, the method of the present invention can successfully stimulate the difference between the two classes. To account for the gradient of this loss, the gradients with respect to Z ^K and Z ^N★ can be analyzed separately.

The gradient of Z ^K can be calculated as:

The gradient with respect to Z ^N* can be calculated as:

The above loss calculation formula may cause the autoencoder to adapt to noise. In order to reduce the negative impact of such a negative situation, the intra-class variance is further added as a penalty:

Finally, the divergence loss L _d ( ) is:

in,

Z ^N* is the feature element value of most sample class samples in the bottleneck layer;

Z ^K is the feature element value of the minority sample class sample in the bottleneck layer;

σ( ) is the sigmoid function;

v is a function that copies m rows of data so that it has the same size as the subtracted matrix;

m ₁ is the number of samples in the majority sample class;

is the bottleneck layer data of the i-th majority sample class data sample;

is the bottleneck layer data of the i-th minority sample class data sample;

is the jth dimension data of the bottleneck layer data of the ith majority sample class data sample;

is the jth dimension data of the bottleneck layer data of the ith minority sample class data sample;

is the output of the sigmoid function of the j-th dimension of the i-th minority sample class data sample;

After considering the intra-class variance, the gradient corresponding to the divergence loss, including the gradient of Z ^N* and the gradient of Z ^K , can be modified into a matrix format;

where, the gradient of Z ^N* for:

Among them, the gradient of Z ^K for:

In an embodiment of the present invention, a classification method for a divergence-excited autoencoder for unbalanced data classification is also provided, as shown in FIG. 2 , including the following steps:

S1. Build a divergence-excited autoencoder;

S2, determine the training set input to the training of the divergence-excitation encoder, and input the data input layer;

The data in the training set in the above S2 includes the training data X, the label of the data y _n of the majority sample class in the training data X, and the label of the data y _k of the minority class of the training data X.

S3. Take minimizing the reconstruction loss and optimizing the divergence loss as the training objectives in turn, and complete the training of the divergence-excitation encoder;

S4. Input the data in the training set and the test data into the trained divergence-excitation autoencoder, and use the cosine distance classifier to classify the test data.

Equations (4) (6) (11) and (12) can be used to calculate the gradient and train the autoencoder in the above step S3. However, regarding the detailed training process, there are still two issues that need to be further determined: the training paradigm and Convergence of networks.

First: training with two losses at the same time, it is difficult to merge gradients during training since the loss functions belong to different layers. Therefore, in the autoencoder in the present invention, a two-stage training paradigm is adopted, instead of explicitly incorporating gradients and feeding them into the network: at each epoch, the reconstruction loss is first minimized (equations 4 or 6) The divergence loss is then optimized (Equation 11 and Equation 12). When training with divergence loss, sample from most sample classes times as an alternative to form a balanced label training subset.

Among them, the training process is similar to GANs: in GANs we alternately train the discriminator and the generator, while in the training method of the present invention, the data reconstruction and intra-class discrimination performance of the autoencoder are sequentially improved. In fact, these two goals Not necessarily "adversarial". In practice, however, in the first few iterations, it is normal if the two losses reflect some kind of "conflict" because they focus on different problems. Separating the two stages of training can bring another advantage: the optimization of the reconstruction loss can "compress" the values of zi ^,j to a relatively restricted range, which can alleviate instability during optimization of the divergence loss.

Second: the fusion of networks, the goal of the method of the present invention is to minimize the inverse divergence loss, so the objective function (Equation 10) itself is not necessarily a convex function. In fact, considering the 1-d case of σ(z ^K ) and σ(z ^N* ), the Hessian is indeterminate, and the sphere is as in Figure 3; so we will analyze the case of optimizing the divergence loss here.

To sum up, as shown in FIG. 4 , the above step S3 is specifically:

S31, input the training data X in the training set, the label of the data y _n of the majority sample class in the training data X, and the label of the data y _k of the minority sample class in the training data X into the divergence-excitation autoencoder in turn;

S32, respectively determining the label index of the data y _k of the minority sample class and the label index of the data y _n of the majority sample class;

Among them, the label index of the data y _k of the minority sample class is

The label index of the data y _n of the majority sample class is

S32. Perform the input excitation-autoencoder on the label index of the data y _k of the minority sample class and the label index of the data y _n of the majority sample class Calculate the gradient corresponding to the reconstruction loss for each time, and back-propagate the gradient corresponding to the reconstruction loss for each calculation to the data output layer to form a training subset with balanced labels;

S34, according to the training subset with balanced labels, calculate The gradient corresponding to the sub-divergence loss is back-propagated to the bottleneck layer;

S35 , respectively evaluate the reconstruction loss and the divergence loss of the current divergence-excitation autoencoder, and complete the training of the divergence-excitation autoencoder when the reconstruction loss and the divergence loss have no significant gradients.

When computing the gradient of each variable with Equation (11), since there is no interaction between dimensions in the divergence loss, only a single data for the one-dimensional case (scalar input) is considered for convenience. Extending the gradient in the sigmoid form of z ^K , we get:

For the gradient of Z ^N* , one can get:

Similarly, ∝ steps are also obtained by assuming z ^K as a given value. From the following equations, we can find that the gradient of z ^K (Equation 13) is a concave/convex function, and it is obvious that when z ^K reaches large positive/negative values, its gradient will saturate close to to 0, which indicates the convergence of the loss. Likewise, the gradient of z ^N* (expressed in Equation 14) will saturate close to 1 for large negative values and close to 0 for large positive values, while being monotonic in the interval . This shows that for a given z ^K , an optimization objective of approximately stationary z ^N* can also be found. Therefore, our training method is highly effective. In practice, it is often possible to find flat gradient points for divergence loss; gradient 12 and gradient 13 are shown in Figure 5.

The analysis of the gradient curve also provides a basis for two-phase training: if the divergence gradient is directly applied to the network, since the Z value is in a large range, it may encounter gradient instability; if two-phase training is used, the Z value will be in a relatively dense range after optimizing the reconstruction loss, so the divergence optimization will be more stable. According to the above analysis, the training process is simply based on the gradient disappearance of reconstruction and divergence loss, and it is reasonable to terminate this process.

As shown in Figure 6, the above step S4 is specifically:

S41. Input the test data X ^* into the trained divergence-excited autoencoder;

S42, obtain the characteristic representation z ^* of each sample x ^* in the test data X ^* according to the trained divergence-excitation autoencoder;

S43, input the training data X into the trained divergence-excited autoencoder, and obtain the data X ^K of the minority sample class with the classification label and the data X ^N of the majority sample class;

S44. Combine the feature representation z ^* of each sample x ^* in the test data X ^* , the samples in the data ^XK of the minority sample class and the samples in the data ^XN of the majority sample class, and determine the test in turn according to the cosine distance classifier The category of each sample x ^* in the data X ^* , complete the classification of the test data X ^*

The classification method of a sample x ^* in the above step S44 is specifically:

S441, calculate the cosine distance D _N between each sample ^zi and z ^* in the data X ^N of most sample classes;

The cosine distance D _N in the above step S431 is:

S442, calculate the cosine distance D _K between each sample z ^j and z ^* in the data X ^K of the minority sample class;

The cosine distance D _K in the above-mentioned steps S432 is:

Among them, |||| is the operator for taking the general number.

S443. Determine the median values of D _N and D _K respectively, and compare their sizes;

S444. Classify the sample x ^* corresponding to z ^* and the data with the data label into one category according to the smaller median value.

In one embodiment of the present invention, by comparing with the convolutional neural network and the sensitive convolutional neural network, the process of verifying the validity and effect of the method of the present invention:

An unbalanced dataset was generated by sampling from the benchmark MNIST dataset, and its performance was tested with convolutional neural networks, loss-sensitive convolutional neural networks, and the proposed divergence-excited autoencoder-based algorithm. Several commonly used non-equilibrium classification metrics are used to evaluate the performance of the network, including TPr and TNr, which represent the recall rate of the majority and minority sample classes, TPv and TNv, which represent the precision rate of the classification results _; in addition, the F1 score and G mean is used as a measure of "overall performance", to illustrate the computation of these measures, we use TP, TN, FP, FN to denote the sample size, and these samples are classified as true positives, true negatives, false positives, and false negatives, respectively, Then, the precision/recall of the two classes can be expressed as follows:

_The F1 score can be expressed as:

Similarly, G-means can be expressed as:

Based on the experimental situation, adjusting the parameter η=12.0, the divergence-excited autoencoder proposed by the present invention is composed of 3 layers of convolution and 3 layers of deconvolution layers, and the bottleneck layer is 100 dimensions. Detailed descriptions of the two convolutional neural networks and autoencoders used in the experiments are shown in Table 1.

Table 1: Structure and Parameters of Convolutional Neural Networks and Loss-Sensitive Convolutional Neural Networks

For a clearer comparison, the classification results of the three methods for the minority class with only 30 and 10 samples were analyzed, and the recall of the key minority class was selected to plot. The classification results are compared in Figures 7 and 7. shown in Figure 8. Three subgraphs (a), (b) and (c) are selected from left to right to represent the traditional convolutional neural network, the improved loss-sensitive convolutional neural network and the effect of the proposed method of the present invention. Each small square is the classification result of its corresponding horizontal and vertical handwritten digits. The depth of its color represents the level of recall, and the darker the color, the higher the recall of key minority sample classes (the specific value has been set in marked in the figure). It can be seen from the figure that the method proposed by the present invention has the highest recall ability, and also has a good effect on the case where the sample size is further reduced (10 samples).

In order to further examine the change of the recall rate of the minority sample class with the reduction of sample data in the minority sample class, the convolutional neural network, the improved loss-sensitive convolutional neural network and the method proposed by the present invention are respectively calculated and plotted in Fig. 9, The figure shows that when the handwritten digits are classified again, the recall rate of the proposed divergence-excited autoencoder method is always higher than other methods. The robustness of the method is better.

In addition, compared with the aforementioned two-category unbalanced data classification task, the keyword recognition task of recognizing a few keywords from a noisy background has higher requirements on the classifier, which is also tested in this embodiment. FIG. 10 and FIG. 11 respectively show the comparison of the interception performance of keywords 3 and 1, showing that the recall, F1 value and G-Mean value of the method proposed by the present invention are better than other methods.

In the process of the above comparison experiment, the performance comparison of the ordinary convolutional neural network, the loss-sensitive convolutional neural network and the divergence-excited autoencoder proposed by the present invention is illustrated under different tasks and parameter settings. It can be observed that the proposed divergence-excited autoencoders are generally more favorable than convolutional networks:

(1) Task performance when the number of minority samples is small: For the tested models, it is found that the performance of the convolutional network tends to fluctuate. Satisfactory performance; while the autoencoder proposed by the present invention is more stable.

(2) Convergence speed: From the above data, it can be seen that the divergence-excited autoencoder-based method generally converges faster than the convolutional neural network.

(3) F1 value and G mean: The F1 score and G mean of the minority sample class are used to evaluate the overall performance of the algorithm. Under the two evaluation indicators, the divergence-excited autoencoder proposed by the present invention is superior to the convolutional neural network.

To sum up, the divergence-excited autoencoder and its classification method for dividing balanced data classification proposed by the present invention have the following advantages: First, in terms of F1 value and convergence speed, the The encoder-based approach outperforms the convolutional neural network-based approach; secondly, the autoencoder requires fewer parameters for manual tuning, unlike loss-sensitive convolutional neural networks that require optimization of hyperparameters; finally, when a small number of samples When the number of samples of the class is small, the autoencoder model proposed in the present invention performs well.

Claims (10)

1. A divergence-excitation self-encoder for non-equalized data classification is characterized by comprising a data input layer, an encoding layer unit, a bottleneck layer, a decoding layer unit and a data output layer which are connected in sequence;

the data input layer is connected with the data output layer;

the bottleneck layer unit and the data output layer unit both have loss functions;

the loss function in the bottleneck layer is used for calculating divergence loss of data input to the bottleneck layer so as to excite and distinguish samples with different labels;

the loss function in the data output layer unit is used for calculating the reconstruction loss of the output data of the decoding layer unit.

2. A divergence-autocoder for non-equalized data classification in accordance with claim 1, characterized in that the data input layer is used to input data to be classified;

the coding layer unit comprises a plurality of convolution layers which are connected in sequence and is used for coding the data to be classified input by the data input layer;

the bottleneck layer comprises a factor layer and a function loss layer; the element layer is used for deriving output data of the coding layer unit and dividing the output data into a few samples and a plurality of samples according to the label value of the element layer, and the function loss layer is used for calculating divergence loss and gradient of the few samples and the plurality of samples;

the decoding layer unit comprises a plurality of deconvolution layers which are connected in sequence and is used for decoding data output by the bottleneck layer and reconstructing data to be classified;

and the data output layer calculates the reconstruction loss and gradient of the output data according to the reconstructed data to be classified and the data to be classified, and outputs the classified data.

3. A divergence-self encoder for non-equalized data classification in accordance with claim 1, characterized in that in the data output layer:

when the output data of the decoding layer unit is standardized data, the decoding layer unit is connected with the data output layer through a sigmoid function, and the reconstruction loss calculated by the loss function of the data output layer is cross entropy loss;

when the output data of the decoding layer unit is not standardized data, the decoding layer unit is directly connected with the data output layer, and the reconstruction loss calculated by the loss function of the data output layer is the minimum square loss;

when the output data of the decoding layer unit is standardized data, the sigmoid function sigma (-) is as follows:

wherein,pixel values at coordinates (i, j) for the reconstructed data to be classified;

cross entropy loss L of data output layer_r(. is):

wherein, X_i,jThe pixel value of the input data to be classified at the coordinate (i, j);

x is input data to be classified;

classifying the reconstructed data to be classified;

gradient corresponding to cross entropy lossComprises the following steps:

wherein, is the pairwise multiplication operator;

least square loss L of data output layer when output data of decoding layer unit is not standardized data_r(. is):

wherein, the superscript T is the transpose operator;

gradient of least square loss correspondenceComprises the following steps:

where m is the number of samples input to the data output layer.

4. Divergence-excitation self encoder for non-equalized data classification in accordance with claim 1 characterized in that the divergence loss in the bottleneck layer is L_d(. is):

wherein,

Z^N*characteristic element values of most sample samples in the bottleneck layer;

Z^Kcharacteristic element values of a few sample class samples in a bottleneck layer;

σ (-) is a sigmoid function;

v is a function that copies the data m rows so that it is the same size as the matrix being reduced;

m₁is the number of samples in the majority sample class;

bottleneck layer data of an ith majority sample class data sample;

bottleneck layer data of the ith few sample class data samples;

bottle for ith majority sample class data sampleJ-th dimension data of the neck layer data;

j dimension data of the bottleneck layer data of the ith few sample class data samples;

the output of the sigmoid function of the j dimension of the ith few sample class data samples;

the gradient corresponding to the divergence loss comprises Z^N*Gradient and Z^KA gradient of (a);

wherein Z is^N*Gradient of (2)Comprises the following steps:

wherein Z is^KGradient of (2)Comprises the following steps:

5. a method of classification for a divergence-excitation autocoder for non-equalized data classification, comprising the steps of:

s1, constructing a divergence-excitation self-encoder;

s2, determining a training set input to the divergence-excitation encoder training, and inputting a data input layer;

s3, sequentially taking the minimized reconstruction loss and the optimized divergence loss as training targets to finish the training of the divergence-excitation encoder;

and S4, inputting the data in the training set and the test data into a trained divergence-excitation self-encoder, and classifying the test data by using a cosine distance classifier.

6. The method for classifying a divergence-excitation autocoder for non-equalized data classification as claimed in claim 5, wherein the data in the training set in the step S2 includes training data X, and data y of a majority sample class in the training data X_nData y of a few sample classes in label and training data X_kThe label of (1).

7. The method for classifying a divergence-excitation autocoder for non-equalized data classification as claimed in claim 6, wherein the step S3 is specifically:

s31, training data X in training set, data y of most samples in training data X_nLabel of (2) and data y of a few sample classes in the training data X_kThe labels are sequentially input into a divergence-excitation self-encoder;

s32, determining data y of a few sample classes respectively_kLabel index of (d) and data y of majority sample class_nThe tag index of (a);

wherein, data y of a few sample classes_kHas a label index of

Data y of majority sample class_nHas a label index of

S32, data y according to minority sample class_kLabel index of (d) and data y of majority sample class_nTag index of input excitation-self encoderCalculating the gradient corresponding to the reconstruction loss of each time, and calculating each timeThe gradient corresponding to the reconstruction loss is reversely propagated to a data output layer to form a training subset with a balance label;

s34, calculating according to the training subset with the balance labelThe corresponding gradient of the hypo-divergence loss is reversely propagated to the bottleneck layer;

and S35, respectively carrying out reconstruction loss and divergence loss evaluation on the current divergence-excitation self-encoder, and completing the training of the divergence-excitation self-encoder when the reconstruction loss and the divergence loss have no obvious gradient.

8. The method for classifying a divergence-excitation autocoder for non-equalized data classification as claimed in claim 7, wherein the step S4 is specifically:

s41, testing data X^*Inputting the data to a trained divergence-excitation self-encoder;

s42, obtaining test data X according to the trained divergence-excitation self-encoder^*Of each sample x^*Is a characteristic of^*；

S43, inputting the training data X into the trained divergence-excitation self-encoder to obtain the data X of a few sample classes with classification labels^KAnd data X of a majority sample class^N；

S44 Combined test data X^*Of each sample x^*Is a characteristic of^*Data X of minority sample class^KSample in (1) and data X of a majority of sample classes^NThe sample in (1) sequentially determines the test data X according to the cosine distance classifier^*Of each sample x^*Class of completion test data X^*Classification of (3).

9. The training method of divergence-excitation autocoder for non-equalized data classification as claimed in claim 8, wherein one sample x in the step S44^*The classification method is concretely：

S441, calculating data X of a plurality of sample classes^NEach sample z inⁱAnd z^*Cosine distance D between_N；

S442, calculating data X of a few sample classes^KEach sample z in^jAnd z^*Cosine distance D between_K；

S443, determining D respectively_NAnd D_KAnd comparing the magnitudes thereof;

s444, according to the smaller median value, dividing z in the value^*Corresponding sample x^*And data classes with data tags.

10. The method of classification for a divergence-excitation autocoder for unequal data classification as claimed in claim 9, characterized in that;

cosine distance D in the step S431_NComprises the following steps:

the cosine distance D in the step S432_KComprises the following steps:

wherein, | | | is an operational operator for pan number.