CN111738343A - An image annotation method based on semi-supervised learning - Google Patents

Abstract

The invention discloses an image labeling method based on semi-supervised learning, which is characterized in that different classifiers are designed aiming at different types of samples, the classifiers are trained by using the labeled partial samples, the results of the different classifiers are voted, and the type with the highest accuracy is selected, so that an unknown sample is labeled. In order to reduce the influence caused by error classification, random linear mixing operation is carried out on the samples in each category obtained by the classifier and the samples in the labeled corresponding categories, so that the error classification result also contains the characteristics of the corresponding category, and a new thought is provided for the field of deep learning and machine learning of semi-supervised learning.

Description

An image annotation method based on semi-supervised learning

technical field

The invention belongs to the field of semi-supervised learning, and relates to an image labeling method based on semi-supervised learning.

Background technique

In recent years, the introduction of deep learning technology and the maturity of machine learning technology have made breakthroughs in the field of computer vision. Many traditional problems in the field of computer vision, such as classification problems, detection problems, and semantic segmentation problems, have been better solved. However, most computer vision tasks are supervised learning tasks, which means that all input data needs to be manually labeled. People need to collect relevant pictures and use relevant software to carefully label the pictures. This process is very expensive. Human and material resources, in the field of target detection, such as the coco data set contains 80 categories, a total of 170,000 pictures, each of these pictures needs to manually find out the targets to be labeled one by one and mark the corresponding objects with a rectangular frame It usually takes several weeks or even months for hundreds of people to label such large datasets. It is inevitable that mistakes will occur during human labeling. Errors in labeling will have some impact on subsequent training, and error correction is also difficult. Therefore, there is a need for a method that can improve the problem of data set labeling, improve the accuracy of labeling and save some labor. cost. Semi-supervised learning has always been a research hotspot in the field of machine learning. It is a method that combines supervised learning and unsupervised learning. This method trains the model through a part of annotated and a large number of unlabeled files. To a large extent, the burden of the staff is reduced, but the current semi-supervised learning methods cannot make the results of model training equal to those of supervised training. Therefore, supervised learning is still the mainstream in the field of computer vision. The process of semi-supervised learning is similar to that of supervised learning. Supervised learning has some commonalities. Semi-supervised learning needs to label part of the data. This part of the labeled data can be used to classify and label the unlabeled data, and through the current technology, a small number of samples are trained. The resulting model also has a good classification effect, so this paper proposes a method for auxiliary labeling using a semi-supervised learning method.

SUMMARY OF THE INVENTION

In order to solve the above problems, the technical solution of the present invention is an image labeling method based on semi-supervised learning, which includes the following steps:

Include the following steps:

S10, add background class: the goal is to divide into class A and class B, first introduce the background class composed of random sampling of other classes other than class A or class B;

S20, build a cross-network classification model: label the M classified image data, and use a deep learning network to train a model between every two classes, so there are a total of M*(M-1) models in different orders. Select different networks for class A and B training, train the labeled data through M*(M-1) models, and record M-1 independent classification models of a certain class as a class learner, with a total of M classes learner;

S30, constitute a sub-voter: the unlabeled data is used to predict the category through M class learners, and all the results including a certain category are formed into a voting subset sum. According to the number of categories, there are a total of M sub-voters. The voter contains 2M-2 groups of different prediction results, each of which contains a prediction for a certain class. When the class Avs class B and class Bvs class A overlap, the networks trained by the two are different. also different;

S40, vote according to the mutually exclusive voter: each of the M sub-voters generates 2M-2 prediction probabilities for the same picture, and sets a series of rules and thresholds, as long as the voting results generated by the sub-voters exceed the threshold, that is, It is considered that the sample belongs to the category corresponding to this set, and the voting results have only one predicted category of pictures retained, and marked with prediction labels.

S50, correcting the wrong labels based on random linear mixing: the newly labeled samples are randomly mixed with the original samples with the same label, so as to suppress the interference caused by the wrongly labeled samples to the network training.

Preferably, the voting includes the following steps:

S41, each sub-voter calculates the weighted average of the accuracy, the accuracy of each classification network is different, and calculates the corresponding weight coefficient according to the accuracy, so that the result predicted by the model with high accuracy has a greater impact on the final labeling;

S42, each sub-voter set scores the picture, and counts the number N of sub-voters that exceed the threshold;

S43, only keep the data of N=1, and add a prediction label to it.

During the voting process, when the probability of the occurrence of two sub-voters exceeds the threshold, the representative sample may belong to two different labels, and the sub-voter determines that the voting fails and discards the sample.

Preferably, the random linear mixing fuses similar data without generating new labels, and the calculation formula is:

Among them, x _i is the labeled image of the same category as x _j , x _j is the labeled file output by the model, and β is a random parameter, ranging from 0 to 1.

Preferably, the threshold value is 0.95.

The beneficial effects of the present invention are as follows: the present invention proposes a method for predicting a large amount of unlabeled data by adopting semi-supervised ensemble learning for the existence of a small amount of labeled data and a large amount of unlabeled data in image labeling, thereby realizing the For accurate category prediction of data, according to the process of deep learning network learning image data, a random linear hybrid method is proposed to achieve more accurate image annotation.

Description of drawings

1 is a flow chart of the steps of an image labeling method based on semi-supervised learning according to a specific embodiment of the method of the present invention;

FIG. 2 is a schematic diagram of an image labeling method based on semi-supervised learning according to a specific embodiment of the method of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

On the contrary, the present invention covers any alternatives, modifications, equivalents and arrangements within the spirit and scope of the present invention as defined by the appended claims. Further, in order to give the public a better understanding of the present invention, some specific details are described in detail in the following detailed description of the present invention. The present invention can be fully understood by those skilled in the art without the description of these detailed parts.

Referring to FIG. 1, it is a flowchart of steps of an image labeling method based on semi-supervised learning according to an embodiment of the present invention, including the following steps:

S10, add background class: the goal is to divide into class A and class B, first introduce the background class composed of random sampling of other classes other than class A or class B;

First, a background class is introduced for each two-class base classifier to combat the prediction interference problem of unknown class samples. In this way, the two-class model of the base learner is transformed into a three-class model. The ratio of the three types of A-B-background classes is 1:1:1, and the composition method of background class images is randomly selected from other classes in equal proportions;

S20, build a cross-network classification model: label the M classified image data, and use a deep learning network to train a model between every two classes, so there are a total of M*(M-1) models in different orders. Select different networks for class A and B training, train the labeled data through M*(M-1) models, and record M-1 independent classification models of a certain class as a class learner, with a total of M classes learner;

Taking M as 20 as a specific example, the construction of the data labeling system is based on ensemble learning, which integrates the weak classifiers trained by deep learning between every two different classes A-B. Finally, the image is annotated based on the model that has been integrated and learned. Annotate 20 classified image data, and use a deep learning network to train a model between every two classes, so there are a total of 380 models (20*19), and the weak classifiers involved in the integration should be as much as possible. Independence, in order to avoid the repetition of A-B and B-A categories, the same network is used for training, and the category is indexed. When the index of A is greater than B, the deep learning model PnasNet (batch_size=64) is selected. When the index of X is smaller than Y, deep learning is selected. Model SeNet (batch_size=32). Different batch_sizes can make the features learned by the network inconsistent, so different batch_sizes are selected for the above two cases. A small amount of labeled data is trained through 380 models, and 19 independent classification models of a certain class are recorded as a class learner, so there are a total of 20 class learners

S30, form a sub-voter: predict the category of the unlabeled data through M class learners, and form a voting subset sum with all the results of a certain category. According to the number of categories, there are a total of M voting subset sums, Each voting subset sum contains 2M-2 groups of different prediction results, each of which contains predictions for a certain class. When there is an overlap between class Avs class B and class Bvs class A, the two trained Different networks have different predictions;

A large amount of unlabeled data is passed through 20 classifiers to predict categories. All results containing a class form a subvoter. Therefore, according to the number of categories, there are a total of 20 sub-voters, each sub-voter contains 38 sets of different prediction results, and each of these 38 groups of prediction results contains predictions for a certain category. As mentioned above, when there is an overlap of "class Avs class B" and "class B vs class A", the networks trained by the two are different, so the predictions are also different.

S40, vote according to the mutually exclusive voter: each of the M sub-voters generates 2M-2 prediction probabilities for the same picture, and sets a series of rules and thresholds, as long as the voting results generated by the sub-voters exceed the threshold, that is, It is considered that the sample belongs to the category corresponding to this sub-voter set, and the voting result has only one predicted category of pictures retained, and marked with the predicted label;

Each sub-voter set contains 38 prediction probabilities for the same picture. Each sub-voter calculates the weighted average of the accuracy. The accuracy of each classification network is different. The result predicted by the model has a greater impact on the final annotation; then, each sub-voter set scores the image, and counts the number N of sub-voters that exceed the threshold of 0.95. Finally, only keep the data of N=1, and label it with prediction to complete the prediction of mutually exclusive voting;

S50, correcting the wrong labels based on random linear mixing: the newly labeled samples are randomly mixed with the original samples with the same label, so as to suppress the interference of the wrongly labeled samples on network training;

The correct rate of newly generated sample labels can reach more than 94%, but some labels are still wrong. In order to correct the adverse effects of a small number of wrong labels on possible future network training, the invention proposes RLB data hybrid enhancement algorithm. (Random linear blending), the newly labeled samples are randomly mixed with the original samples with the same label to suppress the interference caused by the wrongly labeled samples to network training.

Voting includes the following steps:

S41, each sub-voter calculates the weighted average of the accuracy, the accuracy of each classification network is different, and calculates the corresponding weight coefficient according to the accuracy, so that the result predicted by the model with high accuracy has a greater impact on the final labeling;

S42, each sub-voter set scores the picture, and counts the number N of sub-voters that exceed the threshold;

S43, only keep the data of N=1, and add a prediction label to it.

During the voting process, when the probability of the occurrence of two sub-voters exceeds the threshold, the representative sample may belong to two different labels, and the sub-voter determines that the voting fails and discards the sample.

Random linear mixing fuses similar data without generating new labels. The calculation formula is:

Among them, x _i is the labeled image of the same category as x _j , x _j is the labeled file output by the model, and β is a random parameter, ranging from 0 to 1. Referring to Fig. 2, the specific effect of the method operation of the present invention is shown.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (4)

1. An image labeling method based on semi-supervised learning is characterized by comprising the following steps:

s10, add background class: the target is divided into A type and B type, and a background type formed by random sampling of other types other than A type or B type is introduced;

s20, constructing a cross network classification model: labeling M classified picture data, wherein a deep learning network is used for training a model between every two classes, so that the number of M (M-1) models is total, different networks are selected during training of A, B classes in different sequences, the labeled data are trained through the M (M-1) models, the M-1 independent classification models of a certain class are labeled as a class learner, and the number of M class learners is total;

s30, forming a sub-voter: predicting classes of unlabelled data through M class learners, forming a sub-voter by using all results containing a certain class, wherein according to the number of classes, the total number of the M sub-voters is M, the M sub-voters form a voter, each sub-voter contains 2M-2 groups of different prediction results, each group of the prediction results contains prediction of a certain class, and when the overlapping condition of Avs class B and Bvs class A occurs, the training networks of the two classes are different, and the predictions are also different;

s40, voting according to the mutual exclusion voter: each sub-voter in the M sub-voters generates 2M-2 prediction probabilities for the same picture, a series of rules and thresholds are set, as long as the voting result generated by the sub-voter exceeds the threshold, the sample is considered to belong to the category corresponding to the set, the voting result is only the picture of one prediction category is reserved, and a prediction label is marked;

s50, correcting the error label based on the random linear mixture: and mixing the newly marked sample with the original sample with the label in a random proportion, and inhibiting the interference of the wrongly marked sample on network training.

2. The method of claim 1, wherein the voting comprises the steps of:

s41, each sub-voter calculates the weighted average of the accuracy, the accuracy of each classification network is different, and the corresponding weight coefficient is calculated according to the accuracy, so that the influence of the result predicted by the model with high accuracy on the final labeling is larger;

s42, scoring the picture by each sub-voter set, and counting the number N of the sub-voters exceeding a threshold value;

s43, only keeping the data with N being 1, and marking a prediction label for the data;

in the voting process, when the probabilities of two sub-voters exceed the threshold, the representative sample may belong to two different labels, and the voter judges that the voting fails and abandons the sample.

3. The method of claim 1, wherein the random linear mixture fuses homogeneous data without generating new labels, and the calculation formula is:

wherein x_iAre marked with x_jPictures of the same category, x_jAnd β is a random parameter which is an annotation file output by the model and has a value range of 0-1.

4. The method of claim 1, wherein the threshold is 0.95.

Images