CN109376786A - An image classification method, apparatus, terminal device and readable storage medium - Google Patents

Abstract

The present invention is applicable to the technical field of image processing, and provides an image classification method, device, terminal device and readable storage medium. The method includes: training a deep convolutional neural network by using images of known categories to obtain a network training model; The network training model establishes a probability distribution model for each type of sample in the known category image; corrects the activation value of the known category image according to the probability distribution model; The activation value obtains the activation value of the image of the unknown category; the images are classified according to the activation value of the image of the known category and the activation value of the image of the unknown category. The present invention can reasonably and accurately classify images out of the training set category in the known category images in practical applications.

Description

An image classification method, apparatus, terminal device and readable storage medium

technical field

The invention belongs to the technical field of image processing, and in particular relates to an image classification method, device, terminal device and readable storage medium.

Background technique

Image classification can extract, process and analyze the characteristic data of the image through the computer, identify different targets and objects, and classify the images according to different characteristics.

At present, when performing image classification, the algorithm based on deep neural network uses a trained image classification model to classify known image data, wherein the trained image classification model is based on the training image data in the same category space. and test image data; or determine the unknown image category according to the activation value of the correctly classified image sample of a certain category; however, for the unknown category of images outside the training set of known image categories, it is impossible to make reasonable judgments. classification, as well as the defects of inaccurate image classification.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present invention provide an image classification method, apparatus, terminal device, and readable storage medium, so as to solve the problem in the prior art for images of unknown categories outside the training set of known image categories. Reasonable classification is not possible, and there is a problem of inaccurate image classification defects.

A first aspect of the embodiments of the present invention provides an image classification method, including:

The deep convolutional neural network is trained through the known category images to obtain the network training model;

According to the network training model, a probability distribution model is separately established for each type of sample in the known type of image;

Modify the activation value of the known category image according to the probability distribution model;

Obtain the activation value of the unknown category image according to the activation value of the known category image data;

The images are classified according to the activation values of the known class images and the activation values of the unknown class images.

In one embodiment, a deep convolutional neural network is trained by using images of known categories to obtain a network training model, including:

Divide the acquired images of known categories into a training set and a test set;

The deep convolutional neural network is trained by the images of the training set, the classification performance of the deep convolutional neural network is tested by the images of the test set, and the network classification result is output;

Perform a supervised operation on the network classification result through a loss function to obtain a supervised operation result;

Adjust the network parameters of the deep convolutional neural network according to the result of the supervision operation.

In one embodiment, according to the network training model, a probability distribution model is separately established for each type of sample in the known type of image, including:

Obtain the mean vector of each class of samples in the known class image;

Calculate the distance between each class of samples in the known class image and the mean vector;

According to the distance, select a number of input samples from each type of samples according to a preset ratio;

The model parameters of the probability distribution model corresponding to the input sample categories are estimated according to the several input samples.

In one embodiment, correcting the activation value of the known category image according to the probability distribution model includes:

Extracting the first activation value of the test sample in the known category image by using the network training model;

According to the first activation value, a preset number of sample categories are selected from the test samples;

calculating, according to the probability distribution model corresponding to the preset number of sample categories and the first activation value of the test samples in the sample category, the probability of belonging to the test samples in the preset number of sample categories;

The first activation value of the test samples in the preset number of sample categories is modified according to the belonging probability, and the second activation value is obtained.

In one embodiment, obtaining the activation value of the unknown category image according to the activation value of the known category image data includes:

Calculate the activation value of the unknown category image according to the first activation value and the second activation value of the test sample in the selected preset number of sample categories The calculation formula is:

in, is the first activation value of the test sample, is the corrected second activation value, C is the total number of categories of images of known categories, and c is any c-th sample in the total number of categories.

In one embodiment, classifying images according to the activation value of the known category image and the activation value of the unknown category image includes:

Normalize the activation value of the image of the known category and the activation value of the image of the unknown category to obtain a new activation value of the image;

Select the undetermined category value corresponding to the current test image with the largest activation value from the new activation values;

judging whether the pending category value corresponds to the unknown category value;

If so, refuse to identify the current test image, and determine that the current test image is an undefined category;

If not, then determine whether the activation value corresponding to the current measurement image is less than a preset threshold;

If so, refuse to identify the current test image, and determine that the current test image is an undefined category;

If not, it is determined that the current test image belongs to a known category, and the current test image is classified into a known category.

A second aspect of the embodiments of the present invention provides an image classification apparatus, including:

a first model obtaining unit, used for training a deep convolutional neural network by using images of known categories to obtain a network training model;

a second model obtaining unit, configured to respectively establish a probability distribution model for each type of sample in the known type of image according to the network training model;

a correction unit, configured to correct the activation value of the known category image according to the probability distribution model;

an activation value obtaining unit, configured to obtain the activation value of the unknown class image according to the activation value of the known class image data;

An image classification determination unit, configured to classify images according to the activation value of the known class image and the activation value of the unknown class image.

In one embodiment, the first model obtaining unit includes:

a data division module, used to divide the acquired images of known categories into a training set and a test set;

a first result generation module, used for training the deep convolutional neural network by using the image data of the training set, and performing a classification performance test on the trained deep convolutional neural network by using the image data of the test set, Output network classification results;

A second result generation module, configured to perform a supervised operation on the network classification result through a loss function, and obtain a supervised operation result;

A parameter adjustment module, configured to adjust the network parameters of the deep convolutional neural network according to the result of the supervision operation.

A third aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program Steps to implement the above image classification method.

A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, implements the steps of the above image classification method.

Compared with the prior art, the embodiments of the present invention have the following beneficial effects: through the embodiments of the present invention, a deep convolutional neural network is trained by using images of known categories to obtain a network training model; A probability distribution model is established for each type of sample in the known category image; the activation value of the known category image is corrected according to the probability distribution model; the activation value of the unknown category image is obtained according to the activation value of the known category image data. According to the activation value of the known category image and the activation value of the unknown category image, the image is classified; through the training of the deep convolutional neural network, the recognition ability of the image is optimized, and the network training model is improved. The performance of image classification; by establishing a probability distribution model for each type of samples and modifying the image activation value, the image classification can be better described, the accuracy and rationality of image classification can be improved, and the unknown classification can be avoided. The problem of images being misclassified; it realizes the reasonable classification of images other than the training set category in the known category images in practical applications; it has strong ease of use and practicability.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present invention. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a schematic diagram of an implementation flowchart of an image classification method provided in Embodiment 1 of the present invention;

2 is a schematic diagram of network supervision provided by a loss function according to Embodiment 1 of the present invention;

3 is a schematic diagram of an implementation process flow of acquiring a network training model provided by Embodiment 1 of the present invention;

4 is a schematic diagram of an implementation process flow of establishing a probability distribution model provided by Embodiment 1 of the present invention;

FIG. 5 is a schematic flowchart of an implementation of a modified activation value provided in Embodiment 1 of the present invention;

6 is a schematic diagram of an implementation flowchart for classifying images provided in Embodiment 1 of the present invention;

7 is a schematic diagram of an image classification apparatus provided in Embodiment 2 of the present invention;

FIG. 8 is a schematic diagram of a terminal device provided by an embodiment of the present invention.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

It is to be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described feature, integer, step, operation, element and/or component, but does not exclude one or more other features , whole, step, operation, element, component and/or the presence or addition of a collection thereof.

It is also to be understood that the terminology used in this specification of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise.

The term "comprising" and any variations thereof in the description and claims of the present invention and the above drawings are intended to cover non-exclusive inclusions. For example, a process, method or system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes Other steps or units inherent in these processes, methods, products or devices. Also, the terms "first," "second," and "third," etc. are used to distinguish between different objects, rather than to describe a particular order.

It should further be understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items .

In order to illustrate the technical solutions of the present invention, the following specific embodiments are used for description.

Referring to FIG. 1, it is a schematic diagram of the implementation process of the image classification method provided by the embodiment of the present invention. The method aims to solve the problem that the current image classification method cannot be directly extended and applied to identify image categories that have not appeared in the network model training process. The image classification method Lack of migration and flexibility issues. This method can not only realize reasonable and correct identification and classification of image categories that have been discovered and recorded, but also select images of unknown categories, which meets the actual needs of open-set image recognition application scenarios. As shown in the figure, the method includes the following steps:

In step S101, a deep convolutional neural network is trained by using images of known categories to obtain a network training model.

In the embodiment of the present invention, the image of the known category is a selected picture of a fixed category, and the deep convolutional neural network is trained under the condition of the closed set; the categories of the images under the condition of the open set are all known category, while the images in the open set condition include images of unknown categories. Known category images include training images and test images for deep convolutional neural networks, and the category space of training images is exactly the same as that of test images; for example, known category images include 500 images, including a total of 5 categories, Each category includes 100 images, of which 80 belong to training images and 20 belong to testing images.

In addition, according to the number of categories of the dataset of known category images and the complexity of the dataset, the corresponding classification basic network is selected. For example, the selected basic network may include the residual network ResNet50 network, but is not limited to this network.

It should be noted that in the process of training the deep convolutional neural network through the known category images, a network supervision part is also set for the output results of the network training model; the network supervision part calculates the loss function of the deep convolutional neural network through the loss function. value, backpropagation realizes the gradient rotation of the network training model, optimizes the deep convolutional neural network and updates the parameters, and improves the image recognition ability of the deep convolutional neural network in the training stage. The loss function can include the superposition of various loss functions. For example, the cross-entropy loss function and the center loss function are used to jointly supervise the network classification results. As shown in Figure 2, the loss function is a schematic diagram of network supervision. After the image is outputted by the network training model, the network training model is supervised by the cross entropy loss function and the central loss function, and the gradient is reversed by using the supervision result to promote the update of the parameters of the network training model and improve the image recognition ability of the network training model.

As a specific implementation example of step S101, as shown in the schematic diagram of the implementation flow of obtaining a network training model as shown in FIG. 3 , the deep convolutional neural network is trained through known category images, and the network training model is obtained, including:

Step S1011, dividing the acquired images of known categories into a training set and a test set.

Step S1012: Train the deep convolutional neural network by using the images in the training set, test the classification performance of the deep convolutional neural network by using the images in the test set, and output a network classification result.

In this embodiment, a dataset of images of known categories is collected and sorted, and the dataset is divided as a closed space set, which can be divided into training set and test; for example, the dataset of images of known categories includes 500 images, a total of Contains 5 categories, each category includes 100 images, of which 80 belong to the training set and 20 belong to the test set; the test set has the same category space as the training, and both contain 5 categories of images. The deep convolutional neural network is trained through the images of the training set, the classification performance of the deep convolutional neural network is detected by the images of the test set, and the network classification results are output.

Step S1013, performing a supervised operation on the network classification result through a loss function to obtain a supervised operation result.

In this embodiment, the set loss function may be a superposition of multiple loss functions, including a cross-entropy loss function (Cross-Entropy Loss), and the expression is:

Among them, x is the image input to the network training model, C is the total number of image categories in the dataset, y _i indicates whether the input image belongs to the i-th category image (1 means belonging, 0 means not belonging), P(y _i |x ) represents the probability that the input image belongs to the i-th class.

The set loss function also includes a center loss function (Center Loss), which supervises the network classification results. The center loss function is expressed as:

L _center = 0.5*||xx _c || (2),

Among them, x represents the feature extracted by the input image through the deep convolutional neural network, and x _c represents the central feature vector of the category to which the input image belongs.

In the training process of the deep convolutional neural network, combined with the above two loss functions, the two loss functions are used to jointly supervise the deep convolutional neural network, so as to obtain the comprehensive loss function, which is expressed as:

L _total = λ _{cross_entropy} L _{cross_entropy} + λ _center L _center (3),

Among them, λ _{cross_entropy} and λ _center respectively represent the corresponding weights of the two loss functions, usually λ _{cross_entropy} is set to 1, and λ _center is set to 8×10 ^-7 .

Through the set loss function, the loss function value of the deep convolutional neural network is calculated, and the loss function value is back-propagated to realize the gradient return of the deep convolutional neural network, thereby generating the supervised operation result.

Step S1014: Adjust the network parameters of the deep convolutional neural network according to the result of the supervision operation.

In this embodiment, the network parameters are hyperparameters set before the deep convolutional neural network learning, including but not limited to the learning rate and batch size of the deep convolutional neural network, and parameters are obtained by selecting appropriate hyperparameters for training The data is supervised by the loss function, and the hyperparameters of the deep convolutional neural network that are set are continuously updated and optimized, and better hyperparameters are selected to improve the image recognition and classification performance of the network.

Through this embodiment, a variety of loss functions are used to jointly supervise the network classification results. In addition to the introduction of the cross entropy function, a central loss function is also set, which is beneficial to reduce the intra-class distance of each type of samples in the image during the training process. , increase the inter-class distance between samples of different categories, and make the classification results more distinguishable; at the same time, the center loss function is also conducive to the calculation of the center vector of each class of samples in the future; when synthesizing the ratio of the two loss functions, the two These loss functions are combined in a weighted way, which improves the accuracy of image classification of the network training model.

Step S102 , respectively establishing a probability distribution model for each type of sample in the known type of image according to the network training model.

In this embodiment, it is known that the distribution of each class of samples in the class images has a certain probability, and a corresponding probability distribution model is established for each class of samples through the trained network training model. Before determining the probability distribution model , it is necessary to calculate the model parameters corresponding to this class of samples through the existing samples, and then determine the probability distribution of the samples in each class, so as to provide a probability judgment for determining whether the image belongs to a known class.

Through the probability distribution model, the analysis of the sample distribution of each category of the image can better describe the image classification, determine whether the image belongs to a certain category according to the probability, and improve the accuracy of the image classification of the network training model.

As a specific implementation example of step S102, as shown in FIG. 4 , as shown in the schematic diagram of the implementation flow of establishing a probability distribution model, a probability distribution model is respectively established for each type of sample in the known category image according to the network training model, include:

Step S1021: Obtain the mean vector of each type of sample in the known type of image.

In this embodiment, the correctly classified samples in the training set among the known categories of images are selected, the activation value (activation) of the input image is calculated through the network training model, the input image is the correctly classified sample, and the activation value of each type of sample activation value is calculated. The average value is obtained to obtain the mean vector of each type of samples. The calculation formula is:

Among them, uc is the mean vector of the samples of the _c -th class, and N _c is the number of correctly classified samples of the c-th class, is the activation value of the nth sample in the c-th sample.

The activation value of each class of correctly classified samples is extracted through the network training model, and the mean vector corresponding to each class of samples is calculated.

Step S1022: Calculate the distance between each class of samples in the known class image and the mean vector.

In this embodiment, in the training set of images of known categories, for the correctly classified samples, the distance between each category of samples and the mean vector of the category of samples is calculated, and the distance may include Euclidean distance (Euclidean distance) and Cosine distance, which is not limited here. Taking the calculation methods of Euclidean distance and cosine distance as an example, the calculated Euclidean distance d _euclidea and cosine distance d _cosine are combined by weighting as the final distance. The calculation formula is:

d _total = λ _euclidean d _euclidean + λ _cosine d _cosine (5);

Among them, λ _euclidean and λ _cosine are the weighting coefficients corresponding to the two distances, respectively, λ _euclidean is set to 1/200, and λ _cosine is set to 1.

Step S1023, according to the distance, select a number of input samples from each type of samples according to a preset ratio.

In this embodiment, for the final distance calculated for each type of samples, according to the extreme value estimation theory, the calculated final distances are sorted by size, a number of samples are obtained in a preset ratio, and the selected samples are the corresponding final distances. The samples with the highest distance value, that is, the samples with larger distances.

The formula for obtaining a number of samples according to a preset ratio is:

N _df =λN _c (6);

Among them, N _c is the number of samples that are correctly classified in the c class, λ is the proportion of samples selected from the c class samples, λ can be 20% to 40%, and N _df is the c class sample selected for use as a Enter the number of samples.

Several selected samples of a certain class are used as input samples of the probability distribution model, and model parameters in the probability distribution model corresponding to the samples of this class are estimated according to the input samples.

Step S1024, estimating model parameters of the probability distribution model corresponding to the input sample categories according to the plurality of input samples.

In this embodiment, the probability distribution model may be a Weber distribution model, and the expression of the probability distribution model is:

Among them, w _n is the probability that a certain type of input sample is input into the Weber distribution model, a ^test is the activation value extracted from the input sample, α is the number of selected sample categories with larger activation values, and the value range of n is [ 1,α], γ is the control coefficient of the probability of the Weber distribution model, τ _n , κ _n , λ _n represent the parameters of the nth Weber distribution model.

Input the selected input samples into the above expression (7), and by controlling the sample ratio of the tail of the Weber distribution, the parameters of the Weber distribution model corresponding to the sample category can be calculated. The corresponding Weber distribution model.

Through this embodiment, when determining the probability distribution model of each type of sample, first calculate the mean vector of each type of sample, that is, the center vector, calculate the distance between this type of sample and this type of sample mean vector, and select samples according to a certain proportion. , as the model parameters of the estimated probability distribution model for the input samples; the combination of Euclidean distance and cosine distance is used in the calculation of the distance, and the corresponding probability model is estimated by the distance between each type of sample and the corresponding center vector based on the angle of the sample probability distribution. , which improves the estimation accuracy of the probability distribution model; whether the input test sample belongs to a certain class of samples and the corresponding probability of belonging to this class of samples can be obtained according to the probability distribution model, which improves the performance of image classification and the correctness of image classification. Practical.

Step S103, correcting the activation value of the known category image according to the probability distribution model.

In this embodiment, according to the images of the training set and the test set of known images, by inputting the test image into the network training model, the activation value corresponding to the test image can be extracted, and the extracted activation value is in the normalized function The activation value of the output before processing.

The probability value that a test image of a certain category belongs to the category is calculated by the probability distribution model, and the corrected activation value is obtained by multiplying the probability value by the activation value of the test image.

Specifically, as shown in the schematic flowchart of the implementation of correcting the activation value as shown in FIG. 5 , correcting the activation value of the known category image according to the probability distribution model includes:

Step S1031, extracting the first activation value of the test sample in the known category image through the network training model.

In this embodiment, the test samples of different categories in the known category graph are input into the network training model, and the first activation value a ^test of the test samples of different categories can be extracted, and the first activation value is in the pair test. The sample activation value is the activation value before normalization function processing.

It should be noted that, in this embodiment, the first activation value does not represent only one activation value, but multiple activation values obtained for different categories of test samples that have not been processed by a normalization function.

Step S1032: Select a preset number of sample categories from the test samples according to the first activation value.

In this embodiment, according to the extreme value estimation theory, for multiple sample categories, the obtained first activation values are sorted by size, and a preset number of sample categories with larger activation values are selected. The number of sample categories is α.

Step S1033, according to the probability distribution model corresponding to the preset number of sample categories and the first activation value of the test samples in the sample category, calculate the belonging of the test samples in the preset number of sample categories probability.

In this embodiment, according to the expression (7) of the probability distribution model determined in step S1024, and the activation value of the test sample selected in the sample category with a larger activation value, it is calculated that the selected test sample corresponds to a certain The probability of belonging to the category.

It should be noted that the probability control coefficient γ is added to the probability distribution model, which reduces the probability value and avoids the occurrence of the activation value of a specific category of test samples occupying the main component, so that some samples are misclassified as unknown picture categories. .

Step S1034, correcting the first activation value of the test samples in the preset number of sample categories according to the belonging probability, and obtaining a second activation value.

In this embodiment, the probability of belonging to the test sample is obtained through the probability distribution model, and the activation value of the selected test sample of the sample category with a larger activation value is corrected according to the probability of belonging, and the revised calculation formula is:

in, is the first activation value of the test sample in the selected sample category with a larger activation value, w _c is the corresponding probability of the test sample, is the second activation value of the test sample.

Through this embodiment, by selecting a sample category with a larger activation value of the test sample in the known category images, the activation value of the test sample is input into the probability distribution model to calculate the probability of belonging of the test sample, and the original activation value of the test sample is calculated according to the probability of belonging. Correction, obtain the corrected activation value, better control the size of the activation value of the test sample, and prevent the sample activation value of a specific category from occupying the main component, which leads to the problem that some samples are wrongly classified into unknown image categories, which improves the Image classification accuracy.

Step S104, obtaining the activation value of the unknown category image according to the activation value of the known category image data.

In this embodiment, for the selected α sample categories, the activation value of the image of the unknown category is constructed according to the first activation value and the second activation value of the test sample.

Specifically, obtaining the activation value of the unknown category image according to the activation value of the known category image data includes:

Calculate the activation value of the unknown category image according to the first activation value and the second activation value of the test sample in the sample category The calculation formula is:

in, is the first activation value of the test sample, is the corrected second activation value, C is the total number of categories of images of known categories, and c is any c-th sample in the total number of categories.

Step S105, classify the images according to the activation value of the known category image and the activation value of the unknown category image.

In this embodiment, through the construction of the activation value of the unknown category image, combined with the activation value of the known category image, the activation value of C+1 dimension is formed, where C is the total number of categories under the condition of the closed set of the known category image; The activation value of C+1 dimension is normalized, and the image classification under the condition of closed set is extended to the image classification under the condition of open set.

Specifically, as shown in the schematic diagram of the implementation process of classifying images as shown in FIG. 6 , classifying images according to the activation value of the known category image and the activation value of the unknown category image includes:

Step S1051, normalize the activation value of the image of the known category and the activation value of the image of the unknown category to obtain a new activation value of the image.

In this embodiment, the normalization function softmax function is used to normalize the corrected C+1 micro-activation value, that is, the activation value of the known category image and the activation value of the unknown category image are both normalized. , to obtain a new C+1-dimensional activation value with a new range from 0 to 1, the new activation value includes the activation value of the unknown category image, and the calculation formula of the new activation value is:

in, is the activation value of the corrected image of class c, and p _c is the probability that the current test image belongs to the image of class c after normalization.

According to formula (10), the probability vector P _c of C+1 dimension can be obtained.

Step S1052: Select the pending category value corresponding to the current test image with the largest activation value from the new activation values.

In this embodiment, the C+1-dimensional probability vector is indexed, the largest activation value among the new activation values is extracted, the current test image corresponding to the largest activation value is obtained, and the pending category value c corresponding to the current test image is further obtained. ^* .

Step S1053, judging whether the pending category value corresponds to the unknown category value.

In this embodiment, according to the acquired pending category value c ^* corresponding to the current test image, it is determined whether the pending category value c ^* is equal to the unknown category value C+1.

Step S1054, if yes, refuse to identify the current test image, and determine that the current test image is of an undefined category.

In this embodiment, if the undetermined category value c ^* is equal to the unknown category value C+1, the current test image is rejected and identified, and the current test image is determined to be an undefined category, thereby realizing image classification in the case of an open set of images.

Step S1055, if no, determine whether the activation value corresponding to the current measurement image is less than a preset threshold.

In this embodiment, the threshold for image recognition is set, the normalized activation value of the current test image is extracted, and it is determined whether the activation value of the current test image is smaller than the preset threshold for image recognition. The preset threshold may be a rejection threshold, that is, a threshold at which the system refuses to recognize an image; or it may be an acceptance threshold, that is, a threshold that the system can accept the image recognition result.

Step S1056, if yes, refuse to identify the current test image, and determine that the current test image is of an undefined category.

In this embodiment, if the activation value corresponding to the current test image is smaller than the preset threshold, the current test image is refused to be recognized, and the current test image is judged to be of an undefined category; thus, when encountering images of known categories Images outside the training set can also be classified reasonably correctly.

Step S1057, if not, it is determined that the current test image belongs to a known category, and the current test image is classified into a known category.

In this embodiment, if the current test image does not belong to the undefined category, the images of the known category are classified according to the closed-set image situation, and it is determined that the current test image belongs to the known category c ^* .

Through this embodiment, a certain category of pictures is selected to train a deep convolutional neural network model under the condition of closed-set images, and by selecting appropriate hyperparameters, the network parameters are updated to obtain a network training model with good image classification performance; The probability distribution model corresponding to each sample category is estimated below, and the sample distribution probability of each category is reasonably described; the activation value of the image under the condition of the closed set image is corrected by the sample distribution probability, the activation value of the unknown category is constructed, and all activation values are normalized. It uses the normalized activation value to identify the image under the condition of the boot image, realizes the rejection of the image of the unknown category, and correctly classifies the image of the known category, which improves the rationality and accuracy of the image classification and meets the actual needs. The needs of application scenarios are more practical.

It should be noted that, those skilled in the art can easily think of other sorting solutions within the technical scope disclosed by the present invention, which should also be within the protection scope of the present invention, and will not be repeated here.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

Embodiment 2

Referring to FIG. 7 , it is a schematic diagram of an image classification apparatus provided by an embodiment of the present invention. For convenience of description, only parts related to the embodiment of the present invention are shown.

The image classification device includes:

The first model obtaining unit 71 is used for training the deep convolutional neural network by using images of known categories to obtain a network training model;

A second model obtaining unit 72, configured to respectively establish a probability distribution model for each type of sample in the known type of image according to the network training model;

A correction unit 73, configured to correct the activation value of the known category image according to the probability distribution model;

an activation value obtaining unit 74, configured to obtain the activation value of the unknown class image according to the activation value of the known class image data;

The image classification determination unit 75 is configured to classify images according to the activation value of the known class image and the activation value of the unknown class image.

Optionally, the first model obtaining unit includes:

a data division module, used to divide the acquired images of known categories into a training set and a test set;

a first result generation module, configured to train the deep convolutional neural network through images in the training set, test the classification performance of the deep convolutional neural network through images in the test set, and output a network classification result;

A second result generation module, configured to perform a supervised operation on the network classification result through a loss function, and obtain a supervised operation result;

A parameter adjustment module, configured to adjust the network parameters of the deep convolutional neural network according to the result of the supervision operation.

Through this embodiment, a certain category of pictures is selected to train a deep convolutional neural network model under the condition of closed-set images, and by selecting appropriate hyperparameters, the network parameters are updated to obtain a network training model with good image classification performance; The probability distribution model corresponding to each sample category is estimated below, and the sample distribution probability of each category is reasonably described; the image activation value under the condition of closed-set image is corrected by the sample distribution probability, the activation value of the unknown category is constructed, and all activation values are normalized. It uses the normalized activation value to identify the image under the condition of the boot image, realizes the rejection of the image of the unknown category, and correctly classifies the image of the known category, which improves the rationality and accuracy of the image classification and meets the actual needs. application scenario requirements.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated by different functional units and modules as required. , that is, dividing the internal structure of the mobile terminal into different functional units or modules to complete all or part of the functions described above. Each functional module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit, and the above-mentioned integrated units may be implemented in the form of hardware. , can also be implemented in the form of software functional units. In addition, the specific names of the functional modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working process of the modules in the above mobile terminal, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

Embodiment 3

FIG. 8 is a schematic diagram of a terminal device provided by an embodiment of the present invention. As shown in FIG. 8 , the terminal device 8 of this embodiment includes: a processor 80 , a memory 81 , and a computer program 82 stored in the memory 81 and executable on the processor 80 . When the processor 80 executes the computer program 82, the steps in each of the above embodiments of the graphic classification method are implemented, for example, steps 101 to 105 shown in FIG. 1 . Alternatively, when the processor 80 executes the computer program 82, the functions of the modules/units in the above-mentioned apparatus embodiments, for example, the functions of the modules 71 to 75 shown in FIG. 7 are implemented.

Exemplarily, the computer program 82 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 81 and executed by the processor 80 to complete the this invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 82 in the terminal device 8 . For example, the computer program 82 can be divided into a first model acquisition unit, a second model acquisition unit, a correction unit, an activation value acquisition unit, and an image classification determination unit. The specific functions of each module are as follows:

a first model obtaining unit, used for training a deep convolutional neural network by using images of known categories to obtain a network training model;

a second model obtaining unit, configured to respectively establish a probability distribution model for each type of sample in the known type of image according to the network training model;

a correction unit, configured to correct the activation value of the known category image according to the probability distribution model;

an activation value obtaining unit, configured to obtain the activation value of the unknown class image according to the activation value of the known class image data;

An image classification determination unit, configured to classify images according to the activation value of the known class image and the activation value of the unknown class image.

The terminal device 8 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The terminal device may include, but is not limited to, the processor 80 and the memory 81 . Those skilled in the art can understand that FIG. 8 is only an example of the terminal device 8, and does not constitute a limitation on the terminal device 8, and may include more or less components than those shown in the figure, or combine some components, or different components For example, the terminal device may further include an input and output device, a network access device, a bus, and the like.

The so-called processor 80 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 81 may be an internal storage unit of the device/terminal device 8 , such as a hard disk or a memory of the terminal device 8 . The memory 81 may also be an external storage device of the terminal device 8, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) equipped on the terminal device 8. card, flash card (Flash Card) and so on. Further, the memory 81 may also include both an internal storage unit of the terminal device 8 and an external storage device. The memory 81 is used to store the computer program and other programs and data required by the terminal device. The memory 81 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit, and the above-mentioned integrated units may adopt hardware. It can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present invention. For the specific working processes of the units and modules in the above-mentioned system, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the apparatus/terminal device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units. Or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated modules/units, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable media Excluded are electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it is still possible to implement the foregoing implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the within the protection scope of the present invention.

Claims (10)

1. An image classification method, comprising:

training the deep convolution neural network through the known class image to obtain a network training model;

respectively establishing a probability distribution model for each type of sample in the known type of image according to the network training model;

correcting the activation value of the known class image according to the probability distribution model;

acquiring an activation value of an unknown class image according to the activation value of the known class image data;

and classifying the images according to the activation values of the known class images and the activation values of the unknown class images.

2. The image classification method of claim 1, wherein training the deep convolutional neural network with known class images to obtain a network training model comprises:

dividing the acquired known class images into a training set and a test set;

training the deep convolutional neural network through the images of the training set, testing the classification performance of the deep convolutional neural network through the images of the test set, and outputting a network classification result;

performing supervision operation on the network classification result through a loss function to obtain a supervision operation result;

and adjusting the network parameters of the deep convolutional neural network according to the supervision operation result.

3. The image classification method according to claim 1, wherein the establishing of the probability distribution model for each class of samples in the known class of images according to the network training model comprises:

obtaining a mean vector of each type of sample in the known type of image;

calculating the distance between each type of sample in the known type of image and the mean vector;

selecting a plurality of input samples from each type of samples according to the distance and a preset proportion;

and estimating model parameters of the probability distribution model corresponding to the input sample category according to the input samples.

4. The image classification method of claim 1, wherein modifying the activation values of the images of the known class according to the probability distribution model comprises:

extracting a first activation value of a test sample in the known class image through the network training model;

selecting a preset number of sample categories from the test samples according to the first activation value;

calculating the probability of the test samples in the sample classes of the preset number according to the probability distribution model corresponding to the sample classes of the preset number and the first activation value of the test samples in the sample classes;

and correcting the first activation values of the test samples in the preset number of sample categories according to the probability to obtain second activation values.

5. The image classification method of claim 4, wherein obtaining the activation value of the unknown class of image from the activation value of the known class of image data comprises:

calculating the activation value of the unknown class image according to the first activation value and the second activation value of the test samples in the selected preset number of sample classesThe calculation formula is as follows:

wherein,to test the first activation value of the sample,and C is any class C sample in the total class number.

6. The image classification method according to claim 1, wherein classifying an image according to the activation value of the known class image and the activation value of the unknown class image comprises:

normalizing the activation value of the image of the known type and the activation value of the image of the unknown type to obtain a new activation value of the image;

selecting a pending class value corresponding to the current test image with the maximum activation value from the new activation values;

judging whether the undetermined class value corresponds to the unknown class value;

if so, refusing to identify the current test image, and judging that the current test image is in an undefined class;

if not, judging whether the activation value corresponding to the current image to be detected is smaller than a preset threshold value;

if so, refusing to identify the current test image, and judging that the current test image is in an undefined class;

if not, judging that the current test image belongs to a known class, and classifying the current test image according to the known class.

7. An image classification apparatus, comprising:

the first model acquisition unit is used for training the deep convolution neural network through the known class image to acquire a network training model;

the second model obtaining unit is used for respectively establishing a probability distribution model for each type of sample in the known type of image according to the network training model;

the correcting unit is used for correcting the activation value of the known class image according to the probability distribution model;

the activation value acquisition unit is used for acquiring the activation value of the unknown class image according to the activation value of the known class image data;

and the image classification judging unit is used for classifying the images according to the activation values of the known class images and the activation values of the unknown class images.

8. The image classification device according to claim 7, wherein the first model acquisition unit includes:

the data dividing module is used for dividing the acquired known class images into a training set and a test set;

the first result generation module is used for training the deep convolutional neural network through the images of the training set, testing the classification performance of the deep convolutional neural network through the images of the test set and outputting a network classification result;

the second result generation module is used for carrying out supervision operation on the network classification result through a loss function to obtain a supervision operation result;

and the parameter adjusting module is used for adjusting the network parameters of the deep convolutional neural network according to the supervision operation result.

9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.