CN115099294A - Flower image classification algorithm based on feature enhancement and decision fusion - Google Patents
Flower image classification algorithm based on feature enhancement and decision fusion Download PDFInfo
- Publication number
- CN115099294A CN115099294A CN202210275922.8A CN202210275922A CN115099294A CN 115099294 A CN115099294 A CN 115099294A CN 202210275922 A CN202210275922 A CN 202210275922A CN 115099294 A CN115099294 A CN 115099294A
- Authority
- CN
- China
- Prior art keywords
- decision
- equal
- fusion
- dec
- stage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 44
- 238000007635 classification algorithm Methods 0.000 title claims abstract description 15
- 238000012549 training Methods 0.000 claims abstract description 35
- 238000010606 normalization Methods 0.000 claims abstract description 14
- 238000000034 method Methods 0.000 claims abstract description 13
- 238000012360 testing method Methods 0.000 claims description 33
- 238000010586 diagram Methods 0.000 claims description 21
- 238000012795 verification Methods 0.000 claims description 21
- 230000004044 response Effects 0.000 claims description 13
- 125000000205 L-threonino group Chemical group [H]OC(=O)[C@@]([H])(N([H])[*])[C@](C([H])([H])[H])([H])O[H] 0.000 claims description 7
- 230000002708 enhancing effect Effects 0.000 claims description 7
- 230000000694 effects Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 abstract description 7
- 238000007781 pre-processing Methods 0.000 abstract description 7
- 230000006870 function Effects 0.000 description 5
- 241000219000 Populus Species 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000000873 masking effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000006386 neutralization reaction Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a flower image classification algorithm based on feature enhancement and decision fusion, and belongs to the technical field of image classification. The method comprises a data preprocessing stage, a feature extraction and enhancement stage, a training stage and a decision fusion classification stage. In the data preprocessing stage, corresponding operations such as cutting, scaling, normalization and the like are carried out on the data set; in the characteristic extraction and enhancement stage, the preprocessed data is sent into a VGG16 model pre-trained by ImageNet to extract the characteristics of the multilayer depth image, and a characteristic enhancement strategy is introduced to adaptively distribute the characteristic weight; in the training stage, a plurality of user-defined softmax classifiers are trained by using a plurality of groups of features in the previous stage; and in the decision fusion classification stage, information entropy is introduced to express the certainty degree of each classifier, fusion weight is determined according to the information entropy, and fusion decision is performed to realize classification. The features of the invention which are subjected to self-adaptive enhancement have stronger expression capability, and have stronger classification capability compared with the traditional softmax classifier.
Description
Technical Field
The invention relates to a flower image classification algorithm based on feature enhancement and decision fusion, and belongs to the technical field of image classification.
Background
Image classification is a fundamental problem in computer vision, and is the basis of image positioning, image detection, image segmentation and other technologies. There are many image classification algorithms, such as classification algorithms based on artificially defined features, algorithms based on neural networks such as convolutional neural networks, antagonistic neural networks, attention-driven mechanisms, and the like.
The quality of the extracted features is a key for determining the classification effect, and previous research algorithms usually focus on improving image preprocessing methods, network structures, activation functions and the like, and aim to extract more excellent image features. However, these efforts are less focused on optimizing the extracted features and a plurality of softmax classifiers are jointly used for decision-related research, so that the classification accuracy is to be improved.
Disclosure of Invention
The invention aims to solve the technical problem of providing a flower image classification algorithm based on feature enhancement and decision fusion, so as to solve the problem of misclassification caused by insufficient feature expression and excessive breakage of a softmax classifier.
The technical scheme of the invention is as follows: a flower image classification algorithm based on feature enhancement and decision fusion comprises a data preprocessing stage, a feature extraction and enhancement stage, a training stage and a decision fusion classification stage. And in the data preprocessing stage, corresponding operations such as cutting, scaling, normalization and the like are carried out on the data set. And in the characteristic extraction and enhancement stage, the preprocessed data is sent into a VGG16 model pre-trained by ImageNet to extract the characteristics of the multilayer depth image, and a characteristic enhancement strategy is introduced to adaptively distribute characteristic weights. The training phase trains a plurality of custom softmax classifiers using the sets of features of the previous phase. And the decision fusion classification stage introduces information entropy to express the certainty degree of each classifier, determines a fusion weight according to the information entropy, and fuses decisions to realize classification.
The method comprises the following specific steps:
step 1: and carrying out corresponding operations of cutting, scaling and normalization on the data set.
The Step1 specifically comprises the following steps:
step1.1: the image data set is divided into a training set, a verification set and a test set.
Step1.2: and (3) cutting each image in the training set into k square images by adopting a normal random cutting strategy (each image can be cut for the required times to expand the data of the training set), and cutting each image in the verification set and the test set into one square image by adopting a central cutting strategy.
Step1.3: and (3) scaling the images in the training set, the test set and the verification set after the cropping to be 224 × 224 of the size required by the VGG16 model by using a Lanczos interpolation mode.
Step1.4: and linearly transforming the pixel values of the images in the training set, the verification set and the test set after the scaling from the range of [0,255] to the range of [0,1] by adopting a maximum and minimum normalization mode.
Step 2: sending the preprocessed data into a VGG16 model pre-trained by ImageNet to extract multilayer high-level features, and introducing a feature enhancement strategy to adaptively distribute feature weights so as to realize the effect of adaptively enhancing the features.
The Step2 is specifically as follows:
step2.1: and removing three full-connection layers of the pre-trained VGG16 model, and sending the training set, the verification set and the test set to extract high-level features.
Step2.2: the characteristics of the block i _ convj layer and the block5_ pool layer of the training set, the verification set and the test set are respectively saved.
Step2.3: and self-adaptive enhancing is carried out on the features of the blocki _ convj layer by using a feature enhancing masking strategy to obtain blocki _ convj _ en.
If the block i _ convj output characteristic diagram is assumed to be M ij ＝F ij The output characteristic diagram of (x, y, p) block i _ convj _ en is M ij_en ＝F ij_en (x, y, p), introduction of feature enhancement mask M en ＝F en (x,y,p)，M ij_en ＝M ij ×M en Wherein x is more than or equal to 1, y is more than or equal to W, P is more than or equal to 1 and less than or equal to P, wherein W represents the side length of the characteristic diagram, and P represents the channel number of the characteristic diagram.
Step2.4: and splicing the block i _ convj _ en layer and the block5_ pool layer into a new tensor Concat _ ij.
The Step2.3 specifically comprises the following steps:
step2.3.1: block i _ convj layer feature map, called M ij The dimension is W × W × P, that is, each of the P channels has a pixel matrix with the size of W × W, and the P channels are compressed into one channel to obtain a superposition map M with the dimension of W × W stack And calculating an average M thereof average Wherein M is ij ＝F ij (x, y, p) if one of the channel characteristics is marked as m p Then M is stack ＝m 1 +m 2 +…+m P ,M average ＝M stack /P。
Step2.3.2: order to
Representing a threshold that distinguishes high response areas from non-high response areas, the high response areas being more likely to be flower expressing areas.
Step2.3.3: if M is ij If F (x, y, p) is greater than or equal to thres, then M en ＝1+(M ij (x, y, p) -thres)/(a-thres). If M is ij If F (x, y, p) < thres, then M en B, wherein a and b are hyperparameters.
Step2.3.4：M ij_en ＝M ij ×M en 。
Step 3: multiple custom softmax classifiers are trained using the previous stage's sets of features.
The Step3 is specifically as follows:
step3.1: and connecting a global average pooling layer (GAP) with a full connection layer containing 1024 nodes, and constructing a softmax classifier by using a softmax function as an activation function.
Step3.2: and (3) respectively accessing the tensor Concat _ ij into the classifier, and training and verifying by adopting the output characteristics of the block i _ convj and the block5_ pool layers.
Step3.3: and storing the model trained by Concat _ ij to obtain a decision model dec _ brij.
Step 4: and introducing information entropy to express the certainty degree of each classifier, so that decision fusion weight is adaptively distributed, and classification is realized through decision fusion.
The Step4 is specifically as follows:
step4.1: testing the images of the test set by using dec _ brij to obtain the probability that each sample belongs to each class, and for a sample x n Test result p of decision branch dec _ brij ij (x n )＝[p ij1 (x n )p ij2 (x n )…p ijL (x n )]Wherein L represents the total number of classes of the sample, p ijl (x n ) Prediction sample x representing decision dec _ brij n Probability of belonging to class i.
Step4.2: the entropy h (x) of information is introduced to represent the degree of certainty that the decision model has for the decision,
wherein i is more than or equal to 1 and less than or equal to 5, and j is more than or equal to 1 and less than or equal to 3.
H ij (x) Indicating the degree of certainty that the decision branch dec _ brij has with respect to the classification result, p ijl (x) Indicating the probability that the decision branch dec _ brij decides sample x as class i.
Step4.3: a decision weight ω is introduced to determine a specific method of decision fusion, where ω is defined as follows,
wherein i is more than or equal to 1 and less than or equal to 5, and j is more than or equal to 1 and less than or equal to 3. Omega ij Indicating the decision weight assigned to dec _ brij.
Step4.4: for sample x n The output result of dec _ brij integrated with the decision weight is
Step4.5: for sample x n The decision fusion probability belonging to each class is P (x) n )，
Representing the probability that the prediction sample x belongs to the class L after decision fusion.
Step4.6: for sample x, the final prediction class label (x) ═ argmax (p (x)).
The invention has the beneficial effects that: the invention combines the characteristic enhancement and the characteristic fusion, combines the softmax classifier and the information entropy, and has the following advantages:
1. the features subjected to self-adaption enhancement have stronger expression capability;
2. compared with the conventional softmax classifier, the soft max classifier has stronger classification capability.
Drawings
FIG. 1 is a diagram of a feature extraction and enhancement network of the present invention;
FIG. 2 is a block diagram of a decision fusion of the present invention;
FIG. 3 is a dimension diagram of a network layer output characteristic diagram, wherein W represents a side length of the characteristic diagram, and P represents a channel number;
FIG. 4 is an overlay visualization of features of the present invention;
FIG. 5 is a feature overlay visualization after feature enhancement of the present invention;
FIG. 6 is a flow chart of the steps of the present invention.
Detailed Description
The invention is further described with reference to the following drawings and detailed description.
As shown in fig. 6, a flower image classification algorithm based on feature enhancement and decision fusion includes a data preprocessing stage, a feature extraction and enhancement stage, a training stage, and a decision fusion classification stage. In the data preprocessing stage, corresponding operations such as cutting, scaling, normalization and the like are carried out on the data set; in the characteristic extraction and enhancement stage, the preprocessed data is sent into a VGG16 model which is pre-trained by ImageNet to extract the characteristics of the multilayer depth image, and a characteristic enhancement strategy is introduced to adaptively distribute characteristic weights; in the training stage, a plurality of user-defined softmax classifiers are trained by using a plurality of groups of characteristics in the previous stage; and in the decision fusion classification stage, information entropy is introduced to express the certainty degree of each classifier, fusion weight values are determined according to the information entropy, and fusion decision classification is realized.
The method comprises the following specific steps:
step 1: and carrying out corresponding operations such as cutting, scaling, normalization and the like on the data set.
Step 2: and sending the preprocessed data into a VGG16 model pre-trained by ImageNet to extract multilayer depth image features, and introducing a feature enhancement strategy to adaptively distribute feature weights so as to realize adaptive enhancement features.
Step 3: multiple custom softmax classifiers are trained using the previous stage's sets of features.
Step 4: and introducing information entropy to express the certainty degree of each classifier, thereby adaptively allocating decision fusion weights and realizing classification through decision fusion.
The Step1 is specifically as follows:
step1.1: the image data set is divided into a training set, a verification set and a test set.
In this example, the public data set oxford-17flowers provided by oxford university was selected from the Visual Geometry Group of oxford university, which is a common 17flowers in the uk. Each flower had 80 pictures and the entire data set had 1360 pictures. Because the number of data sets is small, 50 flowers of each type are randomly selected as a training set, 15 flowers are selected as a verification set, and 15 flowers are selected as a testing set. Furthermore, since the features extracted from four of these classes cannot be used, most of the literature has been tested on 13 classes of flowers, which is also used herein to evaluate the performance of the present invention.
Step1.2: and (3) cutting each image in the training set into k square images by adopting a normal random cutting strategy (each image can be cut for the required times to expand the data of the training set), and cutting each image in the verification set and the test set into one square image by adopting a central cutting strategy.
In this embodiment, the present invention randomly cuts the training set, the verification set, and the test set images 5 times, respectively, and expands the data to 5 times the original data, that is, 3250 training sets, and 975 verification sets, respectively, and 975 test sets.
Step1.3: and scaling the clipped training set, the test set and the verification set to be 224 multiplied by 224 required by the VGG16 model by using a Lanczos interpolation mode.
Step1.4: and linearly transforming the pixel values of the training set, the verification set and the test set after the scaling from the range of [0,255] to the range of [0,1] by adopting a maximum and minimum normalization mode.
In this embodiment, sample x is used 1 For example, as shown in FIG. 1, the sample resolution before normalization is 540X 500, with pixel values in the range of [0,255]]After normalization, the sample resolution is 224 × 224 required by the model, and the pixel value range is [0,1]]As in the left image normalization portion of fig. 1. Pixels before normalization were: [[[56,56,56,...,26,26,26],[57,58,58,...,26,26, 26],[59,60,61,...,26,26,26],...,[6,5,5,...,14,14,14],[6,5,5,...,14,14,14],[6,5, 5,...,13,13,13]],[[78,78,78,...,29,29,29],[79,80,80,...,29,29,29],[82,83,84,...,29,29, 29],...,[6,5,5,...,16,16,16],[6,5,5,...,16,16,16],[6,5,5,...,15,15,15]]] 540×500×3 Where "540 × 500" represents the sample resolution and "3" represents the RGB three channels of samples. Pixels after normalization are [ [ [0.21960784,0.21960784,0.21960784,. ] 9,0.10196079,0.10196079, 0.10196079],[0.22352941, 0.22745098,0.22745098,...,0.10196079,0.10196079,0.10196079],...,[0.02352941,0.01960784, 0.01960784,...,0.05098039,0.05098039,0.05098039]],[[0.11372549,0.11372549,0.11372549,..., 0.03921569,0.03921569,0.03921569],[0.11764706,0.12156863,0.12156863,...,0.03921569, 0.03921569,0.03921569],...,[0.02352941,0.01960784,0.02745098,...,0.04705882,0.04705882, 0.04705882]]] 224×224×3 。
Step2.1: and removing three full connection layers of the pre-trained VGG16 model, and sending the training set, the verification set and the test set into the model to extract high-level features.
Step2.2: the characteristics of the block i _ convj layer and the block5_ pool layer of the training set, the verification set and the test set are respectively saved.
In this embodiment, i in block i _ convj represents the ith network block, j represents the jth convolutional layer, block5_ conv2 represents the 2 nd convolutional layer of the 5 th network block of the VGG16 model, and taking block5_ conv2 as an example, the feature diagram dimensions of the layers are (14, 512), that is, the feature diagram size is 14 × 14, and the number of channels is 512.
Step2.3: and carrying out self-adaptive enhancement on the feature graph of the blocki _ convj layer by using a feature enhancement masking strategy to obtain blocki _ convj _ en.
If the block i _ convj output characteristic diagram is assumed to be M ij ＝F ij The output characteristic diagram of (x, y, p) block i _ convj _ en is M ij_en ＝F ij_en (x, y, p), introduction of feature enhancement mask M en ＝F en (x,y,p)，M ij_en ＝M ij ×M en Wherein x is more than or equal to 1, y is more than or equal to W, P is more than or equal to 1 and less than or equal to P, wherein W represents the side length of the characteristic diagram, and P represents the channel number of the characteristic diagram.
In this embodiment, taking block5_ conv2 and block5_ conv2_ en as examples, block5_ conv2 and block5_ conv2_ en output characteristic diagrams M 52 、M 52_en The dimensions are 14 × 14 × 512, i.e., W is 14 and P is 512.
Step2.4: and splicing the block i _ convj _ en layer and the block5_ pool layer into a new tensor Concat _ ij.
In this embodiment, taking block5_ conv2_ en and block5_ pool layers as examples, the feature graph dimension of the block5_ conv2_ en layer is 14 × 14 × 512, and a one-dimensional tensor a with 512 parameters is obtained through a global average pooling layer. The dimension of the block5_ pool layer characteristic graph is 7 × 7 × 512, and a one-dimensional tensor B with 512 parameters is generated through the global average pooling layer. The two tensors a and B are merged to generate a one-dimensional tensor Concat _52 containing 1024 parameters.
Step2.3.1: block i _ convj layer signature graph (hereinafter referred to as M) ij For example, as shown in fig. 1), dimension W × P (each of P channels has a pixel matrix with size W × W), and compressing P channels into one channel results in a superimposed graph M with dimension W × W stack And calculating an average M thereof average Wherein M is ij ＝F ij (x, y, p) if one of the channel signatures is denoted as m p Then M is stack ＝m 1 +m 2 +…+m P ，M average ＝M stack /P。
In this embodiment, the block5_ conv2 layer feature maps of 512 channels are compressed into one channel, so as to obtain a 14 × 14 overlay map M stack The overlay of the block5_ conv2 layer characteristic diagram is shown in fig. 4.
Step2.3.2: order to
Representing a threshold that distinguishes high response regions from non-high response regions, the high response regions being more likely to be flower expressing regions.
In this embodiment, the threshold thres of the sample for distinguishing the high response region from the non-high response region is 0.24598.
Step2.3.3: if M is ij If F (x, y, p) is greater than or equal to thres, then M en ＝1+(M ij (x, y, p) -thres)/(a-thres). If it is
M ij If F (x, y, p) < thres, then M en ＝b。
In the embodiment, a and b are hyperparameters, a is a factor for enhancing the characteristics of the high-response region and used for adjusting the degree of characteristic enhancement, and b is a factor for weakening the non-high-response region, wherein a is more than or equal to 1 and less than or equal to 2, and b is more than or equal to 0 and less than or equal to 1.
Step2.3.4：M ij_en ＝M ij ×M en 。
In this embodiment, after the adaptive enhancement features are performed, an overlay of feature maps of the block5_ conv2 layer and the block5_ pool layer is shown in fig. 5.
Step3.1: and connecting a global average pooling layer (GAP) with a full connection layer containing 1024 nodes, and constructing a softmax classifier by using a softmax function as an activation function.
Step3.2: and (3) respectively accessing the tensor Concat _ ij into the classifier, and training and verifying by adopting the output characteristics of the block i _ convj and the block5_ pool layers.
In the embodiment, an Adam optimizer is adopted for training, the sample batch is set to be 32, 50 epochs are trained each time, and the model is saved when the average accuracy rate is higher than that of the previous Epoch each time in the training process.
Step3.3: and storing the model trained by Concat _ ij to obtain a decision model dec _ brij.
In this embodiment, for example, dec _ br52 is used, and dec _ br52 represents a decision model obtained by splicing block5_ conv2_ en and block5_ pool to train a classifier.
Step4.1: testing the images of the test set by using dec _ brij to obtain the probability that each sample belongs to each class, and regarding the sample x n Test result p of decision branch dec _ brij ij (x n )＝[p ij1 (x n )p ij2 (x n )…p ijL (x n )]Where L represents the total number of classes of the sample, p ijl (x n ) Prediction sample x representing decision dec _ brij n Probability of belonging to class i.
In this embodiment, taking dec _ br51, dec _ br52 and dec _ br53 as examples, the test result p of dec _ br51 51 (x 1 )＝ [1.0000000e+00,4.1446593e-17,1.9549345e-15,7.9526921e-20,2.0109854e-20,3.1513676e-19, 3.7332855e-20,2.1863510e-23,1.3979170e-16,2.2339542e-20,9.2374559e-15, 5.8055036e-11,3.0085874e-23]。
Test result p of dec _ br52 52 (x 1 )＝[9.9999547e-01,7.5567144e-09,7.8939694e-10, 5.6319975e-17,7.2288855e-13,3.3209550e-14,4.4501590e-12,7.1541192e-16,5.3460147e-10, 4.9500820e-12,6.0213007e-10,4.4727053e-06,1.1179825e-15]。
Test result p of dec _ br53 53 (x 1 )＝[9.9999964e-01,8.5260909e-11,6.1670391e-08, 6.0080515e-14,2.3101764e-11,6.7670508e-11,1.8991200e-09,4.3540345e-12,2.2171498e-07, 7.6678863e-10,3.7856211e-09,3.0988421e-08,1.2395125e-10]。
Step4.2: introducing information entropy H (x) to representThe degree of certainty that the decision model has with respect to the decision,
wherein i is more than or equal to 1 and less than or equal to 5, and j is more than or equal to 1 and less than or equal to 3. H ij (x) Indicating the degree of certainty that the decision branch dec _ brij has with respect to the classification result, p ijl (x) Indicating the probability that the decision branch dec _ brij decides sample x as class i.
In this embodiment, the information entropy H of dec _ br51 51 (x 1 ) Information entropy H of dec _ br 52-3.986313764776386 e-08 52 (x 2 ) Information entropy H of dec _ br 53-8.630309145106366 e-05 53 (x 3 )＝7.871773456765137e-06。
Step4.3: a specific method for determining decision fusion by introducing a decision weight ω, which is defined as follows,
wherein i is more than or equal to 1 and less than or equal to 5, and j is more than or equal to 1 and less than or equal to 3. Omega ij Indicating the decision weight assigned to dec _ brij.
In this embodiment, the decision weight ω corresponding to dec _ br51 51 0.33334378825983924, dec _ br52 corresponds to the decision weight ω 52 0.3333150341887531, dec _ br53 corresponds to the decision weight ω 53 ＝0.33334117755140763。
Step4.4: for sample x n The output result of dec _ brij integrated with the decision weight is
In this embodiment, the output results of the decision weights incorporated into the dec _ br51, dec _ br52, and dec _ br53 are respectively:
step4.5: for sample x n The decision fusion probability belonging to each class is P (x) n )，
Representing the probability that the prediction sample x after decision fusion belongs to class L.
In this embodiment, sample x 1 The decision fusion probability belonging to each class is P (x) 1 )＝[9.99998371e-01, 2.58543500e-09,2.08537332e-08,1.00000001e-10,1.00000001e-10,1.00000001e-10, 6.99720771e-10,1.00000001e-10,7.41182590e-08,3.22268109e-10,1.49593680e-09, 1.50118298e-06,1.07983939e-10]。
Step4.6: for sample x, the final prediction class label (x) ═ argmax (p (x)).
In this embodiment, P (x) 1 ) Can be seen that the first probability value is the largest, so the sample x 1 Belonging to class 0.
The present invention can be further illustrated by the following experimental results.
The experimental environment is as follows: the CPU is Intel (R) core (TM) i7-7700 CPU @3.60GHz, the memory is 32GB, the operating system is windows 10, and the compiling environment is jupyter notebook.
Experimental data: the invention adopts the oxford 17flowers public data set to classify the images.
And (3) analyzing an experimental result: the invention introduces the enhancement mask to enhance the characteristics in a self-adaptive manner, thereby improving the expression capability of the characteristics; and introducing information entropy to fuse a plurality of decisions for classification. The classification accuracy of the poplar method reached 95.41% in the data set, and table 1 is a statistical table of the classification accuracy of the method of the present invention under the feature enhancement effect in the data set, and data in the table shows that the feature enhancement method of the present invention is superior to the method of poplar in most cases, and the classification accuracy of the method of the present invention in the data set can reach 97.03%.
TABLE 1
Table 2 shows the comparison of classification accuracy under the fusion of single decision and decision in this dataset. Experimental data show that when the two decisions are close to each other, decision fusion can achieve a better effect, but when the two decisions are far from each other, a decision neutralization effect may occur.
TABLE 2
While the present invention has been described in detail with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, and various changes can be made without departing from the spirit of the present invention within the knowledge of those skilled in the art.
Claims (6)
1. A flower image classification algorithm based on feature enhancement and decision fusion is characterized in that:
step 1: performing corresponding cutting, scaling and normalization operations on the data set;
step 2: sending the preprocessed data into a VGG16 model pre-trained by ImageNet to extract multilayer high-level features, and introducing a feature enhancement strategy to adaptively distribute feature weights so as to realize the effect of adaptively enhancing the features;
step 3: training a plurality of self-defined softmax classifiers by using the plurality of groups of characteristics of the previous stage;
step 4: and introducing information entropy to express the certainty degree of each classifier, so that decision fusion weight is adaptively distributed, and classification is realized through decision fusion.
2. A floral image classification algorithm based on feature enhancement and decision fusion as claimed in claim 1, wherein Step1 is specifically:
step1.1: dividing an image data set into a training set, a verification set and a test set;
step1.2: cutting each image in the training set into k square images by adopting a normal random cutting strategy, and cutting each image in the verification set and the test set into a square image by adopting a center cutting strategy;
step1.3: scaling the images in the training set, the test set and the verification set after clipping to be 224 multiplied by 224 of the size required by the VGG16 model by using a Lanczos interpolation mode;
step1.4: and linearly transforming the pixel values of the images in the training set, the verification set and the test set after the scaling from the range of [0,255] to the range of [0,1] by adopting a maximum and minimum normalization mode.
3. A floral image classification algorithm based on feature enhancement and decision fusion as claimed in claim 1, wherein Step2 is specifically:
step 2.1: removing three full-connection layers of a pre-trained VGG16 model, and sending a training set, a verification set and a test set into the three full-connection layers to extract high-level features;
step2.2: respectively storing the characteristics of a block i _ convj layer and a block5_ pool layer of the training set, the verification set and the test set;
step2.3: self-adaptive enhancing is carried out on the characteristics of the blocki _ convj layer by using a characteristic enhancing mask strategy to obtain blocki _ convj _ en;
if the block i _ convj output characteristic diagram is assumed to be M ij ＝F ij The output characteristic diagram of (x, y, p) block i _ convj _ en is M ij_en ＝F ij_en (x, y, p), introduction of feature enhancement mask M en ＝F en (x,y,p)，M ij_en ＝M ij ×M en Wherein x is more than or equal to 1, y is more than or equal to W, P is more than or equal to 1 and less than or equal to P, wherein W represents the side length of the characteristic diagram, and P represents the channel number of the characteristic diagram;
step2.4: and splicing the block i _ convj _ en layer and the block5_ pool layer into a new tensor Concat _ ij.
4. A floral image classification algorithm based on feature enhancement and decision fusion as claimed in claim 3, wherein said step2.3 is specifically:
step2.3.1: block i _ convj layer feature map, called M ij Dimension is W multiplied by P, namely, each of P channels has a pixel matrix with the size of W multiplied by W, and the P channels are compressed to one channel to obtain a superposition map M with the dimension of W multiplied by W stack And calculating an average M thereof average Wherein M is ij ＝F ij (x, y, p) if one of the channel characteristics is marked as m p Then M is stack ＝m 1 +m 2 +…+m P ,M average ＝M stack /P；
Step2.3.2: order to
A threshold representing a distinguishing high response zone from a non-high response zone, the high response zone being more likely to be a zone in which flowers are expressed;
step2.3.3: if M is ij If F (x, y, p) is greater than or equal to thres, then M en ＝1+(M ij (x, y, p) -thres)/(a-thres); if M is ij If F (x, y, p) < thres, then M en B, wherein a and b are hyperparameters;
Step2.3.4：M ij_en ＝M ij ×M en 。
5. a floral image classification algorithm based on feature enhancement and decision fusion as claimed in claim 1, wherein Step3 is specifically:
step3.1: connecting the global average pooling layer with a full-connection layer containing 1024 nodes, and constructing a softmax classifier by taking a softmax function as an activation function;
step3.2: respectively accessing the tensor Concat _ ij into a classifier, and training and verifying by adopting output characteristics of a block i _ convj layer and a block5_ pool layer;
step3.3: and storing the model trained by Concat _ ij to obtain a decision model dec _ brij.
6. A floral image classification algorithm based on feature enhancement and decision fusion as claimed in claim 1, wherein Step4 is specifically:
step4.1: testing the images of the test set by using dec _ brij to obtain the probability that each sample belongs to each class, and for a sample x n Test result p of decision branch dec _ brij ij (x n )＝[p ij1 (x n )p ij2 (x n )…p ijL (x n )]Wherein L represents the total number of classes of the sample, p ijl (x n ) Prediction sample x representing decision dec _ brij n Probability of belonging to class i;
step4.2: the entropy h (x) of information is introduced to represent the degree of certainty that the decision model has for the decision,
wherein i is more than or equal to 1 and less than or equal to 5, and j is more than or equal to 1 and less than or equal to 3;
H ij (x) Indicating the degree of certainty that the decision branch dec _ brij has with respect to the classification result, p ijl (x) The probability that the decision branch dec _ brij decides the sample x as class i is represented;
step4.3: a decision weight ω is introduced to determine a specific method of decision fusion, where ω is defined as follows,
wherein i is more than or equal to 1 and less than or equal to 5, j is more than or equal to 1 and less than or equal to 3, omega ij Represents the decision weight assigned to dec _ brij;
step4.4: for sample x n The output result of dec _ brij integrated with the decision weight is
Step4.5: for sample x n The decision fusion probability belonging to each class is P (x) n )，
Representing the probability that the prediction sample x belongs to the class L after decision fusion;
step4.6: for sample x, the final prediction class label (x) ═ argmax (p (x)).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210275922.8A CN115099294B (en) | 2022-03-21 | 2022-03-21 | Flower image classification algorithm based on feature enhancement and decision fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210275922.8A CN115099294B (en) | 2022-03-21 | 2022-03-21 | Flower image classification algorithm based on feature enhancement and decision fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115099294A true CN115099294A (en) | 2022-09-23 |
| CN115099294B CN115099294B (en) | 2024-07-19 |
Family
ID=83287883
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210275922.8A Active CN115099294B (en) | 2022-03-21 | 2022-03-21 | Flower image classification algorithm based on feature enhancement and decision fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115099294B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130156301A1 (en) * | 2011-12-19 | 2013-06-20 | Industrial Technology Research Institute | Method and system for recognizing images |
| GB201703310D0 (en) * | 2017-03-01 | 2017-04-12 | Toshiba Kk | A feature extraction system, an automatic speech recognition system, a feature extraction method, an automatic speech recognition method and a method of train |
| CN110222562A (en) * | 2019-04-26 | 2019-09-10 | 昆明理工大学 | A kind of method for detecting human face based on Fast R-CNN |
| CN110569905A (en) * | 2019-09-10 | 2019-12-13 | 江苏鸿信系统集成有限公司 | Fine-grained image classification method based on generation of confrontation network and attention network |
| CN111028207A (en) * | 2019-11-22 | 2020-04-17 | 东华大学 | Button flaw detection method based on brain-like immediate-universal feature extraction network |
| WO2020077940A1 (en) * | 2018-10-16 | 2020-04-23 | Boe Technology Group Co., Ltd. | Method and device for automatic identification of labels of image |
| WO2020204525A1 (en) * | 2019-04-01 | 2020-10-08 | 한양대학교 산학협력단 | Combined learning method and device using transformed loss function and feature enhancement based on deep neural network for speaker recognition that is robust in noisy environment |
| WO2020215984A1 (en) * | 2019-04-22 | 2020-10-29 | 腾讯科技（深圳）有限公司 | Medical image detection method based on deep learning, and related device |
| US20200388287A1 (en) * | 2018-11-13 | 2020-12-10 | CurieAI, Inc. | Intelligent health monitoring |
| CN112614131A (en) * | 2021-01-10 | 2021-04-06 | 复旦大学 | Pathological image analysis method based on deformation representation learning |
| CN113887610A (en) * | 2021-09-29 | 2022-01-04 | 内蒙古工业大学 | Pollen image classification method based on cross attention distillation transducer |
| CN114092819A (en) * | 2022-01-19 | 2022-02-25 | 成都四方伟业软件股份有限公司 | Image classification method and device |
| WO2023221951A2 (en) * | 2022-05-14 | 2023-11-23 | 北京大学 | Cell differentiation based on machine learning using dynamic cell images |
| CN117195148A (en) * | 2023-09-07 | 2023-12-08 | 西安科技大学 | Ore emotion recognition method based on expression, electroencephalogram and voice multi-mode fusion |
| CN117350925A (en) * | 2023-10-31 | 2024-01-05 | 云南电网有限责任公司电力科学研究院 | Inspection image infrared visible light image fusion method, device and equipment |
-
2022
- 2022-03-21 CN CN202210275922.8A patent/CN115099294B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130156301A1 (en) * | 2011-12-19 | 2013-06-20 | Industrial Technology Research Institute | Method and system for recognizing images |
| GB201703310D0 (en) * | 2017-03-01 | 2017-04-12 | Toshiba Kk | A feature extraction system, an automatic speech recognition system, a feature extraction method, an automatic speech recognition method and a method of train |
| WO2020077940A1 (en) * | 2018-10-16 | 2020-04-23 | Boe Technology Group Co., Ltd. | Method and device for automatic identification of labels of image |
| US20200388287A1 (en) * | 2018-11-13 | 2020-12-10 | CurieAI, Inc. | Intelligent health monitoring |
| WO2020204525A1 (en) * | 2019-04-01 | 2020-10-08 | 한양대학교 산학협력단 | Combined learning method and device using transformed loss function and feature enhancement based on deep neural network for speaker recognition that is robust in noisy environment |
| WO2020215984A1 (en) * | 2019-04-22 | 2020-10-29 | 腾讯科技（深圳）有限公司 | Medical image detection method based on deep learning, and related device |
| CN110222562A (en) * | 2019-04-26 | 2019-09-10 | 昆明理工大学 | A kind of method for detecting human face based on Fast R-CNN |
| CN110569905A (en) * | 2019-09-10 | 2019-12-13 | 江苏鸿信系统集成有限公司 | Fine-grained image classification method based on generation of confrontation network and attention network |
| CN111028207A (en) * | 2019-11-22 | 2020-04-17 | 东华大学 | Button flaw detection method based on brain-like immediate-universal feature extraction network |
| CN112614131A (en) * | 2021-01-10 | 2021-04-06 | 复旦大学 | Pathological image analysis method based on deformation representation learning |
| CN113887610A (en) * | 2021-09-29 | 2022-01-04 | 内蒙古工业大学 | Pollen image classification method based on cross attention distillation transducer |
| CN114092819A (en) * | 2022-01-19 | 2022-02-25 | 成都四方伟业软件股份有限公司 | Image classification method and device |
| WO2023221951A2 (en) * | 2022-05-14 | 2023-11-23 | 北京大学 | Cell differentiation based on machine learning using dynamic cell images |
| CN117195148A (en) * | 2023-09-07 | 2023-12-08 | 西安科技大学 | Ore emotion recognition method based on expression, electroencephalogram and voice multi-mode fusion |
| CN117350925A (en) * | 2023-10-31 | 2024-01-05 | 云南电网有限责任公司电力科学研究院 | Inspection image infrared visible light image fusion method, device and equipment |
Non-Patent Citations (6)
| Title |
|---|
| JIA, LY等: "A Parallel Convolution and Decision Fusion-Based Flower Classification Method", 《MATHEMATICS》, vol. 10, no. 15, 31 August 2022 (2022-08-31), pages 1 - 15 * |
| LI, R等: "Endoscopic Segmentation of Kidney Stone based on Transfer Learning", 《2021 PROCEEDINGS OF THE 40TH CHINESE CONTROL CONFERENCE (CCC)》, 1 January 2021 (2021-01-01), pages 8145 - 8150 * |
| LIU, J等: "A Method for Plant Diseases Detection Based on Transfer Learning and Data Enhancement", 《2022 INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE BIG DATA AND INTELLIGENT SYSTEMS (HDIS)》, 17 March 2022 (2022-03-17), pages 154 - 8 * |
| QING LI: ""Deep Hierarchical Semantic Segmentation Algorithm Based On Image Information Entropy"", 《ICIC EXPRESS LETTERS, PART B: APPLICATIONS 》, vol. 11, no. 01, 31 January 2020 (2020-01-31), pages 25 - 32 * |
| 严春满等: "基于特征增强的SAR图像舰船小目标检测算法", 《控制与决策》, vol. 38, no. 01, 8 October 2021 (2021-10-08), pages 239 - 247 * |
| 陈程军等: "基于激活-熵的分层迭代剪枝策略的CNN模型压缩", 《计算机应用》, vol. 40, no. 05, 13 April 2020 (2020-04-13), pages 1260 - 1265 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115099294B (en) | 2024-07-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106845421B (en) | Face feature recognition method and system based on multi-region feature and metric learning | |
| CN112818862B (en) | Face tampering detection method and system based on multi-source clues and mixed attention | |
| CN112686331B (en) | Forged image recognition model training method and forged image recognition method | |
| Shojae Chaeikar et al. | PSW statistical LSB image steganalysis | |
| WO2023000872A1 (en) | Supervised learning method and apparatus for image features, device, and storage medium | |
| KR102036957B1 (en) | Safety classification method of the city image using deep learning-based data feature | |
| CN112949572A (en) | Slim-YOLOv 3-based mask wearing condition detection method | |
| CN111160286B (en) | Video authenticity identification method | |
| CN110929745A (en) | Neural network based classification method and classification device thereof | |
| Laishram et al. | A novel minimal distortion-based edge adaptive image steganography scheme using local complexity: (BEASS) | |
| CN114187311A (en) | Image semantic segmentation method, device, equipment and storage medium | |
| CN112257758A (en) | Fine-grained image recognition method, convolutional neural network and training method thereof | |
| CN114511576B (en) | Image segmentation method and system of scale self-adaptive feature enhanced deep neural network | |
| CN114444566B (en) | Image forgery detection method and device and computer storage medium | |
| CN110348320B (en) | Face anti-counterfeiting method based on multi-loss depth fusion | |
| CN110555439A (en) | identification recognition method, training method and device of model thereof and electronic system | |
| CN114549913A (en) | Semantic segmentation method and device, computer equipment and storage medium | |
| CN111079864A (en) | Short video classification method and system based on optimized video key frame extraction | |
| CN114500396A (en) | MFD chromatographic characteristic extraction method and system for distinguishing anonymous Tor application flow | |
| US20070016576A1 (en) | Method and apparatus for blocking objectionable multimedia information | |
| WO2021213188A1 (en) | Training method and apparatus for image processing model, and image processing method and apparatus | |
| CN116342931A (en) | Fur image classification method, system and storage medium for multi-scale attention | |
| CN113011307A (en) | Face recognition identity authentication method based on deep residual error network | |
| CN113033371B (en) | Multi-level feature fusion pedestrian detection method based on CSP model | |
| CN115099294B (en) | Flower image classification algorithm based on feature enhancement and decision fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |