JP2022138266A - Method of discriminating class of discriminated data using machine learning model, information processing apparatus, and computer program - Google Patents

Abstract

To provide a technique that discriminates a class of discriminated data by a method other than a normal method using a discrimination result at an output layer of a neural network.SOLUTION: A class discrimination method includes: a step (a) of preparing M pairs of known characteristic spectrum groups corresponding to M machine learning models; and a step (b) of performing class discrimination processing of discriminated data using the M machine learning models and M known characteristic spectrum groups. The step (b) includes: a step (b1) of calculating M characteristic spectra from an output of a specific layer of the M machine learning models; a step (b2) of calculating a similarity for each class relating to each of the machine learning models as a similarity between the characteristic spectrum and the known characteristic spectrum group, and obtaining a reliability for each class depending on the similarity for each class; and a step (b3) of outputting a discrimination result list in which a plurality of classes are arranged in an order of the reliability of each of the classes.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to a method, an information processing device, and a computer program for discriminating a class of data to be discriminated using a machine learning model.

Patent Documents 1 and 2 disclose what is called a capsule network as a vector neural network type machine learning model using vector neurons. A vector neuron means a neuron whose inputs and outputs are vectors. A capsule network is a machine learning model whose network nodes are vector neurons called capsules. Vector neural network type machine learning models such as capsule networks can be used for class discrimination of input data.

U.S. Pat. No. 5,210,798 International Publication No. 2019/083553

The inventor of the present disclosure has the problem that in the conventional technology, the features extracted in the intermediate layer of the neural network are not correctly reflected in the class discrimination results in the output layer, and the class discrimination results may be erroneous. I found out. The present disclosure provides a technique for performing class discrimination of data to be discriminated by a method different from the usual method of directly using the discrimination result in the output layer of the neural network. In addition, the present disclosure provides a training data processing technique that contributes to improvement in discrimination accuracy, whether combined with or separated from the other method.

According to the first aspect of the present disclosure, when M is an integer of 1 or more, the class of data to be discriminated is determined using M vector neural network type machine learning models having a plurality of vector neuron layers. A method is provided. The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. and The method includes the step of: (a) preparing M sets of known feature spectra groups associated with the M machine learning models, wherein the M sets of known feature spectra groups are Nm for each machine learning model; (b) a known feature spectrum group obtained from the output of a specific layer of the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of the classes is input; using the M machine learning models and the M known feature spectra groups to perform class discrimination processing of the data to be discriminated. The step (b) includes (b1) calculating M feature spectra from the output of the specific layer in response to the input of the data to be discriminated to the M machine learning models; and (b2) the As the similarity between the M feature spectra and the M sets of known feature spectra, calculate the class similarity for each of the Nm classes in each machine learning model, and the class similarity (b3) obtaining M sets of class-wise confidences associated with the M machine learning models by obtaining dependent class-wise confidences; and outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in order of reliability of each class.

According to the second form of the present disclosure, when M is an integer of 2 or more, the class of data to be discriminated is determined using M vector neural network type machine learning models having a plurality of vector neuron layers. A method is provided. This method includes (a) a step of preparing M sets of known feature spectra groups associated with the M machine learning models, wherein the M sets of known feature spectra groups are Nm for each machine learning model. (b) a known feature spectrum group obtained from the output of a specific layer of the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of the classes is input; using the M machine learning models and the M known feature spectra groups to perform class discrimination processing of the data to be discriminated. The step (b) includes (b1) calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and generating M discriminant classes (b2) calculating a discriminant class similarity for each of the M discriminant classes as a similarity between the M feature spectra and the M sets of known feature spectra, and obtaining a model confidence for each of the M machine learning models by obtaining a model confidence dependent on the discriminant class similarity; (b3) among the M discriminant classes, the model confidence; outputting the discriminant class obtained from the machine learning model with the highest , as a class discrimination result for the data to be discriminated.

According to the third aspect of the present disclosure, when M is an integer of 1 or more, the class of data to be discriminated is determined using M vector neural network type machine learning models having a plurality of vector neuron layers. An information processing device is provided. This information processing apparatus includes a memory that stores the M machine learning models, and a processor that performs calculations using the M machine learning models. The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. and The processor performs (a) a process of reading M sets of known feature spectra groups associated with the M machine learning models from the memory, wherein the M sets of known feature spectra groups are each machine learning model A process including a group of known feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of Nm classes is input to ( b) using the M machine learning models and the M known feature spectrum groups to perform class discrimination processing of the data to be discriminated. The process (b) includes (b1) a process of calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and (b2) the As the similarity between the M feature spectra and the M sets of known feature spectra, calculate the class similarity for each of the Nm classes in each machine learning model, and the class similarity (b3) a process of obtaining M sets of class confidences associated with the M machine learning models by obtaining the class confidences that depend on them; and (b3) the M sets of class confidences represented by a process of outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in order of reliability of each class.

According to the fourth aspect of the present disclosure, when M is an integer of 2 or more, the class of data to be discriminated is determined using M vector neural network type machine learning models having a plurality of vector neuron layers. An information processing device is provided. This information processing apparatus includes a memory that stores the M machine learning models, and a processor that performs calculations using the M machine learning models. The processor performs (a) a process of reading M sets of known feature spectra groups associated with the M machine learning models from the memory, wherein the M sets of known feature spectra groups are each machine learning model A process including a group of known feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of Nm classes is input to ( b) using the M machine learning models and the M known feature spectrum groups to perform class discrimination processing of the data to be discriminated. The process (b) includes (b1) calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and generating M discriminant classes and (b2) calculating the discriminant class similarity for each of the M discriminant classes as the similarity between the M feature spectra and the M sets of known feature spectra, (b3) a process of obtaining model reliability for each of the M machine learning models by obtaining model reliability that depends on the discriminant class similarity; and (b3) the model reliability among the M discriminant classes outputting the discriminant class obtained from the machine learning model with the highest value as the class discrimination result for the data to be discriminated.

According to the fifth aspect of the present disclosure, when M is an integer of 1 or more, the class of data to be discriminated is determined using M vector neural network type machine learning models having a plurality of vector neuron layers. A computer program is provided that causes a processor to perform a class determination process for The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. and The computer program includes (a) a process of reading from memory M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are each machine learning model; A process including a group of known feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of Nm classes is input to ( b) a computer program that causes the processor to execute a process of classifying the data to be classified using the M machine learning models and the M known feature spectrum groups. The process (b) includes (b1) a process of calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and (b2) the As the similarity between the M feature spectra and the M sets of known feature spectra, calculate the class similarity for each of the Nm classes in each machine learning model, and the class similarity (b3) a process of obtaining M sets of class confidences associated with the M machine learning models by obtaining the class confidences that depend on them; and (b3) the M sets of class confidences represented by a process of outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in order of reliability of each class.

According to the sixth aspect of the present disclosure, when M is an integer of 2 or more, the class of data to be discriminated is determined using M vector neural network type machine learning models having a plurality of vector neuron layers. A computer program is provided that causes a processor to perform a class determination process for This computer program includes (a) a process of reading out from a memory M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are each machine learning model; A process including a group of known feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of Nm classes is input to ( b) a computer program that causes the processor to execute a process of classifying the data to be classified using the M machine learning models and the M known feature spectrum groups. The process (b) includes (b1) calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and generating M discriminant classes and (b2) calculating the discriminant class similarity for each of the M discriminant classes as the similarity between the M feature spectra and the M sets of known feature spectra, (b3) a process of obtaining model reliability for each of the M machine learning models by obtaining model reliability that depends on the discriminant class similarity; and (b3) the model reliability among the M discriminant classes outputting the discriminant class obtained from the machine learning model with the highest value as the class discrimination result for the data to be discriminated.

1 is a block diagram of a class determination system according to an embodiment; FIG. 1 is a block diagram of an information processing device; FIG. Explanatory drawing which shows the structure of a machine-learning model. Explanatory drawing which shows the other structure of a machine-learning model. FIG. 4 is a block diagram showing functions of a class determination processing unit according to the first embodiment; A flow chart showing a preparation process of a machine learning model. Explanatory drawing which shows the teacher data by which the clustering process was carried out. Explanatory drawing which shows a characteristic spectrum. FIG. 4 is an explanatory diagram showing how a group of known feature spectra is created using teacher data; Explanatory drawing which shows the structure of a known-feature spectrum group. 4 is a flow chart showing a processing procedure of a medium discrimination/printing process in the first embodiment; FIG. 10 is an explanatory diagram showing how a similarity degree by class regarding data to be discriminated is obtained; Explanatory drawing which shows an example of the discrimination|determination result list|wrist. The block diagram which shows the function of the class discrimination|determination process part in 2nd Embodiment. The block diagram which shows the function of the class discrimination|determination process part in 3rd Embodiment. 10 is a flow chart showing a processing procedure of a medium discrimination/printing process in the third embodiment; The block diagram which shows the function of the class discrimination|determination process part in 4th Embodiment. The block diagram which shows the function of the class discrimination|determination process part in 5th Embodiment. Explanatory drawing which shows the 1st calculation method of the similarity degree according to class. Explanatory drawing which shows the 2nd calculation method of the similarity degree according to class. Explanatory drawing which shows the 3rd calculation method of the similarity according to class. FIG. 10 is an explanatory diagram showing a method of selecting similarities by class when using a plurality of specific layers;

A. First Embodiment:
FIG. 1 is a block diagram showing a class discrimination system according to the first embodiment. This class determination system is a printing system having a printer 10 , an information processing device 20 and a spectrometer 30 . The spectrometer 30 can spectroscopically measure the unprinted print medium PM used in the printer 10 to obtain the spectral reflectance. In the present disclosure, spectral reflectance is also referred to as "spectral data". The spectrometer 30 includes, for example, a tunable interference spectrum filter and a monochrome image sensor. The spectroscopic data obtained by the spectrometer 30 is used as discriminated data input to a machine learning model, which will be described later. The information processing apparatus 20 uses a machine learning model to perform class determination processing of spectral data, and determines to which of a plurality of classes the print medium PM belongs. The “class of print medium PM” means the type of print medium PM. The information processing device 20 controls the printer 10 so as to execute printing under appropriate printing conditions according to the type of print medium PM. Further, the information processing apparatus 20 may display the determined type of the printing medium PM on the printer display. By doing this, it is possible to prevent the printer from being driven while a medium different from the one intended by the user is installed. The class determination system according to the present disclosure can also be implemented as a system other than a printing system. For example, images to be determined, one-dimensional data other than spectral data, spectral images, time series data, etc. It may be realized as a system that performs determination.

FIG. 2 is a block diagram showing functions of the information processing device 20. As shown in FIG. The information processing apparatus 20 has a processor 110 , a memory 120 , an interface circuit 130 , an input device 140 and a display section 150 connected to the interface circuit 130 . The spectrometer 30 and the printer 10 are also connected to the interface circuit 130 . For example, without limitation, the processor 110 not only has the function of executing the processing detailed below, but also displays data obtained by the processing and data generated during the processing on the display unit 150. It also has the function to

The processor 110 functions as a print processing unit 112 that executes print processing using the printer 10, and functions as a class determination processing unit 114 that executes class determination processing for spectral data of the print medium PM. The class determination processing unit 114 includes a similarity calculation unit 310 and a comprehensive determination unit 320 . The print processing unit 112 and the class determination processing unit 114 are implemented by the processor 110 executing computer programs stored in the memory 120 . However, these units 112 and 114 may be realized by hardware circuits. A processor herein is a term that also includes such hardware circuitry. Also, the processor that executes the class determination process may be a processor included in a remote computer connected to the information processing device 20 via a network.

The memory 120 stores a plurality of machine learning models 200, a plurality of teacher data groups TD, a plurality of known feature spectrum groups KSp, and a print setting table PST. The machine learning model 200 is used for processing by the class determination processing unit 114 . A configuration example and operation of the machine learning model 200 will be described later. A teacher data group TD is a set of labeled data used for learning of the machine learning model 200 . In this embodiment, the teacher data group TD is a set of spectral data. The known feature spectrum group KSp is a set of feature spectra obtained when the teacher data group TD is input to the trained machine learning model 200 . A feature spectrum will be described later. The print setting table PST is a table in which print settings suitable for each print medium are registered. When the number of machine learning models 200 is represented by an integer M, M can be set to any number equal to or greater than 1. In this embodiment, the case of using two machine learning models 200 will be described. As the teacher data group TD and the known feature spectrum group KSp, those corresponding to the machine learning model 200 are used.

FIG. 3 is an explanatory diagram showing the configuration of the machine learning model 200. As shown in FIG. This machine learning model 200 includes a convolutional layer 210, a primary vector neuron layer 220, a first convolutional vector neuron layer 230, a second convolutional vector neuron layer 240, and a classification vector neuron layer 250 in order from the input data IM side. and Of these five layers 210-250, the convolutional layer 210 is the lowest layer and the classification vector neuron layer 250 is the highest layer. In the following discussion, layers 210-250 are also referred to as "Conv layer 210," "PrimeVN layer 220," "ConvVN1 layer 230," "ConvVN2 layer 240," and "ClassVN layer 250," respectively.

In the present embodiment, the input data IM is spectral data, so it is one-dimensional array data. For example, the input data IM is data obtained by extracting 36 representative values every 10 nm from spectral data in the range of 380 nm to 730 nm.

Although two convolution vector neuron layers 230 and 240 are used in the example of FIG. 3, the number of convolution vector neuron layers is arbitrary and the convolution vector neuron layers may be omitted. However, it is preferable to use one or more convolutional vector neuron layers.

The configuration of each layer 210-250 in FIG. 3 can be described as follows.
<Description of Configuration of Machine Learning Model 200>
Conv layer 210: Conv[32,6,2]
- PrimeVN layer 220: PrimeVN[26,1,1]
ConvVN1 layer 230: ConvVN1[20,5,2]
ConvVN2 layer 240: ConvVN2[16,4,1]
- ClassVN layer 250: ClassVN[Nm,3,1]
・Vector dimension VD: VD=16
In the description of each of these layers 210-250, the character string before the parentheses is the layer name, and the numbers inside the parentheses are the number of channels, the surface size of the kernel, and the stride, respectively. For example, the Conv layer 210 has the layer name “Conv”, the number of channels is 32, the kernel surface size is 1×6, and the stride is 2. These descriptions are shown under each layer in FIG. The hatched rectangle drawn in each layer represents the surface size of the kernel used in calculating the output vector of the adjacent upper layer. In this embodiment, since the input data IM is one-dimensional array data, the surface size of the kernel is also one-dimensional. Note that the parameter values used in the description of each layer 210 to 250 are examples and can be changed arbitrarily.

The Conv layer 210 is a layer composed of scalar neurons. The other four layers 220-250 are layers composed of vector neurons. A vector neuron is a neuron whose input and output are vectors. In the above description, the dimensions of the output vectors of the individual vector neurons are 16 and constant. In the following, the term "node" is used as a superordinate concept for scalar neurons and vector neurons.

In FIG. 3, for the Conv layer 210, a first axis x and a second axis y defining the plane coordinates of the node array and a third axis z representing depth are shown. It also shows that the Conv layer 210 has sizes of 1, 16, and 32 in the x, y, and z directions. The size in the x direction and the size in the y direction are called "resolution". In this embodiment, the x-direction resolution is always one. The size in the z direction is the number of channels. These three axes x, y, and z are also used as coordinate axes indicating the position of each node in other layers. However, in FIG. 3, illustration of these axes x, y, and z in layers other than the Conv layer 210 is omitted.

As is well known, the y-direction resolution W1 after convolution is given by the following equation.
W1=Ceil{(W0-Wk+1)/S} (1)
Here, W0 is the resolution before convolution, Wk is the surface size of the kernel, S is the stride, and Ceil{X} is a function for rounding up the decimal part of X.
The resolution of each layer shown in FIG. 3 is an example when the y-direction resolution of the input data IM is 36, and the actual resolution of each layer is appropriately changed according to the size of the input data IM.

The ClassVN layer 250 has Nm channels. In the example of FIG. 3, Nm=3. In general, Nm is an integer greater than or equal to 2 and is the number of known classes discriminable using machine learning model 200 . A different value can be set for each machine learning model 200 for the number of distinguishable classes Nm. The total number of classes discriminable by the M machine learning models 200 is represented by ΣNm. The three channels of the ClassVN layer 250 output judgment values Class1 to Class3 for the three known classes. Usually, the class having the largest value among these judgment values Class1 to Class3 is used as the class discrimination result of the input data IM. Further, when the largest value among the determination values Class1 to Class3 is less than a predetermined threshold value, it may be determined that the class of the input data IM is unknown.

In the present disclosure, as described later, instead of using the judgment values Class1 to Class3 of the ClassVN layer 250, which is the output layer, the similarity calculated from the output of a specific vector neuron layer is used to determine the discriminant class. It's a good way to do it. It is also possible to output a discrimination result list in which a plurality of classes are arranged in order of similarity or in order of reliability dependent on similarity, instead of determining one discrimination class. In the first embodiment, a method of outputting a discrimination result list and presenting it to the user is adopted.

In FIG. 3, partial regions Rn in each layer 210, 220, 230, 240, 250 are also depicted. The suffix “n” of the partial region Rn is the code for each layer. For example, partial region R210 indicates a partial region in Conv layer 210. FIG. The “partial region Rn” is specified by a plane position (x, y) defined by the position of the first axis x and the position of the second axis y in each layer, and a plurality of channels along the third axis z is a region containing The partial region Rn has dimensions of “Width”דHeight”דDepth” corresponding to the first axis x, the second axis y, and the third axis z. In this embodiment, the number of nodes included in one “partial region Rn” is “1×1×number of depths”, that is, “1×1×number of channels”.

As shown in FIG. 3 , a feature spectrum Sp_ConvVN1, which will be described later, is calculated from the output of the ConvVN1 layer 230 and input to the similarity calculation section 310 . Similarly, feature spectra Sp_ConvVN2 and Sp_ClassVN are calculated from the outputs of the ConvVN2 layer 240 and the ClassVN layer 250 and input to the similarity calculation section 310 . The similarity calculation unit 310 uses these feature spectra Sp_ConvVN1, Sp_ConvVN, Sp_ClassVN and a known feature spectrum group KSp created in advance to calculate similarities by class, which will be described later, and depends on the similarities by class. Calculate the reliability for each class.

In the present disclosure, the vector neuron layer used for similarity calculation is also called "specific layer". Any number of one or more vector neuron layers can be used as the specific layer. The structure of the feature spectrum and the method of calculating the degree of similarity and the degree of reliability using the feature spectrum will be described later.

FIG. 4 is an explanatory diagram showing another configuration of the machine learning model 200. As shown in FIG. This machine learning model 200 differs from the machine learning model 200 of FIG. 3 using one-dimensional array input data in that the input data IM is two-dimensional array data. The configuration of each layer 210-250 in FIG. 4 can be described as follows.
<Description of the configuration of each layer>
Conv layer 210: Conv[32,5,2]
- PrimeVN layer 220: PrimeVN[16,1,1]
ConvVN1 layer 230: ConvVN1[12,3,2]
ConvVN2 layer 240: ConvVN2[6,3,1]
- ClassVN layer 250: ClassVN[Nm,4,1]
・Vector dimension VD: VD=16

The machine learning model 200 shown in FIG. 4 can be used, for example, in a classification system that classifies images to be classified. However, in the following description, the machine learning model 200 shown in FIG. 3 is used.

FIG. 5 is a block diagram showing functions of the class determination processing unit 114a in the first embodiment. The class determination processing unit 114a includes a similarity calculation unit 310a and a comprehensive determination unit 320a. The "a" attached to the end of these symbols indicates that they are those of the first embodiment. Comprehensive determination section 320 a includes list creation section 322 . In this example, two machine learning models 200_1 and 200_2 are used. "_1" and "_2" at the end of these codes are additional codes to distinguish between the two machine learning models. However, when there is no need to distinguish between a plurality of machine learning models, the code "200" with no additional code is used.

When the data to be discriminated IM is input to the two machine learning models 200, the feature spectra Sp are calculated from the specific layers of the two machine learning models 200 and input to the similarity calculation section 310a. The similarity calculation unit 310a calculates a class-based similarity Sclass that is a similarity between the feature spectrum Sp and the corresponding known feature spectrum group KSp of the specific layer, and calculates a class-based similarity Sclass that depends on the class-based similarity Sclass. Obtain the reliability Rclass. Boxes drawn with dashed lines in the similarity calculation unit 310a in FIG. 5 are drawn for convenience to show the input/output relationship. The class similarity Sclass includes a parameter m indicating the machine learning model 200, a parameter i indicating the class, and a similarity value S_value of the class i. A method of calculating the similarity by class Sclass will be described later.

The class-specific reliability Rclass includes a parameter m indicating the machine learning model 200, a parameter i indicating the class, and the reliability value R_value of the class i. As shown in the following equation, the reliability value R_value is calculated according to a reliability function H that depends on the similarity value S_value of the class similarity Sclass.
R_value(i)=H[S_value(i)] (2)
In this formula, the machine learning model parameter m is omitted. Preferably, the reliability function H determines the reliability by class Rclass so that the reliability by class Rclass has a positive correlation with the similarity by class Sclass. A specific example of the reliability function H will be described later.

In the example of FIG. 5, it is assumed that each of the two machine learning models 200 has three discriminable classes. The list creation unit 322 outputs a discrimination result list RL in which a plurality of classes are arranged in order of reliability according to the class-by-class reliability Rclass. This discrimination result list RL is displayed on the display unit 150 . Specific examples of the class-by-class reliability Rclass and the discrimination result list RL will be described later.

FIG. 6 is a flow chart showing the processing procedure of the machine learning model preparation process. This preparation process is, for example, a process executed by the manufacturer of the printer 10 .

In step S100 of FIG. 6, the class determination processing unit 114a performs grouping by clustering a plurality of teacher data. FIG. 7 shows teacher data grouped by clustering processing. In this example, a plurality of teacher data are grouped into a first teacher data group TD1 and a second teacher data group TD2. As the clustering process, for example, the k-means method can be used. It is also possible to perform clustering processing using distances such as the Euclidean distance or the Mahalanobis distance for the optical spectrum of the print medium. For example, in clustering processing using the Euclidean distance, an average spectrum is obtained by averaging the results of multiple spectrum measurements for each print medium, and the print media are collected in order of closest Euclidean distance to the average spectrum of the reference print medium. The collection of the first group is terminated when the number of collected print media reaches the reference value. Then, a new reference is set again, and the print media are collected in the order of the Euclidean distance to form the second group, and the process is repeated. Considering the case of using the k-means method, the teacher data groups TD1 and TD2 have representative points G1 and G2 representing the respective teacher data groups TD1 and TD2. These representative points G1 and G2 are, for example, the center of gravity. If the teacher data consists of reflectances at n wavelengths, one teacher data can be regarded as data representing one point in an n-dimensional space, so that distances between teacher data and differences between multiple teacher data can be calculated. It is possible to calculate the centroid. In FIG. 7, for convenience of illustration, a plurality of teaching data points are drawn in a two-dimensional space, but in practice, teaching data can be expressed as points in an n-dimensional space. These representative points G1 and G2 are used when determining which of the plurality of teacher data groups TD1 and TD2 the teacher data for the added class is closest to when adding a new class as a target of class discrimination processing. can be used for If the teacher data of the additional class is added to one of the plurality of teacher data groups TD1 and TD2 using this judgment, the teacher data groups TD1 and TD2 after addition are equivalent to the state grouped by the clustering process. state can be maintained. Note that points other than the center of gravity may be used as the representative points G1 and G2.

In this embodiment, a plurality of teacher data are grouped into two teacher data groups TD1 and TD2, but only one teacher data group may be created, or three or more teacher data groups may be created. . Also, a plurality of training data groups may be created by a method other than clustering processing. However, if a plurality of teacher data are grouped by clustering processing, teacher data that are similar to each other can be put together in the same group. By learning a plurality of machine learning models 200 using such a plurality of teacher data groups, the accuracy of class discrimination processing by individual machine learning models 200 can be improved compared to the case where clustering processing is not performed. This is for the following reasons. That is, since a plurality of machine learning models are independent of each other, it is assumed that there are two machine learning models A and B, and very similar spectroscopic data Am and Bm, for example. If the spectroscopic data Am and Bm are learned by separate machine learning models A and B, the individual machine learning models A and B do not perform learning to separate the features of the spectroscopic data Am and Bm. . As a result, the difference between the spectral data Am and Bm cannot be distinguished, and both the two machine learning models A and B may judge the spectral data Am as Bm and the spectral data Bm as Am. On the other hand, if the spectroscopic data Am and Bm are learned by the same machine learning model, the spectroscopic data Am and Bm can be discriminated because the difference between the spectroscopic data Am and Bm is learned. By clustering the spectral data in this way, print media with similar features are learned by the same machine learning model, so these print media can be correctly discriminated. Also, other machine learning models are more likely to discriminate unknown print media.

In step S110 of FIG. 6, the class determination processing unit 114a executes learning of the machine learning model 200 using a plurality of teacher data groups TD. Each piece of teacher data is labeled in advance. In this embodiment, it is assumed that any one of labels 1 to 3 is assigned to teacher data of the first machine learning model 200_1. These labels correspond to the three classes Class1 to Class3 of the machine learning model 200 shown in FIG. It is also assumed that the teacher data of the second machine learning model 200_2 is given a label of any one of 11-13. In this disclosure, "label" and "class" mean the same thing.

After completing the learning using the plurality of teacher data groups TD, the learned machine learning model 200 is stored in the memory 120 . In step S120 of FIG. 6, the machine learning model 200 that has already been trained is input again with a plurality of teacher data to generate the known feature spectrum group KSp. The known feature spectrum group KSp is a set of feature spectra described below.

FIG. 8 is an explanatory diagram showing a feature spectrum Sp obtained by inputting arbitrary input data to the learned machine learning model 200. As shown in FIG. Here, the feature spectrum Sp obtained from the output of the ConvVN1 layer 230 is explained. The horizontal axis of FIG. 8 is the position of the vector elements for the output vectors of the multiple nodes included in one partial region R230 of the ConvVN1 layer 230. In FIG. The position of this vector element is represented by a combination of the output vector element number ND and the channel number NC at each node. In this embodiment, since the vector dimension is 16 (the number of elements of the output vector output by each node), the element numbers ND of the output vector are 16 from 0 to 15. FIG. Also, since the number of channels in the ConvVN1 layer 230 is 20, there are 20 channel numbers NC from 0 to 19. In other words, this feature spectrum Sp is obtained by arranging a plurality of element values of the output vector of each vector neuron included in one partial region R230 over a plurality of channels along the third axis z.

The vertical axis in FIG. 8 indicates the feature value C _V at each spectral position. In this example, the feature value C _V is the value V _ND of each element of the output vector. As the feature value CV, a value obtained by _multiplying the value _VND of each element of the output vector by a normalization factor described later may be used, or the normalization factor may be used as it is. In the latter case, the number of feature values C _V included in the feature spectrum Sp is equal to the number of channels, which is twenty. Note that the normalization coefficient is a value corresponding to the vector length of the output vector of that node.

The number of feature spectra Sp obtained from the output of the ConvVN1 layer 230 for one input data is equal to the number of plane positions (x, y) of the ConvVN1 layer 230, that is, the number of subregions R230, so 6 be. Similarly, three feature spectra Sp are obtained from the output of the ConvVN2 layer 240 and one feature spectrum Sp is obtained from the output of the ClassVN layer 250 for one input data.

When teacher data is input again to the learned machine learning model 200, the similarity calculation unit 310a calculates the feature spectrum Sp shown in FIG. 8 and registers it in the memory 120 as a known feature spectrum group KSp.

FIG. 9 is an explanatory diagram showing how the known feature spectrum group KSp is created using the teacher data TD. In this example, by inputting the teacher data TD whose labels are 1 to 3 to the trained machine learning model 200, three vector neuron layers, that is, the ConvVN1 layer 230, the ConvVN2 layer 240 and the ClassVN layer 250 output , feature spectra KSp_ConvVN1, KSp_ConvVN2, and KSp_ClassVN associated with respective labels or classes are obtained. These feature spectra KSp_ConvVN1, KSp_ConvVN2, and KSp_ClassVN are stored in the memory 120 as a group of known feature spectra KSp.

FIG. 10 is an explanatory diagram showing the structure of the known feature spectrum group KSp. In this example, the known feature spectrum group KSp_ConvVN1 obtained from the output of the ConvVN1 layer 230 of the first machine learning model 200_1 is shown. The known feature spectrum group KSp_ConvVN2 obtained from the output of the ConvVN2 layer 240 and the known feature spectrum group KSp_ConvVN1 obtained from the output of the ClassVN layer 250 have similar configurations, but are omitted in FIG. . As the known feature spectrum group KSp, it suffices if the one obtained from the output of at least one vector neuron layer is registered.

Each record of the known feature spectrum group KSp_ConvVN1 indicates a parameter m indicating the order of the machine learning model, a parameter i indicating the order of the label or class, a parameter j indicating the order of the specific layer, and the order of the partial regions Rn. It contains a parameter k, a parameter q indicating a data number, and a known feature spectrum KSp. The known feature spectrum KSp is the same as the feature spectrum Sp in FIG.

The class parameter i takes the same value from 1 to 3 as the label. The specific layer parameter j takes a value from 1 to 3 indicating which of the three specific layers 230, 240, 250 is. The parameter k of the partial region Rn takes a value indicating which of the plurality of partial regions Rn included in each specific layer, that is, which plane position (x, y). Since the ConvVN1 layer 230 has six partial regions R230, k=1-6. The data number parameter q indicates the number of teacher data with the same label, and takes values from 1 to max1 for class 1, from 1 to max2 for class 2, and from 1 to max3 for class 3.

Note that the plurality of teaching data TD used in step S120 need not be the same as the plurality of teaching data TD used in step S110. However, even in step S120, if some or all of the plurality of teaching data TD used in step S110 are used, there is an advantage that there is no need to prepare new teaching data.

FIG. 11 is a flow chart showing a processing procedure of a medium discrimination/printing process using a learned machine learning model. This medium identification/printing process is performed by a user using the printer 10, for example.

In step S210, the user instructs the class determination processing unit 114a as to whether or not the target print medium, which is the print medium to be processed, requires the class determination process. Even if the user knows the type of the target print medium, it may indicate that the class determination process is required for confirmation. If the class determination process is unnecessary, the process proceeds to step S260, where the user selects print settings suitable for the target print medium, and in step S270, the print processing unit 112 causes the printer 10 to execute printing using the target print medium. . On the other hand, if the type of target print medium is unknown and the class determination process is required, the process proceeds to step S220.

In step S220, the class determination processing unit 114a acquires spectral data by causing the spectrometer 30 to perform spectroscopic measurement of the target print medium. This spectral data is used as discriminated data input to the machine learning model 200 .

In step S230, the class discrimination processing unit 114a inputs data to be discriminated to the learned machine learning models 200_1 and 200_2, respectively, and calculates feature spectra Sp from the outputs of the machine learning models 200_1 and 200_2. In step S240, the similarity calculation unit 310a calculates the similarity by class from the feature spectrum Sp obtained according to the input of the data to be discriminated and the registered known feature spectrum group KSp, and the similarity by class Depends on class reliability.

FIG. 12 is an explanatory diagram showing how the similarity by class Sclass and the reliability by class Rclass regarding the data to be discriminated are obtained. When the data to be discriminated is input to the m-th machine learning model 200_m, the class discrimination processing unit 114a extracts feature spectra Sp_ConvVN1, Sp_ConvVN2, Calculate Sp_ClassVN respectively. The similarity calculator 310a uses the feature spectrum Sp_ConvVN1 obtained from the output of the ConvVN1 layer 230 and the known feature spectrum group KSp_ConvVN1 to calculate the similarity by class Sclass_ConvVN1. A specific method for calculating the degree of similarity by class will be described later. For the ConvVN2 layer 240 and the ClassVN layer 250, similarities by class Sclass_ConvVN2 and Sclass_ClassVN are similarly calculated.

It is not necessary to generate all the class similarities Sclass_ConvVN1, Sclass_ConvVN2, Sclass_ClassVN using the three vector neuron layers 230, 240 and 250 respectively, but one or more of these vector neuron layers may be used to generate the class similarities is preferably calculated. As described above, in the present disclosure, a vector neuron layer used for similarity calculation is referred to as a "specific layer".

The similarity calculator 310a calculates the similarity by class Sclass using at least one of these similarities by class Sclass_ConvVN1, Sclass_ConvVN2, and Sclass_ClassVN, and obtains the reliability by class Rclass from this similarity by class Sclass. The similarity by class Sclass represents the similarity value S_value for each class i of the m-th machine learning model. Class-specific reliability Rclass represents the reliability value R_value for each class i of the m-th machine learning model. As the reliability function for obtaining the reliability value R_value from the similarity value S_value, for example, any of the following can be used.
R_value( _i )=H1 [S_value(i)]=S_value(i) (3a)
R_value(i)=H2[ _{S_value} (i)]=Ac(i)×Wt+S_value(i)×(1-Wt) (3b)
R_value(i)=H ₃ [S_value(i)]=Ac(i)×S_value(i) (3c)
Here, Ac(i) is an activation value corresponding to the judgment value of class i in the output layer of the machine learning model 200, and Wt is a weighting coefficient in the range of 0<Wt<1. The activation value Ac(i) is the same as the determination values Class1 to Class3 for each class shown in FIG.

_The reliability function H1 in the above equation (3a) is an identity function that takes the similarity value S_value itself as the reliability value R_value. The reliability function H2 in the above equation (3b) is a function that obtains the reliability value R_value by taking the weighted _average of the similarity value S_value and the activation value Ac. The reliability function H3 in the above equation ( _3c ) is a function that obtains the reliability value R_value by multiplying the similarity value S_value by the activation value Ac. Also, reliability functions other than these may be used. For example, a function may be used in which the power of the similarity value S_value is used as the reliability value R_value. In this way, the reliability by class Rclass can be determined as depending on the similarity by class Sclass. Further, it is preferable that the class-wise reliability Rclass has a positive correlation with the class-wise similarity Sclass.

The similarity by class Sclass is an index indicating the degree of similarity between the data to be discriminated and the teacher data for each class. Also, the class-by-class reliability Rclass can be used as an index indicating whether or not the data to be discriminated corresponds to each class.

In step S250, the list creation unit 322 outputs a discrimination result list RL in which a plurality of classes are arranged in order of reliability value R_value according to the class-by-class reliability Rclass. This discrimination result list RL is displayed on the display unit 150 .

FIG. 13 is an explanatory diagram showing an example of the determination result list RL. In this example, the discrimination result list RL includes priority, parameter m indicating the machine learning model, parameter i indicating the class, class name, and reliability value R_value. Multiple classes are arranged. In other embodiments, the discrimination result list RL may not include the parameter m indicating the machine learning model. Priority indicates the order of the confidence value R_value. The name of the print medium is exemplified as the class name. Note that the priority may be omitted. Also, one of the class parameter i and the class name may be omitted from the class information. Furthermore, the discrimination result list RL does not need to include all classes that can be discriminated by the M machine learning models, and may include only some of them. In general, when the number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, the total number of classes distinguishable by the M machine learning models is represented by ΣNm. . The discrimination result list RL may include only some of the ΣNm classes with higher reliability values R_value.

When the determination result list RL is displayed on the display unit 150, the user can observe the determination result list RL and determine the type of target print medium to be determined. For example, if a class assumed in advance by the user as the type of target print medium exists at the top of the discrimination result list RL, the expected class can be determined as the type of target print medium. . Also, if the user does not anticipate the type of target print media in advance, the class at the top of the discrimination result list RL can be determined as the type of target print media by comparing it with the reliability value. can do. If the reliability of the class at the top of the determination result list RL is low, it may be treated as an unknown print medium without being determined as the type of target print medium, and an operation may be performed accordingly. For example, the user may be warned that an unknown print medium is set.

In step S260, the user selects the class of the target print medium, ie, the type of the target print medium, from the determination result list RL, and instructs the print processing unit 112 to select it. Step S260 means (i) that the processor 110 selects one print medium type from the discrimination result list RL in response to an instruction from the user, and (ii) that the user displays "Accept". (iii) if there is no user's instruction within a predetermined period of time, the processor 110 selects the type of print medium at the top of the determination result list RL when giving an instruction; It may include any one of selecting the print media type at the top of the RL and/or stopping the process. In step S270, the print processing unit 112 selects print settings by referring to the print setting table PST according to the type of target print medium. In step S280, the print processing unit 112 executes printing according to the print settings. According to the procedure of FIG. 11, even if the type of target print medium is unclear, the machine learning model 200 can be used to determine the target print medium type. It is possible to run

As described above, in the first embodiment, since the discrimination result list RL in which a plurality of classes are arranged in order of reliability of each class is output, highly reliable class discrimination results can be obtained.

B. Second embodiment:
FIG. 14 is a block diagram showing functions of the class determination processing unit 114b in the second embodiment. The class determination processing unit 114b includes a similarity calculation unit 310b and a comprehensive determination unit 320b. The comprehensive determination section 320 b includes an in-class selection section 321 and a list creation section 322 . The configuration of the apparatus shown in FIGS. 1 and 2 and the procedure of processing shown in FIGS. 6 and 11 are substantially the same as in the first embodiment.

In the second embodiment, a plurality of discriminated data IM_1, IM_2, IM_3 obtained from the same discriminated object are input to machine learning models 200_1, 200_2, respectively. Note that when the number of pieces of data to be determined IM is represented by an integer P, P can be set to any number equal to or greater than 2. A parameter p is used to distinguish individual discriminated data IM. In the second embodiment, an example using three data to be determined IM_1, IM_2, IM_3 will be described. These discriminated data IM_1, IM_2, and IM_3 are spectral data measured by the spectrometer 30 from the same print medium.

The similarity calculation unit 310b calculates a similarity by class Sclass, which is the similarity between the feature spectrum Sp and the group of known feature spectra KSp, for each piece of data to be discriminated IM_p. Obtain another reliability Rclass_p. The reliability by class Rclass_p includes a parameter m indicating the machine learning model, a parameter i indicating the class, a parameter p indicating the data to be discriminated, and a reliability value R_value of the class. In other words, in the second embodiment, P pieces of class-by-class reliability Rclass_p are obtained for each class according to P pieces of data to be discriminated. These class-by-class reliability Rclass_p are the same as the class-by-class reliability Rclass described in FIG.

For each class, the intra-class selection unit 321 selects a reliability representative value, which is a statistical representative value among P reliability values R_value represented by P class-specific reliability Rclass_p, as the reliability of the class. Determined as a degree value. A "statistically representative value" means a median value, an average value, or a mode value. As a result, the class-by-class reliability Rclass similar to that described with reference to FIG. 5 is output from the in-class selector 321 .

The list creation unit 322 outputs a discrimination result list RL in which a plurality of classes are arranged in order of reliability values R_value according to the class-by-class reliability Rclass given from the in-class selection unit 321 . This processing is the same as in the first embodiment.

As described above, in the second embodiment, the reliability representative for each class is selected from the P reliability levels Rclass_p calculated using P pieces of data to be determined obtained from the same object to be determined. Values are obtained, and a discrimination result list RL is output using these reliability representative values, so that class discrimination results with higher reliability can be obtained.

C. Third Embodiment:
FIG. 15 is a block diagram showing functions of the class determination processing unit 114c in the third embodiment. The class determination processing unit 114c includes a similarity calculation unit 310c and a comprehensive determination unit 320c. Comprehensive determination section 320 c includes result selection section 325 . The configuration of the apparatus shown in FIGS. 1 and 2 and the procedure of processing shown in FIG. 6 are substantially the same as those of the first embodiment.

The similarity calculation unit 310c of the third embodiment is provided with the class discrimination result Dc obtained in the output layer of the machine learning model 200 in addition to the feature spectrum Sp. This class discrimination result Dc is a result of specifying one of a plurality of classes that can be discriminated by the machine learning model 200 as the discriminant class. The similarity calculator 310c calculates a discriminant class similarity Sdc regarding the discriminant class indicated by the class discrimination result Dc, and obtains a model reliability Rmodel dependent on the discriminant class similarity Sdc. The discriminant class similarity Sdc includes only the similarity values related to the discriminant class, and does not include the similarity values of classes other than the discriminant class. It is the same as Sclass and is calculated by the same formula as the similarity by class Sclass. Also, the same method as the method of obtaining the class-by-class reliability Rclass from the class-by-class similarity Sclass described in the first embodiment can be used as the method of obtaining the model reliability Rmodel from the discriminant class similarity Sdc. The model reliability Rmodel includes a parameter m indicating the machine learning model 200, a parameter Dc indicating the discriminant class, and a reliability value Rb.

The result selection unit 325 selects the discriminant class obtained from the machine learning model with the highest model reliability Rmodel among the M discriminant classes discriminated by the M machine learning models 200 as the class discrimination result for the data to be discriminated. Output as FRD. Alternatively, as the class discrimination result FRD, a list in which a plurality of classes are arranged in descending order of model reliability Rmodel may be output. In the example of FIG. 15, the number M of machine learning models 200 is 2, but in the third embodiment, an arbitrary value of 2 or more can be adopted as the integer M.

FIG. 16 is a flow chart showing the processing procedure of the medium discrimination/printing process in the third embodiment. The procedure in FIG. 16 is obtained by replacing steps S230 to S260 in FIG. 11 described in the first embodiment with steps S310 to S340, and other steps are the same as in FIG.

In step S310, the class discrimination processing unit 114c inputs data to be discriminated to a plurality of machine learning models 200, and calculates a discriminant class Dc and a feature spectrum Sp from the output of each machine learning model 200, respectively. In step S320, the similarity calculator 310c calculates the discriminant class similarity Sdc of the discriminant class Dc from the feature spectrum Sp obtained in response to the input of the data to be discriminated and the registered known feature spectrum group KSp. , a model reliability Rmodel that depends on the discriminant class similarity Sdc. In step S330, the result selection unit 325 outputs the class discrimination result FRD obtained from the machine learning model with high model reliability Rmodel. This class determination result FRD is given to the print processing unit 112 . In step S340, the print processing unit 112 selects the type of target print medium according to the class determination result FRD. The steps after the next step S270 are the same as in the first embodiment.

As described above, in the third embodiment, among the M discriminant classes obtained from the M machine learning models, the discriminant class obtained from the machine learning model with the highest model reliability Rmodel is applied to the data to be discriminated. Since it is output as a class discrimination result FRD, a highly reliable class discrimination result can be obtained.

D. Fourth Embodiment:
FIG. 17 is a block diagram showing functions of the class determination processing unit 114d in the fourth embodiment. The class determination processing unit 114d includes a similarity calculation unit 310d and a comprehensive determination unit 320d. The comprehensive determination section 320 d includes an intra-model selection section 324 and a result selection section 325 . 1 and 2 and the processing procedure shown in FIG. 6 are substantially the same as in the first embodiment, and the processing procedure shown in FIG. 16 is substantially the same as in the third embodiment. be.

In the fourth embodiment, as in the second embodiment, a plurality of discriminated data IM_1, IM_2, IM_3 obtained from the same discriminated object are input to machine learning models 200_1, 200_2, respectively. Similar to the third embodiment, the similarity calculator 310d is provided with the feature spectrum Sp and the class discrimination result Dc from each machine learning model 200. FIG.

The similarity calculator 310d calculates a discriminant class similarity Sdc regarding the discriminant class Dc for each discriminated data IM_p, and obtains a model reliability Rmodel_p that depends on the discriminant class similarity Sdc. The model reliability Rmodel_p includes a parameter m indicating the machine learning model, a parameter Dc indicating the discriminant class, a parameter p indicating data to be discriminated, and a model reliability value Rb. In other words, in the fourth embodiment, P model reliability levels Rmodel_p are obtained for each machine learning model according to P pieces of data to be discriminated. These model reliability degrees Rmodel_p are the same as the model reliability degrees Rmodel described in FIG. 15 in the third embodiment, except that the parameter p indicating the data to be discriminated is included.

For each machine learning model 200, the intra-model selection unit 324 selects the largest class among the P discriminant classes Dc included in the P model reliability Rmodel_p as the discriminant class Dc in the machine learning model 200. , adopt the model confidence including its discriminant class Dc. As a result, the intra-model selector 324 outputs the model reliability Rmodel similar to that described with reference to FIG. The result selection unit 325 outputs the class discrimination result FRD obtained from the machine learning model with the highest model reliability Rmodel. This processing is the same as in the third embodiment.

As described above, in the fourth embodiment, among the P discriminant classes obtained from each machine learning model using P discriminant data obtained from the same discriminant object, the class with the largest number is Since it is adopted as the discriminant class of the machine learning model, a more reliable class discrimination result can be obtained.

E. Fifth Embodiment:
FIG. 18 is a block diagram showing functions of the class determination processing unit 114e in the fifth embodiment. The class determination processing unit 114e includes a similarity calculation unit 310e and a comprehensive determination unit 320e. The comprehensive determination section 320 e includes an in-class selection section 326 , an in-model selection section 327 and a result selection section 328 . 1 and 2 and the processing procedure shown in FIG. 6 are substantially the same as in the first embodiment, and the processing procedure shown in FIG. 16 is substantially the same as in the third embodiment. be.

In the fifth embodiment, as in the second embodiment, a plurality of discriminated data IM_1, IM_2, IM_3 obtained from the same discriminated object are input to machine learning models 200_1, 200_2, respectively. The similarity calculator 310e is provided with the feature spectrum Sp from each machine learning model 200, as in the second embodiment.

The similarity calculator 310e calculates a similarity by class Sclass_p, which is the similarity between the feature spectrum Sp and the known feature spectrum group KSp, for each piece of data to be discriminated IM_p. The class similarity Sclass_p includes a parameter m indicating a machine learning model, a parameter i indicating a class, a parameter p indicating data to be discriminated, and a class similarity value S_value. In other words, in the fifth embodiment, P class similarities Sclass_p are obtained according to P pieces of data to be discriminated.

For each class, the intra-class selection unit 326 selects a similarity representative value, which is a statistical representative value among the P similarity values S_value represented by the P class similarities Sclass_p, as the similarity of the class. Determined as a degree value. A "statistically representative value" means a median value, an average value, or a mode value. As a result, the in-class selection unit 326 outputs a class-by-class similarity Sclass including a parameter m indicating a machine learning model, a parameter i indicating a class, and a class similarity value S_value.

For each machine learning model 200, the intra-model selection unit 327 selects the class corresponding to the highest similarity value among the similarity values S_value of each class represented by the class similarity Sclass as the machine learning model 200. Determine the corresponding discriminant class Dc. Further, the intra-model selection unit 327 adopts the similarity value S_value for this discriminant class Dc as the discriminant class similarity of the discriminant class Dc, and obtains the confidence value Rb of the model reliability Rmodel that depends on the discriminant class similarity S_value. . As a method for obtaining the model reliability Rmodel from the discriminant class similarity S_value, the same method as the method for obtaining the class-by-class reliability Rclass from the class-by-class similarity Sclass described in the first embodiment can be used. As a result, the in-model selector 327 outputs the model reliability Rmodel including the parameter m indicating the model, the parameter Dc indicating the discriminant class, and the reliability value Rb, as in the case of FIG. The result selection unit 325 outputs the class discrimination result FRD obtained from the machine learning model with the highest model reliability Rmodel. This processing is the same as in the fourth embodiment.

In the fifth embodiment, in step S310 of the procedure shown in FIG. 16, instead of using the discrimination result obtained from the output layer of the machine learning model 200, the similarity of each class represented by the similarity by class Sclass The difference from the fourth embodiment is that the class corresponding to the highest similarity value among the degree values S_value is determined as the discrimination class Dc corresponding to the machine learning model 200 . In this fifth embodiment, it is also possible to consider that the process of determining the discriminant class is performed in step S320 of FIG.

As described above, in the fifth embodiment, using P pieces of discriminated data obtained from the same object to be discriminated, P similarities for each class are obtained for each machine learning model. Since the discriminant class is determined using the degree of similarity, a more reliable class discrimination result can be obtained.

F. Method of calculating similarity:
For example, any one of the following three methods can be adopted as a method for calculating the similarity by class described above.
(1) A first calculation method M1 for obtaining similarity by class without considering the correspondence between the feature spectrum Sp and the partial region Rn in the group of known feature spectra KSp
(2) A second calculation method M2 for obtaining the similarity by class between the corresponding partial regions Rn of the feature spectrum Sp and the group of known feature spectra KSp.
(3) A third calculation method M3 for obtaining similarity by class without considering partial regions Rn at all
A method for calculating the similarity by class Sclass_ConvVN1 from the output of the ConvVN1 layer 230 will be sequentially described below in accordance with these three calculation methods M1, M2, and M3. In the following description, the parameter m of the machine learning model 200 and the parameter q of the discriminated data are omitted.

FIG. 19 is an explanatory diagram showing the first calculation method M1 of similarity by class. In the first calculation method M1, first, from the output of the ConvVN1 layer 230, which is a specific layer, a local similarity S(i,j,k) representing the similarity to each class i is calculated for each partial region k. Then, from these local similarities S(i,j,k), one of the three class similarities Sclass(i,j) shown on the right side of FIG. 19 is calculated. The similarity by class Sclass(i,j) is the same as the similarity by class Sclass_ConvVN1 shown in FIGS.

In the first calculation method M1, the local similarity S(i,j,k) is calculated using the following equation.
S(i,j,k)=max[G{Sp(j,k), KSp(i,j,k=all,q=all)}] (c1)
here,
i is a parameter indicating the class,
j is a parameter indicating a specific layer;
k is a parameter indicating the partial region Rn;
q is a parameter indicating a data number;
G{a,b} is a function for obtaining the similarity between a and b,
Sp(j,k) is a feature spectrum obtained from the output of a specific subregion k of a specific layer j according to the data to be discriminated;
KSp(i,j,k=all,q=all) is all data numbers in all partial regions k of a specific layer j associated with class i in the known feature spectrum group KSp shown in FIG. known feature spectrum of q,
max[X] is a logical operation that takes the maximum of the X values.
As the function G{a,b} for obtaining the degree of similarity, for example, a formula for obtaining cosine similarity or a formula for obtaining similarity according to distance can be used.

The three types of similarities by class Sclass(i,j) shown on the right side of FIG. , or obtained by taking the minimum value. Whether to use the maximum value, the average value, or the minimum value depends on the intended use of the class discrimination process. For example, when the purpose is to discriminate an object using a natural image, the class similarity Sclass(i, j) is preferably determined. Also, when the purpose is to discriminate the type of printing medium, or when the purpose is to judge quality using an image of an industrial product, the local similarity S(i, j, k) for each class i It is preferable to obtain the similarity by class Sclass(i,j) by taking the minimum value. Also, there may be cases where it is preferable to find the similarity by class Sclass(i,j) by averaging the local similarities S(i,j,k) for each class i. Which of these three types of calculations is used is preset by the user experimentally or empirically.

As described above, in the first calculation method M1 of similarity by class,
(1) Similarity between the feature spectrum Sp obtained from the output of a specific subregion k of a specific layer j and all known feature spectra KSp associated with that specific layer j and each class i, depending on the data to be discriminated Find the local similarity S(i,j,k), which is the degree of
(2) Class-specific similarity Sclass(i,j) by taking the maximum value, average value, or minimum value of local similarity S(i,j,k) for a plurality of partial regions k for each class i. ).
According to this first calculation method M1, the similarity by class Sclass(i,j) can be obtained by a relatively simple calculation and procedure.

FIG. 20 is an explanatory diagram showing the second calculation method M2 of similarity by class. In the second calculation method M2, the local similarity S(i,j,k) is calculated using the following equation instead of the above equation (c1).
S(i,j,k)=max[G{Sp(j,k), KSp(i,j,k,q=all)}] (c2)
here,
KSp(i,j,k,q=all) is the number of all data numbers q in a specific subregion k of a specific layer j associated with class i in the group of known feature spectra KSp shown in FIG. Known feature spectrum.

In the above-described first calculation method M1, the known feature spectrum KSp (i, j, k=all, q=all) in all partial regions k of the specific layer j is used, whereas the second calculation method M1 Method M2 uses only the known feature spectrum KSp(i,j,k,q=all) for the same subregion k as the subregion k of the feature spectrum Sp(j,k). Other methods in the second calculation method M2 are the same as the first calculation method M1.

In the second class similarity calculation method M2,
(1) Depending on the data to be discriminated, the feature spectrum Sp obtained from the output of a particular sub-region k of a particular layer j and all known sub-regions k of that particular layer j and all known Find the local similarity S(i,j,k), which is the similarity with the feature spectrum KSp,
(2) By taking the maximum value, average value, or minimum value of the local similarities S(i,j,k) for a plurality of partial regions k for each class i, the similarity by class Sclass(i, j).
The class similarity Sclass(i,j) can also be obtained by the second calculation method M2 through relatively simple calculations and procedures.

FIG. 21 is an explanatory diagram showing the third calculation method M3 of similarity by class. In the third calculation method M3, the class similarity Sclass(i,j) is calculated from the output of the ConvVN1 layer 230, which is the specific layer, without obtaining the local similarity S(i,j,k).

The class similarity Sclass(i,j) obtained by the third calculation method M3 is calculated using the following equation.
Sclass(i,j)=max[G{Sp(j,k=all), KSp(i,j,k=all,q=all)}] (c3)
here,
Sp(j, k=all) is the feature spectrum obtained from the output of all sub-regions k of the specific layer j according to the data to be discriminated.

As described above, in the third calculation method M3 of similarity by class,
(1) By class, which is the similarity between all feature spectra Sp obtained from the output of a specific layer j according to the data to be discriminated and all known feature spectra KSp associated with the specific layer j and each class i A similarity Sclass(i,j) is obtained for each class.
According to the third calculation method M3, the similarity by class Sclass(i,j) can be obtained by a simpler calculation and procedure.

All of the three calculation methods M1 to M3 described above are methods for calculating the similarity by class with respect to each specific layer i. As described above, in this embodiment, one or more of the vector neuron layers 230, 240, and 250 shown in FIG. 3 can be used as specific layers to calculate similarity by class. When using a plurality of specific layers, it is possible to determine similarity by class, for example, as follows.

FIG. 22 is an explanatory diagram showing a method of selecting similarities by class when using a plurality of specific layers. This method selects the similarity by class obtained from the specific layer showing the most statistically significant determination result among the plurality of specific layers. In the example of FIG. 22, the ConvVN1 layer 230 and the ConvVN2 layer 240 are used as specific layers. First, for each subregion k of the ConvVN1 layer 230, a class having the maximum local similarity S(i,j,k) is determined, and the class parameter value i of that class is assigned to each subregion k. do. Note that the class parameter value i is a value that indicates the order in a plurality of classes. Similarly, for the ConvVN2 layer 240, a class with the maximum local similarity S(i,j,k) is determined, and the class parameter value i of that class is assigned to each partial region k.

In the method of FIG. 22, the variance is further calculated for the distribution of the class parameter values i in the plurality of partial regions k in each specific layer. This variance is the statistical variance value for the class parameter value i. In the example of FIG. 22, the ConvVN1 layer 230 has a variance of 0.14 and the ConvVN2 layer 240 has a variance of 0.22. Among these specific layers 230 and 240, it is expected that the more biased the distribution of the class parameter value i, the clearer the determination result. In other words, among the plurality of specific layers, the similarity by class obtained for the specific layer with the smallest variance is adopted. According to this method, even when a plurality of similarities by class are obtained using a plurality of specific layers, an appropriate similarity by class can be selected from among them.

G. How to calculate the output vector of each layer of the machine learning model:
The method of calculating the output of each layer in the machine learning model 200 shown in FIG. 3 is as follows. The machine learning model 200 shown in FIG. 4 is also the same except for the individual parameter values.

Each node of the PrimeVN layer 220 takes the scalar output of the 1x1x32 nodes of the Conv layer 210 as a 32-dimensional vector and multiplies this vector by the transformation matrix to obtain the vector output of that node. This transformation matrix is an element of a kernel with a surface size of 1×1 and is updated as the machine learning model 200 learns. Note that it is also possible to integrate the processing of the Conv layer 210 and the PrimeVN layer 220 into one primary vector neuron layer.

ここで、
Ｍ^L _iは、下位層Ｌにおけるｉ番目のノードの出力ベクトル、
Ｍ^L+1 _jは、上位層Ｌ＋１におけるｊ番目のノードの出力ベクトル、
ｖ_ijは、出力ベクトルＭ^L+1 _jの予測ベクトル、
Ｗ^L _ijは、下位層Ｌの出力ベクトルＭ^L _iから予測ベクトルｖ_ijを算出するための予測行列、
ｕ_jは、予測ベクトルｖ_ijの和、すなわち線形結合、である和ベクトル、
ａ_jは、和ベクトルｕ_jのノルム|ｕ_j|を正規化することによって得られる正規化係数であるアクティベーション値、
Ｆ（Ｘ）は、Ｘを正規化する正規化関数である。 When the PrimeVN layer 220 is called the "lower layer L" and the ConvVN1 layer 230 adjacent to it on the upper side is called the "upper layer L+1", the output of each node in the upper layer L+1 is determined using the following equation. .

here,
M ^L _i is the output vector of the i-th node in the lower layer L,
M ^L+1 _j is the output vector of the j-th node in the upper layer L+1,
v _ij is the prediction vector of the output vector M ^L+1 _j ;
W ^L _ij is a prediction matrix for calculating the prediction vector v _ij from the output vector M ^L _i of the lower layer L;
u _j is the sum vector that is the sum, a linear combination, of the prediction vectors v _ij ;
a _j is the activation value, the normalization factor obtained by normalizing the norm |u _j | of the sum vector u _j ;
F(X) is the normalization function that normalizes X.

ここで、
ｋは、上位層Ｌ＋１のすべてのノードに対する序数、
βは、任意の正の係数である調整パラメーターであり、例えばβ＝１である。 As the normalization function F(X), for example, the following formula (E3a) or formula (E3b) can be used.

here,
k is the ordinal number for all nodes in the upper layer L+1,
β is a tuning parameter that is any positive coefficient, eg β=1.

In the above equation (E3a), the activation value a _j is obtained by normalizing the norm |u _j | of the sum vector u _j for all the nodes of the upper layer L+1 with the softmax function. On the other hand, in equation (E3b), the activation value a _j is obtained by dividing the norm |u _j | of the sum vector u _j by the sum of the norms |u _j | for all nodes in the upper layer L+1. As the normalization function F(X), functions other than the formulas (E3a) and (E3b) may be used.

The ordinal number i in the above equation (E2) is conveniently assigned to the nodes of the lower layer L used to determine the output vector M ^L+1 _j of the j-th node in the upper layer L+1. take a value. Also, the integer n is the number of nodes in the lower layer L used to determine the output vector M ^L+1 _j of the j-th node in the upper layer L+1. Therefore, the integer n is given by the following equation.
n=Nk×Nc (E5)
Here, Nk is the surface size of the kernel, and Nc is the number of channels in the PrimeVN layer 220, which is the lower layer. In the example of FIG. 3, Nk=5 and Nc=26, so n=130.

One kernel used to determine the output vector of the ConvVN1 layer 230 has 1×5×26=130 elements with a surface size of kernel size 1×5 and a depth of 26 channels in the lower layer. , and each of these elements is the prediction matrix W ^L _ij . Also, 20 sets of these kernels are required to generate output vectors for the 20 channels of the ConvVN1 layer 230 . Therefore, the number of kernel prediction matrices W ^L _ij used to determine the output vectors of the ConvVN1 layer 230 is 130×20=2600. These prediction matrices W ^L _ij are updated as the machine learning model 200 learns.

As can be seen from the above equations (E1) to (E4), the output vector M ^L+1 _j of each node of the upper layer L+1 is obtained by the following calculation.
(a) Multiply the output vector M ^L _i of each node of the lower layer L by the prediction matrix W ^L _ij to obtain the prediction vector v _ij ,
(b) obtaining a sum vector u _j that is the sum of the prediction vectors v _ij obtained from each node of the lower layer L, that is, a linear combination;
(c) normalizing the norm |u _j | of the sum vector u _j to obtain an activation value a _j that is a normalization factor;
(d) Divide the sum vector u _j by the norm |u _j | and multiply it by the activation value a _j .

Note that the activation value a _j is a normalization factor obtained by normalizing the norm |u _j | with respect to all nodes in the upper layer L+1. Therefore, the activation value a _j can be considered as an index indicating the relative output strength of each node among all nodes in the upper layer L+1. The norms used in equations (E3), (E3a), (E3b), and (4) are typically L2 norms representing vector lengths. At this time, the activation value a _j corresponds to the vector length of the output vector M ^L+1 _j . Since the activation value a _j is only used in the above equations (E3) and (E4), it need not be output from the node. However, it is also possible to configure the upper layer L+1 so as to output the activation value a _j to the outside.

The configuration of the vector neural network is almost the same as that of the capsule network, and the vector neurons of the vector neural network correspond to the capsules of the capsule network. However, the calculations according to the above equations (E1) to (E4) used in the vector neural network are different from those used in the capsule network. The biggest difference between the two is that in the capsule network, the prediction vector v _ij on the right side of the above equation (E2) is multiplied by a weight, and the weight is searched for by repeating dynamic routing multiple times. be. On the other hand, in the vector neural network of this embodiment, since the output vector M ^L+1 _j is obtained by calculating the above-described equations (E1) to (E4) once in order, there is no need to repeat dynamic routing. , which has the advantage of being faster to compute. In addition, the vector neural network of the present embodiment requires a smaller amount of memory for calculation than the capsule network, and according to experiments by the inventor of the present disclosure, the amount of memory is about 1/2 to 1/3. There is also an advantage that it can be done with

A vector neural network is similar to a capsule network in that it uses nodes whose inputs and outputs are vectors. Therefore, the advantages of using vector neurons are also shared with capsule networks. In addition, the plurality of layers 210 to 250 represent features of larger regions as they go higher, and features of smaller regions as they go lower, which is the same as an ordinary convolutional neural network. Here, the "feature" means a characteristic part included in the input data to the neural network. Vector neural networks and capsule networks are superior to ordinary convolutional neural networks in that the output vector of a node contains spatial information representing the spatial information of the feature represented by that node. That is, the vector length of the output vector of a certain node represents the existence probability of the feature represented by that node, and the vector direction represents spatial information such as the direction and scale of that feature. Therefore, the vector directions of the output vectors of two nodes belonging to the same layer represent the positional relationship of their respective features. Alternatively, it can be said that the vector directions of the output vectors of the two nodes represent variations of the feature. For example, for a node corresponding to the "eyes" feature, the direction of the output vector could represent variations in the fineness of the eyes, how they are hung, and so on. In a normal convolutional neural network, it is said that spatial information of features disappears due to pooling processing. As a result, vector neural networks and capsule networks have the advantage of being superior to ordinary convolutional neural networks in their ability to identify input data.

The advantages of vector neural networks can also be considered as follows. That is, the vector neural network has the advantage that the output vectors of the nodes represent the features of the input data as coordinates in a continuous space. Therefore, the output vectors can be evaluated such that if the vector directions are close, the features are similar. In addition, even if the feature included in the input data is not covered by the teacher data, there is an advantage that the feature can be determined by interpolation. On the other hand, a conventional convolutional neural network suffers from chaotic compression due to pooling processing, and thus has the disadvantage that the features of input data cannot be expressed as coordinates in a continuous space.

The output of each node of the ConvVN2 layer 240 and the ClassVN layer 250 is similarly determined using the above-described equations (E1) to (E4), so detailed description is omitted. The resolution of the ClassVN layer 250, which is the highest layer, is 1×1, and the number of channels is n1.

The output of ClassVN layer 250 is transformed into a plurality of decision values Class0-Class2 for known classes. These judgment values are values normalized by a softmax function. Specifically, for example, from the output vector of each node of the ClassVN layer 250, the vector length of the output vector is calculated, and the vector length of each node is normalized by the softmax function. to obtain the decision value for each class. As described above, the activation value a _j obtained by the above equation (E3) is a value corresponding to the vector length of the output vector M ^L+1 _j and is normalized. Therefore, the activation value a _j in each node of the ClassVN layer 250 may be output and used as it is as the judgment value for each class.

In the above-described embodiment, as the machine learning model 200, a vector neural network that obtains an output vector by calculating the above equations (E1) to (E4) was used. A capsule network disclosed in Japanese Patent Publication No. 2009/083553 may be used.

・Other embodiments:
The present disclosure is not limited to the embodiments described above, and can be implemented in various forms without departing from the scope of the present disclosure. For example, the present disclosure can also be implemented in the following aspects. The technical features in the above embodiments corresponding to the technical features in each form described below are used to solve some or all of the problems of the present disclosure, or to achieve some or all of the effects of the present disclosure. In order to achieve the above, it is possible to appropriately replace or combine them. Also, if the technical features are not described as essential in this specification, they can be deleted as appropriate.

<1> According to the first aspect of the present disclosure, when M is an integer of 1 or more, using M vector neural network type machine learning models having a plurality of vector neuron layers, A method is provided for determining a class. The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. and The method includes the step of: (a) preparing M sets of known feature spectra groups associated with the M machine learning models, wherein the M sets of known feature spectra groups are Nm for each machine learning model; (b) a known feature spectrum group obtained from the output of a specific layer of the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of the classes is input; using the M machine learning models and the M known feature spectra groups to perform class discrimination processing of the data to be discriminated. The step (b) includes (b1) calculating M feature spectra from the output of the specific layer in response to the input of the data to be discriminated to the M machine learning models; and (b2) the As the similarity between the M feature spectra and the M sets of known feature spectra, calculate the class similarity for each of the Nm classes in each machine learning model, and the class similarity (b3) obtaining M sets of class-wise confidences associated with the M machine learning models by obtaining dependent class-wise confidences; and outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in order of reliability of each class.
According to this method, since a discrimination result list in which a plurality of classes are arranged in order of reliability of each class is output, highly reliable class discrimination results can be obtained.

<2> In the above method, when P is an integer of 2 or more, the discriminated data includes P discriminated data obtained from the same discriminated object. The step (b1) includes a step of calculating P feature spectra from the output of each machine learning model in response to the input of the P pieces of discriminated data to each machine learning model, and the step (b2) is , with respect to each set of reliability by class among the M sets of reliability by class, using the P feature spectra obtained from each machine learning model, as the reliability by class for each set, P a step of obtaining the confidences by class; determining a confidence value for each class represented by the set of class confidences.
According to this method, a reliability representative value for each class is obtained from among P reliability degrees for each class calculated using P pieces of data to be discriminated obtained from the same object to be discriminated. Since the classification result list is output using the reliability representative value, it is possible to obtain class classification results with higher reliability.

<3> In the above method, the reliability by class is (1) equal to the similarity by class, or (2) the similarity value of each class in the similarity by class and the machine learning model Activation value corresponding to the judgment value of each class in the output layer and the result of taking a weighted average, or (3) the similarity value of each class of the similarity by class and the activation for each class It may be either the result of multiplying the value with
According to this method, the class-by-class reliability can be obtained by a simple calculation.

<4> According to the second aspect of the present disclosure, when M is an integer of 2 or more, using M vector neural network type machine learning models having a plurality of vector neuron layers, A method is provided for determining a class. This method includes (a) a step of preparing M sets of known feature spectra groups associated with the M machine learning models, wherein the M sets of known feature spectra groups are Nm for each machine learning model. (b) a known feature spectrum group obtained from the output of a specific layer of the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of the classes is input; using the M machine learning models and the M known feature spectra groups to perform class discrimination processing of the data to be discriminated. The step (b) includes (b1) calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and generating M discriminant classes (b2) calculating a discriminant class similarity for each of the M discriminant classes as a similarity between the M feature spectra and the M sets of known feature spectra, and obtaining a model confidence for each of the M machine learning models by obtaining a model confidence dependent on the discriminant class similarity; (b3) among the M discriminant classes, the model confidence; outputting the discriminant class obtained from the machine learning model with the highest , as a class discrimination result for the data to be discriminated.
According to this method, of the M discriminant classes obtained from the M machine learning models, the discriminant class obtained from the machine learning model with the highest model reliability is output as the class discrimination result for the data to be discriminated. Therefore, highly reliable class discrimination results can be obtained.

<5> In the above method, when P is an integer of 2 or more, the data to be determined includes P pieces of data to be determined obtained from the same object to be determined, and the step (b1) includes: a step of determining P discriminant classes according to the input of the P discriminant data to the machine learning model; selecting as discriminant classes in the machine learning model.
According to this method, the class with the largest number of P discriminant classes obtained from each machine learning model using P discriminant data obtained from the same discriminant object is selected as the class of the machine learning model. Since it is adopted as a discriminant class, a more reliable class discrimination result can be obtained.

<6> In the above method, when P is an integer of 2 or more, the data to be determined includes P pieces of data to be determined obtained from the same object to be determined, and the step (b1) includes: For each of the M sets of machine learning models, using the P feature spectra obtained from each machine learning model, as the class similarity associated with the machine learning model, P class similarities and a similarity representative value, which is a statistically representative value among the P similarity values for each class represented by the P class similarities, associated with the machine learning model determining the similarity value of each class represented by the similarity by class, and the class corresponding to the highest similarity value among the similarity values of each class represented by the similarity by class, and determining the discriminant class corresponding to the machine learning model. The step (b2) includes adopting the highest similarity value used in determining the discriminant class corresponding to each machine learning model in the step (b1) as the discriminant class similarity. It can be a thing.
According to this method, P similarities by class are obtained for each machine learning model using P pieces of data to be discriminated obtained from the same object to be discriminated, and P similarities by class are used. Since the discriminant class is determined by the method, a more reliable class discrimination result can be obtained.

<7> In the above method, the model reliability is (1) equal to the discriminant class similarity, or (2) the discriminant class similarity and the discriminant class judgment value in the output layer of the machine learning model. or (3) the result of multiplying the discriminant class similarity by the activation value for the discriminant class. good too.
According to this method, the model reliability can be obtained by a simple calculation.

<8> In the above method, the integer M is 2 or more, the M machine learning models are trained using the corresponding M teacher data groups, and the M teacher data groups may be in a state equivalent to being grouped into N sets by clustering processing.
According to this method, the teacher data group used for learning each machine learning model is grouped by the clustering process, so that the accuracy of the class discrimination process by the machine learning model can be improved.

<9> In the above method, the specific layer includes vector neurons arranged on a plane defined by two axes, a first axis and a second axis, along a third axis in a direction different from the two axes. in the specific layer, specified by a plane position defined by the position of the first axis and the position of the second axis, and along the third axis A region containing multiple channels is called a partial region. For each partial region among the plurality of partial regions included in the specific layer, the feature spectrum is obtained by: and (ii) by multiplying each element value of said first feature spectrum by a normalization factor corresponding to the vector length of said output vector. and (iii) a third feature spectrum in which said normalization factors are arranged across said plurality of channels along said third axis. may be
According to this method, the degree of similarity can be obtained using any one of three types of characteristic spectra obtained from the output vector of a specific layer.

<10> In the above method, in the step (b2), the feature spectrum obtained from the output of the specific partial region of the specific layer, and the characteristic spectrum associated with the specific layer and each class according to the data to be discriminated a step of obtaining a plurality of local similarities indicating similarity to each class with respect to the plurality of partial regions of the specific layer by obtaining local similarities that are degrees of similarity to all the known feature spectra; and obtaining the class similarity or the discriminant class similarity by taking the maximum value, average value, or minimum value of the plurality of local similarities for the plurality of partial regions. good too.
According to this method, the similarity by class or the discriminant class similarity can be calculated by relatively simple calculations.

<11> In the above method, the step (b2) includes the characteristic spectrum obtained from the output of the specific partial region of the specific layer, the specific partial region of the specific layer and the Obtaining a plurality of local similarities indicating similarities to each class with respect to the plurality of partial regions of the specific layer by obtaining local similarities that are similarities to all the known feature spectra associated with each class. and determining the similarity by class or the discriminant class similarity by taking the maximum value, average value, or minimum value of the plurality of local similarities for the plurality of partial regions for each class. and may be included.
According to this method, the similarity by class or the discriminant class similarity can be calculated by relatively simple calculations.

<12> In the above method, the step (b2) includes all the characteristic spectra obtained from the output of the specific layer and all the characteristic spectra associated with the specific layer and each class according to the data to be discriminated. A step of determining the similarity by class or the discriminant class similarity may be included by determining the similarity with the known feature spectrum for each class.
According to this method, the similarity by class or the discriminant class similarity can be calculated by a simpler calculation.

<13> According to the third aspect of the present disclosure, when M is an integer of 1 or more, using M vector neural network type machine learning models having a plurality of vector neuron layers, An information processing device for discriminating classes is provided. This information processing apparatus includes a memory that stores the M machine learning models, and a processor that performs calculations using the M machine learning models. The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. and The processor performs (a) a process of reading M sets of known feature spectra groups associated with the M machine learning models from the memory, wherein the M sets of known feature spectra groups are each machine learning model A process including a group of known feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of Nm classes is input to ( b) using the M machine learning models and the M known feature spectrum groups to perform class discrimination processing of the data to be discriminated. The process (b) includes (b1) a process of calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and (b2) the As the similarity between the M feature spectra and the M sets of known feature spectra, calculate the class similarity for each of the Nm classes in each machine learning model, and the class similarity (b3) a process of obtaining M sets of class confidences associated with the M machine learning models by obtaining the class confidences that depend on them; and (b3) the M sets of class confidences represented by a process of outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in order of reliability of each class.
According to this information processing apparatus, since a discrimination result list in which a plurality of classes are arranged in order of reliability of each class is output, highly reliable class discrimination results can be obtained.

<14> According to the fourth aspect of the present disclosure, when M is an integer of 2 or more, using M vector neural network type machine learning models having a plurality of vector neuron layers, An information processing device for discriminating classes is provided. This information processing apparatus includes a memory that stores the M machine learning models, and a processor that performs calculations using the M machine learning models. The processor performs (a) a process of reading M sets of known feature spectra groups associated with the M machine learning models from the memory, wherein the M sets of known feature spectra groups are each machine learning model A process including a group of known feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of Nm classes is input to ( b) using the M machine learning models and the M known feature spectrum groups to perform class discrimination processing of the data to be discriminated. The process (b) includes (b1) calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and generating M discriminant classes and (b2) calculating the discriminant class similarity for each of the M discriminant classes as the similarity between the M feature spectra and the M sets of known feature spectra, (b3) a process of obtaining model reliability for each of the M machine learning models by obtaining model reliability that depends on the discriminant class similarity; and (b3) the model reliability among the M discriminant classes outputting the discriminant class obtained from the machine learning model with the highest value as the class discrimination result for the data to be discriminated.
According to this information processing device, among the M discriminant classes obtained from the M machine learning models, the discriminant class obtained from the machine learning model with the highest model reliability is used as the class discrimination result for the data to be discriminated. Since it is output, highly reliable class discrimination results can be obtained.

<15> According to the fifth aspect of the present disclosure, when M is an integer of 1 or more, using M vector neural network type machine learning models having a plurality of vector neuron layers, A computer program is provided that causes a processor to perform a class determination process to determine a class. The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. and The computer program includes (a) a process of reading from memory M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are each machine learning model; A process including a group of known feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of Nm classes is input to ( b) a computer program that causes the processor to execute a process of classifying the data to be classified using the M machine learning models and the M known feature spectrum groups. The process (b) includes (b1) a process of calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and (b2) the As the similarity between the M feature spectra and the M sets of known feature spectra, calculate the class similarity for each of the Nm classes in each machine learning model, and the class similarity (b3) a process of obtaining M sets of class confidences associated with the M machine learning models by obtaining the class confidences that depend on them; and (b3) the M sets of class confidences represented by a process of outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in order of reliability of each class.
According to this computer program, since a discrimination result list in which a plurality of classes are arranged in order of reliability of each class is output, a highly reliable class discrimination result can be obtained.

<16> According to the sixth aspect of the present disclosure, when M is an integer of 2 or more, using M vector neural network type machine learning models having a plurality of vector neuron layers, A computer program is provided that causes a processor to perform a class determination process to determine a class. This computer program includes (a) a process of reading out from a memory M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are each machine learning model; A process including a group of known feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of Nm classes is input to ( b) a computer program that causes the processor to execute a process of classifying the data to be classified using the M machine learning models and the M known feature spectrum groups. The process (b) includes (b1) calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models, and generating M discriminant classes and (b2) calculating the discriminant class similarity for each of the M discriminant classes as the similarity between the M feature spectra and the M sets of known feature spectra, (b3) a process of obtaining model reliability for each of the M machine learning models by obtaining model reliability that depends on the discriminant class similarity; and (b3) the model reliability among the M discriminant classes outputting the discriminant class obtained from the machine learning model with the highest value as the class discrimination result for the data to be discriminated.
According to this computer program, among the M discriminant classes obtained from the M machine learning models, the discriminant class obtained from the machine learning model with the highest model reliability is output as the class discrimination result for the data to be discriminated. Therefore, highly reliable class discrimination results can be obtained.

The present disclosure can also be implemented in various forms other than those described above. For example, it can be realized in the form of a computer program for realizing the function of the class discrimination device, a non-transitory storage medium in which the computer program is recorded, or the like.

DESCRIPTION OF SYMBOLS 10... Printer, 20... Information processing apparatus, 30... Spectral measuring instrument, 110... Processor, 112... Print process part, 114... Class discrimination process part, 120... Memory, 130... Interface circuit, 150... Display part, 200... Machine Learning model 210 Convolution layer 220 Primary vector neuron layer 230 First convolution vector neuron layer 240 Second convolution vector neuron layer 250 Classification vector neuron layer 310 Similarity calculator 320 Synthesis Determination unit 321 In-class selection unit 322 List creation unit 324 In-model selection unit 325 Result selection unit 326 In-class selection unit 327 In-model selection unit 328 Result selection unit

Claims (16)

A method for discriminating a class of data to be discriminated using M machine learning models of a vector neural network type having a plurality of vector neuron layers, where M is an integer of 1 or more,
The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. When
The method includes:
(a) A step of preparing M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are Nm classes for each machine learning model a known feature spectrum group obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each is input;
(b) using the M machine learning models and the M known feature spectrum groups to perform class discrimination processing of the data to be discriminated;
including
The step (b) is
(b1) calculating M feature spectra from the output of the specific layer in response to the input of the data to be discriminated to the M machine learning models;
(b2) Calculate the similarity by class for each of the Nm classes in each machine learning model as the similarity between the M feature spectra and the M sets of known feature spectra, and obtaining M sets of class confidences associated with the M machine learning models by determining class confidences that depend on class similarities;
(b3) outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in the order of reliability of each class represented by the M sets of reliability by class; ,
A method, including 2. The method of claim 1, wherein
When P is an integer of 2 or more, the data to be discriminated includes P pieces of data to be discriminated obtained from the same object to be discriminated,
The step (b1) includes a step of calculating P feature spectra from the output of each machine learning model in response to the input of the P pieces of discriminated data to each machine learning model,
In the step (b2), with respect to each set of class-based reliability among the M sets of class-based reliability,
Using the P feature spectra obtained from each machine learning model, obtaining P class confidences as the class confidences of each set;
A reliability representative value, which is a statistically representative value among the P reliability values for each class represented by the P class confidences, is calculated for each set of class confidences determining as a confidence value for the class;
A method, including 3. A method according to claim 1 or 2,
The reliability by class is
(1) equal to the similarity by class, or
(2) A result of taking a weighted average of the similarity value of each class in the similarity by class and the activation value corresponding to the judgment value of each class in the output layer of the machine learning model, or
(3) As a result of multiplying the similarity value of each class of the similarity by class and the activation value for each class,
A method that is either A method for discriminating a class of data to be discriminated using M machine learning models of a vector neural network type having a plurality of vector neuron layers, where M is an integer of 2 or more,
(a) A step of preparing M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are Nm classes for each machine learning model a known feature spectrum group obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each is input;
(b) using the M machine learning models and the M known feature spectrum groups to perform class discrimination processing of the data to be discriminated;
including
The step (b) is
(b1) calculating M feature spectra from the output of the specific layer and determining M discrimination classes in response to the input of the data to be discriminated to the M machine learning models;
(b2) Calculate the discriminant class similarity for each of the M discriminant classes as the similarity between the M feature spectra and the M sets of known feature spectra, and calculate the discriminant class similarity for each of the M discriminant classes obtaining a model confidence for each of the M machine learning models by determining a dependent model confidence;
(b3) outputting the discriminant class obtained from the machine learning model with the highest model reliability among the M discriminant classes as a class discrimination result for the data to be discriminated;
A method, including 5. The method of claim 4, wherein
When P is an integer of 2 or more, the data to be discriminated includes P pieces of data to be discriminated obtained from the same object to be discriminated,
The step (b1) is
determining P discriminant classes in response to the P discriminated data inputs to each machine learning model;
A step of selecting the class with the largest number of P discriminant classes obtained from each machine learning model as the discriminant class in the machine learning model;
A method, including 5. The method of claim 4, wherein
When P is an integer of 2 or more, the data to be discriminated includes P pieces of data to be discriminated obtained from the same object to be discriminated,
The step (b1) includes, for each of the M sets of machine learning models,
Using the P feature spectra obtained from each machine learning model, obtaining P class similarities as class similarities associated with the machine learning models;
A similarity representative value, which is a statistically representative value among the P similarity values for each class represented by the P class similarities, is a class similarity associated with the machine learning model determining as a similarity value for each class represented by
determining the class corresponding to the highest similarity value among the similarity values of each class represented by the similarity by class as the discriminant class corresponding to the machine learning model;
including
The step (b2) includes adopting the highest similarity value used in determining the discriminant class corresponding to each machine learning model in the step (b1) as the discriminant class similarity. ,
Method. The method according to any one of claims 4 to 6,
The model reliability is
(1) equal to the discriminant class similarity, or
(2) a weighted average result of the discriminant class similarity and the activation value corresponding to the discriminant class decision value in the output layer of the machine learning model, or
(3) As a result of multiplying the discriminant class similarity by the activation value for the discriminant class,
A method that is either A method according to any one of claims 1 to 7,
The integer M is 2 or more,
The M machine learning models are learned using corresponding M teacher data groups,
The method according to claim 1, wherein the M training data groups are in a state equivalent to being grouped into N sets by clustering processing. A method according to any one of claims 1 to 8,
In the specific layer, vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes. having a configuration that
In the specific layer, when a region specified by a planar position defined by the position of the first axis and the position of the second axis and including the plurality of channels along the third axis is called a partial region,
For each partial region of a plurality of partial regions included in the specific layer, the characteristic spectrum is
(i) a first type feature spectrum in which a plurality of element values of an output vector of each vector neuron included in the partial region are arranged over the plurality of channels along the third axis;
(ii) a feature spectrum of the second type obtained by multiplying each element value of the feature spectrum of the first type by a normalization factor corresponding to the vector length of the output vector;
(iii) a third type of feature spectrum in which the normalization factors are arranged across the plurality of channels along the third axis;
A method, called as one of 10. The method of claim 9, wherein
The step (b2) is
Local similarity, which is a degree of similarity between the feature spectrum obtained from the output of a specific partial region of the specific layer and all the known feature spectra associated with the specific layer and each class, according to the data to be discriminated obtaining a plurality of local similarities indicative of the similarity for each class with respect to the plurality of sub-regions of the specific layer;
obtaining the similarity by class or the discriminant class similarity by taking the maximum value, average value, or minimum value of the plurality of local similarities regarding the plurality of partial regions for each class;
A method, including 10. The method of claim 9, wherein
The step (b2) is
the feature spectrum obtained from the output of the specific partial region of the specific layer and all the known feature spectra associated with the specific partial region and each class of the specific layer, according to the data to be discriminated; a step of obtaining a plurality of local similarities indicating similarities for each class with respect to the plurality of partial regions of the specific layer by obtaining local similarities that are similarities;
obtaining the similarity by class or the discriminant class similarity by taking the maximum value, average value, or minimum value of the plurality of local similarities regarding the plurality of partial regions for each class;
A method, including 10. The method of claim 9, wherein
The step (b2) is
obtaining, for each class, similarities between all the feature spectra obtained from the output of the specific layer and all the known feature spectra associated with the specific layer and each class, according to the data to be discriminated; determining the class-wise similarity or the discriminant class similarity by: An information processing device that executes a discrimination process for discriminating a class of data to be discriminated using M machine learning models of a vector neural network type having a plurality of vector neuron layers, where M is an integer of 1 or more. hand,
a memory that stores the M machine learning models;
a processor that performs operations using the M machine learning models;
with
The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. When
The processor
(a) A process of reading from the memory M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are Nm for each machine learning model A process including a known feature spectrum group obtained from the output of a specific layer of the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of the classes is input;
(b) using the M machine learning models and the M known feature spectrum groups, a process of performing class discrimination processing of the data to be discriminated;
and run
The processing (b) is
(b1) a process of calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models;
(b2) Calculate the similarity by class for each of the Nm classes in each machine learning model as the similarity between the M feature spectra and the M sets of known feature spectra, and A process of obtaining M sets of class confidences associated with the M machine learning models by obtaining class confidences that depend on class similarities;
(b3) a process of outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in the order of reliability of each class represented by the M sets of reliability by class; ,
An information processing device, including An information processing device that executes discrimination processing for discriminating classes of data to be discriminated using M machine learning models of a vector neural network type having a plurality of vector neuron layers, where M is an integer of 2 or more. hand,
a memory that stores the M machine learning models;
a processor that performs operations using the M machine learning models;
with
The processor
(a) A process of reading from the memory M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are Nm for each machine learning model A process including a known feature spectrum group obtained from the output of a specific layer of the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of the classes is input;
(b) using the M machine learning models and the M known feature spectrum groups, a process of performing class discrimination processing of the data to be discriminated;
and run
The processing (b) is
(b1) a process of calculating M feature spectra from the output of the specific layer and determining M discrimination classes according to the input of the data to be discriminated to the M machine learning models;
(b2) calculating a discriminant class similarity for each of the M discriminant classes as the similarity between the M feature spectra and the M sets of known feature spectra, and calculating the discriminant class similarity for each of the M discriminant classes; obtaining a model confidence for each of the M machine learning models by determining a dependent model confidence;
(b3) a process of outputting a discriminant class obtained from the machine learning model with the highest model reliability among the M discriminant classes as a class discrimination result for the data to be discriminated;
An information processing device, including A computer program that causes a processor to execute class discrimination processing for discriminating classes of data to be discriminated using M machine learning models of a vector neural network type having a plurality of vector neuron layers, where M is an integer of 1 or more. and
The number of classes distinguishable by the m-th machine learning model out of the M machine learning models is Nm, which is an integer of 2 or more, and the total number of classes distinguishable by the M machine learning models is ΣNm. When
Said computer program comprises:
(a) A process of reading from memory M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are Nm classes for each machine learning model A process including a known feature spectrum group obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of is input;
(b) using the M machine learning models and the M known feature spectrum groups, a process of performing class discrimination processing of the data to be discriminated;
a computer program that causes the processor to execute
The processing (b) is
(b1) a process of calculating M feature spectra from the output of the specific layer according to the input of the data to be discriminated to the M machine learning models;
(b2) Calculate the similarity by class for each of the Nm classes in each machine learning model as the similarity between the M feature spectra and the M sets of known feature spectra, and A process of obtaining M sets of class confidences associated with the M machine learning models by obtaining class confidences that depend on class similarities;
(b3) a process of outputting a discrimination result list in which a plurality of classes, which are at least part of the ΣNm classes, are arranged in the order of reliability of each class represented by the M sets of reliability by class; ,
computer programs, including A computer program that causes a processor to execute class discrimination processing for discriminating classes of data to be discriminated using M machine learning models of a vector neural network type having a plurality of vector neuron layers, where M is an integer of 2 or more. and
(a) A process of reading from memory M sets of known feature spectrum groups associated with the M machine learning models, wherein the M sets of known feature spectrum groups are Nm classes for each machine learning model A process including a known feature spectrum group obtained from the output of a specific layer among the plurality of vector neuron layers of each machine learning model when a plurality of teacher data for each of is input;
(b) using the M machine learning models and the M known feature spectrum groups, a process of performing class discrimination processing of the data to be discriminated;
a computer program that causes the processor to execute
The processing (b) is
(b1) a process of calculating M feature spectra from the output of the specific layer and determining M discrimination classes according to the input of the data to be discriminated to the M machine learning models;
(b2) calculating a discriminant class similarity for each of the M discriminant classes as the similarity between the M feature spectra and the M sets of known feature spectra, and calculating the discriminant class similarity for each of the M discriminant classes; obtaining a model confidence for each of the M machine learning models by determining a dependent model confidence;
(b3) a process of outputting a discriminant class obtained from the machine learning model with the highest model reliability among the M discriminant classes as a class discrimination result for the data to be discriminated;
computer programs, including

Images