CN104598720B - Cmp time setting method based on cluster and multi-task learning - Google Patents

Abstract

The chemical mechanical grinding time setting method based on clustering and multi-task learning belongs to the field of automatic control, information technology and advanced manufacturing. The method takes the process index and product state as input, and the key factor affecting the process index - grinding time is used as the output to establish a reverse model to optimize the setting of grinding time. In the process of constructing the above reverse model, in view of the problem of many types of production varieties and little data of a single variety, similar varieties are clustered according to product characteristics, and a multi-task learning method based on common parameter extraction is used for modeling in each category; The calculated model parameters are divided into the common part of this kind of variety and the private part of single variety.

Description

A chemical mechanical grinding time setting method based on clustering and multi-task learning

technical field

The invention belongs to the fields of automatic control, information technology and advanced manufacturing. In order to solve the problems of high rework rate and low production efficiency of the entire production line caused by manual setting of chemical mechanical grinding time in the multi-variety mixed production mode in modern microelectronics production and manufacturing, the present invention adopts an optimal setting method based on a reverse model , taking the process index and product status as input, and the key factor affecting the process index—grinding time as output—to establish a reverse model to optimize the setting of grinding time. Aiming at the problem that there are many product varieties in the production data and the data volume of a single variety is small, a chemical mechanical grinding time setting method based on clustering and multi-task learning is proposed, which can realize the optimal setting of chemical mechanical grinding time. Increases productivity in chemical mechanical polishing processes.

Background technique

Chemical mechanical polishing is a key process in the microelectronics manufacturing process, which affects the production efficiency of the entire production line. Due to the multi-variety and small-batch production characteristics of the modern microelectronics manufacturing industry, enterprises often adopt the mode of multi-variety mixed production. In this mixed production mode, the traditional Run-to-Run (RtR) optimization control method is difficult to achieve ideal results. The main idea of RtR is to use the production information of the previous batch or even several batches to guide the production of new batches. Generally, it is assumed that the products of the same variety are processed continuously on one equipment or several equipments are assumed to process a small number of varieties in a regular cycle. . However, due to the multi-variety mixed production, several equipment may continuously process any variety of products, and there are differences between different varieties and equipment, and the effect of the traditional RtR method is poor. Therefore, at present, the grinding time still depends on manual experience setting and advanced sheet testing. Once unqualified products appear, the entire batch will have to be reworked. It is difficult to achieve high production efficiency. It is urgent to optimize the processing time of chemical mechanical grinding. Certainly. The traditional optimization setting method of operating parameters needs to firstly model the process index, and then optimize the operating parameters on the basis of the index model. The process index model is a forward model with product status and operating parameters as input and process index as output. However, when there is less data, it is difficult to accurately establish the index model, resulting in poor operation optimization results. Drawing lessons from the idea of using historical batches to guide future production in RtR, and aiming at the characteristics of many types of chemical mechanical grinding data and less data in each type, the present invention uses process indicators and product status as input, and the key factor affecting process indicators-grinding time As an output, an inverse model is established to optimize the setting of grinding time, and based on this, a chemical-mechanical grinding time setting method based on clustering and multi-task learning is proposed.

Contents of the invention

In order to solve the problem of optimal setting of chemical mechanical grinding time in a multi-variety mixed production environment, the present invention takes the process index and product status as input, and the key factor affecting the process index-grinding time as output to establish a reverse model for optimizing the grinding time Based on this, a chemical mechanical grinding time setting method based on clustering and multi-task learning is proposed. In order to deal with the problem of many varieties of products and little data of each variety in the actual production data, a two-step modeling method based on clustering and multi-task learning is proposed, which integrates the clustering process and parameter learning in traditional clustering multi-task learning The process is considered separately. The variety clustering process assumes that the data in each variety conforms to a multidimensional Gaussian distribution, and uses maximum likelihood estimation to estimate the mean vector and variance matrix of the multidimensional Gaussian distribution. According to the estimated results, the Bhattacharyachian distance is used to represent the similarity of two multidimensional Gaussian distributions. The similarity matrix is obtained by calculating the similarity between the probability distributions of different varieties. Using the similarity matrix as input, the affine propagation algorithm was used to complete the clustering of varieties. At the same time, in order to ensure that the number of samples in each category in the results of clustering under small samples is large enough, the number of samples of each variety is embedded in the process of variety clustering as a priori knowledge.

The model parameters are computed in each category using the proposed multi-task learning algorithm based on common parameter extraction. To solve the problem of small number of samples in a single task, the proposed multi-task learning algorithm based on common parameter extraction decomposes model parameters into common and private parts and learns both parts simultaneously. The common part can make up for the model bias caused by insufficient sample data.

The chemical mechanical grinding time setting method based on clustering and multi-task learning is characterized in that the method is implemented on a computer in the following steps:

Step (1): Data curation

The optimal setting model established in this method uses 4 process indicators and product status to form a model input row vector x, including: abrasive material removal rate, incoming sheet thickness, random measurement of the thickness of the preceding sheet and the thickness of the sheet measured by lot, Take the chemical mechanical grinding time as the model output y; assume that the relationship between the model input and output satisfies the following formula:

y=xw+δ

The column vector w represents the model parameters to be determined, and δ is the noise.

Without loss of generality, suppose there are m product varieties; N _i sample inputs of the i-th product variety are recorded as matrix Its j-th row x _i(j) represents the input vector of the j-th sample of the i-th product variety, d is the number of model input variables, and d=4 in this method; N _i samples of the i-th product variety output denoted as a column vector its jth element Indicates the output of the j-th sample of the i-th product variety, that is, the chemical-mechanical grinding time; for subsequent representation, X is used to represent the input matrix [X ₁ , X ₂ ,..., X _m-1 , X _m ] The matrix formed by the vertical arrangement; Y is also used to represent the column vector formed by the vertical arrangement of the output vector [y ₁ , y ₂ ,..., y _m-1 , y _m ]; both X and Y have N _i rows, N represents the total number of samples; column vector Indicates the label of the variety to which each sample belongs, and the value range is {1, 2, ..., m-1, m};

Step (2): Calculate the similarity matrix of different varieties

Assuming that the data of each variety obeys different multidimensional Gaussian distributions, the probability distribution function of each variety is calculated using the maximum likelihood estimation method; for example, for the i-th variety, the mean vector of its multidimensional Gaussian distribution and and variance matrix Estimated value of :

where matrix The row vector z _i(j) represents the jth row of the matrix Z _i , including 4 input variables and 1 output variable;

The similarity between different varieties is compared by using the Barrett's distance, that is, the similarity of multidimensional Gaussian distribution of different varieties is compared:

Under the assumption of multidimensional Gaussian distribution, the Bhattacharyian distance has an analytical expression, assuming that the probability distribution of two multidimensional Gaussian distributions is G ₁ ~ N(μ ₁ , ∑ ₁ ), G ₂ ~ N(μ ₂ , ∑ ₂ ) The distance calculation method is:

in |A| represents the determinant of matrix A;

According to the above calculation method of difference degree, based on the mean vector of multivariate Gaussian distribution of each variety and variance matrix The estimated value of is calculated to obtain the similarity matrix; because the input required by the subsequent affine propagation clustering algorithm is the similarity, the difference is negative to obtain the similarity;

Step (3): Product feature clustering based on affine propagation

Affine propagation clustering is a clustering algorithm based on information accumulation. The cluster center is determined according to the accumulated information of different points, and the similarity matrix is mainly used to calculate two kinds of information, γ(i, k), a(i, k):

Start to set a(i,k)=0, then iteratively update γ(i,k) and a(i,k) according to the above formula until convergence;

In the affine propagation clustering, the preference vector is used to represent the possibility of each task in the prior knowledge becoming the cluster center; in the iteration, the preference vector is used to replace the diagonal elements in the similarity matrix to affect the selection of the cluster center; because Products with a large number of samples will get more accurate models later, which are more suitable as cluster centers. In order to use the prior of the number of samples in the process of clustering, the following method is used to set the preference vector in the affine propagation algorithm ;

Assume that the set value of the preference vector is p=[p ₁ , p ₂ ,..., p _m-1 , p _m ]:

Among them, N _i represents the number of samples for each task, and L indicates that varieties with a sample size larger than L are more likely to become cluster centers; typical values are set to a=0.005, b=2000, L=50;

Step (4): Multi-task learning based on common parameter extraction

After clustering, L categories are obtained, and a multi-task learning algorithm based on common parameter extraction is used for the varieties in each category. The main idea of the algorithm is to divide the model parameters of each variety into two parts: shared parameters and private parameters. ; Shared parameters are the same parts of the data model for all varieties in each category, with a column vector Represents; while the private parameters are different parts of the data model for each variety in each category, with a column vector If there are γ varieties in a category, then γ column vectors v ⁱ can form a matrix V of γ columns; the multi-task learning algorithm based on the extraction of common parameters can learn the two parts according to the data in each category Parameters to get the parameters of the final model, for example, for the i-th task, the final model parameters are:

w ⁱ =u+v ⁱ

Before learning the model, it is necessary to normalize the data; then set the parameters of the model, including λ ₁ , λ ₂ , λ ₃ , and randomly initialize the common parameter vector u and the private parameter matrix V;

The iterative process is as follows, where X represents the matrix formed by the vertical arrangement of input matrices [X ₁ , X ₂ , ..., XX _r-1 , X _r ] of γ varieties in one class; Y also represents γ in one class The output vectors [y ₁ , y ₂ , ..., y _r-1 , y _r ] of varieties are arranged vertically to form a column vector for the k-th iteration calculation:

In the above formula:

Update u k , V _k according to p _k , u _k-1 and Q _k , V _{k -1}

Where α∈[0,1], α ⁰ =0 can be set; t ₀ =1,

In the iterative process, the step size l _k is determined by the following method:

l _k ＝2 ^jk l _k-1 , where j _k is the smallest non-negative positive integer that makes the following formula true:

Using the above methods for the categories obtained by the L clusters respectively, L model libraries can be obtained, which include all models of m varieties.

Description of drawings

Figure 1: Flow chart of the method for setting chemical mechanical grinding time based on clustering and multi-task learning

Figure 2: The hardware and software composition diagram of the chemical mechanical grinding time setting method based on clustering and multi-task learning.

Detailed ways

The present invention proposes a chemical mechanical grinding time setting method based on clustering and multi-task learning. Its main advantage is that it can be used in multi-variety mixed production, and can improve production efficiency compared with manual setting. In the actual application process, if a new production batch arrives, the grinding time can be calculated according to its variety and processing layer type and other batch information. The learning algorithm based on clustering and multi-task of the present invention depends on related hardware equipment, including: data acquisition system, algorithm server and user client, and is realized by intelligent optimization software. The flow chart of the method proposed by the present invention is shown in FIG. 2 .

Step (1): Data Acquisition

The collected production information includes lot product variety, processing level, material removal rate, incoming sheet thickness, leading sheet output thickness, random inspection output sheet thickness, incoming sheet thickness range, and output sheet thickness range, and the initialization information is stored in the production process database middle;

Step (2): Data Wrangling

On the model training server, first remove the abnormal historical records, including the obvious abnormal records entered by human beings in the historical data and the abnormal records missing individual data items. Then organize the data according to the requirements of model learning, in which the combination of product variety and variety is positioned as a generalized variety, and different levels of the same variety are also considered as different varieties. The dataset composed of where line j Consists of 4-dimensional data, including material removal rate, thickness of the incoming piece, thickness of the leading piece and the thickness of the piece measured by lot; jth element is the chemical mechanical grinding time; the variety number vector I.

Step (3): Model training

On the model training server, model learning is performed according to the sorted data set {X _i , y _i }, i={1, 2, . . . , m−1, m} and the variety number vector I. It includes two steps: product feature clustering and multi-task learning of common parameter extraction. Complete the entire model training process according to the calculation process in the invention description. The flow chart of the method proposed by the present invention is shown in FIG. 1 .

Step (4): Parameter Optimization

In the multi-task learning based on product feature clustering and common parameter extraction, the main model parameters include sample number threshold L, cluster number parameter a and multi-task learning parameters λ ₁ , λ ₂ , λ ₃ . The sample threshold L can be determined by counting the number of samples of each variety in the entire data set. The remaining three parameters are optimized by cross-validation method. The main process is to split the entire data set and use it as training set and test set respectively. For example, if 3 cross-validation is performed, the data of each variety is divided into 3 parts, and then 1 part is used as test data and 2 parts are used as training data. Use the training data to train the model, and then use the test data to test the optimization setting effect of the model. By setting different a, λ ₁ , λ ₂ , λ ₃ to select the parameter with the best effect. Finally the best model is transferred to an on-site server.

Step (4): Online Application

On the on-site server, according to the current processing batch data transmitted from the actual production data, select the data corresponding to the variety and level to determine the model parameters. The data x in the current batch includes the material removal rate, the thickness of the incoming piece, the thickness of the missing piece of the leading piece, and the thickness of the piece measured by lot. Here, the standard sheet thickness of this variety and level is used to replace the sheet thickness of the preceding sheet and the thickness of the sheet measured by lot sampling. The optimal setting value of the chemical mechanical polishing time can be obtained by multiplying the vector x with the model parameters.

Based on the above-mentioned proposed multi-task learning method based on product feature clustering and common parameter extraction, the present invention has done a large number of simulation experiments. Due to space limitations, only the simulation of the application of the invention to the optimal setting of chemical mechanical grinding time is given here. result. The input data consists of 4-dimensional data, including the material removal rate, the thickness of the incoming piece, the thickness of the piece out of the first piece and the thickness of the piece measured by lot. The data is taken from the industrial field data between 2011-1-1 and 2013-7-25, 441 varieties, a total of 11250 records, using 3 cross-validation to determine the model parameters.

The present invention is compared with typical multi-task learning algorithm CMTL-convex (Clustered multi-task learningA convex formulation) and nonlinear modeling methods Kernel Extreme Learning Machine (KELM) and Support Vector Machine (SVM). Among them, in order to prove the effectiveness of the variety clustering algorithm, KELM and SVM without the variety clustering method are compared with Cluster-KELM and Cluster-SVM after the variety clustering. The Gaussian kernel function used in KELM and SVM is expressed as:

exp(-γ*||uv|| ² )

Another parameter is the weight of the regular term in the model. The relevant parameter values are shown in the table below.

Table 1 Algorithm-related parameter settings

Data with different ratios (0.2, 0.4, 0.6) were randomly selected as test data for algorithm performance comparison. The performance index selection regularized mean square error (normalized Mean Squared Error, nMSE), average mean square error (averaged Means Squared Error, aMSE) and the proportion of 10% error (error> 10%) comparison results are shown in Table 2

Table 2 Performance comparison results of LS-IELM and OS-ELM, Fixed-LSSVM algorithm

It can be seen from the table that TwCMTL proposed by the present invention has better test accuracy and better generalization ability than CMTL-convex, SVM, KELM, Cluster-SVM, and Cluster-KELM.

Claims (1)

1. based on cluster and multi-task learning cmp time setting method, it is characterised in that methods described be Realized according to the following steps successively on computer：

Step (1)：Data preparation

The optimal setting model established in this method with 4 technic indexs and Product Status composition model line of input vector x, its Include：Grinding-material removal rate, carry out piece thickness, take a sample test leading piece slice thickness and lot and take a sample test slice thickness, with chemical machine Tool milling time is that model exports y；Relation between hypothesized model input and output meets following formula：

Y=xw+ δ

Wherein column vector w represents model parameter to be determined, and δ is noise；

Assuming that there is m product variety；The N of i-th of product variety_iIndividual sample input is designated as matrixIts jth row x_i(j) Represent the input vector of j-th of sample of i-th of product variety, d is mode input variable number, d=4；I-th of product variety N_iIndividual sample output is designated as column vectorIts j-th of elementRepresent the defeated of j-th of sample of i-th of product variety Go out, i.e. the cmp time；Represent for ease of follow-up, represented with X by input matrix [X₁, X₂..., X_m-1, X_m] longitudinal direction row The formed matrix of row；Equally represented with Y by output vector [y₁, y₂..., y_m-1, y_m] longitudinal arrangement form column vector；X and Y hasOK, N represents all total sample numbers；Column vectorRepresent the mark of the affiliated kind of each sample Number, span is { 1,2 ..., m-1, m }；

Step (2)：Calculate the similarity matrix of different cultivars

The probability-distribution function of each kind is calculated using Maximum Likelihood Estimation, each product under Multi-dimensional Gaussian distribution hypothesis The corresponding mean vector of kindAnd variance matrixEstimate be：

<mrow> <msub> <mover> <mi>&amp;mu;</mi> <mo>^</mo> </mover> <mi>i</mi> </msub> <mo>=</mo> <mfrac> <mn>1</mn> <msub> <mi>N</mi> <mi>i</mi> </msub> </mfrac> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>j</mi> <mo>=</mo> <mn>1</mn> </mrow> <msub> <mi>N</mi> <mi>i</mi> </msub> </munderover> <msub> <mi>z</mi> <mrow> <mi>i</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>)</mo> </mrow> </mrow> </msub> <mo>,</mo> <mi>i</mi> <mo>=</mo> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mn>2</mn> <mo>,</mo> <mo>...</mo> <mo>,</mo> <mi>m</mi> <mo>-</mo> <mn>1</mn> <mo>,</mo> <mi>m</mi> <mo>}</mo> </mrow>

<mrow> <msub> <mover> <mo>&amp;Sigma;</mo> <mo>^</mo> </mover> <mi>i</mi> </msub> <mo>=</mo> <mfrac> <mn>1</mn> <mrow> <msub> <mi>N</mi> <mi>i</mi> </msub> <mo>-</mo> <mn>1</mn> </mrow> </mfrac> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>j</mi> <mo>=</mo> <mn>1</mn> </mrow> <msub> <mi>N</mi> <mi>i</mi> </msub> </munderover> <mrow> <mo>(</mo> <msub> <mi>z</mi> <mrow> <mi>i</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>)</mo> </mrow> </mrow> </msub> <mo>-</mo> <msub> <mover> <mi>&amp;mu;</mi> <mo>^</mo> </mover> <mi>i</mi> </msub> <mo>)</mo> </mrow> <msup> <mrow> <mo>(</mo> <msub> <mi>z</mi> <mrow> <mi>i</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>)</mo> </mrow> </mrow> </msub> <mo>-</mo> <msub> <mover> <mi>&amp;mu;</mi> <mo>^</mo> </mover> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mo>&amp;prime;</mo> </msup> <mo>,</mo> <mi>i</mi> <mo>=</mo> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mn>2</mn> <mo>,</mo> <mo>...</mo> <mo>,</mo> <mi>m</mi> <mo>-</mo> <mn>1</mn> <mo>,</mo> <mi>m</mi> <mo>}</mo> </mrow>

Wherein matrixRow vector z_i(j)Representing matrix Z_iJth row, comprising 4 input variables and 1 output variable；

Diversity factor between different cultivars is compared using Pasteur's distance, that is, compares the diversity factor of different cultivars Multi-dimensional Gaussian distribution； The definition of Pasteur's distance is：

<mrow> <msub> <mi>d</mi> <mi>B</mi> </msub> <mrow> <mo>(</mo> <mi>p</mi> <mo>(</mo> <mi>x</mi> <mo>)</mo> <mo>,</mo> <mi>q</mi> <mo>(</mo> <mi>x</mi> <mo>)</mo> <mo>)</mo> </mrow> <mo>=</mo> <mo>-</mo> <mi>l</mi> <mi>n</mi> <mrow> <mo>(</mo> <mo>&amp;Integral;</mo> <msqrt> <mrow> <mi>p</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> <mi>q</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> </mrow> </msqrt> <mi>d</mi> <mi>x</mi> <mo>)</mo> </mrow> </mrow>

Under the hypothesis of Multi-dimensional Gaussian distribution, Pasteur's distance has analytical expression：Two Multi-dimensional Gaussian distribution G₁~N (μ₁, ∑₁) And G₂~N (μ₂, ∑₂) Pasteur apart from calculating formula：

<mrow> <msub> <mi>d</mi> <mi>B</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>G</mi> <mn>1</mn> </msub> <mo>,</mo> <msub> <mi>G</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mn>1</mn> <mn>8</mn> </mfrac> <msup> <mrow> <mo>(</mo> <msub> <mi>&amp;mu;</mi> <mn>1</mn> </msub> <mo>-</mo> <msub> <mi>&amp;mu;</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mi>T</mi> </msup> <msup> <mi>&amp;Gamma;</mi> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </msup> <mrow> <mo>(</mo> <msub> <mi>&amp;mu;</mi> <mn>1</mn> </msub> <mo>-</mo> <msub> <mi>&amp;mu;</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>+</mo> <mfrac> <mn>1</mn> <mn>2</mn> </mfrac> <mi>ln</mi> <mrow> <mo>(</mo> <mo>|</mo> <msub> <mi>&amp;Sigma;</mi> <mn>1</mn> </msub> <msup> <mo>|</mo> <mrow> <mo>-</mo> <mfrac> <mn>1</mn> <mn>2</mn> </mfrac> </mrow> </msup> <mo>|</mo> <msub> <mi>&amp;Sigma;</mi> <mn>2</mn> </msub> <msup> <mo>|</mo> <mrow> <mo>-</mo> <mfrac> <mn>1</mn> <mn>2</mn> </mfrac> </mrow> </msup> <mo>|</mo> <mi>&amp;Gamma;</mi> <mo>|</mo> <mo>)</mo> </mrow> </mrow>

Wherein| A | representing matrix A determinant；

Because subsequent affine propagation clustering method required input is similarity, diversity factor is taken and negative obtains similarity；It is based on The polynary Gaussian Profile mean vector of each kindAnd variance matrixEstimate, similarity matrix is calculated；

Step (3)：Product feature cluster based on affine propagation

Affine propagation clustering is a kind of clustering method based on information accumulation, is determined according to the cumulative information amount of difference in cluster The heart, two kinds of information content are calculated using similarity matrix；To point i and point k, involved two kinds of information content r (i, k) and a (i, k) For：

<mrow> <mi>r</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>&amp;LeftArrow;</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>-</mo> <munder> <mrow> <mi>m</mi> <mi>a</mi> <mi>x</mi> </mrow> <mrow> <mi>k</mi> <mo>*</mo> <mi>s</mi> <mo>.</mo> <mi>t</mi> <mo>.</mo> <mi>k</mi> <mo>*</mo> <mo>&amp;NotEqual;</mo> <mi>k</mi> </mrow> </munder> <mo>{</mo> <mi>a</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>k</mi> <mo>*</mo> <mo>)</mo> </mrow> <mo>+</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>k</mi> <mo>*</mo> <mo>)</mo> </mrow> <mo>}</mo> </mrow>

<mrow> <mi>a</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>&amp;LeftArrow;</mo> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>*</mo> <mo>&amp;NotEqual;</mo> <mi>k</mi> </mrow> </munder> <mi>m</mi> <mi>a</mi> <mi>x</mi> <mo>{</mo> <mn>0</mn> <mo>,</mo> <mi>r</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>*</mo> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>}</mo> <mo>,</mo> <mrow> <mo>(</mo> <mi>i</mi> <mo>=</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow>

Iteration starts setting up a (i, k)=0, then according to the renewal r (i, k) of above formula iteration and a (i, k) until convergence；

Represent that each task is as the possibility of cluster centre in priori with preference vector in affine propagation clustering；In iteration It is middle to replace the diagonal entry in similarity matrix with preference vector and then influence the selection of cluster centre；Because number of samples compared with Extended meeting obtains more accurate model after more product varietys, is more suitable for cluster centre, in order to by this elder generation of number of samples During testing for clustering, the preference vector in affine transmission method is set with the following method；

If the setting value of preference vector is p=[p₁, p₂..., p_m-1, p_m]：

<mrow> <msub> <mi>p</mi> <mi>i</mi> </msub> <mo>=</mo> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <mo>(</mo> <msub> <mi>N</mi> <mi>i</mi> </msub> <mo>-</mo> <mi>M</mi> <mi>a</mi> <mi>x</mi> <mo>(</mo> <msub> <mi>N</mi> <mi>i</mi> </msub> <mo>)</mo> <mo>)</mo> <mo>&amp;times;</mo> <mi>a</mi> </mrow> </mtd> <mtd> <mrow> <msub> <mi>N</mi> <mi>i</mi> </msub> <mo>&gt;</mo> <msub> <mi>L</mi> <mn>1</mn> </msub> </mrow> </mtd> <mtd> <mrow> <mi>i</mi> <mo>=</mo> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mn>2</mn> <mo>,</mo> <mo>...</mo> <mo>,</mo> <mi>m</mi> <mo>-</mo> <mn>1</mn> <mo>,</mo> <mi>m</mi> <mo>}</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mo>-</mo> <msub> <mi>L</mi> <mn>1</mn> </msub> <mo>&amp;times;</mo> <mi>a</mi> </mrow> </mtd> <mtd> <mrow> <msub> <mi>N</mi> <mi>i</mi> </msub> <mo>&amp;le;</mo> <msub> <mi>L</mi> <mn>1</mn> </msub> </mrow> </mtd> <mtd> <mrow> <mi>i</mi> <mo>=</mo> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mn>2</mn> <mo>,</mo> <mo>...</mo> <mo>,</mo> <mi>m</mi> <mo>-</mo> <mn>1</mn> <mo>,</mo> <mi>m</mi> <mo>}</mo> </mrow> </mtd> </mtr> </mtable> </mfenced> </mrow>

Wherein L₁Expression wishes that sample size is more than L₁Kind be more likely to turn into cluster centre；Set a=0.005, b= 2000, L=50；

Step (4)：Multi-task learning based on shared parameter extraction

The L classification obtained after cluster, the multi-task learning based on shared parameter extraction is used to the kind in each classification Method, its main thought are that the model parameter of each kind is divided into two parts：Shared parameter and privately owned parameter；Shared parameter is Identical part in the data model of all kinds, uses column vector in each classificationRepresent；And privately owned parameter is each Part different in the data model of each kind, uses column vector in classificationRepresent；If there are r in a classification Kind, then r column vector v⁽ⁱ⁾It may make up the matrix V of a r row；Multi-task learning method based on shared parameter extraction can root Learn this two-part parameter according to the data in each classification so as to obtain the parameter of final mask, to i-th of task, finally Model parameter be：

wⁱ=u+v⁽ⁱ⁾

, it is necessary to which first data are normalized before model learning is carried out；Then the parameter of setting model, including λ₁, λ₂, λ₃, random initializtion shares parameter vector u and privately owned parameter matrix V；

Iterative process is as follows, the input matrix [X of r kind during wherein X represents a kind of₁, X₂..., X_r-1, X_r] longitudinal arrangement institute The matrix of composition；Output vector [the y of r kind in one kind is equally represented with Y₁, y₂..., y_r-1, y_r] longitudinal arrangement composition Column vector

To kth time iterative calculation：

<mrow> <msub> <mi>p</mi> <mi>k</mi> </msub> <mo>=</mo> <msub> <mi>u</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>-</mo> <mfrac> <mn>1</mn> <msub> <mi>l</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> </mfrac> <msub> <mo>&amp;dtri;</mo> <mi>u</mi> </msub> <mi>f</mi> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>,</mo> <msub> <mi>V</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>)</mo> </mrow> </mrow>

<mrow> <msubsup> <mi>q</mi> <mi>k</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <mi>m</mi> <mi>a</mi> <mi>x</mi> <mrow> <mo>(</mo> <mn>0</mn> <mo>,</mo> <mn>1</mn> <mo>-</mo> <mfrac> <msub> <mi>&amp;lambda;</mi> <mn>3</mn> </msub> <mrow> <msub> <mi>l</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>|</mo> <mo>|</mo> <msubsup> <mi>s</mi> <mi>k</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msubsup> <mo>|</mo> <msub> <mo>|</mo> <mn>2</mn> </msub> </mrow> </mfrac> <mo>)</mo> </mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msubsup> </mrow>

In above formula：

<mrow> <msub> <mi>S</mi> <mi>k</mi> </msub> <mo>=</mo> <msub> <mi>V</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>-</mo> <mfrac> <mn>1</mn> <msub> <mi>l</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> </mfrac> <msub> <mo>&amp;dtri;</mo> <mi>V</mi> </msub> <mi>f</mi> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>,</mo> <msub> <mi>V</mi> <mrow> <mi>t</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>)</mo> </mrow> </mrow>

<mrow> <msub> <mo>&amp;dtri;</mo> <mi>u</mi> </msub> <mi>f</mi> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <msup> <mi>X</mi> <mi>T</mi> </msup> <mfrac> <mrow> <mi>Y</mi> <mo>-</mo> <msub> <mi>X</mi> <msub> <mi>u</mi> <mi>k</mi> </msub> </msub> </mrow> <mrow> <mo>|</mo> <mo>|</mo> <mi>Y</mi> <mo>-</mo> <msub> <mi>X</mi> <msub> <mi>u</mi> <mi>k</mi> </msub> </msub> <mo>|</mo> <msub> <mo>|</mo> <mn>2</mn> </msub> </mrow> </mfrac> <mo>-</mo> <mn>2</mn> <msub> <mi>&amp;lambda;</mi> <mn>1</mn> </msub> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>r</mi> </munderover> <msubsup> <mi>X</mi> <mi>i</mi> <mi>T</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>X</mi> <mi>i</mi> </msub> <mo>(</mo> <mrow> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>+</mo> <msubsup> <mi>v</mi> <mi>k</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msubsup> </mrow> <mo>)</mo> <mo>)</mo> </mrow> <mo>+</mo> <mn>2</mn> <msub> <mi>&amp;lambda;</mi> <mn>2</mn> </msub> <mo>&amp;CenterDot;</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> </mrow>

<mrow> <msub> <mo>&amp;dtri;</mo> <mi>V</mi> </msub> <mi>f</mi> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mo>-</mo> <mn>2</mn> <msub> <mi>&amp;lambda;</mi> <mn>1</mn> </msub> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>r</mi> </munderover> <msubsup> <mi>X</mi> <mi>i</mi> <mi>T</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>X</mi> <mi>i</mi> </msub> <mo>(</mo> <mrow> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>+</mo> <msubsup> <mi>v</mi> <mi>k</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msubsup> </mrow> <mo>)</mo> <mo>)</mo> </mrow> </mrow>

According to p_k, u_k-1And Q_k, V_k-1Update u_k, V_k

u_k=p_k+α_k(p_k-p_k-1)

V_k=Q_k+α_k(Q_k-Q_k-1)

Wherein α ∈ [0,1], can make α₀=0；t₀=1,

In an iterative process, step-length l_kDetermine with the following method：

Wherein j_kTo cause the minimum non-negative positive integer of following formula establishment：

<mrow> <mi>f</mi> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>)</mo> </mrow> <mo>&amp;le;</mo> <msub> <mi>F</mi> <mrow> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>l</mi> <mi>k</mi> </msub> </mrow> </msub> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>,</mo> <msub> <mi>V</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>)</mo> </mrow> </mrow>

<mrow> <mi>f</mi> <mrow> <mo>(</mo> <mi>u</mi> <mo>,</mo> <mi>V</mi> <mo>)</mo> </mrow> <mo>=</mo> <mo>|</mo> <mo>|</mo> <mi>Y</mi> <mo>-</mo> <mi>X</mi> <mi>u</mi> <mo>|</mo> <msub> <mo>|</mo> <mn>2</mn> </msub> <mo>+</mo> <msub> <mi>&amp;lambda;</mi> <mn>1</mn> </msub> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>r</mi> </munderover> <mo>|</mo> <mo>|</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>X</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>u</mi> <mo>+</mo> <msub> <mi>v</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mo>|</mo> <msubsup> <mo>|</mo> <mn>2</mn> <mn>2</mn> </msubsup> <mo>+</mo> <msub> <mi>&amp;lambda;</mi> <mn>2</mn> </msub> <mo>|</mo> <mo>|</mo> <mi>u</mi> <mo>|</mo> <msubsup> <mo>|</mo> <mn>2</mn> <mn>2</mn> </msubsup> </mrow>

<mfenced open = "" close = ""> <mtable> <mtr> <mtd> <mrow> <msub> <mi>F</mi> <mrow> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>l</mi> <mi>k</mi> </msub> </mrow> </msub> <mrow> <mo>(</mo> <mi>u</mi> <mo>,</mo> <mi>V</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>f</mi> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>)</mo> </mrow> <mo>+</mo> <mo>&lt;</mo> <mi>u</mi> <mo>-</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mo>&amp;dtri;</mo> <mi>u</mi> </msub> <mi>f</mi> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>)</mo> </mrow> <mo>&gt;</mo> <mo>+</mo> <mfrac> <msub> <mi>l</mi> <mi>k</mi> </msub> <mn>2</mn> </mfrac> <mo>|</mo> <mo>|</mo> <mi>u</mi> <mo>-</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>|</mo> <msubsup> <mo>|</mo> <mn>2</mn> <mn>2</mn> </msubsup> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mo>+</mo> <mo>&lt;</mo> <mi>V</mi> <mo>-</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mo>&amp;dtri;</mo> <mi>V</mi> </msub> <mi>f</mi> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>,</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>)</mo> </mrow> <mo>&gt;</mo> <mo>+</mo> <mfrac> <msub> <mi>l</mi> <mi>k</mi> </msub> <mn>2</mn> </mfrac> <mo>|</mo> <mo>|</mo> <mi>V</mi> <mo>-</mo> <msub> <mi>V</mi> <mi>k</mi> </msub> <mo>|</mo> <msubsup> <mo>|</mo> <mi>F</mi> <mn>2</mn> </msubsup> </mrow> </mtd> </mtr> </mtable> </mfenced>

L model library i.e. can obtain using the above method respectively to the classification of L cluster gained, it comprises all m kinds Model.