CN109871702B - Federal model training method, system, apparatus, and computer-readable storage medium - Google Patents
Federal model training method, system, apparatus, and computer-readable storage medium Download PDFInfo
- Publication number
- CN109871702B CN109871702B CN201910121269.8A CN201910121269A CN109871702B CN 109871702 B CN109871702 B CN 109871702B CN 201910121269 A CN201910121269 A CN 201910121269A CN 109871702 B CN109871702 B CN 109871702B
- Authority
- CN
- China
- Prior art keywords
- model
- prediction
- global
- parameters
- client terminals
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 148
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000002776 aggregation Effects 0.000 claims abstract description 21
- 238000004220 aggregation Methods 0.000 claims abstract description 21
- 238000012360 testing method Methods 0.000 claims description 133
- 238000004364 calculation method Methods 0.000 claims description 17
- 230000004931 aggregating effect Effects 0.000 claims description 7
- 238000001514 detection method Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 16
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 5
- 238000012935 Averaging Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006116 polymerization reaction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a federal model training method, a federal model training system, federal model training equipment and a computer-readable storage medium, wherein the federal model training method comprises the following steps: after the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, acquiring weight coefficients corresponding to each model parameter according to a preset acquisition rule; according to the model parameters and the weight coefficient corresponding to each model parameter, a second global model is obtained through aggregation; detecting whether the second global model converges; if the second global model is detected to be in a convergence state, the second global model is determined to be a final result of federal model training, and the second global model with the encrypted parameters is issued to a plurality of client terminals. The method and the device realize that the collaboration terminal updates and obtains the new global model according to the model parameters and the weight coefficients of each client terminal, and promote the prediction effect of the federal model.
Description
Technical Field
The present invention relates to the field of machine learning technologies, and in particular, to a federal model training method, system, device, and computer readable storage medium.
Background
The federal model is a machine learning model built by utilizing technical algorithm encryption, a plurality of federal clients in the federal learning system do not need to give own data when the model is trained, but train a local model according to a global model encrypted by parameters issued by a collaboration terminal and a data set local to the client, return the parameters of the local model for the collaboration terminal to aggregate and update the global model, and the updated global model is issued to the client again, and the loop is repeated until convergence. The federal learning protects the data privacy of the client by means of parameter exchange under an encryption mechanism, the local model of the client and the local model of the client cannot be transmitted, the local data cannot be anti-guessed, and the federal model guarantees the data privacy while maintaining the data integrity to a higher degree.
At present, when the collaboration terminal updates the global model according to local model parameters returned by a plurality of clients, the model parameters of the clients are simply averaged, the averaged model parameters are used as new global model parameters to be issued to the clients for continuous iterative training, however, in actual training, the prediction performance of the trained local model is uneven due to different training data of each client, and the effect of the global model is not ideal due to the existing simple average aggregation method.
Disclosure of Invention
The invention mainly aims to provide a federation model training method, a federation model training system, federation model training equipment and a federation model training computer readable storage medium, and aims to solve the technical problem that the federation model effect is not ideal due to the fact that the model parameters of a plurality of federation clients are updated in a simple average aggregation mode by the existing collaboration end.
To achieve the above object, the present invention provides a federal model training method, including the steps of:
after the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, acquiring weight coefficients corresponding to each model parameter according to a preset acquisition rule; the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters;
According to the model parameters and the weight coefficient corresponding to each model parameter, a second global model is obtained through aggregation;
detecting whether the second global model converges;
And if the second global model is detected to be in a convergence state, determining the second global model as a final result of federal model training, and issuing a second global model with encrypted parameters to the plurality of client terminals.
Optionally, after the collaboration terminal receives the model parameters sent by the plurality of client terminals, the step of obtaining the weight coefficient corresponding to each model parameter according to a preset obtaining rule includes:
After receiving model parameters respectively sent by a plurality of client terminals, the cooperative terminal tests and obtains a prediction error rate of a prediction model corresponding to each model parameter according to a preset test sample set;
And respectively calculating to obtain a weight coefficient corresponding to each model parameter based on the prediction error rate of each prediction model and a preset calculation formula.
Optionally, after the collaboration terminal receives the model parameters sent by the plurality of client terminals, the step of testing and obtaining the prediction error rate of the prediction model corresponding to each model parameter according to the preset test sample set includes:
after receiving model parameters respectively sent by a plurality of client terminals, the cooperative terminal inputs a plurality of test samples in a preset test sample set into a prediction model corresponding to the model parameters for prediction, and a prediction value of the prediction model for each test sample is obtained;
obtaining the number of test samples with wrong prediction results in the test sample set according to a plurality of prediction values;
And determining the ratio of the number of the test samples with the wrong prediction results to the number of all the test samples in the test sample set as the prediction error rate of the pre-prediction model.
Optionally, the step of aggregating to obtain the second global model according to the plurality of model parameters and the weight coefficient corresponding to each model parameter includes:
Multiplying each model parameter by the corresponding weight coefficient to obtain a plurality of multiplied results;
and adding the multiplied results, determining the added result as a model parameter of a second global model, and obtaining the second global model.
Optionally, after the collaboration terminal receives the model parameters sent by the plurality of client terminals, the step of acquiring the weight coefficient corresponding to each model parameter according to a preset acquisition rule further includes:
sending the first global model encrypted by the parameters to a plurality of client terminals respectively;
receiving model parameters respectively sent by the plurality of client terminals;
After receiving the first global model issued by the collaboration terminal, the client terminal predicts a first training sample set according to the first global model to obtain a predicted value, samples the first training sample set according to the predicted value to obtain a second training sample set, trains the first global model based on the second training sample set, and obtains the model parameters after training.
Optionally, the step of detecting whether the second global model converges further includes:
and if the second global model is detected to be in an unconverged state, issuing the second global model with parameter encryption to a plurality of client terminals respectively, so that the client terminals continue to train iteratively according to the second global model issued by the cooperative terminal respectively to return model parameters to the cooperative terminal.
Optionally, after the collaboration terminal receives the model parameters sent by the plurality of client terminals, the step of acquiring the weight coefficient corresponding to each model parameter according to a preset acquisition rule further includes:
Receiving model parameters respectively sent by a plurality of client terminals;
After the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, the step of acquiring the weight coefficient corresponding to each model parameter according to a preset acquisition rule comprises the following steps:
Receiving weight coefficients corresponding to the model parameters, which are respectively sent by the plurality of client terminals; and the plurality of client terminals respectively test and obtain the prediction error rate of the prediction model corresponding to the model parameters according to a preset test sample set, and calculate and obtain the weight coefficient corresponding to the model parameters according to the prediction error rate and a preset calculation formula.
In addition, the invention also provides a federal model training system, the system comprises a cooperative terminal and a plurality of client terminals which are respectively in communication connection with the cooperative terminal, and the cooperative terminal comprises:
The acquisition module is used for acquiring a weight coefficient corresponding to each model parameter according to a preset acquisition rule after receiving the model parameters respectively sent by the plurality of client terminals; the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters;
The aggregation updating module is used for aggregating to obtain a second global model according to the plurality of model parameters and the weight coefficient corresponding to each model parameter;
The detection module is used for detecting whether the second global model converges or not;
And the determining module is used for determining the second global model as a final result of federal model training and issuing the second global model with parameter encryption to the plurality of client terminals when the detecting module detects that the second global model is in a convergence state.
Optionally, the acquiring module includes:
The test unit is used for testing and obtaining the prediction error rate of the prediction model corresponding to each model parameter according to a preset test sample set after receiving the model parameters respectively sent by the plurality of client terminals;
And the calculating unit is used for respectively calculating and obtaining the weight coefficient corresponding to each model parameter based on the prediction error rate of each prediction model and a preset calculating formula.
Optionally, the test unit includes:
The testing subunit is used for inputting a plurality of test samples in a preset test sample set into a prediction model corresponding to the model parameters to predict after receiving the model parameters respectively sent by a plurality of client terminals, so as to obtain a predicted value of the prediction model for each test sample;
the obtaining subunit is used for obtaining the number of the test samples with wrong prediction results in the test sample set according to a plurality of prediction values;
and the determining subunit is used for determining the ratio of the number of the test samples with the wrong prediction result to the number of all the test samples in the test sample set as the prediction error rate of the pre-prediction model.
Optionally, the aggregation update module includes:
The multiplication processing unit is used for multiplying each model parameter with the corresponding weight coefficient to obtain a plurality of multiplied results;
and the updating unit is used for adding the multiplied results and determining the added result as a model parameter of a second global model to obtain the second global model.
Optionally, the collaboration terminal further includes:
the first issuing module is used for sending the first global model encrypted by the parameters to a plurality of client terminals respectively;
The receiving module is used for receiving model parameters respectively sent by the plurality of client terminals;
after receiving the first global model issued by the first issuing module, the client terminal predicts a first training sample set according to the first global model to obtain a predicted value, samples the first training sample set according to the predicted value to obtain a second training sample set, trains the first global model based on the second training sample set, and obtains model parameters after training.
Optionally, the collaboration terminal further includes:
and the second issuing module is used for issuing the second global model with encrypted parameters to a plurality of client terminals respectively when the detection module detects that the second global model is in an unconverged state, so that the client terminals continue to iterate and train according to the second global model issued by the second issuing module respectively to return model parameters to the collaborative terminal.
Optionally, the acquiring module is further configured to receive weight coefficients corresponding to the model parameters sent by the plurality of client terminals respectively; and the plurality of client terminals respectively test and obtain the prediction error rate of the prediction model corresponding to the model parameters according to a preset test sample set, and calculate and obtain the weight coefficient corresponding to the model parameters according to the prediction error rate and a preset calculation formula.
In addition, to achieve the above object, the present invention also proposes a federal model training apparatus comprising a memory, a processor, and a federal model training program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the federal model training method as described above.
In addition, to achieve the above object, the present invention also proposes a computer readable storage medium having stored thereon a federal model training program which, when executed by a processor, implements the steps of the federal model training method as described above.
According to the method, after the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, the weight coefficient corresponding to each model parameter is obtained according to a preset obtaining rule; the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters; according to the model parameters and the weight coefficient corresponding to each model parameter, a second global model is obtained through aggregation; detecting whether the second global model converges; if the second global model is detected to be in a convergence state, determining the second global model as a final result of federal model training, and issuing a second global model with encrypted parameters to the plurality of client terminals; therefore, when the collaboration terminal aggregates the global model according to model parameters returned by clients of federation multiparties, instead of simply averaging a plurality of model parameters, a new global model is updated by combining weight coefficients of each model parameter, the weight coefficients are determined according to prediction accuracy of training models of each client terminal, prediction effects of the federation model are improved, and the problem that the existing collaboration terminal updates the global model by simply averaging model parameters of a plurality of federation clients to cause non-ideal federation model effects is avoided.
Drawings
FIG. 1 is a schematic diagram of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a first embodiment of the federal model training method according to the present invention;
FIG. 3 is a schematic flow chart of a second embodiment of the federal model training method according to the present invention;
FIG. 4 is a schematic flow chart of a third embodiment of the federal model training method according to the present invention;
FIG. 5 is a flow chart of a fourth embodiment of the federal model training method according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware running environment according to an embodiment of the present invention.
It should be noted that fig. 1 is a schematic structural diagram of a hardware running environment of the federal model training apparatus. The federal model training device in the embodiment of the invention can be a terminal device such as a PC, a portable computer and the like.
As shown in fig. 1, the federal model training apparatus may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the federal model training apparatus configuration shown in fig. 1 is not limiting of the federal model training apparatus, and may include more or fewer components than illustrated, or may combine certain components, or may have a different arrangement of components.
As shown in FIG. 1, an operating system, a network communication module, a user interface module, and federal model training program may be included in memory 1005, which is a type of computer storage medium. The operating system is a program for managing and controlling the hardware and software resources of the federal model training device, and supports the operation of the federal model training program and other software or programs.
In the federal model training apparatus shown in fig. 1, the user interface 1003 is mainly used for connecting to a client terminal or the like, and performing data communication with each terminal; the network interface 1004 is mainly used for connecting a background server and carrying out data communication with the background server; and the processor 1001 may be configured to invoke the federal model training program stored in the memory 1005 and perform the following operations:
after the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, acquiring weight coefficients corresponding to each model parameter according to a preset acquisition rule; the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters;
According to the model parameters and the weight coefficient corresponding to each model parameter, a second global model is obtained through aggregation;
detecting whether the second global model converges;
And if the second global model is detected to be in a convergence state, determining the second global model as a final result of federal model training, and issuing a second global model with encrypted parameters to the plurality of client terminals.
Further, after the collaboration terminal receives the model parameters respectively sent by the plurality of client terminals, the step of obtaining the weight coefficient corresponding to each model parameter according to a preset obtaining rule includes:
After receiving model parameters respectively sent by a plurality of client terminals, the cooperative terminal tests and obtains a prediction error rate of a prediction model corresponding to each model parameter according to a preset test sample set;
And respectively calculating to obtain a weight coefficient corresponding to each model parameter based on the prediction error rate of each prediction model and a preset calculation formula.
Further, after the collaboration terminal receives the model parameters sent by the plurality of client terminals, the step of testing and obtaining the prediction error rate of the prediction model corresponding to each model parameter according to the preset test sample set includes:
after receiving model parameters respectively sent by a plurality of client terminals, the cooperative terminal inputs a plurality of test samples in a preset test sample set into a prediction model corresponding to the model parameters for prediction, and a prediction value of the prediction model for each test sample is obtained;
obtaining the number of test samples with wrong prediction results in the test sample set according to a plurality of prediction values;
And determining the ratio of the number of the test samples with the wrong prediction results to the number of all the test samples in the test sample set as the prediction error rate of the pre-prediction model.
Further, the step of aggregating the plurality of model parameters and the weight coefficients corresponding to each model parameter to obtain the second global model includes:
Multiplying each model parameter by the corresponding weight coefficient to obtain a plurality of multiplied results;
and adding the multiplied results, determining the added result as a model parameter of a second global model, and obtaining the second global model.
Further, before the step of acquiring the weight coefficient corresponding to each model parameter according to the preset acquiring rule after the collaboration terminal receives the model parameters sent by the plurality of client terminals, the processor 1001 may be further configured to invoke the federal model training program stored in the memory 1005, and execute the following steps:
sending the first global model encrypted by the parameters to a plurality of client terminals respectively;
receiving model parameters respectively sent by the plurality of client terminals;
After receiving the first global model issued by the collaboration terminal, the client terminal predicts a first training sample set according to the first global model to obtain a predicted value, samples the first training sample set according to the predicted value to obtain a second training sample set, trains the first global model based on the second training sample set, and obtains the model parameters after training.
Further, after the step of detecting whether the second global model converges, the processor 1001 may be further configured to invoke a federal model training program stored in the memory 1005, and perform the following steps:
and if the second global model is detected to be in an unconverged state, issuing the second global model with parameter encryption to a plurality of client terminals respectively, so that the client terminals continue to train iteratively according to the second global model issued by the cooperative terminal respectively to return model parameters to the cooperative terminal.
Further, before the step of acquiring the weight coefficient corresponding to each model parameter according to the preset acquiring rule after the collaboration terminal receives the model parameters sent by the plurality of client terminals, the processor 1001 may be further configured to invoke the federal model training program stored in the memory 1005, and execute the following steps:
Receiving model parameters respectively sent by a plurality of client terminals;
After the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, the step of acquiring the weight coefficient corresponding to each model parameter according to a preset acquisition rule comprises the following steps:
Receiving weight coefficients corresponding to the model parameters, which are respectively sent by the plurality of client terminals; and the plurality of client terminals respectively test and obtain the prediction error rate of the prediction model corresponding to the model parameters according to a preset test sample set, and calculate and obtain the weight coefficient corresponding to the model parameters according to the prediction error rate and a preset calculation formula.
Based on the above structure, various embodiments of a federal model training method are presented.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the federal model training method according to the present invention.
Embodiments of the present invention provide embodiments of federal model training methods, it being noted that although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than that illustrated herein.
The federal model training method comprises the following steps:
Step S100, after receiving model parameters respectively sent by a plurality of client terminals, the cooperative terminal acquires weight coefficients corresponding to each model parameter according to a preset acquisition rule;
the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters.
The federal model is a machine learning model built by encryption by using a technical algorithm, the federal learning system is used for ensuring the confidentiality of data of federal multiparty clients in a training process, encryption training is carried out by means of a third-party cooperation terminal, a plurality of federal clients in the federal learning system do not need to give own data in model training, but train a local model according to a global model encrypted by parameters issued by a cooperation terminal and a data set local to the client, and return the parameters of the local model to the cooperation terminal for aggregation and updating of the global model, and the updated global model is issued to the client again and is cycled and reciprocated until convergence. The federal learning protects the data privacy of the client by means of parameter exchange under an encryption mechanism, the local model of the client and the client cannot be transmitted, the local data cannot be anti-guessed, and the data privacy can be ensured while the data integrity is maintained to a higher degree.
However, when the existing collaboration terminal updates the global model according to local model parameters returned by a plurality of clients, the model parameters of the clients are simply averaged, the averaged model parameters are used as new global model parameters to be issued to the clients for continuous iterative training, however, in actual training, the prediction performance of the trained local model is uneven for each client due to different training data, the simple average aggregation method in the prior art can cause the effect of the global model to be not ideal, if the prediction accuracy difference of the local model of each client in the federal learning system is large, the simple average aggregation of the plurality of parameters can reduce the model effect of the clients with high prediction accuracy of the local model, and the effect of the federal model finally obtained by the global model is not ideal.
In this embodiment, the collaboration terminal encrypts parameters of the first global model by using a preset encryption algorithm, and issues the first global model with encrypted parameters to a plurality of client terminals in the federal learning system, where the preset encryption algorithm is not specifically limited in this embodiment, and may be an asymmetric encryption algorithm or the like, and the first global model is a global model obtained after the federal model to be trained in this embodiment completes a plurality of iterative operations.
Further, after the client terminal receives the first global model encrypted by the parameters issued by the collaboration terminal, each client terminal trains the first global model according to the local training sample data to obtain the local model of each client terminal, and in this embodiment, the training sample types of the plurality of clients all obey independent same distribution, and the client terminal returns the model parameters of the obtained local model to the collaboration terminal.
After the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, a weight coefficient corresponding to each model parameter is obtained according to a preset obtaining rule, the weight coefficient is determined based on the prediction accuracy of a prediction model corresponding to the model parameters, as an implementation mode, the collaboration terminal stores a test sample set, the test samples in the test sample set and training samples of the plurality of client terminals have the same characteristic dimension, after the model parameters respectively sent by the plurality of client terminals are received, the collaboration terminal adopts the test sample set to test the prediction accuracy of the prediction model corresponding to each model parameter, specifically, a plurality of test samples in the test sample set are input into the prediction model returned by each client terminal, the prediction result of the prediction model on the plurality of test samples in the test sample set is obtained, the number of test samples with prediction errors is screened out, the total number of test samples in the test sample set is divided by the number of the test samples with the screened out prediction errors, and the prediction error rate of the prediction model corresponding to the model is obtained.
Further, the weight coefficient of each model parameter when the model parameter participates in global model polymerization is determined according to the prediction error rate of each model parameter, the prediction error rate of each model parameter is inversely related to the weight coefficient of the model parameter, that is, the smaller the prediction error rate of the prediction model corresponding to the model parameter is, the larger the weight coefficient of the model parameter is, when the model parameters sent by a plurality of client terminals are polymerized, the cooperative terminal in the embodiment increases the weight of the model parameter of the model with high prediction accuracy, and decreases the weight of the model parameter of the model with low prediction accuracy, so that the updated new global model ensures the increase of the model effect of each client terminal. As one embodiment, the weight coefficient may be calculated according to a calculation formulaAnd calculating, wherein epsilon i is the prediction error rate of the prediction model of the ith client terminal, alpha i is the weight coefficient corresponding to the model parameter of the prediction model of the ith client terminal, i is an integer greater than zero, and the cooperative terminal acquires the weight coefficient corresponding to each model parameter.
It should be noted that, in other embodiments, multiple client terminals in the federal learning system may all store the same test sample set, the test samples in the test sample set and the training samples of the multiple client terminals all have the same feature dimensions, the weight coefficient corresponding to the model parameter of each client terminal may be the predicted error rate of the model parameter of the client terminal tested by the client terminal according to the locally stored test sample set, so as to obtain the weight coefficient corresponding to the model parameter, and the client terminal sends the model parameter and sends the weight coefficient corresponding to the calculated model parameter to the cooperative terminal for the cooperative terminal to aggregate.
Step S200, according to a plurality of model parameters and weight coefficients corresponding to each model parameter, a second global model is obtained through aggregation;
in this embodiment, the prediction error rate of the prediction model corresponding to each model parameter is inversely related to the weight coefficient of the model parameter, that is, the smaller the prediction error rate is, the larger the weight coefficient of the model parameter is, the larger the prediction error rate is, the smaller the weight coefficient of the model parameter is, each model parameter is multiplied by the corresponding weight coefficient, and then the plurality of model parameters multiplied by the weight coefficient are added to obtain the parameter of the new global model, so as to obtain the new global model, that is, the second global model.
When the collaboration terminal is aggregated according to model parameters sent by a plurality of client terminals, the model parameters of the collaboration terminal are weighted for a model with high prediction accuracy, the model parameters of the model with low prediction accuracy are weighted for a model with low prediction accuracy, the new global model obtained through updating ensures the increase of the model effect of each client terminal, and the problem that the model effect of the federation model is not ideal due to the fact that the existing collaboration terminal updates the global model by adopting a simple average aggregation mode for the model parameters of a plurality of federation clients is avoided.
Step S300, detecting whether the second global model converges;
In this embodiment, as an implementation manner, the collaboration terminal obtains a loss value according to a loss function of the second global model, determines whether the second global model converges according to the loss value, specifically, the collaboration terminal stores a first loss value under the first global model, obtains the second loss value according to the loss function of the second global model, calculates a difference value between the first loss value and the second loss value, and determines whether the difference value is smaller than or equal to a preset threshold, if the difference value is smaller than or equal to the preset threshold, it is determined that the second global model is in a converging state, federal model training is completed, and when practical training is performed, the preset threshold can be set according to a user requirement, and the preset threshold is not particularly limited in this embodiment.
And step S400, if the second global model is detected to be in a convergence state, determining the second global model as a final result of federal model training, and issuing the second global model with encrypted parameters to the plurality of client terminals.
If the second global model is detected to be in a convergence state, the federal model training is completed, the second global model is determined to be a final result of the federal model training, the collaboration terminal issues the second global model with encrypted parameters to the plurality of client terminals, and the plurality of client terminals realize the effect growth of the local model on the premise of not giving own data, and improve the prediction accuracy while guaranteeing the data privacy.
According to the embodiment, after the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, the weight coefficient corresponding to each model parameter is obtained according to a preset obtaining rule; the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters; according to the model parameters and the weight coefficient corresponding to each model parameter, a second global model is obtained through aggregation; detecting whether the second global model converges; if the second global model is detected to be in a convergence state, determining the second global model as a final result of federal model training, and issuing a second global model with encrypted parameters to the plurality of client terminals; therefore, when the collaboration terminal aggregates the global model according to model parameters returned by clients of federation multiparties, instead of simply averaging a plurality of model parameters, a new global model is updated by combining weight coefficients of each model parameter, the weight coefficients are determined according to prediction accuracy of training models of each client terminal, prediction effects of the federation model are improved, and the problem that the existing collaboration terminal updates the global model by simply averaging model parameters of a plurality of federation clients to cause non-ideal federation model effects is avoided.
Further, a second embodiment of the federal model training method of the present invention is presented.
Referring to fig. 3, fig. 3 is a flowchart of a second embodiment of the federal model training method according to the present invention, based on the embodiment shown in fig. 2, in this embodiment, step S100, after the collaboration terminal receives model parameters sent by a plurality of client terminals respectively, the step of obtaining weight coefficients corresponding to each model parameter according to a preset obtaining rule includes:
Step S101, after receiving model parameters respectively sent by a plurality of client terminals, a cooperative terminal tests and obtains a prediction error rate of a prediction model corresponding to each model parameter according to a preset test sample set;
Step S102, calculating to obtain the weight coefficient corresponding to each model parameter based on the prediction error rate of each prediction model and a preset calculation formula.
Specifically, in this embodiment, the collaboration terminal stores a test sample set, where the test sample set includes a plurality of test samples, where the plurality of test samples and local training samples of the plurality of client terminals have the same feature dimensions, and the collaboration terminal inputs the plurality of test samples in the test sample set into a prediction model returned by each client terminal to obtain a prediction result of the prediction model on the plurality of test samples in the test sample set, screens a number of test samples with incorrect prediction results, divides the number of test samples in the test sample set by the number of test samples with incorrect prediction results, so as to obtain a prediction error rate of the current prediction model, and further, adopts the same method to obtain a prediction error rate of the prediction model corresponding to the model parameters sent by each client terminal.
In this embodiment, the preset calculation formula is: Wherein epsilon i is the prediction error rate of the prediction model of the ith client terminal, alpha i is the weight coefficient corresponding to the model parameter of the prediction model of the ith client terminal, and i is an integer greater than zero; after the cooperative terminal calculates the prediction error rate of the prediction model of each client terminal, each prediction error rate is substituted into the calculation formula to calculate, and the obtained result is the weight coefficient corresponding to each model parameter.
Further, based on the embodiment shown in fig. 2, in this embodiment, step S200, the step of aggregating to obtain the second global model according to the plurality of model parameters and the weight coefficient corresponding to each model parameter includes:
step S201, multiplying each model parameter by a corresponding weight coefficient to obtain a plurality of multiplied results;
Step S202, adding the multiplied results, determining the added result as a model parameter of a second global model, and obtaining the second global model.
In this embodiment, each model parameter is multiplied by its corresponding weight coefficient, and then a plurality of model parameters multiplied by the weight coefficient are added to obtain parameters of a new global model, so as to obtain a new global model, i.e., a second global model.
When the collaboration terminal is aggregated according to model parameters sent by a plurality of client terminals, the model parameters of the collaboration terminal are weighted for a model with high prediction accuracy, the model parameters of the model with low prediction accuracy are weighted for a model with low prediction accuracy, the new global model obtained through updating ensures the increase of the model effect of each client terminal, and the problem that the model effect of the federation model is not ideal due to the fact that the existing collaboration terminal updates the global model by adopting a simple average aggregation mode for the model parameters of a plurality of federation clients is avoided.
Further, a third embodiment of the federal model training method of the present invention is presented.
Referring to fig. 4, fig. 4 is a flowchart of a third embodiment of the federal model training method according to the present invention, based on the embodiment shown in fig. 2, in this embodiment, step S100, after the collaboration terminal receives model parameters sent by a plurality of client terminals respectively, further includes, before the step of acquiring weight coefficients corresponding to each model parameter according to a preset acquisition rule:
Step S110, sending the first global model encrypted by the parameters to a plurality of client terminals respectively;
step S120, receiving model parameters respectively sent by the plurality of client terminals;
After receiving the first global model issued by the collaboration terminal, the client terminal predicts a first training sample set according to the first global model to obtain a predicted value, samples the first training sample set according to the predicted value to obtain a second training sample set, trains the first global model based on the second training sample set, and obtains the model parameters after training.
In this embodiment, as an implementation manner, it is assumed that the federal model to be trained has completed the kth iterative operation and obtained a first global model k, where k is an integer greater than zero, and the federal model training method specifically includes the following steps;
the method comprises the steps that a collaboration terminal sends a first global model k with encrypted parameters to each federal client terminal;
Step b, the ith client terminal receives the model k, predicts the local training sample set X i by using the model k, and divides the training samples in X i into two sets according to the prediction result: a mispredicted sample data set (X 1,x2,...,xn) and a correctly predicted sample data set (y 1,y2,...,ym), where n represents the number of mispredicted samples in X i, m represents the number of correctly predicted samples in X i, n may be equal to m, and the embodiment is not specifically limited, so there are:
x i＝(x1,x2,...,xn)∪(y1,y2,...,ym) and Establishment;
The ith client terminal samples X i before training model k, specifically selects the sample data set (X 1,x2,...,xn) with incorrect prediction, extracts partial samples (Y 1,y2,...,yk) from the sample data set (Y 1,y2,...,ym) with correct prediction, forms a sampled training data set Y i, namely Y i＝(x1,x2,...,xn)∪(y1,y2,...,yk) with k < n, trains model k with training data set Y i, and obtains a new local prediction model after training
Step c, the ith client terminal sends the trained local prediction model to the cooperative terminal, the cooperative terminal inputs a plurality of test samples in a test sample set stored in the cooperative terminal to the prediction model returned by the ith client terminal to obtain a prediction result of the prediction model on the plurality of test samples in the test sample set, the number of test samples with prediction errors is screened out, and the total number of test samples in the test sample set is divided by the number of the screened test samples with prediction errors to obtain a prediction error rate of the prediction model of the ith client terminal
Step d, the cooperative terminal calculates the prediction error rate of the calculated i-th client terminal prediction modelSubstitution into a calculation formulaIn the method, a weight coefficient corresponding to the model parameter of the ith client terminal is calculated and obtained
Step e, the collaboration terminal aggregates and updates the first global model k to obtain a second global model k+1 according to the model parameters sent by each client terminal and the calculated weight coefficients corresponding to each model parameter, whereinQ is the total number of federal client terminals in this embodiment, the collaboration terminal detects whether the model k+1 converges, and if so, takes the model k+1 as the final training result of the federal model in this embodiment, and encrypts and transmits the model parameters of the model k+1 to each client terminal.
And if the collaboration terminal detects that the model k+1 is not converged, repeating the steps a-e until the federal model is converged.
When the client terminal trains according to the first global model encrypted by the parameter issued by the cooperative terminal, the client terminal predicts a local first training sample set of the client terminal according to the first global model to obtain a predicted value, samples the first training sample set according to the predicted value to obtain a second training sample set, trains the first global model based on the second training sample set, obtains model parameters after training and returns the model parameters to the cooperative terminal, so that the weight of sample data with prediction errors is improved when the sample data is iterated next time, the performance of the local training model is optimized, namely the quality of model parameters sent to the cooperative terminal by each client terminal is improved, and the prediction accuracy of the global model, namely the federal model in the embodiment is improved.
Further, a fourth embodiment of the federal model training method of the present invention is presented.
Referring to fig. 5, fig. 5 is a flowchart of a fourth embodiment of the federal model training method according to the present invention, based on the embodiment shown in fig. 2, in this embodiment, step S300, after the step of detecting whether the second global model converges, further includes:
And step S500, if the second global model is detected to be in an unconverged state, issuing the second global model with parameter encryption to a plurality of client terminals respectively, so that the client terminals continue to iterate training according to the second global model issued by the cooperative terminal respectively to return model parameters to the cooperative terminal.
In this embodiment, if the second global model is detected to be in an unconverged state, the second global model encrypted by parameters is issued to a plurality of client terminals respectively, and after the collaboration terminal receives the model parameters sent by the plurality of client terminals respectively, the weighting coefficient corresponding to each model parameter is obtained according to a preset obtaining rule; the model parameters are obtained by performing federal model training on a second global model encrypted by the client terminal according to parameters issued by the cooperative terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters; according to the model parameters and the weight coefficient corresponding to each model parameter, a third global model is obtained through aggregation; detecting whether the third global model converges; if the third global model is detected to be in a convergence state, determining the third global model as a final result of federal model training, and issuing a parameter-encrypted third global model to the plurality of client terminals, and if the third global model is detected to be in an unconverged state, issuing the parameter-encrypted third global model to the plurality of client terminals respectively, repeating the steps of any embodiment of the invention, and continuing training until the model converges.
Further, a fifth embodiment of the federal model training method of the present invention is presented.
Based on the embodiment shown in fig. 2, in this embodiment, step S100, after the cooperative terminal receives the model parameters sent by the plurality of client terminals, further includes the steps before the step of obtaining the weight coefficient corresponding to each model parameter according to a preset obtaining rule:
Receiving model parameters respectively sent by a plurality of client terminals;
Step S100, when the collaboration terminal receives model parameters sent by a plurality of client terminals respectively, the step of obtaining weight coefficients corresponding to each model parameter according to a preset obtaining rule includes:
Receiving weight coefficients corresponding to the model parameters, which are respectively sent by the plurality of client terminals; and the plurality of client terminals respectively test and obtain the prediction error rate of the prediction model corresponding to the model parameters according to a preset test sample set, and calculate and obtain the weight coefficient corresponding to the model parameters according to the prediction error rate and a preset calculation formula.
In this embodiment, as an implementation manner, the calculation of the weight coefficient corresponding to the model parameter of each client terminal is performed in each client terminal, multiple client terminals in the federal learning system all store the same test sample set, the test samples in the test sample set and the training samples of multiple client terminals all have the same feature dimensions, the weight coefficient corresponding to the model parameter of each client terminal may be the prediction error rate of the client terminal for testing its prediction model according to the locally stored test sample set, so as to obtain the weight coefficient corresponding to the model parameter thereof, and the client terminal sends the model parameter and simultaneously sends the weight coefficient corresponding to the calculated model parameter to the collaboration terminal for the collaboration terminal to aggregate to obtain the global model.
In addition, the embodiment of the invention also provides a federal model training system, which comprises a cooperative terminal and a plurality of client terminals which are respectively in communication connection with the cooperative terminal, wherein the cooperative terminal comprises:
The acquisition module is used for acquiring a weight coefficient corresponding to each model parameter according to a preset acquisition rule after receiving the model parameters respectively sent by the plurality of client terminals; the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters;
The aggregation updating module is used for aggregating to obtain a second global model according to the plurality of model parameters and the weight coefficient corresponding to each model parameter;
The detection module is used for detecting whether the second global model converges or not;
And the determining module is used for determining the second global model as a final result of federal model training and issuing the second global model with parameter encryption to the plurality of client terminals when the detecting module detects that the second global model is in a convergence state.
Preferably, the acquiring module includes:
The test unit is used for testing and obtaining the prediction error rate of the prediction model corresponding to each model parameter according to a preset test sample set after receiving the model parameters respectively sent by the plurality of client terminals;
And the calculating unit is used for respectively calculating and obtaining the weight coefficient corresponding to each model parameter based on the prediction error rate of each prediction model and a preset calculating formula.
Preferably, the test unit includes:
The testing subunit is used for inputting a plurality of test samples in a preset test sample set into a prediction model corresponding to the model parameters to predict after receiving the model parameters respectively sent by a plurality of client terminals, so as to obtain a predicted value of the prediction model for each test sample;
the obtaining subunit is used for obtaining the number of the test samples with wrong prediction results in the test sample set according to a plurality of prediction values;
and the determining subunit is used for determining the ratio of the number of the test samples with the wrong prediction result to the number of all the test samples in the test sample set as the prediction error rate of the pre-prediction model.
Preferably, the preset calculation formula is: Wherein epsilon i is a prediction error rate of a prediction model of the ith client terminal, alpha i is a weight coefficient corresponding to a model parameter of the prediction model of the ith client terminal, and i is an integer greater than zero.
Preferably, the aggregation update module includes:
The multiplication processing unit is used for multiplying each model parameter with the corresponding weight coefficient to obtain a plurality of multiplied results;
and the updating unit is used for adding the multiplied results and determining the added result as a model parameter of a second global model to obtain the second global model.
Preferably, the cooperative terminal further includes:
the first issuing module is used for sending the first global model encrypted by the parameters to a plurality of client terminals respectively;
The receiving module is used for receiving model parameters respectively sent by the plurality of client terminals;
after receiving the first global model issued by the first issuing module, the client terminal predicts a first training sample set according to the first global model to obtain a predicted value, samples the first training sample set according to the predicted value to obtain a second training sample set, trains the first global model based on the second training sample set, and obtains model parameters after training.
Preferably, the cooperative terminal further includes:
and the second issuing module is used for issuing the second global model with encrypted parameters to a plurality of client terminals respectively when the detection module detects that the second global model is in an unconverged state, so that the client terminals continue to iterate and train according to the second global model issued by the second issuing module respectively to return model parameters to the collaborative terminal.
Preferably, the obtaining module is further configured to receive weight coefficients corresponding to the model parameters sent by the plurality of client terminals respectively; and the plurality of client terminals respectively test and obtain the prediction error rate of the prediction model corresponding to the model parameters according to a preset test sample set, and calculate and obtain the weight coefficient corresponding to the model parameters according to the prediction error rate and a preset calculation formula.
The specific implementation of the federal model training system is basically the same as the above-mentioned embodiments of the federal model training method, and will not be described here again.
In addition, the embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium is stored with a federal model training program, and the federal model training program realizes the steps of the rewards sending method when being executed by a processor.
The specific implementation of the computer readable storage medium of the present invention is substantially the same as the above embodiments of the federal model training method, and will not be described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (12)
1. A federal model training method, comprising the steps of:
After the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, acquiring weight coefficients corresponding to each model parameter according to a preset acquisition rule; the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters; after the collaboration terminal receives model parameters respectively sent by a plurality of client terminals, the step of acquiring the weight coefficient corresponding to each model parameter according to a preset acquisition rule comprises the following steps: after receiving model parameters respectively sent by a plurality of client terminals, the cooperative terminal tests and obtains a prediction error rate of a prediction model corresponding to each model parameter according to a preset test sample set; respectively calculating to obtain weight coefficients corresponding to each model parameter based on the prediction error rate of each prediction model and a preset calculation formula, wherein the prediction error rate of the prediction model corresponding to each model parameter is inversely related to the weight coefficients of the model parameters;
according to the model parameters and the weight coefficient corresponding to each model parameter, a second global model is obtained through aggregation; wherein, according to a plurality of model parameters and weight coefficients corresponding to each model parameter, the step of aggregating to obtain the second global model comprises: multiplying each model parameter by the corresponding weight coefficient to obtain a plurality of multiplied results; adding the multiplied results, determining the added result as a model parameter of a second global model, and obtaining the second global model;
detecting whether the second global model converges;
And if the second global model is detected to be in a convergence state, determining the second global model as a final result of federal model training, and issuing a second global model with encrypted parameters to the plurality of client terminals.
2. The federal model training method according to claim 1, wherein the step of testing and obtaining the prediction error rate of the prediction model corresponding to each model parameter according to a preset test sample set after the cooperative terminal receives the model parameters respectively transmitted by the plurality of client terminals comprises:
after receiving model parameters respectively sent by a plurality of client terminals, the cooperative terminal inputs a plurality of test samples in a preset test sample set into a prediction model corresponding to the model parameters for prediction, and a prediction value of the prediction model for each test sample is obtained;
obtaining the number of test samples with wrong prediction results in the test sample set according to a plurality of prediction values;
And determining the ratio of the number of the test samples with the wrong prediction result to the number of all the test samples in the test sample set as the prediction error rate of the prediction model.
3. The federal model training method according to any one of claims 1-2, wherein, after the cooperative terminal receives the model parameters respectively sent by the plurality of client terminals, the step of acquiring the weight coefficient corresponding to each model parameter according to a preset acquisition rule further includes:
sending the first global model encrypted by the parameters to a plurality of client terminals respectively;
receiving model parameters respectively sent by the plurality of client terminals;
After receiving the first global model issued by the collaboration terminal, the client terminal predicts a first training sample set according to the first global model to obtain a predicted value, samples the first training sample set according to the predicted value to obtain a second training sample set, trains the first global model based on the second training sample set, and obtains the model parameters after training.
4. The federal model training method according to claim 1, wherein the step of detecting whether the second global model converges further comprises:
and if the second global model is detected to be in an unconverged state, issuing the second global model with parameter encryption to a plurality of client terminals respectively, so that the client terminals continue to train iteratively according to the second global model issued by the cooperative terminal respectively to return model parameters to the cooperative terminal.
5. The federal model training method according to claim 1, wherein, after the cooperative terminal receives the model parameters respectively sent by the plurality of client terminals, the step of acquiring the weight coefficient corresponding to each model parameter according to a preset acquisition rule further comprises:
Receiving model parameters respectively sent by a plurality of client terminals;
After the collaboration terminal receives the model parameters respectively sent by the plurality of client terminals, the step of obtaining the weight coefficient corresponding to each model parameter according to a preset obtaining rule comprises the following steps:
Receiving weight coefficients corresponding to the model parameters, which are respectively sent by the plurality of client terminals; and the plurality of client terminals respectively test and obtain the prediction error rate of the prediction model corresponding to the model parameters according to a preset test sample set, and calculate and obtain the weight coefficient corresponding to the model parameters according to the prediction error rate and a preset calculation formula.
6. A federal model training system, the system comprising a collaboration terminal and a plurality of client terminals communicatively connected to the collaboration terminal, respectively, the collaboration terminal comprising:
The acquisition module is used for acquiring a weight coefficient corresponding to each model parameter according to a preset acquisition rule after receiving the model parameters respectively sent by the plurality of client terminals; the model parameters are obtained by performing federal model training on a first global model encrypted according to parameters issued by a cooperative terminal by a client terminal, and the weight coefficients are determined based on the prediction accuracy of a prediction model corresponding to the model parameters;
The aggregation updating module is used for aggregating to obtain a second global model according to the plurality of model parameters and the weight coefficient corresponding to each model parameter; wherein, the aggregation update module comprises: the multiplication processing unit is used for multiplying each model parameter with the corresponding weight coefficient to obtain a plurality of multiplied results; the updating unit is used for adding the multiplied results and determining the added result as a model parameter of a second global model to obtain the second global model;
The detection module is used for detecting whether the second global model converges or not;
The determining module is used for determining the second global model as a final result of federal model training and issuing the second global model with parameter encryption to the plurality of client terminals when the detecting module detects that the second global model is in a convergence state; wherein, the acquisition module includes:
The test unit is used for testing and obtaining the prediction error rate of the prediction model corresponding to each model parameter according to a preset test sample set after receiving the model parameters respectively sent by the plurality of client terminals;
And the calculating unit is used for respectively calculating and obtaining the weight coefficient corresponding to each model parameter based on the prediction error rate of each prediction model and a preset calculating formula, wherein the prediction error rate of the prediction model corresponding to each model parameter is inversely related to the weight coefficient of the model parameter.
7. The federal model training system according to claim 6, wherein the test unit comprises:
The testing subunit is used for inputting a plurality of test samples in a preset test sample set into a prediction model corresponding to the model parameters to predict after receiving the model parameters respectively sent by a plurality of client terminals, so as to obtain a predicted value of the prediction model for each test sample;
the obtaining subunit is used for obtaining the number of the test samples with wrong prediction results in the test sample set according to a plurality of prediction values;
and the determining subunit is used for determining the ratio of the number of the test samples with the wrong prediction result to the number of all the test samples in the test sample set as the prediction error rate of the prediction model.
8. The federal model training system according to any one of claims 6-7, wherein the collaboration terminal further comprises:
the first issuing module is used for sending the first global model encrypted by the parameters to a plurality of client terminals respectively;
The receiving module is used for receiving model parameters respectively sent by the plurality of client terminals;
after receiving the first global model issued by the first issuing module, the client terminal predicts a first training sample set according to the first global model to obtain a predicted value, samples the first training sample set according to the predicted value to obtain a second training sample set, trains the first global model based on the second training sample set, and obtains model parameters after training.
9. The federal model training system according to claim 6, wherein the collaboration terminal further comprises:
and the second issuing module is used for issuing the second global model with encrypted parameters to a plurality of client terminals respectively when the detection module detects that the second global model is in an unconverged state, so that the client terminals continue to iterate and train according to the second global model issued by the second issuing module respectively to return model parameters to the collaborative terminal.
10. The federal model training system according to claim 6, wherein the acquisition module is further configured to receive weight coefficients corresponding to the model parameters respectively transmitted by the plurality of client terminals; and the plurality of client terminals respectively test and obtain the prediction error rate of the prediction model corresponding to the model parameters according to a preset test sample set, and calculate and obtain the weight coefficient corresponding to the model parameters according to the prediction error rate and a preset calculation formula.
11. Federal model training apparatus, characterized in that it comprises a memory, a processor and a federal model training program stored on the memory and executable on the processor, which federal model training program, when executed by the processor, implements the steps of the federal model training method according to any of claims 1 to 5.
12. A computer readable storage medium, having stored thereon a federal model training program, which when executed by a processor, implements the steps of the federal model training method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910121269.8A CN109871702B (en) | 2019-02-18 | 2019-02-18 | Federal model training method, system, apparatus, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910121269.8A CN109871702B (en) | 2019-02-18 | 2019-02-18 | Federal model training method, system, apparatus, and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109871702A CN109871702A (en) | 2019-06-11 |
| CN109871702B true CN109871702B (en) | 2024-06-28 |
Family
ID=66918815
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910121269.8A Active CN109871702B (en) | 2019-02-18 | 2019-02-18 | Federal model training method, system, apparatus, and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109871702B (en) |
Families Citing this family (65)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110334544B (en) * | 2019-06-26 | 2023-07-25 | 深圳前海微众银行股份有限公司 | Federal model degradation processing method and device, federal training system and storage medium |
| CN112183757B (en) * | 2019-07-04 | 2023-10-27 | 创新先进技术有限公司 | Model training method, device and system |
| CN110378487B (en) * | 2019-07-18 | 2021-02-26 | 深圳前海微众银行股份有限公司 | Method, device, equipment and medium for verifying model parameters in horizontal federal learning |
| CN110378749B (en) * | 2019-07-25 | 2023-09-26 | 深圳前海微众银行股份有限公司 | Client similarity evaluation method and device, terminal equipment and storage medium |
| CN110443416A (en) * | 2019-07-30 | 2019-11-12 | 卓尔智联(武汉)研究院有限公司 | Federal model building device, method and readable storage medium storing program for executing based on shared data |
| CN110490738A (en) * | 2019-08-06 | 2019-11-22 | 深圳前海微众银行股份有限公司 | A kind of federal learning method of mixing and framework |
| US12093837B2 (en) | 2019-08-09 | 2024-09-17 | International Business Machines Corporation | Building a federated learning framework |
| CN110442457A (en) * | 2019-08-12 | 2019-11-12 | 北京大学深圳研究生院 | Model training method, device and server based on federation's study |
| CN110380917B (en) * | 2019-08-26 | 2022-01-07 | 深圳前海微众银行股份有限公司 | Control method and device of federal learning system, terminal equipment and storage medium |
| CN110503207A (en) * | 2019-08-28 | 2019-11-26 | 深圳前海微众银行股份有限公司 | Federation's study credit management method, device, equipment and readable storage medium storing program for executing |
| CN110632554A (en) * | 2019-09-20 | 2019-12-31 | 深圳前海微众银行股份有限公司 | Indoor positioning method, device, terminal equipment and medium based on federal learning |
| CN110674528B (en) * | 2019-09-20 | 2024-04-09 | 深圳前海微众银行股份有限公司 | Federal learning privacy data processing method, device, system and storage medium |
| CN110601814B (en) * | 2019-09-24 | 2021-08-27 | 深圳前海微众银行股份有限公司 | Federal learning data encryption method, device, equipment and readable storage medium |
| CN110705627A (en) * | 2019-09-26 | 2020-01-17 | 深圳前海微众银行股份有限公司 | Target detection method, target detection system, target detection device and readable storage medium |
| CN110838069A (en) * | 2019-10-15 | 2020-02-25 | 支付宝(杭州)信息技术有限公司 | Data processing method, device and system |
| CN112656401B (en) * | 2019-10-15 | 2023-08-22 | 梅州市青塘实业有限公司 | Intelligent monitoring method, device and equipment |
| CN112749812A (en) * | 2019-10-29 | 2021-05-04 | 华为技术有限公司 | Joint learning system, training result aggregation method and equipment |
| CN110782042B (en) * | 2019-10-29 | 2022-02-11 | 深圳前海微众银行股份有限公司 | Method, device, equipment and medium for combining horizontal federation and vertical federation |
| CN110766169A (en) * | 2019-10-31 | 2020-02-07 | 深圳前海微众银行股份有限公司 | Transfer training optimization method and device for reinforcement learning, terminal and storage medium |
| CN110929260A (en) * | 2019-11-29 | 2020-03-27 | 杭州安恒信息技术股份有限公司 | Malicious software detection method, device, server and readable storage medium |
| CN110992936A (en) * | 2019-12-06 | 2020-04-10 | 支付宝(杭州)信息技术有限公司 | Method and apparatus for model training using private data |
| CN111027715B (en) * | 2019-12-11 | 2021-04-02 | 支付宝(杭州)信息技术有限公司 | Monte Carlo-based federated learning model training method and device |
| CN111241582B (en) * | 2020-01-10 | 2022-06-10 | 鹏城实验室 | Data privacy protection method and device and computer readable storage medium |
| CN111275207B (en) * | 2020-02-10 | 2024-04-30 | 深圳前海微众银行股份有限公司 | Semi-supervision-based transverse federal learning optimization method, equipment and storage medium |
| CN111369042B (en) * | 2020-02-27 | 2021-09-24 | 山东大学 | Wireless service flow prediction method based on weighted federal learning |
| CN111477290B (en) * | 2020-03-05 | 2023-10-31 | 上海交通大学 | Federal learning and image classification method, system and terminal for protecting user privacy |
| CN111382875A (en) * | 2020-03-06 | 2020-07-07 | 深圳前海微众银行股份有限公司 | Federal model parameter determination method, device, equipment and storage medium |
| WO2021179196A1 (en) * | 2020-03-11 | 2021-09-16 | Oppo广东移动通信有限公司 | Federated learning-based model training method, electronic device, and storage medium |
| CN113470806B (en) * | 2020-03-31 | 2024-05-24 | 中移(成都)信息通信科技有限公司 | Method, device, equipment and computer storage medium for determining disease detection model |
| CN111461329B (en) * | 2020-04-08 | 2024-01-23 | 中国银行股份有限公司 | Model training method, device, equipment and readable storage medium |
| CN111460511B (en) * | 2020-04-17 | 2023-05-02 | 支付宝(杭州)信息技术有限公司 | Federal learning and virtual object distribution method and device based on privacy protection |
| WO2021227069A1 (en) * | 2020-05-15 | 2021-11-18 | Oppo广东移动通信有限公司 | Model updating method and apparatus, and communication device |
| CN113688855B (en) * | 2020-05-19 | 2023-07-28 | 华为技术有限公司 | Data processing method, federal learning training method, related device and equipment |
| CN111340150B (en) * | 2020-05-22 | 2020-09-04 | 支付宝(杭州)信息技术有限公司 | Method and device for training first classification model |
| CN113705823A (en) * | 2020-05-22 | 2021-11-26 | 华为技术有限公司 | Model training method based on federal learning and electronic equipment |
| US11715044B2 (en) | 2020-06-02 | 2023-08-01 | Huawei Cloud Computing Technologies Co., Ltd. | Methods and systems for horizontal federated learning using non-IID data |
| CN113259147B (en) * | 2020-06-28 | 2022-07-26 | 中兴通讯股份有限公司 | Network element management method, device, computer equipment and medium |
| CN113282933B (en) * | 2020-07-17 | 2022-03-01 | 中兴通讯股份有限公司 | Federal learning method, device and system, electronic equipment and storage medium |
| CN111882133B (en) * | 2020-08-03 | 2022-02-01 | 重庆大学 | Prediction-based federated learning communication optimization method and system |
| CN111932367A (en) * | 2020-08-13 | 2020-11-13 | 中国银行股份有限公司 | Pre-credit evaluation method and device |
| CN112001452B (en) * | 2020-08-27 | 2021-08-27 | 深圳前海微众银行股份有限公司 | Feature selection method, device, equipment and readable storage medium |
| CN112214342B (en) * | 2020-09-14 | 2024-05-24 | 德清阿尔法创新研究院 | Efficient error data detection method in federal learning scene |
| CN112261137B (en) * | 2020-10-22 | 2022-06-14 | 无锡禹空间智能科技有限公司 | Model training method and system based on joint learning |
| CN112149171B (en) * | 2020-10-27 | 2021-07-09 | 腾讯科技（深圳）有限公司 | Method, device, equipment and storage medium for training federal neural network model |
| CN112365007B (en) * | 2020-11-11 | 2024-06-25 | 深圳前海微众银行股份有限公司 | Model parameter determining method, device, equipment and storage medium |
| CN112465043B (en) * | 2020-12-02 | 2024-05-14 | 平安科技（深圳）有限公司 | Model training method, device and equipment |
| CN112217706B (en) * | 2020-12-02 | 2021-03-19 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN114610475A (en) * | 2020-12-03 | 2022-06-10 | 深圳清华大学研究院 | Training method of intelligent resource arrangement model |
| CN112819177B (en) * | 2021-01-26 | 2022-07-12 | 支付宝(杭州)信息技术有限公司 | Personalized privacy protection learning method, device and equipment |
| CN112885337A (en) * | 2021-01-29 | 2021-06-01 | 深圳前海微众银行股份有限公司 | Data processing method, device, equipment and storage medium |
| CN112989929B (en) * | 2021-02-04 | 2022-08-09 | 支付宝(杭州)信息技术有限公司 | Target user identification method and device and electronic equipment |
| CN113077060A (en) * | 2021-03-30 | 2021-07-06 | 中国科学院计算技术研究所 | Federal learning system and method aiming at edge cloud cooperation |
| CN113128701A (en) * | 2021-04-07 | 2021-07-16 | 中国科学院计算技术研究所 | Sample sparsity-oriented federal learning method and system |
| CN113095513A (en) * | 2021-04-25 | 2021-07-09 | 中山大学 | Double-layer fair federal learning method, device and storage medium |
| CN113268758B (en) * | 2021-06-17 | 2022-11-04 | 上海万向区块链股份公司 | Data sharing system, method, medium and device based on federal learning |
| CN113361721B (en) * | 2021-06-29 | 2023-07-18 | 北京百度网讯科技有限公司 | Model training method, device, electronic equipment, storage medium and program product |
| CN113487043A (en) * | 2021-07-22 | 2021-10-08 | 深圳前海微众银行股份有限公司 | Federal learning modeling optimization method, apparatus, medium, and computer program product |
| CN113642737B (en) * | 2021-08-12 | 2024-03-05 | 广域铭岛数字科技有限公司 | Federal learning method and system based on automobile user data |
| CN113658689B (en) * | 2021-08-25 | 2024-07-30 | 深圳前海微众银行股份有限公司 | Training method and device for multi-agent model, electronic equipment and storage medium |
| CN113850396B (en) * | 2021-09-28 | 2022-04-19 | 北京邮电大学 | Privacy enhanced federal decision method, device, system and storage medium |
| CN114912581A (en) * | 2022-05-07 | 2022-08-16 | 奇安信科技集团股份有限公司 | Training method and device for detection model, electronic equipment and storage medium |
| CN117279014A (en) * | 2022-06-13 | 2023-12-22 | 中兴通讯股份有限公司 | Fault prediction method and device, electronic equipment and storage medium |
| WO2024026583A1 (en) * | 2022-07-30 | 2024-02-08 | 华为技术有限公司 | Communication method and communication apparatus |
| CN115526339B (en) * | 2022-11-03 | 2024-05-17 | 中国电信股份有限公司 | Federal learning method, federal learning device, electronic apparatus, and computer-readable storage medium |
| CN117251276B (en) * | 2023-11-20 | 2024-02-09 | 清华大学 | Flexible scheduling method and device for collaborative learning platform |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107133805A (en) * | 2017-05-09 | 2017-09-05 | 北京小度信息科技有限公司 | Method of adjustment, device and the equipment of user's cheating category forecasting Model Parameter |
| CN109284313A (en) * | 2018-08-10 | 2019-01-29 | 深圳前海微众银行股份有限公司 | Federal modeling method, equipment and readable storage medium storing program for executing based on semi-supervised learning |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160147943A1 (en) * | 2014-11-21 | 2016-05-26 | Argo Data Resource Corporation | Semantic Address Parsing Using a Graphical Discriminative Probabilistic Model |
| CN107229518B (en) * | 2016-03-26 | 2020-06-30 | 阿里巴巴集团控股有限公司 | Distributed cluster training method and device |
| KR20190032433A (en) * | 2016-07-18 | 2019-03-27 | 난토믹스, 엘엘씨 | Distributed machine learning systems, apparatus, and methods |
| WO2018098230A1 (en) * | 2016-11-22 | 2018-05-31 | Massachusetts Institute Of Technology | Systems and methods for training neural networks |
| CN108648023A (en) * | 2018-05-18 | 2018-10-12 | 上海电机学院 | A kind of businessman's passenger flow forecast method of fusion history mean value and boosted tree |
| CN109102157A (en) * | 2018-07-11 | 2018-12-28 | 交通银行股份有限公司 | A kind of bank's work order worksheet processing method and system based on deep learning |
| CN109165683B (en) * | 2018-08-10 | 2023-09-12 | 深圳前海微众银行股份有限公司 | Sample prediction method, device and storage medium based on federal training |
| CN109165725B (en) * | 2018-08-10 | 2022-03-29 | 深圳前海微众银行股份有限公司 | Neural network federal modeling method, equipment and storage medium based on transfer learning |
| CN109165515A (en) * | 2018-08-10 | 2019-01-08 | 深圳前海微众银行股份有限公司 | Model parameter acquisition methods, system and readable storage medium storing program for executing based on federation's study |
-
2019
- 2019-02-18 CN CN201910121269.8A patent/CN109871702B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107133805A (en) * | 2017-05-09 | 2017-09-05 | 北京小度信息科技有限公司 | Method of adjustment, device and the equipment of user's cheating category forecasting Model Parameter |
| CN109284313A (en) * | 2018-08-10 | 2019-01-29 | 深圳前海微众银行股份有限公司 | Federal modeling method, equipment and readable storage medium storing program for executing based on semi-supervised learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109871702A (en) | 2019-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109871702B (en) | Federal model training method, system, apparatus, and computer-readable storage medium | |
| Han et al. | Adaptive gradient sparsification for efficient federated learning: An online learning approach | |
| CN110378488B (en) | Client-side change federal training method, device, training terminal and storage medium | |
| Angjelichinoski et al. | A statistical learning approach to ultra-reliable low latency communication | |
| CN111538598B (en) | Federal learning modeling method, apparatus, device, and readable storage medium | |
| CN113516255B (en) | Federal learning modeling optimization method, apparatus, readable storage medium, and program product | |
| CN113139662B (en) | Global and local gradient processing method, device, equipment and medium for federal learning | |
| US8976656B2 (en) | Available bandwidth measurement system, transmission device, available bandwidth measurement method and recording medium | |
| KR100961399B1 (en) | Method for closed-loop subspace transmission and reception in a two transmit N-receive antenna system | |
| CN111226238A (en) | Prediction method, terminal and server | |
| US10263858B2 (en) | Environment simulator for user percentile | |
| US12047658B2 (en) | Control apparatus, control method, and program | |
| CN112766497A (en) | Deep reinforcement learning model training method, device, medium and equipment | |
| AU2015203099A1 (en) | Bandwidth prediction apparatus, bandwidth prediction system, bandwidth prediction program, bandwidth prediction method and recording medium | |
| JP6093791B2 (en) | POSITIONING DEVICE, POSITIONING METHOD, POSITIONING PROGRAM, AND POSITIONING SYSTEM | |
| CN116776155A (en) | Model training method, device, equipment and medium based on federal learning | |
| JP5804060B2 (en) | Parameter estimation apparatus, parameter estimation method, and parameter estimation program | |
| US9942828B2 (en) | Wireless communication apparatus, wireless communication method, and computer-readable recording medium | |
| CN113516254A (en) | Method, apparatus, medium, and program product for optimizing horizontal federated learning modeling | |
| CN115424079B (en) | Image classification method based on federal edge learning and related equipment | |
| CN117273119A (en) | Dynamic fairness federal learning method and device based on reinforcement learning | |
| CN111194048A (en) | 1-bit parameter estimation method based on EM | |
| US10250476B2 (en) | Identifying modifications to technical characteristics of a communication channel to alter user perception of communication channel performance | |
| Yang et al. | Privacy-Preserving Digital Twin for Vehicular Edge Computing Networks | |
| CN112929907A (en) | Method and device for determining antenna parameters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |