CN114862400A

CN114862400A - Data processing method, device and storage medium

Info

Publication number: CN114862400A
Application number: CN202110152070.9A
Authority: CN
Inventors: 郭俊华; 曹结松; 陈永; 王磊
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-02-03
Filing date: 2021-02-03
Publication date: 2022-08-05
Also published as: WO2022166278A1

Abstract

The application relates to a data processing method, a device and a storage medium, wherein the method comprises the following steps: receiving first response information of a first user, receiving second response information of a second user, obtaining the target annotation data set by a sandbox according to the first response information, obtaining the target algorithm model by the sandbox according to the second response information, evaluating the target algorithm model by using the target annotation data set to obtain an evaluation result, wherein the evaluation result is provided for a third user, and the first user, the second user and the third user forbid to access the target annotation data set and the target algorithm model in the sandbox. According to the embodiment of the application, the marked data set and the algorithm model can be guaranteed not to be sold and abused for the second time, the process is traceable and controllable, the potential distrust problem of a user is solved, the problem that a user cannot download and use a large amount of data is solved, and the marked data set and the algorithm model can be shared for many times after being uploaded once.

Description

Data processing method, device and storage medium

Technical Field

The present application relates to the field of data processing, and in particular, to a data processing method, apparatus, and storage medium.

Background

Now having entered the big data era, the exchange of data and algorithmic models within various industries has become more important, for example, the training of the autodrive perception model requires a large amount of labeled data, the types of labeled data include: laser point cloud 3D detection, laser point cloud segmentation, picture 2D detection, picture segmentation and the like, wherein the data need to cover different traffic conditions such as weather, road conditions, traffic participants and the like; on the aspect of testing and verifying the automatic driving regulation and control algorithm, the industry generally considers that a set of commercially available automatic driving vehicles needs road testing of at least more than 100 hundred million miles, and a perception model needs to output key perception results for the automatic driving vehicles, so as to support the automatic driving regulation and control algorithm decision and planning. On one hand, enterprises need to enrich own data sets to acquire data, and meanwhile, a large amount of data collected by the enterprises and owned algorithms can be expected to be revealed through transactions.

Because digital commodities have the particularity of being acquired once and used for unlimited times, under the current technical means, the traditional electronic commerce platform B2B is generally used for trading of the digital commodities, the behaviors of resale, abuse and the like of the digital commodities after sale cannot be avoided, and a novel credible, traceable and controllable technical means is urgently needed.

Disclosure of Invention

In view of the above, a data processing method, apparatus and storage medium are provided.

In a first aspect, an embodiment of the present application provides a data processing method, where the method includes: receiving first response information of a first user, wherein the first user provides at least one annotation data set, the first response information comprises a storage address of a target object determined by the first user, and the target object comprises a target annotation data set requested to be used by a third user; receiving second response information of a second user, wherein the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model requested to be used by a third user; the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; and evaluating a target algorithm model by using the target annotation data set to obtain an evaluation result, wherein the evaluation result is used for being provided for a third user, and the first user, the second user and the third user forbid accessing the target annotation data set and the target algorithm model in the sandbox.

According to the embodiment of the application, the sandbox acquires the target annotation data set according to the first response information by receiving the response information of the first user and receiving the second response information of the second user; the sandbox acquires the target algorithm model according to the second response information; the target annotation data set is used for evaluating the target algorithm model to obtain an evaluation result, sharing and transaction of the annotation data set and the algorithm model among users can be realized, a user (a third user) can also obtain evaluation of the target algorithm model to assist the user in freely selecting the target algorithm model, the first user, the second user and the third user forbid access to the target annotation data set and the target algorithm model in the sandbox, the user cannot access source data, safe sharing of the annotation data set and the algorithm model can be realized, and the problems of secondary resale, abusing and abuse of the source data are prevented. The sandbox is used as a third party to acquire and process data, the process is traceable and controllable, and the potential distrust problem of a user and all parties is solved. In addition, a user (a third user) does not directly obtain the annotation data set and the algorithm model, so that the problem that the user cannot download and use a large amount of data is solved, and the annotation data set and the algorithm model can be shared for many times by uploading once.

According to the first aspect, in a first possible implementation manner of the data processing method, the first user and the second user are the same user.

According to the embodiment of the application, the first user and the second user are the same user, the marked data set and the algorithm model can be provided by the same user, and the range of resources which can be shared by the users is expanded.

According to the first aspect, in a second possible implementation manner of the data processing method, the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the method further comprises the following steps: destroying the target labeling data set obtained by the sandbox according to the use strategy of the target labeling data set; and destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

According to the embodiment of the application, the target labeling data set acquired by the sandbox is destroyed according to the use strategy of the target labeling data set; according to the use strategy of the target algorithm model, the target algorithm model obtained by the sandbox is destroyed, so that the marking data set and the algorithm model can be incinerated after use, and the marking data set and the algorithm model are prevented from being re-sold and abused for the second time.

According to the second possible implementation manner of the first aspect, in a third possible implementation manner of the data processing method, according to a usage policy of the target annotation data set, destroying the target annotation data set acquired by the sandbox includes: under the condition that the number of times of using the target labeling data set reaches a preset first threshold value, destroying the target labeling data set; and/or destroying the target annotation data set under the condition that the use time of the target annotation data set reaches a preset second threshold value; according to the using strategy of the target algorithm model, destroying the target algorithm model obtained by the sandbox, which comprises the following steps: destroying the target algorithm model under the condition that the using times of the target algorithm model reaches a preset third threshold value; and/or destroying the target algorithm model in case the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

According to the embodiment of the application, the destruction is carried out according to the use times or use time of the target labeling data set and the target algorithm model, so that the destruction strategy can be flexibly selected by a user, and different destruction requirements of different labeling data sets, different algorithm models and different users are met.

According to the first aspect and at least one of the first, second, and third possible implementation manners of the first aspect, in a fourth possible implementation manner of the data processing method, the obtaining, by a sandbox, the target annotation data set according to the first response information includes: the sandbox acquires the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set; the sandbox obtains the target algorithm model according to the second response information, and the method comprises the following steps: and the sandbox acquires the target algorithm model stored in the cloud storage unit according to the storage address of the target algorithm model.

According to the embodiment of the application, the sandbox is used for acquiring the target labeling data set and the target algorithm model stored in the cloud storage unit according to the storage addresses of the target labeling data set and the target algorithm model, so that the labeling data set and the algorithm model can be uploaded once and shared for many times without being uploaded again during each use, repeated network transmission of large batches of data is avoided, network bandwidth is wasted, and the problems of storage and sharing difficulty caused by large-volume labeling data sets can be solved by means of cloud storage resources and expansion capacity of the cloud storage resources.

According to the first aspect, at least one of the first, second, third, and fourth possible implementation manners of the first aspect, in a fifth possible implementation manner of the data processing method, the sandbox converts the obtained target algorithm model according to a conversion algorithm model to match a compiling environment of the sandbox.

According to the embodiment of the application, the obtained target algorithm model is converted according to the conversion algorithm model, so that the sandbox can support algorithm models of different compiling environments for analysis and calculation, and sharing and transaction of diversified algorithm models are supported.

In a sixth possible implementation manner of the data processing method according to the first aspect as well as at least one of the first, second, third, fourth, and fifth possible implementation manners of the first aspect, the method further includes: determining whether the first user is authentic according to first authentication information, the first authentication information being provided when the first user provides information of at least one annotation data set; determining whether the second user is trusted according to second identity verification information, wherein the second identity verification information is provided when the second user provides information of at least one algorithm model; determining whether the third user is authentic according to third authentication information, the third authentication information being provided when the third user requests to use the target annotation data set and the target algorithm model.

The problem that users are not trusted with each other can be solved by respectively verifying whether the first user, the second user and the third user are trusted when the first user provides information of at least one labeled data set, the second user provides information of at least one algorithm model and the third user requests to use a target labeled data set and the target algorithm model, so that safe sharing of the labeled data set and the algorithm model is realized, and the process is supervised and traceable.

According to the first aspect, and at least one of the first, second, third, fourth, fifth, and sixth possible implementation manners of the first aspect, in a seventh possible implementation manner of the data processing method, information of a first object to be requested sent by the first user is received, where the first object to be requested includes at least one labeled data set provided by the first user; receiving information of a second object to be requested sent by the second user, wherein the second object to be requested comprises at least one algorithm model provided by the second user; notifying the third user of the information of the first object to be requested and the information of the second object to be requested; receiving first request information sent by the third user, wherein the first request information indicates an annotation data set in the first object to be requested, which is requested to be used by the third user; receiving second request information sent by the third user, wherein the second request information indicates an algorithm model in the second object to be requested, which is requested to be used by the third user; sending the first request information to the first user, wherein the first request information is used for determining the first response information; and sending the second request information to the second user, wherein the second request information is used for determining the second response information.

According to the embodiment of the application, the sharing and transaction between users for the labeled data set and the algorithm model can be realized by receiving the information of the first object to be requested sent by the first user, receiving the information of the second object to be requested sent by the second user, informing the third user of the information of the first object to be requested and the information of the second object to be requested, receiving the first request information sent by the third user, receiving the second request information sent by the third user, sending the first request information to the first user, and sending the second request information to the second user, so that the requirements of the users for the exchange and the change of the labeled data set and the algorithm model are met.

According to a seventh possible implementation manner of the first aspect, in an eighth possible implementation manner of the data processing method, the information of the first object to be requested further includes first authentication information, the information of the second object to be requested further includes second authentication information, and the first request information and the second request information further include third authentication information, and the method further includes: when the information of the first object to be requested is received, determining whether the first user is credible according to the first identity verification information; when receiving the information of the second object to be requested, determining whether the second user is credible according to the second identity authentication information; when the first request information is received, determining whether the third user is credible according to the third identity authentication information; and when the second request message is received, determining whether the third user is credible according to the third identity authentication information.

According to the embodiment of the application, when the first object information to be requested, the second object information to be requested, the first request information and the second request information are received, whether the users are credible or not is determined according to the identity verification information, the problem that the users are not credible mutually can be solved, the safe sharing of the labeled data set and the algorithm model is realized, and the process can be supervised and traced.

In a ninth possible implementation manner of the data processing method according to the first aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh and eighth possible implementation manners of the first aspect, the method further includes: receiving demand information sent by the third user, wherein the demand information represents a labeled data set and/or an algorithm model required by the third user; notifying the first user and/or the second user of the demand information.

According to the embodiment of the application, the demand information sent by the third user is received, and the first user and/or the second user are/is informed of the demand information, so that the users can release demands for the labeled data sets and the algorithm models, the exchange of the labeled data sets and the algorithm models among the users is more convenient, and the personalized customization of the labeled data sets and the algorithm models is supported.

In a tenth possible implementation form of the data processing method according to the second possible implementation form of the first aspect, the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to the indication of the third user.

According to the embodiment of the application, a third user can put forward a use strategy of the labeled data set or the algorithm model according to the self requirement, so that the flexibility of the use strategy is improved.

In an eleventh possible implementation manner of the data processing method according to the seventh possible implementation manner of the first aspect, the first request information further includes a usage policy of the annotation data set indicated by the first request information, where the usage policy is used to determine a condition for destroying the annotation data set indicated by the first request information; the second request information further comprises a use strategy of the algorithm model indicated by the second request information, and the use strategy is used for determining the condition of destroying the algorithm model indicated by the second request information; the use strategy of the target annotation data set is determined according to the use strategy included in the first request message, and the use strategy of the target algorithm model is determined according to the use strategy included in the second request message.

According to the embodiment of the application, the first request information further comprises the use strategy of the annotation data set indicated by the first request information, the second request information further comprises the use strategy of the algorithm model indicated by the second request information, the use strategy of the target annotation data set is determined according to the use strategy included by the first request information, and the use strategy of the target algorithm model is determined according to the use strategy included by the second request information, so that the purpose of burning the target annotation data set and the target algorithm model after use can be realized, and the annotation data set and the algorithm model are prevented from being resold and abused for a second time. The third user can put forward the use strategy of the labeling data set or the algorithm model according to the self requirement, and the flexibility of the use strategy is improved.

In a twelfth possible implementation form of the data processing method according to the first aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth and eleventh possible implementation forms of the first aspect, the algorithmic model is a machine learning model.

According to the embodiment of the application, safe sharing and fair transaction of the machine learning model among different users can be realized, abuse of source data is prevented, the user can obtain a required evaluation result without downloading the source data of the machine learning model, and the machine learning model meeting requirements can be conveniently selected.

In a thirteenth possible implementation form of the data processing method according to the first aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh, ninth, tenth and eleventh possible implementation forms of the first aspect, the sandbox is supervised by a blockchain, the first response information is used for determining intelligent contracts made by the first user and the third user, and the second response information is used for determining intelligent contracts made by the second user and the third user.

According to the embodiment of the application, by means of the block chain technology, the uplink data of the users can be not tampered, the problem of distrust among the users is solved, the safe sharing of the labeled data set and the algorithm model is realized, the process is transparent to the users, the traceability is controllable, and the transaction information can not be tampered.

In a second aspect, an embodiment of the present application provides a data processing method, where the method includes: receiving first response information of a first user, wherein the first user provides at least one annotation data set, the first response information comprises a storage address of a target object determined by the first user, and the target object comprises the target annotation data set provided by the first user; receiving second response information of a second user, wherein the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model requested to be used by the first user; the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; and evaluating a target algorithm model by using the target labeling data set to obtain an evaluation result, wherein the evaluation result is used for being provided for a first user, and the first user and the second user forbid accessing the target labeling data set and the target algorithm model in the sandbox.

According to the embodiment of the application, first response information of a first user is received, second response information of a second user is received, and the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; and evaluating the target algorithm model by using the target labeling data set to obtain an evaluation result, so that sharing and transaction of the algorithm model by the user can be realized, and the first user can evaluate the algorithm model provided by the second user through the own labeling data set so as to assist the first user in selecting the required algorithm model. The first user and the second user forbid access to the target labeling data set and the target algorithm model in the sandbox, the user cannot access to the source data, the secure sharing of the algorithm model can be achieved, and the problems of secondary resale, excessive distribution and abuse of the source data are prevented. The sandbox is used as a third party to acquire and process data, the process is traceable and controllable, and the potential distrust problem of a user and all parties is solved. In addition, a user (a first user) does not directly obtain the annotation data set and the algorithm model, so that the problem that the user cannot download and use a large amount of data is solved, and the annotation data set and the algorithm model can be shared for many times by uploading once.

According to a second aspect, in a first possible implementation manner of the data processing method, the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the method further comprises the following steps: destroying the target labeling data set obtained by the sandbox according to the use strategy of the target labeling data set; and destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

According to the first possible implementation manner of the second aspect, in a second possible implementation manner of the data processing method, according to a usage policy of the target annotation data set, destroying the target annotation data set acquired by the sandbox includes: destroying the target annotation data set under the condition that the using times of the target annotation data set reach a preset first threshold value; and/or destroying the target annotation data set when the using time of the target annotation data set reaches a preset second threshold value; according to the using strategy of the target algorithm model, destroying the target algorithm model obtained by the sandbox, which comprises the following steps: destroying the target algorithm model under the condition that the using times of the target algorithm model reach a preset third threshold value; and/or destroying the target algorithm model in the case that the usage time of the target algorithm model used reaches a predetermined fourth threshold value.

According to the second aspect and at least one of the first and second possible implementation manners of the second aspect, in a third possible implementation manner of the data processing method, the obtaining, by the sandbox, the target annotation data set according to the first response information includes: the sandbox acquires the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set; the sandbox obtains the target algorithm model according to the second response information, and the method comprises the following steps: and the sandbox acquires the target algorithm model stored in the cloud storage unit according to the storage address of the target algorithm model.

In a fourth possible implementation manner of the data processing method according to the second aspect as well as at least one of the first, second, and third possible implementation manners of the second aspect, the method further includes: and the sandbox converts the acquired target labeling data set and/or the target algorithm model according to a conversion algorithm model so as to match the compiling environment of the sandbox.

In a fifth possible implementation manner of the data processing method according to the second aspect as well as at least one of the first, second, third and fourth possible implementation manners of the second aspect, the method further includes: determining whether the first user is authentic according to first authentication information provided when the first user requests to use a target algorithm model; and determining whether the second user is credible according to second identity verification information, wherein the second identity verification information is provided when the second user provides information of at least one algorithm model.

According to the embodiment of the application, when the first user requests to use the target algorithm model and the second user provides information of at least one algorithm model, whether the first user and the second user are credible or not is verified respectively, the problem that the users are not credible with each other can be solved, the safe sharing of the labeled data set and the algorithm model is realized, and the process can be supervised and traced.

In a sixth possible implementation manner of the data processing method according to the second aspect as well as at least one of the first, second, third, fourth and fifth possible implementation manners of the second aspect, the method further includes: receiving information of an object to be requested sent by the second user, wherein the object to be requested comprises at least one algorithm model provided by the second user; notifying the first user of information of the object to be requested; receiving request information sent by the first user, wherein the request information indicates an algorithm model in the object to be requested, which is requested to be used by the first user; and sending the request information to the second user, wherein the request information is used for determining the second response information.

According to the embodiment of the application, the information of the object to be requested, which is sent by the second user, is received, the information of the object to be requested is notified to the first user, the request information sent by the first user is received, and the request information is sent to the second user, so that sharing and transaction of algorithm models among users can be realized, and the requirements of users on exchange and change of the algorithm models are met.

In a seventh possible implementation manner of the data processing method according to the sixth possible implementation manner of the second aspect, the information of the object to be requested further includes first authentication information, and the request information further includes second authentication information, and the method further includes: when the information of the object to be requested is received, determining whether the second user is credible according to the first identity verification information; and when the request information is received, determining whether the first user is credible according to the second identity authentication information.

According to the embodiment of the application, when the information of the object to be requested and the request information are received, whether the user is credible or not is determined according to the identity verification information, the problem that the users are not credible mutually can be solved, the algorithm model is safely shared, and the process can be supervised and traceable.

In an eighth possible implementation manner of the data processing method according to the second aspect as well as at least one of the first, second, third, fourth, fifth, sixth and seventh possible implementation manners of the second aspect, the method further includes: receiving demand information sent by the first user, wherein the demand information represents an algorithm model required by the first user; notifying the second user of the demand information.

According to the embodiment of the application, the requirement information sent by the first user is received, and the requirement information is notified to the second user, so that the user can issue the requirement for the algorithm model, the algorithm model is more conveniently exchanged among the users, and personalized customization of the algorithm model is supported.

In a ninth possible implementation form of the data processing method according to the first possible implementation form of the second aspect, the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to the indication of the first user.

According to the embodiment of the application, the first user can provide the use strategy of the labeled data set or the algorithm model according to the requirement of the first user, and the flexibility of the use strategy is improved.

In a tenth possible implementation manner of the data processing method according to the sixth possible implementation manner of the second aspect, the request information further includes: the use strategy of the algorithm model indicated by the request information is used for determining the condition of destroying the algorithm model indicated by the request information; the use strategy of the target algorithm model is determined according to the use strategy included in the request information.

According to the embodiment of the application, the request information further comprises the use strategy of the algorithm model indicated by the request information, and the use strategy of the target algorithm model is determined according to the use strategy included by the request information, so that the target algorithm model can be incinerated after use, and the algorithm model is prevented from being secondarily reselled and abused. The first user can provide the use strategy of the algorithm model according to the self requirement, and the flexibility of the use strategy is improved.

In an eleventh possible implementation form of the data processing method according to the second aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth and tenth possible implementation forms of the second aspect, the algorithmic model is a machine learning model.

In a twelfth possible implementation form of the data processing method according to the second aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth and eleventh possible implementation forms of the second aspect, the sandbox is supervised by a blockchain, and the second response information is used to determine the smart contracts made by the first user and the second user.

According to the embodiment of the application, by means of the block chain technology, the uplink data of the users can be not tampered, the problem of distrustment among the users is solved, the algorithm model is safely shared, the process is transparent to the users, the source can be controlled and traced, and the transaction information can not be tampered.

In a third aspect, an embodiment of the present application provides a data processing method, where the method includes: receiving first response information of a first user, wherein the first user provides at least one annotation data set, the first response information comprises a storage address of a target object determined by the first user, and the target object comprises a target annotation data set requested to be used by a second user; receiving second response information of a second user, wherein the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model provided by the second user; the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; and training the target algorithm model by using the target labeling data set to obtain a trained algorithm model, wherein the trained algorithm model is provided for the second user, and the first user and the second user forbid accessing the target labeling data set and the target algorithm model in the sandbox.

According to the embodiment of the application, first response information of a first user and second response information of a second user are received, and the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; the target labeling data set is used for training the target algorithm model to obtain the trained algorithm model, sharing and transaction of a user on the labeling data set can be achieved, a second user can train own algorithm model through the labeling data set provided by the first user, so that the second user can also train the algorithm model under the condition that the second user lacks enough labeling data, the first user and the second user forbid access to the target labeling data set and the target algorithm model in the sandbox, the user cannot access to source data, safe sharing of the labeling data set can be achieved, and the problems of secondary resale, distribution and abuse of the source data are prevented. The sandbox is used as a third party to acquire and process data, the process is traceable and controllable, and the potential distrust problem of a user and all parties is solved. In addition, a user (a second user) does not directly obtain the source data of the labeled data set and the algorithm model, so that the problem that the user cannot download and use a large amount of data is solved, and the labeled data set and the algorithm model can be shared for many times by uploading at one time.

According to a third aspect, in a first possible implementation manner of the data processing method, the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the method further comprises the following steps: destroying the target labeling data set obtained by the sandbox according to the use strategy of the target labeling data set; and destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

According to a first possible implementation manner of the third aspect, in a second possible implementation manner of the data processing method, according to a usage policy of the target annotation data set, destroying the target annotation data set acquired by the sandbox includes: destroying the target annotation data set under the condition that the using times of the target annotation data set reach a preset first threshold value; and/or destroying the target annotation data set when the using time of the target annotation data set reaches a preset second threshold value; according to the using strategy of the target algorithm model, destroying the target algorithm model obtained by the sandbox comprises the following steps: destroying the target algorithm model under the condition that the using times of the target algorithm model reach a preset third threshold value; and/or destroying the target algorithm model in case the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

According to the third aspect and at least one of the first and second possible implementation manners of the third aspect, in a third possible implementation manner of the data processing method, the obtaining, by the sandbox, the target annotation data set according to the first response information includes: the sandbox acquires the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set; the sandbox obtains the target algorithm model according to the second response information, and the method comprises the following steps: and the sandbox acquires the target algorithm model stored in the cloud storage unit according to the storage address of the target algorithm model.

According to the third aspect, at least one of the first, second and third possible implementation manners of the third aspect, in a fourth possible implementation manner of the data processing method, the method further includes: and the sandbox converts the acquired target labeling data set and/or the target algorithm model according to a conversion algorithm model so as to match the compiling environment of the sandbox.

According to the embodiment of the application, the acquired target algorithm model is converted according to the conversion algorithm model, so that the sandbox can support algorithm models of different compiling environments for analysis and calculation, and the use of diversified algorithm models by a user is supported.

According to the third aspect and at least one of the first, second, third and fourth possible implementation manners of the third aspect, in a fifth possible implementation manner of the data processing method, the method further includes: determining whether the first user is authentic according to first authentication information, the first authentication information being provided when the first user provides information of at least one annotation data set; and determining whether the second user is credible according to second identity verification information, wherein the second identity verification information is provided when the second user requests to use the target annotation data set.

According to the embodiment of the application, when the first user provides the information of at least one labeled data set and when the second user requests to use the target labeled data set, whether the first user and the second user are credible or not is verified respectively, the problem that the users are not credible with each other can be solved, the safe sharing of the labeled data set and the algorithm model is realized, and the process can be supervised and traced.

According to the third aspect, and at least one of the first, second, third, fourth and fifth possible implementation manners of the third aspect, in a sixth possible implementation manner of the data processing method, the method further includes: receiving information of an object to be requested sent by the first user, wherein the object to be requested comprises at least one annotation data set provided by the first user; notifying the second user of information of the object to be requested; receiving request information sent by the second user, wherein the request information indicates a labeled data set in the object to be requested, which is requested to be used by the second user; and sending the request information to the first user, wherein the request information is used for determining the first response information.

According to the embodiment of the application, receiving information of an object to be requested, which is sent by a first user, and informing a second user of the information of the object to be requested; and receiving request information sent by the second user, and sending the request information to the first user, so that sharing and transaction of the labeled data set among users can be realized, and the requirements of the users on exchange and change of the labeled data set are met.

According to a sixth possible implementation manner of the third aspect, in a seventh possible implementation manner of the data processing method, the information of the object to be requested further includes first authentication information, the request information further includes second authentication information, and the method further includes: when the information of the object to be requested is received, determining whether the first user is credible according to the first identity verification information; and when the request information is received, determining whether the second user is credible according to the second identity verification information.

According to the embodiment of the application, when the information of the object to be requested and the request information are received, whether the user is credible or not is determined according to the identity authentication information, the problem that the users are not credible mutually can be solved, the safe sharing of the labeled data set is realized, and the process can be supervised and traceable.

According to the third aspect, at least one of the first, second, third, fourth, fifth, sixth and seventh possible implementation manners of the third aspect, in an eighth possible implementation manner of the data processing method, the method further includes: receiving demand information sent by the second user, wherein the demand information represents a labeled data set required by the second user; notifying the first user of the demand information.

According to the embodiment of the application, the demand information sent by the second user is received, and the first user is informed of the demand information, so that the exchange of the labeled data sets among users is more convenient, and the personalized customization of the labeled data sets is supported.

In a ninth possible implementation form of the data processing method according to the first possible implementation form of the third aspect, the usage policy of the target annotation data set is determined according to an indication of the second user.

According to the embodiment of the application, the second user can provide the use strategy of the labeled data set or the algorithm model according to the requirement of the second user, and the flexibility of the use strategy is improved.

According to a sixth possible implementation manner of the third aspect, in a tenth possible implementation manner of the data processing method, the request information further includes: the use strategy of the marked data set indicated by the request information is used for determining the condition of destroying the marked data set indicated by the request information; and the use strategy of the target annotation data set is determined according to the use strategy included in the request information.

According to the embodiment of the application, the request information further comprises the use strategy of the labeled data set indicated by the request information, and the use strategy of the target labeled data set is determined according to the use strategy included by the request information, so that the target labeled data set can be incinerated after use, and the target labeled data set is prevented from being secondarily resaled and abused. The second user can put forward the use strategy of the target marking data set according to the requirement of the second user, and the flexibility of the use strategy is improved.

In a twelfth possible implementation form of the data processing method according to the third aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth and eleventh possible implementation forms of the third aspect, the algorithmic model is a machine learning model.

According to the method and the device, the machine learning model can be safely used by a user, abuse of source data is prevented, and the trained algorithm model can be obtained by using the self machine learning model by the user.

In a twelfth possible implementation form of the data processing method according to the third aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh or eighth, ninth and tenth possible implementation forms of the third aspect, the sandbox is supervised by a blockchain, and the first response information is used to determine the intelligent contracts made by the first user and the second user.

According to the embodiment of the application, by means of the block chain technology, the uplink data of the users can be not tampered, the problem of distrustment among the users is solved, the safe sharing of the labeled data set is realized, the process is transparent to the users, the source can be controlled and traced, and the transaction information can not be tampered.

In a fourth aspect, an embodiment of the present application provides a data processing apparatus, including: a first receiving module, configured to receive first response information of a first user, where the first user provides at least one annotation data set, the first response information includes a storage address of a target object determined by the first user, and the target object includes a target annotation data set requested to be used by a third user; the second receiving module is used for receiving second response information of a second user, the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model requested to be used by a third user; the first obtaining module is used for obtaining the target annotation data set by the sandbox according to the first response information; the second obtaining module is used for obtaining the target algorithm model by the sandbox according to the second response information; and the first evaluation module is used for evaluating a target algorithm model by using the target labeling data set to obtain an evaluation result, wherein the evaluation result is used for being provided for a third user, and the first user, the second user and the third user forbid accessing the target labeling data set and the target algorithm model in the sandbox.

In a first possible implementation form of the data processing apparatus according to the fourth aspect, the first user and the second user are the same user.

In a second possible implementation manner of the data processing apparatus according to the fourth aspect, the first response information further includes: the use strategy of the target labeling data set is used for determining the destroy condition of the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the device further comprises: the first destruction module is used for destroying the target labeling data set acquired by the sandbox according to the use strategy of the target labeling data set; and the second destruction module is used for destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

According to a second possible implementation manner of the fourth aspect, in a third possible implementation manner of the data processing apparatus, the first destruction module further includes: the first destruction submodule is used for destroying the target labeling data set under the condition that the number of times of using the target labeling data set reaches a preset first threshold value; and/or a second destruction submodule, configured to destroy the target annotation data set when a usage time for which the target annotation data set is used reaches a predetermined second threshold; the second destruction module, comprising: a third destruction submodule, configured to destroy the target algorithm model when the number of times that the target algorithm model is used reaches a predetermined third threshold; and/or a fourth destroy submodule for destroying the target algorithm model if the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

According to the fourth aspect and at least one of the first, second and third possible implementation manners of the fourth aspect, in a fourth possible implementation manner of the data processing apparatus, the first obtaining module includes: the first obtaining submodule is used for the sandbox to obtain the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set; the second acquisition module includes: and the second obtaining submodule is used for obtaining the target algorithm model stored in the cloud storage unit by the sandbox according to the storage address of the target algorithm model.

In a fifth possible implementation form of the data processing apparatus according to the fourth aspect as well as at least one of the first, second, third and fourth possible implementation forms of the fourth aspect, the apparatus further comprises: and the first conversion module is used for converting the obtained target algorithm model by the sandbox according to the conversion algorithm model so as to match the compiling environment of the sandbox.

In a sixth possible implementation manner of the data processing apparatus according to the fourth aspect as well as at least one of the first, second, third, fourth and fifth possible implementation manners of the fourth aspect, the apparatus further includes: a first determining module, configured to determine whether the first user is authentic according to first authentication information, where the first authentication information is provided when the first user provides information of at least one annotation data set; the second determining module is used for determining whether the second user is credible according to second identity verification information, and the second identity verification information is provided when the second user provides information of at least one algorithm model; a third determining module, configured to determine whether the third user is trusted according to third authentication information, where the third authentication information is provided when the third user requests to use the target annotation data set and the target algorithm model.

In a seventh possible implementation manner of the data processing apparatus according to the fourth aspect as well as at least one of the first, second, third, fourth, fifth and sixth possible implementation manners of the fourth aspect, the apparatus further includes: a fifth receiving module, configured to receive information of a first object to be requested sent by the first user, where the first object to be requested includes at least one labeled data set provided by the first user; a sixth receiving module, configured to receive information of a second object to be requested sent by the second user, where the second object to be requested includes at least one algorithm model provided by the second user; a first notification module, configured to notify the third user of information of the first object to be requested and information of the second object to be requested; a seventh receiving module, configured to receive first request information sent by the third user, where the first request information indicates an annotated data set in the first object to be requested, which is requested to be used by the third user; an eighth receiving module, configured to receive second request information sent by the third user, where the second request information indicates an algorithm model in the second object to be requested, where the third user requests to use the algorithm model; a first sending module, configured to send the first request information to the first user, where the first request information is used to determine the first response information; and a second sending module, configured to send the second request information to the second user, where the second request information is used to determine the second response information.

In an eighth possible implementation manner of the data processing apparatus according to the seventh possible implementation manner of the fourth aspect, the information of the first object to be requested further includes first authentication information, the information of the second object to be requested further includes second authentication information, and the first request information and the second request information further include third authentication information, the apparatus further includes: a sixth determining module, configured to determine, when information of the first object to be requested is received, whether the first user is trusted according to the first authentication information; a seventh determining module, configured to determine, when information of the second object to be requested is received, whether the second user is trusted according to the second authentication information; an eighth determining module, configured to determine, when the first request information is received, whether the third user is trusted according to the third authentication information; and the ninth determining module is used for determining whether the third user is credible according to the third identity authentication information when the second request information is received.

In a ninth possible implementation form of the data processing apparatus according to the fourth aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh and eighth possible implementation forms of the fourth aspect, the apparatus further includes: a ninth receiving module, configured to receive requirement information sent by the third user, where the requirement information represents a labeled data set and/or an algorithm model required by the third user; and the second notification module is used for notifying the first user and/or the second user of the demand information.

In a tenth possible implementation manner of the data processing apparatus according to the seventh possible implementation manner of the fourth aspect, the first request information further includes a usage policy of the annotation data set indicated by the first request information, where the usage policy is used to determine a condition for destroying the annotation data set indicated by the first request information; the second request information further comprises a use strategy of the algorithm model indicated by the second request information, and the use strategy is used for determining the condition of destroying the algorithm model indicated by the second request information; the usage policy of the target annotation data set is determined according to the usage policy included in the first request message, and the usage policy of the target algorithm model is determined according to the usage policy included in the second request message.

In a tenth possible implementation form of the data processing apparatus according to the ninth possible implementation form of the fourth aspect, the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to the instruction of the third user.

In a twelfth possible implementation form of the data processing device the algorithmic model is a machine learning model according to the fourth aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth and eleventh possible implementation forms of the fourth aspect.

In a thirteenth possible implementation form of the data processing apparatus according to the fourth aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, eleventh and twelfth possible implementation forms of the fourth aspect, the sandbox is supervised by a blockchain, the first response information is used to determine smart contracts made by the first user and the third user, and the second response information is used to determine smart contracts made by the second user and the third user.

In a fifth aspect, an embodiment of the present application provides a data processing apparatus, including: a tenth receiving module, configured to receive first response information of a first user, where the first user provides at least one annotation data set, the first response information includes a storage address of a target object determined by the first user, and the target object includes the target annotation data set provided by the first user; an eleventh receiving module, configured to receive second response information of a second user, where the second user provides at least one algorithm model, and the second response information includes a storage address of a target object determined by the second user, where the target object includes a target algorithm model requested to be used by the first user; a fifth obtaining module, configured to obtain, by the sandbox, the target annotation data set according to the first response information; the sixth obtaining module is used for obtaining the target algorithm model by the sandbox according to the second response information; and the second evaluation module is used for evaluating a target algorithm model by using the target labeling data set to obtain an evaluation result, the evaluation result is provided for a first user, and the first user and the second user forbid accessing the target labeling data set and the target algorithm model in the sandbox.

In a first possible implementation manner of the data processing apparatus according to the fifth aspect, the first response information further includes: the use strategy of the target labeling data set is used for determining the destroy condition of the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the device further comprises: a fifth destruction module, configured to destroy the target annotation data set obtained by the sandbox according to a usage policy of the target annotation data set; and the sixth destruction module is used for destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

In a second possible implementation manner of the data processing apparatus according to the first possible implementation manner of the fifth aspect, the fifth destruction module includes: a ninth destruction sub-module, configured to destroy the target annotation data set when the number of times that the target annotation data set is used reaches a predetermined first threshold; and/or a tenth destruction submodule, configured to destroy the target annotation data set when a usage time for which the target annotation data set is used reaches a predetermined second threshold; the sixth destruction module comprises: an eleventh destruction submodule for destroying the target algorithm model when the number of times the target algorithm model is used reaches a predetermined third threshold; and/or a twelfth destruction submodule for destroying the target algorithm model if the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

According to the fifth aspect and at least one of the first and second possible implementation manners of the fifth aspect, in a third possible implementation manner of the data processing apparatus, the fifth obtaining module includes: a fifth obtaining submodule, configured to obtain, by the sandbox, the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set; the sixth obtaining module includes: and the sixth obtaining submodule is used for obtaining the target algorithm model stored in the cloud storage unit by the sandbox according to the storage address of the target algorithm model.

In a fourth possible implementation form of the data processing apparatus according to the fifth aspect as well as at least one of the first, second and third possible implementation forms of the fifth aspect, the apparatus further comprises: and the second conversion module is used for converting the obtained target labeling data set and/or the target algorithm model by the sandbox according to a conversion algorithm model so as to match the compiling environment of the sandbox.

In a fifth possible implementation form of the data processing apparatus according to the fifth aspect as well as at least one of the first, second, third and fourth possible implementation forms of the fifth aspect, the apparatus further comprises: a fourteenth determining module, configured to determine whether the first user is authentic according to first authentication information, where the first authentication information is provided when the first user requests to use the target algorithm model; and a fifteenth determining module, configured to determine whether the second user is trusted according to second authentication information, where the second authentication information is provided when the second user provides information of at least one algorithm model.

In a sixth possible implementation form of the data processing apparatus according to the fifth aspect as well as at least one of the first, second, third, fourth and fifth possible implementation forms of the fifth aspect, the apparatus further comprises: a twelfth receiving module, configured to receive information of an object to be requested sent by the second user, where the object to be requested includes at least one algorithm model provided by the second user; a third notifying module, configured to notify the first user of information of the object to be requested; a thirteenth receiving module, configured to receive request information sent by the first user, where the request information indicates an algorithm model in the object to be requested, which the first user requests to use; and a third sending module, configured to send the request information to the second user, where the request information is used to determine the second response information.

In a seventh possible implementation manner of the data processing apparatus according to the sixth possible implementation manner of the fifth aspect, the information of the object to be requested further includes first authentication information, the request information further includes second authentication information, and the apparatus further includes: a tenth determining module, configured to determine, when receiving the information of the object to be requested, whether the second user is trusted according to the first identity verification information; and the eleventh determining module is used for determining whether the first user is credible according to the second identity authentication information when the request information is received.

In an eighth possible implementation manner of the data processing apparatus according to the fifth aspect as well as at least one of the first, second, third, fourth, fifth, sixth and seventh possible implementation manners of the fifth aspect, the apparatus further includes: a fourteenth receiving module, configured to receive requirement information sent by the first user, where the requirement information indicates an algorithm model required by the first user; and the fourth notification module is used for notifying the second user of the requirement information.

In a ninth possible implementation manner of the data processing apparatus according to the sixth possible implementation manner of the fifth aspect, the request information further includes: the use strategy of the algorithm model indicated by the request information is used for determining the condition of destroying the algorithm model indicated by the request information; the use strategy of the target algorithm model is determined according to the use strategy included in the request information.

In a tenth possible implementation form of the data processing apparatus according to the first possible implementation form of the fifth aspect, the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to the indication of the first user.

In an eleventh possible implementation form of the data processing device according to the fifth aspect as such or according to at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth and tenth possible implementation forms of the fifth aspect, the algorithmic model is a machine learning model.

In a twelfth possible implementation form of the data processing apparatus according to the fifth aspect as such or according to at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth and eleventh possible implementation forms of the fifth aspect, the sandbox is supervised by a blockchain, and the second response information is used to determine smart contracts made by the first user and the second user.

In a sixth aspect, an embodiment of the present application provides a data processing apparatus, including: a third receiving module, configured to receive first response information of a first user, where the first user provides at least one tagged data set, where the first response information includes a storage address of a target object determined by the first user, and the target object includes a target tagged data set requested to be used by a second user; a fourth receiving module, configured to receive second response information of a second user, where the second user provides at least one algorithm model, the second response information includes a storage address of a target object determined by the second user, and the target object includes a target algorithm model provided by the second user; the third obtaining module is used for obtaining the target annotation data set by the sandbox according to the first response information; the fourth obtaining module is used for obtaining the target algorithm model by the sandbox according to the second response information; and the training module is used for training the target algorithm model by using the target labeling data set to obtain a trained algorithm model, wherein the trained algorithm model is provided for the second user, and the first user and the second user forbid accessing the target labeling data set and the target algorithm model in the sandbox.

In a first possible implementation manner of the data processing apparatus according to the sixth aspect, the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the device further comprises: the third destroying module is used for destroying the target labeling data set acquired by the sandbox according to the use strategy of the target labeling data set; and the fourth destroying module is used for destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

In a second possible implementation manner of the data processing apparatus according to the first possible implementation manner of the sixth aspect, the third destruction module includes: a fifth destruction submodule, configured to destroy the target annotation data set when the number of times that the target annotation data set is used reaches a predetermined first threshold; and/or a sixth destruction submodule, configured to destroy the target annotation data set when a usage time for which the target annotation data set is used reaches a predetermined second threshold; the fourth destruct module comprises: a seventh destruction submodule, configured to destroy the target algorithm model when the number of times that the target algorithm model is used reaches a predetermined third threshold; and/or an eighth destruction submodule for destroying the target algorithm model if the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

In a third possible implementation manner of the data processing apparatus according to the sixth aspect as well as at least one of the first and second possible implementation manners of the sixth aspect, the third obtaining module includes: the third obtaining submodule is used for obtaining the target annotation data set stored in the cloud storage unit by the sandbox according to the storage address of the target annotation data set; the fourth obtaining module comprises: and the fourth obtaining submodule is used for obtaining the target algorithm model stored in the cloud storage unit by the sandbox according to the storage address of the target algorithm model.

In a fourth possible implementation form of the data processing apparatus according to the sixth aspect as well as at least one of the first, second and third possible implementation forms of the sixth aspect, the apparatus further comprises: and the third conversion module is used for converting the obtained target labeling data set and/or the target algorithm model by the sandbox according to a conversion algorithm model so as to match the compiling environment of the sandbox.

In a fifth possible implementation form of the data processing apparatus according to the sixth aspect as well as at least one of the first, second, third and fourth possible implementation forms of the sixth aspect, the apparatus further comprises: a fourth determining module, configured to determine whether the first user is trusted according to first authentication information, where the first authentication information is provided when the first user provides information of at least one annotation data set; and the fifth determining module is used for determining whether the second user is credible according to second authentication information, and the second authentication information is provided when the second user requests to use the target annotation data set.

In a sixth possible implementation manner of the data processing apparatus according to the sixth aspect as well as at least one of the first, second, third, fourth and fifth possible implementation manners of the sixth aspect, the apparatus further includes: a fifteenth receiving module, configured to receive information of an object to be requested sent by the first user, where the object to be requested includes at least one annotation data set provided by the first user; a fifth notification module, configured to notify the second user of information of the object to be requested; a sixteenth receiving module, configured to receive request information sent by the second user, where the request information indicates a tagged data set in the object to be requested, where the tagged data set is requested to be used by the second user; a fourth sending module, configured to send the request information to the first user, where the request information is used to determine the first response information.

In a seventh possible implementation manner of the data processing apparatus according to the sixth possible implementation manner of the sixth aspect, the information of the object to be requested further includes first authentication information, the request information further includes second authentication information, and the apparatus further includes: a twelfth determining module, configured to determine, when receiving the information of the object to be requested, whether the first user is trusted according to the first authentication information; and the thirteenth determining module is used for determining whether the second user is credible according to the second identity authentication information when the request information is received.

In an eighth possible implementation manner of the data processing apparatus according to the sixth aspect as well as at least one of the first, second, third, fourth, fifth, sixth and seventh possible implementation manners of the sixth aspect, the apparatus further includes: a seventeenth receiving module, configured to receive requirement information sent by the second user, where the requirement information indicates a labeled data set required by the second user; a sixth notification module, configured to notify the first user of the demand information.

In a ninth possible implementation manner of the data processing apparatus according to the fifth possible implementation manner of the sixth aspect, the request information further includes: the use strategy of the marked data set indicated by the request information is used for determining the condition of destroying the marked data set indicated by the request information; and the use strategy of the target annotation data set is determined according to the use strategy included in the request information.

In a tenth possible implementation form of the data processing apparatus according to the first possible implementation form of the sixth aspect, the usage policy of the target annotation data set is determined according to the indication of the second user.

In an eleventh possible implementation form of the data processing device according to the sixth aspect as such or according to at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth and tenth possible implementation forms of the sixth aspect, the algorithmic model is a machine learning model.

In a twelfth possible implementation form of the data processing apparatus according to the sixth aspect as well as at least one of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth and eleventh possible implementation forms of the sixth aspect, the sandbox is supervised by a blockchain, and the first response information is used to determine the smart contracts made by the first user and the second user.

In a seventh aspect, an embodiment of the present application provides a data processing apparatus, including: a processor; a memory for storing processor-executable instructions; the processor is configured to implement, when executing the instruction, a data processing method of one or more of the first aspect or multiple possible implementations of the first aspect, or a data processing method of one or more of the second aspect or multiple possible implementations of the second aspect, or a data set processing method of one or more of the third aspect or multiple possible implementations of the third aspect.

In an eighth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium, on which computer program instructions are stored, which, when executed by a processor, implement a data processing method of the first aspect or one or more of the multiple possible implementations of the first aspect, or implement a data processing method of the second aspect or one or more of the multiple possible implementations of the second aspect, or implement a data processing method of the third aspect or one or more of the multiple possible implementations of the third aspect.

In a ninth aspect, an embodiment of the present application provides a terminal device, where the terminal device may execute the data processing method in one or more of the foregoing first aspect or multiple possible implementations of the first aspect, or implement the data processing method in one or more of the foregoing second aspect or multiple possible implementations of the second aspect, or implement the data processing method in one or more of the foregoing third aspect or multiple possible implementations of the third aspect.

In a tenth aspect, an embodiment of the present application provides a computer program product, which includes computer readable code or a non-volatile computer readable storage medium carrying computer readable code, and when the computer readable code runs in an electronic device, a processor in the electronic device performs a data processing method of one or more of the first aspect or the multiple possible implementations of the first aspect, or performs a data processing method of one or more of the second aspect or the multiple possible implementations of the second aspect, or performs a data processing method of one or more of the third aspect or the multiple possible implementations of the third aspect.

These and other aspects of the present application will be more readily apparent in the following description of the embodiment(s).

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the application and, together with the description, serve to explain the principles of the application.

Fig. 1 shows a schematic diagram of an application scenario according to an embodiment of the present application.

FIG. 2 shows a block diagram of a data processing device according to an embodiment of the present application.

FIG. 3 shows a block diagram of a sandbox according to an embodiment of the present application.

FIG. 4 shows a flow diagram of a data processing method according to an embodiment of the present application.

FIG. 5 shows a flow diagram of a data preparation phase, a security computation phase, and a data usage phase of a data processing method according to an embodiment of the application.

FIG. 6 shows a flow diagram of user interaction with a data processing device according to an embodiment of the application.

FIG. 7 shows a schematic diagram of utilizing a blockchain technique according to an embodiment of the present application.

FIG. 8 shows a flow diagram of a data processing method according to an embodiment of the present application.

FIG. 9 shows a flow diagram of a data processing method according to an embodiment of the present application.

FIG. 10 shows a flow diagram of a data processing method according to an embodiment of the present application.

FIG. 11 shows a flow diagram of a data processing method according to an embodiment of the present application.

FIG. 12 shows a flow diagram of a data processing method according to an embodiment of the present application.

FIG. 13 shows a block diagram of a data processing device according to an embodiment of the present application.

FIG. 14 shows a block diagram of a data processing device according to an embodiment of the present application.

FIG. 15 shows a block diagram of a data processing device according to an embodiment of the present application.

Fig. 16 shows a block diagram of a data processing apparatus according to an embodiment of the present application.

Detailed Description

Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present application. It will be understood by those skilled in the art that the present application may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present application.

Data transactions are typically conducted in the prior art via the e-commerce platform of B2B. The roles of main participants of the existing e-commerce platform include: commercial tenant, user, network platform, authentication center, payment system etc. through: the processes of information release, inquiry of a buyer, payment and commodity acquisition are carried out. Such a platform can support common goods well. However, as for the data set and the algorithm, the data set and the algorithm have particularity as transaction products, and the problems are that the data set and the algorithm are used as digital commodities and are acquired once and used for unlimited times, and the problems of secondary sale, abuse and the like after the users buy cannot be avoided in the prior art; secondly, the volume of the data set is huge and is measured by T level, the prior art basically supports the capacity of the volume data, and users cannot directly download and share the volume data; moreover, the prior art is difficult to trace the source and monitor the transaction of the digital commodity, and the problem of potential mutual distrust of both transaction parties cannot be solved. The existence of the problems causes related enterprises to have strong appeal for sharing and realizing the data sets and related algorithms, but the problems cannot be solved and can only be hoped.

In order to solve the technical problem, the data processing method provided by the application obtains the target algorithm model and the target label data set through the sandbox according to response information provided by all parties or users of the label data set or the algorithm model, evaluates or trains the target algorithm model by using the target label data set to obtain an evaluation result or trained algorithm model to provide the evaluation result or trained algorithm model for the users, ensures that the label data set and the algorithm model are not sold and abused for the second time by enabling all the parties and the users to forbid access to the target label data set and the target algorithm model in the sandbox, obtains and processes data by using the sandbox as a third party, has a traceable and controllable process, and solves the potential distrust problem of the users and all the parties. In addition, the user does not directly acquire the labeled data set and the algorithm model, so that the problem that the user cannot download and use a large amount of data is solved, and the labeled data set and the algorithm model can be shared for many times by uploading at one time.

Fig. 1 shows a schematic diagram of an application scenario according to an embodiment of the present application. In a possible implementation manner, the data processing method provided by the embodiment of the application can be applied to an application scenario in which a user performs a transaction of a labeled data set and an algorithm model. As shown in fig. 1, the user may include an auto-pilot, a parts supplier, an automobile manufacturer, etc., and the present application is not limited to the type of user. In a possible implementation manner, according to the data processing method provided by the embodiment of the application, a user can implement a transaction of the labeled data set and the algorithm model through the trusted area. The labeled data set may include any type of data set including labeled data, such as a labeled data set collected in a road test, simulation, and other scenarios and used for training a machine learning model. The algorithmic model may comprise a set of instructions or programs to implement any type of algorithm (e.g., a machine learning algorithm), and the type of annotation data set and algorithmic model is not a limitation of the present application.

The following description will be given taking as an example an application scenario in which an autopilot, a part supplier, and an automobile manufacturer perform transactions of a labeled data set and an algorithm model, where the autopilot and the part supplier serve as examples of all parties who label the data set and the algorithm model, and the automobile manufacturer serves as an example of a user who labels the data set and the algorithm model. Those skilled in the art should understand that the embodiments of the present application are not limited to such application scenarios.

In one possible implementation, the automatic driving enterprise may publish an algorithm model (e.g., a machine learning model) on a trusted area, the component supplier may publish a related labeled data set (e.g., a pedestrian labeled data set, a road-based labeled data set, etc.) on the trusted area, the automobile manufacturer that needs to optimize the automobile perception capability may purchase the algorithm model provided by the automatic driving enterprise and the labeled data set provided by the component supplier respectively on the trusted area, and specify a labeled data set and an algorithm model that need to be analyzed (which may include the labeled data set and/or the algorithm model purchased by the automobile manufacturer, and may also include the labeled data set and the algorithm model provided by the automobile manufacturer itself), and the trusted area may automatically perform analysis and calculation using the labeled data set according to the labeled data set and the algorithm model specified by the automobile manufacturer (e.g., evaluate the algorithm model or advance the algorithm model) Line training), generating an analysis result (e.g., an evaluation result of the algorithm model or the trained algorithm model); for an automobile manufacturer, after the calculation is completed in the trusted area, the analysis calculation result can be obtained in the trusted area, but the purchased labeled data set and/or the source data of the algorithm model cannot be downloaded; the marked data set and/or the algorithm model purchased by the automobile manufacturer can be destroyed in a trusted area according to the use strategy determined in the transaction process, so that secondary sale and abuse of resources are prevented. The automatic driving enterprises, the part suppliers and the automobile manufacturers can also carry out identity verification in a credible area through an authentication center, so that credibility among transaction entities is ensured, and the transactions are safe and reliable.

FIG. 2 shows a block diagram of a data processing device according to an embodiment of the present application. As shown in fig. 2, in the embodiment of the present application, the data processing apparatus (corresponding to the trusted area in the above) may include a sandbox (or referred to as a sandbox system), an Object Based Storage (OBS) access channel, and a network platform (or referred to as a cyberspace, a network infrastructure). Wherein a sandbox may be used for security analysis calculations based on resources from outside the sandbox, FIG. 3 shows a block diagram of a sandbox according to an embodiment of the present application; the OBS access channel can be used for accessing a large amount of safe storage OBSs and acquiring resources; the network platform can be used for carrying out identity authentication of users, information release of the users, transaction among the users and the like.

As shown in fig. 3, the sandbox may include a target annotation data set module, a conversion algorithm model module, a target algorithm model module, a usage policy module, a computing platform, a data search engine, and an Information Technology (IT) infrastructure. The target annotation data set module can be used for storing a target annotation data set provided by a user; the target algorithm model module can be used for storing a target algorithm model provided by a user; the conversion algorithm model module can be used for storing a conversion algorithm model of the sandbox, and the conversion algorithm model can be used for converting the target algorithm model to be matched with the compiling environment of the sandbox; the use strategy module can be used for storing a target labeling data set and a use strategy of the target algorithm model; the computing platform can be used for carrying out analysis and computation according to the target algorithm model, the conversion algorithm model and the target labeling data set and obtaining an analysis result; the data search engine can be used for searching the labeled data set and/or the algorithm model for the OBS through the OBS access channel so as to obtain a target labeled data set and a target algorithm model; IT infrastructure can be used to make sandboxes.

In one possible implementation, the user may conduct transactions that label the dataset and/or the algorithmic model, and the user may be divided into all parties and a user party. All parties have a labeled data set and/or an algorithm model, and the labeled data set and/or the algorithm model are core assets of the all parties; and the user performs analysis processing by using the annotation data set and the algorithm model, and specifies the annotation data set and/or the algorithm model provided by all the users to use, but does not acquire the source data of the annotation data set and/or the algorithm model of all the users. All parties can trade with a plurality of users at the same time, and the users can also trade with a plurality of all parties at the same time. All parties can publish information of the sold labeling data sets and/or algorithm models on the network platform, a user can publish request information of the labeling data sets and/or algorithm models on the network platform, all parties and the user can conduct transactions on the network platform, the user can conduct calculation and analysis in a sandbox by means of the purchased labeling data sets and/or algorithm models and own labeling data sets and/or algorithm models to obtain analysis results, the sandbox informs all parties of the analysis results through the network platform, and all parties publish the analysis results for the user. The sandbox can also destroy the marked data set and/or the algorithm model regularly or repeatedly according to the use strategy, so that secondary sale and abuse of the marked data set and/or the algorithm model are prevented.

The embodiments of the present application are not limited to the types of the labeled data sets and the algorithm models, and several exemplary transaction scenarios for labeling the data sets and the algorithm models are described below as examples, and it should be understood by those skilled in the art that the embodiments of the present application are not limited to such application scenarios.

In an exemplary application scenario, user a may publish an algorithm model and/or an annotation data set on a network platform and upload published source data to an OBS, user B may make a request for analysis and calculation on the network platform, if the request information includes a request for using the algorithm model and/or the annotation data set of user a (which may also include a plurality of users, such as the algorithm model and/or the annotation data set of user C and user D), user B may purchase the algorithm model and/or the annotation data set of user a (which may also purchase from a plurality of users), user a may see a transaction request of user B on the network platform, and after confirmation by user a, a transaction is established, a sandbox may download the relevant algorithm model and/or annotation data set from the OBS according to response information of user a, automatically perform analysis and calculation, and the obtained algorithm model and/or the obtained annotation data set can be destroyed according to the use strategy confirmed by both parties during transaction after the analysis and calculation are completed. And the user B can download the analysis result from the network platform after the analysis calculation is completed.

In this application scenario, fig. 4 shows a flowchart of a data processing method according to an embodiment of the present application. As shown in fig. 4, the data processing method includes:

step S10, data preparation phase.

Before the data preparation stage, all parties upload the labeled data set and/or the algorithm model to the OBS, and specific addresses accessed by the labeled data set and/or the algorithm model are obtained.

For example, if an owner prepares to sell an algorithm model for implementing a pedestrian labeling algorithm on a network platform, the algorithm script of the algorithm model needs to be uploaded to the OBS for storage, and a specific storage address of source data on the OBS is obtained, so that a subsequent sandbox can obtain the algorithm model from the OBS.

Step S20, a security calculation phase.

The method comprises the steps that all parties and users can determine to conduct transactions on a network platform, the sandbox waits for analysis and calculation according to algorithm models and labeled data sets from all parties and users, analysis results are obtained, and download addresses of the analysis results are provided for all parties through the network platform.

For example, a user may make a request for evaluating an algorithm model for implementing a pedestrian labeling algorithm issued by an owner according to a self-owned pedestrian labeling data set on a network platform, purchase the algorithm model to the owner, confirm a transaction by the owner, determine response information by the owner, obtain the algorithm model and the pedestrian labeling data set according to the response information by a sandbox, calculate to obtain an evaluation result of the algorithm model, and notify the owner of a download address of the evaluation result through the network platform.

Step S30, data use phase.

And all parties can issue the download address of the analysis result to the user on the network platform, and the user downloads the analysis result from the network platform to complete the transaction.

For example, all parties can encrypt the download address of the analysis result, and after the user acquires the download authority on the network platform, the user downloads the analysis result according to the download address by using the download authority on the network platform.

FIG. 5 shows a flow diagram of a data preparation phase, a security computation phase, and a data usage phase of a data processing method according to an embodiment of the application. As shown in fig. 5, the interactive modules participating in the flow of the data preparation phase, the secure computation phase, and the data usage phase may include an owner, a network platform, an OBS access channel, a sandbox, and a user.

As shown in fig. 5, the flow of the data preparation phase may include:

and S101, all parties issue information of the annotation data set and/or the algorithm model to be traded on the network platform.

The tagged data set information issued by the owner may include digest data (e.g., including a digest of data content in the data set), data description information (e.g., including a subject, a type, and the like of the data), a data storage address (e.g., a storage address of the tagged data set on the OBS), a use policy (e.g., the tagged data set or the algorithm model is destroyed by time or destroyed by time), identity verification information, and the algorithm model information may include algorithm description information, an algorithm storage address, a use policy, identity verification information, and the like of the algorithm model.

Wherein, the identity verification information can be provided by a third party certification center on the network platform for proving the credibility of all parties.

For example, if the owner wants to sell the pedestrian labeling dataset, the owner may publish, on the network platform, related information of the pedestrian labeling dataset, which may include an abstract of the content of the pedestrian labeling dataset, data description information (for example, the subject is the pedestrian labeling dataset), a specific access address obtained after the owner has uploaded the data of the dataset source to the OBS, a destruction policy of the dataset (for example, destruction after one-time use or three days after transaction confirmation), certification information of the owner (for example, enterprise qualification certification), and the like; if the owner wants to sell the algorithm model of the machine learning algorithm, the owner can publish the relevant information of the machine learning algorithm on the network platform, which may include algorithm description (for example, the algorithm labels a data set for pedestrians), relevant parameter information of the algorithm, a compiling environment on which the algorithm depends, a specific access address obtained after the owner uploads an algorithm script to the OBS, identity verification information of the owner (for example, enterprise qualification certification of the owner), and the like.

The method comprises the steps that all parties can apply for identity verification information to a third-party authentication center on a network platform before issuing information of a data set and/or an algorithm model to be traded.

And S102, the user issues the demand information of the annotation data set and/or the algorithm model on the network platform.

The requirement information issued by the user can comprise the identity verification information of the user, and the identity verification information can be provided by a third-party authentication center on the network platform.

For example, if a user has a need for a pedestrian labeling data set, the user may publish requirement information on the network platform, where the requirement information may include a type or a subject of the needed labeling data set (e.g., a data set that needs a pedestrian labeling subject), additional requirements on the data set (e.g., a requirement on the amount of the labeled data set), and authentication information of the user (e.g., an enterprise qualification certificate of the user).

Step S103, the network platform broadcasts the release events of the owner and the user.

The network platform can broadcast the information of the annotation data set and/or the algorithm model to be traded and the requirement information of a user. It should be noted that all parties and users may choose to publish or not publish relevant information according to needs, for example, all parties may publish information of the annotation data set and/or the algorithm model to be traded in a single aspect, and users may choose a needed annotation data set and/or algorithm model directly according to information published by all parties and make requests without publishing their own needs.

As shown in fig. 5, the flow of the secure computation phase may include:

step S201, the user makes a request to the network platform.

Wherein the request may include at least one of a target annotation data set to use, a target algorithm model to use, and a usage policy.

Wherein the identity verification information of the user can also be included in the request, and the identity verification information can be provided by a third party authentication center on the network platform.

Wherein, the user can choose to purchase the annotation data set and/or algorithm model of a plurality of owners for use, and the user can choose to use the own annotation data set and/or algorithm model.

Wherein, the target annotation data set can comprise an annotation data set transacted with one or more owners or an own annotation data set of a user; the target algorithmic model may comprise an algorithmic model of a transaction with one or more owners, or an algorithmic model owned by the user.

For example, if a user wants to evaluate an algorithm model sold by an owner on a network platform using his own pedestrian tagging data set, the request from the user to the owner may include the algorithm model of the owner (i.e., the target algorithm model used) and a destruction policy for the algorithm model (i.e., the usage policy).

The usage policy of the algorithm model to be purchased by the user can be determined by referring to the usage policy in the algorithm model information issued by all the users.

And S202, all parties repeat the request on the network platform, determine response information and apply for starting the sandbox.

The response information determined by the owner may include a transaction order number, a specific access address of a related resource (including the target annotation dataset and the target algorithm model, the dataset and/or the algorithm model provided by the owner), a usage policy, and the like.

For example, if the request of the user involves purchasing resources of multiple owners, the network platform may send multiple requests to the multiple owners, respectively, and the multiple owners may confirm the corresponding requests, respectively, and generate corresponding response information after the multiple owners confirm the requests, respectively.

In the case that the target annotation data set and/or the target algorithm model proposed by the user includes the own annotation data set and/or algorithm model, the user may also provide the network platform with response information, which may include a transaction order number, and a specific access address, a use policy, etc. of the user's own annotation data set and/or algorithm model. The response information of the user may further include an analysis requirement of the user on the target annotation data set and/or the target algorithm model, for example, training or evaluating the target algorithm model by using the target annotation data set.

The using strategy of the user for the own label data set and/or algorithm model can be self-defined (for example, the sandbox is destroyed after being used once or the sandbox is destroyed after being used for one day);

step S203, the network platform sends the response information to the sandbox.

And step S204, the sandbox acquires related resources from the OBS through the OBS access channel, and executes a calculation analysis task to obtain an analysis result.

The sandbox can obtain a specific access address of the target labeling data set (such as a specific access address of the pedestrian labeling data set and the algorithm model) according to the response information, searches the target labeling data set through an OBS access channel through a data search engine, downloads the target labeling data set and stores the target labeling data set in a target labeling data set module; the sandbox can also obtain a specific access address of the target algorithm model in the OBS according to the response information, download the specific access address and store the specific access address in the target algorithm model module. After downloading, the sandbox can use the computing platform to import the relevant algorithm models from the target algorithm model module and the conversion algorithm model module, and the analysis result is obtained by computing with the target labeling data set.

For example, the algorithm model may be a machine learning model, and training or evaluation of the model may be achieved by importing the algorithm model and the target labeling data set on the computing platform and inputting the target labeling data set into the machine learning model.

The conversion algorithm model in the conversion algorithm model module is used for adapting to a compiling environment of a sandbox (e.g., a computing platform in the sandbox) to perform computing under the condition that a compiling environment on which each algorithm implemented in the target algorithm model depends is not consistent with the compiling environment of the sandbox, and may be implemented in an open neural network exchange (ONNX) format, for example. The present application does not limit the type of compilation environment for a sandbox.

After the calculation is performed, the sandbox may destroy the target algorithm model and the target labeling data set according to the response information by using the policy module, and the destruction policy may be the number of times of using the target algorithm model and/or the target labeling data set or the time of using the target algorithm model and/or the target labeling data set according to the sandbox.

And S205, uploading the sandbox encrypted evaluation result or the trained algorithm model to a network platform.

And step S206, the sandbox informs all parties through the network platform, the calculation analysis process is finished, and the sandbox is closed.

The sandbox can send the evaluation result or the download address of the trained algorithm model to all parties through the network platform.

As shown in fig. 5, the flow of the data usage phase may include:

Step S301, all parties issue evaluation results or download addresses of the trained algorithm models on the network platform.

Wherein the download address can be encrypted by all parties.

For example, the owner can encrypt the download address through authentication limitation, and can also pay the user through the network platform with the encrypted download address and the symmetric key encrypted by the public key of the owner.

Step S302, the network platform sends the downloadable event of the analysis result to the user.

Step S303, the user downloads the analysis result, such as the evaluation result of the algorithm model or the trained algorithm model, on the network platform.

The user can obtain the download address from the notification of the downloadable event and decrypt the download address, and after the analysis result is downloaded according to the decrypted download address, the user can transfer the authentication information to all parties through the network platform to confirm the receipt of the analysis result.

Wherein, the user can also provide the evaluation of the annotation data set and/or the algorithm model and the calculation analysis evaluation on the network platform.

In this application scenario, fig. 6 shows a flowchart of a user interaction with a data processing apparatus according to an embodiment of the present application. As shown in fig. 6, the interaction flow of the user with the data processing apparatus includes:

In step S61, all parties issue the information of the annotation data set and/or the algorithm model to be traded on the data processing device.

In step S62, the user requests the data processing device.

In step S63, all parties reply the request of the user.

In step S64, the data processing apparatus returns a downloadable notification of the analysis result to the owner.

The data processing device can perform analysis and calculation by using the algorithm model and the related labeled data set according to response information sent after all parties repeat requests to obtain an analysis result.

In step S65, the owner sends a downloadable notification of the analysis result to the user through the data processing apparatus.

In step S66, the user downloads the analysis result from the data processing apparatus based on the downloadable notification of the analysis result.

In a possible implementation manner, the data processing method according to an embodiment of the present application may be implemented in combination with a block chain technique. For example, a sandbox may be implemented in virtual machine technology using a blockchain supervisory sandbox mode; the network platform may be implemented in conjunction with blockchain and intelligent contract technologies.

FIG. 7 shows a schematic diagram of utilizing a blockchain technique according to an embodiment of the present application. As shown in fig. 7, the sandbox is connected with an intelligent contract, and the intelligent contract is connected with a blockchain, and an exemplary process of implementing the data processing method according to the embodiment of the present application by using the blockchain technology is as follows:

All parties can release the labeled data set and/or the algorithm model, and in the case of releasing the labeled data set, the data directory, the metadata, the Hash and the certification (such as identity verification information) information can be linked; where an algorithmic model is published, the algorithmic description, Hash, attestation information (e.g., authentication information) may be stored uplink. The user can select the labeled data set and/or the algorithm model issued by all the parties, apply for calculation to the sandbox, and can submit the labeled data set and/or the algorithm model of the user. After all parties approve the request of the user, the approval records are linked and stored, the algorithm model submitted by the user and the storage address of the marked data set are linked, and an intelligent contract (the token is locked, and the automatic transfer is carried out after the contract content is finished) and the public keys of all the parties and the user are linked. And submitting the encrypted acquisition address of the marked data set and/or the algorithm model to the sandbox from all directions, and starting the sandbox. And the sandbox executes analysis and calculation, destroys data according to the use strategy after execution is finished, and stores the execution record and the destruction record in a chaining mode through the intelligent contract. The sandbox submits the encrypted analysis results and the symmetric key encrypted by the owner's public key to the owner. All parties issue analysis results, download addresses to the block chain, re-encrypt the symmetric key with the user public key and uplink the encrypted symmetric key for storage. And the user acquires the analysis result download address from the block chain. And the user downloads a calculation analysis report based on the address acquired on the chain, completes evaluation and stores the evaluation uplink.

Fig. 8 shows a flow chart of a data processing method according to an embodiment of the present application, and as shown in fig. 8, the method includes:

step S801, receiving first response information of a first user, wherein the first user provides at least one annotation data set, the first response information comprises a storage address of a target object determined by the first user, and the target object comprises a target annotation data set requested to be used by a third user;

step S802, receiving second response information of a second user, wherein the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model requested to be used by a third user;

step S803, the sandbox obtains the target annotation data set according to the first response information;

step S804, the sandbox obtains the target algorithm model according to the second response information;

step S805, evaluating a target algorithm model by using the target annotation data set to obtain an evaluation result, where the evaluation result is provided to a third user, and the first user, the second user, and the third user forbid access to the target annotation data set and the target algorithm model in the sandbox.

Wherein the first user may be an owner of the annotation data set, the second user may be an owner of the algorithm model, and the third user may be a user of the annotation data set and the algorithm model. The target annotation data set may comprise target annotation data sets to be traded by the first user and the third user, and the target algorithmic model may comprise an algorithmic model to be traded between the second user and the third user. The evaluation result may include evaluation of the accuracy of the algorithm model and the time complexity. For example, the target labeling data set may include a picture set labeled with pedestrians, and the algorithm model may include a machine learning model for pedestrian identification, where the machine learning model for pedestrian identification is used to identify pedestrians in the picture set, and then compared with labels of pedestrians in the picture set, the identification accuracy of the machine learning model may be evaluated, so as to assist a third user in selecting a desired machine learning model.

For example, the first response information may be determined by the first user from the request information of the third user, which may include information of the target annotation data set requested for use. Receiving first response information from a first user through a network platform in the data processing device shown in fig. 2-7, and according to the first response information, obtaining a target annotation data set pre-stored in an OBS through the sandbox through an OBS access channel shown in fig. 2-7, for example; the second response information may be determined by the second user based on request information of the third user, and the request information of the third user may include information of a target algorithm model requested to be used. The second response information from the second user may be received by the network platform in the data processing apparatus as shown in fig. 2 to fig. 7, and according to the second response information, the sandbox may access the channel through the OBS to obtain the target algorithm model pre-stored in the OBS.

In one possible implementation, the first user and the second user are the same user.

For example, the user providing the at least one annotation data set and the user of the at least one algorithm model may be the same user.

Fig. 9 is a flowchart illustrating a data processing method according to an embodiment of the present application, where the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: the usage policy of the target algorithm model, which is used for determining the condition for destroying the target algorithm model, as shown in fig. 9, the method further includes:

step S901, destroying the target annotation data set obtained by the sandbox according to a usage policy of the target annotation data set;

and S902, destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model. According to the embodiment of the application, the target labeling data set acquired by the sandbox is destroyed according to the use strategy of the target labeling data set; according to the use strategy of the target algorithm model, the target algorithm model obtained by the sandbox is destroyed, so that the marking data set and the algorithm model can be incinerated after use, and the marking data set and the algorithm model are prevented from being re-sold and abused for the second time.

For example, the sandbox may destroy the obtained source data of the target algorithm model and the annotation data set according to the usage policy of the target algorithm model and the target annotation data set.

In a possible implementation manner, destroying the target annotation data set acquired by the sandbox according to the usage policy of the target annotation data set includes: destroying the target annotation data set under the condition that the using times of the target annotation data set reach a preset first threshold value; and/or destroying the target annotation data set when the using time of the target annotation data set reaches a preset second threshold value; according to the using strategy of the target algorithm model, destroying the target algorithm model obtained by the sandbox comprises the following steps: destroying the target algorithm model under the condition that the using times of the target algorithm model reach a preset third threshold value; and/or destroying the target algorithm model in case the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

The number of times of use may be the number of times the annotation data set is calculated, or the number of times the algorithm model is calculated, and the time of use may be the time at which the annotation data set and/or the algorithm model is imported into the sandbox as a starting point.

Wherein the first threshold and the second threshold may be determined by a first user, the third threshold and the fourth threshold may be determined by a second user, and at least one of the first threshold, the second threshold, the third threshold, and the fourth threshold may also be determined by a third user.

In a possible implementation manner, the obtaining, by the sandbox, the target annotation data set according to the first response information includes: the sandbox acquires the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set; the sandbox obtains the target algorithm model according to the second response information, and the method comprises the following steps: and the sandbox acquires the target algorithm model stored in the cloud storage unit according to the storage address of the target algorithm model.

The target labeling data set and the target algorithm model are obtained by importing the target labeling data set into a target labeling data set algorithm module of the sandbox and importing the target algorithm model into a target algorithm model module of the sandbox for subsequent calculation.

The cloud storage unit can be realized through the OBS, and the embodiment of the application does not limit the specific implementation mode of the cloud storage unit.

In one possible implementation manner, the sandbox converts the obtained target algorithm model according to a conversion algorithm model so as to match the compiling environment of the sandbox.

The conversion algorithm model may be stored in the conversion algorithm model module shown in fig. 3, and the conversion algorithm model may be carried by a sandbox or obtained by the sandbox before analysis and calculation. For example, the conversion algorithm model may be implemented in ONNX format.

In one possible implementation, the method further includes: determining whether the first user is authentic according to first authentication information, the first authentication information being provided when the first user provides information of at least one annotation data set; determining whether the second user is trusted according to second identity verification information, wherein the second identity verification information is provided when the second user provides information of at least one algorithm model; determining whether the third user is authentic according to third authentication information, the third authentication information being provided when the third user requests to use the target annotation data set and the target algorithm model.

According to the embodiment of the application, whether the user is trusted or not is determined according to the identity authentication information, the problem that the users are not trusted mutually can be solved, the safe sharing of the labeled data set and the algorithm model is realized, and the process can be supervised and traceable.

The first authentication information and the second authentication information may be provided when the first user and the second user issue information of an annotated data set or an algorithm model (which is equivalent to the following when the first user sends the first information to be requested and the second user sends the second information to be requested) which can be requested by the third user.

The identity authentication information may include any type of identity authentication information such as qualification information of the enterprise.

In a possible implementation manner, receiving information of a first object to be requested sent by the first user, where the first object to be requested includes at least one labeled data set provided by the first user; receiving information of a second object to be requested sent by the second user, wherein the second object to be requested comprises at least one algorithm model provided by the second user; notifying the third user of the information of the first object to be requested and the information of the second object to be requested; receiving first request information sent by the third user, wherein the first request information indicates an annotated data set in the first object to be requested, which is requested to be used by the third user; receiving second request information sent by the third user, wherein the second request information indicates an algorithm model in the second object to be requested, which is requested to be used by the third user; sending the first request information to the first user, wherein the first request information is used for determining the first response information; and sending the second request information to the second user, wherein the second request information is used for determining the second response information.

In a possible implementation manner, the information of the first object to be requested further includes first authentication information, the information of the second object to be requested further includes second authentication information, and the first request information and the second request information further include third authentication information, and the method further includes: when the information of the first object to be requested is received, determining whether the first user is credible according to the first identity verification information; when receiving the information of the second object to be requested, determining whether the second user is credible according to the second identity authentication information; when the first request information is received, determining whether the third user is credible according to the third identity authentication information; and when the second request message is received, determining whether the third user is credible according to the third identity authentication information.

In one possible implementation, the method further includes: receiving demand information sent by the third user, wherein the demand information represents a labeled data set and/or an algorithm model required by the third user; notifying the first user and/or the second user of the demand information.

Wherein the first user and/or the second user is notified of the need information, e.g., the need information is broadcast on a network platform.

In one possible implementation, the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to the indication of the third user.

According to the embodiment of the application, the target marking data set and the target algorithm model can be incinerated after use, and secondary resale and abuse of the marking data set and the algorithm model are avoided. The third user can put forward the use strategy of the labeling data set or the algorithm model according to the self requirement, and the flexibility of the use strategy is improved.

For example, the usage policy of the target annotation data set and the usage policy of the target algorithm model may be indicated when the owner (third user) sends request information for using the target annotation data set or the target algorithm model to the user (first user and second user).

In a possible implementation manner, the first request information further includes a usage policy of the annotation data set indicated by the first request information, where the usage policy is used to determine a condition for destroying the annotation data set indicated by the first request information; the second request information further comprises a use strategy of the algorithm model indicated by the second request information, and the use strategy is used for determining the condition of destroying the algorithm model indicated by the second request information; the use strategy of the target annotation data set is determined according to the use strategy included in the first request message, and the use strategy of the target algorithm model is determined according to the use strategy included in the second request message.

According to the embodiment of the application, the first request information further comprises the use strategy of the annotation data set indicated by the first request information, the second request information further comprises the use strategy of the algorithm model indicated by the second request information, the use strategy of the target annotation data set is determined according to the use strategy included by the first request information, and the use strategy of the target algorithm model is determined according to the use strategy included by the second request information, so that the target annotation data set and the target algorithm model can be incinerated after use, and the annotation data set and the algorithm model are prevented from being resold and abused again. The third user can put forward the use strategy of the labeling data set or the algorithm model according to the self requirement, and the flexibility of the use strategy is improved.

In one possible implementation, the algorithmic model is a machine learning model.

In one possible implementation, the sandbox is supervised by a blockchain, the first response information is used for determining intelligent contracts generated by the first user and the third user, and the second response information is used for determining intelligent contracts generated by the second user and the third user.

Fig. 10 shows a flow chart of a data processing method according to an embodiment of the present application, as shown in fig. 10, the method comprising:

step S1001, receiving first response information of a first user, wherein the first user provides at least one annotation data set, the first response information includes a storage address of a target object determined by the first user, and the target object includes the target annotation data set provided by the first user;

step S1002, receiving second response information of a second user, wherein the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model requested to be used by the first user;

Step S1003, the sandbox acquires the target annotation data set according to the first response information;

step S1004, the sandbox acquires the target algorithm model according to the second response information;

step S1005, evaluating a target algorithm model by using the target annotation data set to obtain an evaluation result, where the evaluation result is provided to a first user, and the first user and the second user forbid access to the target annotation data set and the target algorithm model in the sandbox.

According to the embodiment of the application, first response information of a first user and second response information of a second user are received, and the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; and evaluating the target algorithm model by using the target labeling data set to obtain an evaluation result, so that sharing and transaction of the algorithm model by the user can be realized, and the first user can evaluate the algorithm model provided by the second user through the own labeling data set so as to assist the first user in selecting the required algorithm model. The first user and the second user forbid access to the target labeling data set and the target algorithm model in the sandbox, the user cannot access to the source data, the secure sharing of the algorithm model can be achieved, and the problems of secondary resale, excessive distribution and abuse of the source data are prevented. The sandbox is used as a third party to acquire and process data, the process is traceable and controllable, and the potential distrust problem of a user and all parties is solved. In addition, a user (a first user) does not directly obtain the annotation data set and the algorithm model, so that the problem that the user cannot download and use a large amount of data is solved, and the annotation data set and the algorithm model can be shared for many times by uploading once.

The first user can be a user using the annotation data set and the algorithm model, the second user can be all parties of the algorithm model, the target algorithm model can comprise a target algorithm model to be traded by the first user and the second user, and the target annotation data set can comprise the annotation data set owned by the first user.

For example, the first response information may be determined by the first user, the first response information may be received from the first user through a network platform in the data processing apparatus shown in fig. 2 to 7, and according to the first response information, the sandbox may obtain the target annotation data set prestored in the OBS through an OBS access channel shown in fig. 2 to 7, for example; the second response information may be determined by the second user based on the request information of the first user, which may include information of the target algorithm model requested to be used. The second response information from the second user may be received by the network platform in the data processing apparatus as shown in fig. 2 to fig. 7, and according to the second response information, the sandbox may access the channel through the OBS to obtain the target algorithm model pre-stored in the OBS. The first response information may further include an analysis requirement of the first user on the target annotation data set and the target algorithm model, for example, an evaluation of the target algorithm model is obtained.

In one possible implementation manner, the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the method further comprises the following steps: destroying the target labeling data set obtained by the sandbox according to the use strategy of the target labeling data set; and destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

In a possible implementation manner, destroying the target annotation data set obtained by the sandbox according to the usage policy of the target annotation data set includes: destroying the target annotation data set under the condition that the using times of the target annotation data set reach a preset first threshold value; and/or destroying the target annotation data set when the using time of the target annotation data set reaches a preset second threshold value; according to the using strategy of the target algorithm model, destroying the target algorithm model obtained by the sandbox comprises the following steps: destroying the target algorithm model under the condition that the using times of the target algorithm model reach a preset third threshold value; and/or destroying the target algorithm model in case the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

According to the embodiment of the application, the destruction is carried out according to the use times or use time of the target labeling data sets and the target algorithm models, so that the destruction strategy can be flexibly selected by users, and different destruction requirements for different labeling data sets, different algorithm models and different users are met.

Wherein the number of times of use may be the number of times the annotation data set is calculated, or the number of times the algorithmic model is calculated, and the time of use may be from the time the annotation data set and/or the algorithmic model is imported into the sandbox.

Wherein the first threshold and the second threshold may be determined by a first user and the third threshold and the fourth threshold may be determined by a second user. At least one of the third threshold and the fourth threshold may also be determined by the first user.

In one possible implementation, the method further includes: and the sandbox converts the acquired target labeling data set and/or the target algorithm model according to a conversion algorithm model so as to match the compiling environment of the sandbox.

The conversion algorithm model may be stored in the conversion algorithm model module shown in fig. 3, and the conversion algorithm model may be carried by a sandbox or obtained by the sandbox before analysis and calculation. For example, the conversion algorithm model may be implemented by an ONNX structure.

In one possible implementation, the method further includes: determining whether the first user is authentic according to first authentication information provided when the first user requests to use a target algorithm model; and determining whether the second user is credible according to second identity verification information, wherein the second identity verification information is provided when the second user provides information of at least one algorithm model.

The first authentication information and the second authentication information may be provided when the first user requests to use the algorithm model (corresponding to the first user sending the request information hereinafter), and the second user issues information of the algorithm model which can be requested by the first user (corresponding to the second user sending the object-to-be-requested information hereinafter), respectively.

In one possible implementation, the method further includes: receiving information of an object to be requested sent by the second user, wherein the object to be requested comprises at least one algorithm model provided by the second user; notifying the first user of information of the object to be requested; receiving request information sent by the first user, wherein the request information indicates an algorithm model in the object to be requested, which is requested to be used by the first user; and sending the request information to the second user, wherein the request information is used for determining the second response information.

In a possible implementation manner, the information of the object to be requested further includes first authentication information, the request information further includes second authentication information, and the method further includes: when the information of the object to be requested is received, determining whether the second user is credible according to the first identity verification information; and when the request information is received, determining whether the first user is credible according to the second identity authentication information.

In one possible implementation, the method further includes: receiving demand information sent by the first user, wherein the demand information represents an algorithm model required by the first user; notifying the second user of the demand information.

Wherein the second user is notified of the need information, e.g., the need information is broadcast on a network platform.

In one possible implementation, the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to the indication of the first user.

For example, the usage policy of the target algorithm model may be indicated when the user (first user) sends request information for using the target algorithm model to all the users (second users).

In a possible implementation manner, the request information further includes: the use strategy of the algorithm model indicated by the request information is used for determining the condition of destroying the algorithm model indicated by the request information; and the use strategy of the target algorithm model is determined according to the use strategy included in the request information.

According to the embodiment of the application, the request information further comprises the use strategy of the algorithm model indicated by the request information, and the use strategy of the target algorithm model is determined according to the use strategy included by the request information, so that the target algorithm model can be incinerated after use, and the algorithm model is prevented from being secondarily reselled and abused. The first user can provide the use strategy of the algorithm model according to the self requirement, and the flexibility of the use strategy is improved. In one possible implementation, the algorithmic model is a machine learning model.

According to the embodiment of the application, safe sharing and fair transaction of the machine learning model among different users can be realized, and abuse of source data is prevented, so that the user can obtain a required evaluation result without downloading the source data of the machine learning model.

In one possible implementation, the sandbox is supervised by a blockchain, and the second response information is used for determining the intelligent contracts generated by the first user and the second user.

Fig. 11 shows a flow chart of a data processing method according to an embodiment of the present application, as shown in fig. 11, the method comprising:

step S1101, receiving first response information of a first user, where the first user provides at least one annotation data set, where the first response information includes a storage address of a target object determined by the first user, and the target object includes a target annotation data set requested to be used by a second user;

step S1102, receiving second response information of a second user, where the second user provides at least one algorithm model, the second response information includes a storage address of a target object determined by the second user, and the target object includes the target algorithm model provided by the second user;

Step S1103, the sandbox obtains the target annotation data set according to the first response information;

step S1104, the sandbox obtains the target algorithm model according to the second response information;

step S1105, training the target algorithm model with the target labeling data set to obtain a trained algorithm model, where the trained algorithm model is provided to the second user, and the first user and the second user forbid access to the target labeling data set and the target algorithm model in the sandbox.

According to the embodiment of the application, first response information of a first user and second response information of a second user are received, and the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; the target labeling data set is used for training a target algorithm model to obtain the trained algorithm model, sharing and trading of the algorithm model by users can be achieved, a second user can train the own algorithm model through the labeling data set provided by the first user, so that the second user can also achieve training of the algorithm model under the condition that the second user lacks enough labeling data, the first user and the second user forbid access to the target labeling data set and the target algorithm model in the sandbox, the user cannot access to source data, safe sharing of the algorithm model can be achieved, and the problems of secondary resale, abusing and abusing of the source data are prevented. The sandbox is used as a third party to acquire and process data, the process is traceable and controllable, and the potential distrust problem of a user and all parties is solved. In addition, a user (a second user) does not directly obtain the annotation data set and the algorithm model, so that the problem that the user cannot download and use a large amount of data is solved, and the annotation data set and the algorithm model can be shared for many times by uploading once.

The first user can mark all parties of the data set, the second user can mark users of the data set and the algorithm model, the target marking data set can comprise a target marking data set to be traded by the first user and the second user, and the target algorithm model can comprise the algorithm model owned by the second user. The trained algorithm model is available to the second user.

For example, the first response information may be determined by the first user according to the request information of the second user, the first response information from the first user may be received through the network platform in the data processing apparatus shown in fig. 2 to 7, and according to the first response information, the sandbox may obtain the target annotation data set prestored in the OBS through the OBS access channel shown in fig. 2 to 7, for example; the second response information from the second user may be received by the network platform in the data processing apparatus shown in fig. 2 to fig. 7, and according to the second response information, the sandbox may obtain the target algorithm model pre-stored in the OBS through the OBS access channel. The second response message may also include the analysis requirements of the second user for the target annotation data set and the target algorithm model, for example, training the target algorithm model using the target annotation data set.

Fig. 12 is a flowchart illustrating a data processing method according to an embodiment of the present application, where the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: as shown in fig. 12, the information of the object to be requested further includes first authentication information, the request information further includes second authentication information, and the method may further include:

step S1201, destroying the target labeling data set obtained by the sandbox according to the use strategy of the target labeling data set;

step S1202, destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

For example, the sandbox may destroy the obtained annotation data set and the source data of the target algorithm model according to the usage policy of the target annotation data set and the target algorithm model.

In a possible implementation manner, destroying the target annotation data set obtained by the sandbox according to the usage policy of the target annotation data set includes: under the condition that the number of times of using the target labeling data set reaches a preset first threshold value, destroying the target labeling data set; and/or destroying the target annotation data set when the using time of the target annotation data set reaches a preset second threshold value; according to the using strategy of the target algorithm model, destroying the target algorithm model obtained by the sandbox comprises the following steps: destroying the target algorithm model under the condition that the using times of the target algorithm model reach a preset third threshold value; and/or destroying the target algorithm model in case the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

Wherein the first threshold and the second threshold may be determined by a first user and the third threshold and the fourth threshold may be determined by a second user. At least one of the first threshold and the second threshold may also be determined by the second user.

The obtaining of the target annotation data set and the target algorithm model may be implemented by importing the target annotation data set into a target annotation data set algorithm module of the sandbox and importing the target algorithm model into a target algorithm model module of the sandbox for subsequent calculation.

According to the embodiment of the application, the acquired target algorithm model is converted according to the conversion algorithm model, so that the sandbox can support algorithm models of different compiling environments for analysis and calculation, and the use of diversified algorithm models by a user is supported. For example, the conversion algorithm model may be implemented by an ONNX structure.

In one possible implementation, the method further includes: determining whether the first user is authentic according to first authentication information, the first authentication information being provided when the first user provides information of at least one annotation data set; and determining whether the second user is credible according to second identity verification information, wherein the second identity verification information is provided when the second user requests to use the target annotation data set.

According to the embodiment of the application, whether the user is trusted or not is determined according to the identity authentication information, the problem that the users are not trusted mutually can be solved, the algorithm model is safely shared, and the process can be supervised and traceable.

The first authentication information may be provided when the first user issues information of the annotation data set requested by the second user (which is equivalent to the aforementioned information of the object to be requested sent by the first user), and the second authentication information may be provided when the second user sends the request information.

The authentication information may include any type of authentication information such as qualification information of the enterprise.

In one possible implementation, the method further includes: receiving information of an object to be requested sent by the first user, wherein the object to be requested comprises at least one annotation data set provided by the first user; notifying the second user of the information of the object to be requested; receiving request information sent by the second user, wherein the request information indicates a labeled data set in the object to be requested, which is requested to be used by the second user; and sending the request information to the first user, wherein the request information is used for determining the first response information.

In a possible implementation manner, the information of the object to be requested further includes first authentication information, the request information further includes second authentication information, and the method further includes: when the information of the object to be requested is received, determining whether the first user is credible according to the first identity verification information; and when the request information is received, determining whether the second user is credible according to the second identity verification information.

In one possible implementation, the method further includes: receiving demand information sent by the second user, wherein the demand information represents a labeled data set required by the second user; notifying the first user of the demand information.

In one possible implementation, the usage policy of the target annotation data set is determined according to the indication of the second user.

According to the embodiment of the application, the burning-after-use of the target algorithm model can be realized, and the algorithm model is prevented from being secondarily resaled and abused. The first user can provide the use strategy of the algorithm model according to the self requirement, and the flexibility of the use strategy is improved.

For example, the usage policy of the target annotation data set may be indicated when the user (second user) sends request information for using the target annotation data set to all the users (first users).

In one possible implementation manner, the request information further includes: the use strategy of the marked data set indicated by the request information is used for determining the condition of destroying the marked data set indicated by the request information; and the use strategy of the target annotation data set is determined according to the use strategy included in the request information.

According to the embodiment of the application, the request information further comprises the use strategy of the annotation data set indicated by the request information, and the use strategy of the target annotation data set is determined according to the use strategy included by the request information, so that the target annotation data set can be burned after use, and the target annotation data set is prevented from being re-sold and abused for the second time. The second user can put forward the use strategy of the target marking data set according to the self requirement, and the flexibility of the use strategy is improved.

According to the embodiment of the application, the machine learning model can be safely used by a user, abuse of source data is prevented, and the trained algorithm model can be obtained by using the self machine learning model by the user.

In one possible implementation, the sandbox is supervised by a blockchain, and the first response information is used for determining intelligent contracts generated by the first user and the second user.

Fig. 13 is a block diagram illustrating a data processing apparatus according to an embodiment of the present application, and as shown in fig. 13, the apparatus includes:

a first receiving module 1301, configured to receive first response information of a first user, the first user providing at least one annotation data set, the first response information including a storage address of a target object determined by the first user, the target object including a target annotation data set requested to be used by a third user;

a second receiving module 1302, configured to receive second response information of a second user, where the second user provides at least one algorithm model, the second response information includes a storage address of a target object determined by the second user, and the target object includes a target algorithm model requested to be used by a third user;

a first obtaining module 1303, configured to obtain, by the sandbox, the target annotation data set according to the first response information;

a second obtaining module 1304, configured to obtain, by the sandbox, the target algorithm model according to the second response information;

a first evaluation module 1305, configured to evaluate a target algorithm model by using the target annotation data set to obtain an evaluation result, where the evaluation result is provided to a third user, and the first user, the second user, and the third user prohibit access to the target annotation data set and the target algorithm model in the sandbox.

In a possible implementation manner of the data processing apparatus, the first user and the second user are the same user.

In a possible implementation manner of the data processing apparatus, the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model, and the device further comprises: the first destruction module is used for destroying the target labeling data set acquired by the sandbox according to the use strategy of the target labeling data set; and the second destruction module is used for destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

According to the embodiment of the application, the target labeling data set acquired by the sandbox is destroyed according to the use strategy of the target labeling data set; according to the use strategy of the target algorithm model, the target algorithm model obtained by the sandbox is destroyed, so that the marking data set and the algorithm model can be incinerated after use, and the marking data set and the algorithm model are prevented from being re-sold and abused for the second time. In a possible implementation manner of the data processing apparatus, the first destruction module includes: the first destruction submodule is used for destroying the target labeling data set under the condition that the number of times of using the target labeling data set reaches a preset first threshold value; and/or a second destruction submodule, configured to destroy the target annotation data set when a usage time for which the target annotation data set is used reaches a predetermined second threshold; the second destruction module comprises: a third destruction submodule, configured to destroy the target algorithm model when the number of times that the target algorithm model is used reaches a predetermined third threshold; and/or a fourth destroy submodule for destroying the target algorithm model if the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

In a possible implementation manner of the data processing apparatus, the first obtaining module includes: the first obtaining submodule is used for the sandbox to obtain the target labeling data set stored in the cloud storage unit according to the storage address of the target labeling data set; the second acquisition module comprises: and the second obtaining submodule is used for obtaining the target algorithm model stored in the cloud storage unit by the sandbox according to the storage address of the target algorithm model.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: and the first conversion module is used for converting the obtained target algorithm model by the sandbox according to the conversion algorithm model so as to match the compiling environment of the sandbox.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a first determining module, configured to determine whether the first user is authentic according to first authentication information, where the first authentication information is provided when the first user provides information of at least one annotation data set; the second determining module is used for determining whether the second user is credible according to second identity verification information, and the second identity verification information is provided when the second user provides information of at least one algorithm model; and the third determining module is used for determining whether the third user is credible according to third authentication information, and the third authentication information is provided when the third user requests the target annotation data set and the target algorithm model.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a fifth receiving module, configured to receive information of a first object to be requested sent by the first user, where the first object to be requested includes at least one labeled data set provided by the first user; a sixth receiving module, configured to receive information of a second object to be requested sent by the second user, where the second object to be requested includes at least one algorithm model provided by the second user; a first notification module, configured to notify the third user of information of the first object to be requested and information of the second object to be requested; a seventh receiving module, configured to receive first request information sent by the third user, where the first request information indicates an annotated data set in the first object to be requested, which is requested to be used by the third user; an eighth receiving module, configured to receive second request information sent by the third user, where the second request information indicates an algorithm model in the second object to be requested, where the third user requests to use the algorithm model; a first sending module, configured to send the first request information to the first user, where the first request information is used to determine the first response information; and a second sending module, configured to send the second request information to the second user, where the second request information is used to determine the second response information.

In a possible implementation manner of the data processing apparatus, the information of the first object to be requested further includes first authentication information, the information of the second object to be requested further includes second authentication information, and the first request information and the second request information further include third authentication information, and the apparatus further includes: a sixth determining module, configured to determine, when receiving the information of the first object to be requested, whether the first user is trusted according to the first authentication information; a seventh determining module, configured to determine, when receiving information of the second object to be requested, whether the second user is trusted according to the second authentication information; an eighth determining module, configured to determine, when the first request information is received, whether the third user is trusted according to the third authentication information; and the ninth determining module is used for determining whether the third user is credible according to the third identity authentication information when the second request information is received.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a ninth receiving module, configured to receive requirement information sent by the third user, where the requirement information represents a labeled data set and/or an algorithm model required by the third user; and the second notification module is used for notifying the first user and/or the second user of the demand information.

In a possible implementation of the data processing apparatus, the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to an instruction of the third user.

In a possible implementation manner of the data processing apparatus, the first request information further includes a usage policy of the annotation data set indicated by the first request information, where the usage policy is used to determine a condition for destroying the annotation data set indicated by the first request information; the second request information further comprises a use strategy of the algorithm model indicated by the second request information, and the use strategy is used for determining the condition of destroying the algorithm model indicated by the second request information; the use strategy of the target annotation data set is determined according to the use strategy included in the first request message, and the use strategy of the target algorithm model is determined according to the use strategy included in the second request message.

According to the embodiment of the application, the first response information further comprises the use policy of the annotation data set indicated by the third user; the second response information further includes a usage policy of the algorithm model indicated by the third user; the use strategy of the target labeling data set is determined according to the use strategy of the labeling data set indicated by the third user, and the use strategy of the target algorithm model is determined according to the use strategy of the algorithm model indicated by the third user, so that the target labeling data set and the target algorithm model can be incinerated after use, and the labeling data set and the algorithm model are prevented from being re-sold and abused. The third user can put forward the use strategy of the labeled data set or the algorithm model according to the self requirement, and the flexibility of the use strategy is improved.

In one possible implementation of the data processing apparatus, the algorithmic model is a machine learning model.

In one possible implementation of the data processing apparatus, the sandbox is supervised by a blockchain, the first response information is used for determining intelligent contracts made by the first user and the third user, and the second response information is used for determining intelligent contracts made by the second user and the third user.

According to the embodiment of the application, by means of the block chain technology, the uplink data of the users can be not tampered, the problem of distrust among the users is solved, the safe sharing of the labeled data set and the algorithm model is realized, the process is transparent to the users, the traceability is controlled, and the transaction information can not be tampered.

Fig. 14 is a block diagram showing a data processing apparatus according to an embodiment of the present application, and as shown in fig. 14, the apparatus includes:

A tenth receiving module 1401, configured to receive first response information of a first user, where the first user provides at least one annotation data set, and the first response information includes a storage address of a target object determined by the first user, where the target object includes the target annotation data set provided by the first user;

an eleventh receiving module 1402, configured to receive second response information of a second user, the second user providing at least one algorithm model, the second response information including a storage address of a target object determined by the second user, the target object including a target algorithm model requested to be used by the first user;

a fifth obtaining module 1403, configured to, according to the first response information, obtain the target annotation data set by the sandbox;

a sixth obtaining module 1404, configured to, according to the second response information, obtain the target algorithm model by the sandbox;

a second evaluation module 1405, configured to evaluate a target algorithm model by using the target annotation data set to obtain an evaluation result, where the evaluation result is provided to a first user, and the first user and the second user prohibit access to the target annotation data set and the target algorithm model in the sandbox.

According to the embodiment of the application, first response information of a first user and second response information of a second user are received, and the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; and evaluating the target algorithm model by using the target labeling data set to obtain an evaluation result, so that sharing and transaction of the algorithm model by a user can be realized, the first user and the second user forbid access to the target labeling data set and the target algorithm model in the sandbox, the user cannot access to source data, the safe sharing of the algorithm model can be realized, and the problems of secondary resale, excessive distribution and abuse of the source data are prevented. The sandbox is used as a third party to acquire and process data, the process is traceable and controllable, and the potential distrust problem of a user and all parties is solved. In addition, a user (a first user) does not directly obtain the annotation data set and the algorithm model, so that the problem that the user cannot download and use a large amount of data is solved, and the annotation data set and the algorithm model can be shared for many times by uploading once.

In a possible implementation manner of the data processing apparatus, the first response information further includes: the use strategy of the target labeling data set is used for determining the destroy condition of the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the device further comprises: a fifth destruction module, configured to destroy the target annotation data set obtained by the sandbox according to a usage policy of the target annotation data set; and the sixth destruction module is used for destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

According to the embodiment of the application, the target labeling data set acquired by the sandbox is destroyed according to the use strategy of the target labeling data set; according to the use strategy of the target algorithm model, the target algorithm model obtained by the sandbox is destroyed, so that the marking data set and the algorithm model can be incinerated after use, and the marking data set and the algorithm model are prevented from being re-sold and abused for the second time. According to the first possible implementation manner of the second aspect, in a second possible implementation manner of the data processing method, according to a usage policy of the target annotation data set, destroying the target annotation data set acquired by the sandbox includes: destroying the target annotation data set under the condition that the using times of the target annotation data set reach a preset first threshold value; and/or destroying the target annotation data set when the using time of the target annotation data set reaches a preset second threshold value; according to the using strategy of the target algorithm model, destroying the target algorithm model obtained by the sandbox comprises the following steps: destroying the target algorithm model under the condition that the using times of the target algorithm model reach a preset third threshold value; and/or destroying the target algorithm model in case the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

In a possible implementation manner of the data processing apparatus, the fifth destruction module includes: a ninth destruction sub-module, configured to destroy the target annotation data set when the number of times that the target annotation data set is used reaches a predetermined first threshold; and/or a tenth destruction submodule, configured to destroy the target annotation data set when a usage time for which the target annotation data set is used reaches a predetermined second threshold; the sixth destruction module comprises: an eleventh destruction submodule for destroying the target algorithm model when the number of times the target algorithm model is used reaches a predetermined third threshold; and/or a twelfth destruction submodule for destroying the target algorithm model if the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

In a possible implementation manner of the data processing apparatus, the fifth obtaining module includes: a fifth obtaining submodule, configured to obtain, by the sandbox, the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set; the sixth obtaining module includes: and the sixth obtaining submodule is used for obtaining the target algorithm model stored in the cloud storage unit by the sandbox according to the storage address of the target algorithm model.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: and the second conversion module is used for converting the obtained target labeling data set and/or the target algorithm model by the sandbox according to a conversion algorithm model so as to match the compiling environment of the sandbox.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a fourteenth determining module, configured to determine whether the first user is trusted according to first authentication information, the first authentication information being provided when the first user requests to use the target algorithm model; and a fifteenth determining module, configured to determine whether the second user is trusted according to second authentication information, where the second authentication information is provided when the second user provides information of at least one algorithm model.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a twelfth receiving module, configured to receive information of an object to be requested sent by the second user, where the object to be requested includes at least one algorithm model provided by the second user; a third notifying module, configured to notify the first user of information of the object to be requested; a thirteenth receiving module, configured to receive request information sent by the first user, where the request information indicates an algorithm model in the object to be requested, which the first user requests to use; and a third sending module, configured to send the request information to the second user, where the request information is used to determine the second response information.

In a possible implementation manner of the data processing apparatus, the information of the object to be requested further includes first authentication information, the request information further includes second authentication information, and the apparatus further includes: a tenth determining module, configured to determine, when receiving the information of the object to be requested, whether the second user is trusted according to the first identity verification information; and the eleventh determining module is used for determining whether the first user is credible according to the second identity authentication information when the request information is received.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a fourteenth receiving module, configured to receive requirement information sent by the first user, where the requirement information represents an algorithm model required by the first user; and the fourth notification module is used for notifying the second user of the demand information.

In one possible implementation of the data processing apparatus, the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to an instruction of the first user.

According to the embodiment of the application, the first user can put forward the use strategy of the labeled data set or the algorithm model according to the requirement of the first user, and the flexibility of the use strategy is improved.

In a possible implementation manner of the data processing apparatus, the request information further includes: the use strategy of the algorithm model indicated by the request information is used for determining the condition of destroying the algorithm model indicated by the request information; the use strategy of the target algorithm model is determined according to the use strategy included in the request information.

According to the embodiment of the application, the request information further comprises the use strategy of the algorithm model indicated by the request information, and the use strategy of the target algorithm model is determined according to the use strategy included by the request information, so that the target algorithm model can be incinerated after use, and the algorithm model is prevented from being secondarily reselled and abused. The first user can put forward the use strategy of the algorithm model according to the self requirement, and the flexibility of the use strategy is improved. In one possible implementation of the data processing apparatus, the algorithmic model is a machine learning model.

In one possible implementation of the data processing apparatus, the sandbox is supervised by a blockchain, and the second response information is used to determine the smart contracts generated by the first user and the second user.

Fig. 15 is a block diagram showing a data processing apparatus according to an embodiment of the present application, and as shown in fig. 15, the apparatus includes:

a third receiving module 1501, configured to receive first response information of a first user, where the first user provides at least one annotation data set, the first response information includes a storage address of a target object determined by the first user, and the target object includes a target annotation data set requested to be used by a second user;

a fourth receiving module 1502, configured to receive second response information of a second user, where the second user provides at least one algorithm model, the second response information includes a storage address of a target object determined by the second user, and the target object includes a target algorithm model provided by the second user;

a third obtaining module 1503, configured to obtain, by the sandbox, the target annotation data set according to the first response information;

a fourth obtaining module 1504, configured to, according to the second response information, obtain the target algorithm model by the sandbox;

a training module 1505 configured to train the target algorithm model with the target labeling data set to obtain a trained algorithm model, wherein the trained algorithm model is provided to the second user, and the first user and the second user forbid access to the target labeling data set and the target algorithm model in the sandbox.

According to the embodiment of the application, first response information of a first user and second response information of a second user are received, and the sandbox acquires the target annotation data set according to the first response information; the sandbox acquires the target algorithm model according to the second response information; the target labeling data set is used for evaluating the target algorithm model to obtain an evaluation result, sharing and trading of the algorithm model by a user can be realized, a second user can train the own algorithm model through the labeling data set provided by the first user, so that the second user can train the algorithm model under the condition that the second user lacks enough labeling data, the first user and the second user forbid access to the target labeling data set and the target algorithm model in the sandbox, the user cannot access to source data, safe sharing of the algorithm model can be realized, and the problems of secondary resale, abusing and abusing of the source data are prevented. The sandbox is used as a third party to acquire and process data, the process is traceable and controllable, and the potential distrust problem of a user and all parties is solved. In addition, a user (a first user) does not directly obtain the annotation data set and the algorithm model, so that the problem that the user cannot download and use a large amount of data is solved, and the annotation data set and the algorithm model can be shared for many times by uploading once.

In a possible implementation manner of the data processing apparatus, the first response information further includes: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set; the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model; the device further comprises: the third destroying module is used for destroying the target labeling data set acquired by the sandbox according to the use strategy of the target labeling data set; and the fourth destroying module is used for destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

In a possible implementation manner of the data processing apparatus, the third destruction module includes: a fifth destruction submodule, configured to destroy the target annotation data set when the number of times that the target annotation data set is used reaches a predetermined first threshold; and/or a sixth destruction submodule, configured to destroy the target annotation data set when a usage time for which the target annotation data set is used reaches a predetermined second threshold; the fourth destruct module comprises: a seventh destruction submodule, configured to destroy the target algorithm model when the number of times that the target algorithm model is used reaches a predetermined third threshold; and/or an eighth destruction submodule for destroying the target algorithm model if the usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

In a possible implementation manner of the data processing apparatus, the third obtaining module includes: the third obtaining submodule is used for obtaining the target annotation data set stored in the cloud storage unit by the sandbox according to the storage address of the target annotation data set; the fourth obtaining module includes: and the fourth obtaining submodule is used for obtaining the target algorithm model stored in the cloud storage unit by the sandbox according to the storage address of the target algorithm model.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: and the third conversion module is used for converting the obtained target labeling data set and/or the target algorithm model by the sandbox according to a conversion algorithm model so as to match the compiling environment of the sandbox.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a fourth determining module, configured to determine whether the first user is trusted according to first authentication information, where the first authentication information is provided when the first user provides information of at least one annotation data set; and the fifth determining module is used for determining whether the second user is credible according to second identity verification information, and the second identity verification information is provided when the second user requests a target annotation data set to be used.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a fifteenth receiving module, configured to receive information of an object to be requested sent by the first user, where the object to be requested includes at least one annotation data set provided by the first user; a fifth notification module, configured to notify the second user of information of the object to be requested; a sixteenth receiving module, configured to receive request information sent by the second user, where the request information indicates a tagged data set in the object to be requested, where the tagged data set is requested to be used by the second user; a fourth sending module, configured to send the request information to the first user, where the request information is used to determine the first response information.

In a possible implementation manner of the data processing apparatus, the information of the object to be requested further includes first authentication information, the request information further includes second authentication information, and the apparatus further includes: a twelfth determining module, configured to determine, when information of the object to be requested is received, whether the first user is trusted according to the first authentication information; and the thirteenth determining module is used for determining whether the second user is credible according to the second identity authentication information when the request information is received.

In one possible implementation manner of the data processing apparatus, the apparatus further includes: a seventeenth receiving module, configured to receive requirement information sent by the second user, where the requirement information indicates a labeled data set required by the second user; a sixth notification module, configured to notify the first user of the demand information.

In a possible implementation of the data processing apparatus, the usage policy of the target annotation data set is determined according to the indication of the second user.

In a possible implementation manner of the data processing apparatus, the request information further includes: the use strategy of the marked data set indicated by the request information is used for determining the condition of destroying the marked data set indicated by the request information; and the use strategy of the target annotation data set is determined according to the use strategy included in the request information.

According to the embodiment of the application, the machine learning model can be safely used by a user, abuse of source data is prevented, and the trained algorithm model can be obtained by using the self-owned machine learning model.

In one possible implementation of the data processing apparatus, the sandbox is supervised by a blockchain, and the first response information is used to determine the smart contracts generated by the first user and the second user.

According to the embodiment of the application, by means of the block chain technology, the uplink data of the users can be not tampered, the problem of distrust among the users is solved, the algorithm model is safely shared, the process is transparent to the users, the traceability is controllable, and the transaction information can not be tampered.

Fig. 16 shows a block diagram of a data processing apparatus according to an embodiment of the present application. As shown in fig. 16, the apparatus 40 includes at least one processor 1801, at least one memory 1802, and at least one communication interface 1803. In addition, the device may also include common components such as an antenna, which will not be described in detail herein.

The processor 1801 may be a general purpose Central Processing Unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to control the execution of programs according to the above schemes.

Communication interface 1803 may be adapted to communicate with other devices or a communication network, such as an ethernet network, a Radio Access Network (RAN), a core network, a Wireless Local Area Network (WLAN), etc.

The Memory 1802 may be, but is not limited to, a Read-Only Memory (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be self-contained and coupled to the processor via a bus. The memory may also be integral to the processor.

The memory 1802 is used for storing application program codes for executing the above schemes, and the execution of the application program codes is controlled by the processor 1801. The processor 1801 is configured to execute application code stored in the memory 1802.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units is only one type of division of logical functions, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

An embodiment of the present application provides a data processing apparatus, including: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions.

Embodiments of the present application provide a non-transitory computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.

Embodiments of the present application provide a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, the processor in the electronic device performs the above method.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an erasable Programmable Read-Only Memory (EPROM or flash Memory), a Static Random Access Memory (SRAM), a portable Compact Disc Read-Only Memory (CD-ROM), a Digital Versatile Disc (DVD), a Memory stick, a floppy disk, a mechanical coding device, a punch card or an in-groove protrusion structure, for example, having instructions stored thereon, and any suitable combination of the foregoing.

The computer readable program instructions or code described herein may be downloaded to the respective computing/processing device from a computer readable storage medium, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present application may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry can execute computer-readable program instructions to implement aspects of the present application by utilizing state information of the computer-readable program instructions to personalize custom electronic circuitry, such as Programmable Logic circuits, Field-Programmable Gate arrays (FPGAs), or Programmable Logic Arrays (PLAs).

Various aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

It is also noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by hardware (e.g., a Circuit or an ASIC) for performing the corresponding function or action, or by combinations of hardware and software, such as firmware.

While the invention has been described in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a review of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method of data processing, the method comprising:

receiving first response information of a first user, wherein the first user provides at least one annotation data set, the first response information comprises a storage address of a target object determined by the first user, and the target object comprises a target annotation data set requested to be used by a third user;

receiving second response information of a second user, wherein the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model requested to be used by a third user;

the sandbox acquires the target annotation data set according to the first response information;

the sandbox acquires the target algorithm model according to the second response information;

and evaluating a target algorithm model by using the target annotation data set to obtain an evaluation result, wherein the evaluation result is used for being provided for a third user, and the first user, the second user and the third user forbid accessing the target annotation data set and the target algorithm model in the sandbox.

2. The method of claim 1, wherein the first response information further comprises: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set;

the second response information further includes: the use strategy of the target algorithm model is used for determining the destroy condition of the target algorithm model;

the method further comprises the following steps:

destroying the target labeling data set obtained by the sandbox according to the use strategy of the target labeling data set;

and destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

3. The method of claim 2,

the destroying the target labeling data set acquired by the sandbox according to the use strategy of the target labeling data set includes:

destroying the target annotation data set under the condition that the using times of the target annotation data set reach a preset first threshold value; and/or

Destroying the target annotation data set under the condition that the use time of the target annotation data set reaches a preset second threshold value;

The destroying the target algorithm model obtained by the sandbox according to the using strategy of the target algorithm model comprises the following steps:

destroying the target algorithm model under the condition that the using times of the target algorithm model reach a preset third threshold value; and/or

And destroying the target algorithm model under the condition that the use time of the target algorithm model used reaches a preset fourth threshold value.

4. The method according to at least one of claims 1 to 3, wherein the sandbox obtains the target annotation data set according to the first response information, including:

the sandbox acquires the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set;

the sandbox obtains the target algorithm model according to the second response information, and the method comprises the following steps:

and the sandbox acquires the target algorithm model stored in the cloud storage unit according to the storage address of the target algorithm model.

5. The method according to at least one of claims 1-4, characterized in that the method further comprises:

determining whether the first user is authentic according to first authentication information, the first authentication information being provided when the first user provides information of at least one annotation data set;

Determining whether the second user is trusted according to second identity verification information, wherein the second identity verification information is provided when the second user provides information of at least one algorithm model;

determining whether the third user is authentic according to third authentication information, the third authentication information being provided when the third user requests to use the target annotation data set and the target algorithm model.

6. The method of claim 2, wherein the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to the indication of the third user.

7. The method of at least one of claims 1-6, wherein the sandbox is supervised by a blockchain, the first response information is used to determine smart contracts made by the first user and the third user, and the second response information is used to determine smart contracts made by the second user and the third user.

8. A method of data processing, the method comprising:

receiving first response information of a first user, wherein the first user provides at least one annotation data set, the first response information comprises a storage address of a target object determined by the first user, and the target object comprises a target annotation data set requested to be used by a second user;

Receiving second response information of a second user, wherein the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model provided by the second user;

and training the target algorithm model by using the target labeling data set to obtain a trained algorithm model, wherein the trained algorithm model is provided for the second user, and the first user and the second user forbid accessing the target labeling data set and the target algorithm model in the sandbox.

9. The method of claim 8, wherein the first response information further comprises: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set;

The method further comprises the following steps:

10. The method of claim 9,

the destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model comprises the following steps:

11. The method according to at least one of claims 8 to 10, wherein the sandbox obtains the target annotation data set according to the first response information, including:

12. The method according to at least one of claims 8-11, characterized in that the method further comprises:

and determining whether the second user is credible according to second identity verification information, wherein the second identity verification information is provided when the second user requests to use the target annotation data set.

13. The method of claim 9, wherein the policy of use of the target annotation data set is determined in accordance with the second user's instructions.

14. The method of at least one of claims 8-13, wherein the sandbox is supervised by a blockchain, and the first response information is used to determine a smart contract generated by the first user and the second user.

15. A data processing apparatus, characterized in that the apparatus comprises:

a first receiving module, configured to receive first response information of a first user, where the first user provides at least one annotation data set, the first response information includes a storage address of a target object determined by the first user, and the target object includes a target annotation data set requested to be used by a third user;

the second receiving module is used for receiving second response information of a second user, the second user provides at least one algorithm model, the second response information comprises a storage address of a target object determined by the second user, and the target object comprises a target algorithm model requested to be used by a third user;

the first obtaining module is used for obtaining the target annotation data set by the sandbox according to the first response information;

the second obtaining module is used for obtaining the target algorithm model by the sandbox according to the second response information;

And the first evaluation module is used for evaluating a target algorithm model by using the target labeling data set to obtain an evaluation result, wherein the evaluation result is used for being provided for a third user, and the first user, the second user and the third user forbid accessing the target labeling data set and the target algorithm model in the sandbox.

16. The apparatus of claim 15, wherein the first response information further comprises: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set;

the device further comprises:

the first destruction module is used for destroying the target labeling data set acquired by the sandbox according to the use strategy of the target labeling data set;

and the second destruction module is used for destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

17. The apparatus of claim 16,

the first destruction module comprises:

the first destruction submodule is used for destroying the target labeling data set under the condition that the number of times of using the target labeling data set reaches a preset first threshold value; and/or

The second destroying submodule is used for destroying the target labeling data set under the condition that the using time of the target labeling data set reaches a preset second threshold value;

the second destruction module comprises:

a third destruction submodule, configured to destroy the target algorithm model when the number of times that the target algorithm model is used reaches a predetermined third threshold; and/or

And the fourth destroy submodule is used for destroying the target algorithm model under the condition that the using time of the target algorithm model reaches a preset fourth threshold value.

18. The apparatus according to at least one of the claims 15 to 17,

the first obtaining module comprises:

the first obtaining submodule is used for the sandbox to obtain the target annotation data set stored in the cloud storage unit according to the storage address of the target annotation data set;

The second acquisition module includes:

and the second obtaining submodule is used for obtaining the target algorithm model stored in the cloud storage unit by the sandbox according to the storage address of the target algorithm model.

19. The apparatus according to at least one of claims 15-18, characterized in that the apparatus further comprises:

a first determining module, configured to determine whether the first user is authentic according to first authentication information, where the first authentication information is provided when the first user provides information of at least one annotation data set;

the second determining module is used for determining whether the second user is credible according to second identity verification information, and the second identity verification information is provided when the second user provides information of at least one algorithm model;

a third determining module, configured to determine whether the third user is trusted according to third authentication information, where the third authentication information is provided when the third user requests to use the target annotation data set and the target algorithm model.

20. The apparatus of claim 16, wherein the usage policy of the target annotation data set and the usage policy of the target algorithm model are determined according to the indication of the third user.

21. The apparatus according to at least one of claims 15-20, wherein the sandbox is supervised by a blockchain, the first response information is used to determine smart contracts made by the first user and the third user, and the second response information is used to determine smart contracts made by the second user and the third user.

22. A data processing apparatus, characterized in that the apparatus comprises:

a third receiving module, configured to receive first response information of a first user, where the first user provides at least one tagged data set, where the first response information includes a storage address of a target object determined by the first user, and the target object includes a target tagged data set requested to be used by a second user;

a fourth receiving module, configured to receive second response information of a second user, where the second user provides at least one algorithm model, the second response information includes a storage address of a target object determined by the second user, and the target object includes a target algorithm model provided by the second user;

the third obtaining module is used for obtaining the target annotation data set by the sandbox according to the first response information;

The fourth obtaining module is used for obtaining the target algorithm model by the sandbox according to the second response information;

and the training module is used for training the target algorithm model by using the target labeling data set to obtain a trained algorithm model, wherein the trained algorithm model is provided for the second user, and the first user and the second user forbid accessing the target labeling data set and the target algorithm model in the sandbox.

23. The apparatus of claim 22, wherein the first response information further comprises: the use strategy of the target labeling data set is used for determining the condition for destroying the target labeling data set;

the device further comprises:

the third destroying module is used for destroying the target labeling data set acquired by the sandbox according to the use strategy of the target labeling data set;

and the fourth destroying module is used for destroying the target algorithm model obtained by the sandbox according to the use strategy of the target algorithm model.

24. The apparatus of claim 23, wherein the third destruction module comprises:

a fifth destruction submodule, configured to destroy the target annotation data set when the number of times that the target annotation data set is used reaches a predetermined first threshold; and/or

A sixth destruction sub-module, configured to destroy the target annotation data set when the usage time for which the target annotation data set is used reaches a predetermined second threshold;

the fourth destruct module comprises:

a seventh destruction submodule, configured to destroy the target algorithm model when the number of times that the target algorithm model is used reaches a predetermined third threshold; and/or

An eighth destruction submodule, configured to destroy the target algorithm model when a usage time for which the target algorithm model is used reaches a predetermined fourth threshold value.

25. The apparatus according to at least one of claims 22-24, characterized in that the third obtaining module comprises:

the third obtaining submodule is used for obtaining the target annotation data set stored in the cloud storage unit by the sandbox according to the storage address of the target annotation data set;

The fourth obtaining module includes:

and the fourth obtaining submodule is used for obtaining the target algorithm model stored in the cloud storage unit by the sandbox according to the storage address of the target algorithm model.

26. The apparatus according to at least one of claims 22-25, characterized in that the apparatus further comprises:

a fourth determining module, configured to determine whether the first user is trusted according to first authentication information, where the first authentication information is provided when the first user provides information of at least one annotation data set;

and the fifth determining module is used for determining whether the second user is credible according to second authentication information, and the second authentication information is provided when the second user requests to use the target annotation data set.

27. The apparatus of claim 23, wherein the policy for use of the target annotation data set is determined in accordance with an indication from the second user.

28. The apparatus according to at least one of claims 22-27, wherein the sandbox is supervised by a blockchain, and the first response information is used to determine a smart contract generated by the first user and the second user.

29. A data processing apparatus, comprising:

A processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to carry out the instructions when executing the method of at least one of claims 1-7 or to carry out the method of at least one of claims 8-14.

30. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method of at least one of claims 1-7 or the method of at least one of claims 8-14.

31. A computer program product comprising computer readable code or a non-transitory computer readable storage medium carrying computer readable code which, when run in an electronic device, a processor in the electronic device performs the method of at least one of claims 1-7 or performs the method of at least one of claims 8-14.