CN114328480B - Data processing method and related device - Google Patents
Data processing method and related device Download PDFInfo
- Publication number
- CN114328480B CN114328480B CN202111494696.4A CN202111494696A CN114328480B CN 114328480 B CN114328480 B CN 114328480B CN 202111494696 A CN202111494696 A CN 202111494696A CN 114328480 B CN114328480 B CN 114328480B
- Authority
- CN
- China
- Prior art keywords
- log
- data
- negative feedback
- content
- characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 238000012549 training Methods 0.000 claims abstract description 60
- 238000000034 method Methods 0.000 claims abstract description 51
- 238000012545 processing Methods 0.000 claims description 123
- 238000012937 correction Methods 0.000 claims description 65
- 238000003860 storage Methods 0.000 claims description 41
- 238000004590 computer program Methods 0.000 claims description 20
- 238000007781 pre-processing Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 10
- 230000015654 memory Effects 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 description 18
- 238000013473 artificial intelligence Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 8
- 238000010801 machine learning Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data processing method and a related device, wherein the data processing method comprises the following steps: acquiring log data of a content recommendation system, wherein the log data comprises a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; the corresponding issued data characteristics of the issued log are stored in a database; inquiring the first sample data characteristic from the database according to the access log, and inquiring the second sample data characteristic from the database according to the negative feedback log; and training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model is used for correcting the negative feedback data characteristic corresponding to the negative feedback log. By the method and the device, the accuracy of the negative feedback data characteristic can be improved.
Description
Technical Field
The present application relates to the field of computer technology, and in particular, to a data processing method, a data processing apparatus, a computer device, a computer readable storage medium, and a computer program product.
Background
In various recommendation scenes such as public number recommendation, in order to enable recommended content to be effectively adopted and accepted by users, negative feedback prediction models are often utilized to predict and identify negative feedback of the users, so that negative feedback data features for training the negative feedback prediction models become an important data index in the recommendation scenes.
At present, after a sample is obtained online, the obtained sample is subjected to feature processing in an offline environment, and then the corresponding negative feedback data feature in the offline environment is obtained. Because of the conditions of large difference between the sample acquisition time and the time for determining the negative feedback data characteristics in the offline environment, the finally extracted negative feedback data characteristics are not accurate enough. Therefore, how to improve the accuracy of the negative feedback data features is a technical problem to be solved currently.
Disclosure of Invention
The embodiment of the application provides a data processing method, a data processing device, computer equipment, a computer readable storage medium and a computer program product, which can improve the accuracy of negative feedback data characteristics.
In one aspect, an embodiment of the present application provides a data processing method, where the method includes:
Acquiring log data of a content recommendation system, wherein the log data comprises a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; the corresponding issued data characteristics of the issued log are stored in a database;
inquiring the first sample data characteristic from the database according to the access log, and inquiring the second sample data characteristic from the database according to the negative feedback log;
and training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model is used for correcting the negative feedback data characteristic corresponding to the negative feedback log.
In one aspect, an embodiment of the present application provides a data processing apparatus, including:
the acquisition unit is used for acquiring log data of the content recommendation system, wherein the log data comprises a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; the corresponding issued data characteristics of the issued log are stored in a database;
The processing unit is used for inquiring the first sample data characteristic from the database according to the access log and inquiring the second sample data characteristic from the database according to the negative feedback log;
the processing unit is further used for training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model is used for correcting the negative feedback data characteristic corresponding to the negative feedback log.
In one possible implementation, the content set includes content i, which is any pushed content in the content set; the object set comprises N objects which receive the content i; i. n is a positive integer;
the issuing log comprises an issuing log item corresponding to the content i, wherein the issuing log item corresponding to the content i records the portrait characteristic of the content i and the portrait characteristics of N objects; the processing unit is also configured to perform the following operations:
Performing feature extraction processing on the issuing log item corresponding to the content i to obtain N issuing data features corresponding to the content i; each issuing data feature consists of the portrait feature of the content i and the portrait feature of one object in N objects;
and storing N downlink data features corresponding to the content i into a database.
In one possible implementation, the access log records information that the content i is subjected to an access operation by an object j after pushing, where the object j is any one of the N objects;
the processing unit queries the first sample data characteristic from the database according to the access log and is used for executing the following operations:
Extracting access data features corresponding to the content i from the access log, wherein the access data features comprise portrait features of the content i and portrait features of the object j;
inquiring in a database according to the access data characteristic corresponding to the content i to obtain a first downlink data characteristic corresponding to the content i matched with the access data characteristic corresponding to the content i;
The first downstream data characteristic is determined as a first sample data characteristic.
In a possible implementation manner, before the processing unit queries the database according to the access data feature corresponding to the content i, the processing unit is further configured to perform the following operations:
detecting click through rate of the content i;
and when the click through rate of the content i is greater than or equal to the click through rate threshold, executing the step of inquiring in the database according to the access data characteristic corresponding to the content i.
In one possible implementation, the negative feedback log records information that the content i is subjected to a negative feedback operation by an object k after being pushed, where the object k is any one of N objects;
the processing unit queries the second sample data characteristic from the database according to the negative feedback log and is used for executing the following operations:
Extracting negative feedback data characteristics corresponding to the content i from the negative feedback log, wherein the negative feedback data characteristics comprise portrait characteristics of the content i and portrait characteristics of the object k;
Inquiring in a database according to the negative feedback data characteristic corresponding to the content i to obtain a second issuing data characteristic corresponding to the content i, wherein the second issuing data characteristic is matched with the negative feedback data characteristic corresponding to the content i;
the second issue data characteristic is determined as a second sample data characteristic.
In one possible implementation, the processing unit trains the feature correction model using the first sample data feature and the second sample data feature for performing the following operations:
training a characteristic deviation correcting model by adopting the first sample data characteristic to obtain a trained characteristic deviation correcting model;
and performing fine adjustment processing on the trained characteristic deviation correcting model by adopting the second sample data characteristics to obtain the trained characteristic deviation correcting model.
In one possible implementation, the processing unit is further configured to perform the following operations:
Performing feature preprocessing on the first sample data features in a configuration processing mode to obtain preprocessed first sample data features, and performing feature preprocessing on the second sample data features in a configuration processing mode to obtain preprocessed second sample data features;
the configuration processing mode may include any one or more of the following: the method comprises the steps of converting enumerated type features into letter identification processing modes, constructing a feature dictionary processing mode, discretizing continuous features and normalizing features.
In one possible implementation, the processing unit is further configured to perform the following operations:
acquiring negative feedback data characteristics corresponding to the negative feedback log;
and calling the trained characteristic deviation correction model to correct the negative feedback data characteristics corresponding to the negative feedback log, so as to obtain consistency characteristics.
In one possible implementation, the processing unit is further configured to perform the following operations:
invoking a negative feedback pre-estimation model, and training the negative feedback pre-estimation model by utilizing consistency characteristics;
and when the trained negative feedback pre-estimation model meets the model convergence condition, determining the trained negative feedback model as the trained negative feedback pre-estimation model.
In one possible implementation, the processing unit is further configured to perform the following operations:
Invoking a trained negative feedback estimation model, and predicting data to be pushed in the content recommendation system to obtain target data;
deleting the target data from the data to be pushed;
And pushing the deleted data to be pushed to the target object.
In one aspect, an embodiment of the present application provides a computer apparatus, where the computer apparatus includes a memory and a processor, and the memory stores a computer program, and when the computer program is executed by the processor, causes the processor to execute the data processing method described above.
In one aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program that, when read and executed by a processor of a computer device, causes the computer device to perform the above-described data processing method.
In one aspect, embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the data processing method described above.
In the embodiment of the application, firstly, the log data of the content recommendation system can be obtained, wherein the log data can comprise a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; and the corresponding issuing data characteristics of the issuing log are stored in a database. The first sample data characteristic may then be queried from the database based on the access log and the second sample data characteristic may be queried from the database based on the negative feedback log. And then, training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model can be used for correcting the negative feedback data characteristic corresponding to the negative feedback log. Therefore, in the embodiment of the application, the first sample data characteristic and the second sample data characteristic are both obtained by inquiring from the database, and in this way, the first sample data characteristic and the second sample data characteristic can be both fixed as the issuing data characteristic in the issuing state; and then training a characteristic correction model based on the first sample data characteristic and the second sample data characteristic in the issuing state, and correcting the negative feedback data characteristic corresponding to the negative feedback log by using the trained characteristic correction model to finally obtain the issuing data characteristic in the issuing state. Therefore, the state of the negative feedback data characteristics of the content set when the negative feedback operation is executed after pushing can be rectified into a sending state, the problems of characteristic crossing and characteristic invalidation are avoided, the negative feedback data characteristics are more accurate, and the accuracy of the negative feedback data characteristics is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a data processing scheme according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a data processing system according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a database according to an embodiment of the present application;
FIG. 5 is a schematic flow chart of a model training method according to an embodiment of the present application;
FIG. 6 is a flow chart of another model training method according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
Fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application.
The embodiment of the application provides a data processing scheme which can improve the accuracy of negative feedback data characteristics, can be applied to various recommendation scenes such as public number recommendation, can train to obtain a negative feedback estimation model with good effect based on the accurate negative feedback data characteristics, and further improves the effectiveness of a content recommendation system. Referring to fig. 1, fig. 1 is a schematic diagram of a data processing scheme according to an embodiment of the present application, and as shown in fig. 1, the general principle of the data processing scheme is as follows: first, log data may be extracted from a content recommendation system (e.g., in an advertisement recommendation scenario, the content recommendation system may be an advertisement recommendation system) as a data source for data processing. The log data may include a issuing log, an access log, and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of access operations (such as clicking operations performed by a user after the content set is exposed) performed on the content set after pushing; the negative feedback log is used for recording information that the content set is subjected to a negative feedback operation (e.g., a complaint operation of a user, etc.) after pushing. In one possible implementation, the issue log in the data source may be sampled, and then the issue data feature corresponding to the sampled issue log is stored in the database. The first sample data characteristic may then be queried from the database based on the access log and the second sample data characteristic may be queried from the database based on the negative feedback log. Next, a feature correction model may be trained using the first sample data feature and the second sample data feature. And finally, a trained characteristic deviation correction model can be called to correct the negative feedback data characteristics corresponding to the negative feedback log, so as to obtain consistency characteristics. Subsequently, a big packet reporting tool (e.g., mmbizsec Tlrecfeaturereporter tool) may be used to report the consistency characteristics and the first sample data characteristics, and periodically generate training data of the negative feedback prediction model, and export the training data of the negative feedback prediction model to a storage platform WFS (a decentralised storage platform).
Therefore, in the embodiment of the application, the first sample data characteristic and the second sample data characteristic are all obtained by inquiring from the database, and in this way, the first sample data characteristic and the second sample data characteristic can be both fixed as the issuing data characteristic in the issuing state; and then training a characteristic correction model based on the first sample data characteristic and the second sample data characteristic in the issuing state, and correcting the negative feedback data characteristic corresponding to the negative feedback log by utilizing the trained characteristic correction model, wherein the finally obtained consistency characteristic is also the issuing data characteristic in the issuing state, so that the state correction of the negative feedback data characteristic when the content set is executed with the negative feedback operation after being pushed can be changed into the issuing state, the problems of characteristic crossing and characteristic invalidation are avoided, the consistency characteristic is more accurate, and the accuracy of the negative feedback data characteristic is improved.
The above mentioned data processing schemes are described in connection with technical terms to which the present application relates:
1. artificial intelligence:
Artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is the theory, method, technique, and application system that simulates, extends, and extends human intelligence using a digital computer or a machine controlled by a digital computer, perceives the environment, obtains knowledge, and uses the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision. The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
In one possible implementation, the data processing scheme of the present application may be combined with machine learning techniques in the field of artificial intelligence. For example, the feature correction model may be trained using machine learning techniques, the negative feedback predictive model may be trained using machine learning techniques, and so on. Among them, the machine learning (MACHINE LEARNING, ML) is a multi-domain interdisciplinary, and involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and the like. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, confidence networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.
2. Blockchain:
A blockchain (Blockchain) network is a network composed of a point-to-point network (P2P network) and a blockchain, and a blockchain refers to a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm, etc., which is essentially a decentralised database, and is a string of data blocks (or referred to as blocks) generated by association using a cryptographic method.
In one possible implementation, the data processing scheme of the present application may be combined with blockchain technology. For example, log data (including a release log, an access log, a negative feedback log, and the like) of the content recommendation system can be uploaded to a blockchain of the blockchain network for storage, so that internal data of the computer device is prevented from being tampered, and safety and privacy of the log data are improved.
3. Cloud technology:
Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside.
In one possible implementation manner, when the data processing scheme of the present application is executed, the issued data features corresponding to the issued log are stored in the database, and this process involves larger-scale calculation and requires larger calculation power and storage space, so in one possible implementation manner of the present application, the computer device can obtain enough calculation power and storage space through the cloud storage technology, and further execute the storage of the issued data features corresponding to the issued log related in the present application.
It can be appreciated that the data processing scheme provided by the embodiment of the application can be applied to various recommendation scenes such as public number recommendation. Specifically, the data processing scheme provided by the embodiment of the application can be utilized to call the trained characteristic deviation correction model to correct the negative feedback data characteristics corresponding to the negative feedback log in the log data, so as to obtain consistency characteristics, then the negative feedback prediction model is trained through the consistency characteristics, when the content recommendation system needs to push the data to be pushed to the object, the trained negative feedback prediction model can be called to perform prediction recognition on the data to be pushed, so that the target data which is possibly subjected to negative feedback operation by a user in the data to be pushed is recognized, the content recommendation system can delete or intercept the target data, and the deleted data to be pushed to the target object, so that the effectiveness of the content recommendation system can be improved, the recommended data can be pushed to the user in a targeted manner according to the user requirement, the possibility of the user to perform the negative feedback operation is reduced, and the user experience can be improved.
With reference to fig. 2, fig. 2 is a schematic diagram of a data processing system according to an embodiment of the present application. As shown in fig. 2, the schematic structural diagram of the data processing system may include: a terminal device 201 and a server 202. The terminal device 201 and the server 202 may be directly or indirectly connected through wired or wireless communication, which is not limited herein; in addition, the number of terminal devices 201 in the embodiment of the present application is also merely an example, and the present application is not limited thereto.
The terminal device 201 shown in fig. 2 may include, but is not limited to: a mobile phone, a tablet computer, a notebook computer, a palm computer, a Mobile Internet Device (MID) INTERNET DEVICE, an intelligent voice interaction device, an in-vehicle terminal, a roadside device, an aircraft, a wearable device, an intelligent home appliance, or a wearable device with a data processing function such as a smart watch, a smart bracelet, a pedometer, and the like.
The server 202 shown in fig. 2 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), basic cloud computing services such as big data and artificial intelligence platforms, and the like.
In one possible implementation manner, when the terminal device 201 and the server 202 jointly execute the data processing scheme in the present application, the terminal device 201 may obtain log data of the content recommendation system, where the log data includes a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; and the corresponding issuing data characteristics of the issuing log are stored in a database. Then, the terminal device 201 may send the log data to the server 202, and after the server 202 obtains the log data sent from the terminal device 201, the server 202 may query the database for the first sample data feature according to the access log, and query the database for the second sample data feature according to the negative feedback log. Next, the server 202 may train a feature deskew model using the first and second sample data features.
Finally, the server 202 may invoke the trained feature correction model to correct the negative feedback data feature corresponding to the negative feedback log, so as to obtain a consistency feature. Subsequently, the server 202 may send the consistency characteristics to the terminal device 201, and the terminal device 201 may train the negative feedback predictive model using the consistency characteristics.
It should be understood that the foregoing is merely illustrative of the steps that the terminal device 201 and the server 202 are responsible for performing, and the embodiments of the present application are not limited thereto. For example, querying a database for a first sample data characteristic based on the access log and querying a database for a second sample data characteristic based on the negative feedback log; training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic; and invoking the trained characteristic deviation correction model to correct the negative feedback data characteristic corresponding to the negative feedback log to obtain a consistency characteristic, wherein the processes can also be executed by the terminal equipment 201. For another example, after the server 202 sends the training feature correction model to the terminal device 201, the terminal device 201 executes the subsequent steps of calling the trained feature correction model to correct the negative feedback data feature corresponding to the negative feedback log, so as to obtain the consistency feature.
Further, the data processing system provided in FIG. 2 may be deployed at a node of a blockchain, e.g., server 202 and terminal device 201 may each be considered as a blockchain node device, together forming a blockchain network. Therefore, the data processing flow related in the application can be executed on the blockchain, so that the fairness and fairness of the data processing flow can be ensured, and the data processing flow can be provided with traceability, thereby improving the safety of the data processing flow.
It may be understood that the schematic diagram of the system architecture described in the embodiment of the present application is for more clearly describing the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application, and those skilled in the art can know that, with the evolution of the system architecture and the appearance of a new service scenario, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.
Based on the related description of the data processing scheme, the embodiment of the application provides a data processing method. Referring to fig. 3, fig. 3 is a schematic flow chart of a data processing method according to an embodiment of the present application, where the data processing method may be performed by the above-mentioned terminal device or server, or may be performed by both the terminal device and the server. For ease of explanation, the following description will take a computer device to execute the data processing method as an example. The data processing method may include the following steps S301 to S303:
S301: acquiring log data of a content recommendation system, wherein the log data comprises a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; and the corresponding issuing data characteristics of the issuing log are stored in a database.
In the embodiment of the application, the content set refers to a set of one or more pieces of content to be pushed; an object set refers to a set of one or more objects (e.g., users) that receive content to be pushed. The content recommendation system refers to: a background system that recommends a content set to an object set, in which specific content of the content set, specific content of the object set, a pushed timestamp, and the like may be recorded. For example, in an advertisement push scenario, the content push system may be an advertisement push system (e.g., an XXX advertisement platform); as another example, in a message pushing scenario, the content pushing system may be a social platform or the like.
The issuing log refers to issuing running water corresponding to an issuing state, and the issuing state refers to a state corresponding to pushing of a content set to an object in a content recommendation system. For example, when the advertisement recommendation system needs to recommend advertisement data to an object, when the advertisement recommendation system pushes the advertisement data to a timestamp (for example, 1 month, 1 day, 10:00) corresponding to one or more users (for example, user a, user B, user C, etc.) is a delivery status, a delivery pipeline may record: 1 month 1 day 10:00, targeted advertisement data is pushed to user a, user B, user C, etc.
The access log refers to exposure/click water corresponding to an exposure/click state, and the exposure state refers to a state corresponding to a content set when the content set is presented to an object set, and the click state refers to a state corresponding to a state when the object performs a click operation on a specific piece of content. For example, after the advertisement recommendation system recommends advertisement data to the object, a timestamp (for example, 1 month, 2 days, 10:00) corresponding to the advertisement data when the advertisement data is presented to the object is an exposure state; for another example, the corresponding timestamp (e.g., 10:01 for 1 month and 2 days) is the click state when the object performs the click operation on the exposed advertisement data. Then, the exposure/click stream may be recorded with: 1 month 2 days 10:01, the advertisement data is triggered by the user a, the user B, etc. information. It will be appreciated that the time interval between the exposure state and the click state is short, and thus in one possible implementation, the exposure state and the click state may be collectively referred to as the access state.
The negative feedback log refers to a negative feedback running water corresponding to a negative feedback state, wherein the negative feedback state refers to a state corresponding to a content set when a subject performs a negative feedback operation, and the negative feedback operation refers to an operation of a user that the user dislikes the content, and the negative feedback operation may include, but is not limited to: complaint operations of the user, dislike operations, and the like. For example, after the advertisement recommendation system recommends advertisement data to the object and presents the advertisement data to the object, the timestamp (for example, 10:02 on 1 month and 2 days) corresponding to the time when the object performs the negative feedback operation is the negative feedback state. Then, the negative feedback running water can be recorded with: 1 month 2 days 10:02, the advertisement data is triggered by the user a to perform negative feedback operation and the like.
In one possible implementation, for the issue log, the access log, and the negative feedback log included in the log data, the data links are typically: issue status- > exposure/click status- > negative feedback status. For example, in an advertisement recommendation scenario, the issuing log may record 10000 pieces of information for pushing the content set to the object set, the exposure log may record 3000 pieces of information for presenting the content set to the object after pushing, the clicking log may record 300 pieces of information for performing a clicking operation on the content set after pushing, and the negative feedback log may record 10 pieces of information for performing a negative feedback operation on the content set after pushing. And, the issuing log, the accessing log and the negative feedback log can all be according to: the storage format of the content set-object set-timestamp-behavior state is stored, for example, the storage format may be a table. Next, referring to tables 1-3, the storage formats of the issue log, the access log, and the negative feedback log are illustrated respectively.
TABLE 1 storage format for issue logs
| Content collection | Object set | Time stamp | Behavior state |
| Advertisement 1 | User 1, user 2, user 3 | 1 Month 2 days 10:00 | Issue status |
| Advertisement 2 | User 3, user 4, user 5 | 1 Month 1 day 10:00 | Issue status |
| Advertisement 3 | User 1, user 3, user 6 | 1 Month 1 day 10:00 | Issue status |
| ... | ... | ... | Issue status |
| Advertisement 10000 | User 2, user 5, user 6 | 1 Month 1 day 10:00 | Issue status |
It will be appreciated that, as shown in table 1, the advertisement recommendation system may push the same advertisement content to one or more objects simultaneously, for example, advertisement 1 may be pushed to user 1, user 2, and user 3 simultaneously (e.g., 1 month and 1 day 10:00), and the same object may receive one or more advertisement contents simultaneously, for example, user 1 may receive advertisement 1 and advertisement 3 simultaneously (e.g., 1 month and 1 day 10:00).
TABLE 2 storage formats of Access logs
| Content collection | Object set | Time stamp | Behavior state |
| Advertisement 1 | User 1, user 2 | 1 Month 1 day 10:01 | Click state |
| Advertisement 2 | User 4, user 5 | 1 Month 1 day 10:01 | Click state |
| Advertisement 3 | User 1, user 3, user 6 | 1 Month 1 day 10:01 | Click state |
| ... | ... | ... | ... |
| Advertisement 300 | User 2, user 5, user 6 | 1 Month 1 day 10:01 | Click state |
It will be appreciated that, as shown in Table 2, when certain advertising content is pushed to an object, only a portion of the object may perform a click operation. For example, the advertisement recommendation system pushes advertisement 1 to user 1, user 2, and user 3, but only user 1 and user 2 have clicked on the advertisement 1.
TABLE 3 storage format of negative feedback log
| Content collection | Object set | Time stamp | Behavior state |
| Advertisement 1 | User 1 | 1 Month 1 day 10:02 | Negative feedback state |
| Advertisement 2 | User 4 | 1 Month 1 day 10:02 | Negative feedback state |
| Advertisement 3 | User 1, user 3 | 1 Month 1 day 10:02 | Negative feedback state |
| ... | ... | ... | ... |
| Advertisement 10 | User 6 | 1 Month 1 day 10:02 | Negative feedback state |
It will also be appreciated that, as shown in Table 3, when certain advertising content is pushed to an object, and a click operation is performed by the object, there may be a smaller portion of the object performing a negative feedback operation (e.g., a complaint operation, etc.). For example, the advertisement recommendation system pushes advertisement 1 to user 1, user 2, and user 3, wherein user 1, user 2 clicked on the advertisement 1, and only user 1 performed a negative feedback operation for the advertisement 1.
In one possible implementation, the content set includes content i, which is any pushed content in the content set; the object set comprises N objects which receive the content i; i. n is a positive integer. The issuing log comprises an issuing log item corresponding to the content i, and the issuing log item corresponding to the content i records the portrait characteristic of the content i and the portrait characteristics of N objects. Then, the computer equipment can perform feature extraction processing on the issuing log item corresponding to the content i to obtain N issuing data features corresponding to the content i; each issuing data feature consists of the portrait feature of the content i and the portrait feature of one object in N objects; the computer device then stores the N pieces of the distribution data characteristics corresponding to the content i in a database.
Specifically, in the advertisement recommendation scenario, assuming that the content i is advertisement 1, the profile of advertisement 1 and the profile of N objects (e.g., user 1, user 2, and user 3) may be recorded in the delivery log entry corresponding to advertisement 1. For example, the advertisement 1 may be subjected to feature extraction processing using the feature extraction model to obtain the portrait feature x1 of the advertisement 1, and the users 1,2, and 3 may be subjected to feature extraction processing using the feature extraction model to obtain the portrait feature y1 of the user 1, the portrait feature y2 of the user 2, and the portrait feature y3 of the user 3, respectively. The feature extraction process may include, but is not limited to: the image feature of the target object may be defined by means of data mining, data processing, or the like: the number of tags to be given to the target object may be one or more, for example, the portrait features of user 1 may include: attribute characteristics (e.g., age, gender, etc.), interest characteristics, behavioral characteristics, psychological characteristics, and the like. The image feature of the target content is also: the number of tags that are tagged for the targeted content may be one or more, for example, the portrait characteristics of advertisement 1 may include: domain features (e.g., sports, food, politics, etc.), language features (e.g., chinese, english, korean, etc.), and the like.
In one possible implementation, the issue log entry may be stored in a database in the form of item (content) -user (object), where the database may be ttlkv database (time to live key value, a database with an automatic expiration function), that is, the database has a valid duration, and if the issue data feature stored in the database exceeds the valid duration, the issue data feature is deleted from the database. By the method, the problems of feature failure and the like caused by overlong storage time can be avoided, and the validity of the issued data features stored in the database can be ensured.
It should be noted that the storage format of the database may include, but is not limited to: table form, linked list form (as shown in fig. 4, fig. 4 is a schematic diagram of a database provided in the embodiment of the present application, where the characteristics of the delivered data stored in the database may be stored in the linked list form), and the embodiment of the present application is not limited in this way specifically. For example, assume that the storage format of the database is in tabular form, as shown in table 4 below:
TABLE 4 storage formats of database
| uer1 | user2 | ... | user N | |
| item1 | x1-y1 | x1-y2 | ... | x1-yN |
| item2 | x2-y1 | x2-y2 | x1-yM | |
| ... | ... | ... | ... | ... |
| item N | xN-y1 | x N-y2 | ... | ... |
As shown in table 4 above, for example, item1 may be an issue log item corresponding to advertisement 1, and the issue log item corresponding to advertisement 1 may include: image feature x1 of advertisement 1 and image features y1, y2... y N of N objects. For another example, item2 may be an issue log item corresponding to advertisement 2, and the issue log item corresponding to advertisement 2 may include: image feature x2 of advertisement 2 and image features y1, y2... y M of M objects. Wherein M is a positive integer, and M and N may be the same or different.
In another possible implementation manner, the computer device may sample the issue log first, and then store the issue data feature corresponding to the issue log after the sampling process into the database. The computer device stores the characteristics of the issued data corresponding to the issued log after the sampling processing to the detailed execution step corresponding to the database, and the embodiment of the present application will not be described herein again. Because the issuing log contains more contents, the operation burden of the computer can be reduced by the sampling processing mode, so that the data processing efficiency is improved.
S302: and inquiring the first sample data characteristic from the database according to the access log, and inquiring the second sample data characteristic from the database according to the negative feedback log.
In one possible implementation manner, before the computer device queries in the database according to the access data feature corresponding to the content i, the method may further include: the method comprises the steps that a computer device detects click through rate of content i; and when the click through rate of the content i is greater than or equal to the click through rate threshold, the computer equipment executes the step of inquiring in the database according to the access data characteristic corresponding to the content i. Specifically, when the content i is subjected to the Click operation, a CTR (Click-Through-Rate) of the content i may be detected, where the CTR of the content i may refer to a historical Click-Through Rate, and the historical Click-Through Rate of the content i may be according to: the number of times content i is clicked and the number of times content i is exposed are determined.
For example, the computer device may acquire the number of times content i is clicked during a past period of time (e.g., 24 hours), and acquire the number of times content i is exposed during the period of time. The computer device then determines the ratio between the number of times content i is clicked and the number of times content i is exposed as the click through rate of content i. For example, in the advertisement recommendation scenario, assume that content i is advertisement 1, and that the current time is 1 month, 2 days, 12:00, then 12 from 1 month 1 day can be obtained: 00 to 1 month and 2 days 12:00 the number of times that ad 1 was exposed (assuming 100 times) and the number of times that ad 1 was subjected to a click operation (assuming 20 times) within 24 hours, the computer device may determine ctr=20/100 for ad 1. Assuming that the click through rate threshold is 10%, the computer device may determine that the CTR (20%) of the advertisement 1 is greater than the click through rate threshold (10%), where the click through rate threshold may be set by a user in a user-defined manner, or may be set by a system, and the embodiment of the present application is not specifically limited to this. In this way, since the click through rate of the content i is detected in advance, the hit rate of the database can be ensured. Next, the computer device performs the step of querying the database according to the access data characteristic corresponding to content i.
In one possible implementation, the access log records information that the content i is to be accessed by the object j after pushing, the object j being any one of the N objects. That is, assuming that the advertisement 1 is pushed to the user 1, the user 2, and the user 3, and assuming that the user 1 and the user 2 perform a click operation on the advertisement 1, the object j refers to either one of the user 1 and the user 2, that is, the object j may refer to the user 1, and the object j may refer to the user 2.
Specifically, the computer device querying the database for the first sample data characteristic from the access log may include: first, the computer device extracts access data features corresponding to content i from the access log, the access data features including portrait features of content i and portrait features of object j. Then, the computer equipment queries in the database according to the access data characteristic corresponding to the content i to obtain a first downlink data characteristic corresponding to the content i, wherein the first downlink data characteristic corresponds to the content i and is matched with the access data characteristic corresponding to the content i. Finally, the computer device determines the first downstream data characteristic as a first sample data characteristic.
For example, in the advertisement recommendation scenario, assume that content i is advertisement 1 and object j is user 1. The computer device may extract from the access log the access data characteristic corresponding to advertisement 1, the access data characteristic corresponding to advertisement 1 comprising the portrayal characteristic of advertisement 1 and the portrayal characteristic of user 1, where the portrayal characteristic of advertisement 1 may comprise: the content key of advertisement 1 (i.e., the unique identification of advertisement 1), the portrait characteristics of user 1 may include: the object key of user 1 (i.e., the unique identification of user 1, such as identity (Identity document, id)). Then, the computer device queries in the database according to the unique identification of the advertisement 1 and the unique identification of the user 1, and obtains the portrait characteristic of the advertisement 1 in the "issuing state" and the portrait characteristic of the user 1 in the "issuing state". Finally, the computer device determines the acquired first downlink data feature (i.e. the portrait feature including the advertisement 1 in the "downlink state" and the portrait feature of the user 1 in the "downlink state") corresponding to the advertisement 1 as the first sample data feature.
In one possible implementation, the negative feedback log records information that the content i is subjected to a negative feedback operation by the object k after pushing, where the object k is any one of the N objects. It will be appreciated that k and j may be the same or different. That is, assuming that after the advertisement 1 is pushed to the user 1, the user 2 and the user 3, assuming that the user 1 performs a click operation on the advertisement 1 and the user 2 performs a negative feedback operation on the advertisement 1, k and j are different, that is, the object j refers to the user 1 and the object k refers to the user 2; for another example, assuming that after the advertisement 1 is pushed to the user 1, the user 2 and the user 3, and assuming that the user 1 performs a click operation on the advertisement 1 and the user 1 performs a negative feedback operation on the advertisement 1, k and j are the same, that is, the object j refers to the user 1 and the object k also refers to the user 1. Of course, it will be appreciated that if user 2 performs a negative feedback operation on advertisement 1, user 2 must perform a click operation on advertisement 1.
Specifically, the computer device querying the database for the second sample data characteristic based on the negative feedback log may include: first, the computer device extracts a negative feedback data feature corresponding to the content i from the negative feedback log, wherein the negative feedback data feature comprises a portrait feature of the content i and a portrait feature of the object k. Then, the computer equipment inquires in a database according to the negative feedback data characteristic corresponding to the content i to obtain a second issuing data characteristic corresponding to the content i, wherein the second issuing data characteristic corresponds to the content i and is matched with the negative feedback data characteristic corresponding to the content i; the second issue data characteristic is determined as a second sample data characteristic.
For example, in the advertisement recommendation scenario, assume that content i is advertisement 1 and object k is user 2. The computer device may extract from the access log the negative feedback data characteristic corresponding to advertisement 1, the negative feedback data characteristic corresponding to advertisement 1 comprising the portraiture characteristic of advertisement 1 and the portraiture characteristic of user 2, where the portraiture characteristic of advertisement 1 may comprise: the content key of advertisement 1 (i.e., the unique identification of advertisement 1), the portrait characteristics of user 2 may include: the object key of user 2 (i.e., the unique identification of user 2, such as identity (Identity document, id)). Then, the computer device queries in the database according to the unique identification of the advertisement 1 and the unique identification of the user 2, and obtains the portrait characteristic of the advertisement 1 in the "down state" and the portrait characteristic of the user 2 in the "down state". Finally, the computer device determines the acquired second distribution data feature (i.e., including the portrait feature of advertisement 1 in the "distribution state" and the portrait feature of user 2 in the "distribution state") corresponding to advertisement 1 as a second sample data feature.
It may be understood that, for any pushed content in the content set, the first downlink data feature and the second downlink data feature corresponding to the pushed content may be determined by referring to the first downlink data feature corresponding to the determined content i and the second downlink data feature corresponding to the determined content i, which are not described herein again in detail.
S303: and training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model is used for correcting the data characteristic corresponding to the negative feedback log.
In the embodiment of the application, the time interval between the time stamp corresponding to the "issuing state" and the time stamp corresponding to the "exposing state/clicking state/negative feedback state" is long (for example, 12 hours, even 1 day, etc.), but the time interval between the time stamp corresponding to the "exposing state", the time stamp corresponding to the "clicking state" and the time stamp corresponding to the "negative feedback state" is basically within the minute level, and the minute level refers to that the time interval takes minutes as a measurement unit, for example, the time interval between the time stamp corresponding to the "exposing state" and the time stamp corresponding to the "clicking state" is 1 minute; for another example, the time interval between the time stamp corresponding to the click state "and the time stamp corresponding to the negative feedback state" is 2 minutes, and so on. Based on the above analysis, in the embodiment of the present application, training samples of the feature correction model may be obtained in two ways:
(1) Training sample a: refers to the first sample data feature obtained by hit query from the database according to the access log in the "access state". It should be noted that, since the time interval between the time stamp corresponding to the "access state (i.e., exposure state/click state)" and the time stamp corresponding to the "negative feedback state" is short (e.g., 1 minute). Therefore, in the embodiment of the application, the first sample data characteristic obtained by the hit query from the database can be directly used as the sample data characteristic in the 'negative feedback state' by using the access log in the 'access state'. Then the first sample data may be characterized as a training sample of the feature correction model.
(2) Training sample B: and the second sample data characteristic is obtained by hit query from the database according to the negative feedback log in the 'negative feedback state'. It should be noted that, in the embodiment of the present application, the ratio between the exposure number of the content exposed to the object set and the negative feedback number of the content executed with the negative feedback operation after exposure is 1000:4, a step of; or the ratio between the exposure quantity of the content exposed to the object set and the negative feedback quantity of the content subjected to clicking operation and negative feedback operation after exposure is 1000:4. therefore, it can be understood that the second sample data feature obtained by the negative feedback log hit query in the embodiment of the present application is a relatively accurate sample data feature in the "negative feedback state". Then, the second sample data may be characterized as a training sample at the fine tune of the feature correction model.
It can be understood that the trained feature correction model in the embodiment of the present application has the following functions: the feature in the other state is corrected to the feature in the issued state. In the embodiment of the application, the negative feedback data characteristics corresponding to the negative feedback log can be fully reported because the negative feedback log occupies less amount in the issuing log, and then the trained characteristic correction model is called to correct the negative feedback data characteristics corresponding to the fully reported negative feedback log, so as to obtain the consistency characteristics. That is, in the embodiment of the application, the negative feedback data characteristics obtained in real time in the negative feedback state can be corrected by using the trained characteristic correction model, so as to obtain the negative feedback data characteristics (namely, consistency characteristics) in the issuing state.
In the embodiment of the application, firstly, the log data of the content recommendation system can be obtained, wherein the log data can comprise a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; and the corresponding issuing data characteristics of the issuing log are stored in a database. The first sample data characteristic may then be queried from the database based on the access log and the second sample data characteristic may be queried from the database based on the negative feedback log. Next, a feature correction model may be trained using the first sample data feature and the second sample data feature. And finally, a trained characteristic deviation correction model can be called to correct the negative feedback data characteristics corresponding to the negative feedback log, so as to obtain consistency characteristics. Therefore, in the embodiment of the application, the first sample data characteristic and the second sample data characteristic are both obtained by inquiring from the database, and in this way, the first sample data characteristic and the second sample data characteristic can be both fixed as the issuing data characteristic in the issuing state; and then training a characteristic correction model based on the first sample data characteristic and the second sample data characteristic in the issuing state, and correcting the negative feedback data characteristic corresponding to the negative feedback log by utilizing the trained characteristic correction model, wherein the finally obtained consistency characteristic is also the issuing data characteristic in the issuing state, so that the state correction of the negative feedback data characteristic when the content set is executed with the negative feedback operation after being pushed can be changed into the issuing state, the problems of characteristic crossing and characteristic invalidation are avoided, the consistency characteristic is more accurate, and the accuracy of the negative feedback data characteristic is improved.
Referring to fig. 5, fig. 5 is a schematic flow chart of a model training method according to an embodiment of the application, where the model training method may be executed by a computer device. The model training method may include the following steps S501 to S504:
S501: and acquiring log data of the content recommendation system.
The log data comprises a issuing log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; and the corresponding issuing data characteristics of the issuing log are stored in a database. It should be noted that, for the explanation of the issue log, the access log and the negative feedback log, the description in step S301 in the embodiment of fig. 3 may be specifically referred to, and the embodiments of the present application are not described herein again.
S502: and inquiring the first sample data characteristic from the database according to the access log, and inquiring the second sample data characteristic from the database according to the negative feedback log.
In the embodiment of the present application, it should also be noted that, in the detailed execution process that the computer device queries the first sample data feature from the database according to the access log and the computer device queries the second sample data feature from the database according to the negative feedback log, the step executed by the computer device in step S302 in the embodiment of fig. 3 may be specifically referred to, and the embodiment of the present application is not described herein again.
S503: and training the characteristic deviation correcting model by adopting the first sample data characteristic to obtain a trained characteristic deviation correcting model.
Referring to fig. 6, fig. 6 is a flow chart of another model training method according to an embodiment of the application. As shown in fig. 6, based on the description of step S303 in the foregoing embodiment, in the embodiment of the present application, the feature correction model may be trained using the first sample data feature, to obtain a trained feature correction model.
In the embodiment of the application, the characteristic deviation rectifying model may refer to a neural network model with any structure, for example, the neural network model may include, but is not limited to: DNN (Deep Neural Networks, deep neural network) model, LSTM (Long Short-Term Memory network) model, GRU (Gated Recurrent Neural network ) model and the like, the application does not limit the model structure of the characteristic deviation correcting model specifically. In one possible implementation, the embodiment of the present application may use a DNN model as the feature correction model in consideration of the operation efficiency of the model.
It should be noted that, in the embodiment of the present application, the feature correction model to be obtained by training has the following functions: the feature in the other state is corrected to the feature in the issued state. Therefore, the predicted feature refers to a feature corresponding to the "issued state" obtained after feature correction is performed on the first sample data feature (i.e., the training sample a described above refers to the first sample data feature obtained after the query is hit from the database according to the access log in the "access state") in the "negative feedback state" by the feature correction model, which is approximately the first sample data feature in the "negative feedback state".
In one possible implementation, the computer device training a feature correction model using the first sample data features may include: first, a computer device obtains a first tag characteristic of a first sample data characteristic. The computer device may then input the first sample data characteristic into the characteristic rectification model, outputting a first predicted characteristic corresponding to the first sample data characteristic. Finally, the computer equipment adjusts model parameters of the characteristic correcting model according to the first prediction feature corresponding to the first sample data feature and the difference between the first label features of the first sample data feature, and if the adjusted characteristic correcting model meets the first model convergence condition, the training of the characteristic correcting model is stopped, and the trained characteristic correcting model is obtained.
The first model convergence condition may refer to: when the training times of the feature correction model reach a preset training threshold, for example, 100 times, the feature correction model can be considered to meet the first model convergence condition, that is, the feature correction model after training for 100 times is used as the feature correction model after training. Or when the difference data between the predicted features and the tag features predicted by the model is smaller than the error threshold, the feature deviation correcting model can be considered to meet the first model convergence condition. And then or when the change between the predicted features obtained by training the feature correction model twice is smaller than a change threshold, the feature correction model can be considered to meet the convergence condition of the first model.
In another possible implementation, the computer device may perform feature preprocessing on the first sample data feature by using a configurable processing manner (for example, performing lib normalization processing on the feature by using a feature factory), to obtain a preprocessed first sample data feature. Then, the computer equipment adopts the preprocessed first sample data characteristic training characteristic correction model to obtain the trained characteristic correction model. The configuration processing mode includes but is not limited to: converting the enumerated type of features into one or more of a letter identification processing mode, a feature dictionary construction processing mode, a continuous feature discretization processing mode and a feature normalization processing mode. Among other things, the feature of the enumeration type may generally refer to a constant in life, i.e. features that may be exhaustive by way of example, and may be converted into an alphabetical identification according to a programming specification. For example, the enumerated type is characterized by "twelve months", and the converted to letter identifier may be characterized as: jan (month one), feb (month two), mar (month three), apr (month four), may (month five), jun (month six), july (month seven), aug (month eight), sep (month nine), oct (month ten), nov (month ten), dec (month ten). As another example, if the enumerated type is characterized by "seven days of a week", the characteristic after conversion to letter identification may be: mon (monday), tue (tuesday), wed (wednesday), thu (thursday), fri (friday), sat (friday), sun (sunday). Also, if the enumerated type is characterized by "red, green, and blue", the feature after being converted into the letter identifier may be: red (Red), green (Green), blue (Blue), etc. The feature preprocessing is carried out on the first sample data features in a collocation mode, so that planning processing in the feature processing process can be ensured, the data processing efficiency can be improved, and further, the normalized first sample data features are utilized to train a feature correction model, and the model training efficiency can be improved.
S504: and performing fine adjustment processing on the trained characteristic deviation correcting model by adopting the second sample data characteristics to obtain the trained characteristic deviation correcting model.
In the embodiment of the present application, as can be seen from the above description, the second sample data feature refers to: the second issued data feature obtained by hit query from the database according to the negative feedback log in the "negative feedback state" is a relatively accurate sample data feature in the "negative feedback state", so as shown in fig. 6, the embodiment of the present application may use the second sample data feature to perform fine tuning processing on the feature correction model trained in the step S503, thereby obtaining a more accurate feature correction model.
In one possible implementation manner, the fine tuning processing is performed on the trained feature deviation rectification model by using the second sample data feature by the computer device to obtain the trained feature deviation rectification model, which may include: first, the computer device obtains a second tag characteristic of a second sample data characteristic. The computer device may then input the second sample data feature into the trained feature correction model, outputting a second predicted feature corresponding to the second sample data feature. And finally, the computer equipment carries out fine adjustment processing on model parameters of the trained feature deviation correcting model according to the second prediction features corresponding to the second sample data features and the differences between the second label features of the second sample data features to obtain the trained feature deviation correcting model.
In another possible implementation manner, the computer device may perform feature preprocessing on the second sample data feature in a configurable processing manner, so as to obtain a preprocessed second sample data feature. Then, the computer equipment adopts the preprocessed second sample data characteristic training characteristic correction model to obtain the trained characteristic correction model. The configuration processing mode includes but is not limited to: converting the enumerated type of features into one or more of a letter identification processing mode, a feature dictionary construction processing mode, a continuous feature discretization processing mode and a feature normalization processing mode. The feature preprocessing is carried out on the second sample data features in a collocation mode, so that planning processing in the feature processing process can be ensured, the data processing efficiency can be improved, and further, the normalized first sample data features are utilized to train a feature correction model, and the model training efficiency can be improved.
By the mode, the production and the acquisition of the first sample data characteristic and the second sample data characteristic are unified into on-line environment production, and two sets of on-line and off-line characteristic production systems are not required to be maintained at the same time, so that the characteristic consistency of the production environment and the on-line environment can be ensured. Further, as the feature deviation rectifying model is added, the accuracy of the consistency feature can be further improved.
In one possible implementation manner, the computer device may invoke the trained feature correction model to perform feature correction processing on the negative feedback data feature corresponding to the negative feedback log, so as to obtain a consistency feature. Specifically, because the negative feedback log occupies less of the issuing log, the embodiment of the application can report the negative feedback data characteristics corresponding to the negative feedback log in full quantity, and then call the trained characteristic correction model to correct the negative feedback data characteristics corresponding to the negative feedback log reported in full quantity, so as to obtain the consistency characteristics. That is, in the embodiment of the application, the negative feedback data characteristics obtained in real time in the negative feedback state can be corrected by using the trained characteristic correction model, so as to obtain the negative feedback data characteristics (namely, consistency characteristics) in the issuing state.
Then, the consistency feature obtained by the embodiment of the application can be used as a training sample of the negative feedback predictive model. Specifically, the training of the negative feedback predictive model by the computer device using the consistency feature may include: firstly, invoking a negative feedback pre-estimation model by computer equipment, and training the negative feedback pre-estimation model by utilizing consistency characteristics; and then, when the trained negative feedback pre-estimation model meets the second model convergence condition, determining the trained negative feedback model as the trained negative feedback pre-estimation model. In another possible implementation manner, the first sample data feature may also be used as a training sample of the negative feedback predictive model, that is, the computer device also trains the negative feedback predictive model with the consistency feature and the first sample data feature, and then obtains a trained negative feedback predictive model.
In the embodiment of the present application, the negative feedback pre-estimation model may refer to a neural network model with any network structure, for example, the negative feedback pre-estimation model may be: DNN model, LSTM model, etc. Specifically, the computer device invoking the negative feedback predictive model, and training the negative feedback predictive model using the consistency feature may include: firstly, the computer equipment can acquire the tag characteristics corresponding to the consistency characteristics; then, the computer equipment inputs the consistency characteristics into a negative feedback pre-estimated model, and outputs the predicted characteristics corresponding to the consistency characteristics; finally, the computer equipment adjusts model parameters of the negative feedback pre-estimated model according to the label characteristics corresponding to the consistency characteristics and the differences between the prediction characteristics corresponding to the consistency characteristics.
In one possible implementation manner, in various recommendation scenes such as public number recommendation and advertisement recommendation, the computer equipment can call a trained negative feedback prediction model to predict data to be pushed in the content recommendation system so as to obtain target data; then, the computer equipment can delete the target data from the data to be pushed; and finally, pushing the deleted data to be pushed to the target object. For example, in the embodiment of the present application, for the data to be pushed in the content recommendation system, before the data to be pushed is pushed and sent to the user, the trained negative feedback prediction model is utilized to predict and obtain the target data which may be subjected to the negative feedback operation by the user, and then the target data is intercepted in advance. And finally, recommending the data to be pushed after intercepting the target data to the user, and in this way, the effectiveness of the content recommendation system can be improved, so that the recommended data can be pushed to the user in a targeted manner according to the user demand, the possibility of the user executing negative feedback operation is reduced, and the user experience can be improved. Because the training sample (i.e. consistency characteristic) of the negative feedback pre-estimation model is accurate, the negative feedback pre-estimation model obtained by training is more accurate, and finally, the prediction effect of the trained negative feedback pre-estimation model put into use on line is also better, so that the benefit of the content recommendation system is improved.
It will be appreciated that in particular embodiments of the present application, data relating to user information (e.g., identification, nickname, etc.) may be subject to user approval or consent when the above embodiments of the present application are applied to particular products or technologies, and that the collection, use, and processing of the relevant data may be subject to relevant national and regional laws and regulations and standards.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. The data processing apparatus 700 may be applied to a computer device corresponding to the above-described embodiment. The data processing apparatus 700 may be a computer program (comprising program code) running in a computer device, for example the data processing apparatus 700 is an application software; the device can be used for executing corresponding steps in the method provided by the embodiment of the application. The data processing apparatus 700 may include:
An obtaining unit 701, configured to obtain log data of a content recommendation system, where the log data includes a sending log, an access log, and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; the corresponding issued data characteristics of the issued log are stored in a database;
A processing unit 702, configured to query the database for the first sample data feature according to the access log, and query the database for the second sample data feature according to the negative feedback log;
The processing unit 702 is further configured to train a feature correction model using the first sample data feature and the second sample data feature, where the trained feature correction model is used to correct the negative feedback data feature corresponding to the negative feedback log.
In one possible implementation, the content set includes content i, which is any pushed content in the content set; the object set comprises N objects which receive the content i; i. n is a positive integer;
The issuing log comprises an issuing log item corresponding to the content i, wherein the issuing log item corresponding to the content i records the portrait characteristic of the content i and the portrait characteristics of N objects; the processing unit 702 is further configured to perform the following operations:
Performing feature extraction processing on the issuing log item corresponding to the content i to obtain N issuing data features corresponding to the content i; each issuing data feature consists of the portrait feature of the content i and the portrait feature of one object in N objects;
and storing N downlink data features corresponding to the content i into a database.
In one possible implementation, the access log records information that the content i is subjected to an access operation by an object j after pushing, where the object j is any one of the N objects;
The processing unit 702 queries the database for the first sample data feature according to the access log, for performing the following operations:
Extracting access data features corresponding to the content i from the access log, wherein the access data features comprise portrait features of the content i and portrait features of the object j;
inquiring in a database according to the access data characteristic corresponding to the content i to obtain a first downlink data characteristic corresponding to the content i matched with the access data characteristic corresponding to the content i;
The first downstream data characteristic is determined as a first sample data characteristic.
In a possible implementation manner, before the processing unit 702 performs the query in the database according to the access data feature corresponding to the content i, the processing unit is further configured to perform the following operations:
detecting click through rate of the content i;
and when the click through rate of the content i is greater than or equal to the click through rate threshold, executing the step of inquiring in the database according to the access data characteristic corresponding to the content i.
In one possible implementation, the negative feedback log records information that the content i is subjected to a negative feedback operation by an object k after being pushed, where the object k is any one of N objects;
the processing unit 702 queries the database for the second sample data feature according to the negative feedback log for performing the following operations:
Extracting negative feedback data characteristics corresponding to the content i from the negative feedback log, wherein the negative feedback data characteristics comprise portrait characteristics of the content i and portrait characteristics of the object k;
Inquiring in a database according to the negative feedback data characteristic corresponding to the content i to obtain a second issuing data characteristic corresponding to the content i, wherein the second issuing data characteristic is matched with the negative feedback data characteristic corresponding to the content i;
the second issue data characteristic is determined as a second sample data characteristic.
In one possible implementation, the processing unit 702 trains a feature rectification model using the first sample data features and the second sample data features for performing the following operations:
training a characteristic deviation correcting model by adopting the first sample data characteristic to obtain a trained characteristic deviation correcting model;
and performing fine adjustment processing on the trained characteristic deviation correcting model by adopting the second sample data characteristics to obtain the trained characteristic deviation correcting model.
In one possible implementation, the processing unit 702 is further configured to perform the following operations:
Performing feature preprocessing on the first sample data features in a configuration processing mode to obtain preprocessed first sample data features, and performing feature preprocessing on the second sample data features in a configuration processing mode to obtain preprocessed second sample data features;
the configuration processing mode may include any one or more of the following: the method comprises the steps of converting enumerated type features into letter identification processing modes, constructing a feature dictionary processing mode, discretizing continuous features and normalizing features.
In one possible implementation, the processing unit 702 is further configured to perform the following operations:
acquiring negative feedback data characteristics corresponding to the negative feedback log;
and calling the trained characteristic deviation correction model to correct the negative feedback data characteristics corresponding to the negative feedback log, so as to obtain consistency characteristics.
In one possible implementation, the processing unit 702 is further configured to perform the following operations:
invoking a negative feedback pre-estimation model, and training the negative feedback pre-estimation model by utilizing consistency characteristics;
and when the trained negative feedback pre-estimation model meets the model convergence condition, determining the trained negative feedback model as the trained negative feedback pre-estimation model.
In one possible implementation, the processing unit 702 is further configured to perform the following operations:
Invoking a trained negative feedback estimation model, and predicting data to be pushed in the content recommendation system to obtain target data;
deleting the target data from the data to be pushed;
And pushing the deleted data to be pushed to the target object.
In the embodiment of the application, firstly, the log data of the content recommendation system can be obtained, wherein the log data can comprise a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; and the corresponding issuing data characteristics of the issuing log are stored in a database. The first sample data characteristic may then be queried from the database based on the access log and the second sample data characteristic may be queried from the database based on the negative feedback log. And then, training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model can be used for correcting the negative feedback data characteristic corresponding to the negative feedback log. Therefore, in the embodiment of the application, the first sample data characteristic and the second sample data characteristic are both obtained by inquiring from the database, and in this way, the first sample data characteristic and the second sample data characteristic can be both fixed as the issuing data characteristic in the issuing state; and then training a characteristic correction model based on the first sample data characteristic and the second sample data characteristic in the issuing state, and correcting the negative feedback data characteristic corresponding to the negative feedback log by using the trained characteristic correction model to finally obtain the issuing data characteristic in the issuing state. Therefore, the state of the negative feedback data characteristics of the content set when the negative feedback operation is executed after pushing can be rectified into a sending state, the problems of characteristic crossing and characteristic invalidation are avoided, the negative feedback data characteristics are more accurate, and the accuracy of the negative feedback data characteristics is improved.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the application. The computer device 800 is configured to perform the steps performed by the computer device in the foregoing method embodiment, where the computer device 800 includes: one or more processors 810; one or more input devices 820, one or more output devices 830, and a memory 840. The processor 810, input device 820, output device 1330, and memory 840 are connected by a bus 850. The memory 840 is used to store a computer program comprising program instructions, and the processor 810 is used to invoke the program instructions stored in the memory 840 to perform the following operations:
Acquiring log data of a content recommendation system, wherein the log data comprises a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; the corresponding issued data characteristics of the issued log are stored in a database;
inquiring the first sample data characteristic from the database according to the access log, and inquiring the second sample data characteristic from the database according to the negative feedback log;
and training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model is used for correcting the negative feedback data characteristic corresponding to the negative feedback log.
In one possible implementation, the content set includes content i, which is any pushed content in the content set; the object set comprises N objects which receive the content i; i. n is a positive integer;
The issuing log comprises an issuing log item corresponding to the content i, wherein the issuing log item corresponding to the content i records the portrait characteristic of the content i and the portrait characteristics of N objects; the processor 810 is also configured to perform the following operations:
Performing feature extraction processing on the issuing log item corresponding to the content i to obtain N issuing data features corresponding to the content i; each issuing data feature consists of the portrait feature of the content i and the portrait feature of one object in N objects;
and storing N downlink data features corresponding to the content i into a database.
In one possible implementation, the access log records information that the content i is subjected to an access operation by an object j after pushing, where the object j is any one of the N objects;
The processor 810 queries the database for the first sample data characteristic based on the access log for:
Extracting access data features corresponding to the content i from the access log, wherein the access data features comprise portrait features of the content i and portrait features of the object j;
inquiring in a database according to the access data characteristic corresponding to the content i to obtain a first downlink data characteristic corresponding to the content i matched with the access data characteristic corresponding to the content i;
The first downstream data characteristic is determined as a first sample data characteristic.
In one possible implementation, the processor 810 is further configured to, before performing a query in the database according to the access data feature corresponding to the content i:
detecting click through rate of the content i;
and when the click through rate of the content i is greater than or equal to the click through rate threshold, executing the step of inquiring in the database according to the access data characteristic corresponding to the content i.
In one possible implementation, the negative feedback log records information that the content i is subjected to a negative feedback operation by an object k after being pushed, where the object k is any one of N objects;
The processor 810 queries the database for the second sample data characteristic based on the negative feedback log for performing the following operations:
Extracting negative feedback data characteristics corresponding to the content i from the negative feedback log, wherein the negative feedback data characteristics comprise portrait characteristics of the content i and portrait characteristics of the object k;
Inquiring in a database according to the negative feedback data characteristic corresponding to the content i to obtain a second issuing data characteristic corresponding to the content i, wherein the second issuing data characteristic is matched with the negative feedback data characteristic corresponding to the content i;
the second issue data characteristic is determined as a second sample data characteristic.
In one possible implementation, the processor 810 trains a feature correction model using the first sample data features and the second sample data features for performing the following operations:
training a characteristic deviation correcting model by adopting the first sample data characteristic to obtain a trained characteristic deviation correcting model;
and performing fine adjustment processing on the trained characteristic deviation correcting model by adopting the second sample data characteristics to obtain the trained characteristic deviation correcting model.
In one possible implementation, the processor 810 is further configured to:
Performing feature preprocessing on the first sample data features in a configuration processing mode to obtain preprocessed first sample data features, and performing feature preprocessing on the second sample data features in a configuration processing mode to obtain preprocessed second sample data features;
the configuration processing mode may include any one or more of the following: the method comprises the steps of converting enumerated type features into letter identification processing modes, constructing a feature dictionary processing mode, discretizing continuous features and normalizing features.
In one possible implementation, the processor 810 is further configured to:
acquiring negative feedback data characteristics corresponding to the negative feedback log;
and calling the trained characteristic deviation correction model to correct the negative feedback data characteristics corresponding to the negative feedback log, so as to obtain consistency characteristics.
In one possible implementation, the processor 810 is further configured to:
invoking a negative feedback pre-estimation model, and training the negative feedback pre-estimation model by utilizing consistency characteristics;
and when the trained negative feedback pre-estimation model meets the model convergence condition, determining the trained negative feedback model as the trained negative feedback pre-estimation model.
In one possible implementation, the processor 810 is further configured to:
Invoking a trained negative feedback estimation model, and predicting data to be pushed in the content recommendation system to obtain target data;
deleting the target data from the data to be pushed;
And pushing the deleted data to be pushed to the target object.
In the embodiment of the application, firstly, the log data of the content recommendation system can be obtained, wherein the log data can comprise a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after pushing; the negative feedback log is used for recording information of the content set which is subjected to negative feedback operation after being pushed; and the corresponding issuing data characteristics of the issuing log are stored in a database. The first sample data characteristic may then be queried from the database based on the access log and the second sample data characteristic may be queried from the database based on the negative feedback log. And then, training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model can be used for correcting the negative feedback data characteristic corresponding to the negative feedback log. Therefore, in the embodiment of the application, the first sample data characteristic and the second sample data characteristic are both obtained by inquiring from the database, and in this way, the first sample data characteristic and the second sample data characteristic can be both fixed as the issuing data characteristic in the issuing state; and then training a characteristic correction model based on the first sample data characteristic and the second sample data characteristic in the issuing state, and correcting the negative feedback data characteristic corresponding to the negative feedback log by using the trained characteristic correction model to finally obtain the issuing data characteristic in the issuing state. Therefore, the state of the negative feedback data characteristics of the content set when the negative feedback operation is executed after pushing can be rectified into a sending state, the problems of characteristic crossing and characteristic invalidation are avoided, the negative feedback data characteristics are more accurate, and the accuracy of the negative feedback data characteristics is improved.
Furthermore, it should be noted here that: the embodiment of the present application further provides a computer storage medium, in which a computer program is stored, and the computer program includes program instructions, when executed by a processor, can perform the method in the corresponding embodiment, so that a detailed description will not be given here. For technical details not disclosed in the embodiments of the computer storage medium according to the present application, please refer to the description of the method embodiments of the present application. As an example, the program instructions may be deployed on one computer device or executed on multiple computer devices at one site or distributed across multiple sites and interconnected by a communication network.
According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device can perform the method in the foregoing corresponding embodiment, and therefore, a detailed description will not be given here.
Those skilled in the art will appreciate that implementing all or part of the above-described embodiment methods may be accomplished by way of a computer program for instructing relevant hardware, where the program may be stored on a computer readable storage medium, and where the program, when executed, may comprise the embodiment flow of the above-described methods. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random-access Memory (Random Access Memory, RAM), or the like.
The foregoing disclosure is illustrative of the present application and is not to be construed as limiting the scope of the application, which is defined by the appended claims.
Claims (14)
1. A method of data processing, comprising:
Acquiring log data of a content recommendation system, wherein the log data comprises a issuing log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after being pushed; the negative feedback log is used for recording information that the content set is subjected to negative feedback operation after being pushed; the corresponding issued data characteristics of the issued log are stored in a database;
inquiring a first sample data characteristic from the database according to the access log, and inquiring a second sample data characteristic from the database according to the negative feedback log;
And training a characteristic deviation correcting model by adopting the first sample data characteristic and the second sample data characteristic, wherein the trained characteristic deviation correcting model is used for correcting the negative feedback data characteristic corresponding to the negative feedback log.
2. The method of claim 1, wherein the set of content comprises content i, content i being any pushed content in the set of content; the object set contains N objects which receive the content i; i. n is a positive integer;
The issuing log comprises an issuing log item corresponding to a content i, wherein the issuing log item corresponding to the content i records portrait characteristics of the content i and portrait characteristics of the N objects; the method further comprises the steps of:
Performing feature extraction processing on the issuing log item corresponding to the content i to obtain N issuing data features corresponding to the content i; each issuing data feature consists of the portrait feature of the content i and the portrait feature of one object in N objects;
and storing the N downlink data features corresponding to the content i into a database.
3. The method of claim 2, wherein the access log records information that the content i was subject to an access operation after pushing by subject j, the subject j being any one of the N subjects;
the querying the first sample data feature from the database according to the access log includes:
Extracting access data features corresponding to the content i from the access log, wherein the access data features comprise portrait features of the content i and portrait features of objects j;
Inquiring in the database according to the access data characteristic corresponding to the content i to obtain a first downlink data characteristic corresponding to the content i, wherein the first downlink data characteristic corresponds to the content i and is matched with the access data characteristic corresponding to the content i;
the first downlink data characteristic is determined as a first sample data characteristic.
4. The method of claim 3, wherein prior to querying the database according to the access data characteristic corresponding to the content i, further comprising:
Detecting click through rate of the content i;
And when the click through rate of the content i is greater than or equal to a click through rate threshold, executing the step of inquiring in the database according to the access data characteristic corresponding to the content i.
5. The method of claim 2, wherein the negative feedback log records information that the content i is subject to a negative feedback operation after pushing by subject k, the subject k being any one of the N subjects;
the querying the second sample data feature from the database according to the negative feedback log includes:
Extracting negative feedback data characteristics corresponding to the content i from the negative feedback log, wherein the negative feedback data characteristics comprise portrait characteristics of the content i and portrait characteristics of the object k;
inquiring in the database according to the negative feedback data characteristic corresponding to the content i to obtain a second issuing data characteristic corresponding to the content i, wherein the second issuing data characteristic corresponds to the content i and is matched with the negative feedback data characteristic corresponding to the content i;
and determining the second downlink data characteristic as a second sample data characteristic.
6. The method of claim 1, wherein training a feature correction model using the first sample data feature and the second sample data feature comprises:
training a characteristic deviation correcting model by adopting the first sample data characteristic to obtain a trained characteristic deviation correcting model;
and performing fine adjustment processing on the trained characteristic deviation correcting model by adopting the second sample data characteristics to obtain the trained characteristic deviation correcting model.
7. The method of claim 6, wherein the method further comprises:
Performing feature preprocessing on the first sample data features in a configuration processing mode to obtain preprocessed first sample data features, and performing feature preprocessing on the second sample data features in a configuration processing mode to obtain preprocessed second sample data features;
Wherein, the configuration processing mode can comprise any one or more of the following: the method comprises the steps of converting enumerated type features into letter identification processing modes, constructing a feature dictionary processing mode, discretizing continuous features and normalizing features.
8. The method of claim 1, wherein the method further comprises:
acquiring negative feedback data characteristics corresponding to the negative feedback log;
and calling the trained characteristic deviation correction model to correct the negative feedback data characteristics corresponding to the negative feedback log, so as to obtain consistency characteristics.
9. The method of claim 8, wherein the method further comprises:
invoking a negative feedback pre-estimation model, and training the negative feedback pre-estimation model by utilizing the consistency characteristics;
and when the trained negative feedback pre-estimation model meets the model convergence condition, determining the trained negative feedback model as a trained negative feedback pre-estimation model.
10. The method of claim 9, wherein the method further comprises:
Invoking the trained negative feedback pre-estimation model, and carrying out prediction processing on data to be pushed in the content recommendation system to obtain target data;
deleting the target data from the data to be pushed;
And pushing the deleted data to be pushed to the target object.
11. A data processing apparatus, comprising:
The acquisition unit is used for acquiring log data of the content recommendation system, wherein the log data comprises a release log, an access log and a negative feedback log; the issuing log is used for recording information for pushing the content set to the object set; the access log is used for recording information of the content set which is subjected to access operation after being pushed; the negative feedback log is used for recording information that the content set is subjected to negative feedback operation after being pushed; the corresponding issued data characteristics of the issued log are stored in a database;
The processing unit is used for inquiring first sample data characteristics from the database according to the access log and inquiring second sample data characteristics from the database according to the negative feedback log;
the processing unit is further configured to train a feature correction model by using the first sample data feature and the second sample data feature, where the trained feature correction model is used to correct the negative feedback data feature corresponding to the negative feedback log.
12. A computer device, comprising: a memory and a processor;
A memory in which one or more computer programs are stored;
a processor for loading the one or more computer programs to implement the data processing method of any of claims 1-10.
13. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded by a processor and to perform a data processing method according to any of claims 1-10.
14. A computer program product, characterized in that the computer program product comprises a computer program adapted to be loaded by a processor and to perform the data processing method according to any of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111494696.4A CN114328480B (en) | 2021-12-08 | 2021-12-08 | Data processing method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111494696.4A CN114328480B (en) | 2021-12-08 | 2021-12-08 | Data processing method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114328480A CN114328480A (en) | 2022-04-12 |
| CN114328480B true CN114328480B (en) | 2024-10-22 |
Family
ID=81050558
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111494696.4A Active CN114328480B (en) | 2021-12-08 | 2021-12-08 | Data processing method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114328480B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113032676A (en) * | 2021-03-31 | 2021-06-25 | 上海天旦网络科技发展有限公司 | Recommendation method and system based on micro-feedback |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11636102B2 (en) * | 2019-09-05 | 2023-04-25 | Verizon Patent And Licensing Inc. | Natural language-based content system with corrective feedback and training |
| CN110851706B (en) * | 2019-10-10 | 2022-11-01 | 百度在线网络技术（北京）有限公司 | Training method and device for user click model, electronic equipment and storage medium |
| CN111160566A (en) * | 2019-12-26 | 2020-05-15 | 腾讯科技（深圳）有限公司 | Sample generation method and device, computer readable storage medium and computer equipment |
| CN111460290B (en) * | 2020-03-27 | 2023-06-02 | 喜丈(上海)网络科技有限公司 | Information recommendation method, device, equipment and storage medium |
-
2021
- 2021-12-08 CN CN202111494696.4A patent/CN114328480B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113032676A (en) * | 2021-03-31 | 2021-06-25 | 上海天旦网络科技发展有限公司 | Recommendation method and system based on micro-feedback |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114328480A (en) | 2022-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12223525B2 (en) | System and method of providing a platform for managing data content campaign on social networks | |
| CN111460294B (en) | Message pushing method, device, computer equipment and storage medium | |
| US10715570B1 (en) | Generic event stream processing for machine learning | |
| US11258814B2 (en) | Methods and systems for using embedding from Natural Language Processing (NLP) for enhanced network analytics | |
| CN109344170B (en) | Stream data processing method, system, electronic device and readable storage medium | |
| CN110598070B (en) | Application type identification method and device, server and storage medium | |
| CN107193974B (en) | Regional information determination method and device based on artificial intelligence | |
| US20180253754A1 (en) | System and method for tracking users across a plurality of media platforms | |
| US20250181389A1 (en) | Systems and methods for graphical user interface-based programming for automated workflows | |
| CN112632405A (en) | Recommendation method, device, equipment and storage medium | |
| CN113821418B (en) | Fault root cause analysis method and device, storage medium and electronic equipment | |
| CN119003891B (en) | Method, device and equipment for generating employee search recommended content | |
| CN117407441A (en) | Artificial intelligence platform applied to field data management | |
| CN112686717A (en) | Data processing method and system for advertisement recall | |
| US11115440B2 (en) | Dynamic threat intelligence detection and control system | |
| CN110413770A (en) | Group's message is referred to the method and device of group topic | |
| CN112085566B (en) | Product recommendation method and device based on intelligent decision and computer equipment | |
| CN114328480B (en) | Data processing method and related device | |
| CN108259588A (en) | A kind of method for pushing and device of the cultural cloud platform based on big data | |
| CN109408704B (en) | Fund data association method, system, computer device and storage medium | |
| CN116049272A (en) | Service data processing method, computer equipment and storage medium | |
| CN113761272A (en) | Data processing method, data processing equipment and computer readable storage medium | |
| Mi et al. | An Ensemble Telecom Customers Clustering Model based on Grey Incidence and K-Means. | |
| CN111651672A (en) | Time-interval user activity recommendation method and device based on deep learning | |
| US20250124019A1 (en) | Event hydration and validation for efficient report generation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |