CN112417171B - Knowledge graph representation learning-oriented data augmentation method - Google Patents

Abstract

The application discloses a data augmentation method for knowledge graph representation learning, which comprises the following steps: given a knowledge graph, triples in the current knowledge graph are first taken as observed facts, and logic rules are mined from the observed facts. Next, through a rule instantiation step, the potential facts containing part of the noise are deduced using the facts currently observed, as candidate triples for augmentation. Because the learning model has certain distinguishing capability on the triples, the application trains and obtains a knowledge graph embedded model based on the current observed facts. And finally, verifying the candidate triples by combining the embedded model, filtering out noise, obtaining the augmented triples and supplementing the augmented triples into the current knowledge graph. Repeating the steps until the maximum number of iterations is reached. The application can amplify the data which can be understood by the user for the knowledge graph and improve the learning effect of the knowledge graph representation.

Description

Knowledge graph representation learning-oriented data augmentation method

Technical Field

The application relates to the technical field of knowledge graphs, in particular to a data augmentation method for knowledge graph representation learning.

Background

In 2012, google corporation first proposed a new concept, a knowledge graph, that improves the quality of searching by introducing knowledge graphs to structure information about search targets. From the content perspective, the knowledge graph is mainly composed of interconnected entities and their attributes; in essence, it can be seen as a knowledge base built on a semantic network, where each knowledge can be represented by a triplet. For example (france, capital, paris), a piece of knowledge (facts) in the real world is depicted: the first of france is paris. Since many real world scenes are suitable for being represented by a knowledge graph, in recent years, construction and application work on the knowledge graph become a new research hotspot. Currently, a large array of quality knowledge-graphs emerge in the industry, such as Freebase, etc., which are widely used in real world applications. Since knowledge maps are symbol-based structured data that is not readily applicable directly to other tasks, how to handle such symbolic representations is an important challenge.

This research problem is called representation learning of the knowledge graph (or embedding of the knowledge graph). The purpose of knowledge-graph representation learning is to assist downstream tasks in the real world by vectorizing elements throughout the knowledge-graph. The problem is modeled as a mapping from symbolic representations to vector representations, while requiring that the new representation in vector space be as far as possible implying structural information in the original symbolic space. Research has shown that this process of representation learning typically relies on a large number of triples to capture potential semantic patterns. However, knowledge maps are naturally incomplete, and the knowledge that they contain is often limited, which results in the embedded representation that is modeled to be often poorly expressive. Intuitively, the strategy is a viable solution to data augmentation for knowledge-graph, enriching the knowledge-graph to represent the data material needed for learning.

In the field of computer vision, data augmentation has proven to be a simple, effective technique and is widely used. Due to the characteristics of the picture, the modes of translation, rotation, scaling and the like are very natural data augmentation modes. However, in the knowledge graph, a piece of structured knowledge, as long as a little is changed, the meaning of the expression of the triplet may be changed greatly, which results in a data augmentation operation in the knowledge graph field not trivial. Note that in knowledge graphs, logic rules are typically used for knowledge reasoning, and the logic rules themselves contain some generalized background information, which makes using logic rules for data augmentation a viable solution.

In the above process, one key to data augmentation using logic rules is how to guarantee the quality of the augmented data. Although logic rules are generally accurate, it is still difficult to guarantee the quality of the new knowledge it infers. This approach has significant costs due to the difficulty of the augmented data user to discern, making filtering by manual means. Based on this, it is necessary to design an automated processing method to ensure the quality of the amplified data. At the same time, it is difficult to ensure complete accuracy of the augmented data, and there is a need in the process to provide a potential mechanism to ensure that the augmented data is understandable to the user, so that the overall augmentation process is user-controllable.

Existing methods are focused on data augmentation of knowledge maps, and although some methods generate new triples through various strategies, the generation process is unintelligible to users, and the quality of generating the triples is not carefully considered.

Disclosure of Invention

The application aims to: aiming at the problems and the shortcomings of the prior art, the application aims to provide a data augmentation method for knowledge graph representation learning, which can utilize the abundant background information contained in logic rules to supplement high-quality augmentation triples which can be understood by users for the knowledge graph. The application can enrich the original knowledge graph information, ensure the quality of the augmentation data, promote the learning effect and provide visual interpretation of the augmentation data for users.

The technical scheme is as follows: in order to achieve the above purpose, the technical scheme adopted by the application is a data augmentation method for knowledge-graph-oriented representation learning, comprising the following steps:

(1.1) mining logic rules from the current given knowledge graph by using an association rule mining algorithm;

(1.2) deriving based on the logic rule obtained in the step (1.1) by combining the facts observed in the current knowledge graph, and instantiating the logic rule to obtain a candidate triplet to be amplified;

(1.3) for the currently given knowledge graph, obtaining an embedded representation of the entity and the relationship by using a representation learning model;

(1.4) based on the step (1.2) and the step (1.3), verifying the candidate triples by using the embedded representation of the knowledge graph, filtering noise therefrom, and supplementing the filtered candidate triples as augmented triples into the current knowledge graph;

(1.5) repeating said steps (1.1) to (1.4) until a maximum number of iterations is reached.

Further, the step (1.2) includes the steps of:

(2.1) traversing each mined logic rule, instantiating entity variables in a rule body by using specific entities aiming at the rule body in the logic rule, and simultaneously limiting each triplet in the rule body after the instantiation to come from a current given knowledge graph, and then instantiating a rule head after the rule body is instantiated, so as to obtain an instantiated logic rule;

(2.2) taking a rule head in the instantiated logic rule as a candidate triplet to be amplified, and taking the instantiated logic rule as interpretation of the knowledge graph amplification.

Further, the step (1.4) includes the steps of:

(3.1) scoring all candidate triples using an embedding model of the current knowledge-graph, the score representing the probability that the candidate triplet is true;

(3.2) setting a threshold, and labeling the candidate triples with scores less than the threshold as counterexamples;

(3.3) establishing propagation influence structures for the candidate triples by utilizing the logic rules, wherein each propagation influence structure corresponds to a specific scoring update formula;

(3.4) searching a propagation influence structure established between the candidate triples marked as counterexamples and other candidate triples by using the candidate triples marked as counterexamples, and carrying out scoring update on the other candidate triples according to a corresponding scoring update formula;

and (3.5) selecting the candidate triples with the score being greater than or equal to the threshold value from the candidate triples with the score being corrected as the augmentation triples.

The beneficial effects are that: (1) The automatic data augmentation is carried out for the knowledge graph, external information is not required to be additionally introduced, and the learning capacity of the knowledge graph representation is improved while the quality of the augmented data is ensured. (2) The whole augmentation process is understandable to the user, and in the whole augmentation process, not only is the augmentation triplet provided, but also the basis for the augmentation of the triplet is provided as the explanation for the user, thereby facilitating the verification of the user and enabling the whole augmentation process to be controllable.

Drawings

FIG. 1 is an overall flow chart of the present application;

FIG. 2 is a schematic diagram of propagation affecting structures between triples of the present design.

Detailed Description

The present application is further illustrated in the accompanying drawings and detailed description which are to be understood as being merely illustrative of the application and not limiting of its scope, and various modifications of the application, which are equivalent to those skilled in the art upon reading the application, will fall within the scope of the application as defined in the appended claims.

The application utilizes logic rules to infer potential knowledge as a candidate triplet, and simultaneously the reasoning process is used as interpretation for users. And scoring the candidate triples through an embedded model, establishing a propagation influence mechanism for the triples by utilizing a logic rule, automatically labeling the counterexample triples in the candidate set, and propagating the labeled influence to the rest triples so as to adjust the distribution condition of the scoring of the whole candidate set. And selecting potential positive examples to supplement as augmentation triples according to the final score distribution. The whole process is iterated continuously, so that the data augmentation module and the representation learning module mutually promote.

The overall flow of the application is shown in fig. 1, comprising 5 parts: mining Huo En rules (abbreviated as rules) according to triplet information in a current knowledge graph by using an association rule mining algorithm, deducing a candidate triplet set for augmentation according to the rules, training in the current knowledge graph by using a representation learning model to obtain embedded representations of entities and relations, filtering the candidate triples by using the embedded representations to obtain augmented triples and supplementing the augmented triples into the knowledge graph, and repeating the steps to enable a data augmentation module and a representation learning module to mutually promote.

The specific embodiments are described below:

1. mining rules by using association rule mining algorithm according to triplet information in current knowledge graph

For a currently given knowledge graph, the triples contained therein are taken as known knowledge. And mining by using a mainstream association rule mining algorithm AMIE. The mined rule has a confidence level for indicating the accuracy of the rule. According to the application, according to knowledge maps of different scales, a user can set a corresponding rule confidence coefficient threshold value and limit the length of a rule body, so that the quality of the augmented data is ensured, the time cost of rule mining is within a reasonable range, for example, the rule confidence coefficient threshold value is set to 0.8, and the length of the rule body is limited to 2.

2. Deriving candidate triplet sets for augmentation from rules

The rule is used for generating candidate triples for augmentation, and the candidate triples are mainly divided into the following two substeps:

(1) Because the mined rule contains entity variables, the application instantiates the variables through specific entities, thereby obtaining the instantiated rule. In the instantiation phase, the rule body is limited to be instantiated only by using the currently observed triples, and the rule head is instantiated after the rule body is instantiated. To ensure that the derived triples are unknown, it is necessary to ensure that the instantiated rule header does not appear in the currently observed triples, thereby ensuring that the augmented data has an information amount;

(2) After the instantiated rules are obtained, the result of the instantiation can be considered as a forward reasoning process. The instantiated rule body is known knowledge, and the instantiated rule head is derived unknown knowledge. Thus, the instantiated rule may be used as a basis and explanation for triplet augmentation. Finally, all rule heads of the instantiated rules are collected and combined into a data augmented candidate triplet set.

3. Training in current knowledge graph by using representation learning model to obtain embedded representation of entity and relationship

The application uses a mainstream knowledge graph to represent a learning model-TransE, trains the current knowledge graph to obtain a trained embedded model, and has a certain prediction capability on unknown triples. Because the whole processing flow is an iterative processing mode, the application adopts an incremental learning process, and achieves the effect of incremental learning by setting initialization parameters. The specific measures are as follows:

(1) During the first training, the model parameters are randomly initialized in a conventional mode;

(2) In the following iterative process, the current model is initialized each time by using the model parameters learned in the previous round. By better initial values, the model can converge on better results.

4. Filtering the candidate triples by using the embedded representation to obtain augmented triples and supplementing the augmented triples into the knowledge graph

Scoring the candidate triples by using the embedded model obtained by the current training, thereby obtaining an initial score distribution on the candidate triples, wherein the score represents the possibility that the triples are positive examples;

to ensure the quality of the augmented triples, a threshold value, e.g. 0.5, is set, triples with initial scores less than the threshold value are automatically marked as counterexamples, and these triples can be directly filtered, which must not be supplemented as augmented data into the knowledge-graph. And the candidate triples with the score being greater than or equal to the threshold value are used as potential positive examples, and subsequent further filtering is needed;

a propagation affecting structure is established for the candidate triples using rules. The application mainly aims at guaranteeing the quality of the augmentation data, namely hopefully reducing the score of the counterexample in the candidate set as much as possible, and is mainly designed for the case marked as the counterexample. As shown in fig. 2, the present application mainly designs three propagation structures, where t represents a triplet and μ represents a rule. Core ideaThe equivalence of the inverse proposition and the original proposition is utilized. Rules establish a relationship between candidate triples, by treating the triples as a proposition, then the forward reasoning process can be regarded as a primitive proposition, i.e. one triplet is correct, from which it can be deduced that the other triplet is also correct. If its inverse is a proposition that the triplet is known to be erroneous, then it can be deduced from the rule that another triplet may also be erroneous. In FIG. 2, the solid circles represent triples (t ₁ 、t ₃ And t ₆ ) The open circles in solid lines represent triples (t ₂ 、t ₄ 、t ₅ And t ₈ ) The dashed open circles represent the current observed triplet (t ₇ ). The first propagation structure is the most basic inverse proposition idea; the second propagation structure means that two triples need to be used for deducing a conclusion together in the original proposition, and then in the inverse proposition, one of the original two triples may be wrong; the third propagation structure refers to two original propositions being known and one derivation being erroneous, while the rule confidence of the derivation process is greater than the other original propositions, then the derivation process of the other original propositions may be suspected to be also erroneous. Finally, through the three propagation structures, propagation relations are established for the candidate triples by utilizing rules. Meanwhile, aiming at the characteristics of three propagation structures, a corresponding update formula needs to be designed. The application is mainly based on two visual recognitions: if the confidence of the rule is higher, the reasoning process of the inverse proposition is more accurate, and the influence on the propagation is more drastic; the greater the likelihood that the current counterexample is false, the more severely the propagation will be affected. The following are four specific update formulas, corresponding to the three propagation structures described above:

p(t ₂ )＝min(0,p(t ₂ )-(1-p(t ₁ ))·λ ₁ ),

p(t ₈ )＝min(0,p(t ₈ )-(1-p(t ₆ ))·λ ₄ ),

where p (t) represents the score of the triplet and λ represents the confidence of the corresponding rule μ;

aiming at the propagation influence structures among the candidate triples, traversing the counterexamples of each label, searching the propagation influence structures established among the other triples, and carrying out scoring update on the other triples according to the corresponding scoring update formula. Meanwhile, in order to further expand the propagation influence, assuming that the currently influenced triplet is also a counterexample, continuing to update the scores of the other triples by using the triplet;

and after the score of the whole candidate triplet is corrected, taking the triplet with the final score being more than or equal to the threshold value of 0.5 as augmentation data, and supplementing the augmentation data into the current knowledge graph.

5. Repeating the above parts to make the data augmentation module and the representation learning module mutually promote

The above parts together form a round of iterations. By repeating the above-described portions for multiple iterations, the data augmentation module and the representation learning module may be caused to interact. To reduce the time overhead caused by multiple iterations, 2 is chosen as the maximum number of iterations. On the one hand, as the data of multiple rounds is augmented, the rule mining portion can capture rich semantic patterns and mine to a large number of reliable rules. Table 1 shows the rule mining and data augmentation of the present application on two widely used reference data sets. The number of triples on both data sets may be 4% to 30% of the original data.

Table 1: rule mining and data augmentation in the present application

Data set	Number of rules mined	Original triplet number	Number of triples is increased
				WN18	41	141442	6667
FB15K	896	483142	152760

On the other hand, with multiple iterations, the knowledge graph is greatly enriched, and the embedding model can capture the relation between the entity and the relation in a more explicit way, so that a stronger embedding representation is obtained. The model is tested on the two data sets by using a link prediction task, and the test results are shown in the table 2, so that the effect of the original representation learning model can be obviously improved after multiple rounds of data augmentation. For example, on WN18, the indices are raised by approximately 0.01 compared to the original TransE model, while on the more densely related data set FB15K, it is possible to raise by 0.04 to 0.07.

Table 2: the application promotes knowledge graph representation learning model

Meanwhile, the application can provide the explanation of the augmentation triples for the user as the augmentation basis. Table 3 shows an illustrative example of some of the augmented triples. For example, one of the augmentation triplets (Johnny Depp, relationship, u.s.a.) is based on Johnny Depp's birth location being kentuck, which is a state in the united states, so Johnny Depp's nationality may be in the united states. Such an interpretation may facilitate user verification, making the overall augmentation process controllable.

Table 3: the present application provides examples of augmented triple interpretation for users

Claims (1)

1. The data augmentation method for knowledge graph representation learning is characterized by comprising the following steps:

(1.1) mining logic rules from the current given knowledge graph by using an association rule mining algorithm;

(1.2) deriving based on the logic rule obtained in the step (1.1) in combination with facts observed in the current knowledge graph, and instantiating the logic rule to obtain a candidate triplet to be amplified, wherein the method comprises the following steps:

(1.2.1) traversing each mined logic rule, instantiating entity variables in a rule body by using specific entities aiming at the rule body in the logic rule, and simultaneously limiting each triplet in the rule body after the instantiation to come from a current given knowledge graph, and then instantiating a rule head after the rule body is instantiated, so as to obtain an instantiated logic rule;

(1.2.2) taking a rule head in the instantiated logic rule as a candidate triplet to be amplified, and taking the instantiated logic rule as interpretation of the amplification of the knowledge graph;

(1.3) for the currently given knowledge graph, obtaining an embedded representation of the entity and the relationship by using a representation learning model;

(1.4) based on the step (1.2) and the step (1.3), validating the candidate triples using the embedded representation of the knowledge-graph, filtering noise therefrom, supplementing the filtered candidate triples as augmented triples into the current knowledge-graph, comprising the steps of:

(1.4.1) scoring all candidate triples using an embedded model of the current knowledge-graph, the score representing the probability that the candidate triplet is true;

(1.4.2) setting a threshold, labeling the candidate triples with a score less than the threshold as counterexamples;

(1.4.3) establishing propagation influence structures for the candidate triples by using the logic rules, wherein each propagation influence structure corresponds to a specific scoring update formula;

(1.4.4) searching a propagation influence structure established between the candidate triples marked as the counterexample and other candidate triples by using the candidate triples marked as the counterexample, and carrying out scoring update on the other candidate triples according to a corresponding scoring update formula;

(1.4.5) selecting a candidate triplet with the score being greater than or equal to the threshold value from the candidate triples with the score corrected as an augmentation triplet;

(1.5) repeating said steps (1.1) to (1.4) until a maximum number of iterations is reached.