JP6979909B2 - Information processing equipment, information processing methods, and programs - Google Patents

Description

The present invention relates to an information processing apparatus, an information processing method, and a program.

Conventionally, research and practical application have been carried out on a mechanism for automatically answering a question entered in a natural sentence. Related to this, the step of receiving a query with one or more words, and the step of querying the database to find at least one candidate subject whose name or alias has the same surface format as the subject chunk. A step of querying the database to detect one or more relationship vectors representing one or more relationships associated with the at least one candidate subject, and the query of the one or more relationships, respectively. The step of determining the ranking score indicating the semantic similarity with the correspondence relationship, the relationship having the highest ranking score from the one or more relationships is selected as the predictive relationship, and the at least one candidate subject is the predictive topic subject. Includes a step of selecting as, and a step of querying the database with the predictive relationship and the predictive topic subject to find the answer to the query, where the one or more words are the topic of the query. A computer implementation method for providing an answer to a query characterized by including a subject chunk that describes a subject is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2017-76403

With the conventional technology, the processing may not proceed efficiently or the accuracy may be insufficient.
The present invention has been made in consideration of such circumstances, and provides an information processing device, an information processing method, and a program capable of deriving an answer to a question more efficiently and with high accuracy. One of the purposes is to do.

One aspect of the present invention is a first aspect selected from the plurality of entities based on a first database in which an acquisition unit for acquiring a question, a plurality of entities, and a relationship indicating a relationship between the entities are registered. A feature amount group learned so that the feature amount obtained by adding the feature amount of the first relation indicating the relationship between the first entity and the second entity to the feature amount of the entity approaches the feature amount of the second entity. The derivator learning unit includes a derivator learning unit that learns a derivator that derives the feature amount of the question by referring to the second database including the question, and the derivator learning unit includes the feature amount of the question and the feature of the second entity. It is an information processing apparatus that learns the derivator so as to reduce the difference from the amount and increase the difference between the feature amount of the question and the feature amount of the sampled entity other than the second entity.

According to one aspect of the present invention, it is possible to derive the answer to the question more efficiently and with high accuracy.

It is a figure which shows an example of the structure of the information processing apparatus 100. It is a figure exemplifying the relationship between two entities constituting a triplet and a relation. It is an image diagram schematically showing the content of the data registered in the knowledge base 50. It is a figure which shows the geometric relation of the vector Vhr of a head entity, the vector Vr of a relation, and the vector Vgt of a tail entity schematically. It is an image diagram schematically showing the content of the data registered in the entity relation vector DB 60. It is an image diagram which conceptually shows the function of a derivator. It is an image diagram conceptually showing the content of processing by the 2nd learning unit 30. It is an image diagram schematically showing the content of processing by a comparative example.

[Overview]
Hereinafter, embodiments of the information processing apparatus, information processing method, and program of the present invention will be described with reference to the drawings. The information processing device is realized by one or more processors. The information processing device may be a device that provides a cloud service, or may be a device in which a program such as a tool or firmware is installed and processing can be executed independently. The information processing device may or may not be connected to a network such as the Internet or WAN. That is, there are no particular restrictions on the computer device for realizing the information processing device, and the information processing device may be realized by any computer device as long as the processing described below can be executed.

The information processing device is used, for example, in a device or system that automatically responds to a question input in the form of text or voice by a person or a computer. The device that responds automatically may be one that has a conversational interface, is included in the search device, interprets the query as a question, and returns the answer together with the search result to the terminal device of the query input person. May be.

The information processing device derives the feature amount of the question, and selects an entity having a feature amount close to the feature amount obtained from the question as the answer to the question. In the following description, the feature amount is assumed to be a vector on Euclidean space, but the feature amount does not have to be a vector as long as the distance can be defined and addition / subtraction is possible. It is assumed that the vectors appearing below are normalized so that the L2 norm is between 0 and 1.

[composition]
FIG. 1 is a diagram showing an example of the configuration of the information processing apparatus 100. The information processing apparatus 100 includes, for example, a question acquisition unit 10, a first learning unit 20, a second learning unit 30 (an example of a “derivator learning unit”), and an answer output unit 40. These components are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components are hardware (circuit parts) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit). It may be realized by (including circuits), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or a flash memory, or may be stored in a removable storage medium such as a DVD or a CD-ROM, and the storage medium is stored in the drive device. It may be installed by being attached.

Further, the information processing apparatus 100 stores information such as a knowledge base 50, an entity relation vector DB (database) 60, and a derivator information 70 in a storage device such as an HDD or a flash memory. These information may be held by a database server or the like separate from the information processing apparatus 100. That is, the information processing device 100 does not have to include a storage device for holding such information.

The question acquisition unit 10 acquires a question from another device via, for example, a network NW. The network NW includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a cellular network, and the like. Questions may be retrieved in textual format or in audio format. In the latter case, the information processing device 1 may include a voice recognition unit. In each case, the question shall be composed of natural sentences and not an abstract code.

The first learning unit 20 sets a vector of an entity and a relation (hereinafter, each of which is referred to as an entity vector and a relation vector) based on a knowledge base 50 in which a plurality of entities and a relation indicating a relationship between the entities are registered. Learn and register in the entity relation vector DB60.

[Knowledge Base]
Here, the knowledge base 50 will be described. The knowledge base 50 is a database in which information about things and information about semantic relationships between things are described as a graph. Things in the Knowledge Base 50 are, for example, abstract concepts such as "human", "machine", "building", "organization", "beauty", "academic", "travel", and, for example, a specific person, specific. Including those individuals (hereinafter referred to as "instances") such as buildings, specific organizations, etc. In the present embodiment, among the things, the object for which the information is described in the knowledge base 50 will be described particularly by referring to the “entity”.

An entity may represent, for example, an entity of an instance of an object (eg, an object that exists in the real world), or a concept of an object (eg, a concept defined in the real world or virtual world). ) May be represented. For example, there are entities such as "building" that represent a concept, and there are entities such as "○○ tower" that represent an instance of the concept "building".

The knowledge base 50 is described using classes and relations defined by a vocabulary system called an ontology in order to enable semantic processing by a computer. An ontology is a definition of a class and a relation of things, and is a collection of constraints that hold between the class and the relation.

A class is a group of things that have the same properties in an ontology. The nature of classes and the nature of things are described by the relations described below.

For example, things that have a beak, an oviparous vertebrate, and forelimbs that are wings are classified in the "bird" class or its subordinate classes. Also, in the class of "birds", things that have the property of not being able to fly are classified into lower classes such as "penguins" and "ostriches". In this way, the class system has a hierarchical structure with a higher-level relationship and lower-level relationships, and the properties of the upper-level class are inherited by the lower-level class. In the example above, the "bird" class's "beak-bearing, oviparous vertebrate with winged forelimbs" property is a lower class property of "penguins" and "ostriches". Will also be included. The class name itself for identifying the class does not necessarily represent the meaning of the class, but in the following explanation, it is assumed that the class name representing the meaning of the class is given for the sake of brevity.

Relations are attributes that describe the nature and characteristics of things and the relationships between classes. As a result, the relation becomes information representing the relationship between the entities. For example, a relation may be an attribute that has the property of "having ~ as a component of the body" or the property of "living in ...", or "a class is a higher class and a certain class is a lower class". It may be an attribute indicating the relationship between the upper and lower levels of "class". The relation name itself for identifying the relation does not necessarily represent the meaning of the relation, but in the following description, it is assumed that the relation name representing the meaning of the relation is given for the sake of brevity.

The basic unit of the knowledge base 50 is a set of three pieces of information (hereinafter referred to as triplets) in which entities are connected by relations. FIG. 2 is a diagram illustrating the relationship between the two entities constituting the triplet and the relationship. In the illustrated example, the triplet [entity "Japan", relation "capital", entity "Tokyo"] is given. From such triplets, it is possible to obtain semantic information that "the capital of Japan is Tokyo." By using the knowledge base 50, information about entities and relationships between entities are clearly expressed, and various machine processes become possible. Hereinafter, the entity on the original side with respect to the semantic direction of the relation in "Japan", that is, the triplet in FIG. 2 is the head entity, and the entity on the front side with respect to the "Tokyo" in FIG. 2, that is, the entity on the front side with respect to the semantic direction of the relation in the triplet. Is called a tail entity. The knowledge base 50 includes a plurality of triplets. One head entity can be a tail entity for another head entity and vice versa. FIG. 3 is an image diagram schematically showing the contents of the data registered in the knowledge base 50. In the figure, the arrows indicate the relations and the dashed ellipses indicate the triplets. Entity 1 in the figure is a head entity for entity 2 and a tail entity for entity 3.

[1st learning department]
In the first learning unit 20, for each of the plurality of triplets (known information) sampled from the knowledge base 50, the knowledge base is such that the vector obtained by adding the relation vector to the head entity vector approaches the tail entity vector. Learn the vector of each entity and each relation contained in 50.

Here, the vector of the head entity is represented by Vh, the vector of the relation is represented by Vr, and the vector of the tail entity is represented by Vt. Since the entity space contains a larger amount of information than the relation space, for example, the vector of the entity is generated with a number of dimensions (k) higher than the number of dimensions (d) of the vector Vr of the relation. In the following description, the vector Vhr and Vtr mapped to the space to which the relation vector Vr belongs by multiplying each of the vector Vh of the head entity and the vector Vt of the tail entity by the k × d matrix ^{Mk × d are obtained with the head entity and the vector Vt.} It is called a vector of tail entities (see equations (1) and (2); M is a matrix). The vector Vhr of the head entity is an example of the "features of the first entity", the vector Vtr of the tail entity is an example of the "features of the second entity", and the vector Vr of the relation connecting them is the "first". This is an example of "relation vector". In the figure, O is the origin of the vector space.

Vhr = Vh × M ^{k × d} … (1)
Vtr = Vt × M ^{k × d} … (2)

Ideally, the vector sum of the vector Vhr of the head entity and the vector Vr of the relation matches the vector Vgt of the tail entity. FIG. 4 is a diagram schematically showing the geometrical relationship between the vector Vhr of the head entity, the vector Vr of the relation, and the vector Vgt of the tail entity.

The first learning unit 20 sets the vector Vhr of the head entity, the vector Vr of the relation, and the vector Vgt of the tail entity so that the score C based on the objective function fr represented by the equation (3) becomes sufficiently small and converges. learn. In the equation, || A || ₂ represents the L2 norm of the vector A. The objective function fr becomes zero when the geometrical relationship shown in FIG. 3 is established.

fr (h, t) = {|| Vhr + Vr-Vtr || ₂ } ² ... (3)

The score C is expressed by the equation (4). In the formula, S is a combination of head entities, relations, and tail entities that make up the correct triplet, and S * is a combination of head entities, relations, and tail entities that do not make up the correct triplet. γ is an arbitrarily determined margin parameter.

The vector of the entity and the relation learned by the first learning unit 20 is registered in the entity relation vector DB 60 and used by the second learning unit 30. FIG. 5 is an image diagram schematically showing the contents of the data registered in the entity relation vector DB 60. Focusing on the triplet including the entity 1 and the entity 2, the vector V1r of the entity 1 corresponds to Vhr, the vector V2r of the entity 2 corresponds to Vtr, and the vector V12r of the relation connecting them corresponds to Vr. Focusing on the triplet including the entity 1 and the entity 3, the vector V1r of the entity 1 corresponds to the Vtr, the vector V3r of the entity 3 corresponds to the Vhr, and the vector V31r of the relation connecting them corresponds to the Vr.

[Second learning department]
The second learning unit 30 learns the parameters of the derivator represented by the derivator information 70 and the like. The derivator generates a vector of texts constituting the question (hereinafter, the vector Vq of the question). When the process of the second learning unit 30 is semantically decomposed, the learning of the question vector Vq itself according to the relationship with the head entity, the relation, and the tail entity, and the rule that the text constituting the question is made into the question vector Vq. Includes learning and. The vector Vq in question is a vector in the same vector space as the vectors Vhr, Vr, Vtr, that is, a vector having the same number of dimensions as these.

In the following description, the second learning unit 30 is provided with information associating a question with a head entity and a relation that are considered to represent the meaning of the question, and a triplet including a tail entity that is the correct answer to the question. It shall be given as data. For example, [Question "Where is the capital of Japan", head entity "Japan", relation "capital", tail entity "Tokyo"] is regarded as one record, and data including multiple records is used as teacher data in the second learning unit 30. Given to.

The second learning unit 30 reduces the difference (distance) between the vector Vq of the question and the vector Vtr of the tail entity based on the above teacher data and the entity relation vector DB60, and reduces the difference (distance) between the vector Vq of the question and the vector Vq of the question. The vector Vq of the question is learned so as to increase the difference (distance) from the vector Vtr * of the sampled entity other than the tail entity. That is, the second learning unit 30 learns the vector Vq of the question so as to reduce the objective function g1 of the equation (5).

g1 = {|| Vt-Vq || ₂ } ² ... (5)

Further, the second learning unit 30 further reduces the difference between the vector Vq of the question and the sum of the vector Vhr of the head entity and the vector Vr of the relation, and samples the vector Vq of the question and other than the head entity. The vector Vq of the question is learned so as to increase the difference between the vector Vhr * of the entity and the sum of the vectors Vr * of the sampled relations other than the above relations. That is, the second learning unit 30 learns the vector Vq of the question so as to reduce the objective function g2 of the equation (6).

g2 = {|| Vhr + Vr-Vq || ₂ } ² ... (6)

More specifically, the second learning unit 30 learns the vector Vq of the question so that the score L based on the sum of the objective function g1 and the objective function g2 becomes sufficiently small and converges. The score L is expressed by the equation (7). In the formula, S is a combination of head, relation, and tail entities that make up the correct triplet with the corresponding question, and S * does not make up the correct triplet, or the question does not correspond to other information. A combination of head entities, relationships, tail entities, and questions. γ is an arbitrarily determined margin parameter.

The second learning unit 30 learns the parameters of the derivator for deriving the vector Vq of the question from the text which is a natural sentence. FIG. 6 is an image diagram conceptually showing the function of the deriver. As shown, the deriver is generated based on, for example, an RNN. For example, the code of a word in which a question is separated for each character is sequentially input to the derivator. The RNN outputs the vectors Vq (1), Vq (2), ... Of the provisional question at each time point, and propagates at least a part of the calculation result to the next time point. The output of the RNN at the time when the codes of all the words of the question are input becomes the vector Vq of the question. The second learning unit 30 derives the vector Vq of the question learned so as to reduce the score L based on the respective vectors Vhr, Vr, Vtr of the corresponding known head entity, relation, and tail entity for a certain question. As such, the parameters of RNN are learned. This parameter is held as the deriver information 70.

If the question is in English, the words separated by spaces may be sequentially input to the RNN at each time point, and if it is Japanese or other, the words separated by morphological analysis are sequentially input to the RNN at each time point. May be done. The second learning unit 30 performs learning according to those aspects.

FIG. 7 is an image diagram conceptually showing the contents of processing by the second learning unit 30. As shown in the figure, the vector Vq of the question is obtained by inputting the question into the deriver. In the second learning unit 30, both the distance d1 between the question vector Vq and the tail entity vector Vtr and the distance d2 between the question vector Vq and the sum of the head entity vector Vhr and the relation vector Vr are sufficient. Learn the vector parameters so that they become smaller.

Further, as described above, the knowledge base 50 is provided with information (for example, a class) regarding the properties of the entity. The second learning unit 30 samples the entity to be negatively sampled regardless of the class in the first period from the start of learning until a certain amount of time elapses, and in the second period after the first period, the second learning unit 30 samples. Negatively sampled entities are sampled from the same entity as the correct entity. As a result, efficient learning can be performed and convergence can be accelerated.

The answer output unit 40 inputs, for example, a question acquired from another device via the network NW into the deriver defined by the deriver information 70, and inputs the vector of the entity closest to the vector Vq of the output question to the entity. Extracted from the relation vector DB60. At this time, the answer output unit 40 extracts the vector of the entity closest to the vector Vq of the question by, for example, a method of 1NN (Nearest Neighbor). The answer output unit 40 returns the content of the entity corresponding to the vector of the nearest entity to another device as an answer to the question, for example, via the network NW. When the information processing device performs processing by itself, the information processing device processes a question input to an input unit (not shown) and causes an output unit to output an answer.

According to the information processing apparatus described above, it is possible to derive the answer to the question more efficiently and with high accuracy.

Here, as a comparison target, consider a case of learning so that the vector Vq of the question is simply brought closer to the vector Vhr + Vr (hereinafter, comparative example). FIG. 8 is an image diagram schematically showing the contents of the processing according to the comparative example. In the comparative example, the answer is derived through two-step learning (inference) of learning to bring Vq closer to Vhr + Vr (reducing the distance d2) and learning to bring Vhr + Vr closer to Vtr (reducing the distance d3). However, in the information processing apparatus of the embodiment, since the learning that directly brings Vq closer to Vtr is included in the processing content, the accuracy of deriving the answer can be improved.

The same can be said from the viewpoint of information compression. In learning to bring Vq closer to Vhr + Vr, a kind of dimensional compression is performed. This is because the natural sentence of the question contains more information (hereinafter, additional information) than the abstract semantic information represented by the head entity or relation. In the comparative example, learning to approach Vhr + Vr is performed with the additional information deleted, whereas in the information processing apparatus of the embodiment, learning is performed to approach Vtr with the additional information included, so that Vhr + Vr is used in the first place. Results can be obtained with higher accuracy than learning that approaches Vtr.

Further, according to the information processing apparatus of the embodiment, as shown in the equation (7), Vq is learned under the two constraints of bringing Vq closer to Vtr and bringing Vq closer to Vhr + Vr. Therefore, by lowering the degree of freedom in learning, it is possible to accelerate the convergence of processing. As a result, processing can be performed at a higher speed than in the comparative example.

Further, according to the information processing apparatus of the embodiment, as shown in the equation (7), the term for maximizing when solving the problem of minimizing the distance to the positive sample and maximizing the distance to the negative sample. By making the sign negative, the whole is made a minimization problem. Since the processing such as deep learning is suitable for solving the problem of minimizing the objective function, the processing of the embodiment can be suitably realized by using a computer.

In the above description, the information processing device assumes that the code of the word that separates the questions is sequentially input to the RNN, but it is input to the RNN after making it a regular expression to some extent using a synonym dictionary or the like. You may.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 Information processing device 10 Question acquisition unit 20 1st learning unit 30 2nd learning unit 40 Answer output unit 50 Knowledge base 60 Entity relationship vector DB
70 Deriver information

Claims (9)

Based on the first database in which a plurality of entities and relationships indicating relationships between the entities are registered, the first entity and the second entity are added to the features of the first entity selected from the plurality of entities. Refer to the second database including the feature amount group learned so that the feature amount obtained by adding the feature amount of the first relation indicating the relationship of the second entity approaches the feature amount of the second entity, and the feature of the text constituting the question. Equipped with a derivator learning unit that learns a derivator that derives a quantity
The derivator learning unit
Reduce the difference between the feature amount of the question and the feature amount of the second entity.
The derivator is learned so as to increase the difference between the feature amount of the question and the feature amount of the sampled entity other than the second entity.
Information processing device. The derivator learning unit further
The difference between the feature amount of the question and the sum of the feature amount of the first entity and the feature amount of the first relation is reduced.
The derivator is set so as to increase the difference between the feature amount of the question and the feature amount of the sampled entity other than the first entity and the feature amount of the sampled relation other than the first relation. learn,
The information processing apparatus according to claim 1. The feature amount of the question is a feature amount in the same space as the feature amount of the first entity, the feature amount of the first relation, and the feature amount of the second entity.
The information processing apparatus according to claim 1 or 2. Information about the nature of the entity is attached to the first database.
In the first period, the derivator learning unit samples entities other than the second entity regardless of the property, and in the second period after the first period, the property is the second entity. Sampling entities other than the second entity from the entities common to
The information processing apparatus according to any one of claims 1 to 3. The derivator is configured based on an RNN (Recurrent Neural Networks) in which words that divide the question are sequentially input.
The derivator learning unit learns the parameters of the RNN.
The information processing apparatus according to any one of claims 1 to 4. Based on the first database in which a plurality of entities and relationships indicating relationships between the entities are registered, the first entity and the second entity are added to the features of the first entity selected from the plurality of entities. A feature amount learning unit for learning the feature amount so that the feature amount obtained by adding the feature amount of the first relation indicating the relationship of the second entity approaches the feature amount of the second entity is further provided.
The information processing apparatus according to any one of claims 1 to 5. The acquisition department to get the question and
Further, an answer output unit that extracts an entity close to the feature amount of the question obtained by inputting the question acquired by the acquisition unit into the derivator from the first database and outputs it as an answer to the question. Prepare, prepare
The information processing apparatus according to any one of claims 1 to 6. The computer
Based on the first database in which a plurality of entities and relationships indicating relationships between the entities are registered, the first entity and the second entity are added to the features of the first entity selected from the plurality of entities. Refer to the second database including the feature amount group learned so that the feature amount obtained by adding the feature amount of the first relation indicating the relationship of the second entity approaches the feature amount of the second entity, and the feature of the text constituting the question. Learn a derivator to derive a quantity,
During the above learning
Reduce the difference between the feature amount of the question and the feature amount of the second entity.
The derivator is learned so as to increase the difference between the feature amount of the question and the feature amount of the sampled entity other than the second entity.
Information processing method. On the computer
Based on the first database in which a plurality of entities and relationships indicating relationships between the entities are registered, the first entity and the second entity are added to the features of the first entity selected from the plurality of entities. The feature amount obtained by adding the feature amount of the first relation indicating the relationship of the above is referred to the second database including the feature amount group learned so as to approach the feature amount of the second entity, and the feature of the text constituting the question is referred to. Train a derivator to derive a quantity
When learning the above,
Reduce the difference between the feature amount of the question and the feature amount of the second entity.
The derivator is trained so as to increase the difference between the feature amount of the question and the feature amount of the sampled entity other than the second entity.
program.

Images