JPH11259524A - Information retrieval system, information processing method in information retrieval system and record medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent the omission of information for indicating relation between keywords, to effectively utilize the concept information and to perform highly accurate information retrieval. SOLUTION: By a concept extraction device 101, the concept information for which the information is structured by a concept unit or an attribute unit is extracted based on keyword information extracted from a natural language document and the information for indicating the relation between the keywords. The extracted concept information is registered to a data base 104 by a concept registration device 102 and required concept information is retrieved from the registered concept information corresponding to the retrieval request of a user by a concept retrieval device 103. Then, the concept information including the respective syntax, semantic, context and structural information for indicating the relation between the keywords is extracted and registered, the relation information between the keywords is utilized and the retrieval whose condition is the information of the concept unit or the attribute unit is performed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieval system, an information processing method in the information retrieval system, and a recording medium on which a program for executing the method is recorded. Instead, perform syntactic analysis, semantic analysis, context analysis and structural analysis, etc. to extract and register conceptual information including syntactic, semantic, contextual and structural information representing the relationship between keywords, The present invention relates to an information search system, an information processing method in an information search system, and a recording medium that prevent a loss of information representing a relationship between keywords and enable highly accurate information search by effectively utilizing the conceptual information.

[0002]

2. Description of the Related Art First, a WWW page search system will be described as an example of a conventional information search system (hereinafter, referred to as a conventional example). The WWW page search system is described in the first document "William B. Frakes, Re.
cardo Baeza-Yates edition; "Information Retrieval, Data
Structures & Algorithms ", PRENTICE HALL, ISBN 0-1
3-463837-9].

FIG. 49 is a configuration diagram showing a system configuration of a WWW page search system of the conventional example. FIG.
0 shows a flowchart for explaining a processing procedure when registering information in the WWW page search system of the conventional example.

The WWW page search system of the prior art is
A document group 491 including a natural language which is constructed on a plurality of computers connected to the Internet 4915 and which is distributed and stored on an external storage device of the plurality of computers.
3 is a system constructed for the purpose of immediately finding the location of information desired by the user. As a product of such a WWW page search system, for example, “Microsoft Index Server (Microsoft
Index Server), and a service such as "Yahoo (http://www.yahoo.com/)" exists.

[0005] In this type of information search system, registration and search of information can be generally performed. The information search is performed based on a search sentence input by a user via a keyboard 4912 in a database 4908. The location information (or the document itself) of the natural language document corresponding to the search condition is displayed on the display 4911. Various operations for activating the information registration process can be considered depending on the information retrieval system. Here, for example, the following is presented. First, a user inputs location information of a document via a keyboard 4912 to register information. Second, a search engine called "agent" or "robot" is used by a WW of the Internet.
The information is automatically registered while searching on the W hyperlink in the range specified by the user.

[0006] The information registration process activated by these activation operations accumulates data that can be converted from a search sentence into document location information on the database 4908, and prepares for information retrieval. Note that there are many information search systems in which a user (user) performing information search and a user (administrator) performing information registration are considered clearly.

Next, a general procedure of information registration processing in the WWW page search system of the conventional example will be described with reference to a flowchart shown in FIG. First, in the keyword extraction device 4901, a keyword is extracted from the natural language document group 4913.

First, in step 5001, the morphological analysis device 4902 causes the natural language document 5 to be analyzed to be analyzed.
002 is decomposed into morphemes. That is, natural language document 5
002 is converted into a set of morphemes (words). This disassembly into morphemes is, in English, equivalent to a process of extracting a character string delimited by several symbols such as spaces and periods. The processing of Japanese morphological analysis is complicated, but as a general method, for example, the second document "Yoshimura, Hidaka, Yoshida;" Morphological Analysis ”, IPSJ Transactions Vol.
24, No. 1, pp. 40-46 (1983)].

In the next step 5003, the stop word removal processing by the stop word removal device 4903 eliminates morphemes that are not required as keywords from the set of morphemes extracted in step 5002. Stopwords are words that are too common and likely to appear in any document. For example, in English, "is", "the", "have", "go", etc. are stopwords. However, even in Japanese, particles, auxiliary verbs, case particles, etc. are removed as stop words. These stop words are generally registered in a dictionary called a stop word list.

Next, in step 5004, the morpheme set after the stop word is removed is normalized by the stemming processing performed by the stemming processing device 4904. That is, it normalizes the inflected form to the standard form, and normalizes fluctuations in notation. Note that this stemming process may be omitted.

In step 5005, a keyword weight may be introduced by a keyword weighting process performed by the keyword weighting device 4905. That is, the weighting process for the keyword is also a process step that can be omitted. Note that there are various ways of obtaining the weight, but for example, there is a method in which a keyword appearing uniformly in the target natural language document group 4913 is regarded as not so important and the weight of the keyword is reduced.

The above steps 5001, 5003, 50
04,5005, one natural language document 5
002 is converted into a keyword set 5006. Here, the keyword set is a set of pure keywords when the weighting process is not performed on the keyword in step 5005, and is a set of pairs of keywords and weights when the weighting process is performed. .

Also, the keyword set 5 obtained here
006 is set as the location information (document ID) 5008 of the document acquired in step 5007 and
At 09, it is registered in the database 4908. Here, in the case of WWW, the document ID 5008 corresponds to address information such as “http: // ...” called a URL.

The database registration process at step 5009 is a process performed by the keyword registration device 4906. Generally, a data structure such as an "inverted index" as shown in FIG. Information registration is performed as data from which location information can be searched. Here, the information retrieval using the transposed index is described in the third document, Makoto Nagao, “Natural Language Processing”, Iwanami Shoten, 1996, ISBN 4-00-010355-5, No. 11
Chapters are described in detail. Further, the specific example of the transposed index shown in FIG. 51 is an example in the case where the keyword weighting process in step 5005 is performed.
This is a data structure that can determine the weight for each document for each keyword. If the weighting process is not performed, the weighting value for each document for each keyword in FIG. 51 is a binary flag representing “included in document” or “not included”.

Next, a general procedure of information retrieval processing in the WWW page retrieval system of the conventional example will be described with reference to a flowchart shown in FIG.

In the information search, the user operates the keyboard 4
There are mainly two types of search sentences input through the 912, and the processing partially differs depending on the type of the search sentence. One is to input a search sentence in natural language as it is, for example, input a sentence such as "Tell me about Japanese demographics." The other is a case where a keyword search sentence is input based on a predetermined grammar. For example, a character “&” is defined as a special symbol representing a logical product, and only a document including a plurality of keywords is included. In an information retrieval system that defines a grammar for retrieval, entering "Japan & Population &Statistics" or the like searches for documents that include all three keywords "Japan", "population" and "statistics". Such processing is performed. Which search sentence can be input depends on the information search system, and there are cases where only one can be input, and cases where both can be input and one of them is used while switching. In addition, regarding the switching method,
In general, a method such as designation by a user via a button is used. Also, in the flowchart of FIG.
In 301, the types of these search sentences are determined, and the processing branches.

If it is determined in step 5301 that the search sentence is a natural language search sentence, an information registration process (FIG. 5)
0), the keyword is extracted by the keyword extracting device 4901. However, the keyword weighting device 4905 is not used in the information search process.

First, in step 5302, the input natural language search sentence 5303 is converted into a morphological analyzer 4902.
Decomposes into morphemes. From the set of decomposed morphemes, in step 5304, unnecessary morphemes are removed by the stop word remover 4903, and then, in step 5307, normalization is performed by the stemming processor 4904. Note that the stemming process (Step 5)
307) can be omitted. The set of keywords extracted at this stage is referred to as a search keyword set 5308 here.
Call.

If it is determined in step 5301 that the search sentence is a keyword search sentence, step 5305
In, by the keyword search sentence analysis device 4916,
The input keyword search sentence 5306 is grammatically analyzed. The grammar here depends on the information retrieval system, but in general, logical AND (AND), logical OR (OR), NOT (NOT) operations, and prioritization (hierarchization by "()") Etc. can often be represented by special symbols (operators).

In the case of a keyword search sentence, similarly to the case of a natural language search sentence, a stemming process is performed in the next step 5307, and a search keyword set 5308 is set.
Is converted to In this case, the search keyword set 5308
Needs to hold the information of the above operators, not just a set of keywords. An information search system in which the search keyword set 5308 is automatically generated by some processing and a search is performed using the search keyword set 5308 can be regarded as equivalent to the information search system of the conventional example. In this case, the keyword search sentence analyzer 4916 becomes unnecessary.

As described above, the search content specified by the natural language search sentence or the keyword search sentence is once converted into a search keyword set 5308, and then the process proceeds to step 530.
9, the keyword search device 4907 uses the database 49 based on the search keyword set 5308.
A search process for 08 is performed. Then, the result is displayed on the display 4911 in step 5310.

The contents of the information search process vary depending on what is allowed as the above-mentioned operator and how a simple set of keywords extracted from the natural language search sentence 5303 is used for the search. I do. For example, there are variations in processing such as assuming that a mere set of keywords is the same as those connected by a logical product (AND) or increasing the search score of a document including at least one keyword. Generally, a process for checking which document contains the keyword by using the transposed index in FIG. 51 is included (see Chapter 11 of the third document). As an example of the information retrieval process, for example, when “Keyword 1 Keyword 2” is specified for the transposed index in FIG. 51 and no operator is specified, the weights of the keywords of the respective documents are added. Dividing by the number of keywords, Document 1 is (0.2 + 0) ÷ 2 = 0.1, Document 2 is (0.5 + 0.3) ÷
There is also a method of obtaining a search value (degree of relevance) by setting 2 = 0.4 and document 3 as (0.6 + 0.1) ÷ 2 = 0.35. In this case, Document 2
Is the most suitable for the search sentence.

[0023]

As described above,
In a conventional WWW page search system, an index (index) for obtaining a document location and a degree of relevance from a keyword is created in a database based on a technique for extracting a keyword from a document to be registered. Search is possible. This is based on the idea that the content expressed by text can be approximately expressed in the form of a set of keywords. However, in such conventional information search technology,
The problem is that syntactic, semantic, contextual, and structural information representing the relationship between keywords is ignored by using an approximate expression based on keyword sets, and the accuracy of information retrieval is reduced. There is. Hereinafter, this problem will be described with a specific example.

For example, it is assumed that a document related to “Omron product information” as shown in FIG. 4 is given as an example of a natural language document. When this document is registered as information by the above-mentioned conventional technique (the flowchart of FIG. 50), a keyword set as shown in FIG. 52 is obtained, and the document of FIG. 4 is approximately expressed as the keyword set of FIG. . However, in this keyword set, the following information is already missing.

The first is "lack of context information about a paragraph". In the document of FIG. 4, "Omron product information" is the title of the entire document, and gives the context that all the following document contents are information on Omron products. I don't know.

The second is "lack of context information related to itemization". In the document of FIG. 4, there are bullets such as (a), (b), and (c), and the keywords "sphygmomanometer", "thermometer", and "low-frequency treatment device" are headwords of each bullet. It has become. This gives the context that the items in these bullet points refer to “blood pressure monitor”, “thermometer”, and “low frequency treatment device”, respectively. do not know.

The third is "lack of structural information related to itemization". In the document of FIG. 4, as a further substructure of the item of (a), there are two itemized items indicated by the symbol "●", and these two items describe two independent products. Figure 4
The unit of the keyword set is unknown.

The fourth is "lack of context / structure information related to table". (1.4.4) In FIG. 3, the medical device approval numbers of the sphygmomanometer of (a) are shown in a table, and the heading is attached to each column, so that the medical device approval numbers for the respective types of products are correct. Although it can be understood, it is not understood in FIG.

Fifth, "lack of syntactic and semantic information". In the document of FIG. 4, since the release information of each product is represented by a natural language sentence, it is possible to know which product each word or sentence is described by syntactic and semantic information. In the keyword set of FIG. 4, it is ambiguous.

Since the information lack as described in the first to fifth, that is, the syntactic, semantic, contextual, and structural information are missing, a search that correctly grasps the user's intention is performed. It is difficult to get results.

For example, if a search sentence “12,500 yen & thermometer” is given as a search sentence “I want to know about a thermometer of 12,500 yen”, the document in FIG.
Although there is no information of a thermometer of a circle, both of these two keywords are included in the keyword set of FIG. 52, so that the natural language document of FIG. 4 is adapted to the search.

The present invention has been made in view of such conventional problems. The purpose of the present invention is to perform syntax analysis, semantic analysis, and context analysis instead of extracting keywords at the information extraction stage. Analyze and analyze the structure to extract and register syntactic, semantic, contextual, and structural information representing the relationship between keywords to prevent the loss of information representing the relationship between keywords. Another object of the present invention is to provide an information search system, an information processing method and a recording medium in the information search system, which enable highly accurate information search by effectively utilizing the concept information.

[0033]

Means for Solving the Problems In order to solve the above-mentioned problems, the invention described in claim 1 of the present application is based on keyword information extracted from a natural language document and information representing the relationship between the keywords. A concept extraction device for extracting concept information obtained by structuring the information in concept units or attribute units, a concept registration device for registering concept information extracted by the concept extraction device in a database, and the database in accordance with a user's search request. And a concept search device for searching for necessary concept information from the concept information registered in the database, and performing a search conditioned on the information of the concept unit or the attribute unit using the relationship information between the keywords. is there.

Further, the invention described in claim 2 of the present application is as follows:
A concept extraction device that extracts concept information in which the information is structured in concept units or attribute units, based on keyword information extracted from a natural language document and information representing relationships between the keywords, and a concept extraction device that extracts the concept information. A concept registration device for registering the concept information in a database, and keyword information extracted from a natural language search sentence given by a user and information representing a relationship between the keywords, and structured the information into concept or attribute units. A concept search device for extracting search concept information and searching for necessary concept information from the concept information registered in the database according to the extracted search concept information, and utilizing the relationship information between the keywords. The object of the present invention is to perform a search based on information on a concept unit or the attribute unit.

The invention described in claim 3 of the present application provides
The concept registration device converts the concept information extracted by the concept extraction device into an inverted index format based on a set of document location information of the natural language document, or any of the document location information and the concept information in the document. 3. The information retrieval system according to claim 1, wherein the information is registered in the database in an inverted index format based on a set of corresponding attribute information indicating whether the attribute has been described.

The invention described in claim 4 of the present application provides
The concept extraction device describes a morphological analysis device that extracts a morpheme from the natural language document, a verb extraction device that extracts only a verb component in the extracted morpheme, and information of a semantic on which the verb depends. A verb case frame dictionary, a semantic dictionary for performing surface definition of the semantic, a case element specifying device for specifying a case element corresponding to the verb, and conceptual information based on correspondence information between the case element and the concept. 4. The information retrieval system according to claim 1, further comprising: a concept creation device for generating a concept from a case element, wherein the concept information is extracted by utilizing information of a semantic on which a verb depends. It is in.

The invention described in claim 5 of the present application provides
The concept extraction device describes a morphological analysis device that extracts a morpheme from the natural language document, a verb extraction device that extracts only a verb component in the extracted morpheme, and describes information of a surface case on which a verb depends. A verb case frame dictionary, a case element specifying device that specifies a case element corresponding to the verb,
A device for generating concept information based on correspondence information between the case element and the concept, comprising: a device for generating concept information based on correspondence information between the case element and the concept, wherein the concept information is extracted by utilizing information of a surface case element on which a verb depends. An information retrieval system according to claim 1, 2 or 3.

The invention described in claim 6 of the present application provides
A morphological analyzer for extracting a morpheme from the natural language document, a verb extracting device for extracting only a verb component in the extracted morpheme, information of a semantic on which the verb depends, and a surface case Verb case frame dictionary that describes the information of a verb, a semantic dictionary that performs a surface definition of the semantic, a case element specifying device that specifies a case element corresponding to the verb, and correspondence information between the case element and the concept 2. A concept creation device for creating concept information based on a verb, wherein the concept information is extracted by utilizing information of a semantic on which a verb depends and information of a surface case. , 2 or 3.

The invention described in claim 7 of the present application provides
A morphological analyzer that extracts the morpheme from the natural language document, a condition of a semantic element of each extracted morpheme (hereinafter, referred to as a target element), and a co-occurrence in which the target element co-occurs Information extraction that describes the condition of the element, the condition of the attached element of the noted element and the co-occurring element, the condition of the appearance position of the co-occurring element, and the information of the column of the corresponding concept (hereinafter referred to as corresponding column) 4. The apparatus according to claim 1, further comprising: a knowledge and a corresponding column determination device that calculates a degree of correspondence to each corresponding column and extracts a morpheme from a corresponding column having the highest degree of correspondence. 5. In the information retrieval system.

Further, the invention described in claim 8 of the present application is as follows:
The concept extraction device regards the natural language document as a hierarchical context region, and obtains a context region extraction knowledge describing a start point condition, an end point condition, and a candidate for a lower-order context region for each pulse region. A context hierarchy analysis device for analyzing the hierarchy of the document structure of the natural language document to create context hierarchy information; and a context information extraction knowledge describing semantic and corresponding column information of a concept. A context information extraction device for extracting context information from a sentence in the natural language document and storing context information in the context hierarchy information; and a context information extraction device and a lower level of the context hierarchy information. Utilizing the context information stored in the context area from the context area to the upper context area or from the upper context area to the lower context area, the area where the concept information is described Constant while in the information retrieval system of claim 1, 2 or 3, characterized in that it comprises a conceptual description area determined type conceptual information extraction device for extracting the conceptual information.

The invention described in claim 9 of the present application is
The concept extraction device regards the natural language document as a hierarchical context region, and obtains a context region extraction knowledge describing a start point condition, an end point condition, and a candidate for a lower-order context region for each pulse region. A context hierarchy analysis device that analyzes the hierarchy of the document structure of the natural language document to create context hierarchy information; a morphological analysis device that extracts morphemes from the natural language document; Condition of each morpheme (hereinafter referred to as a focused element), condition of a co-occurring element co-occurring with the focused element, condition of an adjunct element to the focused element and the co-occurring element, and And the information extraction knowledge describing the information of the column of the corresponding concept (hereinafter referred to as “corresponding column”), and the degree of correspondence to each of the corresponding columns is calculated. A corresponding column determination device that extracts a morpheme in a high corresponding column, and a context information extraction device that extracts context information from a sentence in the natural language document and stores context information in the context hierarchical structure information. The concept information is described by utilizing the context hierarchical structure information and the context information stored in the lower context area from the lower context area of the context hierarchical structure information or in the lower context area from the upper context area. 4. A concept description area determining type conceptual information extracting apparatus for extracting the conceptual information while determining an area in which the information is stored.
In the information retrieval system described in 1.

Further, according to the invention as set forth in claim 10 of the present application, the information extraction knowledge includes a condition of the semantic element of the element of interest, a condition of the co-occurring element, a condition of the accessory element, and a condition of the co-occurring element. 10. The information retrieval system according to claim 7, wherein the condition of the appearance position is described as pattern description knowledge such as a regular expression.

Further, according to the invention as set forth in claim 11 of the present application, the concept extraction device activates knowledge that is likely to co-occur with the knowledge when the knowledge of the information extraction knowledge matches. 11. The information retrieval system according to claim 7, wherein the knowledge that is unlikely to be generated is provided with an inter-knowledge correlation rule that suppresses the knowledge.

According to a twelfth aspect of the present invention, the correspondence column determination device determines the correspondence degree in parallel for each information extraction knowledge or determines the correspondence column for each morpheme. 12. The information retrieval system according to claim 7, wherein the information retrieval system performs processing in parallel.

According to a thirteenth aspect of the present invention, the concept extracting device comprises a concept knowledge key in which essential attributes of a concept characterizing the concept to be extracted are described. When the concept information extracting device extracts the concept information while determining the concept description area, the concept attribute knowledge is used to extract the essential attributes, check that a plurality of concept units are not included, and add a new concept unit. 12. The concept information is checked to determine whether the concept information is necessary or sufficient.
Or the information retrieval system described in 12.

According to a fourteenth aspect of the present invention, the concept extracting apparatus further comprises a concept knowledge key in which essential attributes of the concept characterizing the concept to be extracted are described. 11. The concept information extracting apparatus according to claim 8, wherein when extracting the concept information while determining the concept description area, the concept information extracting apparatus uses the concept key knowledge to eliminate duplication of the extracted concept information. , 11, 12 or 1
3. The information retrieval system according to item 3.

[0047] In the invention according to claim 15 of the present application, the concept extraction device extracts a morpheme from the natural language document, a syntax analysis device that analyzes a syntax, and the syntax analysis device. And a concept creation device from syntax elements for creating concept information by extracting preset syntax elements from syntax elements such as ga-case / ヲ -case, subject / predicate, dependency / acceptance, etc., 4. The information retrieval system according to claim 1, wherein concept information is extracted only from a result of the syntax analysis.

Further, according to the invention as set forth in claim 16 of the present application, based on keyword information extracted from a natural language document and information indicating a relationship between the keywords, the information is structured in units of concepts or attributes. A concept extracting step of extracting the concept information extracted, a concept registering step of registering the concept information extracted by the concept extracting step in a database, and necessary concept information from the concept information registered in the database according to a user's search request. A concept search step for searching, and performing a search using the information of the concept unit or the attribute unit as a condition using the relationship information between the keywords. .

Further, according to the invention described in claim 17 of the present application, based on keyword information extracted from a natural language document and information representing the relationship between the keywords, the information is structured in units of concepts or attributes. A concept extracting step of extracting the concept information extracted by the concept extracting step, a concept registering step of registering the concept information extracted by the concept extracting step in a database, and a keyword information extracted from a natural language search sentence given by a user and a relationship between the keywords. Concept search for extracting search concept information obtained by structuring the information in concept units or attribute units based on the information, and searching for necessary concept information from the concept information registered in the database according to the extracted search concept information The concept unit or the attribute by utilizing the relationship information between the keywords. In an information processing method in an information retrieval system and performs a search in which the position information and conditions.

Further, in the invention according to claim 18 of the present application, the concept registering step converts the concept information extracted by the concept extracting step into an inverted index form by a combination with the document location information of the natural language document. 17. The method according to claim 16, wherein the document is registered in the database in an inverted index format based on a combination of the document location information and corresponding attribute information indicating which attribute of the concept information is described in the document. Or an information processing method in the information search system according to item 17.

In the invention described in claim 19 of the present application, the concept extracting step includes a morphological analysis step of extracting a morpheme from the natural language document, and extracting only a verb component in the extracted morpheme. A verb case frame dictionary that describes information on semantics on which the verb depends, and a semantic dictionary that performs surface definition of the semantics, and specifies a case element corresponding to the verb. A case element specifying step, and a step of creating a concept from a case element that creates concept information based on the correspondence information between the case element and the concept. 17. The method according to claim 16, wherein
17. An information processing method in the information search system described in 17 or 18.

Further, in the invention according to claim 20 of the present application, the concept extracting step includes a morphological analysis step of extracting a morpheme from the natural language document, and extracting only a verb component in the extracted morpheme. A verb extraction step, a case element specifying step of specifying a case element corresponding to the verb with reference to a verb case frame dictionary describing information of a surface case on which the verb depends, and a correspondence between the case element and the concept. A step of creating a concept from a case element for creating concept information based on the information, wherein the concept information is extracted by utilizing information of a surface case element on which a verb depends. 18. An information processing method in the information search system according to item 18.

Further, in the invention according to claim 21 of the present application, the concept extracting step includes a morphological analysis step of extracting a morpheme from the natural language document, and extracting only a verb component in the extracted morpheme. A verb extraction step, a verb case frame dictionary that describes information of a semantic on which the verb depends and information of a surface case, and a semantic dictionary that performs a surface definition of the semantic, and corresponds to the verb. A case element specifying step of specifying a case element to perform, and a concept creation step from a case element for creating concept information based on the correspondence information between the case element and the concept, and information of a semantic on which the verb depends. 19. The information processing method in the information search system according to claim 16, wherein concept information is extracted by utilizing surface case information.

Further, in the invention according to claim 22 of the present application, the concept extracting step includes a morphological analysis step of extracting a morpheme from the natural language document, and the extracted morphemes (hereinafter, referred to as a target element). ), The condition of the co-occurring element co-occurring with the noted element, the condition of the attached element of the noted element and the co-occurring element, and the condition of the appearance position of the co-occurring element. Correspondence column determination for calculating the degree of correspondence to each corresponding column and extracting a morpheme to the corresponding column with the highest degree of correspondence by referring to information extraction knowledge describing information of a concept column (hereinafter, corresponding column) 2. The method according to claim 1, further comprising the steps of:
An information processing method in the information search system described in 6, 17 or 18.

Further, in the invention according to claim 23 of the present application, the concept extraction step regards the natural language document as a hierarchical context region, and sets conditions of a start point and an end point of each of the time domain regions. A context hierarchy analysis step of analyzing the hierarchy of the document structure of the natural language document and creating context hierarchy information by referring to context area extraction knowledge describing conditions and candidates for a lower context area; Context information for extracting context information from a sentence in the natural language document with reference to context information extraction knowledge describing information of a column of a concept corresponding to a semantic element, and storing context information in the context hierarchical structure information Extracting, from the context hierarchy structure information and the lower context region of the context hierarchy structure information to the upper context region or from the upper context region to the lower context region. A concept description area determining type concept information extracting step of extracting the concept information while determining an area in which the concept information is described by utilizing the stored context information. 16, 17
Or the information processing method in the information search system described in 18.

Further, in the invention according to claim 24 of the present application, the concept extraction step regards the natural language document as a hierarchical context region, and sets conditions for a start point and an end point for each of the time domain regions. A context hierarchy analysis step of analyzing the hierarchy of the document structure of the natural language document and creating context hierarchy information by referring to context area extraction knowledge describing conditions and candidates for a lower context area; A morphological analysis step of extracting a morpheme from the natural language document, a condition of a semantic element of each extracted morpheme (hereinafter, referred to as a target element), a condition of a co-occurrence element in which the target element co-occurs, The condition of the element of interest and the attached element of the co-occurrence element, the condition of the appearance position of the co-occurrence element, and the information of the column of the corresponding concept (hereinafter, corresponding column) are described. With reference to the information extraction knowledge,
A corresponding column determining step of calculating a degree of correspondence to each corresponding column and extracting a morpheme to a corresponding column having the highest degree of correspondence, extracting context information from a sentence in the natural language document, and A context information extracting step of storing context information in structure information; and storing the context hierarchy structure information and a lower context region of the context hierarchy structure information in a higher context region or a higher context region in a lower context region, respectively. 18. A concept description area determining type concept information extracting step of extracting the concept information while determining an area in which the concept information is described by utilizing the context information thus obtained. Or the information processing method in the information search system described in 18.

Further, in the invention according to claim 25 of the present application, the information extraction knowledge is such that the condition of the semantic element of the element of interest, the condition of the co-occurrence element, the condition of the ancillary element, and the condition of the co-occurrence element 23. The condition of the appearance position is described as pattern description knowledge such as a regular expression.
Or an information processing method in the information search system described in 24.

Further, in the invention according to claim 26 of the present application, the concept extracting step activates knowledge that is likely to co-occur with the knowledge when the knowledge of the information extraction knowledge matches. 26. The information processing method in the information search system according to claim 22, wherein knowledge that is unlikely to be generated refers to an inter-knowledge correlation rule to be suppressed.

Further, in the invention according to claim 27 of the present application, the correspondence column determination step includes calculating the correspondence degree in parallel for each information extraction knowledge or determining the correspondence column for each morpheme. 27. An information processing method in an information retrieval system according to claim 22, wherein the information processing is performed in parallel.

Further, in the invention according to claim 28 of the present application, the concept description area determining type concept information extracting step includes:
When extracting concept information while determining the concept description area,
Using the concept key knowledge describing the essential attribute of the concept characterizing the concept to be extracted, extraction of the essential attribute, checking that plural concept units are not included, and checking that new information is included 23. Checking the necessity and sufficientness of the concept information by performing
An information processing method in the information search system described in 4, 25, 26 or 27.

According to the invention described in claim 29 of the present application, the concept description area determination type concept information extracting step includes the following steps:
When extracting concept information while determining the concept description area,
27. The method according to claim 23, wherein duplication of extracted concept information is eliminated by using concept key knowledge describing essential attributes of the concept characterizing the concept to be extracted.
27. An information processing method in the information search system described in 27 or 28.

Further, the invention described in claim 30 of the present application is directed to claims 16, 17, 18, 19, 20, 21, 2.
A computer-readable recording medium stored as a program for causing a computer to execute the information processing method in the information search system according to 2, 23, 24, 25, 26, 27, 28, or 29.

According to the invention described in claim 1, 16 or 30, the concept extracting device (concept extracting step) extracts the keyword information extracted from the natural language document and the relation between the keywords. Based on the information representing the information, concept information in which information is structured in concept units or attribute units is extracted, and the extracted concept information is registered in a database by a concept registration device (concept registration step),
The concept search device (concept search step) searches for necessary concept information from the concept information registered in the database in accordance with the search request of the user.

As described above, the concept information including the syntactic, semantic, contextual, and structural information representing the relationship between the keywords is extracted and registered, and the concept information or the concept unit is utilized by utilizing the relationship information between the keywords. Since the search is performed on the basis of the information of the attribute unit, it is possible to prevent the loss of the information indicating the relationship between the keywords, and realize the highly accurate information search by effectively utilizing the concept information.

According to the invention described in claim 2, claim 17 or claim 30, the concept extracting device (concept extracting step) converts the keyword information extracted from the natural language document and the relationship between the keywords. Based on the information to be represented, concept information in which information is structured in concept units or attribute units is extracted, and the extracted concept information is registered in a database by a concept registration device (concept registration step). ), Based on the keyword information extracted from the natural language search sentence given by the user and the information indicating the relationship between the keywords, to extract search concept information in which information is structured in concept units or attribute units,
The necessary concept information is searched from the concept information registered in the database according to the extracted search concept information.

As described above, the concept information including the syntactic, semantic, contextual, and structural information representing the relationship between the keywords is extracted and registered, and the search concept information is also extracted for the natural language search sentence. Then, a search is performed on the basis of the information of the concept unit or the attribute by utilizing the relationship information between the keywords, so that the lack of the information representing the relationship between the keywords is prevented, and the concept information is effectively utilized. A highly accurate information search can be realized.

According to the invention described in claim 3, claim 18 or claim 30, in the concept registration device (concept registration step), the extracted concept information is combined with the document location information of the natural language document. In the inverted index format by
Alternatively, it is registered in the database in an inverted index format based on a combination of document location information and corresponding attribute information indicating which attribute of the concept information is described in the document.

As a result, in addition to the highly accurate search based on the concept information, not only the concept information can be used as the output result, but also the information of the original document can be used, thereby increasing the amount of information that can be output as the search result. Since it is possible to specify in which document the attribute information which is a part of the concept information is described, it is possible to present only an appropriate document as a search result.

In particular, if the concept information is expressed by a record of the relational database, a known relational database is used for storing the concept information,
Utilizing well-known relational database high-speed search technology (index etc.) to enable high-speed search,
It is also possible to easily maintain the extracted conceptual information by using a data editing tool of a relational database using QL or GUI.

Further, a plurality of types of concept information are expressed as a plurality of tables, and not only the column names but also the table names are described in the description of the target columns (corresponding columns, extraction columns, target columns, etc.) of the information extraction knowledge. This makes it possible to simultaneously extract a plurality of types of concept information.

Further, by setting a column serving as a key for a relation between a plurality of conceptual information extracted at the same time, a table join in a relational database can be performed, and a complicated search request over a plurality of conceptual information can be accepted. become.

According to the invention described in claim 4, claim 19 or claim 30, in the concept extracting device (concept extracting step), the morpheme is analyzed from the natural language document by the morphological analyzing device (morphological analyzing step). A verb case that extracts only the verb component in the morpheme extracted by the extraction and verb extraction device (verb extraction step), and describes the information of the semantic on which the verb depends by the case element identification device (case element identification step) The case element corresponding to the verb is specified by referring to the frame dictionary and the semantic dictionary for superficially defining the semantic element, and the case element-based concept creating device (case creating step from the case element) is used. Concept information is created based on correspondence information between a concept and a concept.

As described above, utilizing the information of the semantics on which the verb depends, the concept information including the syntactic and semantic information representing the relation between the keywords is extracted and registered, and the relation information between the keywords is extracted. Since the search utilizing the keyword is performed, it is possible to prevent a loss of information representing the relationship between keywords, particularly syntactic and semantic information, and to realize a highly accurate information search by effectively utilizing the conceptual information.

According to the invention described in claim 5, claim 20, or claim 30, in the concept extracting device (concept extracting step), the morpheme is extracted from the natural language document by the morphological analyzing device (morphological analyzing step). A verb case that extracts only the verb component in the morpheme extracted by the extraction and verb extraction device (verb extraction step), and describes information of the surface case on which the verb depends by a case element identification device (case element identification step) The case element corresponding to the verb is specified with reference to the frame dictionary, and the concept information is created based on the correspondence information between the case element and the concept by the concept creating apparatus from the case element (the step of creating the concept from the case element). .

As described above, the concept information including the syntactic and semantic information representing the relationship between the keywords is extracted without utilizing the information of the surface case element on which the verb depends, without having a semantic dictionary. And perform a search utilizing the relationship information between the keywords, so that the information representing the relationship between the keywords, particularly syntactic and semantic information, is prevented from being lost, and the concept information is effectively used to make the search more concise. With a simple configuration, highly accurate information retrieval can be realized.

According to the invention described in claim 6, 21 or 30, in the concept extracting device (concept extracting step), the morpheme is analyzed from the natural language document by the morphological analyzing device (morphological analyzing step). Extraction and extraction of only verb components in morphemes extracted by a verb extraction device (verb extraction step), and information of semantics and surface case information on which the verb depends by a case element identification device (case element identification step) A case element corresponding to a verb is specified with reference to a verb case frame dictionary describing the verb and a semantic dictionary for surface definition of the semantic, and a concept creating apparatus from the case element (a concept creating step from the case element) ), Concept information is created based on correspondence information between case elements and concepts.

As described above, the concept information including the syntactic and semantic information representing the relationship between the keywords is extracted and registered by utilizing the information of the semantics on which the verb depends and the information of the surface case. , It is possible to specify case elements even in sentences in which case particles representing surface cases are omitted, or in cases where the same semantic element is included in multiple sentences. Therefore, it is possible to prevent a loss of information representing the relationship between keywords, particularly syntactic / semantic information, and to realize highly accurate information retrieval by effectively utilizing the concept information.

According to the invention described in claim 7, 22, or 30, in the concept extracting device (concept extracting step), the morpheme is extracted from the natural language document by the morphological analyzing device (morphological analyzing step). Conditions of semantic elements of extracted and extracted morphemes (hereinafter, referred to as target elements), conditions of co-occurring elements where the target elements co-occur, conditions of auxiliary elements of the target elements and co-occurring elements, and co-occurrences With reference to the condition of the appearance position of the element and the information extraction knowledge that describes the information of the column of the corresponding concept (hereinafter, corresponding column),
The corresponding column determining device (corresponding column determining step) calculates the degree of correspondence to each corresponding column and extracts a morpheme into the corresponding column having the highest degree of correspondence.

Thus, a verb predicate sentence, an adjective predicate sentence,
It is possible to extract concept information from a noun predicate sentence, a sentence containing only words, a sentence containing only noun phrases, a sentence containing no words, and the like, and highly accurate information retrieval can be realized by effectively utilizing the concept information.

In particular, the knowledge used for information registration in the information extraction knowledge and the knowledge used for information search are created in the same format, and the intention information for registration / search is added to create the information extraction knowledge. By constructing and calculating the degree of correspondence to a column in the corresponding column determination device (corresponding column determination step), if the registration correspondence and the search correspondence can be separately tabulated, processing during registration / search can be performed. Knowledge can be treated equally.

In particular, in the corresponding column determining apparatus (corresponding column determining step), an initial value of the degree of correspondence of each corresponding column is determined in order to obtain the degree of correspondence to the corresponding column, and the degree of matching with one element of the information extraction knowledge is determined. If the degree of correspondence is added by a fixed amount at a time, and it is determined that a morpheme is to be finally extracted in the corresponding column having the highest degree of correspondence, the data of the degree of correspondence to be added is retained in the information extraction knowledge. There is no need to save storage space.

Further, the information extraction knowledge has information on the degree of certainty (or importance) of each knowledge, and the corresponding column determination device (corresponding column determination step) uses each correspondence to determine the degree of correspondence to the corresponding column. Determine the initial value of the degree of correspondence of the column, add the certainty of that knowledge to the degree of correspondence whenever it matches one element of the information extraction knowledge, and finally extract the morpheme to the corresponding column with the highest degree of correspondence By doing so,
By being able to weight each information extraction knowledge, extraction accuracy and search accuracy can be improved, and knowledge tuning can be easily performed by changing the certainty factor.

According to the invention described in claim 8, 23, or 30, the concept extracting apparatus (concept extracting step) regards the natural language document as a hierarchical context area, and sets each of the time domain areas. The context hierarchical structure analysis device (context hierarchical structure analysis step) refers to the context region extraction knowledge describing the start point condition, the end point condition, and the candidate for the lower-order context region for the document of the natural language document. Analyze the hierarchy of the structure to create context hierarchical structure information, refer to the context information extraction knowledge that describes the semantic and corresponding column information of the concept, and use the context information extraction device (context information extraction step). Extracts context information from a sentence in a natural language document, stores the context information in context hierarchical structure information, and outputs a concept description area determining type concept information extraction device (concept description). Region determining type conceptual information extraction step)
The concept information is described by utilizing the context hierarchical structure information and the context information stored in the lower context area from the lower context area of the context hierarchical structure information or in the lower context area from the upper context area. The concept information is extracted while deciding the region in which the image data is set.

As described above, the hierarchical structure of the document structure is analyzed, and conceptual information including syntactic, semantic, contextual, and structural information representing the relationship between keywords is extracted and registered. By using the relationship information between them to perform a search based on information in terms of concepts or attributes, context information about paragraphs and bullets, structural information about bullets,
It is possible to prevent the lack of information indicating the relationship between keywords, such as context / structure information about tables and syntactic / semantic information, and to achieve more accurate information retrieval by effectively utilizing context information and concept information. . In addition, the context information extraction knowledge is simple knowledge that can be easily constructed by using the information of the column of the concept corresponding to the semantic element, and the extraction of context information from sentences only with words, sentences only with noun phrases, sentences with no words, etc. This is possible with a simpler configuration.

In particular, if tag information is included in the start point and end point of context information extraction knowledge, HTML
And context hierarchical structure information from a markup language such as SGML or SGML.

According to the ninth, twenty-fourth, or thirty-third aspect of the present invention, in the concept extracting device (concept extracting step), the natural language document is regarded as a hierarchical context region, and each of the time domain regions is determined. The context hierarchical structure analysis device (context hierarchical structure analysis step) refers to the context region extraction knowledge describing the start point condition, the end point condition, and the candidate for the lower-order context region for the document of the natural language document. Context hierarchy structure information is created by analyzing the hierarchy of the structure, and in the context information extraction device (context information extraction step), a morpheme is extracted from the natural language document by the morphological analysis device (morphological analysis step) and extracted. Condition of each morpheme (hereinafter, referred to as a target element), a condition of a co-occurrence element in which the target element co-occurs, The corresponding column determining device (corresponding to the corresponding column determining device (corresponding to the corresponding column determining device) Column determination step), the degree of correspondence to each corresponding column is calculated, and the morpheme is extracted to the corresponding column having the highest degree of correspondence. In the concept description area determining type concept information extracting device (conceptual description area determining type concept information extracting step), context hierarchical structure information and a lower context area of the context hierarchical structure information are transferred to a higher context area. Alternatively, the concept information is extracted while utilizing the context information stored in the lower context area from the upper context area while determining the area where the concept information is described.

As described above, the hierarchical structure of the document structure is analyzed, and conceptual information including syntactic, semantic, contextual, and structural information representing the relationship between keywords is extracted and registered. By using the relationship information between them to perform a search based on information in terms of concepts or attributes, context information about paragraphs and bullets, structural information about bullets,
It is possible to prevent the lack of information indicating the relationship between keywords, such as context / structure information about tables and syntactic / semantic information, and to achieve more accurate information retrieval by effectively utilizing context information and concept information. . In particular, multiple co-occurrence elements can be specified, and the appearance position of co-occurrence elements can be described more flexibly, so that verb predicate sentences, adjective predicate sentences, noun predicate sentences,
It is possible to extract context information with high accuracy from sentences including only words, sentences including only noun phrases, sentences with no words, and the like.

According to the tenth, twenty-fifth, and thirty-third aspects of the present invention, the information extraction knowledge is used to determine the semantic condition of the element of interest, the condition of the co-occurring element, the condition of the adjunct element, and the The condition of the appearance position of the element is described and constructed as pattern description knowledge such as a regular expression. By being able to specify multiple co-occurrence elements and describing the appearance position of co-occurrence elements more flexibly, verb predicate sentences, adjective predicate sentences, noun predicate sentences, sentences containing only words, sentences containing only noun phrases, sentences with no speech It is possible to extract the concept information or the context information from the information with high accuracy, and to realize the information search with high accuracy by effectively utilizing the concept information or the context information.

According to the eleventh, twenty-sixth, or thirty-third aspects of the present invention, in the concept extracting device (concept extracting step), when the knowledge of the information extraction knowledge matches, the information extraction knowledge is shared with the knowledge. The knowledge that is likely to be activated is activated, and the knowledge that is unlikely to co-occur is referred to the inter-knowledge correlation rule that is suppressed. As a result, the corresponding column can be determined with high accuracy, and as a result, information search with high accuracy can be performed.

According to the invention described in claim 12, 27 or 30, in the corresponding column determining apparatus (corresponding column determining step), the degree of correspondence is calculated in parallel for each information extraction knowledge. Alternatively, the determination of the corresponding column is processed in parallel for each morpheme. In this way, by utilizing the feature that each information extraction knowledge is processed independently, high-speed processing can be performed by performing the processing of calculating the degree of correspondence by parallel processing using a parallel computer or the like. High-speed processing can be performed by performing the corresponding column determination processing in parallel using a parallel computer or the like by using the features processed independently.

According to the thirteenth, twenty-eighth, or thirty-third aspects of the present invention, a concept description area determining type concept information extracting apparatus (a concept description area determining type concept information extracting step) stores a concept description area. When extracting concept information while determining, using concept key knowledge describing essential attributes of the concept characterizing the concept to be extracted, extracting the essential attributes, checking that a plurality of concept units are not included, and Check that the new information is included, and check the necessity and sufficiency of the conceptual information. As a result, the accuracy of information extraction in concept units is improved, and as a result, information retrieval with high accuracy is possible.

According to the invention described in claim 14, 29 or 30, the concept description area determining type concept information extracting apparatus (concept description area determining type concept information extracting step) stores the concept description area in the concept description area determining type concept information extracting step. When extracting concept information while deciding, the duplication of the extracted concept information is eliminated by using concept key knowledge describing essential attributes of the concept characterizing the concept to be extracted. As a result, the accuracy of information extraction in concept units is improved, and as a result, information retrieval with high accuracy is possible.

Further, according to the invention described in claim 15 of the present application, in the concept extracting device, the morpheme is extracted from the natural language document by the morphological analysis device, the syntax is analyzed by the syntax analysis device, and the By the concept creation device, the ga-case / ガ -case, subject / predicate,
Concept information is created by extracting a pre-set syntax element from the syntax elements such as interlocking and receiving. As described above, without providing a semantic dictionary, the concept information including the syntactic information representing the relationship between keywords is extracted and registered from only the syntax analysis result, and the search utilizing the relationship information between keywords is performed. As a result, it is possible to prevent a loss of information representing the relationship between keywords, particularly syntactic information, and realize a highly accurate information search with a simpler configuration by effectively utilizing the concept information.

[0094]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an information retrieval system, an information processing method and a recording medium in the information retrieval system according to the present invention will be described below in detail with reference to the drawings in the following order of contents.

[Table of Contents] 1. Overview of Embodiment 1-1. Configuration of system 1-2. Information registration process 1-3. 1. Information search processing 2. Representation of concept set Extraction of concept information 3-1. Extraction of concept information using semantic analysis 3-2. 3. Extraction of concept information using context analysis and structural analysis (1) Context hierarchical structure analysis (2) Context information extraction (3) Concept description area determination and concept information extraction Registration of conceptual information 5. Search for conceptual information Effects of the embodiment

1. 1. Overview of Embodiment 1-1. System Configuration FIG. 1 is a configuration diagram of an information search system according to an embodiment of the present invention. Note that the information search system of the present embodiment also
This is an example applied to a “WWW (World Wide Web) page search system” as in the conventional example, and FIG.
The same reference numerals are given to portions overlapping with (conventional example).

The WWW page search system according to the present embodiment is constructed on a plurality of computers connected to the Internet 112, and includes a document group 11 containing a natural language distributed and stored on an external storage device of the plurality of computers.
This system is constructed for the purpose of immediately finding the location of information desired by the user from 0.

In FIG. 1, the WWW page search system according to the present embodiment includes a concept extraction device 101, a concept registration device 102, a concept search device 103,
In addition to having a database 104 and a concept search sentence analyzer 105, the CPU 106, main memory 107, display 108 and keyboard 10
9 except that the components except the database 104 are interconnected via a data bus 111.

The concept extracting apparatus 101 extracts a keyword and information representing the relationship between the keywords from the natural language document 110 and collects the information into “concept information” in which information is structured in concept units or attribute units. Is performed. As a result, the natural language document 110 is converted into a set of concepts.

In the concept registration device 102, the concept information extracted by the concept extraction device 101 is registered and stored in the database 104. It is also possible to acquire the document location information of the natural language document 110 and register and store it in the database 104 as a concept information inverted index based on a set of the extracted concept information and document location information. Corresponding attribute information indicating which attribute of the concept information is described in 110 is added to the database 10 as a concept information inverted index based on a set of the concept information, the document location information and the corresponding attribute information.
4 can also be registered and stored.

The concept search device 103 searches for necessary concept information from the registered and stored concept information in the database 104 in accordance with a search request. There are two types of search requests, one for inputting a search sentence in natural language, as in the conventional example, and the other for inputting a concept search sentence based on a predetermined grammar.

In the concept search sentence analyzing apparatus 104, the information is structured in concept units or attribute units based on the keyword information extracted from the natural language search sentence given by the user and the information indicating the relationship between the keywords. The search concept information (search concept set) is extracted, and necessary concept information is searched from the concept information registered in the database 104 according to the extracted search concept information.

Further, in the database 104, the concept information is represented by a record of the relational database, so that the relational database before storing the concept information is diverted, and a well-known relational database high-speed search technology (index or the like) is used. To enable high-speed search, database language SQL,
It is also possible to easily maintain the extracted conceptual information by using a data editing tool of a relational database using a GUI (graphic user interface).

1-2. Information Registration Process Next, the information registration process in the WWW page search system according to the present embodiment will be described in detail with reference to the flowchart of FIG. 2 illustrating the procedure of the information registration process.

First, in step 201, the concept extraction device 101 extracts a concept from a natural language document 202. As a result, the natural language document 202 becomes a set 203 of concepts.
Is converted to The method of concept extraction in the step,
Various methods are conceivable for the method of holding the concept set 203. The concept extraction method is described in “3. Extraction of Concept Information”, and the concept set holding method is described in “2.
Expressions of concept sets ". Further, a more detailed configuration, operation, and the like of the concept extracting apparatus 101 will be described in “3. Extraction of Concept Information”.

On the other hand, in step 204, document location information (document ID) 205 of the natural language document 110 is obtained by the concept registration device 102. Here, the natural language document 110
The document location information 205 corresponds to a URL (Uniform Resource Locator) in the case of a WWW page search system.

Next, in step 206, the concept registration device 102 extracts the concept set 2 extracted in step 201.
03 and document location information 2 obtained in step 204
05 are set and registered in the database 104. That is, the conventional inverted index (see FIG. 51)
The concept information transposed index (see FIG. 38) of the data structure in which the keyword portion in the is replaced with the concept information is registered. Details of a method of registering concept information in the database 104 and a data structure to be registered will be described in “4. Registration of concept information”.

1-3. Information Search Process Next, the information search process in the WWW page search system according to the present embodiment will be described in detail with reference to the flowchart of FIG. 3 illustrating the procedure of the information search process.

As in the case of the conventional example, there are mainly two types of search text input by the user via the keyboard 109 in the information search process, and the processing partially differs depending on the type of the search text. One is a case where a search sentence in a natural language is directly input, as in the conventional example, and the other is a case where a concept search sentence is input based on a predetermined grammar. Although the concept search sentence is different from the keyword search sentence in the conventional example, the concept search sentence can be regarded as similar in the sense that the extracted information can be directly accessed to specify the condition. Specifically, an example such as SQL is an example, and details regarding this will be described in “5. Search for Conceptual Information”.

In the flowchart of FIG.
In 1, the type of the search sentence (natural language search sentence or concept search sentence) is determined, and the process is branched.

If the search sentence is a natural language search sentence 303, the process proceeds to step 302, where the concept extraction device 101 extracts a search concept. Various methods can be considered for the extraction method of the search concept and the internal expression method. These methods will be described in “5. Search for Concept Information”.

The set of search concepts extracted at this stage is
Here, it is called a search concept set 306.

If the search sentence is a concept search sentence 305, the process proceeds to step 304, where the input concept search sentence 305 is grammatically analyzed by the concept search sentence analyzer 105 and converted into a search concept set 306. You. Although the grammar here depends on the system, it is assumed that a grammar having a capability like an SQL search sentence is defined. in this case,
The search concept set 306 is not only a set of concepts, but also holds information such as what to search and what conditions are. The details will be described in “5. Search for conceptual information”.

Further, a system that automatically generates the search concept set 306 by any method without input from the keyboard 109 is also an object of the present invention. For example, it is conceivable to monitor an electronic mail exchanged between humans and create a search concept set 306 for automatically searching for relevant information from the contents of the electronic mail. In this way, there is an advantage that the user can reduce the work of inputting a search sentence via the keyboard 109.

As described above, the natural language search sentence 303
Alternatively, the search content specified by the concept search sentence 305 is temporarily converted into a search concept set 306, and step 3
At 07, a search process from the database 104 is performed by the concept search device 103 based on this. And
At step 308, the result will be displayed on the display 108.

[0116] 2. Expression of concept set As described in the description of “1-2. Information registration processing”, various methods can be considered as to in what form the extracted concept set 203 is held. As an example, a method of expressing the concept set 203 by a data structure like a record in a database will be described. The method of extracting the concept expression from the natural language document 202 will be described in the next section “3. Extraction of Concept Information”.

In the following, a method for expressing the concept set 203 by a record-like data structure of the database will be described. In this method, since the concept set 203 is a set of records, the table of the database 104 corresponds to the concept set. For example, the natural language document of FIG. 4 cited in the conventional example is converted into a concept set table as shown in FIG. 5 by a concept extraction process (step 201 in FIG. 2). That is, in FIG. 5, the record representing each product corresponds to the extracted concept.

The concept set extracted as shown in FIG. 5 is different from the conventional keyword set (see FIG. 52) in the following two points.
The quality of information is different. The first is “information structuring in concept units”, which are concept units (here, product units).
The information is separated and extracted as a record in the database 104. The second is "information structuring of attribute units of concepts and grasping of meanings of attributes", in which information is separated into attribute units (corresponding to columns of a table in this case) held by the concepts. From another point of view, it can be seen that the keywords related to the concept are arranged in units related to the concept. At the same time, in this case, as can be seen from the column name given to the column, it is understood that the type (or meaning) of the relationship with the concept is grasped.

Due to such a difference in the quality of the information, it can be considered that the information missing in the conventional keyword set (see FIG. 52) is stored. Specifically, the following characteristic items can be mentioned.

First, the context information given by the “Omron Product Information” of the title of the natural language document (FIG. 4), in which all the document contents under the title describe Omron's products, is a conventional example. 52 is omitted in FIG. 52, but the information is stored as a column of “manufacturer” in FIG. 5 of the present embodiment.

Second, in a natural language document (FIG. 4),
The product category context information for each bulleted item, given by the bulleted headings (a), (b), and (c),
Although missing in FIG. 52 of the conventional example, FIG.
In, information is stored as a column of “product”.

Third, in a natural language document (FIG. 4),
Depending on the item indicated by "●", two independent items in (a)
The structure information that there is one product information is shown in FIG.
However, in FIG. 5 of the present embodiment, the information is stored as separate records.

Fourth, in a natural language document (FIG. 4),
Regarding the medical device approval number of the sphygmomanometer of each product shown in the table, information on which product corresponds to which product is missing in FIG. 52 of the conventional example, but in FIG. Stored as the “Medical Device Approval Number” column.

Fifth, in a natural language document (FIG. 4),
In the conventional example shown in FIG. 52, information on the release of each product expressed in a natural language sentence lacks information on which product, but in FIG. 5 of the present embodiment, the “model”, Stored as "Release date" and "Price" columns.

By preventing the information loss as described above from the first to the fifth, it is possible to obtain a search result that correctly grasps the user's intention. For example, it can be seen that “12,500 yen sphygmomanometer” is “HEM737”, and if it is stored in a database together with document location information, document 3
It is also possible to find out. On the contrary, since it is understood that there is no information of "12,500 yen thermometer", unlike the conventional example, the natural language document of FIG.

In FIG. 5, the concept information is explained only with the table relating to the product. However, in practice, a plurality of tables such as a table relating to a company may exist simultaneously. Further, by describing key items of a certain table in columns of another table, a relation can be defined like a relational database. For example, if there is a company table, the company name column is a key column, and one record of the company table is OMRON data, and the company name is “Omron”, the manufacturer column of the product table is “Omron” Would be related to OMRON data in the company table. That is, by simultaneously extracting information from these two tables, it is possible to realize an information retrieval apparatus that can withstand more complicated data retrieval. In this case, an effect equivalent to expressing the concept in the frame of the frame theory can be obtained.

[0127] 3. Extraction of Concept Information Next, as a more detailed explanation of the concept extraction process (step 202 in FIG. 2), a method of extracting concept information, and a method of mainly extracting data like a record in a database as concept information will be described as an example. It will be described in detail.

3-1. Extraction of conceptual information using semantic analysis First, a method of performing conceptual analysis (case analysis) to extract conceptual information will be described. FIG. 6 is a system configuration diagram of a concept extraction device using semantic analysis, and FIG. 7 is a flowchart illustrating a concept information extracting process using semantic analysis.

In FIG. 5, the concept extracting apparatus of this embodiment includes a morphological analyzer 601 for extracting morphemes from a natural language document, a verb extracting apparatus 602 for extracting only verb components in morphemes, and a case element on which the verb depends. Verb case frame dictionary 605 that describes semantic information and surface case condition information, etc., and semantic dictionary 606 that performs surface definition of semantics
Element specifying device 6 for specifying a case element corresponding to a verb
03, case element / concept correspondence information 607 holding correspondence information between case elements and concept information, and case element / concept correspondence information 60
7 is provided with a concept creation device 604 for creating concept information based on the C.7.

In the concept extracting apparatus of the present embodiment, it is possible to extract concept information by utilizing information of semantics on which a verb depends. Further, it is possible to extract concept information by utilizing information of surface case elements on which a verb depends without reference to a semantic dictionary. Furthermore, it extracts concept information by utilizing both the semantic and surface case information on which the verb depends, and in the case of a sentence in which case particles representing the surface case are omitted, or in the case of sentences containing multiple identical semantic words. In such cases, it is possible to specify the case element.

Next, the concept information extracting process using the semantic analysis in the concept extracting apparatus of the present embodiment will be described with reference to the flowchart of FIG. First, as a first specific example, when a natural language document 702 as shown in FIG. 8 is given, the concept extracting apparatus according to the present embodiment extracts concept information as shown in FIG. 711
(Product table) is output. Hereinafter, the first specific example will be described according to the operation of each step of the flowchart in FIG.

First, in step 701, a morphological analysis is performed on the natural language document 702 (see FIG. 8) by the morphological analyzer 601 to output a morpheme 703. Then, in step 704, the verb extraction device 602 uses the morpheme 703
Extract verbs from In this case, the verb “release” is extracted.

Next, in step 705, the case element specifying device 603 examines the condition of the case element on which the verb “release” depends by using the verb case frame dictionary 605. FIG. 10 illustrates a verb case frame dictionary 605 to which correspondence information to a concept is added. The verb case frame dictionary 605 shown in the figure is configured to include a verb case frame dictionary and information corresponding to a concept. The correspondence information to the concept will be described later, and the configuration of the verb case frame dictionary will be described here. As shown in FIG. 10, a general verb case frame dictionary may be considered as a portion in which the semantic of the case element on which the verb depends and the condition of the surface case are described. For convenience, in FIG. 10, each case element is numbered from [1] to [5].

Next, in step 707, the case element specifying device 603 stores the verb case frame dictionary 605 and
The morpheme 70 extracted by the morpheme analysis (step 701)
Then, the morpheme corresponding to each case element is specified from the information of the semantic element or the surface case.

First, in order to specify a morpheme corresponding to a case element from information of a semantic element, the semantic element dictionary 606 is referred to. That is, the semantics are defined in a semantic dictionary 606 (generally called a semantic analysis dictionary or the like) as shown in FIG. The semantic dictionary 606 describes words or patterns included in the semantics. If the words or patterns match, the morpheme is regarded as the semantic. For example, “July 1, 1997” is a semantic of “<date>” because it matches the pattern “<integer> year <integer> month <integer> day” described in the semantic dictionary 606. You can see that. In FIG. 11,
<...> represents a semantic, and <...> :: ... represents the definition of the semantic. Also, or is used as a special character string representing a disjunctive element, and (...) represents a definition by explanation. In addition,: indicates that the same definition is made for various semantics, and ... indicates that the definition content can be considered other. The other characters represent the characters themselves.

In addition, even if a morpheme such as an unknown word does not match the semantic dictionary 606, the case element can be specified if the information of the surface case matches. The surface case is the case particle "ha", "ga", "wo", "de",
Determined by "ni", "kara", etc., "ha", "ga"
Is "ga" and "wo" is "ヲ". However, since the surface case on which the verb depends depends on the passive voice and the embedded sentence, a separate dictionary may be created for the passive voice and the embedded sentence.

As described above, there is a method of specifying a case element only by a semantic element, or specifying a case element only by a surface case. However, in the case of a verb whose semantic depends on the same case elements, it is difficult to identify the case element only by the semantic, and the surface case is often omitted in actual natural language sentences. In such a case, it becomes difficult to specify the case element using only the surface case. In this embodiment, even in such a case, the judgment is made by simultaneously utilizing the semantic element and the surface case,
Case elements can be specified.

With the above processing, the natural language document 702
(See FIG. 8), the following case elements are extracted. That is, [1] Omron, [2] Sphygmomanometer, [3] "HEM-739", [4] July 1997, corresponding to the case element numbers in FIG.
One day, [5] is 16,000 yen.

Further, in step 709, the case element / concept correspondence information 6
07, the assignment to the concept information is performed.
Get 11. As shown in FIG. 10, the verb case frame dictionary 605 used in the present embodiment has a configuration in which information of “corresponding column” is added to a general verb case frame dictionary as information corresponding to a concept. Element / concept correspondence information 60
7 is provided in the verb case frame dictionary 605. That is, in the case element concept creating apparatus 604, the case elements from the numbers [1] to [5] identified in step 707 are referred to by referring to the correspondence information to the concept in the verb case frame dictionary 605 in FIG. Is assigned to each corresponding column of the “product table” shown in FIG. For example, the case element “Omron” extracted by the number [1] is determined by the “manufacturer” of the “product table” according to the information “product table.maker” described in the corresponding column of the verb case frame dictionary 605. Assigned to the column. Here, the reason why the table name is specified in the column of the corresponding column of the verb case frame dictionary 605 is to make it possible to cope with a case where the conceptual information to be extracted is obtained as a plurality of tables. .

By following the procedure described above, the concept information 711 (see FIG. 9) can be obtained from the natural language document 702 (see FIG. 8).
Reference) can be extracted.

In many cases, pure semantic analysis is performed using only verb case frames and semantic information, and the use of surface case information or the like falls in the category of syntactic analysis (ie, syntactic analysis). . Here, the attribute information of the concept is used as an example of the column of the concept information. Naturally, it is also possible to extract the organized concept information. In this case, the semantic dictionary 6
Since it is not necessary to provide the value of "06" and only the syntax analysis needs to be performed, there is an advantage in that knowledge construction is not required.

Next, as a second specific example, when a natural language document 702 as shown in FIG. 12 is given, the concept extracting apparatus of the present embodiment extracts the concept information as a result.
The concept information 711 (product table) as shown in FIG. 13 is output. Hereinafter, the second specific example will be described.

Unlike the first specific example (FIG. 8), the natural language document of FIG. 12 has no verb. Alternatively, it can be regarded as a "be verb" in English. Such a be
In the case of a verb such as a verb used in various situations, it is difficult to specify the case element on which the verb depends, and it is difficult to create the verb case frame dictionary 605. Even if it can be created, it is often useless because various semantics can be handled.

To solve such a problem, FIG.
An attempt is made to generalize the verb case frame dictionary 605 shown in FIG. What is performed by the verb case frame dictionary 605 is as follows.
It is organized from the viewpoint of the action event represented by the verb “release”, and can be searched in the form of case elements on which the verb depends. That is, in the example of FIG. 14, the search can be performed from the "release" 1401 and the related semantics "company" 1402 to "model" 1406 as case elements.

The reason why the rearrangement is performed mainly on the verb is that the situation is limited by the verb, and it is easy to limit the candidates of the case element related thereto. However, even if the verb is limited, as in the case of the be verb, etc., if the case element candidates related to the verb cannot be limited, the verb is not forcibly considered, and even if other nouns, etc. It is considered that the effect is obtained. Therefore, in such a case, a noun case frame is created by using a noun instead of the verb of the verb case frame shown in FIG.
) Can be extracted from the conceptual information (see FIG. 13).

More specifically, morphemes (constituents) in a natural language document are irrespective of verbs, nouns, etc.
In order to request dependency (or co-occurrence) for other morphemes (constituents), create them as "information extraction knowledge", and then apply those knowledges to determine the most appropriate It is possible to determine a high case element (or co-occurrence element) and extract the concept information using the result. The context information extraction knowledge of FIGS. 23 and 24 described in “3-2. Extraction of Concept Information Using Context Analysis and Structure Analysis” can be regarded as an example of this information extraction knowledge.

3-2. Extraction of conceptual information using context analysis and structural analysis In “3-1. Extraction of conceptual information using semantic analysis”, a method of extracting conceptual information from natural language documents using semantic analysis was described. The problem that can be solved is mainly the problem of "the fifth lack of syntactic and semantic information" in the information lack described in the "problem to be solved by the invention". The main problem in spoken languages is this "lack of syntactic and semantic information", but in the text language, other "lack of contextual information about paragraphs"
“Lack of contextual information about bullets”, “lack of structural information about bullets”, and “lack of contextual / structural information about tables” are also important issues. In order to extract the concept set shown in FIG. 5 from the natural language document shown in FIG. 4 while solving such other missing information, the following context analysis and structure analysis are required. .

In the processing of the context analysis and the structure analysis in the present embodiment, a document or a document group is considered as a hierarchical context area. Here, the "context area" means "document", "paragraph", "line",
"Bulleted list", "bulleted item", "table", "table row", "table column", "table cell", "hyperlink"
Such a structure. Each context region may include a lower context region therein. Further, the lower context area can automatically refer to the upper context area, and the concept information extracted in the upper context area of the reference destination can be used as the context information.

FIG. 15 is a system configuration diagram of a concept information extracting apparatus using context / structure analysis in the present embodiment, and FIG. 16 is a flowchart for explaining concept information extraction processing using context / structure analysis. Is shown.

In FIG. 15, the concept extracting apparatus of the present embodiment regards a group of natural language documents as a hierarchical context area, and sets a start point condition, an end point condition, and a subordinate context area for each of the time domain areas. A context hierarchical structure analysis device 1501 that prepares context hierarchical structure information by analyzing the hierarchy of the document structure of a natural language document with context region extraction knowledge describing the candidate With the knowledge of context information extraction describing the information of the information, it is possible to extract the context information from a sentence containing only words, a sentence containing only noun phrases, a sentence containing no words, etc. A context information extraction device 1502 for storing context information in the hierarchical structure information; and a lower-level context utilizing a context hierarchical structure when extracting concept information while determining a concept description area. Concept description area determination type that performs more accurate concept extraction by referring to the context information stored in the upper context area from the area and utilizing the context information stored in the lower context area from the upper context area Concept information extraction device 1
503 (first configuration). Here, the “context hierarchical structure” refers to various logical structures appearing in a natural language document, that is, a document, a paragraph, a bullet, a bulleted item, a table, a row, a column and a cell in the table. The details of other words used will be clarified in the following description.

Next, the concept information extracting process using the context / structure analysis in the concept extracting apparatus of the present embodiment will be described with reference to the flowchart of FIG. As a specific example, when a natural language document as shown in FIG. 4 is given,
Consider a case where a concept set as shown in FIG. 5 is output as a result of extracting the concept information.

(1) Context Hierarchical Structure Analysis First, in step 701, the context hierarchical structure analyzing device 15
01 recognizes the hierarchical structure of the context area in the natural language document (see FIG. 4). 17 and 18,
5 shows a result of performing a context structure analysis on the natural language document of FIG. 4. Here, FIG. 17 shows a correspondence between a natural language document and a context area, and FIG. 18 shows an internal data structure representing a context hierarchical structure.

The context areas AR1 to AR25 in FIG. 17 correspond to the nodes ND1 to ND25 in the internal data structure in FIG. 18, respectively. That is, the internal data structure represents a context area, and from FIG. 18, each context area has a lower context area, and it is possible to refer to the upper context area from the lower context area.
It can be seen that the reference is indicated by an arrow in FIG. For context areas such as paragraphs ND2 and bullet points ND3, there is only one upper context area that can be referred to, but for table cells ND13 to ND19 and the like, rows (ND9 to N
D11) and two context areas (ND12, ND13) can be referred to as upper areas. 17 and FIG.
Although not shown in the specific example of FIG. 8, a hyperlink or the like can refer to an arbitrary number of context regions as upper regions.

There are several methods for analyzing the natural language document shown in FIG. 4 and obtaining the analysis results shown in FIGS. 17 and 18. For example, when a document includes a table using ruled lines or the like, a method for recognizing the table portion is disclosed in
No. 116970, "Automatic Extraction Processing Method for In-Table Data"
May be used, or
As a method of recognizing a bulleted part in the case of a natural language document that does not include a tag, a method described in “Natural language processing device with bullet processing function” in JP-A-7-295981 may be used. good.

In the present embodiment, application to a “WWW page search system” is assumed, and therefore, in this case, especially when the input is a tagged natural language document such as HTML (HyperText Markup Language), The analysis method of the hierarchical structure is described. Note that HTML is a markup language developed from SGML, but please refer to http://www.w3.org/MarkUp/ for HTML. FIG. 19 illustrates a natural language document when the natural language document of FIG. 4 is described in HTML.

As shown in the natural language document illustrated in FIG.
In the HTML description, a portion corresponding to each of the regional regions is separated by a tag. For example, "<TABLE>" and "</ TAB
The area surrounded by “LE>” corresponds to the context area (AR8) of “table”. The hierarchical structure is also expressed by the hierarchical structure of the tags. For example, in the itemized context area (AR3) surrounded by "<OL>" and "</ OL>", "<U
L> ”and“ </ UL> ”in a bulleted context area (AR
5, AR21, AR24) exist hierarchically. Accordingly, FIG. 20 shows the context region, the start point and end point of the context region, and the context region extraction knowledge for specifying the lower context region according to the tag type and the start and end of the tag. So you can have. However, "<LI" indicates the starting point of a bulleted item.
Since “>” can end the terminating symbol “</ LI>”, “<LI>” must also be a candidate as the end point of the bulleted item.

As described above, the context area, the starting point,
By storing the correspondence between the end point and the lower context area as extracted knowledge (FIG. 20), the context hierarchical structure can be analyzed.

FIG. 21 is a flowchart for explaining the processing procedure of the context hierarchy structure analysis processing (step 1601). First, in step 2101, a context area of "document" is created and set as a current context. Next, in step 2102, the leading position of the input natural language document is pointed.

Next, the following processes from step 2103 to step 2108 are repeated processes. First,
In step 2103, it is checked whether the input point of the natural language document matches the start point of the lower context area for the current context. For example, if the current context is “document”, it is determined whether the current context matches the start point of the lower context area “paragraph”.

If they match, the process proceeds to step 2104, where a lower context area is created and connected to the upper context area, and this is set as the current context. In the case of the beginning of the "document", since this position matches the condition of the starting point of the "paragraph", "paragraph" is created as a lower context area of the "document" and combined, and the current context is changed. It will be a "paragraph". Then, in step 2108, the natural language document is read (the input point is advanced), and the process from step 2103 is repeated again.

If it is determined in step 2103 that the input point does not match the start point of the lower context area, the flow advances to step 2105 to check whether the input point of the natural language document matches the end point of the current context. . If they do not match, the flow advances to step 2108 to read the natural language document (advance the input point) and repeat the processing from step 2103 again.

If it is determined in step 2105 that the end point of the current context matches the end point, the process proceeds to step 21.
Proceeding to 06, it is checked whether the current context is the highest context area (here, "document"). If it is the top context area, the whole analysis processing of the context hierarchical structure ends.

If it is not the top context area, the flow advances to step 2107 to set the upper context area as the current context. The processing from step 2103 is repeated again. By the above processing, the context hierarchical structure analysis processing (step 16)
01) becomes possible.

(2) Context Information Extraction Next, step 160 in the flowchart of FIG.
In 2, the context information is extracted by the context information extracting device 1502. Various methods can be considered for the method of expressing and extracting the context information. In the present embodiment, basically, “2. Expression of concept set” and “3-1.
Extraction of Concept Information Using Semantic Analysis ”and a method similar to concept expression and concept extraction by the method described in“ Method of Extracting Concept Information Using Semantic Analysis ”. This is considered to be a necessary and sufficient condition that the information to be extracted by the concept information extracting apparatus of the present embodiment is the concept information in the end, and that it is necessary to extract information equivalent to the concept information also in the context extraction. Because it can be done. However, since the context information is considered to have different characteristics from the concept information in the following points, it is necessary to devise an extraction method.

The first feature is that, in a natural language document expressed using a document structure, since the meaning is expressed by the document structure, it is often not described in complete sentences. That is, in many cases, the meaning is described in a word, a noun phrase, a verbal sentence without a verb, and the like, such as a headword. In such a case, "3-
1. Extraction of Conceptual Information Using Semantic Analysis ", it is conceivable that information cannot be sufficiently extracted by the sentence analysis performed by ordinary case analysis. Therefore, it is necessary to take into account a more flexible information extraction method, such as extracting information only from the semantic class of a word.

As a second feature, the concept information is stored as knowledge extracted by the system, but the context information is temporary knowledge for extracting the concept information.
This can be thought of as a short-term memory of the work of the brain, and is temporarily stored as context area data,
At the stage when the concept extraction is completed, it may be discarded. For this reason, it is necessary to store it in a different area from the concept.

First, the following two methods are considered to make it possible to deal with the first characteristic. The first method is that all context information is extracted based only on semantics. In this case, the extraction accuracy is likely to decrease because information on the case particles and word order cannot be used. .

In this case, the context information extraction knowledge for extracting the context information is held in a form utilizing only the semantics as shown in FIG. 22, and the semantic pattern described in the semantic column of this knowledge (The semantic dictionary 60 as shown in FIG. 11
6 is extracted in the column described in the corresponding column. Note that the table name is also described in the column of the corresponding column in FIG. 22 so that information of a plurality of tables can be simultaneously extracted.

As a second method, the semantic information of a morpheme, the semantics of another morpheme co-occurring, the postposition accompanying the morpheme, and the appearance position of the co-occurrence element Etc. as context extraction knowledge,
In this method, context information is extracted by using only information that can be used (that is, appearing in the text). By using this method, it is possible to collectively analyze various cases where the sentence is described only in words, noun phrases, sentences with no words, ordinary sentences, etc. There is.

This context information extraction knowledge is considered to be a generalization of the verb case frame to which the correspondence relationship with the concept shown in FIG. 10 is added and the knowledge of FIG. 22. Context information extraction knowledge utilizing semantic elements, language markers, word order, etc. of extracted elements / co-occurrence elements as shown in FIG. The column of the concept to be extracted is described in the column of “extraction column” in FIGS. 23 and 24, and the table name is also described. It is possible to simultaneously extract information of a plurality of tables. In order to

A character string corresponding to an extraction column is hereinafter referred to as an “extraction element”. What is described in the column of "semantic element of extracted element" is information of a semantic element which should be satisfied by the extracted element. This is a character string pattern defined by the semantic dictionary 606 as shown in FIG. 11, and can be considered as a character string pattern that the character string to be extracted should satisfy.

Further, what is described in the column of “postfix element of extracted element” is an element corresponding to a so-called “appendix” such as a case particle or an auxiliary verb for the extracted element. For knowledge in which specific elements are described in this column, it is necessary that the post-element described immediately after the extracted element appears. In the column of "co-occurrence element", a character string and a semantic element which co-occur with the extracted element are described. The column of "suffix element of co-occurrence element" describes an auxiliary word that needs to appear immediately after the co-occurrence element. In the column of "co-occurrence element appearance position", the appearance position relation between the extracted element and the co-occurrence element can be defined, and the position before, immediately before, immediately after, after, etc. Whether or not reference is possible can be specified.

The numerical value described in the column of “certainty” indicates the certainty that should be extracted in the extraction column when the extracted element matches the knowledge described above. For example, if the certainty factor is “1.0”, it indicates that the extracted element that matches the knowledge will surely correspond to the extracted column. The “knowledge ID” described in the last column is a knowledge ID numbered for convenience.

Therefore, the context information extracting device 1502 for extracting context information by the first method described above uses the method shown in FIG.
(1st configuration), the context information extraction knowledge as shown in FIG. 22 that describes the information of the column of the concept corresponding to the semantic is provided. It is possible to extract context information from only a sentence, a noun phrase only sentence, a sentence stop sentence or the like, and is realized by storing context information in context hierarchical structure information.

On the other hand, a context information extraction device 1502 for extracting a context by the second method is a morphological analysis device for extracting a morpheme from a natural language document, and a condition of a semantic element of each morpheme (element of interest). FIG. 23 which describes the conditions of the co-occurrence element co-occurring with the element of interest, the conditions of the adjunct elements of the element of interest / co-occurrence element, the conditions of the appearance position of the co-occurrence element, and the information of the column (corresponding column) of the corresponding concept. A context information extraction knowledge as shown in FIG. 24 and a corresponding column determination device for calculating the degree of correspondence to each corresponding column and extracting a morpheme into the corresponding column having the highest degree of correspondence are provided. The verb predicate sentence, adjective predicate sentence, noun It is possible to extract context information with high precision from predicate sentences, sentences containing only words, sentences containing only noun phrases, sentences with no speech, etc.
This is realized by one that stores context information in context hierarchical structure information (second configuration).

Next, the context information extracting device 1502 (the second method) having such a configuration is used to obtain the information shown in FIGS.
The process of extracting the context information by utilizing the context information extraction knowledge as shown in FIG. 4 (step 1602 in FIG. 16) will be described based on the flowchart shown in FIG.

First, in step 2501, the natural language document 2502 to be analyzed by the morphological analyzer is
Is performed, and the morpheme 2503 is stored in the memory. When the input natural language document is an HTML document or the like, tag information and the like are unnecessary at this stage and are deleted.

Next, in step 2504, the extracted morphemes 2503 are focused on one by one from the front, and this is assumed to be an extraction element. For example, in the case of the natural language document shown in FIG. 4, the first extracted element is “Omron”.

Next, in step 2508, the context area where the extracted element exists is pointed as the context information storage area. For example, in the example of the natural language document in FIG. 4 (see FIGS. 17 and 18), since the extraction element “Omron” exists in the context area AR2 corresponding to the paragraph node ND2, the paragraph node ND2 is set as the context information storage area. Point.
The context information storage area is a temporary storage area for storing context information as described above (the second feature of the context information). As the morpheme is read, the bullet node ND3 and the bullet item node ND4,...

Next, in step 2506, the extraction column correspondence degree table is initialized. Here, one extraction column correspondence table is created for each extraction element as shown in FIG. 27, and does not hold any information immediately after initialization.

In the context information extraction knowledge of FIGS. 23 and 24, the context information determination knowledge 2508 for determining whether or not the extracted element can be context information includes one for each record by a column other than the “extraction column”. Described as knowledge. In step 2507, for each extracted element, the context information determination knowledge described in the context information extraction knowledge 2508 is sequentially extracted one by one.

The context information determination knowledge 2508 describes semantics, postpositions, and conditions that the extracted elements and co-occurrence elements must satisfy in word order. Step 2509
Then, it is checked whether or not the extracted element matches the context information determination knowledge 2508.
At 510, the degree of correspondence to the extraction column is updated. If the correspondence degree for the extracted column is already stored in the extraction column correspondence degree table (see FIG. 27), the correspondence degree is recalculated and updated by calculation with the current correspondence degree. If there is no data on the degree of correspondence to the focused extraction column, the degree of correspondence is newly calculated and stored.

Here, various methods of calculating the degree of correspondence can be considered. As a method in the present embodiment, the initial value of the degree of correspondence is set to “0”, and the degree of matching of the extracted element with the context information discrimination knowledge is set. Then, a method of adding the certainty held in the column of “certainty” in the context information extraction knowledge 2508 to the correspondence is adopted. By adopting this method, the context information determination knowledge 2508 can be weighted, and the knowledge can be tuned by changing the certainty factor. However, other methods of calculating the degree of correspondence are also conceivable. For example, the initial value of the degree of correspondence is set to “0”, and the degree of correspondence is added by “1” each time the context information determination knowledge 2508 is matched. A method is also possible, and according to this method, context information extraction knowledge 2
Since it is not necessary to hold the information of “certainty” in the field 508, the necessary storage area can be reduced.

Steps 2509 and 251 will be described with reference to specific examples.
0 will be described. For example, the knowledge ID [RA1.
The context information discrimination knowledge of [1] is knowledge that the semantic element of the extracted element is <company>, and the extracted element that matches the knowledge has a high degree of correspondence to “product table.maker”. That is, in the case of the natural language document shown in FIG.
1], the degree of correspondence to “product table. Maker” is an initial value of the degree of correspondence “0.9”.
It is added to "0" and updated to "0.9".

FIG. 26 shows an example of an inter-knowledge correlation rule. The inter-knowledge correlation rule is a rule for enabling activation or suppression of other knowledge when a certain knowledge is matched. In the figure, the knowledge ID is described in the node frame, the numerical value described on the link is the activation index, and the activation index is 1.
If the activation index is less than 1.0, the activation index is less than 1.0, and the activation index is less than 1.0.

When utilizing the inter-knowledge correlation rule, each knowledge is given an activity, and at step 2504, the initial value of the activity is initialized (for example, to “1.0”). . Then, when updating the degree of correspondence to the extraction column in step 2510 in accordance with the knowledge, instead of simply adding the degree of certainty to the degree of correspondence, the degree of activity of the knowledge at that time is multiplied by the degree of certainty. Value is added to the degree of correspondence.
At this time, the activity level of the linked knowledge is also updated from the matched knowledge. The activity is updated to a value obtained by multiplying the activation index of the link by the original activity. For example, if the extraction element matches the knowledge of the context information discrimination knowledge [RA1.1] in the initial state (state where all the activities are “1.0”), the activity of the knowledge from RA1.2 to RA1.6 Is updated to "1.5". Although there are other methods of updating the activity, any method may be used as long as knowledge that easily co-occurs is activated, and knowledge that hardly co-occurs is suppressed. By introducing such a mechanism, it becomes possible to simultaneously match knowledge that is unlikely to occur at the same time, and to avoid a phenomenon that the degree of correspondence is not obtained well.

As described above, according to the determination at step 2511, RA1.2, RA1.
Steps 2507 to 2510 are repeated to sequentially check all the context information determination knowledge 2508, such as 2,. As a result, the degrees of correspondence to all the extracted columns that match one extracted element are obtained and stored in the extracted column correspondence table. In the case of the natural language document of FIG. 4, since only the context area AR1 is matched, only the degree of correspondence to “product table.
Fig. 27 shows the state of the extracted column correspondence table at this time. The correspondence is considered to be "0" for an extraction column for which the correspondence is not stored.

Next, in step 2512, it is checked whether or not the degrees of correspondence to all the extracted columns are equal to or less than a preset threshold value in the extracted column correspondence degree table. This threshold value may be “0” or a higher value. If there is at least one threshold value or more at step 2512, the process proceeds to step 2513, where the extraction column having the highest degree of correspondence is determined as the corresponding column. Storage area (AR in the example of the natural language document in FIG. 4)
2, ND2), a corresponding column and an extracted element are set and stored as context information 2515. In the example of the natural language document of FIG. 4, the state of the context information 2515 at this point is as shown in FIG.

Further, according to the determination in step 2516,
The series of processes from step 2504 to step 2514 is repeated for all morphemes. As a result, as shown in FIG. 29, finally, extracted elements that match the context information extraction knowledge 2508 are classified into a context area and a corresponding column, and are extracted as context information. The extracted context information is stored in the data structure of the corresponding context area.

The data of the degree of correspondence in the context information in the process of FIG. 28 is omitted from the finally extracted context information 2515 in FIG.
Also in step 5, the data of the degree of correspondence may be stored and used for ambiguity resolution in later processing.

It is to be noted that the repetition processing from step 2507 to step 2511 in the flowchart of FIG. 25 can be performed independently for each context information discrimination knowledge stored in the context information extraction knowledge 2508. , Processing can be performed at high speed. Also, the repetition processing from step 2504 to step 2516 can be performed independently for each of the extracted morphemes 2503. Similarly, by performing the processing in parallel, high-speed processing can be performed. .

Further, the context information extraction knowledge of FIGS. 23 and 24 can be described by a pattern such as a regular expression as shown in FIG. In FIG. 30, the context information extraction knowledge is described by an extended regular expression.
The following are used as special characters:
“X” represents an extraction element currently focused on. "<…>"
Represents a semantic element. “X :: Y” describes a condition where X satisfies the semantic of Y. "." Represents any single character. “*” Indicates 0 or more repetitions of the immediately preceding regular expression element. "?" Indicates that the immediately preceding regular expression element appears 0 or 1 time. "(...)" forms one regular expression element by combining a plurality of elements. "|" Is a disjunction (or or)
, Which means that the elements on both sides appear selectively.
Characters other than the above are treated as normal characters.

In this case, there is a disadvantage that the method of describing knowledge is special, but there are advantages that a plurality of co-occurring elements can be defined and the appearance position of the co-occurring element can be described more flexibly. In this case, the context information extraction device 1502 (second configuration) that implements the second method uses the pattern description knowledge such as a regular expression to define the conditions such as semantics and co-occurrence elements to be satisfied by each morpheme (element of interest). The information extraction knowledge as shown in FIG.

(3) Determination of Concept Description Area and Extraction of Concept Information By the above processing, extraction of context information (step 1602 in FIG. 16) has been performed. Next, in step 1603, concept description area determination type concept information The extraction device 1503 extracts the concept information while determining the concept description area. FIG. 31 is a flowchart illustrating a processing procedure for determining the concept description area and extracting the concept information by the concept description area determination type concept information extraction device 1503 according to the present embodiment.

Here, the "concept description area" refers to a context area in which one unit of information expressing a certain concept is described without excess or deficiency. In the example of the natural language sentence of FIG. 4, it is a context area in which information about one product is described.
7, AR22, AR25. The process of finding these context regions and extracting concept information therefrom is the process of step 1603. The details of the process of determining the concept description area and extracting the concept information are shown in FIG.
This will be described with reference to FIG.

First, in step 3101, the topmost context region is set as the search starting point of the concept description region in the context hierarchical structure analyzed by the context hierarchical structure analysis process of step 1601 in FIG. Scan to find the lowest context area. That is, a context area that is not referenced from anywhere is found. In the case of the natural language document of FIG. 4, in FIG.
Scanning is started from the context area of the document node ND1, and the paragraph node ND2, the bullet node ND3, the bullet item node ND4, the bullet node ND5, and the bullet item node N
By scanning with D6, the itemized item node ND6 having no lower context area is found as the lowest context area.

Next, in step 3102, the lowest context region is assumed to be a candidate region for the concept description region, and in step 3103, extraction of a concept information candidate from the context information 3104 is attempted. At this time, since the attribute information to be extracted may be pushed out of the candidate area as context information, the attribute information is stored in the context area higher than the candidate area by the context information extraction processing of step 1602 in FIG. It is assumed that the context information 3104 can be used as a candidate for attribute information. Further, it is assumed that context information of a context region lower than the assumed candidate region can also be used. That is, the context information stored in the concept description area currently assumed as the candidate area, the context information stored in the lower context area of the candidate area, the lower context of the lower context area Context information stored in the region, context information stored in a higher-level context region referenced by the candidate region,
・ Concept information is extracted by utilizing all the context information stored in a higher context area that is referred to by the upper context area, and the concept information is extracted by effectively utilizing the hierarchical context. Becomes possible. In the example of the natural language document of FIG. 4, in the case of the node ND6, the node ND6 itself and
Nodes ND5, ND4, ND3, N which are upper context areas
By examining all of D2 and ND1, concept information candidates as shown in FIG. 32 are extracted.

Next, in steps 3105 and 3108, the necessity / sufficiency of the conceptual information is checked. here,
The "conditions for which the concept information is necessary and sufficient" must satisfy the following three conditions. That is, the first necessary and sufficient condition is that information of essential attributes characterizing the concept to be extracted is extracted, and the second necessary and sufficient condition is that a plurality of concept units (in the example of the natural language document in FIG. 4, Multiple products) in the candidate area, and
Is that it contains new information.

First, the “necessary attribute characterizing the concept to be extracted” of the first necessary and sufficient condition is a unique key that determines one unit of the concept (one product in the example of the natural language document in FIG. 4). This information is set in advance as concept key knowledge 3106. For example, in the case of the product table shown in FIG. 32, one product can be represented by a set of the manufacturer, product, and model columns, and the concept key knowledge 3106 in this case is as shown in FIG. Become.

Next, whether or not a plurality of concept units satisfying the second necessary and sufficient condition are included is checked by utilizing the concept key knowledge 3106. That is, concept key knowledge 3
It is checked whether or not a plurality of attributes corresponding to 106 have been extracted. If a plurality of attributes are included, it is determined that the concept unit is a plurality and the second necessary and sufficient condition is not satisfied. For example,
When the node ND5 is assumed as a candidate area for the concept description area, the extracted concept information candidates are as shown in FIG. In FIG. 34, it is assumed that the data does not meet the second necessary and sufficient condition because two data of the combination of the manufacturer, the product, and the model are extracted.

Next, in order to check whether or not new information of the third necessary and sufficient condition is included, the already extracted concept information is extracted from the extracted concept information 310.
7 must be stored. That is, the extracted concept information 3107 is checked, and if there is extracted concept information that includes the concept information candidate, the third necessary and sufficient condition is not satisfied. For example, the node ND1
FIG. 35 shows the conceptual information candidates extracted when cell No. 6 is set as the candidate area. At this time, if the conceptual information candidates shown in FIG. Since the concept information candidate of 35 is included in the concept information of FIG. 32, the third necessary and sufficient condition is not satisfied. As another example, when a concept information candidate is extracted using the row of the node ND10, which is the upper context area of the cells of the nodes ND16 and ND17, as a candidate area,
As shown in FIG. 36, since the information in the column of “medical device approval number” is new information as compared with FIG. 32, in this case, on the contrary, the third necessary and sufficient condition is satisfied.

By checking the first to third necessary and sufficient conditions described above, it is possible to check the necessity and sufficientness of the concept information for one concept information candidate. In the example described above, the concept table to be extracted is only the “product table”. Need to be repeated.

In step 3108, if it is not possible to say that the concept information is necessary or sufficient for all the tables from which the concept is to be extracted, the flow advances to step 3109. If the current candidate area is not the highest context area, Step 31 sets the context region above the candidate region as the candidate region.
Return to 03. That is, the context region higher than the assumed one is assumed as a candidate of the concept description region again, and the above processing is repeated. However, when there are a plurality of upper context regions, the search is repeated for the plurality of context regions. For example, in the case of a table, if the necessity and sufficientness of the conceptual information cannot be determined on a cell basis, the next higher context is either a row basis or a column basis, and it is necessary to check both. Also, if there is no higher context area,
That is, if the current candidate area is the topmost context area, concept information duplication resolution processing is performed in step 3111, and the extraction of the concept information ends.

If it is determined in step 3108 that the concept information is necessary and sufficient for at least one table, the flow advances to step 3112 to store the concept information candidates as extracted concept information, and step 3113
In the above, the candidate area of the concept description area is found by searching for the next area at the same level as the current candidate area, for example, the lowest area starting from the node ND7 when the current candidate area is the node ND6. Is repeated.
If the next area does not exist at the same level, the area next to the upper context area, for example, the current candidate area is the node ND7.
In the case of, the node ND8 is set as a starting point of the candidate area search. If it is determined in step 3114 that there is no higher-level context area (currently the highest level), step 31
The process proceeds to step S11, where the concept information duplication solving process is performed, and the extraction of the concept information ends.

Concept information duplication solution processing (step 311)
At the stage before 1), the extracted conceptual information is in a state as shown in FIG. Although this is very similar to the final set of concepts in FIG. 5, the concepts extracted at nodes ND6 and ND10 and nodes ND7 and ND11 describe information for the same product, As a result, duplication of concept information occurs. In order to solve this, in step 3111, the concept key knowledge 3
106 is used again to search for a concept with the same concept key, and if present, duplicate information is eliminated to form one piece of concept information, so that the concept information as shown in FIG. 5 is finally extracted. Will be.

The concept extraction method using context information described in “3-2. Extraction of concept information using context analysis and structure analysis” is a processing method for a natural language document having no document structure. The method of “3-1. Extraction of concept information using semantic analysis” is included, and a natural language document having no document structure as shown in FIGS. . Extraction of conceptual information using semantic analysis ”.

However, when a long sentence of a natural language document (the sentence is intended to convey a plurality of intentions in one document at most) is included, it is assumed that the natural language document alone forms a context hierarchical structure. Because it is conceivable, not only the analysis of the context hierarchical structure using tags as described in “3-2. Extraction of conceptual information using context analysis and structural analysis”, but also the context hierarchical structure based on transition of the subject, conjunction, etc. Must be analyzed. A method to achieve this is described in, for example, the fourth document "Etsuko Yoshida, Shoichi Yokoyama;" A Method of Context Analysis Using Subjects / Focus ", IEICE Technical Report, Vol97,
No. 330, pp. 1-8, (1997)].

[0208] 4. Registration of Concept Information As described above, the concept information described in “2. Expression of Concept Set” was extracted by the concept information extraction process in “3. Extraction of Concept Information”. This corresponds to the process of step 201 in the flowchart of FIG. In the example of the natural language document in FIG. 4, the conversion to the concept set in FIG. 5 has been completed.

[0209] Even in such a concept set, the same weighting process as that of the keyword set can be performed, and the concept set can be formed by a set of the concept and the numerical value of the weight. The concept set obtained here is registered in the database 104 in step 206 together with the document ID 205 in FIG. 2 using the concept registration device 102 in FIG. It is considered that the data structure registered here is obtained by replacing the keyword portion in the transposed index (see FIG. 51) of the conventional example with the concept information, and is referred to herein as a “concept information transposed index”.

FIG. 38 shows an example of the concept information transposed index when the weighting process is performed. The weighted concept information transposed index has a data structure that can determine the weight of each concept of a document. This is a binary flag indicating that no data exists.

Unlike the conventional keyword transposition index (see FIG. 51), the index portion is made up of a combination of a plurality of pieces of information, so that partial information of the same concept is extracted from a plurality of documents. Could have been. That is, the price is described in “Document 1” and the medical device approval number is described in “Document 3”. By utilizing this later, in order to be able to search for “document 1” when it is desired to search for a document on which price information is described, information on the attribute of the corresponding concept (hereinafter, the corresponding attribute (Referred to as information). The corresponding attribute information is stored as a pair with the document ID. When this method is applied, for example, a record of a thermometer of type MC400 in FIG. 38 has a format in which corresponding attribute information is stored in each document column as shown in FIG. In the example shown in FIG. 39, the key manufacturer, product, and model column information is included in all documents, but the release date, price, and medical device approval number column information is document 1, document 1, respectively. 2. It can be shown that they are separately described in the document 3.

When a plurality of pieces of conceptual information (for example, a product and a company) are extracted at the same time, a conceptual information inverted index as shown in FIG. 38 can be created for each type of conceptual information. For example, when the product table and the company table are simultaneously concept extracted, there are two concept information transposed indexes, the product concept information transposed index and the company concept information transposed index.

Further, in the present embodiment, the method of registering the concept information in combination with the location information of the document has been described.
Only the concept information may be registered in the database 104. In this case, in the flowchart of FIG.
The acquisition of the document ID in step 204 and the resulting document ID are omitted, and in step 206 only the concept information is registered in the database 104.

[0214] 5. Retrieval of Concept Information Finally, details of the concept retrieval outlined in “1-3. Information retrieval processing” will be described. When the concept is expressed by a record, the search concept set 306 in FIG. 3 is to express the following two points. That is, first, which table information or which column information of which table is desired to be known, and secondly, which record information is desired to be known.

The first search request can be made by specifying a table or a column by using any means. For example, if "?" Is described in a column of a table, and the information in that column is answered,
The condition for wanting to know the release date of a product can be expressed as shown in FIG.

The second search request is made possible by describing conditions in columns of a table. For example, the condition that the user wants to know a product whose price is 12,500 yen can be expressed as shown in FIG.

FIG. 40 is similar in expression to the result of the conventional information extraction, but differs in that the condition is represented meaningfully. However, the same expression has the advantage that the algorithm and data structure at the time of information registration can be used for information retrieval.

Although FIG. 40 shows an example of specifying the columns of the table, there may be a condition that the user wants to know the entire table. The following three methods are conceivable as a technique for realizing this. As a first method, there is a method in which the condition of a column is described and there is no "?" In the table, and the user wants to know the entire table. As a second method, " As a third method, there is a method of separately preparing an area for expressing the desire to know the entire table.

The first method can be easily realized. However, there is a method for expressing a case where a condition is applied to a plurality of tables by using a reference between tables and all the tables are not desired to be known. There is a disadvantage that there is no. In addition, there is a drawback in that it is necessary to determine whether to perform information registration or information search based on different knowledge. In the second method, information indicating whether the user really wants to know each column or the entire column is missing.

Further, in the third method, although the storage area is large, it is considered that the lack of information is the least and the expressive power is high. Hereinafter, a specific example employing the third method will be described.

According to the third method, for example, a search request “I want to know the release date of an Omron sphygmomanometer with a price of 12,500 yen” can be expressed as shown in FIG. What is described in the notation example of FIG. 42 is that basically the conditions described in the same record are regarded as logical product conditions (ie, AND conditions). If you want to describe the condition of OR (ie, OR condition),
Either a plurality of records can be described in a table, or an OR condition can be described in a column of the table.

Further, when the columns of the table represent numerical values, it may be desired to describe the following inequality conditions. In this case, a simple comparison condition can be described by specifying that a comparison operator is described before a numerical value described as a condition. For example, "Omron's
A search request "I want to know about a sphygmomanometer of 15,000 yen or less" is expressed as shown in FIG. Similarly, there is a method of extending the operator so that the smallest one or the largest one can be obtained.

In addition to the above, a method of extending a column represented by a character string so as to enable a partial character string match by describing a head match condition and a tail match condition, and a condition of a character string itself are described. Instead of writing, it is possible to describe a semantic element so that all character strings classified as semantic elements can be expanded to match.

Further, it may be assumed that the user wants to know the contents of the column describing the condition. In the case of Exact Match of a character string, the character string written in the condition itself is the answer, so it is hard to imagine the use, but in the case of the above inequality condition of numeric value, the substring match of the character string, the meaning This is because, in the case of a prime match, the character string actually described may be different from the character string described in the condition. In order to make this expressible, the above method of writing "?" In the column of the table you want to know cannot be dealt with. Therefore, this problem can be avoided by providing a response request column for describing "?" For each column.

For example, in the specific example shown in FIG. 44, an answer request column for describing "?" Is provided for each column. FIG. 44 shows a search request for “I want to know the model and price of Omron's sphygmomanometer of 15,000 yen or less”. Here, it should be noted that both the condition and “?” Are described in the price column.

Although the discussion so far has mainly been about preparation for considering the search concept extraction processing (step 302) in the flowchart of FIG. 3, the concept search The present invention can also be applied to the case where a search concept set 306 is created directly from the search concept set 306.

First, consider the case where the concept search sentence 305 is equivalent to SQL (or a subset thereof). In this case, the above “?” Corresponds to SELE.
This is the column name described immediately after “SELECT” in the CT statement.
The conditional expression described in each of the above columns is
This corresponds to a conditional expression described after “WHERE” in the T statement. Thus, it is easy to convert the SQL subset into the expressions shown in FIGS. 40 to 44. In this case, the parsing of the search sentence (step 304) corresponds to the conversion process. Thus, for example, "SELECT type, price FROM
Product table WHERE Manufacturer = "Omron" AND Product = "
The SQL of the sphygmomanometer “AND price <= 15000;” is converted into the expression of FIG. Also, the column for knowing the entire table can be regarded as equivalent to "SELECT *".

Also, by realizing the expressions themselves as shown in FIGS. 40 to 44 by using a GUI instead of using the concept search sentence 305 as an SQL-like one, there is no need for specialized knowledge on the description method such as the SQL. It is also possible to realize a GUI or the like that can create a set of search concepts.

Using the expression method of the search concept set 306 as described above, the search concept extraction for converting the natural language search sentence 303 into the search concept set 306 in the information search process (FIG. 3) of the WWW page search system will be described below. A method of realizing the process (step 302) will be described. Here, it is assumed that basically the same processing as the processing described in “3-2. Extraction of conceptual information using context analysis and structural analysis” is performed.

Now, consider a method of extending the context information extraction knowledge of FIGS. 23 and 24 to general information extraction knowledge that can be used for both information registration processing and information search processing. The knowledge of FIGS. 23 and 24 is applied to the “extraction element” currently focused on, and if applicable, the degree of correspondence to the “extraction column” is increased. Itself is stored as concept information. In order to extend this and apply the expression method described with reference to FIGS. 40 to 44, knowledge for increasing the degree of correspondence to the intention of registration and knowledge for increasing the degree of correspondence to the intention of search are required. If they are separately tabulated and finally the degree of correspondence to the search intention is high, "?" May be stored in the column of the corresponding concept information. Then, for the sake of semantic consistency, some of the column names in the context information extraction knowledge of FIGS. 23 and 24 are changed as follows. However, the algorithmic treatment for each is the same.

“Extraction column” → “Target column” “Semantic element of extracted element” → “Element of interest” “Suffix element of extracted element” → “Suffix element of element of interest” Information reflecting such changed part The extracted knowledge is shown in FIG.
This is shown in FIGS. 46, 47 and 48. Here, the above three column names are changed, and a column of “intention” is added. If the knowledge matches “knowledge” described in the column of intention, the element of interest itself is added to the concept information. If you save and match the knowledge described as "search"
Save "?" In the concept information.

When the retrieval concept information is extracted using the information extraction knowledge shown in FIGS. 45 to 48, the process of updating the degree of correspondence to the extraction column (target column) in the flowchart of the context information extraction process of FIG. Step 2510)
At this time, the degree of correspondence is determined for each pair of the target column and the intention. In other words, "product table. Maker (registration)" and "product table. Maker (search)" are considered different things.

In the context information extraction process (FIG. 25) of storing the corresponding column and the extracted element in the context information storage area (step 2514), if the corresponding column is intended to be registered, the extracted element (target element) Is stored as context information, and if the corresponding column is intended to be searched, “?” Is stored as context information.

In the subsequent processing, retrieval concept information can be extracted by treating "?" In exactly the same way as other information. However, only the check using the concept key knowledge 3106 is performed at the time of checking the necessity / sufficiency as the concept information in the flowchart of the concept description area determination and the concept information extraction processing in FIG. 31 (step 3105). If the condition "want to know about the entire table" described in (1) is extracted, it is assumed that this condition alone corresponds to the concept key, and the sufficiency can be said.

The description of the search concept extraction process (step 302) for converting the natural language search sentence 303 into the search concept set 306 in the information search process (FIG. 3) of the WWW page search system is completed. Next, the database 104
(Step 307).

As described above, the expressions in FIGS. 40 to 44 can be considered as a subset of the SQL SELECT statement, and the mutual conversion is easy. Therefore, converting the search concept set 306 into SQL and searching for the concept information stored in a normal relational database can be performed within the range of known technology.

[0237] 6. Effect of Embodiment As described above, in the information search system (WWW page search system) of the present embodiment, the concept extraction device 10
1. Based on the keyword information extracted from the natural language document 202 and the information indicating the relationship between the keywords, concept information in which information is structured in concept units or attribute units is extracted. Registration device 102
Is registered in the database 104 and the concept search device 1
In step 03, necessary concept information is searched from the concept information registered in the database 104 in accordance with the search request of the user. In this way, conceptual information including syntactic, semantic, contextual, and structural information representing the relationship between keywords is extracted and registered, and the concept information or attribute unit is utilized by utilizing the relationship information between keywords. Since the search is performed based on the information, it is possible to prevent a loss of information indicating the relationship between the keywords, and to perform a highly accurate information search by effectively utilizing the conceptual information.

The information retrieval system (W
In the WW page search system), the concept extracting device 101 uses the keyword information extracted from the natural language document 202 based on information indicating the relationship between the keywords.
Concept information in which information is structured in concept units or attribute units is extracted, the extracted concept information is registered in the database 104 by the concept registration device 102, and provided by the user by the concept search sentence analysis device 105 (concept search device). Based on the keyword information extracted from the natural language search sentence and the information indicating the relationship between the keywords, search concept information in which information is structured in concept units or attribute units is extracted and stored in a database according to the extracted search concept information. Search necessary concept information from the registered concept information. As a result, concept information including syntactic, semantic, contextual, and structural information representing the relationship between keywords is extracted and registered, and search concept information is also extracted for natural language search sentences, and keyword information is extracted. Since the search is performed using the information on the concept unit or the attribute unit as a condition by utilizing the relationship information between the keywords, it is possible to prevent the loss of the information indicating the relationship between the keywords, and to effectively utilize the concept information to obtain highly accurate information. Search becomes possible.

[0239] The conventional example is an information search system provided with an information registration device and an information search device based on keyword extraction, and specifies the location of the original information in response to an inquiry of a search sentence including a keyword. Output (or output the original information itself). On the other hand, the present embodiment is an information search system including a concept registration device and a concept search device based on concept extraction, and also holds structured information in the extracted concept information. It is also possible to answer a search without displaying the original information.

For example, when an information search with the contents shown in FIG. 42 is performed on the conceptual information extracted as shown in FIG. 5, the search result is “01/11/1996”. In this case, the location information of the original information may not be necessary. However,"…
In this case, as much information as possible about the product is desired, "the concept information and the document location information are combined in the form of a concept information inverted index as shown in FIG. Can also be displayed, and an accurate and large-volume information search can be performed.

That is, the information retrieval system provided with the concept registration device and the concept search device using the concept extraction can improve the search accuracy of the search function of the conventional information registration device and the information search device based on the keyword extraction. By having information structured in the form of concept information, it is possible to realize a new search function in which a response can be made without using the original document.

[0242]

As is apparent from the above description, according to the present invention, the concept extracting device (concept extracting step)
Based on the keyword information extracted from the natural language document and the information indicating the relationship between the keywords, concept information in which information is structured in concept units or attribute units is extracted, and the extracted concept information is stored in a concept registration device (concept). Registration in the database by a registration step), and a concept search device (concept search step) searches for necessary concept information from concept information registered in the database in accordance with a user's search request, and a syntax representing a relationship between keywords. Extract and register concept information including target, semantic, contextual, and structural information, and perform a search using information on the concept unit or attribute unit as a condition using the relationship information between keywords. Therefore, it is possible to prevent a loss of information representing a relationship between keywords, and to use the concept information effectively to perform information search with high accuracy. It is possible to provide an information processing method and a recording medium search system, in the information retrieval system.

Further, according to the present invention, the concept extracting device (concept extracting step) uses the concept information or the attribute unit based on the keyword information extracted from the natural language document and the information indicating the relation between the keywords. Is extracted from a natural language search sentence provided by a user by a concept search device (concept search step). The extracted concept information is registered in a database by a concept registration device (concept registration step). Based on the keyword information and the information indicating the relationship between the keywords, search concept information in which information is structured in concept units or attribute units is extracted, and necessary concept information is registered from the concept information registered in the database in accordance with the extracted search concept information. And search for syntactic, semantic, contextual, and structural expressions that represent the relationships between keywords. Extract and register concept information including various types of information, and also extract search concept information for natural language search sentences, and make use of the relationship information between keywords to condition information in concept units or attribute units. Since the search is performed, it is possible to prevent loss of information representing the relationship between keywords, and to effectively use the conceptual information to perform an accurate information search, an information search method in the information search system, A recording medium can be provided.

Further, according to the present invention, in the concept registration device (concept registration step), the extracted concept information is converted into an inverted index form in combination with the document location information of the natural language document, or the document location information and It is to be registered in the database in an inverted index format based on a set of corresponding attribute information indicating which attribute of the concept information is described in the document, and not only the concept information is output but also information of the original document. Can be used to increase the amount of information that can be output as a search result. Also, since it is possible to specify in which document the attribute information that is part of the concept information is described, only the appropriate documents are searched. An information search system, an information processing method in the information search system, and a recording medium that can be presented as a result can be provided.

Further, according to the present invention, in the concept extracting device (concept extracting step), a morpheme is extracted from the natural language document by the morphological analyzing device (morphological analyzing step) and extracted by the verb extracting device (verb extracting step). A verb case frame dictionary that describes information on the semantics on which the verb depends and a semantic that performs surface definition of the semantics by extracting only the verb component in the morpheme that has been extracted and using a case element specifying device (case element specifying step) The case element corresponding to the verb is specified with reference to the dictionary, and the concept information is created based on the correspondence information between the case element and the concept by the concept creating apparatus (case creating step from the case element) from the case element. Utilizing information on semantics on which the verb depends, extracts and registers conceptual information including syntactic and semantic information representing the relationship between keywords, and registers Since the search is performed using the relationship information between the keywords, information representing the relationship between the keywords, particularly syntactic and semantic information, is prevented from being lost, and the concept information is effectively used to obtain highly accurate information. An information search system capable of searching, an information processing method and a recording medium in the information search system can be provided.

Further, according to the present invention, in the concept extracting device (concept extracting step), a morpheme is extracted from the natural language document by the morphological analyzing device (morphological analyzing step) and extracted by the verb extracting device (verb extracting step). Only the verb component in the morpheme obtained is extracted, and the case element corresponding to the verb is extracted by the case element specifying device (case element specifying step) by referring to the verb case frame dictionary describing the information of the surface case on which the verb depends. Identify and create the concept information based on the correspondence information between the case element and the concept by the device for creating the concept from the case element (the step of creating the concept from the case element). Without using a semantic dictionary, concept information including syntactic and semantic information representing the relationship between keywords is extracted and registered, and the Since the search is performed using the relation information, information representing the relation between the keywords, in particular, syntactic and semantic information is prevented from being omitted, and the concept information is effectively used to achieve a simpler structure and a higher accuracy. An information search system capable of high-level information search, an information processing method in the information search system, and a recording medium can be provided.

According to the present invention, in the concept extracting device (concept extracting step), a morpheme is extracted from the natural language document by the morphological analyzing device (morphological analyzing step), and is extracted by the verb extracting device (verb extracting step). A verb case frame dictionary that describes the information of the semantics on which the verb depends and the information of the surface case is extracted by the case element specifying device (case element specifying step), and the surface The case element corresponding to the verb is specified by referring to the semantic dictionary to be defined, and based on the correspondence information between the case element and the concept by the concept creation device from the case element (the concept creation step from the case element). Concept information that includes syntactic and semantic information that expresses the relationship between keywords, utilizing information on the semantics on which the verb depends and information on the surface case. Since the information is extracted and registered, it is possible to specify the case element even in the case of a sentence where the case particle representing the surface case is omitted, or in the case of a sentence containing the same semantic plural. In addition, since the search is performed using the relationship information between the keywords, the information representing the relationship between the keywords, particularly the syntactic and semantic information is prevented from being lost, and the concept information is effectively used to perform a highly accurate information search. Information retrieval system, an information processing method in the information retrieval system, and a recording medium.

According to the present invention, in the concept extracting device (concept extracting step), a morpheme is extracted from the natural language document by the morphological analyzing device (morphological analyzing step), and each extracted morpheme (hereinafter referred to as the element of interest) is extracted. ), The condition of the co-occurrence element where the element of interest co-occurs, the condition of the ancillary element of the element of interest and co-occurrence element, the condition of the appearance position of the co-occurrence element, and the column of the corresponding concept. With reference to the information extraction knowledge describing the information of the corresponding column (hereinafter referred to as the corresponding column), the corresponding column determining device (corresponding column determining step) calculates the degree of correspondence to each corresponding column, and the corresponding column having the highest degree of correspondence. Since morphemes are extracted, it is possible to extract concept information from verb predicate sentences, adjective predicate sentences, noun predicate sentences, sentences containing only words, sentences containing only noun phrases, sentences containing nouns, etc. By effectively utilizing the information can be provided an information processing method and a recording medium information with high accuracy search can information retrieval system, in the information retrieval system.

Further, according to the present invention, in the concept extracting apparatus (concept extracting step), a natural language document is regarded as a hierarchical context area, and a start point condition, an end point condition, and a low order With reference to the context region extraction knowledge describing the context region candidates to become, the hierarchy structure of the natural language document is analyzed by the context hierarchy structure analysis device (context hierarchy structure analysis step) to obtain context hierarchy structure information. The context information extraction device (context information extraction step) extracts context information from a sentence in a natural language document by referring to context information extraction knowledge describing information of a column of a concept corresponding to a semantic element. The context information is stored in the context hierarchical structure information, and the concept description area determining type concept information extracting device (concept description area determining type concept information extracting step) Concept information is described by utilizing the context hierarchy structure information and the context information stored in the lower context region from the lower context region of the context hierarchy structure information or in the lower context region from the upper context region. The concept information is extracted while determining the area, and the hierarchy of the document structure is analyzed, and the concept information including the syntactic, semantic, contextual, and structural information representing the relationship between the keywords is obtained. It is extracted and registered, and it is decided to perform a search based on the information of the concept unit or the attribute unit using the relationship information between the keywords, so the context information about paragraphs and bullets, the structural information about bullets, the table context·
An information search system that prevents structural information, syntactic and semantic information, and the like, from lacking information indicating the relationship between keywords, and enables more accurate information search by effectively utilizing context information and concept information; An information processing method and a recording medium in an information search system can be provided.

Further, according to the present invention, in the concept extracting device (concept extracting step), a natural language document is regarded as a hierarchical context region, and the start point condition, the end point condition and the With reference to the context region extraction knowledge describing the context region candidates to become, the hierarchy structure of the natural language document is analyzed by the context hierarchy structure analysis device (context hierarchy structure analysis step) to obtain context hierarchy structure information. In the context information extraction device (context information extraction step), the morpheme is extracted from the natural language document by the morphological analysis device (morphological analysis step), and the meaning of each extracted morpheme (hereinafter, referred to as an element of interest) Element condition, condition of co-occurrence element co-occurring with the element of interest, condition of ancillary element of element of interest and co-occurrence element, appearance of co-occurrence element The corresponding column determination device (corresponding column determining step) determines the degree of correspondence to each corresponding column by referring to the information on the extraction conditions that describe the conditions of the columns and the information of the corresponding concept columns (hereinafter, corresponding columns). By calculating and extracting morphemes in the corresponding column with the highest degree of correspondence, the context information is extracted from the sentences in the natural language document, and the context information is stored in the context hierarchical structure information. In the extraction device (conceptual description area determining type concept information extracting step), the context hierarchical structure information and the context hierarchical structure information are stored in the lower context area in the upper context area or in the upper context area in the lower context area, respectively. Utilizing the context information, the concept information is extracted while determining the area in which the concept information is described. The hierarchy of the document structure is analyzed, and the key word is extracted. Extract and register conceptual information including syntactic, semantic, contextual, and structural information that represents the relationship between keywords, and use the relationship information between keywords to define information in conceptual units or attribute units as conditions. Information that represents the relationship between keywords, such as context information about paragraphs and bullets, structural information about bullets, context and structural information about tables, and syntactic and semantic information. Prevention and effective use of contextual and conceptual information to achieve more accurate information retrieval. In particular, the ability to specify multiple co-occurring elements and more flexible description of the co-occurring element's appearance positions Information retrieval systems and information retrieval systems capable of extracting context information with high precision from predicate sentences, adjective predicate sentences, noun predicate sentences, sentences containing only words, sentences containing only noun phrases, sentences with no speech, etc. An information processing method and a recording medium which can be provided.

According to the present invention, information extraction knowledge is
The condition of the semantic element of the element of interest, the condition of the co-occurring element, the condition of the attached element, and the condition of the appearance position of the co-occurring element are described and constructed as pattern description knowledge such as a regular expression. Can be specified multiple times, and the occurrence position of co-occurrence elements can be described more flexibly, so that verb predicate sentences, adjective predicate sentences, noun predicate sentences, sentences containing only words, sentences containing only noun phrases, sentences containing no An information retrieval system capable of extracting concept information or context information with higher accuracy and capable of performing accurate information retrieval by effectively utilizing the concept information or context information, an information processing method in the information retrieval system, and recording A medium can be provided.

Further, according to the present invention, in the concept extraction device (concept extraction step), when knowledge with information extraction knowledge matches, the knowledge which is likely to co-occur with the knowledge is activated and hardly co-occurs. Since knowledge refers to an inter-knowledge correlation rule that suppresses information, it is possible to determine a corresponding column with high accuracy, and as a result, an information search system capable of performing information search with high accuracy, an information processing method in the information search system, and recording A medium can be provided.

According to the present invention, in the corresponding column determining apparatus (corresponding column determining step), the calculation of the degree of correspondence is performed in parallel for each information extraction knowledge, or the determination of the corresponding column is processed in parallel for each morpheme. Therefore, it is possible to provide an information search system capable of high-speed processing, an information processing method in the information search system, and a recording medium.

Further, according to the present invention, in the concept description area determining type concept information extracting device (concept description area determining type concept information extracting step), the concept description area is extracted when determining the concept information while determining the concept description area. Using the concept key knowledge describing essential attributes of the concept that characterizes the concept to be extracted, the essential attributes are extracted, a check is made not to include a plurality of concept units, and a check is made to include new information. The present invention provides an information search system, an information processing method and a recording medium in an information search system capable of improving the accuracy of information extraction for each concept, and consequently performing information search with high accuracy. be able to.

Further, according to the present invention, when the concept information is extracted while the concept description area is determined in the concept description area determining type concept information extracting apparatus (concept description area determining type concept information extracting step), the concept information is extracted. The duplication of extracted concept information is eliminated by using concept key knowledge that describes the essential attributes of the concept that characterizes the power concept, so the accuracy of concept-based information extraction is improved, and as a result, highly accurate information retrieval is possible. An information retrieval system, an information processing method in the information retrieval system, and a recording medium can be provided.

Further, according to the present invention, in the concept extracting apparatus, a morpheme is extracted from the natural language document by the morphological analyzing apparatus, the syntax is analyzed by the syntactic analyzing apparatus, and the syntactic analyzing apparatus is formed by the concept creating apparatus from the syntactic element. The concept information is created by extracting preset syntax elements from the syntax elements such as ga-case, ヲ -case, subject / predicate, dependency / acceptance, etc. from the result of Concept information including syntactic information representing the relationship between keywords is extracted and registered from only the analysis results, and a search utilizing the relationship information between keywords is performed, so information representing the relationship between keywords, particularly syntactic information Information retrieval system capable of performing highly accurate information retrieval with a simpler configuration while effectively utilizing the concept information, and an information processing method in the information retrieval system. It is possible to provide a fine recording medium.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an information search system according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a procedure of an information registration process in the WWW page search system according to the embodiment.

FIG. 3 is a flowchart illustrating a procedure of an information search process in the WWW page search system according to the embodiment.

FIG. 4 is an explanatory diagram illustrating a natural language document.

FIG. 5 is an explanatory diagram illustrating a concept set (table) extracted from a natural language document;

FIG. 6 is a system configuration diagram of a concept extraction device using semantic analysis.

FIG. 7 is a flowchart illustrating concept information extraction processing using semantic analysis.

FIG. 8 is an explanatory diagram illustrating a natural language document (first specific example) in concept information extraction processing using semantic analysis.

FIG. 9 is an explanatory diagram exemplifying conceptual information (product table) as an extraction result from the natural language document of FIG. 8;

FIG. 10 is an explanatory diagram illustrating a verb case frame dictionary to which correspondence information to a concept is added.

FIG. 11 is an explanatory diagram illustrating a semantic dictionary.

FIG. 12 is an explanatory diagram exemplifying a natural language document (second specific example) in concept information extraction processing using semantic analysis.

13 is an explanatory diagram illustrating conceptual information (product table) as an extraction result from the natural language document of FIG. 8;

FIG. 14 is an explanatory diagram illustrating a correlation between semantics;

FIG. 15 is a system configuration diagram of a concept information extracting apparatus using context / structure analysis in the embodiment.

FIG. 16 is a flowchart illustrating concept information extraction processing using context / structure analysis.

FIG. 17 is a diagram illustrating a result of performing a context structure analysis on the natural language document of FIG. 4 and illustrating a correspondence relationship between the natural language document and a context region;

18 is a diagram illustrating a result of performing a context structure analysis on the natural language document of FIG. 4 and illustrating an internal data structure representing a context hierarchical structure.

FIG. 19 is an explanatory diagram illustrating a natural language document when the natural language document of FIG. 4 is described in HTML.

FIG. 20 is an explanatory diagram illustrating extraction knowledge of a context region.

FIG. 21 is a flowchart illustrating a procedure of a context hierarchical structure analysis process.

FIG. 22 is an explanatory diagram illustrating context information extraction knowledge in a format utilizing only semantics;

FIG. 23 is an explanatory diagram (part 1) illustrating context information extraction knowledge utilizing semantics of extracted elements / co-occurrence elements, language markers, word order, and the like.

FIG. 24 is an explanatory diagram (part 2) illustrating context information extraction knowledge utilizing semantics, language markers, word order, and the like of extracted elements / co-occurrence elements.

FIG. 25 is a flowchart illustrating a processing procedure for extracting context information by utilizing context information extraction knowledge.

FIG. 26 is an explanatory diagram illustrating an inter-knowledge correlation rule;

FIG. 27 is an explanatory diagram illustrating an extraction column correspondence degree table;

FIG. 28 is an explanatory diagram exemplifying a state of context information during processing;

FIG. 29 is an explanatory diagram exemplifying context information extracted by utilizing context information extraction knowledge;

FIG. 30 is an explanatory diagram illustrating context information extraction knowledge described in an extended regular expression.

FIG. 31 is a flowchart illustrating a processing procedure for determining a concept description area and extracting concept information.

FIG. 32 is an explanatory diagram illustrating extracted concept information candidates.

FIG. 33 is an explanatory diagram illustrating concept key knowledge.

FIG. 34 shows a node ND5 as a candidate area for a concept description area.
It is explanatory drawing which illustrates the conceptual information candidate extracted when it is assumed.

FIG. 35 is an explanatory diagram illustrating conceptual information candidates extracted when the cell of the node ND16 is set as a candidate area.

FIG. 36 is an explanatory diagram exemplifying concept information candidates extracted when a row of a node ND10 is set as a candidate area.

FIG. 37 is an explanatory diagram exemplifying a state of extracted concept information at a stage before concept information duplication solving processing;

FIG. 38 is an explanatory diagram exemplifying a conceptual information transposed index when a weighting process is performed;

FIG. 39 is an explanatory diagram exemplifying a conceptual information transposed index in a format holding correspondence attribute information.

FIG. 40 is an explanatory diagram illustrating an example (part 1) of a search concept set.

FIG. 41 is an explanatory diagram illustrating an example (part 2) of a search concept set;

FIG. 42 is an explanatory diagram illustrating an example (part 3) of a search concept set;

FIG. 43 is an explanatory diagram illustrating an example (part 4) of a search concept set;

FIG. 44 is an explanatory diagram illustrating an example (part 5) of a search concept set;

FIG. 45 is an explanatory diagram (part 1) of an example of information extraction knowledge generalized by enabling specification of a storage element;

FIG. 46 is an explanatory diagram (part 2) of an example of information extraction knowledge generalized by enabling specification of a storage element;

FIG. 47 is an explanatory diagram (part 3) of an example of generalized information extraction knowledge by enabling specification of a storage element;

FIG. 48 is an explanatory diagram (part 4) illustrating an example of generalized information extraction knowledge by enabling specification of a storage element;

FIG. 49 is a configuration diagram illustrating a system configuration of a conventional WWW page search system.

FIG. 50 is a flowchart illustrating a general procedure of information registration processing in a conventional WWW page search system.

FIG. 51 is an explanatory diagram illustrating an inverted index when weighting processing is performed on a keyword;

FIG. 52 is an explanatory diagram exemplifying a keyword set extracted when information of the natural language document in FIG. 4 is registered;

FIG. 53 is a flowchart illustrating a general procedure of an information search process in a conventional WWW page search system.

[Explanation of symbols]

101 Concept Extraction Device 102 Concept Registration Device 103 Concept Search Device 104, 4908 Database 105 Concept Search Sentence Analysis Device 106, 4909 CPU 107, 4910 Main Memory 108, 4911 Display 109, 4912 Keyboard 110, 4913 Natural Language Document Group 111, 4914 Data Bus 112, 4915 Internet 202, 303, 702, 2502 Natural language document 203 Concept set 205, 5008 Document ID 305 Concept search sentence 306 Search concept set 601 Morphological analysis device 602 Verb extraction device 603 Case element identification device 604 Concept from case element Creation device 605 Verb case frame dictionary 606 Semantic dictionary 607 Case element / concept correspondence information 703, 2503 Morpheme 711, 3115 Concept information 1501 Context hierarchy Analyzer 1502 contextual information extraction unit 1503 concepts describing area determined type conceptual information extraction apparatus AR1~AR25 context area ND1~ND25 node RA1.1~RA6.2, RB1.1~RB6.2
Knowledge ID 2508 Context information extraction knowledge 2515, 3104 Context information 3106 Concept key knowledge 3107 Extracted concept information RC1.1 to RC7.6 Knowledge ID 4901 Keyword extraction device 4902 Morphological analysis device 4903 Stop word removal device 4904 Stemming processing device 4905 To keyword Weighting device 4906 keyword registration device 4907 keyword search device 4916 keyword search sentence analysis device 5002, 5303 natural language document 5006 keyword set 5306 keyword search sentence 5308 search keyword set

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masamitsu Kamo 10 Okayama Todocho, Ukyo-ku, Kyoto, Kyoto

Claims (30)

[Claims] A concept extraction device for extracting concept information in which the information is structured in concept units or attribute units based on keyword information extracted from a natural language document and information indicating a relationship between the keywords; A concept registration device for registering concept information extracted by the concept extraction device in a database, and a concept search device for searching necessary concept information from concept information registered in the database in accordance with a search request of a user, An information search system, wherein a search is performed using information on the concept unit or the attribute unit as a condition by utilizing relation information between keywords. 2. A concept extraction device for extracting concept information in which the information is structured in concept units or attribute units based on keyword information extracted from a natural language document and information indicating a relationship between the keywords, A concept registration device for registering concept information extracted by the concept extraction device in a database; and keyword information extracted from a natural language search sentence provided by a user and information representing a relationship between the keywords, the concept registration device including a concept unit or an attribute unit. A concept search device that extracts search concept information in which information is structured, and searches for necessary concept information from the concept information registered in the database according to the extracted search concept information; Information characterized by performing a search using information of the concept unit or the attribute unit as a condition by utilizing information Search system. 3. The concept registering device stores the concept information extracted by the concept extracting device in an inverted index format based on a combination with document location information of the natural language document, or
3. The method according to claim 1, wherein the database is registered in an inverted index format based on a set of the document location information and corresponding attribute information indicating which attribute of the concept information is described in the document. Information retrieval system described. 4. The concept extracting device, comprising: a morphological analyzer for extracting a morpheme from the natural language document; a verb extracting device for extracting only a verb component in the extracted morpheme; and a semantic on which the verb depends. Verb case frame dictionary that describes the information of a verb, a semantic dictionary that performs a surface definition of the semantic, a case element specifying device that specifies a case element corresponding to the verb, and correspondence information between the case element and the concept And a device for generating concept information from a case element that generates concept information based on a verb, wherein the concept information is extracted by utilizing information of a semantic on which a verb depends. Information retrieval system described in. 5. A concept extraction device, comprising: a morphological analysis device for extracting a morpheme from the natural language document; a verb extraction device for extracting only a verb component in the extracted morpheme; and a surface case on which the verb depends. Verb case frame dictionary that describes the information of the verb, a case element specifying device that specifies a case element corresponding to the verb, and concept creation from case elements that creates concept information based on correspondence information between the case element and the concept The information retrieval system according to claim 1, further comprising a device, wherein the concept information is extracted by utilizing information of a surface case element on which the verb depends. 6. A concept extracting device, comprising: a morphological analyzer for extracting a morpheme from the natural language document; a verb extracting device for extracting only a verb component in the extracted morpheme; and a semantic on which the verb depends. A verb case frame dictionary that describes the information of the case and the information of the surface case, a semantic dictionary that performs a surface definition of the semantic, a case element specifying device that specifies a case element corresponding to the verb, and the case element. And a device for generating concept information from case elements that generate concept information based on information corresponding to the concept, and extracting the concept information by utilizing the information of the semantics on which the verb depends and the information of the surface case. The information retrieval system according to claim 1, 2 or 3, wherein 7. The concept extracting device, comprising: a morphological analyzer for extracting a morpheme from the natural language document; a condition of a semantic element of each extracted morpheme (hereinafter, referred to as a target element); Conditions of co-occurring elements, conditions of the element of interest and ancillary elements of the co-occurring elements, conditions of appearance positions of the co-occurring elements, and information of corresponding concept columns (hereinafter, corresponding columns) And a corresponding column determination device that calculates a degree of correspondence to each corresponding column and extracts a morpheme to a corresponding column having the highest degree of correspondence, wherein 2 or 3
Information retrieval system described in. 8. The concept extracting device, wherein the natural language document is regarded as a hierarchical context region, and a condition of a start point, a condition of an end point, and a candidate of a lower-order context region are described for each of the time domain regions. A context hierarchy analysis device for analyzing the hierarchy of the document structure of the natural language document to create context hierarchy information with context area extraction knowledge; and describing information of columns of semantics and corresponding concepts. A context information extraction device that includes context information extraction knowledge, extracts context information from a sentence in the natural language document, and stores the context information in the context hierarchy information; and the context hierarchy information and the context hierarchy. Concept information is described using the context information stored in the lower context area from the lower context area of the structure information or in the lower context area from the upper context area. 4. The information retrieval system according to claim 1, further comprising: a concept description area determining type concept information extracting device that extracts the concept information while determining an area that has been set. 9. The concept extraction device, wherein the natural language document is regarded as a hierarchical context region, and a start point condition, an end point condition, and a lower-order context region candidate are described for each of the time domain regions. A context hierarchy analysis device that includes context region extraction knowledge and analyzes the hierarchy of the document structure of the natural language document to create context hierarchy information; and a morphological analysis device that extracts morphemes from the natural language document. A condition of a semantic element of each of the extracted morphemes (hereinafter, referred to as an element of interest), a condition of a co-occurrence element in which the element of interest co-occurs, and a condition of an adjunct element of the element of interest and the co-occurrence element. Calculating the condition of the appearance position of the co-occurrence element, information extraction knowledge describing information of a column of a corresponding concept (hereinafter, corresponding column), and a degree of correspondence to each of the corresponding columns. A corresponding column determining device for extracting a morpheme into a corresponding column having the highest degree of correspondence, extracting context information from a sentence in the natural language document, and storing the context information in the context hierarchical structure information Context information extraction device, utilizing the context hierarchical structure information and the context information stored in the lower context region from the lower context region of the context hierarchy structure information to the upper context region, respectively, 4. The information retrieval system according to claim 1, further comprising: a concept description area determining type concept information extracting device that extracts the concept information while determining an area in which the concept information is described. . 10. The information extraction knowledge describes a condition of a semantic element of the element of interest, a condition of the co-occurrence element, a condition of the adjunct element, and a condition of an appearance position of the co-occurrence element. 10. The information retrieval system according to claim 7, wherein the information is described as knowledge. 11. The inter-knowledge correlation rule that activates knowledge that is likely to co-occur with the knowledge and suppresses knowledge that is unlikely to co-occur when the knowledge of the information extraction knowledge matches. 10. The method according to claim 7, further comprising:
Or the information retrieval system according to 10. 12. The apparatus according to claim 7, wherein the corresponding column determination device performs the calculation of the degree of correspondence in parallel for each information extraction knowledge, or the determination of the corresponding column in parallel for each morpheme. , 9, 10 or 11. 13. The concept extracting device includes a concept knowledge key describing essential attributes of a concept characterizing a concept to be extracted, and the concept description region determining type concept information extracting device determines a concept description region. While extracting the concept information, the concept key knowledge is used to extract the essential attributes, check that a plurality of concept units are not included, and check that new information is included. 9. The method according to claim 8, wherein the necessary and sufficient information is checked.
The information retrieval system according to 11 or 12. 14. The concept extracting device includes a concept knowledge key describing essential attributes of a concept characterizing a concept to be extracted, and the concept description region determining type concept information extracting device determines a concept description region. 14. The information retrieval system according to claim 8, wherein, when extracting concept information, the duplication of the extracted concept information is eliminated using the concept key knowledge. . 15. A concept extraction device, comprising: a morphological analysis device for extracting a morpheme from the natural language document; a syntax analysis device for analyzing syntax; and a ga-case / ヲ -case, subject / predicate, based on a result of the syntax analysis device. ,
A concept creation device for extracting concept information by extracting a preset syntax element from the syntax elements such as dependency and reception, and extracting the concept information only from the syntax analysis result. The information retrieval system according to claim 1, 2 or 3, wherein 16. A concept extracting step of extracting concept information in which the information is structured in concept units or attribute units based on keyword information extracted from a natural language document and information representing a relationship between the keywords, A concept registration step of registering concept information extracted by the concept extraction step in a database; and a concept search step of searching necessary concept information from concept information registered in the database in accordance with a user's search request; An information processing method in an information search system, wherein a search is performed using information on the concept unit or the attribute unit as a condition by utilizing relation information between keywords. 17. A concept extracting step of extracting concept information in which the information is structured in concept units or attribute units, based on keyword information extracted from a natural language document and information indicating a relationship between the keywords, A concept registration step of registering concept information extracted by the concept extraction step in a database; and keyword information extracted from a natural language search sentence given by a user and information indicating a relationship between the keywords, the concept being described in a concept unit or an attribute unit. A concept search step of extracting search concept information in which information is structured, and searching for necessary concept information from the concept information registered in the database according to the extracted search concept information; Utilizing information, it is possible to perform a search based on the information in the concept unit or the attribute unit. And an information processing method in the information search system. 18. The concept registering step includes: converting the concept information extracted in the concept extracting step into an inverted index format by combining with the document location information of the natural language document; or storing the concept information in the document location information and the document. 18. The database according to claim 16, wherein the concept information is registered in the database in an inverted index format based on a set of corresponding attribute information indicating which attribute is described.
An information processing method in the information search system according to item 1. 19. The concept extraction step includes: a morpheme analysis step of extracting a morpheme from the natural language document; a verb extraction step of extracting only a verb component in the extracted morpheme; and a semantic on which the verb depends. A case element specifying step of specifying a case element corresponding to the verb with reference to a verb case frame dictionary describing the information of the verb and a semantic dictionary for performing a surface definition of the semantic, the case element and the concept And generating concept information from case elements based on the information corresponding to the verb, and extracting the concept information by utilizing information of semantics on which the verb depends. , 17 or 18, the information processing method in the information search system. 20. The concept extraction step includes: a morpheme analysis step of extracting a morpheme from the natural language document; a verb extraction step of extracting only a verb component in the extracted morpheme; and a surface case on which the verb depends. A case element specifying step of specifying a case element corresponding to the verb with reference to a verb case frame dictionary describing the information of the case element; and a case element for creating concept information based on correspondence information between the case element and the concept. 19. A concept creating step of extracting concept information by utilizing information of a surface case element on which a verb depends.
An information processing method in the information search system according to item 1. 21. The concept extracting step includes: a morphological analysis step of extracting a morpheme from the natural language document; a verb extracting step of extracting only a verb component in the extracted morpheme; and a semantic on which the verb depends. A verb case frame dictionary describing the information of the surface case information and a semantic dictionary for performing the surface definition of the semantic, and a case element specifying step of specifying a case element corresponding to the verb, A concept element from the case element for creating concept information based on the correspondence information between the case element and the concept, and the concept information is utilized by utilizing the information of the semantic on which the verb depends and the information of the surface case. 18. The method according to claim 16, wherein extraction is performed.
Or an information processing method in the information search system according to 18. 22. The concept extracting step includes: a morphological analysis step of extracting a morpheme from the natural language document; a condition of a semantic element of each extracted morpheme (hereinafter, referred to as a target element); Conditions of co-occurring elements, conditions of the element of interest and ancillary elements of the co-occurring elements, conditions of appearance positions of the co-occurring elements, and information of corresponding concept columns (hereinafter referred to as corresponding columns) A corresponding column determining step of calculating a degree of correspondence to each corresponding column with reference to information extraction knowledge describing the corresponding column and extracting a morpheme into a corresponding column having the highest degree of correspondence. Item 19. An information processing method in the information search system according to Item 16, 17, or 18. 23. The concept extracting step, in which the natural language document is regarded as a hierarchical context area, and a start point condition, an end point condition, and a lower-order context area candidate are described for each of the time domain areas. Referring to the context area extraction knowledge, a context hierarchy analysis step of analyzing the hierarchy of the document structure of the natural language document to create context hierarchy information, and describing information of semantic elements and corresponding concept columns A context information extracting step of extracting context information from a sentence in the natural language document with reference to the context information extraction knowledge obtained, and storing context information in the context hierarchical structure information; Utilizing the context information stored in the lower context region from the lower context region of the hierarchical structure information or in the lower context region from the upper context region, 2. A concept description area determining type concept information extracting step of extracting the concept information while determining an area in which the concept information is described.
An information processing method in the information search system according to 6, 17, or 18. 24. The concept extracting step, in which the natural language document is regarded as a hierarchical context area, and a start point condition, an end point condition, and a lower-order context area candidate for each of the pulse region are described. A context hierarchy analysis step of analyzing the hierarchy of the document structure of the natural language document to create context hierarchy information with reference to the context region extraction knowledge; and a morphological analysis step of extracting a morpheme from the natural language document And the condition of the semantic of each extracted morpheme (hereinafter referred to as the element of interest), the condition of the co-occurrence element co-occurring with the element of interest, and the condition of the adjunct element of the element of interest and the co-occurrence element And the information extraction knowledge that describes the condition of the appearance position of the co-occurrence element and the information of the column of the corresponding concept (hereinafter referred to as the corresponding column). A corresponding column determining step of calculating the degree of correspondence to the corresponding column and extracting a morpheme to the corresponding column having the highest degree of correspondence, extracting context information from a sentence in the natural language document, and A context information extracting step of storing context information in information; and storing the context hierarchical structure information and a lower context area of the context hierarchical structure information in a higher context area or a higher context area in a lower context area. A concept description area determining type concept information extracting step of extracting the concept information while determining the area in which the concept information is described by utilizing the context information.
An information processing method in the information search system according to 6, 17, or 18. 25. The information extraction knowledge describes a condition of a semantic element of the element of interest, a condition of the co-occurring element, a condition of the attached element, and a condition of an appearance position of the co-occurring element in a pattern description such as a regular expression. 25. The information processing method according to claim 22, wherein the information is described as knowledge. 26. The inter-knowledge correlation rule, wherein the concept extraction step activates knowledge that is likely to co-occur with the knowledge and suppresses knowledge that is unlikely to co-occur when the knowledge of the information extraction knowledge matches. 3. The method according to claim 2, wherein
An information processing method in the information search system according to 2, 24 or 25. 27. The correspondence column determination step, wherein the calculation of the correspondence degree is processed in parallel for each information extraction knowledge, or the determination of the correspondence column is processed in parallel for each morpheme. , 24, 25, or 26, the information processing method in the information search system. 28. The concept description area determining type concept information extracting step includes, when extracting concept information while determining a concept description area, extracting concept key knowledge describing essential attributes of a concept characterizing a concept to be extracted. Extracting necessary attributes, checking that a plurality of concept units are not included, and checking that new information is included to check whether the concept information is necessary and sufficient. Item 30. An information processing method in the information search system according to Item 23, 24, 25, 26, or 27. 29. The concept description area determining type concept information extracting step includes, when extracting concept information while determining a concept description area, a concept key knowledge describing essential attributes of a concept characterizing a concept to be extracted. 25. The method according to claim 23, wherein the duplication of the extracted concept information is eliminated by using the extracted concept information.
25. An information processing method in the information search system according to 25, 26, 27 or 28. 30. The method of claim 16, 17, 18, 19, 2
0, 21, 22, 23, 24, 25, 26, 27, 28
Or a computer-readable recording medium stored as a program for causing a computer to execute the information processing method in the information search system according to 29.