JPH1021249A - Method for generating key word extraction rule - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the key word extraction rule generating method for extracting a character string (key word) representing a logical structure which is used to convert an unstructured document into a structured document. SOLUTION: A logical structure extraction part 102 generates element information 103 representing the adjacency relation between logical structure elements containing character strings from given logical structure definitions 101, an output format information extraction part 105 generates output format information 106 as information regarding a layout and a character string at the time of output of the logical structure elements from given output format definitions 104, and a display part 108 actuates a verification part 111 and an input part 109 to determine a key word corresponding element as a character string corresponding element to be extracted as a key word and generate format conditions for extracting the key word, thereby generating an extraction rule 113. The verification part 111 verifies whether or not the key word corresponding element is set sufficiently at a certain point of time and the complementary information input part 109 generates format conditions under which a user extracts the key word from the information 106.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a document (hereinafter referred to as an "unstructured document") that does not include information that explicitly indicates the structure of a document and that is input by means such as a character recognition device or a word processor. The present invention relates to a method for generating a keyword extraction rule used when converting a document into a structured document including information that explicitly expresses the structure of the document.

[0002]

2. Description of the Related Art One of the formats of a structured document is a method of embedding information explicitly representing a logical structure in a text.
In general, a structured document created by a user (hereinafter referred to as a “document instance”) is composed of a part that specifies a file that describes a logical structure definition that defines the logical structure of the document, and a content text part that indicates the contents of the document. There are many. The logical structure definition defines the logical structure of the document and marks (hereinafter, referred to as “tags”) that represent the components. Also,
In the content text portion, a tag defined in the logical structure definition is inserted so that a character string serving as the content of the logical structure corresponding to the tag is uniquely determined, and the logical structure of the document is explicitly expressed.

When outputting a document instance structured in this way, an output format definition that defines in what format each component of the logical structure (hereinafter referred to as an “element”) is output is defined. The image to be output is generated by referring to the described file. According to this method, since the document instance and the output format definition are independent,
Document instances can be exchanged regardless of individual devices or systems used for output.

[0004] Further, the contents of a character string in such a structured document are explicitly expressed by inserting a tag such as "<author name>" or "<title>" that corresponds to the element on a one-to-one basis. Therefore, by combining with a tool such as a full-text search system corresponding to a structured document, a set of document instances can be used as it is as a database. Structured documents and their use are described in detail in the document "Recommendations of SGML" (edited by Makoto Yoshioka, Ohmsha).

[0005] Due to these advantages, as a document management format in a document processing system that stores and uses a large amount of documents,
The adoption of structured document format is progressing. At the same time, a method of converting an unstructured document such as an existing paper document or a word processor input document into a structured document is being studied.

Conventional techniques relating to conversion of an unstructured document into a structured document include Japanese Patent Application Laid-Open No. 62-24970 and
"Conversion method of document image to ODA logical structured document (Transactions of IEICE, D-II, Vol. J76-DI
I, No. 11, pp. 2274-2284) ". This is from Chapter 1 in unstructured documents.
By extracting a character string (hereinafter, referred to as a “keyword”) representing a logical structure such as “.” Or “1.1” and recognizing the logical structure of the document using the extracted keyword as a clue, the structured document is To generate.

However, in the prior art, there is no viewpoint for supporting the creation of a rule for extracting a keyword, and no means has been disclosed yet. Therefore, it is necessary to manually determine all elements to be keywords and to set layout and character string conditions necessary for keyword extraction.

[0008]

Problems with the conventional method are as follows: (1) There is no means for supporting the determination of an element to be extracted as a keyword (hereinafter, referred to as a "keyword corresponding element"). Not all elements having a character string as a keyword are extracted as keywords. In particular, elements that have no characteristic in the layout or character string are not extracted as keywords, and character strings sandwiched between keywords,
That is, it is treated as a non-keyword.

When deciding which element is to be a keyword-corresponding element, a constraint is imposed that "non-keywords must not be adjacent in a document instance". This is because the non-keyword is a “character string sandwiched between keywords”, and therefore the non-keyword must be adjacent to the keyword. However, in the conventional method, there is no means for automatically testing whether or not the set of elements set as the keyword corresponding elements satisfies the constraint condition. Therefore, if the set of keyword-corresponding elements does not satisfy this constraint, inconvenience occurs when creating a rule for recognizing the logical structure or recognizing the logical structure. As a result, it is necessary to set the keyword-corresponding element again. Come. Then, it is necessary to repeat this cycle until an appropriate set of keyword corresponding elements is set.

(2) There is no method for supporting the setting of layout and character string conditions necessary for keyword extraction. Therefore, it is necessary to manually extract the information required for keyword extraction from the target unstructured document itself or from a set of rules that define the description format of the unstructured document. It costs.

An object of the present invention is to solve the above problems and to provide a method for generating a keyword extraction rule for extracting a keyword from an unstructured document when generating a structured document from the unstructured document. It is in.

[0012]

According to the present invention, there is provided a rule for extracting a characteristic character string, that is, a keyword, representing a component of a logical structure of a document from an unstructured document. This is a keyword extraction rule generation method for generating a keyword extraction rule used when generating a structured document from an unstructured document, and extracts logical structure information from a logical structure definition given to a target document and generates a character. A character string corresponding element information generating step of generating column corresponding element information; an output format information generating step of extracting output format information from an output format definition given to the target document to generate output format information; A keyword extraction rule generation step of generating a keyword extraction rule based on the extracted character string corresponding element information and output format information That.

Further, the character string corresponding element information generating step generates a character string corresponding element and a character string corresponding element that can follow the character string corresponding element as a pair as character string corresponding element information, and generates the output format information. The generation step is to extract information on the layout and character strings when outputting the components of the logical structure of the document as output format information.

Further, the keyword extraction rule generation step displays the output format information for each item necessary for keyword extraction to a user, and according to the user's input,
The output format information is modified so as to conform to the output format on the unstructured document, and the missing information is supplemented by the output format information.

Further, in the keyword extraction rule generating step, when the user determines which of the components of the logical structure is to be extracted as a keyword from the output format information, the extraction should be performed based on the character string corresponding element information. The components of the logical structure are indicated and displayed to assist the user's decision.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described with reference to the drawings. In the present embodiment, as an example of a structured document format, SGML (Stand-rd Genera) is used.
It employs a size-dependent Markup Language (DLD) format, and the logical structure definition is DTD (Do), which is the SGML document type definition set for the target document.
document type definition). The processing contents and description rules of SGML and DTD
ISO887 which is the standard agreement of SO (International Organization for Standardization)
9 and details are described in the document "SGML
Introduction (Martin Bryan, ASCII Publishing).

The present embodiment relates to the "keyword extraction rule generation method" shown in FIG. 1. First, the flow of the structured document generation method will be described, and the position of the present invention will be described while showing specific examples. .

FIG. 2 is a block diagram showing the flow of the structured document generation method. This flow itself is described in Japanese Patent Application No. 7-223.
017, Japanese Patent Application Laid-Open No. 62-24970, or "Conversion method of document image to ODA logical structured document (Transactions of IEICE Transactions, D-II, Vol. J76-DII, No.
11, pp. 2274-2284)).

Reference numeral 201 denotes a document which does not include information explicitly indicating the structure of a document, that is, an unstructured document, which is input by means such as a character recognition device or a word processor. FIG. 3 shows an example of an unstructured document. This is the result of performing character recognition on paper documents using laws and regulations as an example. Although there is no explicit notation indicating the logical structure, each component of the document is laid out using spaces, etc., so that it is easy to read. Have been.

In order to utilize such a text format electronic document in a document processing system, a logical structure definition (FIG. 2)
207) is set. FIG. 4 shows an example of a logical structure definition (DTD) corresponding to the unstructured document in FIG. The first reference numeral 401 indicates that this logical structure definition is called “regulation”. Reference numerals 402 to 415 denote element definitions, in which a model name is described after "! ELEMENT", followed by "(" and ")", which is a group of elements constituting the element. Is described. For example, the model group is (promulgation date, regulation number, promulgation text?), (#PCDATA), and the like. The title is an element having “#PCDATA” as an element of the model group, and the ordinance is “title”. , “Promulgation”, and “main rules” are the elements of the model group. The model group is a set including one or more elements or a content token representing data such as “#PCDATA” as an element, and the model group itself may be nested as an element.

Numeral 402 indicates that the element "ordinance" is constituted by a sequence of elements such as "title", "promulgation", and "main rule". Reference numeral 409 indicates that the element “article” is composed of an array of elements such as “heading?”, “Article number”, “article rule”, and “number *”. An element with an asterisk ("*") means that the element can appear zero or more times, and an element with a question ("?") Indicates whether the element exists or not. Also means good. For example, 409 is
The "heading" does not need to be present, and "number" can appear zero or more times. 403, 40 in which the contents of the model group are (#PCDATA)
Elements 5 to 407 and the like mean that elements such as “title”, “promulgation date”, “regulation number”, and “promulgation sentence” respectively hold character strings representing the contents of the model group.

FIG. 5 shows a tree-like representation of the logical structure from 401 to 416. In FIG. 5, 40 in FIG.
Each element defined in 2 to 415 is expressed as a node, and indicates that the element corresponding to the upper node is configured from the element corresponding to the lower node. A node having “#PCDATA” at a lower level means that the corresponding element holds a character string as a model group.

FIG. 6 shows a structured document obtained by structuring the contents of the unstructured document of FIG. 3 according to the logical structure definition of FIG.
The content of the element defined in FIG. 4 (for example, “promulgation” of 404) represents a symbol (here, <promulgation> of 601) and an end indicating the start of the element in the structured document of FIG. It is represented by a symbol and a character string sandwiched between symbols (here, 602 </ promulgation>). The purpose of the structured document generation process shown in FIG. 2 is to generate such a structured document from an unstructured document as shown in FIG.

The structured document generating process shown in FIG. 2 is roughly divided into two. One is a keyword extraction process of 202, and the other is a logical structure recognition process of 204. The keyword extraction process 202 includes a keyword extraction rule 203
With reference to “Article 1” and “2.
This is a process of extracting a keyword that is a characteristic character string representing a logical structure, such as “1.1”. The logical structure recognition processing 204 refers to the logical structure recognition rule 205 and performs logical structure recognition using the keywords extracted in the keyword extraction processing 202 as clues, and converts the tree-like logical structure shown in FIG. 5 into a document. By assigning, FIG.
This is a process for generating a structured document 206 as described above.

Hereinafter, the keyword extracting process 202 will be described in detail. FIG. 7 shows an example of the keyword extraction rule 203. The keyword extraction rule is a set of rules that combine an element name to be extracted as a keyword and a format condition that is a condition related to a layout and a character string for extracting the keyword. FIG. 8 illustrates the description elements of the format condition in FIG. In FIG. 7, the first item of each line is a keyword name, and the second and subsequent items are format conditions. 701 in FIG. 7 indicates that the format condition of the keyword “title” is “centered line, where a character“ 『” exists first, followed by a character string of an arbitrary length, and finally a character string “rule” Or the string "rule"
Ends the line. ". Regarding 702, the format condition of the keyword “promulgation date” is “character string“ Taisho ”or character string“ Showa ”with an arbitrary number of spaces from the beginning of the line. , Integer, "month", integer, "day", and the line ends ".

In the keyword extraction process 202 shown in FIG. 2, it is determined whether or not a character string conforming to the format condition of the keyword extraction rule exists in the unstructured document. Extract. FIG. 9 shows an example of extracting the unstructured document keywords in FIG. In the logical structure recognition processing 204 of FIG. 2, a structured document is generated by performing logical structure recognition using the extracted keywords as clues.
223017, JP-A-62-24970, or "Method of converting document image to ODA logical structured document (Transactions of IEICE Transactions, D-II, Vol. J76-DII,
No. 11, pp. 2274-2284) ".

The keyword extraction rule generation method described in detail in the present embodiment supports the creation of the keyword extraction rule 203 shown in FIG. Conventionally, all of the keyword extraction rules have been manually generated, but the present system uses the given logical structure definition and output format definition to support the creation of the keyword extraction rules.

FIG. 1 is a block diagram showing a configuration of a keyword extraction rule generation method. First, a processing outline of the present system will be described with reference to FIG. Reference numeral 101 denotes a logical structure definition set for the target document, in which elements appearing in the structured document and relationships between the elements are defined. The logical structure information extraction unit 102 refers to the logical structure definition 101, and is an element that is a logical structure element directly corresponding to a character string (hereinafter, referred to as a “character string corresponding element”).
And character string corresponding element information 103 describing the adjacent information between them. The above-mentioned character string corresponding element is shown in FIG.
Is a node element having “#PCDATA” as a lower order in the title, title, promulgation date, regulation number,.
This corresponds to the issue number and issue rules, and in the case of FIG.
3, 405, 406, 407, 410, 411, 41
The elements 2, 414 and 415 correspond to this. 10
Reference numeral 4 denotes an output format definition set for the target document, in which format each element is output. The output format information extraction unit 105 refers to the output format definition 104 and extracts items necessary for creating a keyword extraction rule as much as possible from information on layout and output character strings at the time of output of each element.
Hereinafter, this item itself is referred to as “requirement item”, and information extracted for each item is referred to as “requirement item content”. The output format information 106 describes the contents of requirement items for each character string corresponding element.

In the keyword extraction rule creating section 107,
The requirement item contents in the output format information 106 relating to each character string corresponding element are presented to the user through the input / output device 112. Then, the information input by the user is received, the requirement item contents are corrected, and the keyword extraction rule 113 is generated based on the corrected requirement item contents.

The processing in the keyword extraction rule creation unit 107 will be described more specifically. Keyword information display section 1
At 08, the name of the character string corresponding element described in the character string corresponding element information 103 is displayed to the user. When a certain character string corresponding element is set as a keyword corresponding element and a format condition is given, the format condition is displayed together with the name of the character string corresponding element. The setting of the format condition for each character string corresponding element is performed in the complementary information input unit 109. The supplementary information input unit 109 refers to the output format information 106 and displays the requirement item contents for the character string corresponding element selected by the user. The user corrects the displayed requirement item contents if they differ from the layout and character strings on the unstructured document. In addition, contents are given to requirement items that could not be extracted by the output format information extraction unit 105. In this way, the requirement item contents are edited so that all the requirement item contents conform to the layout and character strings on the unstructured document. 1
Reference numeral 10 denotes a character string condition input unit that assists the user in editing a character string condition that is one of the requirement item contents. When all the requirement items have been edited, the supplementary information input unit 109 generates a format condition to be used for keyword extraction from the requirement item contents. Then, the process returns to the keyword display unit with the format condition as a return value. The keyword information display unit 108 sets the character string corresponding element for which the format condition was generated in the complementary information input unit 109 as a keyword corresponding element, and displays the format condition along with the element name.

The keyword-corresponding element is determined by the above procedure. It is determined whether or not a set of keyword-corresponding elements determined at a certain point in time satisfies the constraint that “non-keywords must not be adjacent”. The test is performed by the adjacency test unit 111. The element adjacency test unit 111
A character string corresponding element other than the keyword corresponding element (hereinafter referred to as a “non-keyword corresponding element”) with reference to the adjacent information between the character string corresponding elements described in the character string corresponding element information 103.
Test whether are adjacent. If the non-keyword corresponding elements can be adjacent to each other, a format condition is generated for one of them and set as a keyword corresponding element. Conversely, if there is no possibility that non-keyword corresponding elements are adjacent to each other, it means that sufficient keyword corresponding elements have been set at that time. At this time, a set of combinations of the name of each keyword
The keyword extraction rule 113 is used. The above is the processing outline of the keyword extraction rule generation method. Hereinafter, each process in FIG. 1 will be described in detail.

The logical structure information extraction unit 102 refers to the logical structure definition 101 as shown in a specific example in FIGS. 4 and 5, and determines the character string corresponding element and the possibility of adjacency between the character string corresponding elements. Information is extracted, and character string corresponding element information 10 is extracted.
Output as 3. The character string-corresponding element is an element having (#PCDATA) meaning a character string as an element of the model group in the logical structure definition. FIG.
FIG. 10 shows character string corresponding elements in the logical structure definition of FIG. In the example of FIG. 10, the elements "title", "promulgation date", "regulation number", "promulgation text", "headline", "article number", "article rule", "number", and "number rule" are extracted as character string corresponding elements. Is done.

The logical structure information extraction unit 102 examines the possibility of adjacent character string corresponding elements. Specifically, the following two processes are performed. 1. For each element, a set of character string corresponding elements that can appear at the beginning and end of the element is obtained. For example, in the structured document of FIG. 6, a character string corresponding element “promulgation date” appears at the beginning of the element “promulgation”, and a character string corresponding element “promulgation sentence” appears at the end of the element “promulgation”. ing. In this processing, elements that can appear at the beginning and end of such an element are derived from the logical structure definition as shown in FIG. 2. Find combinations of adjacent elements in the model group of the logical structure definition. For each combination, a string-corresponding element that may appear at the end of the preceding element,
The string-corresponding element that can appear first at the back of the element will have the possibility of being adjacent.

In the present embodiment, as a preparation for facilitating these two processes, the logical structure definition shown in FIG. 4 is converted into an expression using the BNF (Buckus Naur Form) notation. BNF is a set of rules called “generation rules”. Each production rule is "A: BC"
Thus, it consists of a left side and a right side separated by a colon ':', which means that the elements on the left side are realized by the arrangement of the elements described on the right side. "A: BC"
Means that the element A is composed of a sequence of elements “BC”. Also,
The symbol '|' is a symbol indicating parallelism, for example, "A:
The generation rule “B | C” means that the element A is composed of the element B or the element C. The details of BNF are described in the document "How to use yacc and lex" (Takashi Saito, HBJ Publishing Bureau) and the like.

FIG. 11 shows the logical structure definition (D
FIG. 12 shows a conversion rule when expressing (TD) using BNF notation, and FIG. 12 shows an example of a logical structure definition expressed using BNF notation. For example, the definition of 404 in FIG. 4 is converted into the generation rules shown in 1203 and 1204 in FIG. Here, “promulgation sentence” in FIG. 4 is replaced by “opt0” 1203 in FIG. 12 by the conversion rule 1101 in FIG. And the definition of “opt0” is 1
204. Hereinafter, in the present embodiment,
The right side in each generation rule of the logical structure definition expressed by the BNF notation will be referred to as the “content model” of the element on the left side.

A procedure for obtaining a set of character string corresponding elements that can appear at the beginning and end of each element from the logical structure definition expressed by the BNF notation will be described.
FIG. 13 shows the algorithm of this procedure. In FIG. 13, the procedure starting with A is a procedure that takes an element as an input argument and returns a set of character string corresponding elements that can appear at the beginning of the element as a return value, and includes a recursive call. Here, the variables mg and elem used in this procedure are local variables newly generated each time the procedure proceeds to A. Also, First [xx] is
This is a global variable representing a set of character string corresponding elements that can appear at the beginning of element XX.

To obtain a set of character string corresponding elements that can appear at the beginning of a certain element, the procedure A is executed with the element as an argument (nt in FIG. 13). In procedure A, First [nt] representing a set of character string corresponding elements that can appear at the beginning of nt is set to an empty set (1301). Also, in the content model of nt, the first element string among the element strings separated by the parallel symbol '|' is substituted for the variable mg (1302). If there is no parallel symbol, the whole content model is set to mg.
Then, the first element of mg is substituted for the variable elem (1303). Next, at 1304, ele
It is checked whether or not m is a character string corresponding element. If elem is a character string corresponding element, elem is added to First [nt] (1305), and the flow advances to 1309. Conversely e
If lem is not a character string corresponding element, First
If [elem] is set (1306)
The content of t [elem] is added to First [nt] (1
308), and proceed to 1309. In 1306, F
If rst [elem] is not set, e
recursively executes procedure A with lem as an argument (1
307). Then, the return value, that is, First [e
lem] is added to First [nt] (130
8) Go to 1309. In 1309, it is checked whether or not mg is the last element string delimited by the parallel symbol in the content model of nt. If mg is not the last element sequence, the next element sequence is substituted for the variable mg (1310), and the process returns to 1303. Conversely, if mg is the last element sequence, the process returns to the procedure that called this procedure, using First [nt] as the return value (1311).

By executing the procedure shown in FIG. 13 until First [] is set for all elements, a set of character string corresponding elements that can appear at the beginning of each element can be obtained. Also, a set Las of character string corresponding elements that can appear at the end
In order to obtain t [], the following two substitutions are performed on FIG. 13 to obtain t [] in the same procedure as in FIG. a. First [XXX] in FIG. 13 is changed to Last [XX].
X]. b. The “first element” of 1303 is replaced with the “last element”.

FIG. 14 shows a set of elements corresponding to character strings that can appear at the beginning and end of the elements in the logical structure definition shown in FIG. 4, that is, First [] and Las.
The result of obtaining t [] is shown. With the above procedure, a set First [] of character string corresponding elements that can appear at the beginning and a set Last [] of character string corresponding elements that can appear at the end can be obtained for each element.

Next, a combination of adjacent elements in the content model of the logical structure definition is obtained. For each combination, there is a possibility that the Last [] element of the front element and the First [] element of the rear element are adjacent to each other. FIG. 15 shows an example of this processing. This drawing is an example of processing regarding a generation rule 1201 in FIG. 12, “Ordinance: title promulgation main rule”. In this generation rule, in the content model of the element “regulations”, the title and the promulgation are adjacent, and the promulgation and the main rule are adjacent (1501). Therefore, the element of Last [title] can be followed by the element of First [promulgation] (1502).
That is, the character string corresponding element “Title” can be followed by the character string corresponding element “Promulgation Date” (1504). Also, L
An element of the first [promulgation] can be followed by an element of the first [main rule] (1503). That is, the character string corresponding elements "heading" and "article number" can both follow the character string corresponding elements "promulgation sentence" and "regulation number" (1505).
By applying this procedure to all the production rules in the logical structure definition expressed in the BNF notation, a set of character string corresponding elements that can be followed by all the character string corresponding elements can be obtained. It becomes character string corresponding element information (103 in FIG. 1). Character string corresponding element information 103
FIG. 16 shows an example. As described above, the character string corresponding element information 103 is generated in the logical structure information extraction unit 102 in FIG. 1 by the procedures shown in FIGS.

Next, the process of extracting the output format information 106 from the output format definition 104 in the output format information extraction unit 105 of FIG. 1 will be described. Reference numeral 104 denotes an output format definition set for the target document, in which format each element is output. FIG.
FIG. 7 shows a part of an example of an output format definition prepared for a structured document according to the logical structure definition of FIG. 1701
Indicates that 1701 to 1711 are definitions relating to the output format of the element “title”. [Font type] 1702 indicates that the font type when outputting “title” is Gothic, and [Font size] 1703 indicates that the font size is 12
pt. pt (point) is a unit of length, and 1 pt = 1/72 inch. [Character pitch] 1704 indicates that the character pitch of the “title” is 1
4 pt. [Offset 1] of 1705
And [Offset 2] of 1706 indicate the minimum amount of space from the left end and right end of the output area of this document to output the content of “title”. [Top displacement] 1707 represents a difference from [Offset 1] of the first row, which often takes a special offset compared to other rows. 1708 [Connection with previous element] indicates what kind of character string is output between the element and the element that appears immediately before. The example of 1708 indicates that after outputting the element appearing immediately before, a line feed is performed and “title” is output. [Character string information] 1709 describes what kind of character string is output. In the example of 1709, the character string (CONTENT) corresponding to the title, that is, the tag <title> and This means that a character string sandwiched between tags </ title> is output as it is. [Arrangement] 1710 shows how the content character string is arranged in the section specified by [Offset 1] and [Offset 2]. Start, end, and center according to four types of allocation methods: left alignment, right alignment, centering, and even allocation
It takes four values, r and justify. The example of 1710 indicates that the content character string of “title” is centered and output.

Such an output format definition is originally for outputting a structured document, and is not for expressing the format of an unstructured document. However, for a document such as a legal document having a regular description format, the output format definition is often defined in accordance with the rules. For such documents, much of the information on the layout and character strings in the output format definition can be used as information for extracting keywords from unstructured documents.

The output format information extraction unit 105 refers to the output format definition 104 and selects from the information on the layout at the time of output of each element and the information on the output character string.
Extract items necessary for keyword extraction as much as possible.
As mentioned earlier, this item is itself called a "requirement item"
Information extracted for each item is referred to as “requirement item contents”.

FIG. 18 shows an example of requirement items required for each keyword when creating the keyword extraction rule shown in FIG. [Logical structure element name] 1801 is the name of the target character string corresponding element, and the character string is a value. A [left space] 1802 and a [right space] 1803 are conditions that indicate a space of at least the number of characters from the left end and the right end of the area where the element is output, and describe the content character string. It is. A [first line space] 1804 indicates how many characters of the left side of the first line, which often takes a special offset compared to other lines, start. [Character string condition] 1805 indicates what character string is used to describe this keyword. [Assign] of 1806 is the left space 18
02 is an item indicating how keywords are assigned in an area defined by the right space 1803 and right alignment, left alignment, centering, uniform,
Takes four values. 1807 [prefixed character string] and 1808 [postfixed character string] are character strings representing what character strings are sandwiched between character string corresponding elements appearing before and after the target keyword, respectively. It is.

The output format information extraction unit 105 refers to the output format definition 104 and extracts information on requirement items as shown in FIG. 18, that is, the content of requirement items as much as possible. FIG. 19 shows an example in which the requirement item contents are extracted from the output format definition shown in FIG. In order to extract the requirement item contents of a certain character string corresponding element, the definition of the character string corresponding element in the output format definition is used.
For example, the requirement item related to the article number is 1712 in FIG.
Extracted from the definition related to the article number of ~ 1722. The requirement items [Left space] and [Right space] are [Offset 1] and [Offset 2] in the output format definition, respectively.
Since the item has the same content as the above, it is only necessary to convert the unit of length from pt to the number of characters. Specifically, the values of [Offset 1] and [Offset 2] may be divided by the value of [Character pitch] (1901 and 1902). The requirement item [first line space] is equivalent to [offset 1] in the output format definition plus [head displacement]. Therefore, a value obtained by dividing the sum of the two by [character pitch] is used as the content (1903). The requirement item [character string condition] is created with reference to [character string information] in the output format definition (19).
However, in the example of FIG. 17, the [character string information] is "CONTENT" for all elements, that is, the content character string in the document instance is output as it is. Information cannot be obtained. Since the requirement item [assignment] is an item representing the same concept as [arrangement] in the output format definition, the value is converted according to the rule of 1905. Requirement item [prefix string]
Substitutes the content of [Connection with previous element] in the output format definition as it is (1906). Requirement item: [Postscript string] is a string corresponding element information,
It is obtained by using [connection with previous element] of another element in the output format definition (1907). Specifically, first, using the character string corresponding element information, a character string corresponding element adjacent to the target character string corresponding element (hereinafter, referred to as a “following element”) is obtained. Next, for all the succeeding elements, the [connection to the preceding element] of the element is checked, and if the content is the same for all the succeeding elements, the content of the character string corresponding element of interest is checked. Character string]. If the content of [Connection with previous element] differs depending on the succeeding element, [Back character string]
Is not set. For example, as for the article number, it can be seen from the character string corresponding element information 1606 of FIG. Therefore, “” ”, which is the [connection with the preceding element] in the article rule, is the [suffix character string] of the article number.
It becomes the contents of. By applying the above procedure to all the character string corresponding elements, the output format information 106 shown in FIG.
Is generated.

The keyword extraction rule creating section 107 shown in FIG.
Now, character string corresponding element information 103 and output format information 106
Is presented to the user through the input / output device 112. Then, the supplementary information is input by the user, and the requirement item information is added and corrected, thereby generating the keyword extraction rule 113. Hereinafter, specific processing in the keyword extraction rule creation unit 107 will be described. The keyword information display unit 108 presents the user with a character string corresponding element name and information indicating which character string corresponding element is set as a keyword corresponding element at a certain point in time. Then, when the user instructs to set a certain character string corresponding element as a keyword corresponding element, the supplementary information input unit 109 is activated, and the format condition is generated by complementing the requirement item contents of the character string corresponding element. I do. At that time, if the user instructs whether or not a keyword-corresponding element sufficient to satisfy the constraint that “non-keywords must not be adjacent” is set, the element adjacency test unit Activate 111 and perform verification.

FIG. 20 shows an example of an interface displayed by the keyword information display unit 108 to the user through the input / output device 112, and FIG. 21 shows a processing flow. The operation of the keyword information display unit 108 will be described with reference to these two figures. The keyword information display unit 108 reads the character string corresponding element information 103 at the time of startup, and obtains the name of each character string corresponding element (2101). Reference numeral 2001 denotes a keyword information display window, which is an element name display area 2002 for displaying all character string corresponding element names, and a format condition display area 2003 for displaying format conditions of character string corresponding elements set as keyword corresponding elements. Consists of Process 2102
In, the character string corresponding element name and the format condition of the element set as the keyword corresponding element at that time are displayed, but since no format condition is initially set for any element, the format condition display area 2003 displays Does not display anything. To assign a format condition to a certain character string corresponding element and set the element as a keyword corresponding element, the user can use a mouse, for example, to display the element name display area 20.
By double-clicking on the element name in 02, the complementary information input unit (109 in FIG. 1) is activated (2104).
Although the operation of the supplementary information input unit 109 will be described later, the character string corresponding element name is passed to the supplementary information input unit 109, and the format condition is received as a return value. Then, the character string corresponding element designated by the user is set as a keyword corresponding element (2105), and the format condition is set in the format condition display area 200.
3 is displayed (2102). The example of FIG. 20 shows a display example of the interface at a certain time. At this point, the formatting condition is given to the two character string corresponding elements of the title of 2006 and the item number of 2007, which means that these two character string corresponding elements are set as the keyword corresponding elements. .

Reference numeral 2004 denotes an adjacency check button. When this button is clicked, the set of keyword-corresponding elements set at that time is sufficient to satisfy the constraint that “non-keywords must not be adjacent”. The element adjacency verification unit (111 in FIG. 1) for verifying whether or not is activated is started (2106). The operation of the element adjacency verification unit 111 will be described later. When the verification is performed and it is determined that a keyword-corresponding element sufficient to satisfy the constraint condition is set, the user clicks the end button and clicks the keyword information. It is instructed to end the processing of the display unit 108. The keyword information display unit 108 outputs the keyword corresponding element name and its format condition as a keyword extraction rule (113 in FIG. 1), and ends the processing (210).
7). The above is the processing contents of the keyword information display unit 108.

Next, FIG. 22 shows an interface of the supplementary information input unit 109 which is activated when the element name is double-clicked on the keyword information display unit 108, and FIG. 23 shows a processing flow thereof. The supplementary information input unit 109 reads the element name for which the format condition is to be set as the keyword corresponding element passed from the keyword information display unit 108 (2301), and outputs the requirement item contents corresponding to the element to the output format information (FIG. 1). From (106) of (230)
2). Then, the content of the requirement item is displayed in the requirement item editing window 2201.
(2303). The requirement item edit window 2201 is
This window allows the user to edit the displayed content. If the displayed content is different from the description format on the unstructured document, the user changes the content. Also, the contents of the requirement items that could not be extracted by the output format information extraction unit 105 (for example, FIG. 18 and FIG. 1)
For the “character string condition” in the extraction example of No. 9), the requirement item editing window is blank, so the user inputs the requirement item content in the editing window (2304 → 2303). The character string condition may also be edited on the requirement item edit window, but the character string condition input button 2202 is clicked to activate the character string condition input unit (110 in FIG. 1) (2305), which makes it easier. Can be entered. The processing of the character string condition input unit 110 will be described later. A display example after inputting the character string condition is shown in "After inputting character string condition" in FIG.

When the editing of the requirement item contents is completed and all the requirement item contents conform to the description format on the unstructured document, the user clicks the end button 2203 to terminate the processing of the supplementary information input unit 109. Instruct. The supplementary information input unit 109 generates a format condition from the content of the requirement item of the character string corresponding element in which the requirement item has been edited (2306), and uses the format condition as a return value to process the keyword information display unit 10.
8 (2307). FIG. 24 shows a processing flow for generating a format condition from the contents of a requirement item. An example of converting the contents of the requirement item of the article number shown in “After inputting the character string condition” in FIG. First, the contents of the requirement item [string condition] (for example, ""
"NUM1""") is substituted into the format condition. Then, it is checked whether or not the content of the requirement item [preceding character string] is a line feed (2401). If it is a line feed, go to 2403. If it is not a line feed, the format condition is sandwiched between '[' and ']', and '+' and the contents of [preceding character string] are added immediately before the format condition (2402). At this time, blanks are converted into SPCs {integer}. Next, in processing 2403, it is checked whether or not the content of the requirement item [subsequent character string] is a line feed.
If it is a line feed, add “@” to the end of the format condition (2
405), and proceed to processing 2406. If it is not a line feed, if there is no '[' and ']' in the format condition,
And ']', followed immediately by the contents of [subscript string] and '
+ '(2404, for example, "["
M1 "Section"] SPC1 + "). In process 2406,
Check whether the content of the requirement item [assignment] is centering. In the case of centering, 'C' is added to the beginning of the format condition (2407), and the generation of the format condition ends. Conversely, if it is not centering, process 24
In step 08, the processes of A and B are performed in accordance with the content of [assignment]. If the content of [Assign] is left-justified, A,
If it is right-justified, B is executed, and if it is equal, both A and B are executed, and the generation of the format condition is ended. In A, '@SPCx' is added to the beginning of the format condition (2409). Where x is the content of [start indent] (for example, "@SP
C0 ["No. 1 NUM1"] SPC1
+ "). In B, first, 'SPCy
＄ ′ is added (2410). Here, y is the content of [right space]. Next, if '＾' or '+' does not exist at the beginning of the format condition, '!''Is added (2411). The complementary information input unit 109 returns the process to the keyword information display unit 108 with the format condition obtained by the above procedure as a return value (2307 in FIG. 23).
The above is the processing content of the supplementary information input unit 109.

Next, FIG. 25 shows the interface of the character string condition input section 110 which is activated when the character string condition input button is clicked in the complementary information input section 109, and FIG. 26 shows the processing flow thereof. String condition input unit 110
The object of the present invention is to reduce the trouble of inputting by converting a character string input frequently used in a character string condition into a button. Reference numeral 2501 denotes a character string condition display window on which a user edits a character string condition. Reference numeral 2502 denotes a cursor in the character string condition display window, which indicates that a character to be inserted by the user is to be inserted at a position where the cursor is located. Reference numerals 2503 to 2508 denote editing buttons. When these buttons are clicked, the processing shown in the table of FIG. 26 is performed (2602). Characters that cannot be input with this button, for example, characters following NUM or SPC, are input by the user from the keyboard. Reference numeral 2509 denotes a clear button. When the user clicks this button, the contents in the character string condition display window are cleared (260).
3). Reference numeral 2510 denotes an end button. When the user clicks this button, the character string condition input unit 110 returns the processing to the complementary information input unit 109 with the contents in the character string condition display window 2501 as a return value (2604). The above is the processing content of the character string condition input unit 110.

Next, a keyword information display section (10 in FIG. 1)
In 8), the processing flow of the element adjacency test unit 111 activated when the adjacency check button is clicked is shown in FIG. 27, and FIG. Element adjacency test unit 11
In step 1, a keyword corresponding element name given from the keyword information display unit 108 is read (2701, for example, 2801). Next, character string corresponding element information (10 in FIG. 1)
3) is read (2702). Then, a non-keyword corresponding element group is obtained as a set obtained by subtracting the keyword corresponding elements from all the character string corresponding elements (2703, 2802, for example). In the process 2704, it is checked whether or not the non-keyword corresponding element exists in the succeeding element of the non-keyword corresponding element by referring to the character string corresponding element information (for example, 2
803). If there is, an adjacent non-keyword corresponding element is presented to the user (2705, 280 for example).
4) End the process. If no keyword exists, the user is informed that no non-keyword is adjacent (270)
6), end the process. The above is the processing content of the element adjacency verification unit 111.

As described above, the embodiment shown in this embodiment can support creation of a keyword extraction rule.

[0054]

As described above, according to the present invention, it is possible to support determination of a character string corresponding element to be extracted as a keyword by using adjacent information between character string corresponding elements extracted from a given logical structure definition. In addition, by extracting conditions relating to layout and character strings when extracting keywords from a given output format definition, it is possible to greatly reduce the labor required for creating keyword extraction rules.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an outline of a keyword extraction rule generation method according to an embodiment of the present invention.

FIG. 2 is a diagram showing an overall flow of structured document generation.

FIG. 3 is a diagram showing an example of an unstructured document.

FIG. 4 is an SGML set for the document shown in FIG.
FIG. 3 is a diagram showing a DTD which is a logical structure definition of a format.

FIG. 5 is a diagram expressing the DTD shown in FIG. 4 in a tree shape.

6 is an example in which the unstructured document shown in FIG. 2 is converted into a structured document conforming to the logical structure definition shown in FIG.

FIG. 7 is a diagram illustrating an example of a keyword extraction rule.

8 is a diagram showing description elements of format conditions in the keyword extraction rule shown in FIG.

FIG. 9 is a diagram showing an example of keyword extraction.

FIG. 10 is a diagram showing an example of extracting a character string corresponding element.

FIG. 11 is a diagram showing an example of a conversion rule when a DTD is described in BNF notation.

FIG. 12 is an example in which the DTD of FIG. 4 is described in BNF notation.

FIG. 13 is a diagram showing a procedure for obtaining a character string corresponding element that can appear at the beginning of an element.

14 is a diagram showing character string corresponding elements that can appear at the beginning and end of each element in the logical structure definition of FIG. 12;

FIG. 15 is a diagram illustrating an example of a process of obtaining an adjacent relationship between character string corresponding elements for the logical structure definition of FIG. 12;

FIG. 16 is a diagram illustrating an example of character string corresponding element information.

FIG. 17 is a diagram illustrating an example of an output format definition.

FIG. 18 is a diagram showing an example of requirement items necessary for extracting a keyword.

FIG. 19 is a diagram illustrating an example of a process of extracting the contents of a requirement item from an output format definition.

FIG. 20 is a diagram illustrating an example of an interface of a keyword information display unit.

FIG. 21 is a diagram showing a processing flow of a keyword information display unit.

FIG. 22 is a diagram illustrating an example of an interface of a supplementary information input unit.

FIG. 23 is a diagram showing a processing flow of a supplementary information input unit.

FIG. 24 is a diagram showing a processing flow for generating a format condition.

FIG. 25 is a diagram showing an interface of a character string condition input unit.

FIG. 26 is a diagram showing a processing flow of a character string condition input unit.

FIG. 27 is a diagram showing a processing flow of an element adjacency test unit.

FIG. 28 is a diagram illustrating a processing example of an element adjacency test unit.

[Explanation of symbols]

 101 Logical Structure Definition 102 Logical Structure Information Extraction Unit 103 Character String Corresponding Element Information 104 Output Format Definition 105 Output Format Information Extraction Unit 106 Output Format Information 107 Element Adjacency Verification Unit 108 Keyword Information Display Unit 109 Complementary Information Input Unit 110 Character String Condition Input Part 201 unstructured document 202 keyword extraction process 203 keyword extraction rule 204 logical structure recognition process 205 logical structure recognition rule 206 structured document 207 logical structure definition

Claims (4)

[Claims] 1. A rule for extracting a characteristic character string, that is, a keyword, representing a component of a logical structure of a document from an unstructured document, and is used when a structured document is generated from the unstructured document. A keyword extraction rule generation method for generating a keyword extraction rule, comprising extracting logical structure information from a logical structure definition given to a target document and generating character string corresponding element information. An output format information generating step of extracting output format information from an output format definition given to the target document to generate output format information; and extracting a keyword based on the generated character string corresponding element information and output format information A keyword extraction rule generation method, comprising a keyword extraction rule generation step of generating a rule. 2. The keyword extraction rule generation method according to claim 1, wherein the character string corresponding element information generating step includes, as character string corresponding element information, a character string corresponding element and a character string corresponding to the character string corresponding element. A keyword extraction rule generation method, wherein the output format information generating step extracts information on a layout and a character string when outputting a component of a logical structure of a document as output format information. . 3. The keyword extraction rule generation method according to claim 2, wherein the keyword extraction rule generation step displays the output format information to a user for each item necessary for keyword extraction, A method for generating a keyword extraction rule, wherein the output format information is corrected so as to conform to an output format on an unstructured document, and missing information is supplemented by the output format information. 4. The keyword extraction rule generation method according to claim 2, wherein in the keyword extraction rule generation step, a user determines which of the components of the logical structure is to be extracted as a keyword from the output format information. A keyword extraction rule generation method, which specifies and displays constituent elements of the logical structure to be extracted based on the character string corresponding element information to assist the user in determining the keyword.