JP5191204B2 - Associative search system - Google Patents

Description

  The present invention relates to an associative system, and more particularly, to an associative search technique for extracting a keyword from input document data and outputting a character string such as a company name closely related to the keyword.

A search system is used to extract necessary information from a vast amount of information. In the case of a general search system, there is a mechanism for extracting information that contains the same or similar concept as the input search term. ing. For example, if a search term “Fuji” is given to a database that stores information on a large number of companies, the search system can accurately output a list of companies that include the character string “Fuji” in the name. If you enter "environmental problem" at a search site on the Internet, a list of Web pages that contain the text "environmental problem" is displayed on the display.
As a result, the user can reach the target information, but the search results there are only in the expected range, and even if you look at the search result list, you can expect unexpected discoveries. There wasn't. Of course, new knowledge can be obtained in the process of examining details of individual data in the search result list, but information including other terms closely related to the search term cannot be extracted directly.

In this regard, in the case of the “associative search system” disclosed in Patent Document 1, the related term storage means that stores the related terms of each term and the co-occurrence with each term (appear in the same document) It has a co-occurrence company name storage means that stores company names (high probability). When a search term is entered, it extracts terms related to it and extracts company names with high co-occurrence for each term. It has a mechanism to do.
JP 2004-110386 A

  As a result, when users enter "environmental problems" as a search term, it becomes possible to directly list the names of companies that often appear in documents related to environmental problems. Recognize and connect with investment behavior.

  However, in this conventional associative search system, it is assumed that the user inputs a specific character string to the system as a search term, so the user is required to have some awareness of the problem at this point. It is done. In other words, it is an effective system for aggressive users who are looking to invest in a specific theme, but for users who vaguely want to invest if there are promising companies in the future, the utility value is I have to say it's a low system.

On the other hand, a large number of document data such as news articles, blog articles, papers, white papers, mail magazines, etc. are accumulated on the Internet website today, and users can freely browse these document data via a PC etc. Blessed with the opportunity to do.
These document data contain information on technologies and services that will lead the next generation. If you can list the names of companies that are closely related to these information, search for a vaguely investee as described above. However, there is no such mechanism so far.

  The above problems are not limited to searching for investee companies, but products such as “I want to buy if there is any good product (service)” or “I want to support if there is a wonderful person” This is also true if you are looking for a person vaguely.

  The present invention was devised to overcome such a current situation, and based on the description contents of document data accumulated in large quantities every day, a highly relevant specific company name, product name, person name, etc. It is an object of the present invention to realize a system capable of listing character strings belonging to a specific category.

In order to achieve the above object, the associative search system according to claim 1 is selected based on keyword storage means for storing a plurality of keywords in advance and the high degree of co-occurrence between the keywords. Associative word storage means for storing a plurality of associative words for each keyword, company name storage means for storing character strings related to a plurality of company names in advance, and means for dividing input document data into morpheme units Means for comparing each morpheme with the keyword and detecting a keyword included in the document data; means for referring to the associative word storage means; acquiring an associated word for each detected keyword; and the company name storage means reference and means for extracting one that matches the character string stored in the company name storage means in each associated words as related company name, associated companies list that lists the associated company name And a means for generating a search result display screen having a document display field for displaying the contents of the document data, and between each affiliated company name in the affiliated company list field and a corresponding keyword in the document display field. A link is set, and when a specific company name in the related company list column is selected, the corresponding keyword in the document display column is highlighted (for example, highlighted) .
Here, “high co-occurrence” means the height of the ratio (degree) of appearance in the same document.
The “morpheme” refers to the smallest language unit having a meaning. For example, when “my name is Suzuki” is broken down into morphemes, “I (pronoun)” “no (particle)” “name (general noun)” “ha (counselor)” “Suzuki (proprietary noun)” “ Is (auxiliary verb) ".

An associative search system according to claim 2 is the system according to claim 1 , further comprising: a document storage unit storing a plurality of document data; a plurality of keywords extracted from each of the document data; and the keyword Using the keyword extraction means stored in the storage means, the appearance frequency of each keyword in all the document data, the means for storing in the co-occurrence frequency storage means, and the appearance frequency data in each document data of each keyword, After calculating the degree of association based on the co-occurrence between keywords and sorting the degree of association between the degree of association calculation means stored in the keyword degree of association storage means and other keywords for each keyword, the higher degree of association An associative word extracting unit that extracts a predetermined number of keywords as an associative word of the keyword and stores it in the associative word storage unit is provided.

Associative retrieval system according to claim 3, The system of claim 2, further said keyword extracting means, respectively provided with a plurality of filters for extracting a keyword candidate based on the unique extraction criterion, each filter And the keyword candidates extracted by at least a plurality of filters are recognized as keywords.

The associative search system according to claim 4 is the system according to claim 3 , wherein one of the filters further extracts (1) a noun included in each document data as an attention word, and (2) each attention The frequency of occurrence of words in all document data is calculated, and (3) the range is expanded to the morpheme before and / or after each word of interest, (4) If the appearance frequency calculated by the processing in (3) above is a predetermined number or more, the range is further expanded to the previous or subsequent morpheme, and this expanded range is Repeat the process of calculating the frequency of occurrence of the included attention word in all document data until the appearance frequency is less than the predetermined number. (5) Predetermined among the attention words including the first attention word and the extended range. Select keywords with appearance frequency within the range as keyword candidates It is characterized in that.

The associative search system according to claim 5 is the system according to claims 2 to 4 , wherein the relevance calculation means further includes (1) a record of appearance in the document in units of document data, and the relevance calculation (2) processing for selecting keywords to be targeted, (2) processing for multiplying the appearance frequency between each selected keyword in document data units, and recording the product in a predetermined storage means, and (3) document data units Then, the process of squaring the appearance frequency of each selection keyword and recording the value in a predetermined storage means, (4) the process of totaling the product between the selection keywords over all document data, (5) The process of calculating the square value of the appearance frequency of each selected keyword over all document data, (6) the process of calculating the square root of the calculated value of (5) above, and (7) the above (6) of each keyword By adding the square roots of each other and dividing the sum of (4) above by the sum, It is characterized by executing a process of calculating the degree of relevance between the keyword.

According to the associative search system described in claim 1, it is possible to list specific company names related to the contents of the input document data. Moreover, instead of simply extracting the company name specified in the document data, it is a mechanism that extracts the company name from the association words associated with the keywords appearing in the document data, so it is purely the content of the article. It becomes possible to present a specific investment destination or the like to a user who has become interested based on the relationship hidden behind the article.
In addition, by clicking on the company name listed in the search result display screen, the corresponding keyword displayed in the document is highlighted, so the user can see the company name in the list. It becomes possible to immediately recognize the cited grounds.

In the associative search system according to claim 2 , since it is a mechanism in which the associative word of the keyword is extracted based on the co-occurrence between the keywords in a large number of document data, it is surprisingly beyond human expectation. It is possible to automatically collect high-quality association words.

In the case of the associative search system according to claim 3 and 4 , keyword candidates are independently extracted from document data using a plurality of filters, and at least those extracted by the plurality of filters are formal keywords. Therefore, it is possible to prevent important keywords from being missed and to prevent unimportant noise from being mixed into the keywords.
In particular, the system of claim 4 has a mechanism for determining whether or not to select an attention word as a keyword candidate based on the appearance frequency in a plurality of document data when extracting keyword candidates. It is possible to give objectivity to the importance of the keyword candidates.

According to the associative search system described in claim 5 , first, in a document data unit, keywords that do not need to be calculated for the degree of association with other keywords because the appearance frequency is zero are excluded in advance. After calculating the degree of relevance, the total calculation process can be simplified as a result of adopting a method of tabulating all documents.
In addition, even when new document data is added to the document storage means, the processing of (1) to (3) is performed for each new document data unit, and the calculation results are added to the existing aggregations of (4) and (5). After adding to the value, it is only necessary to redo the calculations of (6) and (7), and there is an advantage that the recalculation processing of the relevance level when document data is added is facilitated.
Furthermore, when it is necessary to eliminate the influence of outdated document data, after subtracting the values of (2) and (3) related to the old document data from the aggregated values of (4) and (5), Since it is only necessary to redo the calculations of (6) and (7), it becomes easy to keep the relevance between keywords up to date.

FIG. 1 is a block diagram showing the overall configuration of an associative search system 10 according to the present invention, and includes an associative search server 12 and a Web server 14.
The associative search server 12 and the Web server 14 are connected to a network.
The Web server 14 is connected to many other Web servers 18 and the user's PC terminal 20 via the Internet 16.

  The associative search server 12 includes a morphological analysis unit 22, a keyword detection unit 24, an associative word extraction unit 26, a related company extraction unit 28, a keyword DB 32, an associative word DB 34, and a company as a specific species character string storage unit. Name DB36.

The morphological analysis unit 22, the keyword detection unit 24, the associative word extraction unit 26, and the related company extraction unit 28 are realized by the CPU of the server 12 executing necessary processes according to the OS and a dedicated application program.
Further, the keyword DB 32, the associative word DB 34, and the company name DB 36 are provided in the hard disk of the server 12.

In the keyword DB 32, as shown in FIG. 2, many correspondences between keyword IDs and keywords are registered.
In the associative word DB 34, as shown in FIG. 3, correspondences between keyword IDs and a plurality of associative words are registered.
In the company name DB 36, as shown in FIG. 4, correspondences between keywords and company names are registered.

  FIG. 5 is a block diagram showing a functional configuration of the preparation system 40 prepared for generating the keyword DB 32 and the associative word DB 34 provided in the associative search server 12. The preparation system 40 can be configured by the associative search server 12 itself, but can also be configured by another computer. When the preparation system 40 is configured by another computer, the generated data of the keyword DB 32 and the associative word DB 34 are respectively copied to the keyword DB 32 and the associative word DB 34 in the associative search server 12.

  The preparation system 40 includes a document DB 42, a keyword extraction unit 44, a keyword DB 32, a relevance calculation unit 48, a keyword co-occurrence frequency table DB 50, a keyword combination frequency sum table DB 52, a keyword frequency sum table DB 54, A keyword relevance table DB 56, an associative word extraction unit 58, and an associative word DB 34 are provided.

  The keyword extracting unit 44, the relevance calculating unit 48, and the associative word extracting unit 58 are realized by the CPU of the associative search server 12 or another computer executing necessary processes according to the OS and a dedicated application program. The

The document DB 42, the keyword DB 32, the keyword co-occurrence frequency table DB 50, the keyword combination frequency sum table DB 52, the keyword frequency sum table DB 54, the keyword relevance table DB 56, and the associative word DB 34 are stored in the hard disk of the computer.
In the document DB 42, a large number of document files (text data) such as newspaper articles, academic journals, and papers are stored in advance.

  As shown in FIG. 6, the keyword extraction unit 44 includes a dependency expression extraction filter 60, a delimiter extraction filter 62, a character string frequency statistical filter 64, a TermExtract filter 66, and a majority decision filter 68.

Next, the keyword extraction process by the keyword extraction unit 44 will be described with reference to the flowchart of FIG.
First, the keyword extraction unit 44 applies a dependency expression extraction filter 60 to each document file stored in the document DB 42, and extracts a character string having a predetermined dependency expression from each document file (S10).
That is, the dependency expression extraction filter 60 includes a large number of dependency expression patterns “XX manufacturer”, “XX is the main force”, and “XX is produced” in advance. After the expression pattern that applies to is detected, a character string corresponding to “XX” is extracted as a keyword candidate.

  Next, the keyword extraction unit 44 applies a delimiter extraction filter 62 to each document file, such as “XX”, “XX”, (XX), [XX], XX, and so on. The part of XX surrounded by delimiters such as parentheses, spaces, tabs, etc. is extracted as a keyword candidate (S12).

Next, the keyword extraction unit 44 applies a character string frequency statistical filter 64 to each document file, and counts how many times each character string included in each document file appears, including other documents. A character string having an appearance frequency is extracted as a keyword candidate (S14).
First, as shown in FIG. 8, the character string frequency statistical filter 64 pays attention to a noun (here, “DVD”) in the document, and the attention word “DVD” appears in each document file stored in the document DB 42. Add up the number you want. Next, the character string frequency statistical filter 64 expands the range to the morpheme before and after this attention word, totals the frequencies that appear in all the documents, and the appearance frequency becomes less than a certain value (for example, 20 or less). Stop character range expansion at this point.

  For example, since the appearance frequency of “done DVD” including the previous morpheme of the DVD is as low as “2”, the range is not expanded to the previous morpheme. On the other hand, since the appearance frequency of “DVD recorder” including the next morpheme of DVD is as many as “862”, the appearance frequencies of “DVD recorder” including the next morpheme are tabulated. Since the appearance frequency is as low as “5”, the expansion of the range to subsequent morphemes is stopped.

Next, the character string frequency statistical filter 64 extracts “DVD” and “DVD recorder” as keyword candidates because they have an appearance frequency within a predetermined range (for example, 20 to 5,000). On the other hand, “done DVD” and “in the DVD recorder” are out of the above range, and are excluded from keyword candidates.
This is because, if the frequency of occurrence is less than 20 in all documents, it is not an important word in the first place, and if it exceeds 5,000, it is considered to be a general word or general word without features. The amount is adjusted as appropriate according to the amount of use and the purpose of use of the search system.

  By the way, since it takes an enormous amount of time to count the appearance frequency of each character string included in a large amount of document files stored in the document DB 42, as shown in FIG. An index (so-called transposed index) is generated that summarizes whether each morpheme appearing in all document files exists in each document file. Therefore, the keyword extraction unit 44 can acquire the appearance frequency in a relatively short time by referring to this index.

Next, the keyword extracting unit 44 applies the TermExtract filter 66 to the document files stored in the document DB 42, and extracts a character string having a predetermined score or more from each document file as a keyword candidate (S16).
This TermExtract is a string extraction algorithm devised to automatically extract technical terms from a specialized corpus (a huge text data consisting mainly of natural language sentences collected mainly for research purposes). A function is provided for extracting single nouns and compound nouns from the document file as candidate words and calculating the respective importance based on the appearance frequency and the connection frequency of each candidate word. Since this TermExtract itself is a known technique, further explanation is omitted.

Next, the keyword extraction unit 44 inputs the keyword candidates extracted by the dependency expression extraction filter 60, the delimiter extraction filter 62, the character string frequency statistical filter 64, and the TermExtract filter 68 to the majority filter 68, and narrows down the keywords.
The majority filter 68 matches the keyword candidates listed by each filter, recognizes a keyword candidate listed by two or more filters as a final keyword, a keyword ID consisting of a unique serial number, The data is stored in the keyword DB 32 in association (S18).

  As described above, by using the four filters of the dependency expression extraction filter 60, the delimiter extraction filter 62, the character string frequency statistical filter 64, and the TermExtract filter 66, important words are leaked when keywords are extracted from the document file. In addition, by using the majority filter 68 to narrow down, it is possible to prevent unnecessary keywords (noise) from being mixed.

As described above, the keyword candidate selected by two or more of the four filters is recognized as an official keyword, and selection by three or more filters may be a requirement for keyword recognition. it can.
Further, the number of filters is not limited to the above, and other effective keyword candidate extraction filters may be provided in the keyword extraction unit 44.

Next, according to the flowchart of FIG. 10, the relevance calculation process between keywords by the relevance calculation unit 48 will be described.
First, the relevance calculating unit 48 generates a keyword co-occurrence frequency table by counting the appearance frequencies of each keyword in each document file, and stores it in the keyword co-occurrence frequency table DB 50 (S20).
FIG. 11 shows a specific example of the keyword co-occurrence frequency table stored in the keyword co-occurrence frequency table DB 50. For each document D1 to Dn stored in the document DB 42, each keyword KW-1 to n is stored. Appearance frequency is described.

Here, the degree of association between a certain keyword X and Y can be theoretically calculated by substituting the appearance frequency of X and Y described in the keyword co-occurrence frequency table DB50 into i of Equation 1. is there.

  Since the numerator of Equation 1 means the sum of the products of the appearance frequencies of the keywords X and Y for all documents, the value increases as the frequency of occurrence of X and Y in the same document increases. However, if the absolute number of occurrence frequencies of X and Y in a specific document is large, the value of the numerator increases accordingly, and it does not necessarily indicate the high co-occurrence of X and Y. . On the other hand, the denominator is obtained by adding the square roots of the sums of all the squares of the appearance frequencies of the keywords X and Y for each document, and the value increases as the appearance frequency in the specific document of X and Y increases. Become. For this reason, by dividing the numerator value by the denominator value, the influence of the large number of occurrence frequencies of X and Y in a specific document is eliminated, and the co-occurrence between X and Y is increased. It is possible to derive the degree of relevance based on it.

However, in the method of simply performing the calculation of Equation 1, if the amount of document files and the total number of keywords are large, a huge amount of calculation occurs, and a lot of processing time is required.
Therefore, in this embodiment, the calculation process is simplified by generating the keyword combination frequency summation table and the keyword frequency summation table based on the keyword co-occurrence frequency table.

FIG. 12 illustrates the procedure. In this case, the keyword co-occurrence frequency table describes the appearance frequencies of the keywords KW-1 to KW-5 in the document D1, but the KW-3 and KW-4 appearance frequencies are 0. The combination of keywords for which the degree of relevance should be calculated is the following three patterns.
(KW-1, KW-2), (KW-1, KW-5), (KW-2, KW-5)
Next, the relevance calculator 48 generates a keyword combination frequency sum table describing values multiplied by the appearance frequency for each combination, and a keyword frequency sum table describing values obtained by squaring the appearance frequency of each keyword, They are stored in the combination frequency summation table DB52 and the keyword frequency summation table DB54 (S22, S24).

In the keyword combination frequency summation table of FIG. 12, only the value for the document D1 is described. However, the same processing is executed for each document, and the value is added based on the result. A value corresponding to the numerator is obtained.
Similarly, in the keyword frequency total table of FIG. 12, only the value for the document D1 is described, but the value obtained by squaring the appearance frequency of each keyword in each document is tabulated, and the final value ( By calculating the square root of (sum), a value corresponding to the denominator of Equation 1 is obtained.

  Finally, as shown in FIG. 13, the degree-of-relevance calculation unit 48 reads the sum of the combination frequencies of the keywords X and Y from the keyword combination frequency sum table DB 52 and also calculates the sum of the square values of the keywords X from the keyword frequency sum table DB 54. After reading the sum of the square values of the keyword Y and calculating the square root of the sum of the square values, substituting these values into Equation 1 calculates the relevance between the keywords X and Y, and the keyword relevance table Store in DB26 (S26). The degree-of-association calculation unit 48 repeats the process until the process is completed for all keyword combinations.

  As described above, the combination pattern between each keyword is extracted for each document file, the product value and the square value of each keyword are obtained, and then the value of each document file is added, so that the appearance frequency This makes it possible to omit the calculation processing related to the keyword with 0.

When a new document file is added to the document DB 42, the values associated with each keyword in the new document file are stored in the existing combined values stored in the keyword combination frequency summation table DB52 and the keyword frequency summation table DB54. By adding to, it is possible to easily recalculate the degree of association between keywords.
Even when the influence of an obsolete document file is eliminated, by subtracting the value related to each keyword in the document file from the existing total value stored in the keyword combination frequency summation table DB52 and the keyword frequency summation table DB54. Thus, it is possible to easily maintain the degree of association between keywords in the latest state.

Finally, the associative word extraction unit 58 is activated to extract the associative word of each keyword from the keyword association degree table DB 56 and store it in the associative word DB 34 (S28).
That is, the associative word extraction unit 58 sorts the relevance values with other keywords for each keyword, recognizes the top predetermined number of keywords (for example, 50) having high relevance as associative words, It is stored in the associative word DB 34 in association with the keyword ID.

As described above, when the necessary data is stored in the keyword DB 32 and the associative word DB 34 by the preparation system 40, the process by the associative search system 10 is started.
Hereinafter, the procedure of the associative search process by the system 10 will be described with reference to the flowchart of FIG.

First, a document file (new news data) is periodically fetched from another web server 18 (for example, a news site) on the Internet by the web server 14 (S30) and transmitted to the associative search server 12.
Upon receiving this, the morpheme analyzer 22 of the associative search server 12 decomposes each document data into “morphemes” which are the smallest units having linguistic meaning (S32).

Next, the keyword detecting unit 24 is activated, compares each morpheme with the keyword stored in the keyword DB 32, detects a matching morpheme as a keyword existing in the document file, and then sets the ID of each keyword as an associative word. The data is output to the extraction unit 26 (S34).
In order to realize the morpheme analysis unit 22 and the keyword detection unit 24, an existing morpheme analysis algorithm (for example, “MeCab” or “ChaSen”) may be used.

  The associative word extraction unit 26 that has received the ID of each keyword searches the associative word DB 34 using this as a key, extracts each associative word (for each 50 items) (S36), and then outputs it to the related company extraction unit 28 To do.

Receiving this, the related company extraction unit 28 refers to the company name DB 36 to extract the association word corresponding to the company name as the related company, and generates search result data (S38).
As shown in FIG. 15, in this search result data, a list of related companies and a keyword in a document paired with each related company are described for each document ID.

Upon receiving this search result data from the associative search server 12, the Web server 14 generates a search result display screen (HTML file) in which the search result data is reflected in the original document data (S40).
This search result display screen is distributed to the PC terminal 20 in response to a request from the user (S42).

  FIG. 16 shows an example of this search result display screen 70, in which a document display field 72 in which the contents of the original text document are described, and a company name closely related to the keyword appearing in the document are listed. A related company list column 74 is provided.

By browsing the search result display screen 70, the user can read that the content of the article is deeply related to Furuta, Takagi, Okabe Electric, and Shimada Construction.
In addition, when the user clicks on Furuta car, which is one of the related company lists, the company name and link are set by the action of the control program (JavaScript, etc.) described in the HTML file on the search result display screen 70. The keyword “hybrid vehicle” is highlighted.
For this reason, users who have determined that “hybrid vehicles” are promising from the content of the article can immediately recognize that Furuta vehicles are closely related to the hybrid vehicles and can recognize them as candidates for investment. It becomes.

  In the associative search system 10, as described above, the web server 14 periodically retrieves a large number of document files from a website on the Internet, and the associative search server 12 automatically includes a list of related companies of each document file. After the result data is generated, the search result display screen 70 on the Web server 14 is not limited to a usage mode for general user browsing.

  For example, a user uploads a specific document file that the user is paying attention to to the Web server 14, and the associative search server 12 performs an associative search process on the individual document file, and displays a list of related companies and search results of the document file It is also possible to operate so that the screen 70 is returned to the user's PC terminal 20 via the Web server 14.

The associative search system 10 is not limited to the purpose of searching for an investment target company as described above, but can also be applied to the purpose of searching for a purchase target product, a support target person, and the like. In this case, instead of the company name DB 36, a product name DB or person name DB storing a large number of product names and person names is prepared as a specific type character string storage means, and a related function having the same function as the related company extraction unit 28 is provided. By using the product extraction unit and the related person extraction unit, it is possible to list product names and person names that are closely related to a specific document.
Of course, a database that stores various character strings, such as company names, product names, and person names, is provided, and the company names, product names, and person names related to specific documents are listed up comprehensively. You can also.

It is a block diagram which shows the function structure of the associative search system which concerns on this invention. It is a figure which shows the structural example of keyword DB. It is a figure which shows the structural example of an associative word DB. It is a figure which shows the structural example of company name DB. It is a block diagram which shows the function structure of a preparation system. It is a block diagram which shows the structure of a keyword extraction part. It is a flowchart which shows a keyword extraction process. It is explanatory drawing which shows operation | movement of a character string frequency statistical filter. It is explanatory drawing which shows a mode that the morpheme index is formed in document DB. It is a flowchart which shows the related degree calculation process between keywords, and an associative word extraction process. It is explanatory drawing which shows an example of a keyword co-occurrence frequency table. It is explanatory drawing which shows the method of simplifying a relevance calculation process. It is explanatory drawing which shows a mode that a keyword relevance table is produced | generated based on a keyword combination frequency total table and a keyword frequency total table. It is a flowchart which shows the procedure of an associative search process. It is a figure which shows an example of search result data. It is a figure which shows an example of a search result display screen.

Explanation of symbols

10 Associative search system
12 Associative search server
14 Web server
16 Internet
18 Other web servers
20 PC terminal
22 Morphological analyzer
24 Keyword detector
26 Associative word extractor
28 Related Company Extraction Department
32 Keyword DB
34 Associative Word DB
36 Company Name DB
40 Preparation system
42 Document DB
44 Keyword extractor
48 Relevance calculator
50 Keyword co-occurrence frequency table DB
52 Keyword combination frequency summary table DB
54 Keyword Frequency Summation Table DB
56 Keyword Relevance Table DB
58 Associative Word Extraction Unit
60 Dependency Expression Extraction Filter
62 Delimiter extraction filter
64 string frequency statistics filter
66 TermExtract filter
68 Majority filter
68 Majority filter
70 Search result display screen
72 Document display field
74 Related company list

Claims (5)

Keyword storage means for storing a plurality of keywords in advance;
An associative word storage means for storing, for each keyword, a plurality of associative words selected based on the high degree of co-occurrence with each keyword;
Company name storage means for storing character strings related to a plurality of company names in advance;
Means for dividing input document data into morpheme units;
Means for comparing each morpheme with the keyword and detecting a keyword included in the document data;
Means for acquiring an associated word of each detected keyword with reference to the associated word storage means;
A means for referring to the company name storage means and extracting, as associated company names , those associated with the character strings stored in the company name storage means among the associative words;
Means for generating a search result display screen having a related company list field for displaying a list of the related company names and a document display field for displaying the contents of the document data;
A link is set between each affiliated company name in the affiliated company list column and the corresponding keyword in the document display column.
An associative search system, wherein when a specific company name in the related company list field is selected, a corresponding keyword in the document display field is highlighted . Document storage means for storing a plurality of document data;
A keyword extracting means for extracting a plurality of keywords from each of the document data and storing them in the keyword storage means;
A means for totalizing the appearance frequency of each keyword in all document data and storing it in a co-occurrence frequency storage means;
Using the appearance frequency data in each document data of each keyword, calculating a relevance level based on the co-occurrence between keywords, and storing the relevance level in a keyword relevance storage unit;
After sorting the degree of association with other keywords for each keyword, an associated word extraction unit that extracts the upper predetermined number of keywords having a high degree of association as an association word of the keyword and stores it in the association word storage unit; The associative search system according to claim 1 , further comprising: The keyword extraction means includes a plurality of filters that extract keyword candidates based on unique extraction criteria,
The associative search system according to claim 2 , wherein keyword candidates extracted by each filter are matched, and keyword candidates extracted by at least a plurality of filters are recognized as keywords. One of the above filters is
(1) Extract nouns included in each document data as attention words,
(2) Calculate the appearance frequency of all the attention words in all document data,
(3) Extend the range to the morpheme one and the next before each attention word, and calculate the appearance frequency of the attention word including this expansion range in all document data,
(4) If the appearance frequency calculated by the processing in (3) above is a predetermined number or more, the range is further expanded to the previous or subsequent morpheme, and all documents of the attention word including this expanded range Repeat the process of calculating the appearance frequency in the data until the appearance frequency falls below a predetermined number,
(5) The associative search system according to claim 3 , wherein among the attention words including the first attention word and the expanded range, words having an appearance frequency within a predetermined range are selected as keyword candidates. The relevance calculation means is
(1) In a document data unit, a process of selecting a keyword that has an appearance record in the document and should be a target of relevance calculation;
(2) Multiply the appearance frequency between each selected keyword in document data units, and record the product in a predetermined storage means;
(3) A process of squaring the appearance frequency of each selected keyword in document data units and recording the value in a predetermined storage means;
(4) a process of summing up the product between the selected keywords over all document data;
(5) A process of summing up the square value of the appearance frequency of each selected keyword over all document data;
(6) A process for calculating the square root of the aggregate value of (5) above,
(7) A process of calculating the degree of association between both keywords by adding the square roots of (6) above for each keyword and dividing the sum of the above (4) by the sum,
The associative search system according to any one of claims 2 to 4 , wherein:

Images