JP2005031867A - Web information collecting device and web information collecting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for efficiently collecting the information of a home page including a keyword at the time of collecting the information of a Web page through the Internet. <P>SOLUTION: Character data across the plurality of hierarchies of a designated home page are downloaded, and the downloaded character data file is retrieved by using a preliminarily set keyword, and when the date of the character data file is collated with already registered data, and they are made incoincident, the overall home page hit by the keyword retrieval is downloaded, and the changed contents of the downloaded home page are notified to a user concerned as a mail. Also, the headline of the home page hit by the keyword is edited, and the information of the home page is distributed as news. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for collecting information on a homepage including a keyword and delivering a new or changed homepage as a result of keyword search to a related person by e-mail.
[0002]
[Prior art]
With the spread of the Internet, it is common to collect desired web information from a WWW (World Wide Web) site on the Internet by using a search engine typified by Google. (For example, refer nonpatent literature 1).
[0003]
However, conventionally, when a keyword search is performed by a search engine, a huge number of homepages that cannot be referred to are hit. In addition, as conditions were added, there were many problems such as the homepage you would like to refer to was often not hit.
Furthermore, when a homepage that has been hit by a keyword search is downloaded, a homepage that the user does not want to refer to is downloaded, so that processing time is required and desired information cannot be specified immediately. In addition, I have to reconfirm homepages that have not been updated, and it is difficult to refer to the information I want. In addition, multiple users who belong to a group and perform collaborative work refer to the same homepage and confirm the same content, which is very inefficient.
[0004]
[Non-Patent Document 1]
Nikkei BP “Nikkei PC” New Year 2001 Special Issue, No. 376 (92-115 pages)
[0005]
[Problems to be solved by the invention]
Therefore, in order to solve the above problem, in the present invention, a page including a keyword is referred to over a plurality of hierarchies of the home page specified by the keyword, and the user is notified if the home page hit by the keyword search has been revised. (Even if keywords are included, if you have visited the homepage before, it will be out of the reference range.) In addition, even if you can find a revised or new homepage by keyword search, there are few homepages that you really want to reference. Therefore, one person confirms the homepage, summarizes the contents (headlines), distributes the contents to the parties concerned by e-mail, and aims to reduce the reference time of the homepage.
[0006]
[Means for Solving the Problems]
According to a first aspect of the present invention, in a web information collecting apparatus via the Internet, a web patrol unit that patrols a designated home page over a plurality of layers and automatically downloads character data on the home page; Character data file storage means for storing the file as a file, keyword search means for searching for the character data file downloaded by the web patrol means with a preset keyword, and the date of the character data file refer to the registered data In the case of mismatch, as a result of the keyword search, a homepage download means for downloading the entire homepage where the keyword is hit, and information for automatically notifying related users of updated or new content information of the downloaded homepage A knowledge unit, related web information collection apparatus characterized by having a.
[0007]
That is, according to the first invention, the web circulating means downloads the text data by patroling a plurality of layers of the designated homepage, searches the downloaded text for a keyword, and the keyword exists as a result of the search. The entire homepage is downloaded and the update date is checked to automatically notify the user of the revised or new homepage.
[0008]
This allows you to search for keywords within the specified range (download file), so you can easily find the required homepage, and display the homepage that contains the specified keyword in multiple levels of the specified homepage. When the homepage is revised, only the newly registered homepage is notified, so there is no need to refer to a homepage that has not been changed, and the search processing time can be greatly shortened.
[0009]
The second invention relates to the web information collecting device according to the first invention, wherein the web patrol means is to circulate only the homepage where the keyword is hit in the keyword search means from the next time. .
In other words, according to the second invention, since only the home page that hits the keyword search in the download file by the first visit is visited, the visit processing time is greatly reduced from the next time, so that the visit processing time is greatly reduced and the total web information is reduced. The management man-hour for collection is reduced.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 shows the basic system configuration of the present invention. The system of the present invention includes a web information collecting device 1 that collects homepage information from a plurality of information providing servers (not shown) connected to the web 3 via the Internet 3, and a URL (Uniform Resource) indicating the location of the homepage. A plurality of user terminals connected to the Web information terminal 1 via a LAN (Local Area Network) such as setting of Locators), keyword setting, search result confirmation, and news posting, etc., and inputting environment settings for searching and searching the Web information 2 and is a system for sharing and managing web information collected within the group.
[0011]
The web information collection apparatus 1 also stores an environment condition for holding a URL management database 10 for patrol of a designated home page, and patrol conditions and keyword conditions for web information patrol and search conditions entered by the user. A web patrol that accesses a file 11, patrol condition acquisition means 12, and an information providing server connected to the Internet according to a designated URL, and patrols across multiple layers of the homepage to automatically download character data (extension designation). Means 13; character data file storage means 14 for storing the character data for each home page; download folder 15 for storing the downloaded character data file; and keyword search for searching the stored download file with a preset keyword. Means 16 and As a result of keyword search, the hit information database 17 for storing the hit information, the home page download means 18 for downloading the entire hit home page, and the update information of the downloaded file if there is any change or new information Information notification means 19 for notifying related users.
[0012]
Here, the web information collection device 1 is a computer, and a program built in advance is executed on the computer, and a tour condition acquisition unit 12, a web tour unit 13, a character data file storage unit 14, a keyword search unit 16, a home page. Each means of the download means 18 and the information notification means 19 is implement | achieved. The program is recorded on a computer-readable recording medium such as a floppy disk, a compact disk, or a CD-ROM. Although not specifically shown in the figure, the program is installed in a built-in or externally connected medium reader. It is good also as an executable state by doing.
[0013]
In the following embodiments, “collection of defect information” will be described as an example of web information collection.
FIG. 2 is a diagram showing a data configuration example of the URL management database according to the embodiment of the present invention. The data structure of the URL management database 10 includes a URL representing the location of a web page on the Internet, a company name, a URL registration date, a patrol collection stop (pause if no patrol occurs), a URL collection date and time, a site with security It includes items such as an authentication ID at the time of access, an authentication password, the number of hierarchies of URLs, initial collection (represented by 1 or 0), and update date / time.
[0014]
FIG. 3 is a diagram showing a data configuration example of the hit information database according to the embodiment of the present invention. The hit information database 17 holds processing information as data (news posting) for notifying related users described later from a file with a specified extension that has been hit by a keyword search of character data in the downloaded homepage. Keep it.
[0015]
The data structure of the hit information database 17 includes items such as presence / absence of keyword hit, number of keyword hits, total number, number of acquisitions, number of impossibility, title, and headline. The presence / absence of keyword hit is expressed by 1 or 0 for the presence / absence of hit for one keyword. For example, if the number of keyword columns is 8, it is represented by 00100010 corresponding to each keyword, and the total number of hits is counted as 2. The total number, the acquisition number, and the impossibility number indicate the number of URLs to be searched. The title is expressed as <TITLE>, and if there is no description, there is no title. The headline is a representation of the number of characters (for example, 100 characters) from the head of the text hit by the keyword.
[0016]
Next, in order to collect the web information according to the present invention, the setting items of the web tour in advance will be described with reference to FIGS.
FIG. 4 is a diagram showing an example of an initial menu screen according to the embodiment of the present invention. The initial menu screen includes three screen areas 101, 102, and 103.
[0017]
The screen area 101 displays data for items of company name, URL, update date, and number of hits for the URL list searched so far. □ is a check box, a blank indicates that a tour is required, and a cross indicates that it has been processed as not subject to a tour. This reflects the result selected by the user in each setting menu of the screen area 103.
[0018]
The URL is represented by (protocol name): // (domain name) / (file name). In the figure, http: // www. aaa. com / index. http is the URL of the company AAA's top homepage, and http: // is a transmission command using a protocol called HTTP (HyperText Transfer Protocol) with which a web client communicates with an information providing server, followed by www. aaa. com represents the name of the WWW server on the Internet where the homepage is stored. htm represents the top page of the home page. In addition, (.htm) represents an extension of a file in a language HTML (HyperText Markup Language) describing a home page.
[0019]
http: // www. aaa. com / xxx1 / index. http: // www. aaa. com / xxx1 / pr. htm represents the second and third level web pages.
In the URL list of the screen area 101, for example, URL [http: // www. aaa. com / xxx1 / index. The web page of [htm] indicates that there were five hits on the month and month dd of yyyy year as a result of the keyword search.
[0020]
In the screen area 102, items such as defect information, new product information, and technical information are displayed. In this embodiment, defect information is displayed.
Further, in the screen area 103, URL addition (setting of URL, hierarchy, identifier), URL deletion (URL list, deletion of data from folder), collection stop / restart (instruction of collection stop / restart for each URL), collection A search environment setting menu to be entered by the user, such as a schedule (execution date / day of the week / time setting), keyword setting (search keyword setting for download data), and news posting, is displayed. By clicking each menu button with the mouse, each setting screen is displayed, and input by the user is performed on the display screen.
[0021]
In the setting menu, first, URL addition and URL deletion are setting functions for adding and deleting pages in units of top homepages of companies such as AAA and BBB.
Hereinafter, functions of other menus will be described.
FIG. 5 shows an example of an environment setting screen (keyword setting) according to the embodiment of the present invention. For example, the keyword column 1 represents a search using a search expression “defect and capacitor and LSI” as a search keyword.
[0022]
FIG. 6 shows an example of the environment setting screen (collection stop / restart setting) according to the embodiment of the present invention. The URL list is instructed to stop / restart collection, and the URL file whose collection has been stopped is temporarily moved out of the search target folder and returned to the search target when restarting. If the result of the previous hit is 0, the file is removed from the cyclic collection target by putting an X in the check box □. By selecting all the checks and finally displaying the selection screen of the collection stop / resume screen, it is possible to collectively stop collecting URLs with 0 hits.
[0023]
FIG. 7 shows an example of an environment setting screen (collection schedule setting) according to the embodiment of the present invention. As setting items, the total cycle, difference cycle, cycle time, collection target file, and search hierarchy are displayed. In the entire tour cycle, the number of times collected in each month and week is set, and in the difference tour, a setting is made in which only the changed portion is extracted and toured in comparison with the previous search. In this sample screen, select all tours, specify once / week, day of the week on Monday, tour time is 1:00, and the files to be collected specify the extensions of html / html, txt, doc, and xls. The collection schedule is set so that the hierarchy, which is the depth of the cyclic search, is three.
[0024]
Here, html / html is an extension of a homepage description language HTML, and txt is an extension of a text format. Doc is a registered trademark of Microsoft Corporation, xls is a registered trademark of Microsoft Corporation, and pdf is an extension of a registered trademark of Adobe Corporation.
FIG. 8 shows a flowchart of the web tour based on the designated condition according to the embodiment of the present invention. First, in step S11, the URL input by the user and designated from the held environment setting file is read. In step S12, the web circulator 13 circulates the home page of the information providing server connected to the Internet over a plurality of layers, downloads the character data file with the designated extension in step S13, and downloads it to the download folder 15 in step S14. save. In step S15, the above processing is repeated until all the specified URLs read from the environment setting file 11 are completed.
[0025]
When all the URLs are completed, the process proceeds to the next processing flow (1).
FIG. 9 shows a flowchart of keyword search for a download file according to the embodiment of the present invention. In step S21, the download folder 15 is referred to in step S21. In step S22, reference to the download file is continued until the downloaded character data file ends. In step S23, a keyword search is performed on the collected character data file. If the search conditions match in step S24, the process proceeds to step S25, and the date of the downloaded individual character data file with the previously downloaded file is checked.
[0026]
If the dates do not match in step S26, necessary information is stored in the hit information database 17 in accordance with the data structure of FIG.
In step S24, if the keyword condition does not match, the flag of the file in the hit information database 17 is turned OFF. If the date condition does not match in step S26, the process returns to step S25 and the download file check is repeated.
[0027]
When the above process flow is completed, the process proceeds to the next process flow (2).
FIG. 10 shows a flowchart of automatic notification of updated homepage contents according to the embodiment of the present invention. In response to the processing flow of FIG. 9, in step S31, the hit information database 17 is referred to, and in step S32, processing is performed until there is no more data in the hit information database 17. In step S33, the hit information database 17 is selected if the download file flag is ON, and in step S34, the entire designated URL homepage with the flag is downloaded. In step S35, the downloaded URL is saved.
[0028]
Next, in step S32, when all the data processing is completed, the downloaded URL is extracted in step S36, and in step S37, the information on the change contents of the homepage of the downloaded URL is edited (news posting). Automatic notification by email.
FIG. 11 shows an example of creating an automatic posting according to the embodiment of the present invention. By checking the keyword search and update date, the entire home page where the keyword exists is downloaded, and information about the updated or new home page is automatically notified to related users. In this screen example, automatic posting and manual posting can be selected. In the automatic posting, for example, URL1 (yyyy / mm / dd) hit for the keyword string 1, URL2 (yyyy / mm / dd)... URLa (yyyy / mm / dd) hit for the keyword string 2, A URL group of URLb (yyyy / mm / dd)... Is automatically listed and notified to related users. For manual posting, a free-form news posting screen is prepared.
[0029]
FIG. 12 shows a mail notification example of the change contents according to the embodiment of the present invention. The URL of the web page that has been changed, the update date, the hit keyword, and the headline about the content are extracted and the relevant users are automatically notified by email.
In the above embodiment, the description has been mainly made assuming the case of “defect information”, but the present invention is of course not limited to this, and “new product information”, “technical information”, “ It goes without saying that the same technique is applied to collecting web information of a wide range of genres such as “patent information”.
[0030]
(Supplementary note 1) In the web information collection device via the Internet,
A web patrol unit that patrols a designated homepage over a plurality of layers and automatically downloads character data on the homepage;
Character data file storage means for storing the character data as a file for each homepage;
Keyword search means for searching for a character data file downloaded by the web patrol means using a preset keyword;
When the date of the character data file is inconsistent with reference to already registered data, as a result of the keyword search, a homepage download means for downloading the entire homepage where the keyword is hit,
An information notification means for automatically notifying related users of updates of the downloaded home page or information on new contents.
[0031]
(Additional remark 2) The notification information in the said information notification means is the headline information automatically extracted by extracting the character string before and behind including the hit keyword, The web information collection apparatus of Additional remark 1 characterized by the above-mentioned.
(Supplementary note 3) The web information collecting device according to supplementary note 1, wherein the web patrol device circulates only the homepage where the keyword is hit in the keyword search step.
[0032]
(Appendix 4) In the web information collection method via the Internet,
A web patrol step of patroling a designated homepage over a plurality of layers and automatically downloading character data on the homepage;
A character data file storing step for storing the character data as a file for each homepage;
A keyword search step for searching for a character data file downloaded by the web patrol step using a preset keyword;
When the date of the character data file is inconsistent with reference to already registered data, the result of the keyword search is a homepage download step of downloading the entire homepage where the keyword is hit;
An information notification step of automatically notifying related users of updated or new content information of the downloaded homepage;
A web information collecting method characterized by comprising:
[0033]
(Appendix 5) In the web information collection program via the Internet,
On the computer,
A web patrol step of patroling a designated homepage over a plurality of layers and automatically downloading character data on the homepage;
A character data file storing step for storing the character data as a file for each homepage;
A keyword search step for searching for a character data file downloaded by the web patrol step using a preset keyword;
When the date of the character data file is inconsistent with reference to already registered data, the result of the keyword search is a homepage download step of downloading the entire homepage where the keyword is hit;
An information notification step of automatically notifying related users of updated or new content information of the downloaded homepage;
Web information collection program to execute.
[0034]
(Supplementary note 6) A recording medium recording a web information collection program via the Internet,
On the computer,
A web patrol step of patroling a designated homepage over a plurality of layers and automatically downloading character data on the homepage;
A character data file storing step for storing the character data as a file for each homepage;
A keyword search step for searching for a character data file downloaded by the web patrol step using a preset keyword;
When the date of the character data file is inconsistent with reference to already registered data, the result of the keyword search is a homepage download step of downloading the entire homepage where the keyword is hit;
An information notification step of automatically notifying related users of updated or new content information of the downloaded homepage;
A computer-readable recording medium having a web information collecting program recorded thereon for recording.
[0035]
【The invention's effect】
As described above, according to the present invention, a home page including a specified keyword is displayed in a plurality of levels of the specified home page, and when the home page is revised, only a newly registered home page is notified. , It is not necessary to refer to a homepage that has not been changed, and the search processing time can be greatly reduced.
[0036]
In addition, because it is possible to search for keywords within the specified range (download file), it is possible to easily find the necessary homepage.
Furthermore, according to the present invention, the person in charge confirms the contents of the homepage, changes the headline and delivers it to the person who needs it by e-mail. Therefore, the contents other than the person in charge (headline) If you do not need to refer to the homepage just by checking, you can reduce the man-hours required for searching.
[Brief description of the drawings]
FIG. 1 is a diagram showing a basic system configuration according to the present invention.
FIG. 2 is a diagram showing a data configuration example of a URL management database according to the embodiment of the present invention.
FIG. 3 is a diagram showing a data configuration example of a hit information database according to the embodiment of the present invention.
FIG. 4 is a diagram showing an example of an initial menu screen according to the embodiment of the present invention.
FIG. 5 is a diagram showing an example of an environment setting screen (keyword setting) according to the embodiment of the present invention.
FIG. 6 is a diagram showing an example of an environment setting screen (collection stop / restart setting) according to the embodiment of the present invention;
FIG. 7 is a diagram showing an example of an environment setting screen (collection schedule setting) according to the embodiment of the present invention.
FIG. 8 is a diagram showing a flowchart of web patrol based on designated conditions according to the embodiment of the present invention.
FIG. 9 is a flowchart showing a keyword search for a download file according to the embodiment of the present invention.
FIG. 10 is a diagram showing a flowchart of automatic notification of updated homepage contents according to the embodiment of the present invention.
FIG. 11 is a diagram showing an example of creating an automatic posting according to the embodiment of the present invention.
FIG. 12 is a diagram showing a mail notification example of change contents according to the embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Web information collection apparatus 2 User terminal 3 Internet 10 URL management database 11 Environment setting file 12 Travel condition acquisition means 13 Web tour means 14 Character data file storage means 15 Download folder 16 Keyword search means 17 Hit information database 18 Homepage download means 19 Information Notification means

Claims (3)

In the web information collection device via the Internet,
A web patrol unit that patrols a designated homepage over a plurality of layers and automatically downloads character data on the homepage;
Character data file storage means for storing the character data as a file for each homepage;
Keyword search means for searching for a character data file downloaded by the web patrol means using a preset keyword;
When the date of the character data file is inconsistent with reference to already registered data, as a result of the keyword search, a homepage download means for downloading the entire homepage where the keyword is hit,
An information notification means for automatically notifying related users of updated or new contents information of the downloaded home page. 2. The web information collecting apparatus according to claim 1, wherein the web patrol unit circulates only the homepage where the keyword is hit in the keyword search unit from the next time. In the web information collection method via the Internet,
A web patrol step of patroling a designated homepage over a plurality of layers and automatically downloading character data on the homepage;
A character data file storing step for storing the character data as a file for each homepage;
A keyword search step for searching for a character data file downloaded by the web patrol step using a preset keyword;
When the date of the character data file is inconsistent with reference to already registered data, the result of the keyword search is a homepage download step of downloading the entire homepage where the keyword is hit;
An information notification step of automatically notifying related users of updated or new content information of the downloaded homepage;
A web information collecting method characterized by comprising:

Images