JP3298516B2 - Internet information search system and method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an Internet information search system, and more particularly, to creating a database necessary for a search engine that searches a large number of documents and data that can be referred to via the Internet, by using documents and URLs (Uns
The present invention relates to an Internet information search system that acquires an iform resource location.

[0002]

2. Description of the Related Art Conventionally, an Internet information search system has been used to acquire URL information over a wide area in advance so that the information desired by the user can be immediately searched from information on the Internet. However, a program for actually searching for a URL and acquiring data is called a robot. However, since the acquisition interval of URL information of each robot is constant and no consideration is given to other activated robots, a robot is used. If the timing to get URL information to the target server is bad, the CP
U or the network may be overloaded.

[0003]

The first problem with the conventional Internet information search system described above is that if the timing at which the robot obtains the URL information from the target server is bad, the load on the CPU and the network is increased. That is to say,

[0004] The reason is that the URL information acquisition interval of each robot is constant, and no consideration is given to other activated robots.

[0005] An object of the present invention is to consider the load of a server from which a URL is to be obtained, and to obtain a URL of the same server at a certain time interval, repeat this operation to a certain extent, and create a database. An object of the present invention is to provide an Internet information search system having a function of adjusting the activation time of a robot and the time for the activated robot to access the next URL according to the status of another robot.

[0006]

SUMMARY OF THE INVENTION An Internet information search system according to the present invention includes a data processing device and a storage device for storing information.
An RL information storage unit, wherein the robot information storage unit includes an ID for identifying the robot, a startup time of the robot,
The time required for connection is stored, the URL information storage unit stores the acquired information of the WWW server and the information including the final acquisition time of the URL, and the data processing device includes a robot management unit, a robot and a URL management unit, The robot management means acquires an unacquired WWW server from the URL management means,
Activate one robot for one WWW server,
The start-up time of the next robot is adjusted by referring to the robot information storage unit, the robot is started by the robot management unit, acquires the target URL, notifies the robot management unit of the acquisition time of the URL, and acquires the acquired URL. After notifying the URL to the URL management means, and after being started, the URL management means is inquired about the next URL, and then the robot management means is inquired about the next start time, and access is made based on the time.
The L management means receives a request for the next WWW server list from the robot management means, receives a request for the next URL to be searched from the robot, and searches the URL information storage unit.
The result is passed to the robot management unit and the robot, a URL notification is received from the robot, and the URL information storage unit is updated or registered.

[0007]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic diagram showing an embodiment of the Internet information search system of the present invention.

As shown in FIG. 1, an embodiment of the present invention employs a search engine (for example, Yahoo or Go) for searching a large number of documents and data that can be referred to via the Internet.
Documents and URLs (Uniform Resource) to create a database required for
e Location) is a function of the Internet information search system that acquires e-Location. The document referred to here is HTML (HyperText Markup Lap).
nguage), and a document and data linked from the document. A program that actually searches for a URL and acquires data is called a robot. When the first URL to be collected by the Internet information search system is designated, the robot for the server in which the designated URL exists is started, and the URL (http in FIG. 1) is used.
p /// server1 / index. html) and register it in the database. Also, the contents of the URL are decrypted to extract link information (http: // in FIG. 1).
/ Server2 / index2. html) to obtain a new URL. New URL
If is on a different server, the robot is started again for the new server (ie, one robot is started per server). Considering the load of the server to be acquired, a certain period of time is set when acquiring the URL of the same server. In the Internet information search system that repeats this operation to a certain extent and creates a database, a function to adjust the start time of the robot and the time for the started robot to access the next URL according to the status of other robots is provided. Is what you do.

FIG. 2 is a block diagram showing an embodiment of the Internet information search system according to the present invention. Referring to FIG. 2, the present embodiment includes a data processing device 1 that operates under program control and a storage device 2 that stores information.

The storage device 2 includes a robot information storage unit 21 and a U
The RL information storage unit 22 is provided.

FIG. 6 is a block diagram showing the operation of an embodiment of the Internet information search system according to the present invention. FIG.
, The robot information storage unit 21 stores an ID for identifying a robot, a startup time of the robot, and a time required for connection. URL information storage unit 22
Stores the information of the WWW server and the information of the URL that have been acquired so far. The URL information includes the last acquisition time.

The data processing device 1 includes a robot management unit 11
, A robot 12 and a URL management means 13.

The robot management means 11 is a URL management means 1
3. An unacquired WWW server is acquired from 3 and one robot is activated for one WWW server. At this time, the time for starting the next robot 12 is adjusted with reference to the robot information storage unit 21.

The robot 12 is started by the robot management means 11 and acquires a target URL. The time required at this time is notified to the robot management means 11, and the obtained UR
L notifies the URL management means 13. After activated, U
After inquiring the RL management means 13 about the next URL, the robot management means 11 is inquired about the next start time, and access is made according to the time.

The URL management means 13 is a robot management means 1
In response to a request for the next WWW server list from 1, a request for the next URL to be searched from the robot 12 is received, the URL information storage unit 22 is searched, and the result is passed to each. Also, it receives a URL notification from the robot 12,
The URL information storage unit 22 is updated or registered.

Next, the operation of the embodiment of the present invention will be described in detail with reference to FIGS.

FIG. 3 is a flowchart showing the operation of the Internet information search system according to the present invention when the robot is started.

FIG. 4 is a flowchart showing the operation of the Internet information search system of the present invention when connecting to the next URL.

FIG. 5 is a sequence chart showing the operation of the conventional robot and the robot of the present invention.

FIG. 6 is a block diagram showing the operation of an embodiment of the Internet information search system according to the present invention.

FIG. 7 is a sequence chart showing a method of calculating the number of activated robots per unit time according to an embodiment of the Internet information search system of the present invention.

The robot 12 adjusts the connection time in two patterns: when the robot management means 11 starts the robot 12, and when the robot 12 tries to connect to the next URL.

3 and 4, respectively. FIG. 3 is a flowchart showing the operation at the time of starting the robot in FIG. 3. First, the robot management unit 11 makes the URL management unit 13 search WWW to be searched next.
Inquire about the server name (step A1). Where UR
The L management unit 13 accesses the URL information storage unit 22,
It is determined whether or not processing has been completed for all WWW servers in the search range (step A2). As a result, if there is no WWW server to be searched next, it waits for the currently operating robot 12 to end. The robot management means 11 accesses the robot information storage unit 21, acquires the next connection time of the other robot 12 and the time required for connection, and checks whether the connection timing does not overlap (step A3). If they overlap, calculate the timing so that they can be connected to the average, and adjust the next time (step A4). Next, the robot management means 11 starts the robot 12 (step A5). Then robot 1
2 connects to the WWW server (step A6) and reports the time taken at this time to the robot management means 11 (step A7).

FIG. 4 shows the robot 1 started in the flow of FIG.
2 is the flow when trying to connect to the next URL,
The robot management means 11 inquires of the URL management means 13 about the next URL to be searched (step B1). Here, the URL management unit 13 accesses the URL information storage unit 22 and determines whether or not processing has been completed for all URLs in the search range (step B2). As a result,
If there is no URL to be searched next, the process waits until the currently operating robot 12 ends. If the URL exists, the final acquisition time is acquired, and it is determined whether or not the acquisition interval has elapsed (step B3). If the acquisition interval has not yet passed, the robot management unit 11 inquires the URL management unit 13 of a URL to be searched next.
If it has passed, the robot management unit 11 accesses the robot information storage unit 21, acquires the next connection time of the other robot 12 and the time required for connection, and checks whether the connection timing does not overlap (step B4). If they overlap, calculate the timing so that you can connect to the average,
The next time is adjusted (step B5). Next, the robot management means 11 starts the robot (step B6). Thereafter, the robot 12 connects to the WWW server (step B).
7) The time taken at this time is transmitted to the robot management means 11 (step B8). This flow is repeated until there is no more URL to search.

Step A4 in FIG. 3 and Step B5 in FIG.
The method for adjusting the time is described in more detail using an example.

FIG. 5 is a sequence chart showing the operation of the conventional robot and the robot of the present invention. In FIG. 5, the operations of a plurality of robots are displayed on a time axis. The black portion indicates the time during which the robot 12 is moving. The gray area represents the time all robots are moving. In parallel operation of conventional robots, there is no consideration of the timing when another robot starts up, so the startup time may be concentrated, and multiple robots move in the same time zone and the robot does not move at all (Fig. 5
White part) may increase.

In the present invention, since the activation time is adjusted so that a plurality of robots 12 are dispersed, the number of robots 12 that move in parallel is reduced, and the time during which no robot 12 moves is shortened.

The robot information storage unit 21 stores data as shown in FIG. Server name (WWW1, WWW
2, WWW3, WWW4, WWW5, WWW6), and the IDs (0001, 0001,
0002,0003,0004,0005,000
6), next start time (October 11: 24; 5)
5, 10/10 11:24; 56, 10/10 1
1:25; 05, 10/101 1:25; 07, 10 /
10 11:24; 58, 10/10 11: 24: 5
7), the last time it took to connect to the server (5,
6, 5, 3, 6, 5) are described for each server.

When accessing the same server, the robot 12 has a predetermined interval. In this example, it is assumed that the connection interval to the WWW server is 15 seconds. Since the connection interval to the server is constant, unless the robot 12 is newly started up or the URL of a certain server disappears and the robot 12 is not terminated, the connection is made on the same schedule every 15 seconds. Therefore, in the present invention, a method of sequentially reviewing the scheduling for the connection interval of 15 seconds when the robot is started and when the next URL is accessed is adopted. In FIG. 7, the horizontal axis indicates time, and each robot 12 is activated and the connection time is extracted from the robot information storage unit 21 and displayed as shown in FIG. The time filled in black is the time when the robot is connected to the target server. The number of connections per unit time is calculated using this figure. Here, the unit time is 1 second. Since the interval between the dotted lines corresponds to the unit time, it is sufficient to calculate how many robots 12 are activated when viewed vertically at this interval. For example, in the first second, only the robot 1 is moving, so the total is 1, and in the next 1 second, the robot 1
And the robot 2 is moving, so the total is 2. In this way, the number of activations for all unit times for 15 seconds is calculated.

The connection time of the robot to be started next is acquired. If the robot 12 for the server has already been started, the connection time described in the robot information storage unit 21 is used, and if the robot 12 is newly started to connect to a new server, the current start is started. The average of the connection times of the robots 12 that are connected is defined as the connection time. FIG.
In the example of (5), the average of the connection time is calculated by referring to the robot information storage unit 21 of FIG. 6, and (5 + 6 + 5 + 3 + 6 + 5) / 6
= 5 seconds. Therefore, the total sum of the robots activated in the unit time from the current time to 5 seconds after the current time is obtained. In this example, the sum from the present to 5 seconds later is 1 + 2 + 3 + 4 +
4 = 14. 2 + 3 + 4 + for 5 seconds from the next 1 second
4 + 3 = 16. In this way, the sum is calculated by shifting the unit time. If this value is small, it means that no load is applied, so the smallest point and a point close to the current time are searched. In this example, since 4 is the smallest, the robot is started from this time and is described in the robot information storage unit 21.

Next, another embodiment of the present invention will be described.
This will be described in detail with reference to FIG.

FIG. 8 is a sequence chart when the connection interval to the server in the embodiment of the Internet information search system of the present invention is adjusted.

In FIG. 8, when the number of robots increases and the number of connections per unit time increases, the robot 1 within the connection interval
Even if the startup time of 2 is adjusted, a load will be applied. Therefore, the upper limit of the number of connections within the connection interval is determined, and if the robot 12 is activated more than that, the connection interval to the WWW server is further adjusted so that no load is applied.

It is to be noted that a computer-readable storage medium storing a processing program for the Internet information search system as described above may be executed by a computer.

[0036]

As described above, the first effect of the Internet information search system of the present invention is that the robot adjusts the time for accessing the next URL.
There is no access at the same time. As a result, the load on the CPU and the network is reduced.

The reason is that the robot access time is managed and adjusted by the robot management means.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an embodiment of an Internet information search system according to the present invention.

FIG. 2 is a block diagram showing an embodiment of the Internet information search system of the present invention.

FIG. 3 is a flowchart showing the operation of the Internet information search system according to the present invention when the robot is started.

FIG. 4 is a flowchart showing the operation of the Internet information search system of the present invention when connecting to the next URL.

FIG. 5 is a sequence chart showing operations of a conventional robot and a robot according to the present invention.

FIG. 6 is a configuration diagram showing the operation of an embodiment of the Internet information search system of the present invention.

FIG. 7 is a sequence chart showing a method for calculating the number of robot activations per unit time according to an embodiment of the Internet information search system of the present invention.

FIG. 8 is a sequence chart when the connection interval to the server in the embodiment of the Internet information search system according to the present invention is adjusted.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Data processing device 2 Storage device 11 Robot management means 12 Robot 13 URL management means 21 Robot information storage part 22 URL information storage part

Continuation of the front page (56) References Takehiko Shimojima, WWW server information search service "NETPLAZA", NEC Technical Report, July 30, 1996, Vol. 49, No. 7, pp. 91 to 96, Shozo Baba , WWW Server Technology, Journal of the Institute of Image Electronics Engineers of Japan, June 25, 1997, Vol. 26, No. 3, pp. 275-284 (58) Fields investigated (Int. Cl. ⁷ , DB name) G06F 17 / 30 JICST file (JOIS)

Claims (24)

(57) [Claims] An information processing apparatus includes a data processing device and a storage device for storing information. The storage device includes a robot information storage portion and a URL information storage portion. The robot information storage portion includes an ID for identifying the robot and the robot. And the time required for connection are stored, and the URL information storage unit stores the acquired W
WW server information and information including the last URL acquisition time are stored. The data processing device includes a robot management unit, a robot and a URL management unit, and the robot management unit includes a URL management unit.
An unacquired WWW server is acquired from the L management means, one robot is started for one WWW server, and the startup time of the next robot is determined by referring to the robot information storage unit.
Average connection time of currently running robots connection time
In the unit time, shift the unit time
The total sum of the robots activated at
Start the robot from a time near the current time
The robot is started by the robot management unit, acquires the target URL, notifies the robot management unit of the acquisition time of the URL, notifies the acquired URL to the URL management unit, and starts the robot. After inquiring of the next URL to the URL management means, the robot management means is inquired about the next start time, and access is made based on the time.
The L management means receives a request for the next WWW server list from the robot management means, receives a request for the next URL to be searched from the robot, and searches the URL information storage unit.
An internet information search system, wherein a result is passed to a robot management unit and a robot, a URL notification is received from the robot, and a URL information storage unit is updated or registered. 2. A data processing device and a storage device for storing information, wherein the storage device includes a robot information storage portion and a URL information storage portion, wherein the robot information storage portion includes an ID for identifying the robot and the robot. And the time required for connection are stored, and the URL information storage unit stores the acquired W
WW server information and information including the last URL acquisition time are stored. The data processing device includes a robot management unit, a robot and a URL management unit, and the robot management unit includes a URL management unit.
An unacquired WWW server is acquired from the L management means, one robot is started for one WWW server, and the startup time of the next robot is determined by referring to the robot information storage unit.
Average connection time of currently running robots connection time
In the unit time, shift the unit time
The total sum of the robots activated at
Start the robot from a time near the current time
The robot is started by the robot management unit, acquires the target URL, notifies the robot management unit of the acquisition time of the URL, notifies the acquired URL to the URL management unit, and starts the robot. After inquiring of the next URL to the URL management means, the robot management means is inquired about the next start time, and access is made based on the time.
The L management means receives a request for the next WWW server list from the robot management means, receives a request for the next URL to be searched from the robot, and searches the URL information storage unit.
An Internet information search method, wherein a result is passed to a robot management unit and a robot, a URL notification is received from the robot, and a URL information storage unit is updated or registered. 3. A data processing device and a storage device for storing information, wherein the storage device includes a robot information storage portion and a URL information storage portion, wherein the robot information storage portion includes an ID for identifying the robot and the robot. And the time required for connection are stored, and the URL information storage unit stores the acquired W
WW server information and information including the last URL acquisition time are stored. The data processing device includes a robot management unit, a robot and a URL management unit, and the robot management unit includes a URL management unit.
An unacquired WWW server is acquired from the L management means, one robot is started for one WWW server, and the startup time of the next robot is determined by referring to the robot information storage unit.
Average connection time of currently running robots connection time
In the unit time, shift the unit time
The total sum of the robots activated at
Start the robot from a time near the current time
The robot is started by the robot management unit, acquires the target URL, notifies the robot management unit of the acquisition time of the URL, notifies the acquired URL to the URL management unit, and starts the robot. After inquiring of the next URL to the URL management means, the robot management means is inquired about the next start time, and access is made based on the time.
The L management means receives a request for the next WWW server list from the robot management means, receives a request for the next URL to be searched from the robot, and searches the URL information storage unit.
The result is passed to the robot management means and the robot, a notification of the URL from the robot is received, and a program for causing the computer to execute an Internet information search process for updating or registering the URL information storage unit is recorded. recoding media. 4. The robot management unit inquires of the URL management unit about the name of the next WWW server to be searched, and the URL management unit accesses the URL information storage unit to determine whether all the WWW servers within the search range have been processed. Judgment, if there is no WWW server to search for, wait for the currently operating robot to end, the robot management means accesses the robot information storage unit, and the next connection time of other robots and the time required for connection , Check if the connection timings do not overlap, and if they overlap, calculate the timing so that the connection can be averaged, adjust the next time, the robot management means starts the robot, and the robot connects to the WWW server Internet information search system characterized by transmitting the time taken to robot management means . 5. The robot management means inquires of the URL management means about the name of the next WWW server to be searched, and the URL management means accesses the URL information storage unit to determine whether or not processing has been completed for all WWW servers in the search range. Judgment, if there is no WWW server to search for, wait for the currently operating robot to end, the robot management means accesses the robot information storage unit, and the next connection time of other robots and the time required for connection , Check if the connection timings do not overlap, and if they overlap, calculate the timing so that the connection can be averaged, adjust the next time, the robot management means starts the robot, and the robot connects to the WWW server And transmitting the time taken to the robot management means. 6. The robot management means inquires of the URL management means about the name of the next WWW server to be searched, and the URL management means accesses the URL information storage unit to determine whether or not processing has been completed for all WWW servers in the search range. Judgment, if there is no WWW server to search for, wait for the currently operating robot to end, the robot management means accesses the robot information storage unit, and the next connection time of other robots and the time required for connection , Check if the connection timings do not overlap, and if they overlap, calculate the timing so that the connection can be averaged, adjust the next time, the robot management means starts the robot, and the robot connects to the WWW server Computer to perform Internet information search processing that tells the robot management means how much time was spent A recording medium characterized by recording a program for causing the recording medium to be executed. 7. The robot management means queries the URL management means for a next URL to be searched, and the URL management means determines
The RL information storage unit is accessed, and it is determined whether or not processing has been completed for all URLs in the search range. If there is no URL to be searched next, the system waits for the currently operating robot to end. If there,
The final acquisition time is acquired, it is determined whether or not the acquisition interval has elapsed, and if the acquisition interval has not yet passed, the robot management unit queries the URL management unit for the next URL to be searched. The robot management means accesses the robot information storage unit, obtains the next connection time of the other robot and the time required for connection, checks whether the connection timings do not overlap, and if so, determines the timing that can be connected on average. An Internet information search system, wherein the time is calculated, the next time is adjusted, the robot management means starts the robot, the robot is connected to the WWW server, and the time taken is reported to the robot management means. 8. The robot management means inquires of the URL management means about the next URL to be searched,
The RL information storage unit is accessed, and it is determined whether or not processing has been completed for all URLs in the search range. If there is no URL to be searched next, the system waits for the currently operating robot to end. If there,
The final acquisition time is acquired, it is determined whether or not the acquisition interval has elapsed, and if the acquisition interval has not yet passed, the robot management unit queries the URL management unit for the next URL to be searched. The robot management means accesses the robot information storage unit, obtains the next connection time of the other robot and the time required for connection, checks whether the connection timings do not overlap, and if so, determines the timing that can be connected on average. An Internet information search method comprising calculating and adjusting the next time, the robot management means activating the robot, connecting the robot to a WWW server, and transmitting the time taken to the robot management means. 9. The robot management means queries the URL management means for the next URL to be searched, and the URL management means
The RL information storage unit is accessed, and it is determined whether or not processing has been completed for all URLs in the search range. If there is no URL to be searched next, the system waits for the currently operating robot to end. If there,
The final acquisition time is acquired, it is determined whether or not the acquisition interval has elapsed, and if the acquisition interval has not yet passed, the robot management unit queries the URL management unit for the next URL to be searched. The robot management means accesses the robot information storage unit, obtains the next connection time of the other robot and the time required for connection, checks whether the connection timings do not overlap, and if so, determines the timing that can be connected on average. Calculate and adjust the next time, the robot management means activates the robot, the robot connects to the WWW server, and records a program for causing the computer to execute an Internet information search process of transmitting the time taken to the robot management means. A recording medium characterized in that: 10. A server information, a server name, an ID of a robot connected to the server, a next startup time, and a time required for connecting to the server last time are described in the robot information storage unit for each server. At a certain interval when accessing the server, if a new robot is started or the URL of a certain server is lost and the robot is not terminated, the connection is made at the same interval at regular intervals.
A new robot is started, or a server
When the URL is gone and the robot ends
Connect the average of the connection time of the currently running robot
The unit time is shifted by the unit time within the unit time
Find the sum of the robots activated at time, and the sum is small
And start the robot from the time near the current time.
An Internet information search system characterized in that it is adjusted to start . 11. The robot information storage unit describes, for each server, a server name, an ID of a robot connected to the server, a next start time, and a time required to connect to the server last time. At a certain interval when accessing the server, if a new robot is started or the URL of a certain server is lost and the robot is not terminated, the connection is made at the same schedule at regular intervals.
A new robot is started, or a server
When the URL is gone and the robot ends
Connect the average of the connection time of the currently running robot
The unit time is shifted by the unit time within the unit time
Find the sum of the robots activated at time, and the sum is small
And start the robot from the time near the current time.
An Internet information search method, wherein the method is adjusted to start . 12. The robot information storage unit describes a server name, an ID of a robot connected to the server, a next startup time, and a time required for connecting to the server last time for each server. At a certain interval when accessing the server, if a new robot is started or the URL of a certain server is lost and the robot is not terminated, the connection is made at the same schedule at regular intervals.
A new robot is started, or a server
When the URL is gone and the robot ends
Connect the average of the connection time of the currently running robot
The unit time is shifted by the unit time within the unit time
Find the sum of the robots activated at time, and the sum is small
And start the robot from the time near the current time.
A recording medium on which is recorded a program for causing a computer to execute an Internet information search process for adjusting to start . 13. A data processing device and a storage device for storing information, wherein the storage device includes a robot information storage unit and a URL.
An information storage unit, wherein the robot information storage unit stores an ID for identifying the robot, a startup time of the robot, and a time required for connection, and the URL information storage unit stores the acquired information of the WWW server and the final URL of the URL. The data processing device includes a robot management unit, a robot and a URL management unit, and the robot management unit stores information including an acquisition time.
An unacquired WWW server is acquired from the RL management means, one robot is started for one WWW server, and the upper limit of the number of connections within the connection interval is determined by referring to the robot information storage unit. If the robot is activated, WW
The next robot is started up by adjusting the connection interval to the W server, the robot is started by the robot management means, acquires the target URL, notifies the robot management means of the acquisition time of this URL, and sends the acquired URL to the robot management means. After being notified, the URL management means is inquired about the next URL to the URL management means. After that, the robot management means is inquired about the next start time, and the robot management means is accessed according to the next start time. Upon receiving a request for a WWW server list, receiving a request for the next URL to be searched from the robot, searching the URL information storage unit, passing the result to the robot management means and the robot,
An Internet information search system, which receives an RL notification and updates or registers a URL information storage unit. 14. A data processing device and a storage device for storing information, wherein the storage device includes a robot information storage unit and a URL.
An information storage unit, wherein the robot information storage unit stores an ID for identifying the robot, a startup time of the robot, and a time required for connection, and the URL information storage unit stores the acquired information of the WWW server and the final URL of the URL. The data processing device includes a robot management unit, a robot and a URL management unit, and the robot management unit stores information including an acquisition time.
An unacquired WWW server is acquired from the RL management means, one robot is started for one WWW server, and the upper limit of the number of connections within the connection interval is determined by referring to the robot information storage unit. If the robot is activated, WW
The next robot is started up by adjusting the connection interval to the W server, the robot is started by the robot management means, acquires the target URL, notifies the robot management means of the acquisition time of this URL, and sends the acquired URL to the robot management means. After being notified, the URL management means is inquired about the next URL to the URL management means. After that, the robot management means is inquired about the next start time, and the robot management means is accessed according to the next start time. Upon receiving a request for a WWW server list, receiving a request for the next URL to be searched from the robot, searching the URL information storage unit, passing the result to the robot management means and the robot,
An Internet information search method, comprising: receiving an RL notification and updating or registering a URL information storage unit. 15. A data processing device and a storage device for storing information, wherein the storage device includes a robot information storage unit and a URL.
An information storage unit, wherein the robot information storage unit stores an ID for identifying the robot, a startup time of the robot, and a time required for connection, and the URL information storage unit stores the acquired information of the WWW server and the final URL of the URL. The data processing device includes a robot management unit, a robot and a URL management unit, and the robot management unit stores information including an acquisition time.
An unacquired WWW server is acquired from the RL management means, one robot is started for one WWW server, and the upper limit of the number of connections within the connection interval is determined by referring to the robot information storage unit. If the robot is activated, WW
The next robot is started up by adjusting the connection interval to the W server, the robot is started by the robot management means, acquires the target URL, notifies the robot management means of the acquisition time of this URL, and sends the acquired URL to the robot management means. After being notified, the URL management means is inquired about the next URL to the URL management means. After that, the robot management means is inquired about the next start time, and the robot management means is accessed according to the next start time. Upon receiving a request for a WWW server list, receiving a request for the next URL to be searched from the robot, searching the URL information storage unit, passing the result to the robot management means and the robot,
A recording medium on which a program for receiving a notification of an RL and causing a computer to execute an Internet information search process for updating or registering a URL information storage unit is recorded. 16. The robot management means inquires of the URL management means about the name of the next WWW server to be searched, and the URL management means accesses the URL information storage unit to determine whether or not processing has been completed for all WWW servers in the search range. Judgment, if there is no WWW server to search for, wait for the currently operating robot to end, the robot management means accesses the robot information storage unit, and the next connection time of other robots and the time required for connection And get
Check if the connection timings do not overlap, and if they do, calculate the timing so that you can connect on average,
The upper limit of the number of connections within the connection interval is determined, and if the robot is started more than that, the next robot is started by adjusting the connection interval to the WWW server, the robot management means starts the robot, and the robot An Internet information search system, wherein the Internet information search system is connected to a WWW server and notifies the robot management means of the time taken. 17. The robot management unit inquires of the URL management unit about the name of the next WWW server to be searched, and the URL management unit accesses the URL information storage unit to determine whether or not processing has been completed for all WWW servers in the search range. Judgment, if there is no WWW server to search for, wait for the currently operating robot to end, the robot management means accesses the robot information storage unit, and the next connection time of other robots and the time required for connection And get
Check if the connection timings do not overlap, and if they do, calculate the timing so that you can connect on average,
The upper limit of the number of connections within the connection interval is determined, and if the robot is started more than that, the next robot is started by adjusting the connection interval to the WWW server, the robot management means starts the robot, and the robot A method for searching for Internet information, comprising connecting to a WWW server and transmitting the time taken to robot management means. 18. The robot management unit inquires of the URL management unit about the name of the next WWW server to be searched, and the URL management unit accesses the URL information storage unit to determine whether all the WWW servers in the search range have been processed. Judgment, if there is no WWW server to search for, wait for the currently operating robot to end, the robot management means accesses the robot information storage unit, and the next connection time of other robots and the time required for connection And get
Check if the connection timings do not overlap, and if they do, calculate the timing so that you can connect on average,
The upper limit of the number of connections within the connection interval is determined, and if the robot is started more than that, the next robot is started by adjusting the connection interval to the WWW server, the robot management means starts the robot, and the robot A recording medium having recorded thereon a program for causing a computer to execute an Internet information search process for connecting to a WWW server and transmitting the time taken to robot management means. 19. The robot management means inquires of the URL management means about the next URL to be searched, and the URL management means accesses the URL information storage unit to access the URL in the search range.
It is determined whether or not all the processes have been completed. If there is no URL to be searched next, the system waits for the currently operating robot to end. If the URL exists, obtains the final acquisition time, and acquires the acquisition interval. To determine if has elapsed, and if it has not passed the acquisition interval,
The robot management means queries the URL management means for the next URL to be searched, and if it has passed, the robot management means accesses the robot information storage unit to obtain the next connection time of another robot and the time required for connection, Check if the connection timings do not overlap. If they do overlap, calculate the timing so that the connection can be averaged, determine the upper limit of the number of connections within the connection interval, and if the robot is activated more than that, go to the WWW server An Internet information search system, wherein the next robot is started up by adjusting the connection interval of the robot, the robot management means activates the robot, the robot connects to the WWW server, and reports the time taken to the robot management means. 20. The robot management means inquires of the URL management means about the next URL to be searched, and the URL management means accesses the URL information storage unit and sets the URL in the search range.
It is determined whether or not all the processes have been completed. If there is no URL to be searched next, the system waits for the currently operating robot to end. If there is a URL, the final acquisition time is acquired, and the acquisition interval is obtained. To determine if has elapsed, and if it has not passed the acquisition interval,
The robot management means queries the URL management means for the next URL to be searched, and if it has passed, the robot management means accesses the robot information storage unit to obtain the next connection time of another robot and the time required for connection, Check if the connection timings do not overlap. If they do overlap, calculate the timing so that the connection can be averaged, determine the upper limit of the number of connections within the connection interval, and if the robot is activated more than that, go to the WWW server A method for searching for Internet information, characterized in that the connection interval is adjusted to start the next robot, the robot management means starts the robot, the robot connects to the WWW server, and reports the time taken to the robot management means. 21. The robot management means inquires of the URL management means about a next URL to be searched, and the URL management means accesses the URL information storage unit and sets a URL in a search range.
It is determined whether or not all the processes have been completed. If there is no URL to be searched next, the system waits for the currently operating robot to end. If the URL exists, obtains the final acquisition time, and acquires the acquisition interval. To determine if has elapsed, and if it has not passed the acquisition interval,
The robot management means queries the URL management means for the next URL to be searched, and if it has passed, the robot management means accesses the robot information storage unit to obtain the next connection time of another robot and the time required for connection, Check if the connection timings do not overlap. If they do overlap, calculate the timing so that the connection can be averaged, determine the upper limit of the number of connections within the connection interval, and if the robot is activated more than that, go to the WWW server The next robot is started up by adjusting the connection interval of the robot, the robot management means starts the robot, the robot connects to the WWW server, and the computer executes an Internet information search process for transmitting the time taken to the robot management means. A recording medium characterized by recording the above program. 22. The robot information storage unit describes the server name, the ID of the robot connected to the server, the next start time, and the time required for connecting to the server last time for each server, and the robot has the same name. When accessing the server, the upper limit of the number of connections within the connection interval is determined, and when the robot is activated beyond that limit, the connection interval to the WWW server is adjusted and the startup connection of the next robot is performed. Internet information search system. 23. The server name, the ID of the robot connected to the server, the next start time, and the time required to connect to the server last time are described for each server in the robot information storage unit. When accessing the server, the upper limit of the number of connections within the connection interval is determined, and when the robot is activated beyond that limit, the connection interval to the WWW server is adjusted and the startup connection of the next robot is performed. Internet information search method. 24. In the robot information storage unit, a server name, an ID of a robot connected to the server, a next startup time, and a time required to connect to the server last time are described for each server, and When accessing the server, determine the upper limit of the number of connections within the connection interval, and if the robot is activated beyond that, adjust the connection interval to the WWW server and start the next robot for Internet connection search A recording medium on which a program for causing a computer to execute processing is recorded.