WO2001090943A2

WO2001090943A2 - Distributed internet content delivery system

Info

Publication number: WO2001090943A2
Application number: PCT/US2001/014905
Authority: WO
Inventors: Mark E. Kriegsman
Original assignee: Clearway Technologies, Llc
Priority date: 2000-05-19
Filing date: 2001-05-09
Publication date: 2001-11-29
Also published as: GB2379307B; GB2379307A; AU2001261293A1; DE10196218T1; JP2004515834A; WO2001090943A3; GB0227022D0

Abstract

A distributed web serving system for delivering content from a primary server to a client includes a database accessible to the client. The database includes information from which the client can itself identify a secondary server from which to obtain a copy of content from the primary server. The system also includes a processor, other than the primary server, for delivering the content to the client. As a result, the primary server does not need to intervene in a client's initial request for content.

Description

DISTRIBUTED WEB SERVING SYSTEM

FIELD OF THE INVENTION

This invention relates to web serving systems, and in particular, to distributed web-serving systems.

BACKGROUND

A crippling problem in conventional content delivery systems on the Internet is the sheer volume of data traffic being tunneled into a limited number of content- providing primary servers. To some extent, this difficulty has been ameliorated by deploying distributed systems of secondary content servers that, to some extent, replicate the content available on the primary servers. By requesting content from one of these secondary servers instead of from the primary server, a browser competes with less data traffic and is therefore more likely to receive prompt service.

However, a system of secondary server is useful only to the extent that a browser is aware of its existence. In known distributed web serving systems, this is achieved by having the primary server provide, in response to a client's request for content, data indicative of a secondary web server from which that content can be obtained.

Although the foregoing method reduces the response time required to service each requesting client, it does not address the fact that the primary server must still intercede in every initial request for content. Instead, this conventional method merely changes the nature ofthe primary server's response to the requesting client from a time-consuming download of content to instructions to seek some or all ofthe content elsewhere.

From the requesting client's point of view, the conventional method requires two network-accessing events. In the first network accessing event, the requesting client asks the primary server where to obtain the content that it seeks. Then, in the second network- accessing event, the requesting client connects to the secondary server specified in the first network-accessing event. This inefficiency is in many ways analogous to telephoning a department store to obtain information on goods sold by a particular department, obtaining a telephone number for that department, hanging up, and then placing a new telephone call to that department.

SUMMARY OF THE INVENTION

The system and method ofthe invention overcomes the foregoing disadvantage by providing a database to the requesting client. From information contained in the database, the client can identify an optimal secondary server from which it can request the desired content with an expectation of rapid content delivery. By empowering the client with this ability, the system dispenses with the primary server's need to intercede in each requests for content by providing to the client, inter alia., a pointer to a secondary server.

A system according to the invention includes at least one secondary server in communication with a primary server for receiving content from the primary server and for transmitting that content, upon request, to a client. The system also includes a database that is both remote from the primary server and accessible to the client. This database contains information leading to the desired content. The system also includes a processor, other than the primary server, for transmitting the desired content to the client.

In one embodiment, the database includes information that leads to a secondary server that contains a copy ofthe content. Examples of such information include a list of secondary servers that can be interrogated in order to determine whether they contain the desired information or a list of secondary servers together with the content stored by each of those secondary servers.

To reduce the processing load on the primary server, the database is typically a local database, stored, for example, on the client on a proxy server, or on a secondary server.

The system can also include a management server for providing the database and, if necessary, for updating the database in response to changes in the list of secondary servers or changes in the list of contents stored on the secondary servers. W

These and other features ofthe invention will be apparent from a review ofthe following detailed description and the accompanying figures in which:

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows the architecture for a distributed web serving system incoφorating the principles ofthe invention;

FIG. 2 shows a method implemented on the system in FIG. 1 for delivering desired content to a client; and

FIG. 3 shows an alternate method implemented on the system in FIG. 1 for delivering desired content to a client.

DETAILED DESCRIPTION

Referring to FIG. 1, a distributed web serving system 10 for delivering content to a client 12 includes a primary server 14 that contains an original copy ofthe content. The primary server 14 is in data communication with a plurality of secondary servers 16a-n, each of which can, but need not necessarily, have a copy of all or part ofthe original content stored on the primary server 14. The distribution of content from the primary server 14 to selected secondary servers 16a-n is under the control of a management server 18 that is in data communication with both the secondary servers 16a-n and the primary server 14.

When a client 12 retrieves a web page, that web page can contain embedded objects. In some cases, some portions ofthe web page are retrieved from a primary server 14 while other portions are retrieved from secondary servers 16a-n. The term "content" is intended to include particular objects embedded in a web page and to include the web page as a whole.

The management server 18 determines which, if any, ofthe secondary servers 16a-n is to receive particular content stored on the primary server 14. If the management server 18 determines that a particular secondary server 16b is to have a copy of particular content, the management server 18 causes the replication of that content and its transmission from the primary server 14 to the particular secondary server 16b. If the content on the primary server 14 changes, the management server 18 also causes an updated copy of that content to be provided to the appropriate secondary servers 16a-n.

By allowing the client 12 to retrieve content from one of several secondary servers 16a-n rather than funneling all such requests to the primary server 14, the replication of content as described above carries with it the potential of reducing the load on the primary server 14. This allows the primary server 14 to operate with reduced bandwidth requirements. In order to realize this advantage, however, the client 12 must have some way of knowing that the content is available from a location other than the primary server 14.

As shown in FIG. 1, both the client 12 and the management server 18 are in communication with a database 20. This database 20 is generally stored locally, for example at the client 12, or on a proxy server. Alternatively, the database can be stored at the management server 18. The database can also be stored in a distributed fashion, with some of its content residing on the management server 18 and the remainder of its content residing at the client 12. What is important is that the information contained in the database 20 be stored at one or more locations in a manner that does not require the intervention ofthe primary server 14 in a client's initial request for content.

The database 20 contains information that leads the client 12 to the content of interest. This information can include the IP addresses ofthe secondary servers, thereby allowing the client 12 to bypass the DNS system. This information is updated by the management server 18 when necessary. The client 12 executes interface software 21 for interacting with the daltabase in a manner described in connection with FIGS. 2 and 3.

In a first embodiment, the database 20 includes a list of secondary servers 16a-n together with the content stored on each secondary server in that list. The database 20 further includes information indicating the geographical relationship between each secondary server and the client 12, and optionally, the topological relationship between each secondary server 16a-n and the client 12. From this information, the client 12 can determine the secondary server 16a-c that would most likely minimize the time required for retrieval of content from that secondary server. The client can then interrogate that secondary server to determine whether it contains the desired content.

Referring now to FIG. 2, in this first embodiment, setup software executed either by the client or by the management server, ranks the secondary servers in an order that represents the expected service time required for receiving content from that server (step 22). In response to a request to retrieve data from the primary server, interfacing software 21 executing on the client initializes a counter (step 24) and searches the database to determine whether the primary server has stored the desired content on the secondary server identified by the current value ofthe counter (step 26). If it has, the client transmits a message to that secondary server to request a copy ofthe desired content (step 28). Otherwise, the client checks to see if there are additional secondary servers (step 30). If there are, the client increments the counter (step 32) and repeats the foregoing steps with a secondary server identified by the new value ofthe counter. Otherwise, the client transmits a message to the primary server requesting a copy ofthe desired content (step 34).

In a second embodiment ofthe invention, the database 20 includes only a list of secondary servers 16a-n. The database 20 further includes information indicating the geographical relationship between each secondary server and the client 12, and optionally, the. topological relationship between each secondary server 16a-n and the client 12. From this information, the client 12 can determine the secondary server 16a- that would most likely minimize the time required for retrieval of content.

Referring now to FIG.3, in this second embodiment, setup software executed either by the client or by the management server ranks the secondary servers in an order that represents the expected service time required for the client to receive content from that server (step 36). In response to a request to retrieve data from the primary server, interfacing software executing on the client initializes a counter (step 38) and interrogates the server identified by that counter (step 40) to determine whether the desired content has been stored on that secondary server. The client then determines, from the secondary server's response, whether that server has a copy ofthe desired content (step 42). If it does, then the client transmits a message to that secondary server to request a copy ofthe desired content (step 44). If that secondary server does not have a copy ofthe desired content, the client checks to see if there are additional secondary servers available for interrogation (step 46). If there are additional secondary servers available for interrogation, the client increments the counter (step 48) and repeats the foregoing steps with a secondary server identified by the new value of the counter. Otherwise, the client transmits a message to the primary server requesting a copy ofthe desired content (step 50) directly from the primary server.

An optional feature ofthe distributed web-serving system 10 is fault-tolerant software 23 for correcting errors in the client's choice ofthe best secondary server from which to retrieve desired content. It may be, for example, that the database 20 stored on the client 12 is obsolete, or that it contains erroneous information. Under these circumstances, a client 12 will attempt to obtain desired content from an incorrect secondary server, such as a non-optimal secondary-server or an optimal secondary-server that lacks a copy ofthe desired content.

The error-correction software 23, which typically executes on the incorrect secondary server, recognizes the client's error and determines the client's identity. Knowing the client's identity, the error-correction software 23 uses another copy ofthe database 20 to identify the best available secondary server for satisfying the client's request. Essentially, what the error-correction software 23 does is to repeat the client's original, but erroneous, determination of an optimal secondary-server. To the extent that the incorrect secondary server has access to a database that does not include the erroneous or obsolete information, the error-correction software 23 will correctly identify the optimal secondary-server for the client. The incorrect secondary-server can then return the identity, either the domain name or the IP address, ofthe optimal secondary- server to the client 12. Alternatively, the incorrect secondary-server can itself retrieve the desired content from the optimal secondary-server and provide that content to the client 12.

An optional feature ofthe distributed web serving system 10 is error correction software 23 for correcting errors in the client's choice ofthe best secondary server from which to retrieve desired content. It may be, for example, that the database 20 stored on the client 12 is obsolete, or that it contains erroneous information. Under these circumstances, a client 12 will attempt to obtain desired content from an incorrect server, such as a non-optimal secondary server or an optimal secondary server that lacks a copy ofthe desired content.

Error correction software 23 executing on the incorrect secondary server recognizes the client's error, determines the client's identity, and uses it, together with information on its own local database, to identify the best available secondary server for satisfying the client's request. Essentially, what the incorrect secondary server does is to repeat the client's original, but erroneous, determination of an optimal secondary server. To the extent that the incorrect secondary server has access to a database that does not include the erroneous or obsolete information, the incorrect secondary server will correctly identify the best available secondary server for the client. The incorrect secondary server can then return the identity ofthe optimal secondary server to the client. Alternatively, the incorrect secondary server can itself retrieve the desired content from the optimal secondary server and provide the desired content to the client.

It is apparent that in the method and system ofthe invention, the processing overhead incurred by the primary server 14 is reduced because the primary server 14 does not need to intercede in each request for content. In particular, it is not necessary for the primary server 14 to determine, for each request for content, the best secondary server 16b to deliver that content to the client 12. Instead, the method and system ofthe invention empower the client 12 itself to determine the best secondary server 16b from which to obtain desired content. This is achieved by maintaining a database 20 containing

In the first embodiment, the processing that is no longer performed by the primary server 14 is carried out by the client 12. To support the processing by the client 12, the management server 18 provides considerable information in the database 20 and frequently updates that information. In the second embodiment, the same processing is split between the client 14 and the secondary servers 16a-n who respond to the client's interrogation. In this second embodiment, the database 20 accessible to the client 14 does not need to list the content available on each secondary server 16a-n. Accordingly, in this second embodiment, it is not necessary for the management server 18 to create a large database 20 and to update it frequently.

In this second embodiment, the database 20 needs to be provided to the client 12 by the management server 18 only once, during a brief set-up procedure. Typically, the first time a client 12 attempts to retrieve content from the primary server 14, the primary server 14 will send the client 12 a message offering it the opportunity to receive the database in order to obtain documents from the primary server 14 more quickly the next time the client 12 has occasion to do so. If the client 12 declines the offer, the primary server 14 either delivers content to the client in a conventional manner, either directly or by providing the client 12 with a pointer to a secondary server, for example, server 16b, from which that content is available. Otherwise, the primary server 14 causes the management server 18 to provide a copy ofthe database 20 to the client 12 for use by the client 12 in subsequent attempts to retrieve content from the primary server 14.

Although the foregoing description is of an embodiment in which the client attends to the task of identifying a suitable secondary server, this task can also be performed by another processor acting on the client's behalf, providing that other processor is not the primary server. For example, the management server, or a proxy server having access to the database can carry out this task on the client's behalf. As discussed above in connection with the error-correction software, a secondary server can also perform this function on behalf of the client. What is common to these cases is that a processor other than the primary server performs this task and that in all these cases, the primary server need not intervene in a client's initial request for content.

The foregoing embodiments are representative of systems and methods in which the subject matter ofthe invention is made manifest. However, these embodiments, being but examples ofthe manner in which the invention can be implemented, do not represent the entire scope ofthe invention. Instead, what is claimed as new and secured by letters patent is:

Claims

1. A system for delivering content from a primary sever to a client, the system comprising:

a secondary server in communication with said primary server for receiving said content therefrom and in communication with said client for transmitting said content thereto;

a database accessible to said client, said database containing information leading to said content; and

a processor other than the primary server for accessing the database to identify an optimal secondary server for transmitting said content to the client.

2. The system of claim 1 wherein the processor is selected from a group consisting of said client, said secondary server, a proxy server, or a management server.

3. The system of claim 1 wherein said information leading to said content comprises information leading to a secondary server to be interrogated to determine whether said secondary server contains a copy of said content stored thereon.

4. The system of claim 1 wherein said information leading to said content comprises a list of content available on said secondary server.

5. The system of claim 1 wherein said database is a database local to said client.

6. The system of claim 5 wherein said database is stored on said client.

7. The system of claim 5 wherein said database is stored on a proxy server.

8. The system of claim 1 further comprising a management server for providing said database accessible to said client.

9. The system of claim 1 further comprising a management server for updating said database.

10. The system of claim 1 further comprising error-correction software for executing on an incorrectly identified secondary server.

11. The system of claim 10 wherein said error-correction software includes instructions for determining a correct secondary-server and providing to said client information leading to said correct secondary-server.

12. The system of claim 10 wherein said error-correction software includes instructions for determining a correct secondary-server, retrieving said content from said correct secondary-server, and providing said content to said client.

13. A system for delivering content from a primary server to a client, said system comprising;

a plurality of secondary servers in communication with said primary server, a subset of said plurality of secondary servers containing said content stored thereon; and

a database accessible to said client, said database containing information leading to said content.

14. The system of claim 13 wherein said information leading to said content comprises information leading to a secondary server to be interrogated to determine whether said secondary server contains a copy of said content stored thereon.

15. The system of claim 13 wherein said information leading to said content comprises a list of content available on said secondary server. ^•

16. A method for delivering content from a primary server to a client, the method comprising:

storing a copy of said content on a secondary server in communication with said client; and updating a database accessible to said client with information leading to said content on said secondary server. . The method of claim 16 wherein updating said database comprises adding to said database information leading to said secondary server. . The method of claim 16 wherein updating said database comprises adding to said database information indicating content available on said secondary server. . The method of claim 16 further comprising storing said database on said client. . The method of claim 16 further comprising storing said database on a proxy server. . The method of claim 16 further comprising providing said database accessible to said client. . The method of claim 16 further comprising updating said database. . Computer-readable media having encoded thereon software for delivering content from a primary server to a client, said software comprising instructions for:

storing a copy of said content on a secondary server in communication with said client; and

updating a database accessible to said client with information leading to said content. . The computer-readable media of claim 23 wherein said instructions for updating said database comprise instructions for adding information leading to said secondary server to said database. . The computer-readable media of claim 23 wherein said instructions for updating said database comprise instructions for adding to said database information indicating content available on said secondary server. . The computer-readable media of claim 23 wherein said software further comprises instructions for storing said database on said client.

27. The computer-readable media of claim 23 wherein said software further comprises instructions for storing said database on a proxy server.

28. The computer-readable media of claim 23 wherein said software further comprises instructions for providing said database accessible to said client.

29. The computer-readable media of claim 23 wherein said software further comprises instructions for updating said database.

30. A computer-readable medium having encoded thereon software for enabling a client to retrieve desired content from a secondary server, said software including instructions for from a primary server to a client, said software comprising instructions for:

directing said client to a database containing information leading to said content on said secondary server; and

retrieving said content from said secondary server.

31. The computer-readable medium having of claim 30 further comprising said database containing information leading to said content on said secondary server.

32. The computer-readable medium of claim 31 wherein said information leading to said content comprises information leading to a secondary server to be interrogated to determine whether said secondary server contains of copy of said content stored thereon.

33. The computer-readable medium of claim 31 wherein said information leading to said content comprises a list of content available on said secondary-server.

34. The computer-readable medium of claim 30 wherein said software further comprises instructions for providing access to said database by a management server for updating said database.

35. A method for retrieving desired content from a secondary server, said method comprising:

consulting a database containing information leading to said desired content, said information including a plurality of secondary servers;

selecting, from said plurality of secondary servers an optimal secondary server having a copy of said desired content stored thereon; and

retrieving said desired content from said optimal secondary-server.

36. The method of claim 35 wherein selecting an optimal secondary server comprises identifying, on the basis of information from said database, an expected service time for providing content to said client from said optimal secondary server.

37. The method of claim 35 wherein selecting an optimal secondary server comprises identifying a secondary server having access to said desired content.

38. The method of claim 37 wherein identifying a secondary server having access to said desired content comprises determining, on the basis of information from said database, whether a secondary server has access to said desired content.

39. The method of claim 37 wherein identifying a secondary server having access to said desired content comprises interrogating a secondary server to determine whether said secondary server has access to said desired content.

40. The method of claim 35 further comprising providing a management server with access to said database for updating said database.

41. A computer readable medium having encoded thereon software for retrieving desired content from a secondary server, said software comprising:

means for consulting a database containing information leading to said desired content, said information including a plurality of secondary servers; means for selecting, from said plurality of secondary servers, an optimal secondary server having a copy of said desired content stored thereon; and

means for retrieving said desired content from said optimal secondary-server.

42. The method of claim 41 wherein said means for selecting an optimal secondary server comprises means for identifying, on the basis of information from said database, an expected service time for providing content to said client from said optimal secondary server.

43. The method of claim 41 wherein said means for selecting an optimal secondary server comprises means for identifying a secondary server having access to said desired content.

44. The method of claim 43 wherein identifying a secondary server having access to said desired content comprises determining, on the basis of information from said database, whether a secondary server has access to said desired content.

45. The method of claim 43 wherein identifying a secondary server having access to said desired content comprises interrogating a secondary server to determine whether said secondary server has access to said desired content.

46. The method of claim 41 further comprising providing a management server with access to said database for updating said database.

47. A system for delivering content from a primary server to a client, said system comprising:

serving means for communicating with said primary server for receiving said content therefrom and for communicating with said client for transmitting said content thereto; and

storage means for storing information leading to said content; and means for accessing said storage means for identifying an optimal serving means for transmitting said content to said client.

48. The system of claim 47 wherein said information leading to said content comprises information leading to a communication means to be interrogated to determine whether said communication means has access to a copy of said content stored thereon.

49. The system of claim 47 wherein said information leading to said content comprises a list of content available on said serving means.

50. The system of claim 47 further comprising means for providing said storage means to said client.

51. The system of claim 47 further comprising means for updating said storage means.