TWI856829B - Load distribution device and method - Google Patents

Abstract

A load distribution device communicatively connects to servers and an end device, wherein the end device is configured to access a first server of the servers. The load distribution device is configured to receive a resource request from the end device. In response to receiving the resource request from the end device, the load distribution device is further configured to determine whether to redirect the end device to a second server of the servers based on an evaluation result. In response to determining to redirect the end device to the second server, the load distribution device is further configured to transmit a resource response to the end device, wherein the resource response corresponds to the second server.

Description

Load distribution device and method

The present disclosure relates to a load distribution device and method, and more particularly to a load distribution device and method for a content delivery network server.

Generally speaking, after receiving the URL of a specific server in the Content Delivery Network (CDN), the front-end player in the network streaming system will continue to download the required streaming files from the same server during the current playback operation. However, if a large number of users access a single server, the server will be overloaded and the streaming service quality will be affected.

In addition, the operating costs of different servers may be different. If traffic is concentrated on servers with higher costs, higher operating costs may also be incurred.

In view of this, how to dynamically adjust the server load in the content delivery network is a goal that the industry urgently needs to work on.

To solve the above problem, the present disclosure provides a load distribution device, including a transceiver interface and a processor. The transceiver interface is communicatively connected to a plurality of servers and a terminal device. The terminal device obtains a first sub-file of a plurality of streaming files from a first server among the servers. The processor is coupled to the transceiver interface and is used to execute the following steps: receiving a resource request from the terminal device; in response to receiving the resource request from the terminal device, determining whether to redirect the terminal device to a second server among the servers based on an evaluation result; and in response to determining to redirect the terminal device to the second server, transmitting a resource response to the terminal device to instruct the terminal device to obtain a second sub-file of the streaming files from the second server.

The present disclosure also provides a load distribution method applicable to an electronic device, wherein the electronic device is communicatively connected to a plurality of servers and a terminal device, and the terminal device obtains a first sub-file of a plurality of streaming files from a first server among the servers. The load distribution method includes: receiving a resource request from the terminal device; in response to receiving the resource request from the terminal device, determining whether to redirect the terminal device to a second server among the servers based on an evaluation result; and in response to determining to redirect the terminal device to the second server, transmitting a resource response to the terminal device to instruct the terminal device to obtain a second sub-file of the streaming files from the second server.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are intended to provide further explanation of the disclosure as claimed.

In order to make the description of the present disclosure more detailed and complete, reference may be made to the attached drawings and various embodiments described below, in which the same numbers in the drawings represent the same or similar elements.

Please refer to FIG. 1, which is a schematic diagram of a load distribution device 1 in a first embodiment of the present disclosure. The load distribution device 1 includes a processor 12 and a transceiver interface 14, wherein the processor 12 is coupled to the transceiver interface 14, and the transceiver interface 14 is communicatively connected to the terminal device E and the servers S1 to Sn.

In some embodiments, the processor 12 may include a central processing unit (CPU), a multi-processor, a distributed processing system, an application specific integrated circuit (ASIC), and/or a suitable computing unit.

The transceiver interface 14 can be connected to the terminal device E and the servers S1 to Sn by wired or wireless communication. For example, the transceiver interface 14 can include one or more interfaces of different communication protocols, such as: Universal Serial Bus (USB), Bluetooth interface, Ethernet interface, Wi-Fi network interface and/or other communication interfaces. In some embodiments, the transceiver interface 14 is connected to the terminal device E and the servers S1 to Sn by Internet communication.

In some embodiments, the load distribution device 1 is a network service server (e.g., a video streaming service server). Servers S1 to Sn are servers in a content delivery network (e.g., a content delivery network server) for providing network services (e.g., video streaming services) to terminal device E, where n can be a positive integer not less than 2. Terminal device E can send a service request to the load distribution device 1, and the load distribution device 1 selects at least one of the servers S1 to Sn based on the service request and generates a corresponding network address (Uniform Resource Locator, URL).

For example, when a user wants to access a specific streaming video service, he can send an access request to the server of the streaming video provider (i.e., load distribution device 1) through terminal device E. Furthermore, if load distribution device 1 selects server S1 to provide services to terminal device E, it sends a network address for connecting to server S1 to terminal device E, so that terminal device E can access streaming video and other services from server S1 through the network address.

Based on efficiency and other considerations, the parent file corresponding to the same service in the network streaming service (for example, a movie file) is cut into multiple smaller sub-files, and an index file is created, where the index file records the multiple sub-files corresponding to the parent file and the order of the sub-files. For example, the index file records that the video parent file is cut into five files named M1 to M5, and the playback order is M1, M2, M3, M4 and M5. In other words, the user's terminal device E can first obtain the M1 file and play it, and download M2 and the remaining files while playing, without having to wait for the complete parent file to be downloaded before watching the video. In this way, when the terminal device E accesses the streaming service, it can first obtain the index file, and then access the corresponding sub-files from the server in sequence according to the content of the index file.

In some embodiments, the servers S1 to Sn all store the same file. For example, the servers S1 to Sn all store sub-files M1, M2, M3, M4, and M5.

It should be noted that, based on the content delivery network architecture, servers S1 to Sn store files required for providing services (such as the aforementioned index files and sub-files), so the load distribution device 1 can select one of the servers S1 to Sn to provide services to the terminal device E based on factors such as the load of the servers S1 to Sn, connection efficiency, and operating costs.

When the terminal device E accesses the service provided by the load distribution device 1, the terminal device E accesses one of the servers S1 to Sn (for example, server S1) to obtain one of the plurality of streaming files (for example, sub-file M1). As for how the load distribution device 1 dynamically adjusts the load of the servers S1 to Sn, please refer to the following embodiment.

First, the processor 12 of the load distribution device 1 receives a resource request from the terminal device E. Then, in response to receiving the resource request from the terminal device E, the processor 12 determines whether to redirect the terminal device E to a second server among the servers S1 to Sn based on an evaluation result. Finally, in response to determining to redirect the terminal device E to the second server, the processor 12 sends a resource response to the terminal device E to instruct the terminal device E to obtain other files (e.g., sub-file M2) in the streaming files from the second server.

For example, the terminal device E may periodically (e.g., every 5 minutes) send a resource request to the load distribution device 1. After the load distribution device 1 receives the resource request, the processor 12 may generate an evaluation result corresponding to one of the servers S1 to Sn based on the operating status of the servers S1 to Sn (e.g., the current load status) (e.g., the processor 12 determines that the server S2 currently has the lowest load, then the evaluation result corresponds to the server S2). Finally, if the processor 12 determines based on the evaluation result that the terminal device E needs to be redirected, then the resource response of the server corresponding to the evaluation result (e.g., server S2) is sent to the terminal device E to redirect the terminal device E to another server. Furthermore, after receiving the resource response, the terminal device E will redirect to the server corresponding to the resource response to access the resource (for example, download a file).

In this way, the load distribution device 1 can periodically determine whether to redirect the terminal device E to other servers, so as to dynamically adjust the load of the servers S1 to Sn. For the specific details of the above operation, please refer to the following paragraphs.

In some embodiments, when accessing a streaming service, the terminal device E may first obtain the sub-file M1 from the server S1, and then before obtaining the sub-file M2, the terminal device E may send a resource request to the load distribution device 1 to confirm the object (i.e., one of the servers S1 to Sn) to continue downloading the sub-file M2.

In some implementations, the terminal device E sends a resource request to the load distribution device 1 based on the request rules established by the load distribution device 1. Specifically, the request rules established by the load distribution device 1 may include conditions such as the environment network status, video delay status, sub-file transmission status, request frequency, etc.

In some implementations, the load distribution device 1 may also transmit the request rule to the terminal device E to update the request rule stored in the terminal device E. For example, when the terminal device E determines that the environment network status is not good, the terminal device E transmits a resource request to the load distribution device 1 based on the request rule.

In some embodiments, the processor 12 of the load distribution device 1 is further used to receive a plurality of operating states from the servers S1 to Sn, wherein each of the operating states corresponds to one of the servers S1 to Sn. Specifically, the processor 12 can receive the corresponding operating states from the servers S1 to Sn through the transceiver interface 14, and the operating states can include the current traffic and load capacity of the server (i.e., the maximum allowable value of the server traffic).

Next, the processor 12 calculates a plurality of load indicators corresponding to the servers based on the operating states. For example, the processor 12 may divide the current traffic of each of the servers S1 to Sn by the load capacity to obtain the load indicators of each of the servers S1 to Sn. In this example, the load indicator is used to represent the utilization rate of the server traffic (for example, if the load indicator is 100%, it means that the corresponding server is fully loaded).

In some embodiments, the processor 12 may also calculate a load index corresponding to the weight of each of the servers S1 to Sn. For example, if the traffic cost of server S2 is twice the traffic cost of server S1, the weight of server S2 may be set to be twice the weight of server S1. In this way, the load index may also reflect the difference in operating costs between the servers.

In some embodiments, the weights are generated by the processor 12 based on the calculation of the unit traffic cost of each server. In some embodiments, the higher the unit traffic cost of the server, the higher the weight.

Finally, the processor 12 generates an evaluation result based on the load indicator.

In some embodiments, the processor 12 may sort the load indicators corresponding to the servers S1 to Sn, and select the server corresponding to the lowest load indicator (e.g., server S1) as the evaluation result to redirect the terminal device E to the server with the lowest load indicator, thereby directing the terminal device E to a server with a lower current load, a lower traffic cost, or a server that is more suitable to take over the traffic after evaluating multiple factors.

The present disclosure also provides another technology for generating evaluation results, wherein the load distribution device may also include a memory (not shown in the figure), which is used to store the historical operating status of the corresponding servers S1 to Sn, wherein the historical operating status includes multiple historical loads of the servers S1 to Sn corresponding to multiple time points.

In some embodiments, the storage may include semiconductor or solid-state memory, magnetic tape, removable computer disk, random access memory (RAM), read-only memory (ROM), hard disk, and/or optical disk.

For example, the historical load stored in the storage may include the past operation status of the servers S1 to Sn, such as past traffic, load capacity, and corresponding time points. The historical load stored in the memory may also include connection records of past services provided by servers S1 to Sn. For example, the connection records may include user identification (user ID), service type (e.g., live broadcast, pre-stored video), connection start time, connection end time, connection type (e.g., 4G network, 5G network, or Wi-Fi), connected server (e.g., server S1), traffic bandwidth (e.g., 4.5Mbps), Internet Protocol Address, user geographic location (e.g., North District, Central District, or South District), and/or Internet Service Provider (ISP).

For details on the operation of generating evaluation results, please refer to the following example.

First, the processor 12 establishes a plurality of load models corresponding to the servers S1 to Sn based on the historical operation states, wherein the load models include a plurality of corresponding relationships between the time points corresponding to the servers S1 to Sn and the historical loads.

In some embodiments, the processor 12 may train a machine learning model based on the historical operation states and use the trained machine learning model as a load model. For example, the processor 12 may cluster the historical operation states using a clustering model (e.g., a k-means model) to obtain correlations between server traffic and time, user geographic location, connection type, and/or other factors.

Finally, the processor 12 generates the evaluation result based on the load models. Specifically, the processor 12 generates the evaluation result based on the corresponding relationship between the traffic and other factors summarized by the trained load model.

In one example, if the load model concludes that between 9:00 a.m. and 12:00 p.m. every Saturday, the server S2 is about twice as large as the server S1, then the processor 12 can direct the terminal device E to the server S1 (i.e., the processor 12 sends a resource response directed to the server S1 to the terminal device E) based on the load model between 9:00 a.m. and 12:00 p.m. on Saturdays when the traffic usage rate of the server S1 (i.e., the ratio obtained by dividing the current traffic by the load capacity) is less than 50%. In this way, the load distribution device 1 can reduce the load difference between the servers S1 and S2 without causing the traffic to be too concentrated on one of them, and the judgment of the traffic usage rate of the server S1 is introduced so that the load distribution device 1 will not excessively direct the traffic to the server S1.

In another example, if the load model summarizes that when the terminal devices of users whose geographical location is located in the North District accessed the service in the past, a proportion exceeding a threshold value (e.g., 80%) was directed to server S2. At this time, based on the existing mechanism of directing traffic in the content delivery network (e.g., directing the terminal devices to the server with faster response), the load distribution device 1 can determine that server S2 is more suitable for providing services to users in the North District than other servers. Therefore, the processor 12 can direct the terminal devices of users whose geographical location is located in the North District to server S2 based on the load model when the traffic usage rate of server S2 is less than 50%. In this way, the load distribution device 1 can determine the server that is better than the terminal device based on past records, improve the connection efficiency, and reduce the burden of real-time system calculations because the load model has been established in advance. In addition, in some embodiments, since the load model can accurately guide traffic and reduce the probability of redirecting the terminal device E to other servers, the terminal device E can also reduce the frequency of the aforementioned regular transmission of resource requests to reduce the burden on the load distribution device 1.

In another example, if the load model concludes that user A often (e.g., with a probability of more than 90%) accesses streaming videos from server S1 via a Wi-Fi network on Saturday nights, based on the existing traffic diversion mechanism of the content delivery network (e.g., diverting the terminal device to a server with a faster response), the load distribution device 1 can determine that server S1 is more suitable for providing services to user A than other servers under the aforementioned conditions. Therefore, the processor 12 can, based on the load model, direct the terminal device used by user A to server S1 when user A accesses streaming videos via a Wi-Fi network on Saturday nights and the traffic usage rate of server S1 is less than 50%. In this way, similar to the above example, the load distribution device 1 can determine the server that is better than the terminal device based on past records, improve the connection efficiency, and reduce the burden of real-time system calculations because the load model has been established in advance. In addition, in some embodiments, since the load model can accurately guide traffic and reduce the probability of redirecting the terminal device E to other servers, the terminal device E can also reduce the frequency of the aforementioned regular transmission of resource requests to reduce the burden on the load distribution device 1.

It should be noted that the above example is used as an example for the sake of convenience. In fact, other load distribution mechanisms can be generated according to the load model trained by different historical operating states.

In this way, the terminal device E can periodically confirm the server accessing resources from the load distribution device 1 and request resources from the corresponding server. The load distribution device 1 can distribute the load of each of the servers S1 to Sn according to the past operation records of each of the servers S1 to Sn. In addition, by establishing a load model, the load distribution device 1 can reduce the probability and/or number of redirections to the terminal device E. Furthermore, it can also reduce the number of times the terminal device E sends resource requests and the number of times the load distribution device 1 sends resource responses to reduce the load of the load distribution device 1.

In some embodiments, the terminal device E periodically transmits the resource request based on a request rule generated by the load distribution device 1. For example, before the terminal device E accesses a service from the load distribution device 1, the load distribution device 1 transmits the request rule to the terminal device E, for example, the application installed in the terminal device E includes the request rule. Thus, when accessing a service, the terminal device E can periodically transmit the resource request to the load distribution device 1 according to the request rule, so that the load distribution device 1 can redirect the request to the terminal device E when necessary to adjust the load of the servers S1 to Sn.

Furthermore, in some embodiments, the request rule includes determining whether the terminal device sends the resource request to the load distribution device based on at least one of an access duration, a transmission flow, and an access file quantity of the terminal device accessing the server.

Specifically, the request rule may record that every certain time interval (e.g., 5 minutes), every time a certain amount of traffic is transmitted (e.g., 200MB), and/or every time a certain number of files are accessed (e.g., 5 files), the terminal device E must send the resource request to the load distribution device 1 to confirm with the load distribution device 1 the object (server) of future file access.

In some embodiments, the processor 12 of the load distribution device 1 can also sort the servers S1 to Sn based on a plurality of traffic costs of the servers S1 to Sn to generate a sequence; and generate the evaluation result based on the sequence, wherein the evaluation result corresponds to a lowest cost server among the servers S1 to Sn.

For example, load distribution device 1 preferentially directs terminal device E to the server with the lowest traffic cost. If the load of the server with the lowest traffic cost exceeds a threshold value (e.g., 80%), terminal device E is redirected to other servers (e.g., the server with the second lowest traffic cost).

In summary, the terminal device E can periodically confirm the server for accessing resources from the load distribution device 1, and request resources from the corresponding server (for example, downloading a video file). The load distribution device 1 can dynamically distribute the load of each of the servers S1 to Sn according to factors such as the current traffic and traffic cost of each of the servers S1 to Sn. In addition, through the operation of the load distribution device 1 to periodically receive status information from the servers S1 to Sn, when a server becomes abnormal and cannot operate, the load distribution device 1 can also confirm the server's situation at the first time and stop directing the terminal device E to the abnormal server.

Please further refer to FIG. 2, which is a flow chart of a load distribution method 20 in the second embodiment of the present disclosure. The load distribution method 20 is applicable to an electronic device (e.g., load distribution device 1), wherein the electronic device is communicatively connected to a plurality of servers (e.g., servers S1 to Sn) and a terminal device (e.g., terminal device E). The load distribution method 20 includes steps S201 to S203.

In step S201, the electronic device receives a resource request from the terminal device.

In step S202, in response to receiving the resource request from the terminal device, the electronic device determines whether to redirect the terminal device to a second server among the servers based on an evaluation result.

In step S203, in response to determining to redirect the terminal device to the second server, the electronic device sends a resource response to the terminal device to instruct the terminal device to obtain a second sub-file of the streaming files from the second server.

In some embodiments, the load distribution method 20 further includes receiving a plurality of operating states from the servers, wherein each of the operating states corresponds to one of the servers; calculating a plurality of load indicators corresponding to the servers based on the operating states; and generating the evaluation result based on the load indicators.

In some embodiments, the operating states include a plurality of flows and a plurality of load capacities of the servers; the operation of calculating the load indicators corresponding to the servers further includes calculating the load indicators corresponding to the servers based on the flows and the load capacities corresponding to the servers; and the operation of generating the evaluation result further includes sorting the load indicators to generate a sequence, and generating the evaluation result based on the sequence.

In some embodiments, the operation of calculating the load indicators corresponding to the servers further includes calculating the load indicators corresponding to the servers based on a plurality of weights.

In some embodiments, the load distribution device further includes a memory for storing a plurality of historical operating states corresponding to a plurality of servers, wherein the historical operating states include a plurality of historical loads of the servers corresponding to a plurality of time points, and the load distribution method 20 further includes establishing a plurality of load models corresponding to the servers based on the historical operating states, wherein the load models include a plurality of corresponding relationships between the time points corresponding to the servers and the historical loads; and generating the evaluation results based on the load models.

In some embodiments, the historical operating states further include the historical loads of the servers corresponding to a plurality of geographical locations, and the load models include the corresponding relationships between the geographical locations corresponding to the servers and the historical loads.

In some embodiments, the historical operating states further include the historical loads of the servers corresponding to a plurality of connection types, and the load models include the corresponding relationships between the connection types corresponding to the servers and the historical loads.

In some embodiments, the terminal device periodically sends the resource request based on a request rule generated by the load distribution device.

In some embodiments, the request rule includes determining whether the terminal device sends the resource request to the load distribution device based on at least one of an access duration, a transmission flow, and an access file quantity of the terminal device accessing the first server.

In some embodiments, the load distribution method 20 further includes sorting the servers based on a plurality of traffic costs of the servers to generate a sequence; and generating the evaluation result based on the sequence, wherein the evaluation result corresponds to a lowest cost server among the servers.

In some embodiments, the resource response includes a network address connected to the access server, and the network address is used to connect to the access server to obtain at least one file corresponding to the resource request.

In this way, the terminal device can periodically confirm the server accessing resources from the electronic device, and request resources from the corresponding server (for example, downloading a video file). The load distribution method 20 can dynamically distribute the load of each of the servers according to factors such as the current traffic and traffic cost of each of the servers. In addition, through the operation of the electronic device regularly receiving status from the servers, when a server is abnormal and cannot operate, the load distribution method 20 can also confirm the server situation at the first time and stop directing the terminal device to the abnormal server.

Although several embodiments are described in detail above as examples, the load distribution device and method proposed in the present disclosure can also be implemented by other systems, hardware, software, storage media or their combination. Therefore, the protection scope of the present disclosure should not be limited to the specific implementation method described in the embodiments of the present disclosure, but should be defined by the scope of the attached patent application.

It is obvious to those with ordinary knowledge in the art to which the present disclosure belongs that various modifications and changes can be made to the structure of the present disclosure without departing from the scope or spirit of the present disclosure. In view of the foregoing, the protection scope of the present disclosure also covers modifications and changes made within the scope of the attached patent application.

1: Load distribution device

12: Processor

14: Transceiver interface

S1~Sn: Server

E: Terminal device

20: Load distribution method

S201~S203: Steps

In order to make the above and other purposes, features, advantages and embodiments of the present disclosure more clearly understandable, the attached drawings are described as follows: Figure 1 is a schematic diagram of a load distribution device in a first embodiment of the present disclosure; and Figure 2 is a flow chart of a load distribution method in a second embodiment of the present disclosure.

Domestic storage information (please note in the order of storage institution, date, and number) None Foreign storage information (please note in the order of storage country, institution, date, and number) None

1: Load distribution device

12: Processor

14: Transceiver interface

S1~Sn: Server

E: Terminal device

Claims (7)

A load distribution device includes: a transceiver interface, which is communicatively connected to a plurality of servers and a terminal device, wherein the terminal device obtains a first sub-file of a plurality of streaming files from a first server among the servers; and a processor, which is coupled to the transceiver interface and is used to perform the following operations: receiving a plurality of operating states from the servers, wherein each of the operating states corresponds to one of the servers; based on the operating states, calculating a plurality of load indicators corresponding to the servers; generating an evaluation result based on the load indicators; receiving a resource request from the terminal device; and in response to receiving the resource request from the terminal device, determining whether to load the terminal device based on the evaluation result. redirecting the terminal device to a second server among the servers; and in response to determining to redirect the terminal device to the second server, transmitting a resource response to the terminal device to instruct the terminal device to obtain a second sub-file of the streaming files from the second server; wherein the operation states include a plurality of flows and a plurality of load capacities of the servers; the operation of calculating the load indicators corresponding to the servers further includes calculating the load indicators corresponding to the servers based on the flows and the load capacities corresponding to the servers; and the operation of generating the evaluation result further includes sorting the load indicators to generate a sequence, and generating the evaluation result based on the sequence. The load distribution device as described in claim 1, wherein the operation of calculating the load indicators corresponding to the servers further includes calculating the load indicators corresponding to the servers based on a plurality of weights. A load distribution device as described in claim 1, wherein the terminal device periodically sends the resource request based on a request rule generated by the load distribution device. The load distribution device as described in claim 3, wherein the request rule includes at least one of an access duration, a transmission flow, and an access file quantity of the terminal device to access the first server to determine whether the terminal device sends the resource request to the load distribution device. A load distribution device includes: a transceiver interface, which is communicatively connected to a plurality of servers and a terminal device, wherein the terminal device obtains a first sub-file of a plurality of streaming files from a first server among the servers; and a processor, coupled to the transceiver interface, for performing the following operations: sorting the servers based on a plurality of traffic costs of the servers to generate a sequence; and generating an evaluation result based on the sequence, wherein the evaluation result corresponds to A lowest cost server among the servers; receiving a resource request from the terminal device; in response to receiving the resource request from the terminal device, determining whether to redirect the terminal device to a second server among the servers based on the evaluation result; and in response to determining to redirect the terminal device to the second server, transmitting a resource response to the terminal device to instruct the terminal device to obtain a second sub-file of the streaming files from the second server. A load distribution device as described in claim 1 or 5, wherein the resource response includes a network address connected to the access server, and the network address is used to connect to the access server to obtain at least one file corresponding to the resource request. A load distribution method is applied to an electronic device, wherein the electronic device is communicatively connected to a plurality of servers and a terminal device, and the terminal device obtains a first sub-file of a plurality of streaming files from a first server among the servers. The load distribution method comprises: receiving a plurality of operating states from the servers, wherein each of the operating states corresponds to one of the servers; based on the operating states, calculating a plurality of load indicators corresponding to the servers; generating an evaluation result based on the load indicators; receiving a resource request from the terminal device; and in response to receiving the resource request from the terminal device, determining whether to re-use the terminal device based on the evaluation result. Directing to a second server among the servers; and in response to determining to redirect the terminal device to the second server, transmitting a resource response to the terminal device to instruct the terminal device to obtain a second sub-file of the streaming files from the second server; wherein the operation states include a plurality of flows and a plurality of load capacities of the servers; the operation of calculating the load indicators corresponding to the servers further includes calculating the load indicators corresponding to the servers based on the flows and the load capacities corresponding to the servers; and the operation of generating the evaluation result further includes sorting the load indicators to generate a sequence, and generating the evaluation result based on the sequence.

Images