CN110912926A - Data resource back-source method and device - Google Patents

Abstract

The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for returning data resources to a source. The method comprises the following steps: and when the target server does not store the data resources requested by the user, sending back the source request to the upstream server through a plurality of data transmission protocols. And after receiving data resources and source returning information returned by the upstream server through each data transmission protocol, calling a machine learning model to calculate the protocol weight of the data transmission protocol. And calculating the source returning speed, the priority and the protocol weight of each data transmission protocol to obtain each transmission value, and feeding back the data resource returned by the data transmission protocol with the maximum transmission value to the user. By applying the method, the data resources required by the user can be obtained by sending a source return request once through a plurality of data transmission protocols, and the data resources returned by the data transmission protocol with the maximum transmission value are fed back to the user according to the transmission value of each data transmission protocol, so that the consumption of network resources is reduced, and the integrity of the data resources is ensured.

Description

Data resource back-source method and device

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for returning data resources to a source.

Background

In a Content Delivery Network (CDN) system, there are multiple cache servers that store data resources, and each server stores a corresponding data resource according to the local heat required for each data resource. When a user needs to acquire a data resource, the data resource can be requested from a local cache server, and if the local cache server does not store the data resource requested by the user, the data resource requested by the user is acquired by returning the data resource to the source, that is, the data request is initiated to other cache servers to acquire the feedback of storing the data resource to the user.

When other cache servers request the back source of the data resource, the back source overtime or the data of the returned data resource part are missed, and each time the back source overtime or the data are missed, the back source request is usually required to be sent to the cache server of the data resource required by each cache user again. However, each time the data resource is re-sourced, the network resource is consumed again, which increases the cost of sourcing the data resource.

Disclosure of Invention

In view of this, the present invention provides a method for returning data resources back to a source, by which a request for returning data resources back to an upstream server can be simultaneously sent through multiple data transmission protocols, so as to increase the success rate of returning data resources back to the source, and under the condition that the upstream server successfully returns data resources, the data resources returned by an optimal data transmission protocol are selected and fed back to a user, thereby not only reducing the consumption of network resources, but also ensuring the integrity of the data resources.

The invention also provides a data resource source returning device which is used for ensuring the realization and the application of the method in practice.

A data resource back-to-source method, comprising:

when detecting that a user sends a resource request to a target server, acquiring resource information contained in the resource request;

judging whether the target server stores the data resource corresponding to the resource information or not;

if the target server does not store the data resources corresponding to the resource information, acquiring each return source address pre-stored by the target server, wherein the return source address is a server address of an upstream server pre-selected to perform data return source;

determining each data transmission protocol supported by each upstream server according to each back source address;

respectively applying each data transmission protocol supported by each upstream server, and sending a source returning request corresponding to the resource information to the upstream server so that the upstream server responds to the source returning request sent by each data transmission protocol;

receiving data resources and source returning information returned by each upstream server through each data transmission protocol, and calculating the source returning speed of each data transmission protocol for returning the data resources;

inputting the source returning information returned by each data transmission protocol into a machine learning model which is trained in advance, and triggering the machine learning model to output the protocol weight of each data transmission protocol;

determining the priority of each data transmission protocol, and calculating the priority, the source return speed and the protocol weight of each data transmission protocol to obtain a transmission value of each data transmission protocol;

and determining the data transmission protocol with the maximum transmission value as a target transmission protocol, and feeding back the data resource returned by the target transmission protocol to the user.

Optionally, the method for determining, according to each back-source address, each data transmission protocol supported by each upstream server includes:

analyzing the back source address to obtain server information corresponding to the upstream server contained in the back source address;

and acquiring each protocol identifier contained in the server information, and determining a data transmission protocol matched with each protocol identifier.

Optionally, the method for calculating a return-to-source speed of each data transmission protocol return data resource includes:

for each data transmission protocol, calculating the back-to-source time of the data transmission protocol for returning data resources after the upstream server responds to the back-to-source request;

determining the byte size of the data resource returned by the data transmission protocol;

and calculating the byte size and the source returning time of the data resource returned by the data transmission protocol to obtain the source returning speed of the data resource returned by the data transmission protocol.

Optionally, the receiving the data resource and the source return information returned by each upstream server through each data transmission protocol includes:

when receiving the data resource and the source returning information returned by the first data transmission protocol, starting a preset timer to time;

and receiving the data resources and the source returning information returned by each data transmission protocol of each upstream server within preset timing time.

The above method, optionally, further includes:

and in the timing time, when a response failure message returned by the data transmission protocol is received or the data transmission protocol returns data resources and source returning information is not received, disconnecting the connection with the data transmission protocol.

Optionally, the method, after determining the data transmission protocol with the largest transmission value as the target transmission protocol, further includes:

and storing the data resources and the source returning information of the target transmission protocol to a preset storage area, and deleting the data resources and the source returning information sent by all the data transmission protocols except the target transmission protocol.

The method, optionally, includes a training process of the machine learning model, including:

when the server time of the target server reaches the end-of-day processing time, acquiring historical source returning information pre-stored in the target server and a protocol speed corresponding to each piece of historical source returning information, wherein the protocol speed is the speed of sending a source returning request to the upstream server by a corresponding data transmission protocol and acquiring data resources;

sequentially inputting the historical source returning information into the machine learning model so as to enable the machine learning model to carry out model training until model parameters of the machine learning model meet preset training conditions;

when each piece of historical source returning information is input into the machine learning model, a training result of the historical source returning information currently input into the machine learning model is obtained; calling a preset loss function, and calculating the training result and the protocol speed corresponding to the current historical source returning information input into the machine learning model to obtain a loss function value; judging whether the model parameters of the machine learning model meet the training conditions or not according to the loss function values; if not, adjusting the model parameters of the machine learning model according to the loss function values; and if so, obtaining the machine learning model which is trained.

A data resource back-source device, comprising:

the system comprises a first acquisition unit, a second acquisition unit and a resource management unit, wherein the first acquisition unit is used for acquiring resource information contained in a resource request when detecting that a user sends the resource request to a target server;

the judging unit is used for judging whether the target server stores the data resource corresponding to the resource information;

a second obtaining unit, configured to obtain, if the target server does not store the data resource corresponding to the resource information, each source returning address pre-stored by the target server, where the source returning address is a server address of an upstream server that is pre-selected to perform data source returning;

a determining unit, configured to determine, according to each return-to-source address, each data transmission protocol supported by each upstream server;

a sending unit, configured to apply each data transmission protocol supported by each upstream server, respectively, and send a source returning request corresponding to the resource information to the upstream server, so that the upstream server responds to the source returning request sent by each data transmission protocol;

the first calculating unit is used for receiving the data resources and the source returning information returned by each upstream server through each data transmission protocol and calculating the source returning speed of each data transmission protocol for returning the data resources;

the triggering unit is used for inputting the source returning information returned by each data transmission protocol into a machine learning model which is trained in advance, and triggering the machine learning model to output the protocol weight of each data transmission protocol;

the second calculation unit is used for determining the priority of each data transmission protocol, calculating the priority, the source return speed and the protocol weight of each data transmission protocol and obtaining the transmission value of each data transmission protocol;

and the feedback unit is used for determining the data transmission protocol with the maximum transmission value as a target transmission protocol and feeding back the data resource returned by the target transmission protocol to the user.

The above apparatus, optionally, the determining unit includes:

the analysis subunit is configured to analyze the return-to-source address to obtain server information corresponding to the upstream server included in the return-to-source address;

and the first determining subunit is used for acquiring each protocol identifier contained in the server information and determining the data transmission protocol matched with each protocol identifier.

The above apparatus, optionally, the first calculating unit includes:

the first calculating subunit is configured to calculate, for each data transmission protocol, a back-source time for the data transmission protocol to return a data resource after the upstream server responds to the back-source request;

the second determining subunit is used for determining the byte size of the data resource returned by the data transmission protocol;

and the second calculating subunit is used for calculating the byte size and the source returning time of the data resource returned by the data transmission protocol to obtain the source returning speed of the data resource returned by the data transmission protocol.

A storage medium, the storage medium comprising stored instructions, wherein when the instructions are executed, the apparatus on which the storage medium is located is controlled to execute the above data resource source returning method.

An electronic device comprising a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by the one or more processors to perform the data resource back-source method.

Compared with the prior art, the invention has the following advantages:

the invention provides a data resource back-to-source method, which comprises the following steps: when the target server does not store the data resources requested by the user, the source request is sent back to each upstream server in a mode of multiple data transmission protocols. After receiving data resources and source returning information returned by each upstream server through each data transmission protocol, calling a machine learning model to calculate the protocol weight of each data transmission protocol, determining the transmission value of each data transmission protocol by calculating the source returning speed, the priority and the protocol weight of each data transmission protocol, and feeding back the data resources corresponding to the data transmission protocol with the maximum transmission value to a user. By applying the method, the data resources required by the user can be obtained only by sending a source returning request to the upstream server once through the plurality of data transmission protocols, the source returning speed, the priority and the weight of each data transmission protocol are calculated to obtain the transmission value of each data transmission protocol, and the data resources returned by the data transmission protocol with the maximum transmission value are selected from each data transmission protocol and fed back to the user, so that the consumption of network resources is reduced, and the integrity of the data resources is ensured.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a flowchart of a method for returning a data resource to a source according to an embodiment of the present invention;

fig. 2 is a flowchart of another method of a data resource source returning method according to an embodiment of the present invention;

fig. 3 is a flowchart of another method of a data resource source returning method according to an embodiment of the present invention;

fig. 4 is a block diagram of an apparatus for returning data resources to a source device according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In this application, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions, and the terms "comprises", "comprising", or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The invention is operational with numerous general purpose or special purpose computing device environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multi-processor apparatus, distributed computing environments that include any of the above devices or equipment, and the like.

An embodiment of the present invention provides a data resource source returning method, which may be applied to multiple system platforms, where an execution subject of the method may be a computer terminal or a processor of various mobile devices, and a flowchart of the method is shown in fig. 1, and specifically includes:

s101: when detecting that a user sends a resource request to a target server, acquiring resource information contained in the resource request;

in the embodiment of the present invention, when a user needs to obtain a certain data resource, a resource request may be sent to any one of the servers in the content delivery network CDN system through the client where the user is located. Wherein, the requested server is the target server. When detecting that the user sends a resource request to the target server, acquiring resource information contained in the resource request.

It should be noted that, in the embodiment of the present invention, the resource information may include resource information such as a resource name and a resource format.

S102: judging whether the target server stores the data resource corresponding to the resource information or not;

in the embodiment of the present invention, after the user sends the resource request to the target server, it is determined whether the target server stores the data resource corresponding to the resource information included in the resource request, that is, whether the target server can satisfy the request of the user for obtaining the data resource.

S103: if the target server does not store the data resources corresponding to the resource information, acquiring each return source address pre-stored by the target server, wherein the return source address is a server address of an upstream server pre-selected to perform data return source;

in the embodiment of the present invention, if the target server does not store the data resource corresponding to the resource information, the data resource required by the user needs to be returned to the source. The target server has stored therein a plurality of back-to-source addresses that need to be back-to-source, each back-to-source address corresponding to an upstream server. The back source address is the server address of the upstream server corresponding to the back source address.

It should be noted that the back-to-source address in the target server may be updated every day. The source-back addresses stored daily in the target server may or may not be the same. When the target server needs to process each stored back source address, the back source address of the upstream server which has failed can be deleted; if the source returning speed of the upstream server is low and does not meet the preset minimum source returning speed in the process of returning the historical data resources to the source, the source returning address of the upstream server can be deleted.

S104: determining each data transmission protocol supported by each upstream server according to each back source address;

in the embodiment of the present invention, the destination server may store back-to-source addresses of a plurality of upstream servers, and determine, according to the back-to-source addresses, respective data transmission protocols supported by each upstream server.

It should be noted that, in the embodiment of the present invention, each upstream server may support multiple data transmission protocols, where the data transmission protocol may be HTTP/TCP, HTTP/QUIC, TCP/UDP, or another proprietary transmission protocol capable of data transmission. Specifically, each upstream server supports at least one data transfer protocol.

S105: respectively applying each data transmission protocol supported by each upstream server, and sending a source returning request corresponding to the resource information to the upstream server so that the upstream server responds to the source returning request sent by each data transmission protocol;

in the embodiment of the invention, after determining the respective data transmission protocols supported by each upstream server, for each upstream server, at least one data transmission protocol supported by each upstream server is respectively applied to send a source return request to the upstream server. The upstream server supports each data transmission protocol for sending back-source requests, so that the upstream server can respond after receiving each back-source request and respectively return the data resource corresponding to each back-source request through each data transmission protocol.

It should be noted that, in the embodiment of the present invention, if the upstream server only supports one data transmission protocol, the back source request is sent through the data transmission protocol supported by the upstream server. If the upstream server supports multiple data transfer protocols, a request is sent back to the source via each data transfer protocol supported by the upstream server.

Alternatively, if the upstream server fails, after sending the back-to-source request to the upstream server, the upstream server may not be able to respond to the back-to-source request, or the upstream server returns a failure message.

S106: receiving data resources and source returning information returned by each upstream server through each data transmission protocol, and calculating the source returning speed of each data transmission protocol for returning the data resources;

in the embodiment of the invention, the upstream server receives the data resource and the source returning information which are returned to the target server by the upstream server after responding to each data transmission protocol and returning the data resource. The source return information includes a server address of the target server, a data transmission protocol for returning the data resource, return time, a resource size of the data resource, a response speed of the server for responding to the source return request, and the like. And simultaneously calculating the source returning speed of each data transmission protocol for returning the data resource.

It should be noted that the source return speed is not the same as the response speed in the source return information. The response speed is the speed of the upstream server responding after receiving the back-to-source request, and the back-to-source speed is the speed of the upstream server returning the data resource and the target server receiving the data resource after responding to the back-to-source request. Specifically, the back-to-source speed is calculated by the target server after receiving the data resource.

S107: inputting the source returning information returned by each data transmission protocol into a machine learning model which is trained in advance, and triggering the machine learning model to output the protocol weight of each data transmission protocol;

in the embodiment of the invention, the source returning information returned by each data transmission protocol from the upstream server is input into the machine learning model, and the machine learning model can train and calculate each source returning information according to the internal model parameters thereof and output the protocol weight corresponding to each data transmission protocol.

It should be noted that the machine learning model may be a neural network model, and may be trained in an iterative training manner to obtain a high-performance machine learning model, and when the source information is input into the machine learning model, the machine learning model may be enabled to output a protocol weight corresponding to a data transmission protocol.

S108: determining the priority of each data transmission protocol, and calculating the priority, the source return speed and the protocol weight of each data transmission protocol to obtain a transmission value of each data transmission protocol;

in the embodiment of the present invention, the data transmission protocols supported by each upstream server have corresponding priorities, and the priority of each data transmission protocol may be set manually, or may be set by the target server according to historical usage data. After the priority of each data transmission protocol is determined, the priority, the source return speed and the protocol weight of each data transmission protocol are calculated. Optionally, the formula for calculating the priority, the source return speed, and the protocol weight of the data transmission protocol is as follows: priority x backsource speed x protocol weight. The transmission value of each data transmission protocol is obtained by calculation.

It should be noted that, in the embodiment of the present invention, if the same upstream server supports multiple data transmission protocols, the priority of each data transmission protocol may be different or the same. If the same data transmission protocol is supported by different upstream servers, the priority of the same data transmission protocol in the two upstream servers may be different or the same. Each data transmission protocol can be uniformly set with the same priority, and different priorities can be set according to the actual situation of each upstream server.

S109: and determining the data transmission protocol with the maximum transmission value as a target transmission protocol, and feeding back the data resource returned by the target transmission protocol to the user.

In the embodiment of the invention, the data transmission protocol with the maximum transmission value is selected from all the data transmission protocols, and the data transmission protocol with the maximum transmission value is determined as the target transmission protocol. And feeding back the data resource sent by the target transmission protocol to the user.

In the method for returning data resources to the source provided by the embodiment of the invention, when a user sends a resource request to a target server, whether the target server stores the data resources requested by the user is determined. If not, determining each data transmission protocol supported by the upstream server corresponding to each source return address according to each source return address pre-stored in the target server. A request is sent back to the source server via a respective data transfer protocol. When receiving the returned data resources and the returned source information of each data transmission protocol of each upstream server, calculating the returned source speed of each data transmission protocol according to each data resource, obtaining the protocol weight of each data transmission protocol through a machine learning model and the returned source information, and then determining the priority of each data transmission protocol. The priority, the source returning speed and the protocol weight of each data transmission protocol are calculated, the transmission value corresponding to each data transmission protocol is obtained, the data transmission protocol with the maximum transmission value is determined as the target transmission protocol, and the data resources returned by the target transmission protocol are determined to be fast and complete due to the fact that the transmission value of the target transmission protocol is the maximum. And feeding back the data resource returned by the target transmission protocol to the user to complete the process of returning the data resource to the source.

Optionally, in the process of returning the data resource to the source, the data resource and the source returning information returned by the target transmission protocol may be stored, and the priority and the protocol weight of the target transmission protocol may also be stored.

Further, during the process of returning the data resource and the source information by each data transmission protocol, a failure may be returned due to other reasons such as a network failure, and at this time, the connection between the target server and the data transmission protocol is disconnected.

Based on the data resource source returning method provided by the above embodiment, the following embodiment is provided for the data resource source returning process:

the user sends a resource request to the server a, and when the data resource requested by the user is not stored in the server a, the back-source address of the upstream server B, C stored in the server in advance is obtained. It is determined that upstream server B supports HTTP/TCP and HTTP/QUIC protocols and upstream server C supports HTTP/QUIC and TCP/UDP protocols. The back-source request is sent to the upstream server B via the HTTP/TCP and HTTP/QUIC protocols, respectively, and to the upstream server C via the HTTP/QUIC and TCP/UDP protocols, respectively, wherein each data transfer protocol supported by each upstream server sends back-source requests. The upstream server B responds to the source returning request sent by the HTTP/TCP and HTTP/QUIC protocol and returns data resources and source returning information through the HTTP/TCP and the HTTP/QUIC protocol; the upstream server C also responds to the back-source requests sent by the HTTP/QUIC and TCP/UDP protocols and returns the data resources and back-source information via the HTTP/QUIC and TCP/UDP protocols. When the server A receives the upstream server B, the upstream server B returns the data resource B1 and the source returning information B1 through the HTTP/TCP protocol, and returns the data resource B2 and the source returning information B2 through the HTTP/QUIC protocol; and the upstream server C returns the data resource C1 and the source information C1 through the HTTP/QUIC protocol, and returns the data resource C2 and the source information C2 through the TCP/UDP protocol. Respectively calculating the source returning speed of each data transmission supported by the two upstream servers, inputting the source returning information B1, the source returning information B2, the source returning information C1 and the source returning information C2 into a machine learning model, respectively obtaining the protocol weight of the data transmission protocol supported by each upstream server, and determining the priority of each data transmission protocol in the corresponding upstream server. And calculating the source returning speed, the priority and the protocol weight of each data transmission protocol to obtain the transmission value of each data transmission protocol. If the transmission value of the HTTP/QUIC protocol supported by the upstream server B is maximum, the HTTP/QUIC protocol supported by the upstream server B is the target transmission protocol, and the data resource B2 is fed back to the user.

By applying the method provided by the embodiment of the invention, the data resources required by the user can be obtained only by sending the source returning request to the upstream server once through the plurality of data transmission protocols, the source returning speed, the priority and the weight of each data transmission protocol are calculated to obtain the transmission value of each data transmission protocol, and the data resources returned by the data transmission protocol with the maximum transmission value are selected from each data transmission protocol and fed back to the user, so that the consumption of network resources is reduced, and the integrity of the data resources is ensured.

In the method provided in this embodiment of the present invention, based on the content in step S104, after obtaining the source return address in the target server, according to the source return address of each upstream server, a specific process of determining each data transmission protocol supported by the service request on each upstream server may include:

analyzing the back source address to obtain server information corresponding to the upstream server contained in the back source address;

and acquiring each protocol identifier contained in the server information, and determining a data transmission protocol matched with each protocol identifier.

In the data resource source returning method provided by the embodiment of the present invention, after obtaining each source returning address pre-stored by the target server, each source returning address is analyzed to obtain server information of an upstream server corresponding to each source returning address. The server information may include information such as a server model and parameters of the upstream server. And acquiring each protocol identifier contained in each server information, wherein each protocol identifier corresponds to one data transmission protocol, and the data transmission protocol supported by the upstream server can be determined according to each protocol identifier contained in the return source address.

By applying the method provided by the embodiment of the invention, the data transmission protocol supported by each upstream server can be determined through the prestored source returning address, so that the source returning request for the data resource can be sent through each data transmission protocol.

In the method provided in the embodiment of the present invention, based on the content in step S106, after obtaining the data resource and the source returning information returned by each data transmission protocol, the source returning speed of each data transmission protocol for returning the data resource needs to be calculated, and as shown in fig. 2, the method specifically includes:

s201: for each data transmission protocol, calculating the back-to-source time of the data transmission protocol for returning data resources after the upstream server responds to the back-to-source request;

in the embodiment of the present invention, the back-to-source time of the data resources returned by each transmission protocol is calculated, where the back-to-source time of the data transmission protocol may be calculated by obtaining the response time of the upstream server in response to the back-to-source request included in the back-to-source information and the receiving time of the target server at the time of receiving the data resources returned by the data transmission protocol, so as to determine the back-to-source time spent by the data resources returned by each data transmission protocol.

S202: determining the byte size of the data resource returned by the data transmission protocol;

in the embodiment of the invention, the byte size of each data resource is determined according to the data resource returned by each data transmission protocol.

It should be noted that, in order to prevent the integrity of the data resource during the data transmission, whether the data resource is complete or not can be determined by determining the byte size of the data resource.

S203: and calculating the byte size and the source returning time of the data resource returned by the data transmission protocol to obtain the source returning speed of the data resource returned by the data transmission protocol.

In the embodiment of the invention, after the source returning time is obtained by calculation and the byte size of the data resource is determined, the source returning speed of the data transmission protocol for returning the data resource is calculated.

In the data resource source returning method provided in the embodiment of the present invention, after receiving a source returning request sent by a target server through a data transmission protocol, an upstream server obtains resource information included in the source returning request, and responds according to the resource information, and returns a data resource corresponding to the resource information and source returning information corresponding to the data resource. When the upstream server returns the data resource and the source information, the upstream server also returns the data resource and the source information through the data transmission protocol. After receiving the data resource and the source returning information returned by the data transmission protocol, the target server calculates the source returning time spent by the upstream server for returning the data resource and the source returning information after responding, namely the time required by the data transmission protocol for returning the data resource and the source returning information. After the data transmission protocol returns the data resource and the source returning information, the byte size of the data resource returned by the data transmission protocol is determined, and the time spent by each byte is calculated to obtain the source returning speed, namely the source returning speed is calculated according to the byte size and the source returning time returned by the data transmission protocol. The calculation method is as follows: the source return speed is byte size/source return time.

By applying the method provided by the embodiment of the invention, the source returning speed of the data resource returned by each data transmission protocol is determined by calculating the source returning time and the byte size of the data resource, and the integrity of the data resource returned by the data transmission protocol is ensured.

In the method provided in the embodiment of the present invention, based on the content in step S106, after sending the source returning request to the upstream server through the data transmission protocol supported by each upstream server, a process of receiving the data resource and the source returning information returned by each upstream server through the data transmission protocol is shown in fig. 3, and specifically may include:

s301: when receiving the data resource and the source returning information returned by the first data transmission protocol, starting a preset timer to time;

in the embodiment of the present invention, after the source returning request is sent to the upstream server through each data transmission protocol, the upstream server needs to respond to the source returning request sent by each data transmission protocol in sequence. In general, the upstream server responds to the back-source request sent by the data transmission protocol with the highest priority supported by the upstream server, so that the data resource and back-source information are returned basically fastest for the optimal data transmission protocol. When the target server receives the data resource and the source returning information returned by the first data transmission protocol, a preset timer is started to time in order to ensure that the data resource can be obtained in the optimal time.

S302: and receiving the data resources and the source returning information returned by each data transmission protocol of each upstream server within preset timing time.

In the embodiment of the invention, in order to ensure that the data resources are obtained within the optimal time, the data resources and the source returning information which are returned within the preset timing time are received.

Optionally, if the timing time is exceeded, the data resource and the source return information returned by any data transmission protocol are not received any more.

The method for returning the data resource to the source provided by the embodiment of the invention specifically comprises the following steps:

and in the timing time, when a response failure message returned by the data transmission protocol is received or the data transmission protocol returns data resources and source returning information is not received, disconnecting the connection with the data transmission protocol.

In the data resource source returning method provided by the embodiment of the invention, the source returning request is sent to the upstream server through each data transmission protocol, and the upstream server responds to the source returning request of each data transmission protocol, and then returns the data resource and the source returning information according to the source returning request. However, in the process of requesting back to the source, there may be some cases where some upstream server fails or network delays occur, resulting in failure or too long delay time for sending back to the source request or returning back to the data resource. Therefore, when the first data transmission protocol returned data resource and the returned source information are received, a preset timer is started to time, and only the returned data resource and the returned source information within the time period are received. And if a data transmission protocol return response failure message is received within the timing time, representing the conditions of error, failure of an upstream server or network interruption and the like in the process of sending a source return request. If no data transmission protocol return data resource and return source information are received within the timing time, the conditions may be due to network delay or too slow response of the upstream server. Whether a data transmission protocol return response failure message is received or data resources and return source information returned by the data transmission protocol are not received within the timing time, the connection with the data transmission protocol is disconnected. For example, the target server and the upstream server A are connected through a data transmission protocol of a source return request through the HTTP/TCP protocol and the HTTP/QUIC protocol, if the HTTP/TCP protocol returns data resources and source return information within a timing period, but the HTTP/QUIC protocol does not return the data resources and the source return information, the HTTP/QUIC protocol is disconnected, and the target server and the upstream server A are disconnected through the HTTP/QUIC protocol.

It should be noted that, when the upstream server does not store the data resource corresponding to the resource information included in the request to return to the source, the upstream server also returns a response failure message through the data transmission protocol to prompt the target server that the upstream server does not store the data resource corresponding to the request to return to the source.

In the embodiment of the present invention, optionally, before receiving the data resource and the source return information returned by the first data transmission protocol, the target server may receive a response failure message returned by any data transmission protocol, and disconnect the connection with the data transmission protocol. When the first data transmission protocol returns the data resource and the source information, the timer is started to start timing.

Specifically, when the timing time is exceeded, even if the upstream server successfully responds to the back source request and returns the data resource through the data transmission protocol, the target server is disconnected from the data transmission protocol, and therefore the target server will not receive back source information of the data resource returned by the data transmission protocol.

By applying the method provided by the embodiment of the invention, in order to prevent the overlong time for waiting for the data resources returned by each data transmission protocol, after the data resources and the source returning information returned by the first data transmission protocol are received, the timer can be started to time, so that the data resources and the source returning information returned by the data transmission protocol in the optimal time period can be ensured to be obtained, and the user experience is improved.

In the method provided by the embodiment of the present invention, after determining the data transmission protocol with the largest transmission value among the data transmission protocols of each returned data resource, the data transmission protocol is determined as the target transmission protocol, that is, the data resource returned by the target transmission protocol is the data resource requested by the user. After determining the target transmission protocol, the method may further include:

and storing the data resources and the source returning information of the target transmission protocol to a preset storage area, and deleting the data resources and the source returning information sent by all the data transmission protocols except the target transmission protocol.

In the data resource source returning method provided in the embodiment of the present invention, after receiving a plurality of data transmission protocols and returning source information, since the storage space in the target server is limited, all the data resources and returning source information returned by all the data transmission protocols cannot be stored, so that after storing the data resources and returning source information returned by a target transmission protocol, the data resources and returning source information returned by other data transmission protocols are deleted, so as to ensure that the target server can have an excess storage space to store the data resources returned next time. Meanwhile, the redundancy condition of the target server is reduced, and the service performance of the target server is improved.

In the method provided in the embodiment of the present invention, based on the content in step S107, after the data resource and the source returning information returned by each data transmission protocol are obtained, the source returning information needs to be input into the machine learning model that is trained in advance, so that the machine learning model outputs the protocol weight of the data transmission protocol corresponding to the source returning information. The training process of the machine learning model may specifically include:

when the server time of the target server reaches the end-of-day processing time, acquiring historical source returning information pre-stored in the target server and a protocol speed corresponding to each piece of historical source returning information, wherein the protocol speed is the speed of sending a source returning request to the upstream server by a corresponding data transmission protocol and acquiring data resources;

sequentially inputting the historical source returning information into the machine learning model so as to enable the machine learning model to carry out model training until model parameters of the machine learning model meet preset training conditions;

when each piece of historical source returning information is input into the machine learning model, a training result of the historical source returning information currently input into the machine learning model is obtained; calling a preset loss function, and calculating the training result and the protocol speed corresponding to the current historical source returning information input into the machine learning model to obtain a loss function value; judging whether the model parameters of the machine learning model meet the training conditions or not according to the loss function values; if not, adjusting the model parameters of the machine learning model according to the loss function values; and if so, obtaining the machine learning model which is trained.

In the data resource source returning method provided by the embodiment of the invention, when the server time of the target server reaches the end-of-day processing time, that is, when the time of the day in the target server is about to be switched to the next day, the machine learning model preset in the target server is trained. And obtaining historical source returning information pre-stored by the target server, wherein the historical source returning information can be source returning information stored after the target server successfully returns the source of the data resource through each target transmission protocol on the current day. Each history source returning information comprises a plurality of history source returning data, wherein the history source returning data comprises a server address of a target server, a data transmission protocol of a returned data resource, a returning time, a resource size of the data resource, a response speed of the server responding to a source returning request and the like. And acquiring a protocol speed corresponding to each historical source returning information, wherein the protocol speed is the speed of sending a source returning request to an upstream server by a corresponding data transmission protocol to acquire a data resource. And sequentially inputting the historical source returning information into the machine learning model so that the machine learning model is trained according to the historical source returning data in each piece of historical source returning information. When the machine learning model outputs a training result corresponding to each piece of historical source returning information, calculating the protocol speed of the training result and the corresponding historical source returning information to obtain a loss function value, and when the loss function value does not meet a preset condition, adjusting model parameters in the machine learning model through the loss function value until the loss function value of the machine learning model meets the preset condition, so as to finish the training of the machine learning model.

In the process of training by using each piece of historical provenance data, the output training result of the machine learning model is the predicted protocol speed of the corresponding data transmission protocol, that is, the machine learning model predicts the protocol speed of the data transmission protocol corresponding to the historical provenance information according to each piece of historical provenance data in the historical provenance information. Each protocol speed corresponds to a certain protocol weight, the larger the protocol speed is, the larger the corresponding protocol weight is, and the protocol weight of each data transmission protocol is obtained through training of a machine learning model.

By applying the method provided by the embodiment of the invention, accurate protocol weight is obtained after each data transmission protocol returns data resources through training the machine learning model, and the accuracy of calculating the transmission value of each data transmission protocol is improved.

The specific implementation procedures and derivatives thereof of the above embodiments are within the scope of the present invention.

Corresponding to the method described in fig. 1, an embodiment of the present invention further provides a data resource source returning device, which is used for implementing the method in fig. 1 specifically, the data resource source returning device provided in the embodiment of the present invention may be applied to a computer terminal or various mobile devices, and a schematic structural diagram of the data resource source returning device is shown in fig. 4, and specifically includes:

a first obtaining unit 401, configured to obtain resource information included in a resource request when it is detected that a user sends the resource request to a target server;

a determining unit 402, configured to determine whether the target server has stored a data resource corresponding to the resource information;

a second obtaining unit 403, configured to obtain, if the target server does not store the data resource corresponding to the resource information, each source returning address pre-stored by the target server, where the source returning address is a server address of an upstream server that is pre-selected to perform data source returning;

a determining unit 404, configured to determine, according to each of the back-to-source addresses, each data transmission protocol supported by each of the upstream servers;

a sending unit 405, configured to apply each data transmission protocol supported by each upstream server, respectively, and send a source returning request corresponding to the resource information to the upstream server, so that the upstream server responds to the source returning request sent by each data transmission protocol;

a first calculating unit 406, configured to receive the data resource and the source returning information returned by each upstream server through each data transmission protocol, and calculate a source returning speed of each data transmission protocol for returning the data resource;

a triggering unit 407, configured to input source returning information returned by each data transmission protocol into a machine learning model trained in advance, and trigger the machine learning model to output a protocol weight of each data transmission protocol;

a second calculating unit 408, configured to determine a priority of each data transmission protocol, and calculate the priority, the source return speed, and the protocol weight of each data transmission protocol to obtain a transmission value of each data transmission protocol;

a feedback unit 409, configured to determine the data transmission protocol with the largest transmission value as a target transmission protocol, and feed back the data resource returned by the target transmission protocol to the user.

In the apparatus provided in an embodiment of the present invention, the determining unit includes:

the analysis subunit is configured to analyze the return-to-source address to obtain server information corresponding to the upstream server included in the return-to-source address;

and the first determining subunit is used for acquiring each protocol identifier contained in the server information and determining the data transmission protocol matched with each protocol identifier.

In the apparatus provided in an embodiment of the present invention, the first calculating unit includes:

the first calculating subunit is configured to calculate, for each data transmission protocol, a back-source time for the data transmission protocol to return a data resource after the upstream server responds to the back-source request;

the second determining subunit is used for determining the byte size of the data resource returned by the data transmission protocol;

and the second calculating subunit is used for calculating the byte size and the source returning time of the data resource returned by the data transmission protocol to obtain the source returning speed of the data resource returned by the data transmission protocol.

In the apparatus provided in the embodiment of the present invention, the receiving unit includes:

the timing subunit is used for starting a preset timer to time when receiving the data resource and the source returning information returned by the first data transmission protocol;

and the receiving subunit is configured to receive, within a preset timing time, data resources and source return information returned by each data transmission protocol of each upstream server.

The device provided by the embodiment of the invention further comprises:

and the interruption unit is used for disconnecting the connection with the data transmission protocol when receiving a response failure message returned by the data transmission protocol or not receiving data resources and source return information returned by the data transmission protocol within the timing time.

The device provided by the embodiment of the invention further comprises:

and the deleting unit is used for storing the data resources and the source returning information of the target transmission protocol to a preset storage area and deleting the data resources and the source returning information sent by all the data transmission protocols except the target transmission protocol.

The device provided by the embodiment of the invention further comprises:

the training unit is used for acquiring historical source returning information pre-stored in the target server and a protocol speed corresponding to each piece of historical source returning information when the server time of the target server reaches the daily final processing time, wherein the protocol speed is the speed of sending a source returning request to the upstream server by a corresponding data transmission protocol and acquiring data resources; sequentially inputting the historical source returning information into the machine learning model so as to enable the machine learning model to carry out model training until model parameters of the machine learning model meet preset training conditions; when each piece of historical source returning information is input into the machine learning model, a training result of the historical source returning information currently input into the machine learning model is obtained; calling a preset loss function, and calculating the training result and the protocol speed corresponding to the current historical source returning information input into the machine learning model to obtain a loss function value; judging whether the model parameters of the machine learning model meet the training conditions or not according to the loss function values; if not, adjusting the model parameters of the machine learning model according to the loss function values; and if so, obtaining the machine learning model which is trained.

The specific working processes of each unit and sub-unit in the data resource source returning device disclosed in the above embodiment of the present invention may refer to corresponding contents in the data resource source returning method disclosed in the above embodiment of the present invention, and are not described herein again.

The embodiment of the invention also provides a storage medium, which comprises a stored instruction, wherein when the instruction runs, the device where the storage medium is located is controlled to execute the data resource back-source method.

An electronic device is provided in an embodiment of the present invention, and the structural diagram of the electronic device is shown in fig. 5, which specifically includes a memory 501 and one or more instructions 502, where the one or more instructions 502 are stored in the memory 501, and are configured to be executed by one or more processors 503 to perform the following operations according to the one or more instructions 502:

when detecting that a user sends a resource request to a target server, acquiring resource information contained in the resource request;

judging whether the target server stores the data resource corresponding to the resource information or not;

if the target server does not store the data resources corresponding to the resource information, acquiring each return source address pre-stored by the target server, wherein the return source address is a server address of an upstream server pre-selected to perform data return source;

determining each data transmission protocol supported by each upstream server according to each back source address;

respectively applying each data transmission protocol supported by each upstream server, and sending a source returning request corresponding to the resource information to the upstream server so that the upstream server responds to the source returning request sent by each data transmission protocol;

receiving data resources and source returning information returned by each upstream server through each data transmission protocol, and calculating the source returning speed of each data transmission protocol for returning the data resources;

inputting the source returning information returned by each data transmission protocol into a machine learning model which is trained in advance, and triggering the machine learning model to output the protocol weight of each data transmission protocol;

determining the priority of each data transmission protocol, and calculating the priority, the source return speed and the protocol weight of each data transmission protocol to obtain a transmission value of each data transmission protocol;

and determining the data transmission protocol with the maximum transmission value as a target transmission protocol, and feeding back the data resource returned by the target transmission protocol to the user.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both.

To clearly illustrate this interchangeability of hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A method for returning data resources to a source is characterized by comprising the following steps:

when detecting that a user sends a resource request to a target server, acquiring resource information contained in the resource request;

judging whether the target server stores the data resource corresponding to the resource information or not;

if the target server does not store the data resources corresponding to the resource information, acquiring each return source address pre-stored by the target server, wherein the return source address is a server address of an upstream server pre-selected to perform data return source;

determining each data transmission protocol supported by each upstream server according to each back source address;

respectively applying each data transmission protocol supported by each upstream server, and sending a source returning request corresponding to the resource information to the upstream server so that the upstream server responds to the source returning request sent by each data transmission protocol;

receiving data resources and source returning information returned by each upstream server through each data transmission protocol, and calculating the source returning speed of each data transmission protocol for returning the data resources;

inputting the source returning information returned by each data transmission protocol into a machine learning model which is trained in advance, and triggering the machine learning model to output the protocol weight of each data transmission protocol;

determining the priority of each data transmission protocol, and calculating the priority, the source return speed and the protocol weight of each data transmission protocol to obtain a transmission value of each data transmission protocol;

and determining the data transmission protocol with the maximum transmission value as a target transmission protocol, and feeding back the data resource returned by the target transmission protocol to the user.

2. The method of claim 1, wherein said determining, from each of said back-to-source addresses, a respective data transfer protocol supported by each of said upstream servers comprises:

analyzing the back source address to obtain server information corresponding to the upstream server contained in the back source address;

and acquiring each protocol identifier contained in the server information, and determining a data transmission protocol matched with each protocol identifier.

3. The method of claim 1, wherein calculating a return-to-source speed for each of the data transfer protocol return data resources comprises:

for each data transmission protocol, calculating the back-to-source time of the data transmission protocol for returning data resources after the upstream server responds to the back-to-source request;

determining the byte size of the data resource returned by the data transmission protocol;

and calculating the byte size and the source returning time of the data resource returned by the data transmission protocol to obtain the source returning speed of the data resource returned by the data transmission protocol.

4. The method of claim 1, wherein the receiving the data resource and source return information returned by each upstream server via each data transmission protocol comprises:

when receiving the data resource and the source returning information returned by the first data transmission protocol, starting a preset timer to time;

and receiving the data resources and the source returning information returned by each data transmission protocol of each upstream server within preset timing time.

5. The method of claim 4, further comprising:

and in the timing time, when a response failure message returned by the data transmission protocol is received or the data transmission protocol returns data resources and source returning information is not received, disconnecting the connection with the data transmission protocol.

6. The method of claim 1, wherein after determining the data transmission protocol with the largest transmission value as the target transmission protocol, further comprising:

and storing the data resources and the source returning information of the target transmission protocol to a preset storage area, and deleting the data resources and the source returning information sent by all the data transmission protocols except the target transmission protocol.

7. The method of claim 1, wherein the training process of the machine learning model comprises:

when the server time of the target server reaches the end-of-day processing time, acquiring historical source returning information pre-stored in the target server and a protocol speed corresponding to each piece of historical source returning information, wherein the protocol speed is the speed of sending a source returning request to the upstream server by a corresponding data transmission protocol and acquiring data resources;

sequentially inputting the historical source returning information into the machine learning model so as to enable the machine learning model to carry out model training until model parameters of the machine learning model meet preset training conditions;

when each piece of historical source returning information is input into the machine learning model, a training result of the historical source returning information currently input into the machine learning model is obtained; calling a preset loss function, and calculating the training result and the protocol speed corresponding to the current historical source returning information input into the machine learning model to obtain a loss function value; judging whether the model parameters of the machine learning model meet the training conditions or not according to the loss function values; if not, adjusting the model parameters of the machine learning model according to the loss function values; and if so, obtaining the machine learning model which is trained.

8. A data resource back-source device, comprising:

the system comprises a first acquisition unit, a second acquisition unit and a resource management unit, wherein the first acquisition unit is used for acquiring resource information contained in a resource request when detecting that a user sends the resource request to a target server;

the judging unit is used for judging whether the target server stores the data resource corresponding to the resource information;

a second obtaining unit, configured to obtain, if the target server does not store the data resource corresponding to the resource information, each source returning address pre-stored by the target server, where the source returning address is a server address of an upstream server that is pre-selected to perform data source returning;

a determining unit, configured to determine, according to each return-to-source address, each data transmission protocol supported by each upstream server;

a sending unit, configured to apply each data transmission protocol supported by each upstream server, respectively, and send a source returning request corresponding to the resource information to the upstream server, so that the upstream server responds to the source returning request sent by each data transmission protocol;

the first calculating unit is used for receiving the data resources and the source returning information returned by each upstream server through each data transmission protocol and calculating the source returning speed of each data transmission protocol for returning the data resources;

the triggering unit is used for inputting the source returning information returned by each data transmission protocol into a machine learning model which is trained in advance, and triggering the machine learning model to output the protocol weight of each data transmission protocol;

the second calculation unit is used for determining the priority of each data transmission protocol, calculating the priority, the source return speed and the protocol weight of each data transmission protocol and obtaining the transmission value of each data transmission protocol;

and the feedback unit is used for determining the data transmission protocol with the maximum transmission value as a target transmission protocol and feeding back the data resource returned by the target transmission protocol to the user.

9. The apparatus of claim 8, wherein the determining unit comprises:

the analysis subunit is configured to analyze the return-to-source address to obtain server information corresponding to the upstream server included in the return-to-source address;

and the first determining subunit is used for acquiring each protocol identifier contained in the server information and determining the data transmission protocol matched with each protocol identifier.

10. The apparatus of claim 8, wherein the first computing unit comprises:

the first calculating subunit is configured to calculate, for each data transmission protocol, a back-source time for the data transmission protocol to return a data resource after the upstream server responds to the back-source request;

the second determining subunit is used for determining the byte size of the data resource returned by the data transmission protocol;

and the second calculating subunit is used for calculating the byte size and the source returning time of the data resource returned by the data transmission protocol to obtain the source returning speed of the data resource returned by the data transmission protocol.

Images