CN113439416B

CN113439416B - Continuously calibrated network system

Info

Publication number: CN113439416B
Application number: CN201980091304.4A
Authority: CN
Inventors: S·D·瓦尔克; J·M·赛尔维奇; 鲁迪·威利斯; 保罗·辛克斯
Original assignee: Amazon Technologies Inc
Current assignee: Amazon Technologies Inc
Priority date: 2018-12-13
Filing date: 2019-12-12
Publication date: 2023-12-26
Anticipated expiration: 2039-12-12
Also published as: WO2020123867A1; EP3895378A1; CN113439416A

Abstract

The application data may be transmitted while the transmission parameters are oscillated. Metrics associated with complementary network characteristics are analyzed to identify transition points between random error states and deterministic error states of the complementary network characteristics. Additional network characteristics or states may be inferred from the transition points, and the transmission of the application data may be optimized based on the inferred additional characteristics or states.

Description

Continuously calibrated network system

Cross Reference to Related Applications

The disclosure of U.S. patent application Ser. No. 16/369,597, entitled "CONTINUOUSLY CALIBRATED NETWORK SYSTEM", filed on 3/29 in 2019, U.S. patent application Ser. No. 16/369,642, entitled "CONTINUOUSLY CALIBRATED NETWORK SYSTEM", filed on 29 in 3/29 in 2019, and U.S. patent application Ser. No. 16/219,865, entitled "CONTINUOUS CALIBRATION OF NETWORK METRICS", filed on 13 in 12/2018, is incorporated herein by reference in its entirety for all purposes.

Background

In data communications, it can be challenging to efficiently understand and utilize the attributes of a network. For example, in the context of a data stream such as a multimedia stream, determining the bandwidth-delay product may be an important step in ensuring that the bit rate of the multimedia stream does not cause network congestion or underflow and does not provide poor results for users of the network. If the amount of data sent over the communication channel is too high (e.g., the bit rate of the multimedia stream exceeds the bandwidth delay product), the recipient of the multimedia stream may experience poor performance, such as periodic buffering, packet loss, increased delay, etc.; if the amount of data is too low, the recipient will not benefit from the full capacity of the network. The tendency of networks to be affected by "external" factors such as network congestion caused by other users further increases the complexity of these problems.

Drawings

Various techniques will be described with reference to the accompanying drawings, in which:

FIG. 1 shows an illustrative example of a computing environment in which various embodiments may be practiced;

FIG. 2 shows an illustrative example of a graph in which network latency and network capacity may be measured by oscillating a complementary network characteristic calibration in accordance with at least one embodiment;

FIG. 3 shows an illustrative example of a graph of relationships between different network parameters that may be used as part of determining a bandwidth-delay product in accordance with at least one embodiment;

FIG. 4 shows an illustrative example of a relationship between different network parameters that may be used as part of determining a bandwidth-delay product in accordance with at least one embodiment;

FIG. 5 shows an illustrative example of a process for calculating a bandwidth-delay product for a network connection in accordance with one embodiment;

FIG. 6 shows an illustrative example of a process for saturating a network connection according to one embodiment;

FIG. 7 shows an illustrative example of a relationship between different network parameters that may be used as part of determining network characteristics in accordance with at least one embodiment;

FIG. 8 shows an illustrative example of identifying changing network conditions in accordance with at least one embodiment;

FIG. 9 shows an illustrative example of continuous network calibration in accordance with at least one embodiment;

FIG. 10 shows further aspects of an illustrative example of continuous network calibration in accordance with at least one embodiment;

FIG. 11 shows an illustrative example of a computing environment that may transmit data in excess of surface network capacity in accordance with at least one embodiment;

FIG. 12 shows an illustrative example of a process for continuously calibrating a network system in accordance with at least one embodiment;

FIG. 13 shows an illustrative example of a process for determining an operational metric by oscillation of an operational parameter in accordance with at least one embodiment; and is also provided with

Fig. 14 illustrates a system in which various embodiments may be implemented.

Detailed Description

The techniques described herein may be used to implement methods, machines, and articles of manufacture for operating a network system or other system using oscillation calibration techniques. In at least one embodiment, the system transmits application data over a network according to transmission parameters. The system identifies a random or deterministic error state of the metric associated with the transmission parameter and adjusts the transmission parameter to cause oscillation of the metric between the random error state and the deterministic error state. A transition point between the two states is identified, and transmission of the application data is adjusted based at least in part on the identification of the transition point.

As one example, in Transmission Control Protocol (TCP) based communications, network characteristics such as bandwidth delay product may be used to determine how much data may be transmitted before a server stops and waits for acknowledgement of a received packet. The Bandwidth Delay Product (BDP) is the product of the capacity of the data link and the round trip delay time or delay of the data. The bandwidth delay product of the communication channel may be equal to the maximum amount of data that can be transmitted through the network circuitry at any given time. Determining a reliable estimate of the bandwidth-delay product may, for example, allow for maximizing data throughput given current network conditions and thereby reducing buffering or delay of communications by ensuring that the data rate transmitted across the network does not exceed the maximum data throughput supported by the network connection. Similar aspects regarding bandwidth delay products are applicable in terms of User Datagram Protocol (UDP) and other communication protocols.

Determining network characteristics such as bandwidth-delay products involves a number of challenges. Even in static systems, there may be a high degree of variance in the measurement of the bandwidth output and delay values used to determine the bandwidth delay product value. For example, for a static network configuration, the first delay measurement and the second delay measurement may be different due to variances in the network. Even in systems where the bandwidth-delay product is known, the bandwidth and delay measurements may vary based on statistical distributions. Thus, in a dynamic network environment (e.g., where the bandwidth-delay product is not static and changes due to changing network conditions), there are difficulties in instrumentation of the network and determining real-time metrics such as the bandwidth-delay product.

Determining network characteristics such as bandwidth-delay products is useful in various situations and can be used to improve the operation of a computer system in a variety of ways. For example, in the context of multimedia streaming (e.g., streaming of multimedia such as movies or television programs at a variable or constant bit rate), a computer system may determine a bandwidth latency product of a network and indicate the network bandwidth latency product to a Content Delivery Network (CDN) or streaming service. A data provider (e.g., CDN) may receive the network bandwidth latency product and use it to determine how to provide multimedia content to a media client. For example, the streaming service may establish a connection with the client computer system, obtain a network bandwidth delay product as part of establishing the connection, and transmit the multimedia stream at a bit rate that does not exceed the network bandwidth delay product. In this way, the multimedia streaming service is able to provide high quality multimedia content to clients and buffering may be reduced or eliminated entirely. For example, if a data provider (e.g., a multimedia streaming service) submits data at a rate that exceeds the bandwidth-delay product, the client cannot receive and process the data at the same rate at which the data is transmitted. Thus, buffering or data loss may occur, which may in some cases result in poor performance (e.g., cause a client to watch a movie, conduct a conference call, interrupt playing a video game).

According to various aspects described in greater detail below, the network characteristics may be determined using an oscillating complementary network characteristic calibration technique. Oscillation complementary network characteristic calibration techniques may refer to techniques in which one of two complementary network characteristics may be used to determine the other. In this context, the term "complementary" may refer to characteristics that are linked to each other in some way, e.g., if an increase in one characteristic is correlated with another characteristic, then the two characteristics may be considered complementary; a constant property may be considered complementary if it relates to another increasing or decreasing property; etc. It should be noted that these are merely illustrative examples of how two characteristics may be considered complementary characteristics. In general, complementary network characteristics are tied together in a manner that allows knowledge of the trend or set of values of one characteristic to be correlated with the values of another network characteristic.

According to various aspects described in greater detail below, monitoring of complementary network characteristics may be performed using a process of transmitting application data at an oscillating rate. More generally, according to various systems and methods described in more detail below, the throughput rate of the system may be adjusted in an oscillating manner to perform monitoring of complementary system characteristics. In various embodiments, the monitoring of the complementary network or system characteristics may be continuous or near continuous. Oscillations in transmission rate or throughput may be based on causing network characteristics (sometimes referred to as metrics) to oscillate between random error states and deterministic error states. As explained in more detail below, random errors refer to distributions of metrics that appear random, such as gaussian distributions, and deterministic errors refer to errors that appear to be related to systematic errors. For example, deterministic errors of metrics may appear to have a linear dependence. By varying transmission parameters such as the transmission rate or throughput of the system such that such oscillations occur, a value of the complementary system characteristic can be observed. Additionally, values of related system characteristics or states may be inferred based on identifying transition points between random error states and deterministic error states.

According to various aspects described in greater detail below, a process that uses oscillating transmission of application data is employed to perform continuous monitoring of a network and corresponding adjustments and optimizations to application utilization of the network. More generally, these techniques are applied to the system to continuously monitor complementary system characteristics and make corresponding adjustments and optimizations to the system. In some cases, continuous monitoring of system characteristics enables more aggressive utilization of the system. For example, given that the current state of the system is known more than would otherwise be available in the absence of the techniques described herein, it is possible to take measures that are typically assumed to be over-utilized with a network or other system.

As one example, to determine the bandwidth delay product, a process for performing a continuous oscillation complementary network characteristic calibration may include determining a network delay or Round Trip Delay (RTD) time (also referred to as Round Trip Time (RTT) throughout this disclosure) as a first network characteristic of the oscillation complementary network characteristic calibration. Round trip time may refer to the amount of time it takes to send data from a source computer system to a target computer system and for the source computer system to receive an acknowledgement from the target computer system that the data has been received. The network latency may also be referred to as a ping time, but in this disclosure, application data may be used to complete a ping operation in addition to data separate from application data (where application data may include data being streamed or transmitted from a source computer to a target computer, such as audiovisual data or other data related to the operation of an application, rather than data transmitted primarily for purposes of measuring network behavior). Various tools and commands may be used to measure the round trip time of a message. For example, pings (or application data perturbations) in various computing environments may operate by sending Internet Control Message Protocol (ICMP) echo request packets from an origin to a destination. However, using ping commands themselves may introduce distortion that may affect observed round trip times and other metrics. This may occur when performing continuous monitoring, as the amount of network traffic associated with ping commands may be high in such cases. Thus, application data may be used instead of discrete probes, such as ping commands.

In general, there may be some variation in network latency due to variations in network conditions including the amount of external traffic being handled by the intermediate routers of the network. As a function of bandwidth input, the delay of the system is substantially constant when the bandwidth input is less than or equal to the bandwidth delay product, because the amount of time (average) of data packets to be received over the network does not change based on the amount of data transmitted without oversaturation of the network, in other words when there is an underflow of data with respect to the upper limit of the bandwidth delay product. The bandwidth input may be an amount of data per unit time (e.g., a rate in bits/second) that is transmitted across the network by the data provider to the data receiver. For example, bandwidth input may refer to the bit rate of multimedia transmitted by a content delivery network (e.g., a data provider) over a network to a data receiver (e.g., a client computer system playing a multimedia stream).

The second network characteristic of the oscillation complementary network characteristic calibration may be a bandwidth output. Bandwidth output may refer to the data rate received and/or processed by a data receiver. In general, when a network connection is undersaturated, there is a one-to-one relationship between bit input and bit output, so that there may be a case where the bandwidth input may be equal to the bandwidth output, because in an undersaturated network connection, there is such a case: there is sufficient capacity to support the data rate.

By performing measurements of network conditions on a set of network parameters, bandwidth delay products can be calculated using oscillation complementary network characteristic calibration. For example, as described above, the network parameters measured as part of the calibration may be latency and bandwidth inputs. In one implementation, the system is designed to determine the value of the first network parameter, then the value of the second network parameter, then the updated value of the first network parameter, then the updated value of the second network parameter, and so on in an oscillating manner.

Determining the network parameter value may involve measuring a complementary network characteristic and thereby obtaining a metric value indicative of the characteristic. As one particular example, it should be noted that when the network is undersaturated, the delay is constant or approximately constant (e.g., the average measurement of the delay over a larger sample size will not differ significantly based on the bandwidth output value), and when the network is oversaturated, the delay increases. This relationship is shown and described in more detail elsewhere in this disclosure, such as in connection with fig. 3. As a second example, when the network is undersaturated, the bandwidth input and the bandwidth output may be positively correlated (e.g., as the data provider's transmission rate increases to a bandwidth delay product, the data receiver's rate of receiving data increases). The positive correlation may be a 1:1 ratio such that the data transfer increases by 1 KB/sec, as does the rate of data received (e.g., no buffering or data loss occurs). However, when the network is oversaturated, the bandwidth output may remain constant because the network is already fully saturated and the data transmitted across the network cannot be delivered to the data receiver at the same rate as the data is transmitted.

As an illustrative example, an accurate maximum bandwidth input (e.g., determining a bandwidth delay product value) may be determined by first instrumentally detecting a network using a correlation between bandwidth input and bandwidth output and measuring a delay of the network. As part of the oscillation complementary network characteristic calibration, the system performing the calibration process may determine a bandwidth output value in KB/sec (e.g., determine data throughput), submit data at the determined rate, and then measure the delay of the network at the bandwidth output value. The delay may be measured by, for example, submitting one or more network packets for transmission. In some embodiments, this may be a "ping" command, but application data may also be used. The system may submit additional network packets to determine a second delay value for the network at a slightly higher second bandwidth output value. Implementations may, for example, adjust the rate at which data packets are submitted, or adjust the size of the submitted data. When application data is used, implementations may limit message transmission, where the transmitted message includes the application data. Additional delay measurements may be made for a range of bandwidth output values, the initial values and incremental steps of which may be determined based on statistical characteristics, in order to achieve certain confidence levels, such as 90%, 95%, 99% and higher (or lower depending on requirements and design). This can be expressed as: 1- α=0.90, such as in the previous example, where the significant level α is selected by a system administrator and may be adjusted based on desired network behavior. A set of network latency measurements may be collected and analyzed using statistical tools and algorithms to detect a positive correlation between latency and bandwidth input, that is, increasing bandwidth input after a certain threshold (e.g., maximum throughput that the system may not have calculated) may result in a trending increase in network latency, which is statistically significant. The correlation may be measured by calculating a pearson product moment correlation coefficient r for the set of collected delay values and indicating that the network bandwidth is oversaturated if the correlation coefficient exceeds a threshold.

After determining the network bandwidth oversaturation, the system performing the oscillation complementary network characteristic calibration may calculate the network capacity by measuring the bandwidth output at a bandwidth input value for which the correlation coefficient indicates the network bandwidth oversaturation. For example, the bandwidth input value used to measure network capacity may be the highest bandwidth input value used in a set of values used to determine the correlation coefficient r. However, this need not be the same, and other implementations are also contemplated within the scope of the present disclosure, as described in more detail below. As described above, network capacity (e.g., bandwidth capacity) may be calculated by taking multiple bandwidth output measurements at selected bandwidth input values. Making multiple bandwidth output measurements may improve the accuracy of the calculated network capacity, at least because of the possible variance in the network measurements. The variance may be caused by various factors and may be modeled or fitted to a normal distribution. The number of bandwidth output measurements made may depend on parameters of the collected data, such as standard deviation, kurtosis, skewness, etc. of the measured bandwidth output values. In some cases, the calibration process includes making bandwidth output measurements until the confidence interval of the measured average bandwidth output has a range that is less than a threshold (e.g., has a width that is less than a threshold width). In this way, examples of complementary network characteristics (e.g., network latency in this case) are used as part of calibration to determine network capacity. Thus, network capacity may be measured.

Once network capacity is determined, an oscillation complementary network characteristic calibration may be performed to oscillate the oscillation to determine network delay. Continuing with the example above, oscillation occurs by measuring a complementary network characteristic (e.g., bandwidth output in this case) that varies with the bandwidth input value and reducing the bandwidth input value. As described above, the bandwidth input is increased until a positive correlation between the network delay and the bandwidth is determined. The bandwidth input is then reduced as the bandwidth output is measured until a strong positive correlation is detected between the bandwidth input and the bandwidth output (e.g., indicating that the bandwidth output is decreasing as the bandwidth input decreases). Once a positive correlation is detected (e.g., the correlation coefficient r exceeds a threshold), the system may measure the network delay at the bandwidth input value at which the system can be assured that the network capacity is undersaturated (e.g., the correlation coefficient r exceeds a threshold). In the event that such a determination is made that the network capacity is undersaturated at a particular bandwidth input value or range of values, the system may measure the network delay by taking one or more samples. In this way, examples of complementary network characteristics (e.g., network capacity in this case) are used as part of the calibration to determine network latency. Thus, network latency may be measured.

By performing these steps, the computer system may continuously determine the bandwidth-delay product on the order of hundreds of milliseconds or even less (or on the order of round trip time or up to several round trip times), where factors such as network bandwidth, measured network capacity, and variance of delay values, and required accuracy are examples of the amount of time that may affect the bandwidth-delay product may be calculated. For broadband-based connections, bandwidth delay products can be accurately calculated using oscillation complementary network characteristic calibration.

After both network capacity and network delay are calculated successively as described above, the calculated bandwidth delay product value may be utilized in various contexts, such as setting the bit rate of the multimedia stream so as to maximize the number and quality of the multimedia stream so that the stream does not experience ongoing buffering if the network conditions are unchanged. In the context of video games, the bandwidth of data (e.g., including graphics data) streamed for the game may be adjusted to achieve a high quality experience for online players and viewers based on the bandwidth latency product value. In the context of remote applications or software applications or desktop virtualization, bandwidth latency products may be utilized by the virtualization host to determine what type and how much data to send to the remote client. In the context of video streaming, the bandwidth-delay product may produce an experience of the desired quality. The use case scenario is not limited to gaming, software virtualization, video streaming, etc., but is open to any scheme that attempts to send data over a network. Thus, these are non-exhaustive examples of implementations in which bandwidth-delay product values can be accurately, efficiently, and quickly calculated using oscillation complementary network characteristic calibration to improve the operation of computer systems, computer networks, and software applications.

In the foregoing and following description, various techniques are described. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the possible ways in which the techniques may be implemented. However, it will also be apparent that the techniques described below may be practiced in different configurations without specific details. Furthermore, well-known features may be omitted or simplified in order not to obscure the described technology.

As will be appreciated by those of skill in the art in light of this disclosure, certain embodiments may be capable of achieving certain advantages, including some or all of the following: reducing buffering of data; increasing the effective data bandwidth of the network; improving the reliability of the network by reducing the rate at which packets are dropped; the bandwidth delay product of the network connection is measured more accurately; etc. The techniques described herein may be used to improve the operation of a computer system by increasing the data communication capabilities of the computer system, such as by increasing data bandwidth, reducing or eliminating buffering (e.g., of multimedia streams), reducing or eliminating dropped data packets, any combination thereof, and so forth.

FIG. 1 shows an illustrative example of a computing environment 100 in which various embodiments may be practiced. The computing environment 100 may include a first computer system 102 and a second computer system 104 connected via a network 120, such as the internet. The first computer system 102 and/or the second computer system 104 may include executable code, such as those described in more detail below, such as in connection with fig. 2 and 5, that when executed by one or more processors of the computer system causes the one or more processors to determine network characteristics, such as a bandwidth delay product, using oscillation complementary network characteristic calibration.

In one embodiment, the first computer system 102 is any suitable computing system, such as those described in connection with fig. 14. The first computer system 102 may be, for example, a personal computer, a laptop computer, a smart phone, an embedded device, an internet of things (IOT) device, or any other various types of computing devices. The first computer system 102 may include hardware and/or software configured to enable the first computer system 102 to communicate with a network, e.g., via a network interface card, a wireless antenna, etc. For clarity, in some implementations, the first computer system 102 may be a data receiver and the second computer system 104 may be a data provider (which may be the same or similar type of device described above with respect to the computer system 102). For example, the first computer system 102 may be a personal computer that includes software (e.g., client code) that the first computer system 102 uses to communicate with the second computer system 104. In one embodiment, the first computer system 102 includes software, such as a video game application, streaming application, etc., that includes executable code that causes the first computer system 102 to communicate with the second computer system 104 to receive data, such as a multimedia stream (including video and/or audio).

The second computer system 104 may be any suitable computer system, such as those described in connection with fig. 14, and may be configured with executable code that, when executed, allows the second computer system 104 to communicate with the first computer system 102. In one embodiment, the second computer system 104 is a data provider, such as a Content Delivery Network (CDN), or a computing resource service provider that provides a data stream as a service for the service provider. Of course, the functions or operations of the first computer system 102 and the second computer system 104 may be reversed or redistributed in any suitable manner, e.g., the first computer system 102 may be a data provider and the second computer system 104 may be a data receiver. In one embodiment, the second computer system 104 is a computer server hosting a CDN and has access to a data storage system having data. In one implementation, the data may be multimedia content that is stored (e.g., persisted on a hard disk drive) or otherwise accessible to the second computer system.

The second computer system 104 may encode the source multimedia content using a codec, which may be encoded at a variable or fixed bit rate that is determined and/or adjusted based on network characteristics, such as bandwidth-delay product, determined using oscillation complementary network characteristic calibration as described in detail herein. The first computer system 102 and/or the second computer system 104 may, for example, be configured to perform a calibration process, such as those described in accordance with fig. 5, which is described in more detail below.

In one embodiment, the second computer system 104 is a multimedia streaming service that determines a bandwidth delay product of a network connection with a client (e.g., the first computer system 102) for the client of the multimedia streaming service and generates and/or provides a multimedia stream to the client that has a bit rate or data transfer rate that does not exceed the bandwidth delay product. Not exceeding the bandwidth-delay product may have various advantages, such as reducing or eliminating buffering of multimedia content on the client.

In one embodiment, the first computer system 102 and the second computer system 104 establish a communication channel and/or perform a handshake. The computer system may communicate using any suitable protocol, such as User Datagram Protocol (UDP) or Transmission Control Protocol (TCP). As part of establishing a communication channel, one or both computer systems may calculate a bandwidth delay product for data transfer with the other computer system. In one embodiment, a computer system (e.g., a client of a multimedia streaming service) uses an oscillating complementary network characteristic calibration technique, such as in a manner that will be described in more detail below, to calculate the bandwidth delay product.

Performing oscillation complementary network characteristic calibration (e.g., by one or both of the computer systems described in connection with fig. 1) may involve two-stage and continuous oscillation network perturbation. As part of the first phase, the system performing calibration may interfere with network conditions by acquiring a first set of RTT measurements 106 when interfering with bandwidth input of the network. For example, the perturbation may be achieved by incrementally adjusting the value of the transmission parameter.

In one embodiment, the first set of measurements is a set of measurements of network latency collected for different bandwidth inputs. However, the set of measurements need not all be taken at different and different bandwidth inputs. For example, in some cases it may be advantageous to obtain multiple measurements at a certain bandwidth input and perform statistical techniques (such as averaging or smoothing) to determine an aggregate measurement. For example, multiple delayed measurements may be taken at one bandwidth input level to obtain an average and standard deviation of the measurements to determine an aggregate measurement that is more accurate than a single measurement. Accuracy in this case may refer to the fact that: when the signal is random (e.g., spread out in a normal distribution), multiple samples may generally result in a more accurate assessment than a single measurement. Each of the first set of measurements may be a datagram, a message, a packet, or any other form suitable for transmission across a network. In some implementations, each of the measurements is acquired sequentially, as the following may be present: attempting to acquire measurements in parallel may affect other measurements (e.g., affect the bandwidth input of the data pipe).

In some embodiments, when the network is undersaturated, the delay is substantially constant or approximately constant, and thus, the observed delay may be considered to be randomly distributed. In some embodiments, when the network is oversaturated, the delay is positively correlated with the bandwidth input, that is, there is a direct relationship between the network delay and the bandwidth input. This may mean that an increase in bandwidth input is positively correlated with an increase in network latency. As part of the oscillation complementary network characteristic calibration, in some implementations, the system analyzes the first set of RTT measurements 106 using statistical techniques to detect a first correlation 108 between the measured network characteristic and the complementary characteristic. For example, in this case, network delays that vary with bandwidth input are collected and statistical techniques (using any suitable statistical technique) are used to determine the correlation coefficient between delay and bandwidth input. If the correlation coefficient is positive, it means that there is a positive correlation between the two variables, and that the increase of one variable is correlated with the increase of the other variable; if the correlation coefficient is negative, it means that there is an inverse relationship between the two variables, and an increase in one variable tends to be associated with a decrease in the other variable; if the correlation coefficient is zero, it means that the value of one variable cannot help determine the value of another variable.

Bandwidth input may refer to the data rate sent by a perturbed client over a network to saturate the network to a particular level. If the rate is greater than the capacity of the network, the network is oversaturated; conversely, if the rate is less than the capacity of the network, the network is undersaturated. It should be noted that in some embodiments, the network is dynamic and various network parameters such as capacity and latency may change over time. In some cases, the change may be abrupt, such as in the case where, for example, the consumer's broadband connection is affected by another user in the home connected to the streaming service, thereby reducing the amount of bandwidth available to other devices in the home.

As part of the calibration, the system may determine whether the first correlation 108 exceeds a particular threshold. Exceeding the threshold may determine that the calculated correlation coefficient is greater than or less than a particular threshold (including or not including the threshold). In some embodiments, the threshold is selected based on a probability factor such as a confidence level of 1- α, such that the probability that the first correlation coefficient exceeds the threshold after multiple measurements n is equal to (or greater than, greater than or equal to, etc.) 1- α assuming that other probability parameters are known (e.g., the mean and standard deviation of previous measurements in previous oscillations). In this way, the system can configure parameters to ensure that a set of measurements is made over a time of n x L at a probability level 1- α (e.g., 99% probability), where n is the number of measurements and L is the delay of each measurement.

If the first correlation 108 exceeds a threshold or probability, the system may use this as an indication that the delay and bandwidth input are sufficiently correlated to infer that the delay increases with increasing bandwidth input. In some implementations, the system uses this information to measure the first complementary network characteristic 110. The complementary network characteristic of the network delay shown in fig. 1 is bandwidth output. The bandwidth output may be measured at a highest bandwidth input value included in the first set of measurements, the bandwidth output determined to increase with bandwidth input. Other bandwidth input values may be used to measure the bandwidth output, for example, values higher than the highest bandwidth input value in the first set may be utilized, as it has been determined that the network is oversaturated with bandwidth input, and as such, the network will be oversaturated with even larger values.

In some embodiments, the system obtains a single measurement of the bandwidth output. However, in other embodiments, the system obtains multiple measurements of the bandwidth output. The measurement results may be normally distributed and statistical techniques may be used as part of determining the bandwidth output. For example, when the standard deviation of the measurement results is large, the number of the obtained measurement results may be large, and when the standard deviation is small, fewer measurement results may be obtained. Thus, the average of one or more measured values may be the network capacity.

After determining the network capacity, the system may perform a second phase of oscillation by perturbing the network to undersaturation. Similar to the first stage, the system may collect a second set of measurements 112. The second set of measurements may be measurements of complementary characteristics, in this case bandwidth output. The bandwidth output may be measured as a function of the bandwidth input, and the measurement may be collected for decreasing values of the bandwidth input. In one embodiment, the bandwidth input value has a value less than the determined network capacity. The system may analyze the second set of measurements 112 to determine a second correlation 114, which is a correlation between the bandwidth output and the bandwidth input. If there is a positive correlation (e.g., a decrease in bandwidth output tends to be associated with a decrease in bandwidth input), the system measures a complementary network characteristic (e.g., network delay 116) at a certain bandwidth input value, which in some embodiments is selected to be less than or equal to the lowest bandwidth input value in the second set. The network latency may be calculated using statistical techniques as described in connection with the first stage.

After completing the first and second phases (which may be reversed in order in some embodiments), the system may be able to calculate additional network characteristics. For example, determining network capacity and network latency is sufficient to determine a bandwidth-latency product. In one embodiment, the bandwidth-delay product is used to configure a communication session, such as a UDP-based communication session. After the oscillation complementary network characteristic calibration is completed, the oscillation may be performed again in a continuous manner, since the network conditions may have changed.

Determining the bandwidth-delay product of a network is useful in a variety of contexts and can be used to improve the operation of a computer system in a variety of ways. For example, in the context of multimedia stream 118 (e.g., streaming of multimedia such as a movie or television program at a variable or constant bit rate), the computer system may determine a bandwidth latency product of the network and indicate the network bandwidth latency product to a Content Delivery Network (CDN) or streaming service. A data provider (e.g., CDN) may receive a network bandwidth-latency product and utilize the bandwidth-latency product to determine how to provide multimedia content to a media client. For example, the streaming service may establish a connection with the client computer system, obtain a network bandwidth delay product as part of establishing the connection, and transmit the multimedia stream at a bit rate that does not exceed the network bandwidth delay product. In this way, the multimedia streaming service is able to provide high quality multimedia content to clients and buffering may be reduced or eliminated entirely. For example, if a data provider (e.g., a multimedia streaming service) submits data at a rate that exceeds the bandwidth-delay product, the client cannot receive and process the data at the same rate at which the data is transmitted. Thus, buffering or data loss may occur, which may in some cases result in poor performance (e.g., cause a client to watch a movie, conduct a conference call, interrupt playing a video game).

In one embodiment, the first computer system 102 and the second computer system 104 are each devices operable to send and/or receive requests, messages, or information over the appropriate network 120 and to communicate the information back to the user of the device. Examples of such client devices include personal computers, cellular or other mobile phones, handheld messaging devices, laptop computers, tablet computers, set-top boxes, personal data assistants, embedded computer systems, electronic book readers, and the like. In one embodiment, the network comprises any suitable network, including an intranet, the internet, a cellular network, a local area network, a satellite network, or any other such network and/or combination thereof, and the components for such a system depend at least in part on the type of network and/or system selected. Many protocols and components for communicating via such networks are well known and will not be discussed in detail herein. In one embodiment, communication over the network is achieved through wired and/or wireless connections, and combinations thereof. In one embodiment, the network includes the Internet and/or other publicly addressable communication networks.

Fig. 2 shows an illustrative example of a graph 200 in which network delay and network capacity may be measured by oscillating a complementary network characteristic calibration in accordance with at least one embodiment. The techniques described in connection with fig. 2 may be implemented in accordance with embodiments described elsewhere in this disclosure, such as those discussed in connection with fig. 1, 5, and 14. Oscillation may refer to the foregoing process: increasing the bandwidth input to determine network capacity, then decreasing the bandwidth input to determine network latency, then increasing the bandwidth input to update network capacity, then decreasing the bandwidth input to update network latency, and so on. This iterative process may be performed continuously (e.g., repeatedly without waiting between measurements, or by performing on some timed schedule, such as every few seconds, milliseconds, or as appropriate for a given system).

As part of the oscillation complementary network characteristic calibration, a suitable computer system may obtain a set of delay measurements. For the embodiment according to fig. 2, the first set of measurements refers to the round trip time or delay of the network connection. The set of measurements may be delay measurements collected at different bandwidth inputs. Obtaining delay measurements at a certain bandwidth input may be performed using a variety of techniques. For example, the system may estimate network latency at rest by sending data to the other party without applying pressure to the network with other data. In some embodiments, this is an implicit assumption: the bandwidth of the system may be measured in an undersaturated state. The counterpart may receive the transmitted data and transmit a response, and the round trip time between transmitting the data and receiving the response may be a delay of static time. In some embodiments, multiple ping measurements are obtained to determine statistically reliable latency measurements. Based on the delay, the bandwidth input may be pressurized or saturated to determine how much data may be transmitted over the network. As one example, consider the case where the delay of an empty pipe is measured as 100 milliseconds (ms). To measure latency at a bandwidth input of 250 kb/sec, the system may submit one or more messages that collectively encode 250 x 100/1000=25 kb of data. More generally, if an initial message is sent and the response time is measured as L, the function f () for describing the amount of data sent to measure the delay at a particular bandwidth input value can be described as:

f(BW _{Input device} )＝L*BW _{Input device}

In some embodiments, a bandwidth input value is determined, the size of the transmitted message is determined, and any data (e.g., data whose value is irrelevant except for the size of the message (in bytes)) is prepended or otherwise transmitted. For example, if the default size of a message is 32 bytes and the bandwidth input is 128 bytes, 96 bytes of random data (96 bytes of 0 or 1 array, 96 bytes of uninitialized data, etc.) may be added before the message, or one or more data messages of 96 bytes total size may be added before the message. Although the contents of the data may not be readable, transmitting this data may be used to stress the network connection, with the amount of data being specified by the bandwidth input value. In some embodiments, the network connection may be stressed by adjusting the rate at which application data is sent. Thus, instead of (or in addition to) sending any data, the rate at which data used by the application is sent can be increased.

By stressing the network connection with a particular amount of data, the system is able to determine when the network connection has reached saturation (e.g., reached capacity). Consider the case where the data connection is undersaturated when sending a message-routers, switches and computing entities on the network are able to commit data as it is received and do not incur additional latency by buffering or waiting for network resources to become available. However, consider the case where the data connection is oversaturated when a message is sent-the message may be queued behind other data being processed in an oversaturated situation, thereby introducing additional delay. It should be noted, however, that in many systems the latency measurements are random in nature, that is, a network with a latency of L may return a different latency measurement/ ₁ 、l ₂ 、l ₃ …, the measurement results are normally distributed around the average value of L.

Thus, in systems where network latency measurements are affected by variance, it is difficult to determine whether the latency at a first bandwidth input value is higher than the latency at a second bandwidth input value, at least because the difference between the two values may be due to variance. However, statistical techniques may be used to determine at a confidence level (e.g., 90%, 95%, 99%, or higher) that the variance of the delay measurement is caused by the change in the underlying delay signal rather than by the variance.

Returning to fig. 2, a set of delay measurements may be obtained for successively increasing bandwidth input values. It should be noted that the group need not be collected in monotonically increasing or decreasing order, and that it is sufficient for the group to include delay measurements at different bandwidth input values. Measurements may be collected over a short duration (e.g., hundreds or thousands of milliseconds) to reduce the chance and/or amount of change in network conditions as measurements are made. The measured delay values are analyzed using a statistical model. The statistical model may be abstracted into a bivariate graph representing delay on one axis and bandwidth input on a second axis to determine the correlation coefficient between the two variables. The correlation may be measured by calculating a pearson product moment correlation coefficient r for the set of collected delay values. If the correlation coefficient exceeds a threshold that indicates with 90%, 95%, 99%, etc. confidence that the measured delay increases as the bandwidth input increases, the system may determine that the network is oversaturated. It may be determined that the network is oversaturated at the highest bandwidth input in the set of measurements, but other bandwidth input measurements may be used.

Once the system has determined that the network is oversaturated at a particular bandwidth input value, the system may determine the network capacity by measuring the bandwidth output at that bandwidth input value. As described above, the bandwidth input value used may be the highest bandwidth input value or another suitable value that is determined to be oversaturated at least at a certain confidence level based on a correlation coefficient indicating an increase in delay relative to an increase in bandwidth input. In one embodiment, measuring bandwidth output at a bandwidth input value is accomplished by submitting data packets to the partner over the network at a bandwidth input rate (kb/sec), the packets having instructions for the partner to respond with corresponding data packets (e.g., echo packets). For example, the data packets may be numbered sequentially. The system may receive the response after a short period of time after transmission and then measure the received responseRate, and thus bandwidth output. Since it has been determined that the network connection is oversaturated with bandwidth input values, the measured data throughput is the capacity of the channel. In one embodiment, the system transmits n data packets of size m, where n x m is the amount of data determined to be sufficient to oversaturate the network and request the other party to reply with a corresponding data packet of the same size. The system may detect a first time t at which a first corresponding packet is received ₁ And the last time t of receipt of the last corresponding data packet _n . The bandwidth capacity can be calculated as:

network capacity

The system can generate a plurality of measurement results b ₁ 、b ₂ …, and uses various statistical techniques to determine network capacity. For example, the system may continue to collect measurement b _i Until the confidence interval of the network capacity is within a certain window size. Thus, in networks that are affected by a larger amount of variance, more measurements may be taken to ensure that the calculated network capacity is accurate.

Thus, in the manner described above, network capacity of the network connection may be obtained. As part of the oscillation complementary network characteristic calibration, the system in turn calibrates network delays based on bandwidth. The system may obtain a set of bandwidth measurements that are collected for samples of bandwidth input values that are less than the network capacity. The system may send data at a rate based on the bw_input and the counterpart may respond with the data or other data of the same size. The bandwidth output measurement may be measured in connection with determining network capacity as described above. Thus, a set of paired bandwidth output and bandwidth input values may be collected.

The set of bandwidth measurements may be collected for successively decreasing bandwidth input values and a statistical model modeling a relationship or correlation between the bandwidth input and the bandwidth output is generated. In one embodiment, a correlation coefficient is calculated for the group. If the correlation coefficient exceeds a threshold (e.g., indicating a strong positive correlation between the bandwidth output and the bandwidth input), the system may determine that the network bandwidth is undersaturated at the bandwidth input value. In other words, if the correlation coefficient is a large positive value (e.g., r≡1), this means that as the bandwidth input decreases, the bandwidth output also tends to decrease.

Once the system has determined that the network is undersaturated at a particular bandwidth input value, the system may determine the network delay by measuring the delay at the bandwidth input value. As described above, the bandwidth input value used may be the lowest bandwidth input value or another suitable value that is determined to be undersaturated at least at a certain confidence level based on a correlation coefficient indicating that the decrease in bandwidth output is due to a decrease in bandwidth input. In one embodiment, measuring network latency at a bandwidth input value is accomplished by submitting data on the network in conjunction with a data rate corresponding to the bandwidth input. For example, the system may submit n messages of size m, which corresponds to a bandwidth input rate determined to undersaturate the network. Metrics related to the transmission of one or more of these messages may then be collected. For example, metrics (which may include measured observations of various system characteristics) may be collected for the last (i.e., nth) message in the series of messages. Thus, the network latency can be described as:

network delay

Wherein the indicated time is the time of transmission of the last message and the time of receipt of a response to said message.

Once the system has calculated both network capacity and network latency, the system may determine the bandwidth-latency product with high confidence (e.g., as determined based on statistical confidence levels that may be set by the system or network administrator). Thus, the bandwidth-delay product can be described as:

bandwidth delay product bdp=b×d

After determining the bandwidth-delay product, the computer system may calibrate various computer-related functionalities based on the bandwidth-delay product. For example, in the context of multimedia streaming, such as those described elsewhere in this disclosure, a computer system may determine a bandwidth latency product of a network and indicate the network bandwidth latency product to a Content Delivery Network (CDN) or streaming service. A data provider (e.g., CDN) may receive the network bandwidth-latency product and utilize the bandwidth-latency product to determine how much data may be flown through the network at a given time before a client acknowledges receipt of the data. In a TCP-based connection, the bandwidth-delay product may be used to calibrate the amount of data sent over the connection per acknowledgement. The techniques described in connection with fig. 2 may be implemented by computing entities such as those described in connection with fig. 1.

It should be noted that some oscillation complementary network characteristic calibration techniques described in this disclosure discuss measuring network delay as a first step, which need not be the first step, as the techniques may be capable of being performed in different combinations and variations as appropriate for different implementations. For example, in one embodiment, the system may first measure the bandwidth output as a function of the bandwidth input, starting from an initial value that may be greater than the network capacity of the system, and perform steps 4-6 to determine the network delay, and then perform steps 1-3 to determine the network capacity. Various other suitable implementations exist and these steps may be applied in a cyclic manner: the bandwidth input is oscillated to alternately oversaturate and undersaturate the network connection to determine network capacity and network delay, respectively.

Fig. 3 shows an illustrative example of a graph 300 of relationships between different network parameters that may be used as part of determining a bandwidth-delay product in accordance with at least one embodiment. The diagram 300 may be used as part of modeling and/or statistical estimation techniques described elsewhere in this disclosure, such as in connection with fig. 1, 2, and 5. In one embodiment, lines (e.g., horizontal and diagonal lines) are models representing modeled relationships between network characteristics, and points represent illustrative examples of hypothetical sampled measurements. It should be noted that the locations of the points are not necessarily to scale and may be merely representative for clarity.

A first network characteristic relation 302 is shown in fig. 3, said first network characteristic relation 302 showing how a delay, also called delay time, round Trip Time (RTT), round Trip Delay (RTD) time, etc., relates to a specific network condition. In general, the first network characteristic relationship 302 shows that the delay is constant or approximately constant when the network is undersaturated. In embodiments where the network connection suffers random loss, the network may tilt slightly upward, even when undersaturated. The delay time measurements may be normally distributed around the modeled delay value. In one embodiment, when the bandwidth input is undersaturated, the measured RTT values (e.g., represented by the points shown in fig. 3) are normally distributed around RTT value L, where L represents the constant network delay expected when the network is not subject to buffering of data packets and other additional delays that may occur when the network connection is oversaturated. However, once the bandwidth input exceeds the network capacity, the round trip time may increase. These increases may be caused by various factors such as latency associated with queuing data in the routing device, storing data in the short-term memory network, and retrieving data from the short-term memory network. In some embodiments, excessive exceeding of network capacity may result in data being discarded, such as in the event of a queue, buffer, or other temporary storage resource exhaustion.

A second network characteristic relationship 304 is shown in fig. 3, the second network characteristic relationship 304 showing that the bandwidth output is related to a particular network condition, such as a bandwidth input. For example, in the case of an undersaturated network, the relationship of bandwidth input to bandwidth output may be described as or approximately described as follows:

bandwidth of a communication device _{Input device} Bandwidth = _{Output of}

It should be noted, however, that this relationship may assume that the random penalty in the system is zero. Sources of random loss may include random loss such as spectral interference, multipath interference/loss, ionospheric interference/loss, and/or variants thereof. When the transmission medium suffers from random loss, there may be a non-zero probability p that the receiver does not receive the data unit sent over the transmission medium (i.e., 1+.p > 0). To account for the random loss of the ratio p, the relationship between bandwidth input and bandwidth output can be refined to:

bandwidth of a communication device _{Input device} = (1-p) x bandwidth _{Output of} Wherein p is more than or equal to 0 and less than or equal to 1

Unless otherwise indicated, the random loss is assumed to be less than 1 (i.e., a system in which all data transmitted is not received by the receiver due to the random loss is generally not considered unless otherwise indicated).

In one embodiment, a challenge may exist around determining an inflection point, where an inflection point refers to a bandwidth input value where the network is just full capacity (e.g., not oversaturated and not undersaturated). Inflection points may also refer to operating points, bias points, or static points. Without prior knowledge of the bandwidth delay product, it may be difficult to determine the network delay based solely on measuring RTT values as a function of bandwidth input values. Considering the illustrative example in fig. 3-a system that relies solely on RTT measurements to determine network latency may observe that network latency is relatively flat in region 306, which includes the first seven measurements, and overestimates the bandwidth input value in terms of the bandwidth latency product. This can lead to performance degradation because bandwidth input can lead to oversaturation of the network and higher than optimal latency.

Fig. 4 shows an illustrative example of a graph 400 of relationships between different network parameters that may be used as part of determining a bandwidth-delay product in accordance with at least one embodiment. The graph 400 may be used as part of modeling and/or statistical estimation techniques described elsewhere in this disclosure, such as in connection with fig. 1, 2, and 5. In one embodiment, lines (e.g., horizontal and diagonal lines) are models representing modeled relationships between network characteristics, and points represent illustrative examples of hypothetical sampled measurements. It should be noted that the locations of the points are not necessarily to scale and may be merely representative for clarity.

Fig. 4 illustrates how techniques described in this disclosure (such as oscillation complementary network characteristic calibration) may be utilized to improve the accuracy of computing network parameters. The first graph 402 shows the steps for detecting a positive correlation of network characteristics and the second graph 404 shows the steps for measuring or calculating complementary network characteristics. In one embodiment, the system gathers measurements 406 of bandwidth output as a function of bandwidth input and uses statistical techniques such as regression analysis to detect the presence of positive correlation between bandwidth output and bandwidth input to determine network undersaturation. Based on this determination, the system may collect delayed samples 408 at points where the bandwidth input is undersaturated. Additionally, the system may collect measurements 410 of latency and detect that latency increases with bandwidth input. The increase in delay with bandwidth input may indicate network oversaturation and the system may collect samples 412 of bandwidth output to determine network capacity. In some embodiments, samples 408 and/or 412 may be a single measurement.

Fig. 5 shows an illustrative example of a process 500 for calculating a bandwidth-delay product for a network connection, according to one embodiment. It should be noted that calculating, estimating, or otherwise determining the bandwidth-delay product of a network is to overcome the problem explicitly present in the field of computer networks, which is necessarily deeply embedded in computer technology. Throughout this disclosure, there may be techniques described herein, such as those described in connection with implementations lacking non-technical and/or non-computer analogy described in accordance with fig. 5. Some or all of process 500 (or any other process described herein, or variations and/or combinations thereof) may be performed under control of one or more computer systems configured with computer-executable instructions, and may be implemented as code (e.g., computer-executable instructions, one or more computer programs, or one or more application programs) that is jointly executed on one or more processors via hardware, software, or a combination thereof. The code may be stored on a computer readable storage medium, for example, in the form of a computer program comprising a plurality of computer readable instructions executable by one or more processors. The computer readable storage medium may be a non-transitory computer readable medium. In some implementations, at least some of the computer readable instructions available to perform process 500 are not stored using only transitory signals (e.g., propagated transient electrical or electromagnetic transmissions). The non-transitory computer readable medium may include non-transitory data storage circuitry (e.g., buffers, caches, and queues) within the transceiver of the transitory signal.

In one embodiment, the system performing process 500 gathers 502 round trip time measurements based on bandwidth input. In one embodiment, the system selects an initial bandwidth input. In one embodiment, the initial bandwidth input is selected based on a previously determined bandwidth delay product point, an estimate of an expected bandwidth input value for network connection undersaturation, or the like. In one embodiment, the initial bandwidth input is initialized to begin with a null message, such as a TCP segment, containing a header including metadata (such as sender and receiver information), and a null data portion following the header. In one embodiment, the null message is a UDP datagram including a null body field. In some embodiments, such as where the protocol or format requires the data body field to have a size greater than zero, a minimum amount of data (e.g., 1 bit or 1 byte of data) may be included in the data body field. The set of round trip time measurements may include one or more measurements of RTT values and corresponding bandwidth input values. The collected values may increase continuously or may be collected in other orders. The set of values need not be stored, collected, or organized in a particular order.

In some embodiments, the number of measurements collected is manually determined by a system or network administrator based on statistical techniques in a programmatic manner to ensure that the number of measurements acquired is sufficient to determine a probability of a correlation coefficient exceeding a threshold, etc. The number of measurement results may be determined as a fixed number. The number of measurements may be determined based on the network delay or estimated network delay such that the collection is performed within an upper bound time or with a probability of being performed within a sufficiently high upper bound. The upper time limit may be determined based on being able to perform oscillations at a predetermined or dynamically determined rate (e.g., 50 oscillations per minute).

Once the measurements are collected, the system may use the measurements to calculate 504 correlation coefficients. Statistical techniques may be used to determine the correlation coefficient to determine the extent to which there is a linear fit or relationship between two variables, such as the measured RTT value and the bandwidth input value. Other techniques described elsewhere in this disclosure may be utilized, such as those discussed in connection with fig. 1 and 2. The correlation coefficient may be calculated as a value between-1 and 1.

The system may determine whether the correlation coefficient calculated in step 504 exceeds 506 a threshold. If the correlation coefficient does not exceed a threshold (e.g., indicating that there is a strong positive correlation between RTT and bandwidth input), then no sufficiently powerful statistical inference may be indicated that RTT increases with increasing bandwidth input relative to the value of bandwidth input for which the delay is collected. As a result of determining that the correlation coefficient does not exceed the threshold or probability, the system may increase 508 the bandwidth input value for another collection. Steps 502 through 506 may be repeated again for a different range of bandwidth input values that may have some overlap with previously collected measurements. Another correlation coefficient may be calculated based on the new set of measurements.

When the system determines that the correlation coefficient exceeds the threshold, the system may determine 510 network capacity using bandwidth input values based on those bandwidth input values from the collected measurements. In one embodiment, the system uses a bandwidth input value equal to the maximum bandwidth input value in the set. In one embodiment, the system uses a bandwidth input value that exceeds a maximum bandwidth input value. One or more samples of the bandwidth output may be collected at the selected bandwidth input value to determine the network capacity.

The system may collect 512 bandwidth output measurements based on the bandwidth input. The bandwidth input values may be selected based on previously collected bandwidth output values such that the input values selected for measurement do not overlap with those included in the set of measurements used to determine a positive correlation between RTT and bandwidth input. The measurements collected in step 512 may be used to calculate 514 a correlation coefficient indicating whether the bandwidth output is proportionally related to the bandwidth input. In other words, reducing the bandwidth input easily results in a reduction in bandwidth output. The system may determine whether the correlation coefficient exceeds 516 a threshold that may be selected using principles and techniques described elsewhere. If the measured correlation coefficient is not sufficiently positive, the system may reduce 518 the bandwidth input value and collect a new set of measurements. In some embodiments, measurements from a previous collection may be reused in a subsequent collection. Steps 512 through 516 may be repeated.

Once the system determines that the calculated linear correlation coefficient between the bandwidth input and the bandwidth output is sufficiently positive, the system may determine 520 a network delay by sampling one or more RTT values at the bandwidth input value at which the system has determined that there is a sufficiently high statistical confidence, i.e., that there is a positive correlation between the bandwidth output and the bandwidth input. The system may collect multiple samples and use statistical techniques to calculate network latency. Once the network capacity and network delay are known, the system may determine 522 a bandwidth delay product by multiplying the network capacity by a network delay value. The bandwidth-delay product may be used to calibrate the rate at which data is transmitted and/or received by a data stream, such as a multimedia stream, thereby increasing the amount of data transmitted. For multimedia streams, the result may be that the viewer experiences higher quality video and/or audio based on factors such as resolution, bit rate, frame rate, lost frames, buffering, etc.

Variations of the process described in connection with fig. 5 are also contemplated within the scope of the present disclosure. For example, the system may first calculate the network delay by performing steps 512 through 520 and then calculate the network capacity by performing steps 502 through 510. In some embodiments, various steps such as statistical analysis for determining correlation coefficients may be performed in parallel.

In some embodiments, the system performs continuous oscillation complementary network characteristic calibration by repeating steps 502 through 520 after determining the bandwidth-delay product such that the calculated bandwidth-delay product value is updated. In dynamic systems, the bandwidth-delay product may change over time, and thus, continuously performing an oscillating complementary network characteristic calibration may result in a more accurate measurement of network conditions.

It should be noted that while fig. 5 relates to an example process 500 for calculating a bandwidth-delay product, the described techniques are applicable to many situations where there are observable complementary network characteristics. More generally, the example process 500 may be adapted to situations where there are observable complementary system characteristics.

Fig. 6 shows an illustrative example of a process 600 for saturating a network connection according to one embodiment. It should be noted that saturating a network is necessary to overcome the problems explicitly present in the field of computer networks, which are necessarily deeply embedded in computer technology. Throughout this disclosure, there may be techniques described herein, such as those described in connection with implementations lacking non-technical and/or non-computer analogy described in accordance with fig. 6. Some or all of process 600 (or any other process described herein, or variations and/or combinations thereof) may be performed under control of one or more computer systems configured with computer-executable instructions, and may be implemented as code (e.g., computer-executable instructions, one or more computer programs, or one or more application programs) that is jointly executed on one or more processors via hardware, software, or a combination thereof. The code may be stored on a computer readable storage medium, for example, in the form of a computer program comprising a plurality of computer readable instructions executable by one or more processors. The computer readable storage medium may be a non-transitory computer readable medium. In some implementations, at least some of the computer readable instructions available to perform process 600 are not stored using only transitory signals (e.g., propagated transient electrical or electromagnetic transmissions). The non-transitory computer readable medium may include non-transitory data storage circuitry (e.g., buffers, caches, and queues) within the transceiver of the transitory signal.

The systems performing process 600 may be those described in connection with fig. 5. In one embodiment, the system detects 602 a positive correlation between latency and bandwidth input. In one embodiment, the correlation will exceed a threshold or probability. This step may be performed in accordance with those discussed in connection with fig. 1-5. In one embodiment, the system may determine 604 network capacity as a result of determining a positive correlation between latency and bandwidth input, such as by utilizing the techniques described in connection with fig. 1-5. In one embodiment, a positive correlation between latency and bandwidth input is an indication of network oversaturation.

The system may also detect 606 a positive correlation between the bandwidth output and the bandwidth input using the techniques described in connection with fig. 1-5, before or after determining the network delay. For example, the system may collect multiple measurements of bandwidth output at evenly spaced bandwidth input values. The positive correlation between the bandwidth output and the bandwidth input may be used to determine a bandwidth input value that the network is likely to be undersaturated, and the system may calculate 608 a network delay based on network conditions under which the network connection is undersaturated. The system may calculate 610 a bandwidth delay product based on the determined network capacity and network delay. In one embodiment, the system then saturates 612 the network by transmitting data over the network at a rate and/or amount based on the bandwidth-delay product.

Fig. 7 shows an illustrative example of a graph 700 of relationships between different network parameters that may be used as part of determining network characteristics in accordance with at least one embodiment. A first network characteristic relation 302 is shown in fig. 3, said first network characteristic relation 302 showing how a delay, also called delay time, round Trip Time (RTT), round Trip Delay (RTD) time, etc., relates to a specific network condition. For example, relationships 702 may be in accordance with those discussed in connection with FIG. 3.

Fig. 7 further illustrates how additional network parameters or characteristics, such as deterministic loss, network queuing capacity, etc., may be measured in accordance with at least one embodiment. In one embodiment, the second network characteristic relationship 704 shows a model of deterministic loss as a function of bandwidth input. Deterministic losses can be understood from network losses in the sense that they are a function of losses that can be determined based on factors that are controllable or manageable by the network entity. Deterministic losses can be contrasted with random losses, which can be caused by environmental factors such as spectral interference, multipath interference/losses, ionospheric interference/losses, and/or variants thereof. In one embodiment, one source of deterministic loss is due to dropped packets due to the inability of the network buffer to accommodate those packets. For example, if the system submits data that exceeds the capacity of the network, routing devices of the network, such as routers and switches, may begin buffering the exceeded data and sending the data when additional bandwidth becomes available (e.g., because other data has been transmitted). Such buffering or storage of network data along the network in the temporary storage structure of the routing device may be referred to as network queuing 706. Continuing with the example, if the routing device continues to receive data at a rate that exceeds its rate of routing or transmitting data, the cache and other types of temporary storage resources may be exhausted, at which point additional data received by the device may be discarded, or otherwise lost. It should be noted that content such as data may not be irrevocably lost because the content may be retransmitted, for example, based on the sender not receiving an acknowledgement from the intended recipient of the data. The intended recipient of the data may be able to determine from the packet header data.

When the queuing capacity of the routing device along the network has been exhausted, transmitting additional data may result in the data being discarded. The aforementioned devices along the network may or may not include the intended recipient device. In some embodiments, ghost 708 refers to modeling round trip time or delay as a function of bandwidth input. When the network is oversaturated and the network queuing capacity is exhausted, the delay of the received packets may be constant or nearly constant, as additional bandwidth is pushed through the network-possibly because additional data is simply discarded at some point along the network. However, such flatness may cause difficulties in determining network oversaturation based on techniques that rely on detecting a positive correlation between RTT and bandwidth input. In various implementations, RTT is modified or normalized to account for dropped packets-e.g., dropped packets may be credited to measurements by including a penalty that adjusts the RTT value to a higher value when collected RTT values are analyzed using statistical techniques such as regression analysis. Thus, even when the network connection is oversaturated and the network queuing capacity is exhausted, a positive correlation can be detected, at least because increasing the bandwidth input beyond that point would result in additional dropped packets, which would increase the penalty factor. The penalty can be modeled as follows:

RTT _{Is adjusted to} ＝RTT _{Measured by} +r.p, where 0<r≤1,0<P

Wherein RTT _{Is adjusted to} Refers to RTT value, RTT for determining correlation coefficient _{Measured by} Refers to the measured network delay, r refers to the data loss rate, and P refers to the penalty factor. In some embodiments, r refers to a deterministic loss rate and is modeled as zero or near zero when network queuing capacity is not exhausted. The techniques described in connection with fig. 7 may be utilized in connection with embodiments described elsewhere in this disclosure, such as those discussed in connection with fig. 1-6, 8, and 14. For example, in connection with fig. 2 and/or fig. 5, the step for measuring network latency may include the further step of adding a penalty factor greater than or equal to zero based on how much data loss is measured. In some embodiments, the deterministic penalty is modeled as zero or near zero when the network is undersaturated and/or oversaturated with available network queuing capacity. In some implementations, techniques for oscillation complementary network characteristic calibration (such as those discussed in connection with) can be used to measure latency and deterministic loss as bandwidth input varies.

Fig. 8 shows an illustrative example of identifying varying network conditions in accordance with at least one embodiment. As shown in the first graph 802, the bandwidth output may be represented as a function of the bandwidth input. The output function shown includes an underflow region and an overflow region. In the underflow region, the bandwidth output increases linearly with the bandwidth input. In the overflow area, the bandwidth output does not increase linearly, but is constrained by network conditions.

An underflow area may be associated with a random error condition. For example, the variance of the associated metric may have a gaussian distribution around the expected value or trend line. In the overflow state, the variance of the metric may change. For example, the variance may become non-gaussian or include significant non-random components.

Transition point 806 exists between the underflow and overflow regions and likewise exists between the random error state and the deterministic error state. As described herein, the network system performing the adjustment of the transmission parameters may find the location of the transition point 806 where the random error state transitions to a deterministic error state and vice versa. The transition point of the error state may correspond to the transition between the overflow and underflow states and thus may be used to identify a point at which overflow may begin to occur. In some embodiments, queuing may occur when the system enters an overflow state, as described above, but the capacity of the system queuing may eventually be exhausted as bandwidth input increases. Embodiments may oscillate within the queuing to prevent data loss or other consequences of queue overflow, or may take action to compensate for queue overflow, such as retransmitting lost data.

As shown in fig. 8, the transition point 808 of the second graph 804 may be different from the transition point 806 in the first graph 802. This may be caused, for example, by changing network conditions, such as external traffic, failed or repaired network components, and so forth. By performing an oscillation adjustment on the transmission parameters, the system can detect a shift in transition point 808 and can thereby also infer that the point at which the system starts to enter an overflow state has changed.

This continuous calibration process can be further understood from fig. 9 and 10. These figures illustrate examples of continuous calibration in accordance with one or more embodiments. Continuous calibration refers to an ongoing process in which calibration of the system is performed on a basis that is at least periodic or frequent with respect to the rate at which the state of the associated system can be changed.

Fig. 9 shows an illustrative example of continuous calibration in accordance with at least one embodiment. In graph 900, a system performing continuous calibration adjusts the value of calibrated parameter 908 over time. The adjustment may be performed continuously or periodically.

The value of the calibrated parameter 908 may be related to another metric. For example, the bandwidth input may be related to the network delay such that the network delay may be predicted to vary with the bandwidth input. However, the actual value of the network delay may exhibit variance with respect to the predicted value. This difference can be described as an error in prediction. The region boundary 906 demarcates a deterministic error region 902 and a random error region 904. In the deterministic error region 902, the difference between the predicted value and the actual value of the correlation metric exhibits a deterministic pattern. Similarly, in the random error region 904, the difference between the predicted value and the actual value of the correlation metric exhibits a random pattern. The boundary between the regions 902, 904 may indicate a boundary between operating conditions (such as between overflows or underflows).

As can be seen in graph 900, the actual boundaries 906 between these regions may change over time and may be difficult to predict. However, by oscillating the value of the calibrated parameter 908, boundaries may be tracked. For example, the calibrated parameter 908 may initially cause the system to be driven into a state where the relevant metric exhibits deterministic errors. This may correspond to an overflow condition. The system may adjust the parameter values downward until a random error is observed. At that time the system may be in an underflow state. At this point, the parameters may be adjusted upward so that the correlation metric again exhibits deterministic errors. In this way, the region boundary 906 may be estimated.

Fig. 10 shows further aspects of an illustrative example of continuous network calibration in accordance with at least one embodiment. Similar to fig. 9, the graph 1000 of fig. 10 shows how the region boundary 1006 between the deterministic error region 1002 and the random error region 1004 may change over time. Graph 1000 further shows how this boundary may exhibit abrupt shifts. These shifts may indicate changes in the operating state of the associated system.

To accommodate sudden and significant shifts in the region boundaries 1006 more quickly, a system employing the disclosed calibration techniques may further employ a search technique in which the system more aggressively searches for transition points between two regions. For example, at location "a", the system may determine to employ more aggressive searching behavior by making more significant adjustments to the calibrated parameter 1008 than were done during normal oscillation until the region boundary 1006 is found again at point B. At point B, the less aggressive oscillation mode is resumed. Likewise, similar adjustment processes in opposite directions may be repeated at points C and D.

In one embodiment, the system employs this seek behavior to begin after a threshold number of adjustments have been made without identifying a transition point. The number of adjustments may be obtained experimentally. In one embodiment, the adjustment made 5 to 10 times in oscillation mode may be described as a normal oscillation or a non-aggressive oscillation, wherein the adjustment is relatively small. If no boundary is found, the normal oscillation mode is followed by a more aggressive "seek" oscillation mode until the boundary is found again. Thus, in an implementation, the transmission of application data may be adjusted based on a number of prior adjustments that did not cause or identify a transition between states after being made.

As described above, these techniques may be employed by adjusting calibrated parameters during application data transmission rather than by employing separate probes. In embodiments, this may yield the dual benefit of eliminating the impact of network probing on the system, and permitting the system to respond more quickly to state changes.

The foregoing techniques may be employed to increase the utilization of a network in a computing environment. FIG. 11 shows an illustrative example of a computing environment 1100 that may transmit data in excess of surface network capacity in accordance with at least one embodiment. The computing environment 1100 may include a first computer system 1102 and a second computer system 1104 connected via a network, such as the internet. One or more routing devices 1106 may be utilized to connect first computer system 1102 with second computer system 1104. The first computer system 1102 and/or the second computer system 1104 may be configured in the manners described elsewhere in this disclosure, such as those described in connection with fig. 2 and 5. In one embodiment, the first computer system 1102 is a data provider computer system and the second computer is a data receiver computer system.

In one embodiment, routing device 1106 is a computer system, such as a router or switch, configured with executable code that, when executed by one or more processors of the device, causes the one or more processors to receive and transmit data. The routing device may include a table that encodes data to route the data across a network, such as the internet. Routing device 1106 may include memory allocated for temporary storage structures, such as queues 1108, but may be implemented using any suitable data structure, such as arrays, vectors, lists, stacks, etc. In one embodiment, a network such as the internet includes a plurality of routing devices, and queuing capacity may refer to the amount of temporary storage capacity that the routing devices of the network are collectively capable of storing.

The bandwidth capacity of the network may be determined by a computer system, such as the first computer system 1102, using techniques such as those described in connection with fig. 2. The queuing capacity of the network may be determined by a computer system, such as the first computer system 1102, using techniques such as those described in connection with fig. 7. In some embodiments, the system determines the amount of data that can be bursty across the network. A burst may include a transmission of an amount of data equal to the network capacity plus the queuing capacity. In one embodiment, the network capacity is exceeded but not at a rate that may cause the network queues to run out, which may cause the data to be discarded. More generally, techniques such as those described with respect to fig. 10 may be employed to perform continuous calibration based on the described oscillation techniques.

In one embodiment, the first computer system 1102 performs techniques described herein, such as oscillation complementary network characteristic calibration 1110, for determining network characteristics, such as network latency, network capacity, and network queuing capacity. The system may use this information to determine the amount of data to send. In some implementations, the data can be a variable bit rate multimedia stream 1112 whose bit rate is determined based on a bandwidth delay product determined by an oscillating complementary network characteristic calibration. In one embodiment, the data includes multimedia content. The amount of data transmitted may be greater than the surface network capacity and the amount of excess surface network capacity is less than the amount of queuing capacity on the network. As the network connection saturates, additional data is temporarily stored in one or more queues by the routing device, which is transmitted when network capacity becomes available again. In this way, in some embodiments, the first computer system 1102 determines the rate at which data is transmitted beyond the surface network capacity, transmits the data at the determined rate, and the second computer system 1104 receives all of the data transmitted without losing any data. This technique relies on continuous or periodic calibration, without which the actual (as opposed to surface) network capacity may be exceeded.

The foregoing techniques may be employed to improve network transmission in various transmission modes. Examples of such transmission modes include one-to-one, one-to-many, and many-to-one. Fig. 11 provides an example of one-to-one communication. An example of a one-to-many transmission mode is an example of a single source transmitting data to multiple recipients. For example, video or game streaming services may transmit audiovisual data from a single source to a large number of clients. The techniques described herein may be employed to continuously calibrate various parameters related to transmission from a service to a client, such as parameters related to the quality of audio or video, the maximum number of participants, and so on. With respect to many-to-one transmissions, examples include server applications that receive data from various clients. It will be appreciated that the examples given are intended to be illustrative and, thus, should not be construed as limiting the scope of the disclosure to only those implementations practicing the particular examples provided.

Fig. 12 shows an illustrative example of a process for continuously calibrating a network system in accordance with at least one embodiment. Although fig. 12 is depicted as a certain order of steps, the depicted order should not be construed as limiting the scope of the disclosure to only those embodiments that conform to the depicted order. For example, unless indicated otherwise or clear from the context (e.g., when the output of one step is used as the input to another step), at least some of the depicted steps may be reordered or performed in parallel.

The example process 1200 may be performed by any of a variety of systems and devices, such as those shown in fig. 1, 11, and 14. Suitable systems and devices include network-connected computing devices, network routers, hubs, and other suitably equipped data transfer devices.

At 1202, the system or apparatus transmits application data over a network according to transmission parameters. The transmission parameter may be associated with a first network characteristic of a pair of complementary network characteristics and is typically selected based on the ability of the transmission parameter to affect this characteristic. The second characteristic in the pair may be tracked by monitoring a metric associated with (e.g., indicative of) the second characteristic. In fig. 12, this is depicted by step 1204.

For example, the transmission parameter may be a system or device characteristic that affects consumption of the output bandwidth. The system or device may then search for transition points between the random error state and the deterministic error state using network characteristics such as the input bandwidth that are complementary to the output bandwidth.

At 1206, the system or device identifies an error state of the metric. For example, a system or device may predict a value of a metric based on a relationship between complementary network characteristics. The error may be calculated based on a deviation between the observed value and the predicted value of the metric. The distribution of these errors may then be analyzed to determine whether the errors are random or deterministic.

In an embodiment, a system or device may identify an underflow or overflow condition based on identifying an error condition. For example, the system or device may infer from the identification of the random error state that the system or device is in an underflow state. Similarly, the system or device may infer from the identification of a deterministic error state that the system or device is in an overflow state. As described herein, with respect to fig. 8, for example, there may be a correlation between a random error state and an underflow state, and a correlation between a deterministic error state and an overflow state.

At 1208, the system or device adjusts the transmission parameters to cause oscillations between the random error state and the deterministic error state of the metric. While in the random error state, the system or device may adjust the transmission parameters so that the system searches for a deterministic state. Also, while in a deterministic error state, the system or device may adjust transmission parameters so that the system searches for a random error state.

In an embodiment, the system may periodically cause the parameter to oscillate. The use of the term period should not be construed herein to imply that the oscillation period between the random state and the deterministic state occurs at a fixed period, but that the system adjusts the transmission parameters on at least a semi-regular basis. In some embodiments, the adjustment may be made on an effectively continuous basis. It should be noted that the system and apparatus may use adjustments to the transmission of application data to detect state changes, rather than involving the use of probes or pings.

At 1210, the system or device identifies transition points between random error states and deterministic error states of the metrics. The system or apparatus analyzes the distribution of errors, such as the difference between the predicted and actual values of the metric, and determines whether the pattern or distribution of errors is random or deterministic.

In an embodiment, the system or apparatus infers relevant characteristics based on the identified transition points. For example, the system or apparatus may infer the bandwidth-delay product as described with respect to fig. 2 and 3.

In an embodiment, the system or device deduces a transition point between an underflow state and an overflow state from the identification of the transition point. For example, fig. 8 depicts how a system or device may infer a point at which a network will exhibit underflow or overflow, even in the event that changing network conditions cause the point to shift.

At 1212, the system or device adjusts transmission of the application data based at least in part on the identification of the transition point between the random error state and the deterministic error state.

In an embodiment, based on the identification of the transition point, the system or device may exceed a predefined threshold for the transmission parameter. For example, assume that the system can find points through the oscillation process where overflow tends to occur and broadcast data at a rate sufficient to saturate the network without causing problematic overflow amounts. However, due to oscillations of network parameters, the system may periodically cause a small amount of overflow to occur. In an embodiment, the system may safely exceed a predefined threshold, such as those imposed by the transmission control protocol ("TCP") standard.

FIG. 13 shows an illustrative example of a process for determining an operational metric by oscillation of an operational parameter in accordance with at least one embodiment. Although fig. 13 is depicted as a certain order of steps, the depicted order should not be construed as limiting the scope of the disclosure to only those embodiments that conform to the depicted order. For example, unless indicated otherwise or clear from the context (e.g., when the output of one step is used as the input to another step), at least some of the depicted steps may be reordered or performed in parallel.

The example process 1300 may be performed by any of a variety of systems and devices, such as those shown in fig. 1, 11, and 14. Suitable systems and devices include network-connected computing devices, network routers, hubs, and other suitably equipped data transfer devices.

At 1302, a system or device operates according to a parameter. The selected parameter is associated with at least one of a pair of complementary characteristics such that operation of the adjustment device causes a change in the characteristics. Thus, the expected range of values of the complementary property may also be affected by the parameter.

At 1304, the system or device identifies an error state of a metric associated with an operation of the system and the complementary characteristic. In other words, the collected metrics are metrics that enable determination of the error distribution of the complementary network characteristics. Analysis of the error state includes determining whether the error distribution is random or deterministic.

At 1306, the system or device responds to identifying the random error condition by adjusting the parameter in a first direction. The direction is selected so that the system searches for a boundary between a random error state and a deterministic error state. For example, if the system is in an underflow state associated with random errors, increasing the output rate of the system may tend to move the system to an overflow state associated with deterministic errors.

At 1308, the system or device responds to identifying a deterministic error state by adjusting a parameter in a second direction. The second direction is opposite to the first direction. As used herein, the terms and phrases opposite, opposite or opposite … refer to the adjustment being generally or substantially in a direction opposite the first direction. For example, a system that makes a slow upward adjustment of a parameter while in a random error state and a significant downward adjustment of a parameter while in a deterministic error state would still be considered to use an opposite direction adjustment.

At 1310, the system or device identifies transition points between the random error states and deterministic error states of the metrics using various analysis techniques described herein.

At 1312, the system or device responds to the identification of the transition point. In an embodiment, one such response is to infer, calculate, or otherwise determine a value of a correlation characteristic. For example, in a network system, a bandwidth-delay product of the network may be determined based on the identified transition points.

Embodiments may also respond to the identification of the transition point by adjusting the operation of the system. For example, based on the dynamic understanding of the operating conditions provided by the oscillation analysis, the system may determine that a predefined threshold for the parameter is exceeded.

Fig. 14 illustrates aspects of an example system 1400 for implementing the aspects in accordance with one embodiment. As will be appreciated, although a web-based system is used for purposes of explanation, different systems may be used as appropriate to implement the various embodiments. In one embodiment, the system includes an electronic client device 1402, the electronic client device 1402 including any suitable device operable to send and/or receive requests, messages, or information over a suitable network 1404 and communicate the information back to a user of the device. Examples of such client devices include personal computers, cellular or other mobile phones, handheld messaging devices, laptop computers, tablet computers, set-top boxes, personal data assistants, embedded computer systems, electronic book readers, and the like. In one embodiment, the network comprises any suitable network, including an intranet, the internet, a cellular network, a local area network, a satellite network, or any other such network and/or combination thereof, and the components for such a system depend at least in part on the type of network and/or system selected. Many protocols and components for communicating via such networks are well known and will not be discussed in detail herein. In one embodiment, communication over the network is achieved through wired and/or wireless connections, and combinations thereof. In one embodiment, the network comprises the Internet and/or other publicly addressable communication networks, as the system includes a web server 1406 for receiving requests and providing content in response to the requests, but alternative means of serving a similar purpose may be used for other networks, as will be apparent to those of ordinary skill in the art. In one embodiment, web server 1406 is an endpoint of a communication sent from electronic client device 1402. Optionally, the application server 1408 is an endpoint. Thus, the observed network characteristics may describe network conditions of a path between the electronic client device 1402 and the communication endpoint.

In one implementation, the electronic client device 1402 performs network characteristic calibration 1418 in accordance with the techniques described in this disclosure. For example, an application executing on the electronic client device 1402 may send application data using the oscillation techniques described above and thereby perform continuous, semi-continuous, or occasional monitoring of network conditions.

In one embodiment, the illustrative system includes at least one application server 1408 and data storage area 1410, and it should be appreciated that there may be several application servers, layers or other elements, processes or components that may be linked or otherwise configured that may interact to perform tasks such as obtaining data from an appropriate data storage area. In one embodiment, the server is implemented as a hardware device, a virtual computer system, a programming module executing on a computer system, and/or other device configured with hardware and/or software to receive and respond to communications over a network (e.g., web service Application Programming Interface (API) requests). As used herein, unless specified otherwise or clear from context, the term "data store" refers to any device or combination of devices capable of storing, accessing, and retrieving data, which may include any combination and any number of data servers, databases, data storage devices, and data storage media in any standard, distributed, virtual, or clustered system. In one embodiment, the data store communicates with a block-level and/or object-level interface. The application server may include any suitable hardware, software, and firmware for integrating with the data store as needed to execute aspects of one or more applications of the client device to handle some or all of the application's data access and business logic.

In one embodiment, an application server cooperates with a data store to provide access control services and generate content including, but not limited to, text, graphics, audio, video, and/or other content provided by a web server to a user associated with a client device in the form of hypertext markup language ("HTML"), extensible markup language ("XML"), javaScript, cascading style sheets ("CSS"), javaScript object notation (JSON), and/or another suitable client-side structured language or other structured language. In one embodiment, the content transmitted to the client device is processed by the client device to provide the content in one or more forms, including but not limited to forms that a user may perceive through hearing, vision, and/or through other sensations. In one embodiment, the handling of all requests and responses and content delivery between client device 1402 and application server 1408 is used by the web server in this example to use PHP: hypertext preprocessor ("PHP"), python, ruby, perl, java, HTML, XML, JSON, and/or another suitable server-side structured language. In one embodiment, operations described herein as being performed by a single device are performed jointly by multiple devices forming a distributed and/or virtual system. In one embodiment, client device 1402 and application server 1408 communicate via network 1404. The client device 1402 and/or the application server 1408 may determine the bandwidth-delay product of the network connection between the client device 1402 and the application server 1408 by performing oscillation complementary network characteristic calibration using the techniques described in connection with fig. 1, 5, and 6. The bandwidth-delay product may indicate the amount of data that may be transmitted across the network without increasing the delay of the data transmitted across the network 1404. In one embodiment, the client device 1402 calculates a bandwidth-delay product and transmits and/or receives an amount or ratio of data 1420 based on the determined bandwidth-delay product. In one embodiment, the data 1420 is a multimedia segment of a multimedia stream that is transmitted from the application server 1408 to the client device 1402 according to a bit rate that does not exceed the bandwidth-delay product.

In one embodiment, data storage area 1410 includes several separate data tables, databases, data documents, dynamic data storage schemes, and/or other data storage mechanisms and media for storing data related to certain aspects of the present disclosure. In one embodiment, the illustrated data store includes a mechanism for storing production data 1412 and user information 1416 for providing content to the production side. The data store is also shown to include a mechanism for storing log data 1414, which in one embodiment is used for reporting, computing resource management, analysis, or other such purposes. In one embodiment, other aspects such as page image information and access rights information (e.g., access control policies or other rights codes) are stored in the data store, optionally in any of the mechanisms listed above, or in another mechanism in the data store 1410.

In one embodiment, the data store 1410 is operable by logic associated therewith to receive instructions from the application server 1408 and obtain, update, or otherwise process data in response to the instructions, and the application server 1408 provides static data, dynamic data, or a combination of static and dynamic data in response to the received instructions. In one embodiment, dynamic data, such as data used in web logs (blogs), shopping applications, news services, and other such applications, is generated by a server-side structured language as described herein or provided by a content management system ("CMS") operating on or under the control of an application server. In one embodiment, a user submits a search request for a certain type of item through a device operated by the user. In this example, the data store accesses user information to verify the identity of the user, accesses directory details to obtain information about the type of item, and returns the information to the user, such as in the form of a list of results on a web page that the user views through a browser on the user device 1402. Continuing with the example, information for a particular item of interest is viewed in a dedicated page or window of the browser. It should be noted, however, that embodiments of the present disclosure are not necessarily limited to the context of web pages, but rather are more generally applicable to processing requests in a general manner, where the request is not necessarily a request for content. Exemplary requests include requests to manage and/or interact with computing resources hosted by system 1400 and/or another system (such as for starting, terminating, deleting, modifying, reading, and/or otherwise accessing such computing resources).

In one embodiment, each server generally includes an operating system that provides executable program instructions for general management and operation of the server, and includes a computer readable storage medium (e.g., hard disk, random access memory, read only memory, etc.) that stores instructions that, if executed by a processor of the server (i.e., as a result of being executed), cause the server or otherwise allow the server to perform its intended functions.

In one embodiment, system 1400 is a distributed and/or virtual computing system that utilizes several computer systems and components that are interconnected via communication links (e.g., transmission Control Protocol (TCP) connections and/or Transport Layer Security (TLS) or other cryptographically secured communication sessions) using one or more computer networks or direct connections. However, those of ordinary skill in the art will appreciate that such a system may operate in a system having a fewer or greater number of components than shown in FIG. 14. Accordingly, the depiction of system 1400 in fig. 14 should be taken as illustrative in nature and not limiting the scope of the disclosure.

The various implementations may also be implemented in a wide variety of operating environments that, in some cases, may include one or more user computers, computing devices, or processing devices that may be used to operate any of a plurality of applications. In one implementation, a user or client device includes any of the following: a plurality of computers, such as desktop, laptop or tablet computers running a standard operating system, and cellular (mobile), wireless and handheld devices running mobile software and capable of supporting a variety of networking protocols and messaging protocols, and such systems also include a plurality of workstations running any of a variety of commercially available operating systems and other known applications for purposes such as development and database management. In one embodiment, these devices also include other electronic devices such as virtual terminals, thin clients, gaming systems, and other devices capable of communicating over a network, as well as virtual devices such as virtual machines, hypervisors, and other virtual or non-virtual devices that utilize operating system level virtualization software containers and support virtualization capable of communicating over a network.

In one embodiment, the system utilizes at least one network, which may be familiar to those skilled in the art, to support communications using any of a variety of commercially available protocols, such as transmission control protocol/internet protocol ("TCP/IP"), user datagram protocol ("UDP"), protocols operating in layers of the open systems interconnection ("OSI") model, file transfer protocol ("FTP"), universal plug and play ("UpnP"), network file system ("NFS"), common internet file system ("CIFS"), and other protocols. In one embodiment, the network is a local area network, wide area network, virtual private network, the internet, an intranet, an extranet, a public switched telephone network, an infrared network, a wireless network, a satellite network, and any combination thereof. In one embodiment, a connection-oriented protocol is used to communicate between network endpoints such that the connection-oriented protocol (sometimes referred to as a connection-based protocol) is capable of streaming data in an ordered stream. In one embodiment, the connection-oriented protocol may be reliable or unreliable. For example, the TCP protocol is a reliable connection-oriented protocol. Asynchronous transfer mode ("ATM") and frame relay are unreliable connection-oriented protocols. Connection-oriented protocols are in contrast to packet-oriented protocols (such as UDP) that transport packets without guaranteeing ordering.

In one embodiment, computing devices such as the depicted electronic client device 1402, web server 1406, and/or application server 1408 may include hardware elements electrically coupled via buses, including, for example, at least one central processing unit ("CPU" or "processor"), at least one input device (e.g., mouse, keyboard, controller, touch screen, or keypad), at least one output device (e.g., display device, printer, or speaker), at least one storage device (such as a hard disk drive, optical storage device, and solid state storage device (such as random access memory ("RAM") or read-only memory ("ROM")), as well as removable media devices, memory cards, flash memory cards, and the like, as well as various combinations thereof.

In one embodiment, such an apparatus further comprises a computer readable storage medium reader, a communication device (e.g., modem, network card (wireless or wired), infrared communication device, etc.), and a working memory as described above, wherein the computer readable storage medium reader is connected to or configured to receive a computer readable storage medium representing a remote, local, fixed, and/or removable storage device, as well as a storage medium for temporarily and/or more permanently containing, storing, transmitting, and retrieving computer readable information. In one embodiment, the system and various devices also typically include a plurality of software applications, modules, services, or other elements located within at least one working memory device, including an operating system and applications such as a client application or web browser. In one embodiment, custom hardware is used and/or specific elements are implemented in hardware, software (including portable software, such as applets), or both. In one embodiment, a connection to other computing devices such as a network input/output device is employed.

In one embodiment, storage media and computer-readable media for containing code or portions of code include any suitable medium known or used in the art, including storage media and communication media, such as but not limited to volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage and/or transmission of information, such as computer readable instructions, data structures, program modules, or other data, including RAM, ROM, electrically erasable programmable read-only memory ("EEPROM"), flash memory or other memory technology, compact disc read-only memory ("CD-ROM"), digital Versatile Discs (DVD) or other optical storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices, or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a system device. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will appreciate other ways and/or methods of implementing the various embodiments.

The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the claims that follow.

Other variations are also within the spirit of the present disclosure. Thus, while the disclosed technology is susceptible to various modifications and alternative constructions, specific embodiments thereof have been shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the invention to the specific form or forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention as defined in the appended claims.

Additionally, embodiments of the present disclosure may be described in view of the following clauses:

1. a system, the system comprising:

at least one processor; and

a memory storing computer-executable instructions that, in response to execution by the at least one processor, cause the system to:

transmitting application data over a network, wherein the system transmits the application data by oscillating at least a transmission parameter;

monitoring network metrics associated with the transmission parameters;

determining a transition point of the network metric between a deterministic error state and a random error state; and

The transmission of the application data is adjusted based at least in part on the identified transition point.

2. The system of clause 1, wherein the identification of the transition point permits the transmission of the application data to be adjusted to exceed a predefined threshold.

3. The system of clauses 1 or 2, wherein the memory stores further computer-executable instructions that, in response to execution by the at least one processor, cause the system to infer a value of a network characteristic based at least in part on the transition point.

4. The system of any of clauses 1-3, wherein the memory stores further computer-executable instructions that, in response to execution by the at least one processor, cause the system to identify an underflow state, the identifying based at least in part on a correlation between the underflow state and the random error state.

5. The system of any of clauses 1-4, wherein the memory stores further computer-executable instructions that, in response to execution by the at least one processor, cause the system to identify an overflow state, the identifying based at least in part on a correlation between the overflow state and the deterministic error state.

6. A computer-implemented method, the computer-implemented method comprising:

transmitting application data, wherein the transmitting includes oscillating transmission parameters;

monitoring a metric associated with the transmission parameter;

identifying transition points of the metric between deterministic error states and random error states; and

7. The computer-implemented method of clause 6, wherein oscillating the transmission parameter comprises at least periodically adjusting the transmission parameter.

8. The computer-implemented method of clause 6 or 7, wherein adjusting the transmission of the application data comprises changing a rate at which the application data is transmitted.

9. The computer-implemented method of any of clauses 6 to 8, further comprising: the method further includes determining, based at least in part on identifying the transition point, to transmit the application data when a predefined threshold for the transmission parameter is exceeded.

10. The computer-implemented method of any of clauses 6 to 9, further comprising: a network state is inferred based at least in part on the identified transition point.

11. The computer-implemented method of any of clauses 6 to 10, further comprising: an underflow state is identified, the underflow state being identified based at least in part on a correlation between the underflow state and the random error state.

12. The computer-implemented method of any of clauses 6 to 11, further comprising: an overflow state is identified, the overflow state being identified based at least in part on a correlation between the overflow state and the deterministic error state.

13. The computer-implemented method of any of clauses 6-12, wherein oscillating the transmission parameter comprises adjusting the transmission parameter to trend the metric toward the deterministic error state based on identifying a random error state, and adjusting the transmission parameter to trend the metric toward a random error state based on identifying a deterministic error state.

14. The computer-implemented method of any of clauses 6 to 13, wherein the transmission of the application data is adjusted based at least in part on prior adjustments that did not cause a transition between states after proceeding.

15. A non-transitory computer-readable storage medium comprising executable instructions that, upon execution by one or more processors of a computer system, cause the computer system to at least:

transmitting the application data according to the transmission parameters;

monitoring a metric associated with the transmission parameter; and

the transmission of the application data is adjusted according to the transmission parameters, wherein the transmission is adjusted to oscillate the metric between a deterministic error state and a random error state.

16. The non-transitory computer-readable storage medium of clause 15, wherein oscillating the transmission parameter comprises at least periodic adjustment of the transmission parameter.

17. The non-transitory computer-readable storage medium of clauses 15 or 16, wherein the instructions comprise further instructions that, as a result of being executed by the one or more processors, cause the computer system to at least increase consumption of output bandwidth until the metric enters the deterministic error state.

18. The non-transitory computer-readable storage medium of any one of clauses 15-17, wherein the instructions comprise further instructions that, as a result of being executed by the one or more processors, cause the computer system to determine to transmit the application data at least when a predefined threshold for the transmission parameter is exceeded based at least in part on identifying a transition point.

19. The non-transitory computer-readable storage medium of any one of clauses 15-18, wherein the instructions comprise further instructions that, as a result of being executed by the one or more processors, cause the computer system to at least identify an underflow state, the underflow state being identified based at least in part on a correlation between the underflow state and the random error state.

20. The non-transitory computer-readable storage medium of any one of clauses 15-19, wherein the instructions comprise further instructions that, as a result of being executed by the one or more processors, cause the computer system to at least identify an overflow state, the overflow state being identified based at least in part on a correlation between the overflow state and the deterministic error state.

21. A system, the system comprising:

at least one processor; and

transmitting data through a network according to the transmission parameters;

identifying an error state of a network metric associated with the transmission parameter;

In response to determining that the error state of the network metric is random, making a first adjustment to the transmission parameter;

responsive to determining that the error state of the network metric is deterministic, making a second adjustment to the transmission parameter, the second adjustment being in a direction opposite the first adjustment;

determining a transition point of the network metric between a random error state and a deterministic error state; and

responsive to said identification of said transition point.

22. The system of clause 21, wherein the memory stores further computer-executable instructions that, in response to execution by the at least one processor, cause the system to alternate between searching the random error state and searching the deterministic error state.

23. The system of clause 21 or 22, wherein the random error state comprises a random distribution of the network metrics.

24. The system of any of clauses 21 to 23, wherein the memory stores further computer-executable instructions that, in response to execution by the at least one processor, cause the system to determine that the network metric is in the deterministic error state based at least in part on identifying a non-random component of error.

25. The system of any one of clauses 21 to 24, wherein the amount of the first adjustment is based at least in part on a prior adjustment that did not identify the transition point after the performance.

26. The system of any one of clauses 21 to 25, wherein the system continuously oscillates the metric between the random error state and the deterministic error state.

27. A computer-implemented method, the computer-implemented method comprising:

transmitting data according to the parameters;

responding to a random error state of a metric associated with the transmission of the data by making at least a first adjustment to the parameter;

responding to a deterministic error state of the metric by making at least a second adjustment to the parameter, the second adjustment being in an opposite direction to the first adjustment;

identifying transition points of the metric between the deterministic error state and the random error state; and

responsive to said identification of said transition point.

28. The computer-implemented method of clause 27, wherein the data corresponds to application data.

29. The computer-implemented method of clause 27 or 28, further comprising: alternating between searching for the random error state and searching for the deterministic error state.

30. The computer-implemented method of any of clauses 27 to 29, further comprising: the random error is identified by identifying at least a random distribution of the metrics.

31. The computer-implemented method of any of clauses 27 to 30, further comprising: the deterministic error state is identified by identifying at least a non-random distribution of the metrics.

32. The computer-implemented method of any of clauses 27 to 31, wherein responding to the identification of the transition point comprises storing a record comprising a metric value inferred from the identification of the transition point.

33. The computer-implemented method of any of clauses 27 to 32, wherein responding to the identification of the transition point comprises alternating the direction of adjustment of the parameter.

34. The computer-implemented method of any of clauses 27 to 33, further comprising: a change in the transition point is identified based at least in part on the oscillation of the parameter.

35. A non-transitory computer-readable storage medium comprising executable instructions that, upon execution by one or more processors of a computer system, cause the computer system to at least:

Transmitting data through a network according to the transmission parameters;

responding to a random error state of a metric associated with the transmission by making at least a first adjustment to the transmission parameter;

responding to a deterministic error state of the metric by making at least a second adjustment to the transmission parameter, the second adjustment being opposite to the first adjustment;

responsive to said identification of said transition point.

36. The non-transitory computer readable storage medium of clause 35, wherein adjusting the parameter causes a change in the data transmission rate.

37. The non-transitory computer-readable storage medium of clause 35 or 36, wherein the instructions comprise further instructions that, as a result of being executed by the one or more processors, cause the computer system to alternate between at least searching for the random error state and searching for the deterministic error state.

38. The non-transitory computer-readable storage medium of any one of clauses 35-37, wherein responding to the identification of the transition point comprises determining a metric value based at least in part on the transition point.

39. The non-transitory computer-readable storage medium of any one of clauses 35-38, wherein responding to the identification of the transition point comprises alternating a direction of adjustment of the transmission parameter.

40. The non-transitory computer readable storage medium of any one of clauses 35-39, wherein the transition point is identified based at least in part on an oscillation of the transmission parameter.

41. A computer-implemented method comprising determining a bandwidth-delay product of a network connection between a first computing entity and a second computing entity by:

transmitting a first plurality of messages to measure a round trip time of the network connection based on a bandwidth input;

calculating a first correlation coefficient based on the first plurality of messages;

calculating a network capacity of the network connection based on the first correlation coefficient exceeding a first threshold;

transmitting a second plurality of messages to measure a bandwidth output of the network connection based on the bandwidth input;

calculating a second correlation coefficient based on the second plurality of messages;

calculating a network delay of the network connection based on the second correlation number exceeding a second threshold; and

The bandwidth delay product is calculated based on the network capacity and the network delay.

42. The computer-implemented method of clause 41, wherein the method further comprises: a data stream is transmitted across the network connection at a rate that oscillates about the bandwidth-delay product.

43. The computer-implemented method of clause 42, wherein the data stream comprises multimedia content.

44. The computer-implemented method of any of clauses 41 to 43, wherein the first plurality of messages and the second plurality of messages comprise application data.

45. A system, the system comprising:

one or more processors; and

a memory storing computer-executable instructions that, when executed, cause the one or more processors to:

determining a first linear correlation of round-trip data points during an overflow state of the system;

measuring bandwidth during the overflow state;

determining a second correlation of bandwidth data points during an underflow state of the system;

measuring a round trip time during the underflow state;

determining the bandwidth and the round trip time until a bandwidth delay product metric is selected; and is also provided with

Continuously oscillating between the overflow and underflow states of the system.

46. The system of clause 45, wherein the bandwidth-delay product metric is calculated over the order of a number of round trip times.

47. The system of clauses 45 or 46, wherein the memory stores further computer-executable instructions that, when executed, cause the one or more processors to:

collecting a plurality of measurements of bandwidth output corresponding to the overflow state and the underflow state; and

the bandwidth-delay product metric is determined by performing a statistical analysis on at least the plurality of measurements.

48. The system of clause 47, wherein the memory stores further computer-executable instructions that, when executed, cause the one or more processors to:

calculating confidence intervals of the plurality of measurement results; and

the confidence interval is determined to have a range that does not exceed a threshold width.

49. The system of any of clauses 45 to 48, wherein the memory stores further computer-executable instructions that, when executed, cause the one or more processors to:

Collecting a plurality of measurements, wherein each measurement of the plurality of measurements measures a round trip time value based on at least one of the overflow state or the underflow state; and

a statistical regression analysis is performed on the plurality of measurements to determine a positive correlation.

50. The system of clause 49, wherein the plurality of measurements includes measurements taken at different bandwidth input values.

51 the system of any of clauses 45-50, wherein the continuous oscillation causes the metric to exhibit oscillation between a random error and a deterministic error.

52. The system of any of clauses 45 to 51, wherein the memory stores further computer-executable instructions that, when executed, cause the one or more processors to obtain a single measurement of bandwidth output as a function of bandwidth input.

53. A non-transitory computer-readable storage medium comprising executable instructions that, upon execution by one or more processors of a computer system, cause the computer system to at least:

determining a first value of a second network characteristic based on a first plurality of measurements of the first network characteristic over a network connection, the second network characteristic being complementary to the first network characteristic;

Determining a second value of the first network characteristic based on a second plurality of measurements of the second network characteristic over the network connection; and

determining a third value of a third network characteristic of the network connection based on the first value and the second value; and

causing data to be transmitted via the network connection in accordance with the third value.

54. The non-transitory computer-readable storage medium of clause 53, wherein the first network characteristic is round trip time and the second network characteristic is bandwidth output.

The non-transitory computer-readable storage medium of clause 53 or 54, wherein the first network characteristic and the second network characteristic are complementary network characteristics.

56. The non-transitory computer readable storage medium of any one of clauses 53-55, wherein:

the first network characteristic is a round trip time;

the second network characteristic is a deterministic penalty; and is also provided with

The third network characteristic is queuing capacity associated with the network connection.

57. The non-transitory computer-readable storage medium of clause 56, wherein the instructions for determining the first value of the second network characteristic based on the first plurality of measurements of the first network characteristic over the network connection comprise instructions for: the instructions, when executed by the one or more processors, cause the computer system to apply a penalty factor to a measurement based on a loss rate associated with the measurement.

58 the non-transitory computer-readable storage medium of clause 57, wherein the loss rate is a deterministic loss rate.

59. The non-transitory computer-readable storage medium of any one of clauses 53-58, wherein the instructions comprise the instructions of: the instructions, upon execution by the one or more processors, further cause the computer system to update the third value by at least:

determining a fourth value of the second network characteristic based on a third plurality of measurements of the first network characteristic over the network connection;

determining a fifth value of the first network characteristic based on a fourth plurality of measurements of the second network characteristic over the network connection; and

the third value is updated based on the fourth value and the fifth value.

60. The non-transitory computer-readable storage medium of any one of clauses 53-59, wherein the network connection is undersaturated for the first plurality of measurements and oversaturated for the second plurality of measurements.

The use of the terms "a" and "an" and "the" and similar referents in the context of describing the disclosed embodiments (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. Similarly, the use of the term "or" should be interpreted to mean "and/or" unless clearly contradicted by context. Unless otherwise indicated, the terms "comprising," "having," "including," and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to"). When unmodified and referring to a physical connection, the term "connected" should be interpreted as partially or wholly contained within, attached to, or joined together, even if an intervening material is present. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. Unless otherwise indicated or contradicted by context, use of the term "set" (e.g., "set of items") or "subset" should be interpreted as a non-empty set comprising one or more members. In addition, unless indicated otherwise or contradicted by context, the term "subset" of a corresponding set does not necessarily refer to the proper subset of the corresponding set, but the subset and the corresponding set may be equal.

Unless specifically stated otherwise or clearly contradicted by context, a connective language such as a phrase in the form of "at least one of A, B, and C" or "at least one of A, B and C" (i.e., the same phrase with or without oxford comma) is otherwise understood in the context of normal use as the expression item, term, etc. may be: any non-empty subset of sets of a or B or C, a and B and C, or any set that contains at least one a, at least one B, or at least one C that is not contextually contradictory or otherwise precluded. For example, in the illustrative example of a set of three members, the connectivity phrases "A, B, and at least one of C" and "A, B, and at least one of C" refer to any of the following sets: { A }, { B }, { C }, { A, B }, { A, C }, { B, C }, { A, B, C }, and any set having { A }, { B } and/or { C } as a subset (e.g., a set having a plurality of "A") without explicit contradiction or contradiction with context. Thus, such connectivity language is not generally intended to imply that certain embodiments require the presence of at least one of a, at least one of B, and at least one of C, respectively. Similarly, phrases such as "A, B, or at least one of C" and "A, B, or at least one of C" are the same as "A, B, and at least one of C" and "A, B, and at least one of C" unless explicitly stated or clear from the context, refer to any one of the following sets: { A }, { B }, { C }, { A, B }, { A, C }, { B, C }, and { A, B, C }. In addition, unless indicated otherwise or contradicted by context, the term "plurality" indicates a state in which it is in multiple (e.g., the term "plurality of items" indicates multiple items). The number of the plurality of items is at least two, but may be more if explicitly indicated or indicated by context. Furthermore, unless otherwise indicated or clear from context, the phrase "based on" means "based at least in part on" rather than "based only on".

The operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. In one embodiment, a process such as the process described herein (or variations and/or combinations thereof) is performed under control of one or more computer systems configured with executable instructions and is implemented by hardware or combinations thereof as code (e.g., executable instructions, one or more computer programs, or one or more application programs) executing jointly on one or more processors. In one embodiment, the code is stored on a computer readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. In one embodiment, the computer-readable storage medium is a non-transitory computer-readable storage medium that does not include transitory signals (e.g., transient-propagating electrical or electromagnetic transmissions) but includes non-transitory data storage circuits (e.g., buffers, caches, and queues) within a transceiver of the transitory signals. In one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions that, when executed by one or more processors of a computer system (i.e., as a result of being executed), cause the computer system to perform operations described herein. In one embodiment, the set of non-transitory computer-readable storage media includes a plurality of non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media in the plurality of non-transitory computer-readable storage media do not contain all code, and the plurality of non-transitory computer-readable storage media collectively store all code. In one embodiment, the executable instructions are executed such that different instructions are executed by different processors, e.g., a non-transitory computer readable storage medium stores instructions, and the main CPU executes some instructions while the graphics processor unit executes other instructions. In one embodiment, different components of the computer system have separate processors, and different processors execute different subsets of the instructions.

Thus, in one embodiment, a computer system is configured to implement one or more services that individually or collectively perform the operations of the processes described herein, and such computer system is configured with suitable hardware and/or software that enable the operations to be performed. In addition, a computer system implementing embodiments of the present disclosure is a single device, and in another embodiment, a distributed computer system comprising a plurality of devices that operate in different ways such that the distributed computer system performs the operations described herein and such that a single device does not perform all of the operations.

The use of any and all examples, or exemplary language (e.g., "such as") provided herein, is intended merely to better illuminate embodiments of the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.

Embodiments of the present disclosure are described herein, including the best mode known to the inventors for carrying out the invention. Variations of those embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the embodiments of the disclosure to be practiced otherwise than as specifically described herein. Accordingly, the scope of the present disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, unless indicated otherwise or otherwise clearly contradicted by context, the scope of the present disclosure encompasses any combination of the above-described elements in all possible variations thereof.

All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

Claims

1. A computer-implemented method, the computer-implemented method comprising:

monitoring a metric associated with the transmission parameter;

identifying transition points of the metric between deterministic error states and random error states;

inferring a value of a network characteristic based at least in part on the transition point; and

a transmission parameter of the application data is adjusted to cause the metric to oscillate between the random error state and the deterministic error state based at least in part on the transition point and a value of the network characteristic.

2. The computer-implemented method of claim 1, wherein oscillating the transmission parameter comprises at least periodic adjustment of the transmission parameter.

3. The computer-implemented method of claim 1, wherein adjusting the transmission of the application data comprises changing a rate at which the application data is transmitted.

4. The computer-implemented method of claim 1, the computer-implemented method further comprising: the method further includes determining, based at least in part on identifying the transition point, to transmit the application data when a predefined threshold for the transmission parameter is exceeded.

5. The computer-implemented method of claim 1, the computer-implemented method further comprising: a network state is inferred based at least in part on the identified transition point.

6. The computer-implemented method of claim 1, the computer-implemented method further comprising: an underflow state is identified, the underflow state being identified based at least in part on a correlation between the underflow state and the random error state.

7. The computer-implemented method of claim 1, the computer-implemented method further comprising: an overflow state is identified, the overflow state being identified based at least in part on a correlation between the overflow state and the deterministic error state.

8. The computer-implemented method of claim 1, wherein oscillating the transmission parameter comprises adjusting the transmission parameter to trend the metric toward the deterministic error state based on identifying a random error state, and adjusting the transmission parameter to trend the metric toward a random error state based on identifying a deterministic error state.

9. The computer-implemented method of claim 1, wherein the transmission of the application data is adjusted based at least in part on a prior adjustment that did not cause a transition between states after proceeding.

10. A system, the system comprising:

one or more processors; and

a memory storing executable instructions that, as a result of being executed by the system, cause the system to at least:

monitoring network metrics associated with the transmission parameters;

a transmission parameter of the application data is adjusted to cause the network metric to oscillate between the random error state and the deterministic error state based at least in part on the transition point and a value of the network characteristic.

11. The system of claim 10, wherein oscillating the transmission parameter comprises at least periodically adjusting the transmission parameter.

12. The system of claim 10, wherein the instructions comprise further instructions that, as a result of execution by the one or more processors, cause the system to increase consumption of output bandwidth at least until the metric enters the deterministic error state.

13. The system of claim 10, wherein the instructions comprise further instructions that, as a result of being executed by the one or more processors, cause the system to at least determine to transmit the application data when a predefined threshold for the transmission parameter is exceeded based at least in part on identifying the transition point.

14. The system of claim 10, wherein the instructions comprise further instructions that, as a result of execution by the one or more processors, cause the system to at least identify an underflow state, the underflow state identified based at least in part on a correlation between the underflow state and the random error state.

15. The system of claim 10, wherein the instructions comprise further instructions that, as a result of execution by the one or more processors, cause the system to at least identify an overflow state, the overflow state identified based at least in part on a correlation between the overflow state and the deterministic error state.