WO2009043291A1

WO2009043291A1 - A method, a system and a stratrgy and control server for implementing stream media service

Info

Publication number: WO2009043291A1
Application number: PCT/CN2008/072523
Authority: WO
Inventors: Zhong Luo
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2007-09-26
Filing date: 2008-09-25
Publication date: 2009-04-09
Also published as: CN101399980B; CN101399980A

Abstract

A method, a system and a network equipment are provided for implementing stream media service. The invention belongs to the network communication technology field. The method includes: firstly, establishing a mathematics optimizing problem according to the user experience quantificational model of the stream media, and calculating it to determine each parameter in the user experience quantificational model; secondly, setting the control strategy of the stream server according to each the determined parameter; thirdly, the stream server reads the stream media data file according to the control strategy, transmits the media stream to the user terminals according to the read stream media data file, and provides stream media service for users. The embodiments of this invention are independent of the concrete representation form of the user experience quantificational model, therefore, the embodiments of this invention provide a universal implementing framework for stream media service. The embodiments of this invention can provide the optimization stream media user experience for users.

Description

Streaming media service implementation method, system and strategy and control server

The present invention relates to the field of network communication technologies, and specifically relates to a streaming media service implementation method, system, and policy and control server. Background of the invention

The streaming media transmits the multimedia data to the user terminal in a "stream" manner through the network, and the user terminal can receive the multimedia data, decode the multimedia data and other necessary processing, and play the same. The multimedia data here includes: video, audio, still images, animations, text, and more. The network here is like an IP network.

There are many different forms of networking for streaming media services. However, various networking forms include some basic elements. The following is a schematic diagram of the streaming media service networking shown in Figure 1 and the unicast shown in Figure 2. The multicast diagram illustrates the basic elements of the streaming media service networking.

In the schematic diagram of the streaming media service networking diagram in Figure 1, the basic elements of the streaming media service networking are as follows:

1. Streaming server and storage device. Multiple storage devices can form a storage array. The streaming media content data is stored as a file on the storage device. The streaming server reads the file from the storage device and sends it as a media stream. Referring to FIG. 2, in the case of Unicast, each media stream sent by the streaming server corresponds to one user terminal. In the case of unicast, the media stream sent by the streaming server can be called a unicast stream. In the case of multicast, each media stream sent by the stream server corresponds to multiple user terminals, that is, when multiple user terminals need to receive the same media stream, the media stream sent by the stream server reaches the backbone network. After the edge, the media stream is copied and sent to different user terminals by the stream replication node device, such as a DSLAM, or a router, or a BRAS device. In the case of multicast, the media stream sent by the stream server may be referred to as a multicast stream. The media stream sent by the replication node device may be referred to as a unicast stream. The streaming server in Figure 1 can be referred to as a central streaming server.

2. Edge stream server and storage device. For a large-scale streaming media system, multiple edge streaming servers are deployed at the edge of the backbone network, and the edge streaming server provides the nearest service to the user terminal. The edge stream server reads the file from its corresponding storage device and sends it as a media stream.

3. Portal server. The portal server provides a series of services such as content query, on-demand, and request for the user terminal. The real business logic is executed by the background management server (group). The portal server also provides an interface function for authenticating, authenticating, etc. of the user terminal, that is, the portal server obtains data required for authentication and authentication by interacting with the user terminal, and transmits the obtained data to the management server (group). The authentication and authentication of the user terminal is completed by the management server (group). The portal server is equivalent to the foreground of the streaming service.

4. Management server (group). A management server (group) can be a server group composed of multiple management servers. The functions provided by the management server (group) include: user management, authentication and authentication, network management, business logic control, billing management, and so on.

5. Strategy and control server. The policy and control server controls and manages the flow server. Policy and Control Servers You can control multiple stream servers simultaneously, including the Central Streaming Server and the Edge Streaming Server. The control policy is sent to the streaming server through the communication interface protocol between the server and the streaming server, and the actual situation data of the policy execution is obtained from the streaming server.

It should be noted that the functional entities described above are actually logical entities and do not represent specific Physical device. In practical applications, multiple logical functional entities can be implemented by one physical device. For example, in extreme cases, all servers except the edge streaming server can be implemented by one physical server.

In the current streaming service implementation process, the policy and control server adopts extensive control when formulating control policies for the streaming server. For example, when the network condition deteriorates, the streaming server or the policy and control server adjusts the encoding bit rate or frame rate of the media stream data file based on the empirical value. The current extensive control approach does not provide streaming services for the optimal user experience. Summary of the invention

The embodiment of the invention provides a streaming media service implementation method, system and policy and control server, which improves the user experience of the streaming media service.

An embodiment of the present invention provides a method for implementing a streaming media service, including:

The mathematical optimization problem is established according to the user experience quantitative model of the streaming media, and the solution is solved to determine various parameters in the user experience quantitative model;

Setting a control policy of the stream server according to the determined parameters;

The streaming server reads the streaming media data file according to the control policy, and sends a media stream to the user terminal according to the read streaming media data file to provide a streaming media service for the user.

An embodiment of the present invention further provides a streaming media service implementation system, where the system includes:

a storage device, configured to store a streaming media data file;

a strategy and control server, configured to solve a mathematical optimization problem established according to a quantitative model of the user experience of the streaming media, to determine a parameter in the user experience quantitative model, and set a control policy according to the parameter, and the control strategy is Transfer to the streaming server;

And a stream server, configured to read the streaming media data file from the storage device according to the received control policy, and send the media stream to the user terminal according to the read streaming media data file.

An embodiment of the present invention further provides a policy and control server, including:

Storage module: used to store mathematical optimization problems established according to a user experience quantitative model of streaming media;

Solving module: used to solve the mathematical optimization problem stored in the storage module to determine various parameters in the user experience quantitative model;

The transmission module is configured to set a control strategy according to each parameter in the user experience quantitative model obtained by solving the mathematical optimization problem by the solution module, and transmit the control strategy to the stream server. According to the description of the foregoing technical solution, the embodiment of the present invention implements a streaming media service based on a user experience quantitative model. Therefore, the embodiment of the present invention can provide an optimal user experience for a user terminal. The service implementation method is independent of the specific manifestation of the user experience quantitative model. Therefore, the embodiment of the present invention can be applied to various user experience quantitative models that appear at present and in the future, and is a technical solution for implementing streaming media services that can be widely applied. BRIEF DESCRIPTION OF THE DRAWINGS

1 is a schematic diagram of a prior art streaming media service networking; 2 is a schematic diagram of unicast and multicast in the prior art;

Figure 3 is a schematic diagram showing the principle of obtaining a packet loss rate;

Figure 4 is a schematic diagram showing the relationship between fecc and packet loss rate, and protection strength parameters;

Figure 5 is a schematic diagram of the working principle of error correction coding;

6 is a schematic diagram of a process of a flow server execution policy and control server according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a correspondence between a media stream and a user terminal according to an embodiment of the present invention. Mode for carrying out the invention

In the process of implementing the streaming media service, the control strategy provided by the policy and control server for the streaming server may include various strategies in various aspects, such as:

A. Bandwidth Policy Management: What kind of bandwidth is allocated to each media stream.

B. Video stream policy management: What parameters are provided for each video stream, such as encoding protocol, image size, frame rate, etc.

C. Audio stream policy management: What parameters are provided for each audio stream, such as encoding protocol, bit rate, etc. D. QoS/QoE (Quality of service/Quality of experience) Policy Management: Give the QoS/QoE management policy for each media stream.

From the perspective of serving users, the most important thing in the implementation of streaming services is to provide users with the best user experience. User experience directly affects whether users are willing to use streaming services. Therefore, the QoE strategy among the various strategies offered for streaming servers is the core strategy. How to provide an optimal QoE policy for streaming servers, so that users can get the best user experience is the key to implementing streaming media services. How to optimize the QoE policy provided for the streaming server is the key to the technical solution of the embodiment of the present invention. The policy and control server in embodiments of the present invention can provide an optimized QoE policy for the streaming server. The policy and control server determines the QoE policy based on the user experience quantitative model. The QoE strategy here includes variables such as: 1. Maximum allowable bandwidth of each media stream; 2. Video of each media stream, audio coding protocol; 3. Bit rate used; 4. Image size; 5. Frame rate; Priority; 7, error correction coding protection type and protection strength parameters. Embodiments of the present invention determine the variables in the QoE strategy by mathematically optimizing the problem of user experience quantitative model conversion.

The user experience quantitative model refers to: a functional relationship that associates the streaming media user experience quantized value with the media stream's various attribute parameters. The independent variable of the function relationship is the attribute parameter of the media stream, and the value of the function relationship is the quantified value of the streaming media user experience. For the video streaming service, various attribute parameters herein include but are not limited to: bit rate, frame rate, compression coding protocol, image format, key frame interval, error correction coding type, protection strength parameter; for audio streaming service Said, the various attribute parameters here include but are not limited to: bit rate, compression coding protocol, error correction coding type, protection strength parameter.

The embodiment of the present invention proposes a general streaming media service implementation framework based on the user experience quantitative model. The universal streaming media service implementation framework can be applied to various current user experience quantitative models, and can also be applied to later. User experience quantitative model. The embodiment of the present invention can provide an optimal user experience for the user through this universal streaming service implementation framework.

The user experience quantitative model can be a quantitative model for the video user experience, or a quantitative model for the audio user experience; when a user terminal receives both the video stream and the audio stream, the user experience quantitative model can also be a quantitative model for the user experience including video and audio. Type.

The video user experience quantitative model and the audio user experience quantitative model in the embodiments of the present invention are respectively described below by taking several specific user experience quantitative models as an example.

First, the video user experience quantitative model (QEV).

Video User Experience Quantitative Model QEV = f(vbr, fr, vcp, pf, kfi, ecct, eccp) ( 1 )

Where f(.) represents a function, and the parameters in the video user experience quantitative model are explained as follows:

Vbr is the video bit rate, such as 128 kbps, 384 kbps, etc., and the unit of vbr is bit/second or kilobit/second.

Fr is the frame rate, and the unit of fr is fps (frames/second), such as 15fps, 30φ _δ . In general applications, the highest frame rate is vcp for the video compression protocol (optional). There are various compression coding protocols, such as international standards H.261, H.263/H.263+/H.263++, H.264, MPEG-2, MPEG-4, Chinese Standard A VS, and proprietary protocols such as MSVC1 and so on.

Pf is the picture format. The image format determines the size of the image. The image formats are CIF, QCIF, SQCIF, 4CIF, VGA, XGA, Dl, 1080i, etc.

Kfi is the key frame interval, represented by the number of non-keyframes separated by two consecutive keyframes. Ecct is an error correction code type (or error control code type).

Eccp is the error correction coding parameter (error correction/control code parameters).

The video user experience quantitative model QEV can be further expressed as:

QEV = f (vbr, fr, vcp, pf, kfi, ecct, eccp) = fcp(vbr, fr, vcp, pf, kfi) + fecc(ecct, eccp) ( 2 ) where, & 01^, &, ¥ 0, 1", 1<; 1"1) represents an item related to video compression coding (0, compression), and fecc (ecct, eccp) represents an item related to error correction coding.

The following takes a specific form of expression as an example to describe fcp (vbr, fr, vcp, pf, kfi) and fecc (ecct, eccp) in the video user experience quantitative model QEV.

Currently, fcp(vbr, fr, vcp, pf, kfi) can be expressed as follows:

Fcp(v&r, fr, vcp, pf, kfi) = 1 + G(ybr, fr; vcp, pf, kfi) ( 3 )

(ln( r) - \n(ofr(fr; vcp, pf, kfi))

Where: G{ybr, fr) = a{ybr; vcp, pf, kfi) exp 2 (4)

2co{ybr, vcp, pf, kfi) vcp, pf, kfi) = a(vcp, pf, kfi) ——, Q < _a ( br; vcp, pf, kfi) < 4 (5 )

6(vcp,pf, kfi),

Ofr(ybr; vcp, pf, kfi) = d (vcp, pf, kfi) + e(vcp, pf, kfi) br, 1≤ ofri br; vcp, pf, kfi) < 30 ( 6) aX br; vcp , pf, kfi) = f (vcp, pf, kfi) + g(vcp, pf, kfi) br , (7) In the video user experience quantitative model shown in the above formulas (3), (4), (5), (6), and (7), a set of coefficients a, b, _C , d, e in the function are accompanied by Br, fr and fcp are related by changes in coding protocols, image formats, and keyframe intervals.

The video user experience quantitative model shown in the above formulas (3), (4), (5), (6), and (7) is merely a specific example exemplified for describing the embodiment of the present invention. Because the embodiment of the present invention is independent of the specific manifestation of the user experience quantitative model, the embodiments of the present invention can be applied to other various user experience quantitative models that exist at present, and can also be applied to various user experience quantitative models that may appear in the future. .

In practical applications, the video bit rate vbr is not continuously changed. Even with the so-called FGS (Fine Granularity Scalability) compression coding technique, vbr is not continuously changed. Fr, vcp, pf, and kfi are non-continuously variable in nature and are discrete variables. Therefore, the five variables involved in fcp(br, fr, cp, pf, kfi) can be discretized, and can be represented by the typical values commonly used by each variable. For the bit rate, discrete values such as 128bkps, 256bkps, 384kbps, 512kbps, 768kbps, 1024kbps, 1536kbps, 1920kbps, 2048kbps, etc. can be considered. It can be seen that fcp(vbr, fr, vcp, pf, kfi) can be represented by a 5-dimensional discrete table.

The analysis of the fecc (ecct, eccp) model in the embodiment of the present invention is as follows:

For a given type of error correction coding, such as RS (Reed-Solomon Code) code, BCH code, Fountain Code, Tornado Code, Raptor Code, etc., regardless of bandwidth Under the constraint, the higher the protection strength used, the better the user experience. For the protection strength, a set of parameters can be used to describe. For many error correction coding types, two parameters (n, m) can be used to represent the total number of nodes and the number of data nodes to jointly represent the protection strength performance. Among them, the total number of nodes = the number of data nodes + the number of check nodes. Embodiments of the invention may use K parameters to describe the protection strength.

For a given type of error correction coding, the higher the end-to-end packet loss rate, the worse the user experience. Therefore, in the fecc (ecct, eccp) model, the packet loss rate is a key data, and the packet loss rate is an external variable that is uncontrollable by the streaming server. However, the variable packet loss rate needs to be used in the control strategy provided for the streaming server. Therefore, the packet loss rate must be obtained in the embodiment of the present invention. The principle of obtaining the packet loss rate is shown in Figure 3.

In FIG. 3, the user terminal is a receiving end of the media stream, and the user terminal collects the number of lost packets and the total number of packets in a period of time, thereby obtaining an average packet loss rate during the period of time, and then, the user terminal passes the QoS reporting mechanism. The average packet loss rate is reported to the streaming server through the RTCP (Real-time Control Protocol) packet. The flow server reports the average packet loss rate reported by the user terminal to the policy and control server, and the policy and control server obtains the packet loss rate of the user terminal.

For a given type of error correction coding, fecc( _e cct, eccp) can be represented by a multidimensional table at different end-to-end packet loss rates. If the packet loss rate and the error correction coding type are also considered as dimensions, a K+2-dimensional table may be used to represent fecc(ecct, eccp), where K is the number of parameters describing eccp. Figure 4 shows the relationship between fecc and packet loss rate, and two parameters of the protection strength parameter. The fecc(ecct, eccp) shown in Fig. 4 is a two-dimensional table, in which eccpl and eccp2 are two parameters describing the strength of protection.

The error correction coding works as shown in Figure 5.

In Figure 5, the streaming server reads video or audio data, video and audio data from a video or audio file in the storage device. It can be stored in different files, or it can be stored in a common container file, such as AVI (Audio Video Interleaved) files. An AVI file can contain multiple video and audio streams at the same time. The streaming server can encode the multimedia stream through the error correction coding link as needed, and then package the video stream and the audio stream into a packet stream through the packet transmission link, and finally, send the packet stream to the network.

After the error correction coding is introduced, for the media stream data, although the protection capability is improved in the case of packet loss, the bit rate of the media stream data is increased. For a given error correction coding type and a given set of protection strength parameters, there is a scale factor kecc(ecct, eccp), that is, after error correction coding, the bit rate of the video stream is increased to the original kecc ( Ecct, eccp) times. Strictly speaking, compressed media stream data will be packaged and there will be additional overhead. This extra overhead is accurately calculated. For a fixed coding protocol and specific parameters such as bit rate, packet protocol (such as RTP, Real Time Protocol, real-time protocol), etc., a bit rate multiplication factor kpac(br, cp, pacp) can be determined, where the variable pacp Indicates the packetization protocol. Since this extra overhead is generally negligible, the embodiment of the present invention can ignore this scale factor and consider kpac(br, cp, pacp) to be equal to one.

Second, the Audio User Experience Quantitative Model (QEA).

Audio User Experience Quantitative Model QEA = g(abr,acp,ecct,eccp) ( 8)

Where g(.) represents a function, and the parameters in the quantitative model of the audio user experience are explained as follows:

Abr is the audio bit rate, such as 32kbps, 48kbps, 64kbps, 192kbps, etc., the unit of abr is bit

/second or kilobits/second.

Acp is an audio compression coding protocol. There are various compression coding protocols currently available, such as ITU-T G.711, G.722, G.723.K G.728, G.729, G. .729.K MPEG AAC, MP3 and IETF iLBC (Internet Low-BitRate Codec) and so on.

Ecct is an error correction code type (or error control code type).

The audio user experience quantitative model QEA can be further expressed as:

QEA = g(abr, acp, ecct, eccp) = gcp(abr, acp) + gecc(ecct, eccp) ( 9 ) where gcp(abr,acp) represents the item related to audio compression encoding, and gecc(ecct , eccp) represents an item related to error correction coding.

The audio user experience quantitative model adopted by the embodiment of the present invention mainly relies on two variables of the audio compression coding protocol acp and the bit rate abr. The different values of acp correspond to different audio compression coding protocols. Many current audio compression coding protocols support multiple bit rates, such as 32 kbps, 48 kbps, 64 kbps, 192 kbps, and the like. The audio stream improves user experience by using error correction coding. gecc(ecct, eccp) is similar to video fecc(ecct, eccp) and can also be represented by a multidimensional discrete table. Gecc (ecct, eccp) also depends mainly on the packet loss rate. At the same time, due to the use of error correction protection coding for the audio stream, the bit rate is also increased, and the rising scale factor is also kecc(ecct, eccp). In addition, after the error correction coding is adopted, an additional increase in the amount of multimedia data processing by the streaming server is caused, which causes the streaming server to reduce the processing efficiency of the multimedia data.

The above embodiment exemplifies a specific user experience quantitative model, and the user experience quantitative model is a target letter of the embodiment of the present invention. The embodiment of the present invention also needs to set a constraint condition of the objective function, thereby converting the user experience quantitative model into a mathematical optimization problem. The constraints here can be obtained from resource constraints corresponding to the media stream. Since the media stream needs to consume the resources of the streaming server, embodiments of the present invention can set constraints from the resources of the streaming server.

The following takes the processing resources of the streaming server as an example to describe the situation in which the media stream needs to consume the resources of the streaming server.

A streaming server is a network device with processing power. The total processing resources of the streaming server are limited. A rough indicator of the processing power of a streaming server is the number of concurrent streams, which is the number of simultaneous streams that the streaming server sends. However, the embodiment of the present invention can analyze the consumption of the media stream convection server resource from the processing flow of the media stream regardless of the index of the concurrent stream number. The processing of the media stream by the streaming server may include the following steps:

1. Reading streaming media data from the storage device;

2. Perform error correction coding on the streaming media data;

3. The streaming media data is packaged and sent.

In the above three links, the reading link and the packet transmission link must pass, and these two links are basically only related to the bit rate, that is, the amount of data read and packaged and sent in unit time. The processing flow of the set stream server to the read and packet transmission process is: Cp(br), that is, the dependent variable is the bit rate br, the bit rate is vbr for the video stream, and the bit rate is abr for the audio stream. The error correction coding step is optional, that is, the stream server may not perform error correction coding processing on the media stream, and may also perform error correction coding processing on the media stream. In addition to the bit rate, the error correction coding link is also related to ecct and eccp. Therefore, the processing overhead of the error correction coding link can be set as follows: Cecc(br, ecct, eccp), that is, the dependent variable is the bit rate (the bit rate is vbr for the video stream, and the bit rate is abr for the audio stream). ), ecct and eccp.

For a media stream, the total processing overhead of the streaming server is:

,, , iCp(br) + Cecc(br, ecct, eccp) with error correction coding, ,

Ct(br, ecct, eccp) , — ( 10) '' ^W lCp(br) No error correction coding

As shown in Equation 10, for a media stream, the total processing overhead of the stream server can be as shown in Table 1: Table 1 The total processing overhead of the stream server for a single media stream

For video streams, Cp(vbr) increases as vbr increases, and Cecc(br,ecct,eccp) also increases as vbr increases. For the audio stream, Cp (abr) increases as abr increases, and Cecc(abr, ecct, eccp) also increases as abr increases.

In practical applications, there is a set of values (e _C ct, _eCC p) _max such that the value of Cecc(vbr, ecct, eccp) or Cecc(abr, ecct, eccp) for each given vbr or abr value maximum. That is, for each given vbr or abr value, the maximum processing resource of the streaming server that the video stream or audio stream needs to consume can be calculated. The video stream needs to consume the maximum processing resources of the streaming server, and the audio processing needs to consume the maximum processing resources of the streaming server, which can be expressed as follows: Mpvpr(vbr) = Cp(vbr) + Cecc(vbr,ecct eccp

Mpapr(abr) = Cp(abr) + Cecc(abr, ecct , eccp ) ( 11 ) where mpvpr (maximum possible video processing resource) indicates that the video stream may need to consume the maximum processing resources of the server; mpapr (maximum The communicating audio processing resource) indicates that the audio stream may need to consume the maximum processing resources of the streaming server. Mpvpr is an increasing function of the bit rate vbr, and mpapr is an increasing function of the bit rate abr.

Set the total processing resource of the streaming server to RT (R, Resource resource; T, Total all). Some resources in the RT are fixed, and have nothing to do with the media stream, such as system management, running the operating system, and so on. When calculating the total processing resources used by the streaming server for the stream, this part of the fixed resources needs to be subtracted from the RT. The above fixed resources are represented by RF (F means fixed or fixed). The real available processing resource RA (A, Available, available) used by the streaming server for the stream is: RA=RT_RF. When setting the constraints of the target function, it should be considered that the maximum processing resource that the audio stream or video stream needs to consume must not exceed the available processing resources RA of the streaming server.

The embodiment of the present invention may also set the constraint condition of the objective function according to the bandwidth resource of the streaming server, the restriction condition of the user terminal to the streaming media service, and the like.

After setting the mathematical optimization problem according to the user experience quantitative model, the strategy and control server can solve the mathematical optimization problem according to various existing solving methods. The solution process can be implemented by using a DSP (Digital Signal Processor) chip. When mathematical optimization problems are solved using so-called neural networks such as Hopfield networks, commercially available commercial neural network digital integrated circuit chips can be utilized. The strategy and control server obtains the parameters in the user experience quantitative model by solving the results, that is, determining the parameters in the control strategy provided for the streaming server. The policy and control server then outputs the control policy to the streaming server. The streaming server needs to perform these control strategies during media stream reading, error correction coding, and packet delivery. The control strategy includes controlling the variables in the process of media stream reading, error correction coding, and packet transmission by the stream server. Variables such as video compression coding protocol, image size, frame rate, bit rate, etc.; such as audio compression coding protocol, bit rate, etc.

In the embodiment of the present invention, the streaming server can execute various variables in the control policy, that is, the policy and control server can control the streaming media service provided by the streaming server for the user according to the control policy. In a specific practical application, if some variables cannot be controlled, the strategy and control server will be simpler in determining the variables, and thus become a special case for determining each variable.

In the embodiment of the present invention, only the case where the video and the audio stream have existed as the compressed media file in the storage device is considered, and the case where the video and audio streams are obtained by the real-time encoding device is not considered. Therefore, some variables in the control strategy are not controllable, at least they cannot be continuously controlled. For example, the frame rate of a video cannot be controlled at will; if the compressed file corresponds to a certain bit rate, the frame rate should be fixed. To change the frame rate, you can use methods such as frame skipping; changing the frame rate is generally impossible or difficult to achieve. Forcing a frame skipping will result in a serious user experience degradation for a period of time.

At present, by using layered coding technology, video data can be compressed and encoded at the same time to support multiple bit rates and image formats. For example, the base layer supports the QCIF (Quarter Common Interchange Format) format, and the bit rate is 128 kbps; the enhancement layer supports the CIF format, the bit rate is 768 kbps, and the like. In general, video data can support one base layer and multiple enhancement layers, so video data can provide a combination of multiple bit rates. Even without layered coding, video data can support multiple bit rates and image formats. For the same video content as a movie, different encoding protocols and bit rates can be used for compression encoding to form multiple versions of video data, and different versions of video data are provided to user terminals having different capabilities.

For the same audio content, such as the same song, you can use AAC compression or MP3 compression to generate different bit rates.

For image formats, different encoding protocols and bit rates can also be used for compression to produce different versions of the image format, which is very practical. For example, the same movie is compressed in 1080i format. The video data of this version can be provided to user terminals that require high-definition quality; compression is performed in XGA format, and the video data of this version can be provided to PC user terminals; CIF or QCIF format is used. Compressed, this version of the video data can be provided to the mobile terminal.

For the frame rate, a similar method as described above can also be employed, that is, by compressing and encoding at different frame rates to obtain multimedia data of different frame rates. However, in general, there is an optimal fit between the frame rate and the bit rate. At a given bit rate, there is an optimal frame rate to match. Equation (6) can reflect the optimal fit of bit rate and frame rate.

The process of the flow server executing the policy and controlling the control policy transmitted by the server is as shown in FIG. 6.

In FIG. 6, when the flow server executes a specific control policy, it is required to select a suitable media data file stored in the storage device according to each variable in the control policy, read the selected media data file, and read the media data file. Perform error correction coding, packet transmission, and the like.

The specific implementation process of the flow server execution policy and the control policy transmitted by the control server in FIG. 6 may include the following four situations:

Case 1: For the video media stream of the same content, the streaming media data file stored in the storage device includes: supporting different bit rates, frame rates, compression coding protocols, image formats, key frame intervals, error correction coding types, and protection strength parameters. Multiple streaming media data files. At this time, after receiving the control policy transmitted by the policy and the control server, the stream server selects according to the bit rate, the frame rate, the compression coding protocol, the image format, the key frame interval, the error correction coding type, and the protection strength parameter in the control policy. Read the streaming media data file stored in the storage device. After successfully reading the streaming media data file, the streaming server directly converts the read streaming media data file into a media stream and sends it to the user terminal.

Case 2: For the video media stream of the same content, the streaming media data file stored in the storage device includes: a plurality of streaming media data files supporting different bit rates, frame rates, compression encoding protocols, image formats, and key frame intervals. At this time, after receiving the control policy transmitted by the policy and control server, the stream server selects to read the streaming media stored in the storage device according to the bit rate, the frame rate, the compression coding protocol, the image format, and the key frame interval in the control policy. data files. After successfully reading the streaming media data file, the streaming server performs error correction coding processing on the read streaming media data file according to the error correction coding type and the protection strength parameter in the control policy. Then, the stream server converts the error correction encoded stream media data file into a media stream and sends the file to the user terminal.

Case 3: For the audio media stream of the same content, the streaming media data file stored in the storage device includes: a plurality of streaming media data files supporting different bit rates, compression coding protocols, error correction coding types, and protection strength parameters. At this time, after receiving the control policy transmitted by the policy and the control server, the stream server selects and reads the streaming media data stored in the storage device according to the bit rate, the compression coding protocol, the error correction coding type, and the protection strength parameter in the control policy. file. After successfully reading the streaming media data file, the streaming server directly converts the read streaming media data file into a media stream and sends the data to the user terminal. Case 4: For the audio media stream of the same content, the streaming media data file stored in the storage device includes: multiple streaming media data files supporting different bit rates and compression encoding protocols. At this time, after receiving the control policy transmitted by the policy and control server, the stream server reads the streaming media data file according to the bit rate and the compression coding protocol in the control policy, and according to the error correction coding type and protection strength in the control policy. The parameter performs error correction coding processing on the read streaming media data file. Then, the stream server converts the error correction encoded stream media data file into a media stream and sends the file to the user terminal.

The flow server in the embodiment of the present invention selects the media data file, the error correction code, and the packet transmission according to the control policy, and the control policy in the embodiment of the present invention is obtained on the basis of the user experience quantitative model. The streaming server provided by the streaming server for the user terminal in the embodiment of the present invention is a streaming media service that optimizes the user terminal experience.

The method for obtaining a control strategy in the embodiment of the present invention will be described below by taking several specific embodiments as an example.

Embodiment 1 A method for obtaining a control strategy for a single video stream.

First, the policy and control server needs to determine the optimal values of the encoding protocol type, bit rate, frame rate, key frame interval, error correction coding pattern, protection strength parameter, etc., in order to optimize the user experience. In the process of determining each variable, it is necessary to combine the end-to-end packet loss rate pl and the packet loss rate as an external variable. The method for obtaining the packet loss rate is as described in the foregoing embodiment, and the embodiment of the present invention does not limit the specific implementation process for obtaining the packet loss rate. The objective functions set by the Policy and Control Server are:

QEV = f (vbr, fr, vcp, pf, kfi, ecct, eccp) = fcp(vbr, fr, vcp, pf, kfi) + eccind x fecc(ecct, eccp)

(12)

A new variable, BP eccind (error correction code indicator), is introduced in the above objective function, and eccind is used to indicate whether error correction coding is used. The value of eccind is as follows:

Using error correction coding, ,

Eccmd = one..., ( 13 )

No error correction coding

The process of constructing constraints for the objective function (12) is:

In terms of the network connection bandwidth of the streaming server: If the video maximum allowable transmission bandwidth amvtb (allowable maximum video transmission bandwidth) is set for each video stream, then a video stream needs to satisfy the following conditions in terms of bandwidth to be consumed:

((kecc(ecct,eccp)- 1 )eccind+ 1 )vbr<amvtb ( 14)

If the maximum allowable transmission bandwidth amvtb is not set for each video stream, then a video stream needs to satisfy the following conditions in terms of bandwidth to be consumed:

((kecc(ecct,eccp)- 1 )eccind+ 1 )vbr≤TAB ( 15 )

Where: TAB (Total Available Bandwidth) is the total available bandwidth of the stream server to send external media streams. If the streaming server still has some other communication needs, such as communicating with other servers, the TAB should be less than the total bandwidth of the streaming server to the network connection.

Processing resources of the streaming server: If the video maximum allowable processing resource amvpr (allowable maximum video processing resource) is set for each video stream, then a video stream needs to satisfy the following conditions in terms of processing resources to be consumed:

Cp(vbr) + eccind x Cecc(vbr, ecct, eccp) < amvpr ( 16) If the video is not allowed to be used for each video stream, the maximum processing resource avp is allowed. Then a video stream needs to meet the following conditions in terms of processing resources that need to be consumed:

Cp(vbr) + eccind x Cecc(vbr, ecct, eccp) < RA ( 17) where RA is the truly available processing resource for the stream server for the stream.

The embodiment of the present invention may set the constraint of the target function (12) according to the network connection bandwidth of the streaming server and the processing resource of the streaming server. The embodiment of the present invention may also be based on the network connection bandwidth of the streaming server and the processing resource of the streaming server. And the resources of the user terminal to set the constraints of the objective function (12). The resources of the user terminal, such as the network bandwidth of the user terminal connection, the limitation of the user terminal to the multimedia stream format, and the like. The user terminal limits the multimedia stream format, such as restrictions on the multimedia stream image format.

If the network bandwidth of the user terminal is limited, and the maximum allowable receiving bandwidth of the video of the user terminal is amvrb (allowable maximum video reception bandwidth), then the constraint (14) needs to be corrected to:

((kecc(ecct,eccp)- 1 )eccind+ 1 )vbr<min(amvtb,amvrb) ( 18)

Where min(x, y) represents the minimum of the two numbers x and y.

If the network bandwidth of the user terminal is limited and the maximum allowable receiving bandwidth of the video of the user terminal is amvrb, then the constraint condition (15) needs to be corrected to:

((kecc(ecct,eccp)- 1 )eccind+ 1 )vbr<min(TAB ,amvrb) ( 19)

Therefore, the mathematical optimization problem can be described as follows:

{vbr,fr,vcp,pf,kfi,ecct,eccp} _0pt = 3TgmaX (fcp(vbr, fr, vcp, pf , kfi) +

{vbr,fr,vcp,pf,kfi,ecct,eccp}eWSxECCVSx{0,l}

Eccind x fecc(ecct, eccp))

S.t.

(1)((kecc(ecct,eccp)-1)eccind + 1 )vbr < min(amvtb,amvrb)

Or (20) ((kecc(ecct,eccp)-1)eccind + l)vbr < min(TAB ,amvrb)

(2)

Cp(vbr) + eccind x Cecc(vbr, ecct, eccp) < amvpr

Or

Cp(vbr) + eccind x Cecc(vbr, ecct, eccp) < RA

Where st (subject to) represents a constraint; WS (video variable space) represents a set of all combinations of values vbr, fr, vcp, pf, kfi; ECCVS (Error Correction Code) Variable Space, error correction coding variable space), represents a collection of all combinations of values ecct, eccp; operator "X" represents the Cartesian product of two spaces (sets); {0, 1 } is the set of values of the variable eccind.

It has been explained in the foregoing embodiments that the values of variables such as vbr, fr, vcp, pf, kfi, ecct, and eccp may all be discrete values. Therefore, both WS and ECCVS are finite sets, that is, WS and ECCVS are elements. A limited number of collections. Thus, the Cartesian product of WS, ECCVS, and { 0, 1 } is also a finite set.

Solving a mathematical optimization problem on the basis of a finite set, there are many methods, such as Brutal force Search), combinatoric optimization methods, and so on. The brute force search method searches for each set of variables in the comparison set under constraint constraints, and finally finds a set of variable values that maximize the value of the objective function. The set of variable values that maximize the value of the objective function is the optimal value of the variables such as the encoding protocol type, bit rate, frame rate, key frame interval, error correction coding pattern, and protection strength parameter. The optimal values of these variables are the parameters in the quantitative model of the video user experience. The parameters in the quantitative model of the video user experience are also the decision parameters in the control strategy.

The method of solving the mathematical optimization problem belongs to the existing mathematical method, and the method for solving the mathematical optimization problem is not described in detail here. Moreover, embodiments of the present invention do not limit the method of solving mathematical optimization problems.

Embodiment 2: A method for obtaining a control strategy for a single audio stream.

First, the policy and control server needs to determine the optimal values of the variable encoding protocol type, bit rate, error correction coding pattern, and protection strength parameters to optimize the user experience. In the process of determining each variable, it is necessary to combine the end-to-end packet loss rate pl and the packet loss rate as an external variable. The method for obtaining the packet loss rate is as described in the foregoing embodiment, and the embodiment of the present invention does not limit the specific implementation process for obtaining the packet loss rate. The objective functions set by the Policy and Control Server are:

QEA = g(abr, acp, ecct, eccp) = gcp(abr, acp) + eccind x gecc(ecct, eccp) (21) The process of constructing a constraint for the objective function (21) is:

The network connection bandwidth aspect of the streaming server: If the audio maximum allowed transmission bandwidth is set for each audio stream amatb

(allowable maximum audio transmission bandwidth), then an audio stream needs to meet the following conditions in terms of bandwidth that needs to be consumed:

((kecc(ecct,eccp)- 1 )eccind+ 1 )abr<amatb (22)

If the audio maximum allowable transmission bandwidth amvtb is not set for each audio stream, then an audio stream needs to satisfy the following conditions in terms of bandwidth to be consumed:

((kecc(ecct,eccp)- 1 )eccind+ 1 )abr<TAB (23 )

In terms of processing resources of the streaming server: If the audio maximum allowed processing resource is set for each audio stream amapr

(allowable maximum audio processing resource), then an audio stream needs to meet the following conditions in terms of needing to consume processing resources:

Cp(abr) + eccind x Cecc(abr, ecct, eccp) < amapr (24) If the audio maximum allowable processing resource amapr is not set for each audio stream, then an audio stream needs to be satisfied in terms of the need to consume processing resources. The following conditions:

Cp(abr) + eccind x Cecc(abr, ecct, eccp) ≤ RA (25) The embodiment of the present invention can set the constraint of the objective function (21) based only on the network connection bandwidth of the streaming server and the processing resources of the streaming server. The embodiment of the present invention may also set the constraint condition of the objective function (21) according to the network connection bandwidth of the streaming server, the processing resource of the streaming server, and the resource of the user terminal. User terminal resources such as network bands connected to user terminals Wide, user terminal restrictions on the multimedia stream format, and so on. The user terminal limits the multimedia stream format, such as restrictions on the audio stream format.

If the network bandwidth of the user terminal is limited, and the maximum allowable audio reception bandwidth of the user terminal is amarb (allowable maximum audio reception bandwidth), the constraint condition (22) needs to be corrected to:

((kecc(ecct,eccp)- 1 )eccind+ 1 )vbr<min(amatb,amarb) (26)

Where min(x, y) represents the minimum of the two numbers x and y.

If the network bandwidth of the user terminal is limited and the maximum allowable receiving bandwidth of the user terminal is amarb, the constraint condition (23) needs to be corrected to:

((kecc(ecct,eccp)- 1 )eccind+ 1 )vbr<min(TAB ,amarb) (27)

Therefore, the mathematical optimization problem established based on the user experience quantitative model of streaming media can be described as follows:

{abr,acp,ecct, eccp} = 3TgmaX (gcp(abr, acp) + eccind x gecc(ecct, eccp))

{abr,acp,ecct,eccp}eAVSxECCVSx{0, 1 }

S.t.

(1)((kecc(ecct, eccp) - l)eccind + l)abr < min(amatb, amarb)

Or

((kecc(ecct, eccp) - l)eccind + l)abr < min(TAB , amarb)

(2)

Cp(abr) + eccind x Cecc(abr, ecct, eccp) < amapr

Or

Cp(abr) + eccind x Cecc(abr, ecct, eccp) < RA

(28)

Where st represents the constraint; AVS (audio variable space), which represents the set of all combinations of all the values of the variable abr, acp; ECCVS, which represents the various combinations of all the values of the variables ecct and eccp The set of constructs; the operator "X" represents the Cartesian product of two spaces (sets); the set {0, 1 } is the set of values of the variable eccind.

It has been explained in the foregoing embodiments that the values of variables such as abr, acp, ecct, and eccp may all be discrete values. Therefore, both AVS and ECCVS are finite sets, that is, AVS and ECCVS are sets of a limited number of elements. Thus, the Cartesian product of AVS, ECCVS, and { 0, 1 } is also a finite set.

There are many methods for solving a mathematical optimization problem based on a finite set. The method for solving the mathematical optimization problem belongs to the existing mathematical method. The method for solving the mathematical optimization problem is not described in detail. Moreover, embodiments of the present invention do not limit the method of solving mathematical optimization problems. Regardless of the solution method used, after determining the set of variable values that maximize the value of the objective function, the decision parameters in the control strategy are determined.

In practical applications, a streaming server tends to provide multiple media streams at the same time and simultaneously provide services for multiple user terminals. The correspondence between the media stream and the user terminal mainly has four cases as shown in FIG. The four cases shown in Figure 7 are:

Case a, one-to-one, that is, a unicast stream is provided to a user terminal. Case a applies to video on demand or the network does not support live streaming (Live) in the case of multicast.

Case b, one-to-many, that is, the network supports multicast, used for live (Live) or simulated live (Simucast). From the streaming server A multicast stream is sent and sent to multiple user terminals through the stream replication node.

Situation (;, many-to-one, that is, one user terminal simultaneously receives multiple unicast media streams, similar to a picture-in-picture of a television set.

Case d, many-to-many, a combination of the above three cases.

In practical applications, video streams and audio streams do not necessarily appear in pairs. For example, in an Internet Radio (IntemetRadio) application, only the audio stream has no video stream; as in a remote monitoring application, only the video stream may have no audio stream. In IPTV applications, video streams and audio streams tend to appear in pairs. However, the number of video streams and the number of audio streams can vary. Therefore, in the process of establishing mathematical optimization, the strategy and control server must take into account the above different situations, making the description of mathematical optimization problems general. The control policy obtaining method for the streaming media service of the multiple media streams may be the following two types in the following cases: Embodiment 3: A control policy obtaining method for a plurality of media streams and considering resource consumption separately.

The media stream that the streaming server needs to send to the user terminal includes NV video streams and NA audio streams. The maximum allowed transmission bandwidth for each video stream is: amvtb (i) , where i=0, 1, 2, ..., NV-1. The maximum value of the maximum allowed transmission bandwidth of each video stream is not necessarily the same. The maximum allowable transmission bandwidth of each audio stream is: amatb (j ) , where j=0, 1, 2, ..., NA-1. The value of the maximum allowable transmission bandwidth of each audio stream is not necessarily the same.

When setting the maximum allowable transmission bandwidth for each video stream and audio stream, all audio streams and all video streams need to meet the following conditions in terms of the bandwidth to be consumed:

NV-1 NA-1

^amvtb(i) + ^amatb(j) ΓΑΒ (29)

i=0 j=0

The total available bandwidth of the streaming server can meet the bandwidth requirements of each video stream and audio stream.

When setting the maximum allowable processing resources of each video stream and audio stream, all audio streams and all video streams need to meet the following conditions in terms of processing resources to be consumed:

NV-1 NA-1

^ mpvpr(amvtb(i)) + ^ mpapr(amatb( j)) <RA (30) i=0 j=0

The available processing resources of the streaming server can satisfy the processing resource requirements of each video stream and audio stream.

After setting the constraints of equations (29) and (30), each user experience quantitative model can be converted into a mathematical optimization problem, and then the mathematical optimization problem is solved. Regardless of the solution method used, after determining a set of variable values that maximize the value of the objective function, each decision parameter in the control strategy for each media stream is determined, thereby providing an optimal user for each user terminal. Experience.

The third embodiment is to consider the resource consumption of each media stream separately. Considering the resource consumption of each media stream is a conservative method. The disadvantage is that the specific situation of each streaming media stream cannot be considered together. Resources cannot be adjusted to each other. Each media stream has a maximum allowed transmission bandwidth. If it is not used, other streams cannot use this extra bandwidth, and the remaining processing resources cannot be used.

Embodiment 4 A control strategy obtaining method for a plurality of media streams and jointly considering resource consumption of each media stream. This method is a comprehensive and optimal control strategy acquisition method.

The media stream that the streaming server needs to send to the user terminal is still set to include NV video streams and NA audio streams, and the number of users of each media stream service is: NVC (i), i=0, 1, 2, , NV- 1 ; NAC (j ) , j = 0, 1, 2, , NA-1 ; Medium: NVC (Number of video customers) is the number of video users, NAC (Number of audio customers) is the number of audio users; and for unicast streams, the number of users is 1, for multicast streams, the number of users may be greater than 1. The QET (Quality of Experience Total) of the user terminal is:

NV-l

QET = NVC(i) x [fcp(vbr _; , fr _; , vcpi, pf , kfij ) + eccindi x fecc(ecct _{i ?} eccpi )] +

(31) ^ NAC j) x [gcp(abrj , acpj ) + eccindj x gecc(ecct _; , eccp _; )] The comprehensive user experience for all users is a weighted sum of user experiences for each media stream. Here, it can be set that each user terminal is equal, and it is also possible to set a difference between the user terminals. If there is a difference between the user terminals, such as the difference between the VIP user and the ordinary user, when setting the weight, the difference between the different user terminals is also considered.

The embodiment of the present invention introduces a weight w indicating a level difference of the user terminal. If the weight of the lowest level user is 1, and the weight of the highest level user is WM, then w=l, 2, 3, ..., WM. A total of WM level. After considering the weight, equation (31) needs to be corrected to:

NV-l NVC(i)-l

QET = ^[fcpCvbr^ fr; , ν. Ρί , pfj , kfij ) + eccindi x fecc(eccti , eccp )] x ^ w(i, k) +

NA-1 ⁱ⁼⁰ NAC(j)- 1 ^k= ° (32)

^ NAC j) x [gcp(abrj , acpj) + eccindj x gecc(ecct _{j ?} eccpj)] x ^ wj _? k)

j=0 k=0

Where: w(i,k), k=0,l,2,..., NVC(i)-l represents the respective level weights of the NVC (i) users receiving the i-th video stream. Where w(j,k), k=0, l, 2, ..., NAC(j)-l represents the respective level weights of the NAC (j) users receiving the jth audio stream.

In equations (31) and (32), there are 8 variables for each video stream to be optimized, and 5 variables for each audio stream need to be optimized. Therefore, the total number of variables that need to be optimized is 8NV+5NA.

All audio streams and all video streams need to meet the following conditions in terms of the bandwidth they need to consume:

All audio streams and all video streams need to meet the following conditions in terms of processing resources that need to be consumed:

NV-l NA-1

^ϋ(ν^) + eccind _i Cecc(vbr _{i ?} ecct _{i ?} eccp _i ) + ^ C (abrj ) + eccindjCecc(abrj _? ecct j _? ecc j )≤ RA i=0 j=0

(34)

Therefore, the mathematical optimization problem obtained by the user experience quantitative model conversion according to each user terminal is:

| i = 0,1,2,..., NV - l } _opt u

{abrj,acpj,eccindj,ecctj,eccpj | j = 0,l,2,..., NA - l} _opt =

NV-l

Argmin ∑ [&ρ(ν^ , , vcpj , , kf^ )

{vbr ,frj ,vcp ,ρζ ,kfi ,eccind ,ecct ,eccp |i=0,l,2,...,NV-l } < ¹⁼ 0

{abrj ,acpj ,eccindj ,ecctj ,eccp _j |j=0,l,2,..,NA-l}eWS ^Nv xAVS ^NA xECCVS ^NVxNA x{0,l}

NVC(i)-l NA-1

+ eccind; x fecc(ecct _; , eccp _; )] x ^ w(i, k) NAC(j) x [gcp(abrj , acpj ) + eccindj

k=0 j=0

NAC(j)- 1

Xgecc(ecctj, eccpj)] x ^w(j,k)

k=0

.t.

NV-l NA-1

^ Cp(vbr; ) + eccindiCecc(vbri , ecct _{i 5} ecc ; ) + ^ Cp(abrj ) + eccindjCecc(abrj , ecctj , ecc j )≤ RA i=0 j=0 where ws x ^na x ^ χ { o, !}^ represents the Cartesian product of NV WS, NA AVS, NVxNA ECCVS, and NVxNA {0, 1}.

The above formula (35) is a typical discrete mathematical optimization problem, and the number of variables to be optimized is 8NV+5NA. The embodiment of the present invention can solve the mathematical optimization problem of the above formula (35) by using various solving methods in the prior art, and will not be described in detail herein.

If CARD (WS) indicates the number of elements in the WS set, CARD (AVS) indicates the number of elements in the AVS set, and CARD (ECCVS) indicates the number of elements in the ECCVS set, then _{x AVS} ⁴

The number of elements in the middle is: ZWxN^CARD WS WtCAM AVS ^CAM ECCVS^wxNA a. In the above embodiments in which the respective media streams are separately considered, although the number of variables to be optimized does not change, it is also 8NV+5NA, but needs to be searched. The number of elements in the collection is:

2(NV X NA) + NV x [CARD( WS)] + NA x [CARD(AVS)] + (NV x NA) x [CARD (ECCVS)] Obviously, in the implementation of each media stream separately, The size of the mathematical optimization problem increases with the number of media streams in accordance with the arithmetic progression. In the implementation of considering various media streams, the size of the mathematical optimization problem increases in geometric progression with the total number of media streams.

Embodiments of the present invention can reduce the size of the problem and simplify the amount of computation for solving the problem by reducing the number of variables that need to be optimized. The method of reducing the number of variables to be optimized is to merge the media streams according to categories. If the user terminal bandwidth of 1000 media streams, the required image size, and the encoding protocol are the same, the 1000 media streams can be merged into one. Class, the variable optimization results of the 1000 media streams are the same. As long as the variables of one media stream are optimized, the obtained optimized variable values can be adapted to other media streams of the same kind. In this way, the mathematical optimization problem is greatly simplified, so that the number of media streams that affect the scale of the mathematical optimization problem is not the number of classes of different types of media streams. At the same time, in practical applications, some variables such as image format, encoding protocol, etc. need to be fixed, which also reduces the scale of mathematical optimization problems.

Embodiments of the present invention also provide a streaming media service implementation system, which includes a storage device, a policy and control server, and a streaming server.

The storage device is mainly used to store streaming media data files. The streaming media data file includes: a media stream for the same content, and multiple streaming media data files supporting different control policies; for example, for a video media stream of the same content, the storage device stores different bit rates, frame rates, and Multiple streaming media data files of compression encoding protocol, image format, key frame interval, error correction coding type, protection strength parameter; or multiple streams supporting different bit rates, frame rates, compression coding protocols, image formats, key frame intervals For example, for the audio media stream of the same content, the storage device stores multiple streaming media data files supporting different bit rates, compression coding protocols, error correction coding types, and protection strength parameters; or supports different bit rates. And compressing multiple streaming media data files of the encoding protocol.

The policy and control server is mainly used to store a mathematical optimization problem established according to a quantitative model of the user experience of the streaming media, and the constraint condition of the mathematical optimization problem may be set according to resource restriction conditions of the streaming media service. The strategy and control server solves the mathematical optimization problem to determine various parameters in the user experience quantitative model. Then, the policy and control server sets the control strategy according to the determined parameters, and transmits the set control strategy to the stream server. The policy and control server may set the constraint condition of the target function according to the resource of the streaming server, and may also set the constraint condition of the target function according to the bandwidth resource of the streaming server, the restriction condition of the user terminal to the streaming media service, and the like. When the policy and control server set the constraint, the constraint of the mathematical optimization problem may be set according to the constraint condition that the single media stream needs to consume the flow server resource; or the media stream needs to be consumed under the condition that the resource of the streaming server is not adjusted. The constraints of the mathematical optimization problem; the constraints of the mathematical optimization problem can also be set under the condition that each media stream needs to consume the resources of the streaming server to allow mutual adjustment. The specific implementation manner of the policy and control server setting constraints, the solving process for the mathematical optimization problem, and the like are described in the above method embodiments.

The streaming server is mainly configured to read the streaming media data file from the storage device according to the received control policy, and convert the read streaming media data file into a media stream and send the data to the user terminal; for example, the streaming server according to the bit in the control policy The rate, frame rate, compression coding protocol, image format, key frame interval, error correction coding type, protection strength parameter are selected to read the video streaming data file stored in the storage device, and then the streaming server directly reads the video streaming media The data file is converted into a media stream and sent to the user terminal; for example, the stream server selects to read the video streaming media data file stored in the storage device according to the bit rate, the frame rate, the compression encoding protocol, the image format, and the key frame interval in the control policy. Then, the stream server performs error correction coding processing on the read video streaming media data file according to the error correction coding type and the protection strength parameter in the control policy; for example, the flow server according to the bit rate in the control policy, the compression coding protocol, Error correction coding type, protection strength Selecting to read the audio streaming media data file stored in the storage device, and then the streaming server directly converts the read audio streaming media data file into a media stream and sends the data to the user terminal; and, the streaming server according to the bit rate in the control policy The compression encoding protocol reads the audio streaming media data file, and the streaming server performs error correction encoding processing on the read audio streaming media data file according to the error correction encoding type and the protection strength parameter in the control strategy, and then the streaming server encodes the error correction encoding. The processed streaming media data file is converted into a media stream and sent to the user terminal. The specific operations performed by the streaming server are as described in the above method embodiments.

The embodiment of the invention also provides a policy and control server. The storage module is set in the policy and control server, and the solution is solved. Module and transmission module.

The storage module is mainly used to store a mathematical optimization problem established according to a user experience quantitative model of the streaming media; the constraint condition of the mathematical optimization problem may be set according to a resource restriction condition of the streaming media service, such as the constraint condition of the mathematical optimization problem may be The setting according to the resource of the streaming server may also be set according to the bandwidth resource of the streaming server, the restriction condition of the streaming terminal service by the user terminal, and the like. The constraint of the mathematical optimization problem stored in the storage module may be set according to a constraint condition that a single media stream needs to consume the streaming server resource; or may be set under the condition that each media stream needs to consume the resources of the streaming server without adjusting the media; It may also be set under the condition that each media stream needs to consume the resources of the streaming server to allow mutual adjustment. The mathematical optimization problems stored in the storage module and their constraints are as described in the above method embodiments.

The solution module is mainly used to solve the mathematical optimization problem stored in the storage module. The solution module can be implemented by using various existing methods and devices, and the specific solution process will not be described in detail herein.

The transmission module is mainly used to set each parameter in the user experience quantitative model obtained by the solution module by solving the mathematical optimization problem as a control strategy, and transmit the control strategy to the stream server. For video streaming services, the control strategies here include, but are not limited to: bit rate, frame rate, compression coding protocol, image format, key frame interval, error correction coding type, protection strength parameters; for audio streaming services The control strategy here includes but is not limited to: bit rate, compression coding protocol, error correction coding type, protection strength parameter.

Since the control strategy in the embodiment of the present invention is accurately calculated according to the user experience quantitative model, and the user experience quantitative model comprehensively considers various aspects of the user experience, the embodiment of the present invention can provide the user with the optimal maximum. User experience. For the corresponding user experience quantitative models such as video streams and audio streams, more quantitative models of user experience will be proposed as the research in related fields continues to deepen. Since the implementation process of the embodiment of the present invention has no relationship with the specific expression form of the user experience quantitative model, the embodiment of the present invention can be applied to various different user experience quantitative models. Moreover, when the user experience quantitative model is improved, the user experience brought by the embodiment of the present invention for the user terminal is also improved, thereby bringing a better user experience to the user terminal.

While the invention has been described by the embodiments of the invention, it will be understood that

Claims

Rights request

A method for implementing a streaming media service, the method comprising:

2. The method according to claim 1, wherein, for the video streaming service, the parameters include: a bit rate, a frame rate, a compression coding protocol, an image format, a key frame interval, and an error correction coding type. , protection strength parameters.

3. The method of claim 2, wherein, for the video media streaming service,

For the video media stream of the same content, the stored streaming media data file includes: multiple streaming media data files supporting different bit rates, frame rates, compression encoding protocols, image formats, key frame intervals, error correction coding types, and protection strength parameters. And the step of the stream server reading the streaming media data file according to the control policy and sending the media stream to the user terminal according to the read streaming media data file comprises: the flow server according to the bit rate and the frame rate in the control policy, The compression coding protocol, the image format, the key frame interval, the error correction coding type, and the protection strength parameter read the streaming media data file, and directly convert the read streaming media data file into a media stream, and send the data to the user terminal; or

For the video media stream of the same content, the stored streaming media data file includes: a plurality of streaming media data files supporting different bit rates, frame rates, compression encoding protocols, image formats, key frame intervals; and the streaming server according to the The control strategy reads the streaming media data file and sends the media stream to the user terminal according to the read streaming media data file, including: the flow server according to the bit rate, the frame rate, the compression encoding protocol, the image format, the key frame in the control policy Reading the streaming media data file at intervals, and performing error correction coding processing on the read streaming media data file according to the error correction coding type and the protection strength parameter in the control policy, and then processing the error correction coding processed streaming media data The file is converted into a media stream and sent to the user terminal.

The method according to claim 1, wherein, for the audio streaming service, the parameters include: a bit rate, a compression coding protocol, an error correction coding type, and a protection strength parameter.

5. The method of claim 4, wherein, for the audio streaming service,

For the audio media stream of the same content, the stored streaming media data file includes: a plurality of streaming media data files supporting different bit rates, compression encoding protocols, error correction encoding types, protection strength parameters; and the streaming server according to the control The policy of reading the streaming media data file and sending the media stream to the user terminal according to the read streaming media data file comprises: the streaming server reading according to the bit rate, the compression encoding protocol, the error correction coding type, and the protection strength parameter in the control policy The streaming media data file is taken, and the read streaming media data file is directly converted into a media stream and sent to the user terminal; or

For the audio media stream of the same content, the stored streaming media data file includes: a plurality of streaming media data files supporting different bit rates, compression encoding protocols; and the streaming server reads the streaming media data files according to the control policy, and The step of sending the media stream to the user terminal according to the read streaming media data file includes: the streaming server reads the streaming media data file according to the bit rate in the control policy, the compression encoding protocol, and according to the error correction coding type and protection in the control policy. Strength parameter for the read streaming media data The file performs error correction coding processing, and then converts the error correction encoded stream media data file into a media stream and sends it to the user terminal.

6. The method according to claim 1, wherein the constraint condition of the mathematical optimization problem is set according to a resource restriction condition of the streaming media service.

The method according to claim 6, wherein the resource restriction condition of the streaming media service comprises one or more of the following: a limitation of a streaming server resource that the media stream needs to consume, a limitation of a user terminal bandwidth, and a user. Terminal restrictions on media format support capabilities.

The method according to claim 7, wherein the resources of the streaming server comprise: a bandwidth resource of the streaming server, and/or a processing resource of the streaming server.

The method according to claim 6 or 7 or 8, wherein the setting method of the constraint condition of the mathematical optimization problem comprises:

When the streaming server provides a single media stream, the constraints of the mathematical optimization problem are set according to the constraints of the streaming server resources that need to be consumed by the single media stream; or

When the streaming server provides multiple media streams at the same time, the constraints of the mathematical optimization problem are set under the condition that each media stream needs to consume the resources of the streaming server;

When the streaming server provides multiple media streams at the same time, the constraints of the mathematical optimization problem are set under the condition that each media stream needs to consume the resources of the streaming server to allow mutual adjustment.

10. The method according to claim 9, wherein in the case of setting a mathematical optimization problem under the condition that each media stream needs to consume the resources of the streaming server to allow mutual adjustment, in the process of solving the mathematical optimization problem The control policy parameter of one media stream obtained is obtained as a control policy parameter of all media streams belonging to the same class having the same user experience quantitative model parameters as the media stream.

The method according to claim 1, wherein, for the media stream of the same content, the stored streaming media data file comprises: a plurality of streaming media data files supporting different control policies; and when the stored streaming media data files The step of the stream server reading the streaming media data file according to the control policy and transmitting the media stream to the user terminal according to the read streaming media data file includes: the streaming server according to the control policy The error correction coding type and the protection strength parameter perform error correction coding processing on the media stream data transmitted by the error correction coding type.

12. A streaming media service implementation system, the system comprising:

a storage device, configured to store a streaming media data file;

13. The system of claim 12, wherein the constraints of the mathematical optimization problem comprise: When the streaming server provides only a single media stream, the constraints of the mathematical optimization problem set by the constraints of the streaming media resource are required according to the single media stream; or

When the streaming server simultaneously provides multiple media streams, the constraints of the mathematical optimization problem set under the condition that each media stream needs to consume the resources of the streaming server are not adjusted; or

When the streaming server simultaneously provides multiple media streams, the constraints of the mathematical optimization problem set under the condition that each media stream needs to consume the resources of the streaming server to allow mutual adjustment.

14. The system of claim 12, wherein:

For the video streaming service, the parameters include: a bit rate, a frame rate, a compression encoding protocol, an image format, a key frame interval, an error correction coding type, and a protection strength parameter;

For the audio streaming service, the parameters include: a bit rate, a compression coding protocol, an error correction coding type, and a protection strength parameter.

15. A policy and control server, wherein the policy and control server comprises:

The transmission module is configured to set a control strategy according to each parameter in the user experience quantitative model obtained by solving the mathematical optimization problem by the solution module, and transmit the control strategy to the stream server.

16. The policy and control server of claim 15 wherein: