JP2023081642A - Receiving terminal and segment acquisition control method - Google Patents

Abstract

Kind Code: A1 In video streaming distribution, video quality is controlled based on the importance of a video scene obtained from the content and meaning contained in the video scene. SOLUTION: A receiving terminal that receives video streaming distribution adjusts the receiving bit rate of a segment within a rate determination cycle based on a scene list that describes the importance of a video scene in units of segments obtained by dividing the video stream. By providing a video quality determination unit that performs control, it is possible to view moving images with video quality that takes into consideration the importance of video scenes. [Selection drawing] Fig. 1

Description

The present invention relates to a receiving terminal for receiving streaming distribution of moving images and a segment acquisition control method in the receiving terminal. More specifically, the present invention relates to a receiving terminal and segment acquisition control method for controlling video quality based on the importance of video scenes in video streaming distribution.

In recent streaming video distribution on the Internet, a method of streaming distribution using the HTTP protocol by a general-purpose Web server has become mainstream in distribution for many devices. Proprietary technology by IT vendors is widely used as a streaming distribution method (adaptive streaming) using the HTTP protocol, and MPEG-DASH (ISO/IEC23009-1), an international standard intended to unify these streaming methods. There is
All technologies have the same basic concept, and a web server stores video streams encoded with one or more video qualities (screen size and bit rate), each of which is several seconds to several tens of seconds long. A manifest file that describes the attributes and URLs of the moving image content is prepared.
The receiving terminal selects the appropriate video quality from the manifest file in consideration of the screen size of the receiving terminal and the state of the network bandwidth of the transmission line, receives the segments one after another, and joins them into one video content. (Non-Patent Document 1).

There is no standardized method for selecting the video quality, and a widely used method is to control the video quality based on network conditions, the remaining buffer capacity of the viewing terminal, and the like (Non-Patent Document 2). Also, as a method of scoring segments and selecting video quality based on this score, a bit rate (video quality) selection method according to the level of scene complexity (Patent Document 1 ) is disclosed.

Japanese Patent Application Publication No. 2014-506748

Mitsuhiro Hirabayashi, “Technical Overview and Standardization of Next-Generation Video Distribution Technology “MPEG-DASH” and Related Technology Trends”, Institute of Image Information and Television Engineers, 2013, 67, No. 2, pp. 109-115 Hongzi Mao et al., “Neural Adaptive Video Streaming with Pensieve”, SIGCOMM ’17, August 21-25, 2017, pp 197-210

However, the scoring of these segments is based on an index of scene complexity (such as the intensity of motion within the scene), and does not take into account the content or meaning contained in the video scene.
The present invention has been made in view of this problem, and its object is to control the video quality based on the importance of the video scene obtained from the content and meaning contained in the video scene in the streaming distribution of moving images. It is in.

In order to solve the above problems, one aspect of the present invention provides (1) a receiving terminal that receives streaming distribution of a moving image, in which a scene list describing the importance of a video scene is described for each segment obtained by dividing a video stream. Based on this, by providing a video quality determination unit that controls the reception bit rate of segments while adjusting within the rate determination cycle, it is possible to view moving images with video quality that takes into consideration the importance of video scenes. .

In addition, a further aspect of the present invention is characterized by (2) controlling the reception bit rate of the segment based on a scene list describing the importance of multiple patterns according to the user of the receiving terminal ( 1) receiving terminal.

Another aspect of the present invention is (3) a segment acquisition control method for controlling video quality in consideration of the importance of video scenes in moving image streaming distribution, wherein each segment obtained by dividing a video stream Based on a scene list that describes the importance of video scenes, segment distribution bitrate control is performed while adjusting within the rate determination cycle, enabling video viewing with video quality that takes into consideration the importance of video scenes. It is characterized by

Furthermore, another further aspect of the present invention is characterized in that (4) the distribution bit rate of the segment is controlled based on a scene list describing a plurality of patterns of importance according to the user viewing the moving image. This is the segment acquisition control method of (3).

According to the present invention, it is possible to control video quality based on the importance of video scenes in adaptive streaming.

FIG. 1 is a diagram showing an example of the configuration of a content delivery system using a receiving terminal and a segment acquisition method according to the present invention.

FIG. 2 is a flow chart showing an example of a method for determining a combination of video qualities for a rate determination period in the present invention.

FIG. 3 is a flow chart showing an example of the segment acquisition control method of the receiving terminal 100 according to the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Receiving terminal 100 shown in FIG. 1 includes communication interface 101, manifest acquisition unit 102, scene list acquisition unit 103, throughput measurement unit 104, segment acquisition unit 105, video quality determination unit 106, buffer 107, It is composed of a playback unit 108 and a display unit 109 . In this embodiment, MPEG-DASH is assumed as an example of the content distribution method.

When a program is selected, the manifest acquisition unit 102 generates a manifest request and transmits it to the content distribution server 200 via the communication interface 101 and the Internet. Then, the manifest file (MPD file) of the desired program is obtained from the content distribution server 200 as a response to the manifest request. Then, based on the manifest file, a segment URL list in which segment start times and segment URLs are associated is generated and output to the video quality determination unit 106 .

The scene list acquisition unit 103 generates a scene list request and requests it to the content distribution server 200 via the Internet via the communication interface 101, thereby obtaining a scene list in which the importance score corresponding to the segment number of the video stream is described. to get

Here, a method for generating a scene list will be described. The scene list is generated by the streaming delivery side (distribution side). In the present invention, it is preferable that the distribution side describes importance scores of multiple patterns in the scene list. By doing so, the receiving side (receiving terminal 100) can control the video quality based on the importance of the video scene obtained from the contents and meanings included in the video scene, as will be described later.

For the scoring of the degree of importance obtained from the contents and meanings contained in the video scene, information on the person appearing in the video scene may be used. For example, if there are three persons A, B, and C appearing in the target content, the importance score for each person is calculated for the video scene, so that three patterns of importance scores can be obtained. Become. Assuming that the length of one segment (segment length) is 1 second and the frame rate is 30 fps, the number of frames in one segment is 30 frames. At this time, the number of frames in which the target person appears is calculated for all frames (30 frames) in one segment, and this may be used as the importance score for the segment. Although the segment length is not limited to 1 second, the following description assumes that the segment length is 1 second.

Table 1 shows an example of the scene list at this time. Importance scores for segment numbers are described for each pattern of persons A, B, and C. FIG. In the example of Table 1, in the segment of segment number 1 (total of 30 frames), character A appears in 10 frames, character B appears in 0 frames, and character C appears in 30 frames. The importance scores of the three patterns for the segments of are 10, 0, and 30, respectively.

As for the method of scoring the degree of importance, the example of character information and the number of frames in which it appears has been given. good. Scoring may also be performed based on the meaning of frames other than information about characters and objects that appear, or multiple patterns of importance scores are set so as to correspond to patterns such as the user's preferences that have been grasped in advance. You may

In any case, in the present invention, the distribution side sets a plurality of patterns of importance scores based on the contents and meanings contained in the video scenes, and describes them in the scene list. Although details will be described later, in the present invention, the receiving terminal 100 controls the video quality of the acquired segment using the importance pattern extracted from the importance scores (importance patterns) of multiple patterns described in the scene list. do.

The segment acquisition unit 105 inputs the URL of the segment to be acquired output by the manifest acquisition unit 102 from the video quality determination unit 106, and distributes content via the Internet via the communication interface 101 to a request including the input segment URL. Make a request to the server 200, receive a segment from the content distribution server 200 as a response to the request, output it to the buffer 107, and measure the throughput of the time required for the request and the number of bytes of the acquired segment. Output to the unit 104 .

The throughput measurement unit 104 calculates a throughput value from the information on the time required for the request input from the segment acquisition unit 105 and the number of bytes of the acquired segment. For example, throughput value=(number of bytes in segment)/(time required for request).

The video quality determination unit 106 calculates the target distribution capacity for the rate determination cycle to be acquired next based on the throughput value input from the throughput measurement unit 104 at the start of playback or at the time of acquisition of the final segment in the rate determination cycle, From the scene list input from the scene list acquisition unit 103, an importance pattern corresponding to the user of the receiving terminal 100 input from the display unit 109 is extracted, and the video quality of each segment for the rate determination cycle to be acquired next is determined. The input of the importance pattern on the display unit 109 will be described later.

Here, the rate determination cycle can be set in advance such as 1 cycle = N segments (for example, N = 5), and within this 1 cycle, segments with high importance of video scenes become high-quality video streams. Adjust accordingly. At this time, an array is derived that represents a combination of video qualities (bit rates) that includes more high-quality segments while remaining within the target distribution capacity. By allocating a bit rate of medium high quality video quality, the video quality of each segment for the rate determination period to be obtained next is determined. A specific array derivation method will be described later.

Note that the remaining buffer capacity (difference between the final time of the segment inserted in the buffer 107 and the current playback time) is calculated from the current playback time input from the playback unit 108 and the buffering information input from the buffer 107. if the remaining capacity is less than a buffer threshold (e.g., 1 second) and the calculated throughput value is less than the determined video quality, select the video quality that is less than the determined video quality and has the maximum bitrate; The URL of the segment is output to segment acquisition section 105 .

The buffer 107 receives the segment from the segment acquisition unit 105, stores the segment in correspondence with the playback start time and the playback continuation time (segment length) of the segment, and sequentially retrieves the segment at the playback time requested by the playback unit 108. Output to the reproducing unit 108 . In addition, it outputs buffering information, which is information on the reproduction times of all segments currently being stored and the reproduction duration time, to the video quality determination unit 106 .

Playback unit 108 sequentially inputs segments from buffer 107 according to the current playback time, decodes the input segments according to the media formats (video, audio, text, etc.) of the segments, and outputs them to display unit 109 .

Display unit 109 displays the decoded media data input from playback unit 108 according to the format of the media. The display unit 109 also receives an input of importance pattern by the user, and outputs information on the importance pattern selected by the user to the video quality determination unit 106 .

Here, a method of determining a combination of video qualities of segments for a rate determination period in the present invention will be described with reference to the flow chart of FIG. Here, in the video quality determining unit 106, it is smaller than the target distribution capacity, which is (throughput value)×(the number of segments N for the rate determining cycle) at the time of acquiring the final segment of the rate determining cycle input from the throughput measuring unit 104, and Derive an array that represents the video quality combinations that contain more high quality segments. There are three types of video quality (bit rate): high quality, medium quality, and low quality. Let X be the first segment number of the rate determination period determined here.

When the video quality combination determination process is started, first, in S101, it is determined whether (low quality bit rate)*(segment number N for rate determination period) is smaller than the target distribution capacity. If it is not smaller (NO in S101), proceed to S102, and if smaller (YES in S101), proceed to S104.

In S102, N low quality bit rates are added to the video quality combination array, and the process proceeds to S103. In S103, the elements of the determined array are assigned in descending order of importance score from segment (X) to segment (X+N), and the process ends.

In S104, it is determined whether or not there is capacity left for adding the high quality bit rate. If it can be added (YES in S104), the process proceeds to S105, and if not (NO in S104), the process proceeds to S107.

In S105, add the high quality bitrate to the array and proceed to S106. In S106, it is determined whether all elements of the array have been determined. If all the elements of the array have been determined (YES in S106), the process proceeds to S103, and if all the elements of the array have not been determined (NO in S106), the process returns to S104.

In S103, the elements of the determined array are assigned in descending order of importance score from segment (X) to segment (X+N), and the process ends. In S107, it is determined whether or not there remains a capacity to add the medium quality bit rate. If it can be added (YES in S107), the process proceeds to S108.

At S108, the medium quality bit rate is added to the array and the process proceeds to S109. In S109, it is determined whether all elements of the array have been determined. If all the elements of the array have been determined (YES in S109), the process proceeds to S103, and if all the elements of the array have not been determined (NO in S109), the process returns to S107.

In S110, it is determined whether or not there is remaining capacity to add the low quality bit rate.

In S111, add the low quality bitrate to the array and proceed to S112. In S112, it is determined whether all elements of the array have been determined. If all the elements of the array have been determined (YES in S112), the process proceeds to S103, and if all the elements of the array have not been determined (NO in S112), the process returns to S110.

In S113, it is determined whether the final element in the array is of low quality. If the final element is of low quality (YES in S113), proceed to S114, and if the final element is not of low quality (NO in S113), skip S114. and proceed to S115.

In S114, the final element is deleted and the process proceeds to S115. In S115, the quality of the final element is further stored, the final element is deleted, and the process proceeds to S116.

At S116, it is determined whether or not there is an element in the array. If there is an element in the array (YES in S116), proceed to S117, and if there is no element in the array, proceed to S118.

At S117, it is determined whether the final element is of high quality. If the final element is of high quality (YES in S117), proceed to S107; if the final element is not of high quality (NO in S117), proceed to S110.

Also, in S118, it is determined whether or not the stored quality is high quality. If the stored quality is high quality (YES in S118), the process proceeds to S107, and if the stored quality is not high quality (NO in S118), the process proceeds to S110.

Through the above flow, it is possible to determine the combination array of video qualities in the number of segments corresponding to the rate determination period.

Next, the segment acquisition control method in receiving terminal 100 of the present invention will be described using the flowchart of FIG. In this flowchart, a manifest file and a scene list have already been acquired, and an importance pattern corresponding to the user is extracted from multiple patterns of importance scores (importance patterns) described in the acquired scene list. It shall start from the completed state.

When segment reproduction is started, first, in S201, it is determined whether the reproduction is started or the final segment of the rate determination cycle is being acquired. If playback is started or the final segment of the rate determination cycle is being acquired (YES in S201), the process proceeds to S202.If playback is not started or the final segment of the rate determination cycle is being acquired (NO in S201), S202 is skipped and S203 is performed. proceed to

In S202, the target distribution capacity for the rate determination period to be acquired next is calculated. determines the video quality of each segment for the rate determination cycle to be acquired next based on the importance pattern corresponding to the user in . Note that, at the start of playback, the target distribution capacity may be calculated based on the throughput value at the time of the previous playback.

Next, in S203, the remaining buffer capacity (difference between the final time of the segment inserted in the buffer and the current playback time) is calculated from the current playback time input from the playback unit 108 and the buffering information input from the buffer 107. Then, it is determined whether the remaining buffer capacity is smaller than the buffer threshold value (for example, 1 second) and the calculated throughput value is smaller than the determined video quality input from the video quality determining unit 106 . If it is smaller (YES in S203), proceed to S204. If not (NO in S203), skip S204 and proceed to S205.

In S204, a video quality that is smaller than the determined video quality and has the maximum bit rate is selected.

Next, in S205, the segment request (including the segment URL of the selected video quality) input from the video quality determining unit 106 is transmitted to the distribution server 200 to receive the segment data corresponding to the request, and the process proceeds to S206.

In S206, the received segment data is buffered, and the process proceeds to S207. In S207, the segment data is reproduced, and the process proceeds to S208.

In S208, it is determined whether the current playback time has reached the end time of the content or whether the stop button has been pressed. If the current playback time has not reached the end time of the content and the stop button has not been pressed (NO in S208), the process returns to S201. If the current playback time has reached the end time of the content or if the stop button has been pressed (YES in S208), the process ends.

As described above, in the streaming distribution of moving images, it is possible to view moving images with image quality that considers the importance of video scenes. Furthermore, since the importance obtained from the contents and meanings contained in video scenes differs depending on the user, by describing multiple patterns of importance scores in the scene list, it is possible to determine the importance of scenes according to the user. It is possible to control the video quality based on this.

100 receiving terminal 101 communication interface 102 manifest acquisition unit 103 scene list acquisition unit 104 throughput measurement unit 105 segment acquisition unit 106 video quality determination unit 107 buffer 108 reproduction unit 109 display unit 200 content delivery server

Claims (4)

A receiving terminal that receives video streaming distribution,
Based on a scene list that describes the importance of video scenes in units of segments obtained by dividing a video stream, a video quality determination unit is provided that controls the reception bit rate of segments while making adjustments within the rate determination cycle. A receiving terminal characterized in that it is possible to view moving images with image quality in consideration of importance. 2. The receiving terminal according to claim 1, wherein the reception bit rate control of the segment is performed based on a scene list describing the importance of a plurality of patterns according to the user of the receiving terminal. A segment acquisition control method for controlling video quality in consideration of the importance of video scenes in video streaming distribution,
Based on a scene list that describes the importance of video scenes in units of segments obtained by dividing the video stream, the distribution bit rate of the segments is controlled while adjusting within the rate determination cycle. A segment acquisition control method characterized by enabling video viewing with quality. 4. The segment acquisition control method according to claim 3, wherein the distribution bit rate of the segment is controlled based on a scene list describing the plurality of patterns of importance according to the user viewing the moving image.

Images