JP2022550528A - Repair Mechanism for Adaptive Bitrate Multicast - Google Patents

Abstract

An exemplary device for retrieving media data includes a memory configured to store media data and one or more processors implemented in circuits, the one or more processors retrieving packets receiving data indicating a loss signaling mechanism, and the packet loss signaling mechanism confirming that the segments are sent in chunks, that the Transport Object Identifier (TOI) is contiguous, or that the last of the objects assigned to the TOI is comprising at least one of a packet having a particular flag set in the header of the last packet, a base URL, a maximum latency, or a synchronization point in the file delivery header; of packet loss signaling mechanisms detecting loss of packets using at least one of the lost packets containing the lost media data; and using information in the file delivery header to generate a byte range request for the lost media data. and delivering the appropriate media object to the media application. [Selection drawing] Fig. 6

Description

priority claim

[0001] This application is the subject of U.S. Application Serial No. 17/039,442, filed September 30, 2020, and filed October 1, 2019, the entire contents of each of which are incorporated herein by reference. No. 62/908,949, filed herein.

[0002] This disclosure relates to the transport of media data.

[0003] Digital video capabilities include digital television, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, digital recording devices, digital media players, video gaming devices, video It can be incorporated into a wide range of devices, including game consoles, cellular or satellite radiotelephones, video teleconferencing devices, and the like. Digital video devices use MPEG-2, MPEG-4, and ITU-T H.265 standards for more efficient transmission and reception of digital video information. 263 or ITU-T H.263. 264/MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.264 (also called High Efficiency Video Coding (HEVC)). implement video compression techniques, such as the video compression techniques described in the standards defined by H.265, and extensions to such standards.

[0004] After the video data is encoded, the video data may be packetized for transmission or storage. Video data may be assembled into video files conforming to any of a variety of standards, such as the International Organization for Standardization (ISO) Base Media File Format and its extensions.

[0005] In general, this disclosure describes techniques for detecting packet loss and retrieving lost media data. In particular, according to the techniques of this disclosure, a server device may signal a packet loss signaling mechanism to be used by a client device to detect packet loss for a streamed media bitstream. The packet loss signaling mechanism is based on whether the segments are sent in chunks, the Transport Object Identifier (TOI) is contiguous, or the last packet of the object assigned to the TOI is set in the header of the last packet. a basic uniform resource locator (URL), a maximum latency, or a synchronization point in the file delivery header. A client device may use packet loss signaling mechanism(s) to detect lost packets in a streamed media bitstream. In response to detecting a lost packet, the client device detects the lost packet using a byte range request that specifies the portion (e.g., segment) of the media file that contains the media data contained in the lost packet. can retrieve some or all of the missing media data. In this way, the client device may obtain the appropriate media object, where the appropriate media object may still be missing some of the missing media data, but the appropriate media object may be missing. It can be properly parsed, decoded and formatted so that it can be presented.

[0006] In one example, a method of retrieving media data includes receiving data indicative of a packet loss signaling mechanism, the packet loss signaling mechanism wherein segments are sent in chunks, transport object identifiers (TOIs) are contiguous. or that the last packet of the object assigned to the TOI has a specific flag set in the last packet's header, the base URL, the maximum latency, or the synchronization point in the file delivery header. detecting loss of packets using at least one of the packet loss signaling mechanisms, the lost packets containing lost media data; using information in the file delivery header. to generate a byte range request for the missing media data; and deliver the appropriate media object to the media application.

[0007] In another example, a device for retrieving media data includes a memory configured to store media data, one or more processors implemented in circuits, and one or more a processor of receiving data indicating a packet loss signaling mechanism, the packet loss signaling mechanism indicating that the segments are sent in chunks, that the Transport Object Identifiers (TOIs) are contiguous, or are assigned to the TOIs; comprising at least one of: that the last packet of the retrieved object has a particular flag set in the header of the last packet, a base URL, a maximum latency, or a synchronization point in the file delivery header; detecting loss of packets using at least one of the loss signaling mechanisms, wherein the lost packets contain lost media data; generating a byte range request; and delivering the appropriate media object to the media application.

[0008] In another example, a computer-readable storage medium having instructions stored thereon that, when executed, instruct a processor to receive data indicative of a packet loss signaling mechanism; the segment is sent in chunks, the Transport Object Identifier (TOI) is contiguous, or the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet. a base URL, a maximum latency, or a synchronization point in the file delivery header; detecting packet loss using at least one of the packet loss signaling mechanisms; , the lost packet contains the lost media data; using information in the file delivery header to generate a byte range request for the lost media data; and delivering the appropriate media object to the media application. ,

[0009] In another example, means for a device for retrieving media data to receive data indicative of a packet loss signaling mechanism, the packet loss signaling mechanism indicating that the segments are sent in chunks, the transport object Identifiers (TOIs) are contiguous, or the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet, base URL, maximum latency, or file delivery header means for detecting loss of packets using at least one of the packet loss signaling mechanisms; and wherein the lost packets contain lost media data. means for generating byte range requests for lost media data using information in the file delivery header; and means for delivering the appropriate media object to the media application.

[0010] In another example, the method of sending media data includes sending data indicating a packet loss signaling mechanism, the packet loss signaling mechanism indicating that the segments are sent in chunks, a transport object identifier (TOI) ) is contiguous, or the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet, base URL, maximum latency, or synchronization in the file delivery header. sending one or more packets containing media data via a broadcast or multicast protocol; and byte range requests indicating the media data of one of the lost packets. , receiving from the client device via the unicast protocol; and sending the byte range of media data to the client device via the unicast protocol in response to the byte range request.

[0011] In another example, a device for delivering media data includes a memory configured to store video data, one or more processors implemented in circuits, one or more The plurality of processors sends data indicating a packet loss signaling mechanism, and the packet loss signaling mechanism indicates that the segments are sent in chunks, that transport object identifiers (TOIs) are contiguous, or are assigned to TOIs. at least one of: that the last packet of the object set has a particular flag set in the header of the last packet, a base URL, a maximum latency, or a synchronization point in the file delivery header; sending one or more packets containing media data via a broadcast or multicast protocol; and receiving a byte range request from a client device via a unicast protocol indicating the media data of one of the lost packets. and sending the byte range media data to the client device via a unicast protocol in response to the byte range request.

[0012] In another example, a computer-readable storage medium having instructions stored thereon that, when executed, send to a processor data indicative of a packet loss signaling mechanism; but the segments are sent in chunks, the Transport Object Identifier (TOI) is contiguous, or the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet. a base URL, a maximum latency, or a synchronization point in the file delivery header; sending out one or more packets containing media data via a broadcast or multicast protocol; receiving a byte range request from the client device via the unicast protocol indicating the media data of one of the lost packets; and sending bytes to the client device via the unicast protocol in response to the byte range request. sending out the range of media data; and

[0013] In another example, a device for sending media data has means for sending data indicative of a packet loss signaling mechanism, and the packet loss signaling mechanism indicates that the segments are sent in chunks, transport Object Identifiers (TOIs) are contiguous or the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet, base URL, maximum latency, or file delivery synchronization points in the header; means for sending out one or more packets containing media data via a broadcast or multicast protocol; and media data of one of the lost packets. means for receiving, via a unicast protocol, from a client device a byte-range request indicating the and means.

[0014] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

[0015] FIG. 2 is a block diagram illustrating an exemplary system that implements techniques for streaming media data over a network. [0016] FIG. 2 is a block diagram illustrating an exemplary set of components of a retrieval unit of a client device; [0017] FIG. 1 is a conceptual diagram illustrating elements of exemplary multimedia content. [0018] FIG. 4 is a block diagram illustrating elements of an exemplary video file that may correspond to segments; [0019] FIG. 1 is a conceptual diagram illustrating an example system including a sender and a receiver, in accordance with the techniques of this disclosure. [0020] A flowchart illustrating an exemplary method, in accordance with the techniques of this disclosure.

[0021] tools. ietf. Available at org/html/rfc6726, T. File Delivery over Unidirectional Transport (FLUTE), as described in Paila et al., "FLUTE-File Delivery over Unidirectional Transport," Network Working Group, RFC 6726, November 2012, and ROUTE. For example, unidirectional content delivery protocols specify different mechanisms for packet loss detection at different levels of multicast protocols. This loss detection makes it possible to trigger repair procedures so that entire transport objects are not lost due to packet loss. Repairs are generally initiated as byte range requests, so the receiver must have sufficient information to initiate such requests.

[0022] The procedure may be set up as follows. The sender provides information (eg, required information) in the content delivery protocol setup and in individual packets to identify lost packets and enable repair procedures. The receiver uses that information to repair it, typically by unicast byte range requests, or if they are unavailable or unsuccessful or delayed, the loss is Either it can be signaled through the HTTP interface.

[0023] Loss detection signaling mechanisms may include: At the lowest level, detection of lost packets can be enabled by signaling directly in the header of transport packets. For example, in ROUTE, when a mode is used in which objects and packets within objects are sent in sequence order, the start_offset field and B flag in the LCT packet header, the EXT_TOL LCT extension header, and the transport object identifier (TOI) field can be used. The start_offset field indicates the starting byte position of the data in the current packet relative to the first byte of the Broadcast Object. Here, a packet consists of contiguous data portion(s) of a delivery object. Therefore, the receiver can detect a lost packet when it receives a packet with a start_offset that does not match the start_offset of the last packet and the size of the payload. The b flag may be set in the last packet of the Broadcast Object. This can be used in conjunction with the EXT_TOL LCT header extension (which signals the length of the transport object) and/or the TOI field to detect packet loss. For example, the start of receiving a packet with a new TOI without receiving a packet with the B flag set may indicate a lost packet.

[0024] Repair signaling mechanisms may include: In ROUTE, as described above, the packet's start_offset and size may be used to determine the lost byte range in the delivery object. Thus, for a lost packet j that follows received packet i and precedes received packet k, the byte range of lost data is "lost data byte range = start_offset(i) + size(i) ~start_offset(k)-1". If the lost packet is the last packet of the Broadcast Object, the exact size cannot be determined in this way and the receiver will have to request all bytes until the end of the Broadcast Object.

[0025] Both FLUTE and ROUTE provide expiration time mechanisms for File Delivery Table (FDT) and Extended FDT (EFDT) instances, respectively. It also implicitly signals that multicast packets signaled by this FDT or EFDT instance may not be expected after the above timeout. Signaling the Alternate-Content-Location-1 and Alternate-Content-Location-2 elements in FDT/EFDT allows gateways to find locations for accessing the same object via unicast . Also, object metadata expiration timeouts in FDT/EFDT instances may be signaled in the control plane in service/session metadata signaling in conjunction with or instead of the signaling described above. The control plane may carry the base URL to enable reconstruction of the location of the same content via unicast.

[0026] The signaling between the multicast gateway and the DASH player may include: If the multicast gateway cannot guarantee error-free delivery of content to the DASH player, special signaling between the multicast gateway and the DASH player may be required to handle such cases.

[0027] According to TS 26.346, Section 7.9.2, streaming applications can accept partial files. Applications that use HTTP GET requests to retrieve files should include the "Accept" request header defined in RFC 7231 with the new 3GPP® defined media type "application/3gpp-partial" in the HTTP GET May be included in the request. Applications using HTTP GET requests to retrieve files may include the "reception" request header defined in RFC7231 with the new 3GPP defined media type "application/3gpp-partial" in the GET request. By providing this header and issuing such a partial file accept request, the application signals that it is capable of receiving partial file responses.

[0028] An example of the use of such a reception header is as follows.
Accept: */*, application/3gpp-partial
[0029] In this example, the application indicates that it accepts all media types in the response, as well as the particular "partial" type specified by application/3gpp-partial.

[0030] When an MBMS client receives a normal HTTP GET request containing an "accept" request with media type "application/3gpp-partial", i.e., a partial file accept request, the MBMS client receives a partial file with at least one byte. If received, the MBMS Client may respond with a Partial File Response defined as follows according to TS 26.346.

• Response code shall be set to 200 OK. • Content-Type header shall be set to application/3gpp-partial.

• The message body is a multipart message body and may be of the same format as the multipart/byteranges media type described in RFC 7233, Annex A. A multipart/byteranges media type, each with its own Content-Type and Content-Range fields as a means to convey the byte range(s) of the partial file being returned or contains multiple body parts. Each Content-Type header is set to the Content-Type value given in the FDT instance for this file. An extension header (eg, 3gpp-access-position) may be added that gives the byte position where the handler assigned to the file's Content-Type can access the file. The value can be created from mbms2015:IndependentUnitPositions, if present.

• A cache directive should be included in the response to prevent intermediate proxies from storing an incomplete file and serving it to another application. Examples for such cache directives are "Cache-Control: max-age=0" or "Cache-Control: no-cache".

[0031] If the MBMS client receives a partial file acceptance request, and the MBMS client receives a partial file with no bytes (i.e., only FDT instances describing the file metadata were received), the MBMS client: It may respond with a partial file response defined as follows.

• The response code is set to 416 Requested Range Not Satisfiable • The Content-Type header is set to the value of the Content-Type given in the FDT instance for this file.

• Content-Range is set to bytes */Content-Length and the value of the Content-Length attribute Content-Length is given in the FDT Instance for this file.

[0032] An example of this is given in TS 26.346, Section 7.9.2.3.

[0033] The techniques of this disclosure may use the ISO Base Media File Format, Scalable Video Coding (SVC) File Format, Advanced Video Coding (AVC) File Format, 3rd Generation Partnership Project (3GPP) File Format, and/or Multiview Video It may be applied to video files conforming to video data encapsulated according to either the video coding (MVC) file format, or other similar video file formats.

[0034] In HTTP streaming, frequently used operations include HEAD, GET, and partial GET. The HEAD operation retrieves the header of the file associated with a given Uniform Resource Locator (URL) or Uniform Resource Name (URN) without retrieving the payload associated with the URL or URN. A GET operation retrieves the entire file associated with the given URL or URN. A partial GET operation receives a byte range as an input parameter and retrieves a number of consecutive bytes of the file, where the number of bytes corresponds to the received byte range. Thus, a partial GET operation can obtain one or more individual movie fragments, thus providing movie fragments for HTTP streaming. There can be several track fragments of different tracks in a movie fragment. With HTTP streaming, a media presentation can be a structured collection of data that is accessible to clients. Clients may request and download media data information to present streaming services to users.

[0035] In the example of streaming 3GPP data using HTTP streaming, there may be multiple representations for the video and/or audio data of the multimedia content. As described below, different representations may refer to different coding characteristics (e.g., different profiles or levels of a video coding standard), different coding standards or extensions of a coding standard (such as multiview and/or scalable extensions), or different It can correspond to bitrate. A manifest of such representations may be defined in a Media Presentation Description (MPD) data structure. A media presentation may correspond to a structured collection of data accessible to an HTTP streaming client device. HTTP streaming client devices may request and download media data information in order to present streaming services to users of the client devices. A media presentation may be described in an MPD data structure, which may contain updates to the MPD.

[0036] A media presentation may include a sequence of one or more periods. Each period may continue until the beginning of the next period or, in the case of the last period, until the end of the media presentation. Each period may contain one or more representations of the same media content. A representation can be audio, video, timed text, or one of several alternative encoded versions of such data. Representations may vary by encoding type, eg, by bit rate, resolution, and/or codec for video data and by bit rate, language, and/or codec for audio data. The term representation may be used to refer to a section of encoded audio or video data that corresponds to a particular period of multimedia content and is encoded in a particular manner.

[0037] Expressions of a particular time period may be assigned to groups indicated by attributes in the MPD that indicate the adaptation set to which they belong. Representations in the same adaptation set are generally considered alternatives to each other in that the client device can dynamically and seamlessly switch between these representations, eg, to implement bandwidth adaptation. For example, each representation of video data for a particular time period is selected for decoding to present media data, such as video data or audio data, of multimedia content for the corresponding time period. can be assigned to the same adaptation set as obtained. Media content within a period of time may, in some examples, be represented by either one representation from group 0, if present, or a combination of at most one representation from each non-zero group. Timing data for each representation of a period may be expressed relative to the start time of the period.

[0038] A representation may include one or more segments. Each representation may contain an initialization segment, or each segment of the representation may be self-initializing. When present, the initialization segment may contain initialization information for accessing the representation. Generally, initialization segments do not contain media data. A segment may be uniquely referenced by an identifier such as a uniform resource locator (URL), uniform resource name (URN), or uniform resource identifier (URI). The MPD may give each segment an identifier. In some examples, the MPD may also provide a byte range in the form of a range attribute that may correspond to data for a segment within a file accessible by a URL, URN, or URI.

[0039] Different representations may be selected for substantially simultaneous retrieval for different types of media data. For example, a client device may select audio, video, and timed text representations from which to retrieve segments. In some examples, a client device may select a particular adaptation set for performing bandwidth adaptation. That is, the client device may select an adaptation set that includes video presentations, an adaptation set that includes audio presentations, and/or an adaptation set that includes timed text. Alternatively, the client device selects adaptation sets for some types of media (e.g., video) and direct representations for other types of media (e.g., audio and/or timed text). can choose.

[0040] FIG. 1 is a block diagram illustrating an exemplary system 10 that implements techniques for streaming media data over a network. In this example, system 10 includes content preparation device 20 , server device 60 and client device 40 . Client device 40 and server device 60 are communicatively coupled by network 74, which may comprise the Internet. In some examples, content creation device 20 and server device 60 may also be coupled by network 74 or another network, or may be directly communicatively coupled. In some examples, content creation device 20 and server device 60 may comprise the same device.

[0041] Content creation device 20 comprises audio source 22 and video source 24 in the example of FIG. Audio source 22 may comprise, for example, a microphone that produces electrical signals representing captured audio data to be encoded by audio encoder 26 . Alternatively, audio source 22 may comprise a storage medium storing previously recorded audio data, an audio data generator such as a computerized synthesizer, or any other source of audio data. Video source 24 may be a video data generating unit such as a video camera that produces video data to be encoded by video encoder 28, a storage medium encoded with previously recorded video data, a computer graphics source, or a video Any other source of data may be provided. Content creation device 20 may store multimedia content on separate media that is read by server device 60, although it is not necessarily communicatively coupled to server device 60 in all instances.

[0042] Raw audio and video data may comprise analog or digital data. Analog data may be digitized before being encoded by audio encoder 26 and/or video encoder 28 . Audio source 22 may acquire audio data from call participants while they are speaking, while video source 24 may acquire video data of the call participants. In other examples, audio source 22 may comprise a computer-readable storage medium with stored audio data, and video source 24 may comprise a computer-readable storage medium with stored video data. In this manner, the techniques described in this disclosure may be applied to live, streaming, real-time audio and video data, or archived, pre-recorded audio and video data.

[0043] Audio frames corresponding to video frames are generally captured (or generated) by audio source 22 concurrently with the video data captured (or generated) by video source 24 contained within the video frames. is an audio frame containing audio data that has been processed). For example, audio source 22 captures audio data while call participants generally generate audio data by speaking, and video source 24 simultaneously captures audio data, i.e., while audio source 22 captures audio data, the call participants to capture the video data of Thus, an audio frame may temporally correspond to one or more specific video frames. Accordingly, an audio frame corresponding to a video frame is generally a situation in which audio and video data were captured simultaneously, and a situation in which the audio and video frames comprise simultaneously captured audio and video data, respectively. corresponds to

[0044] In some examples, audio encoder 26 may encode a timestamp in each encoded audio frame that represents the time the audio data for the encoded audio frame was recorded; , may encode a timestamp in each encoded video frame that represents the time the video data for the encoded video frame was recorded. In such examples, an audio frame corresponding to a video frame may comprise an audio frame with a timestamp and a video frame with the same timestamp. Content creation device 20 may generate timestamps from which audio encoder 26 and/or video encoder 28 may be generated, or may be used by audio source 22 and video source 24 to associate audio and video data with timestamps, respectively. may include an internal clock.

[0045] In some examples, audio source 22 may send data corresponding to the time the audio data was recorded to audio encoder 26, and video source 24 may send data corresponding to the time the video data was recorded. may be sent to video encoder 28 . In some examples, audio encoder 26 encodes sequence identifiers in the encoded audio data to indicate the relative temporal order of the encoded audio data without necessarily indicating the absolute time at which the audio data was recorded. Similarly, video encoder 28 may use the sequence identifiers to indicate the relative temporal order of the encoded video data. Similarly, in some examples, sequence identifiers may be mapped with or otherwise correlated with timestamps.

[0046] Audio encoder 26 generally produces a stream of encoded audio data, and video encoder 28 produces a stream of encoded video data. Each individual stream of data (whether audio or video) is sometimes called an elementary stream. An elementary stream is a digitally encoded (possibly compressed) single component of a representation. For example, the coded video or audio portion of the presentation can be an elementary stream. Elementary streams may be converted to packetized elementary streams (PES) before being encapsulated within a video file. Within the same representation, a stream ID may be used to distinguish PES packets belonging to one elementary stream from others. The basic data unit of an elementary stream is a Packetized Elementary Stream (PES) packet. Thus, coded video data generally correspond to elementary video streams. Similarly, audio data corresponds to one or more respective elementary streams.

[0047] ITU-T H. Many video coding standards, such as H.264/AVC and the upcoming High Efficiency Video Coding (HEVC) standard, define syntax, semantics, and decoding processes for error-free bitstreams, all of which: Comply with a profile or level. Video coding standards generally do not specify encoders, but encoders are tasked with ensuring that the generated bitstreams are standard compliant for decoders. In the context of video coding standards, a "profile" corresponds to a subset of algorithms, features or tools and the constraints applied to them. For example, H. A "profile" defined by the H.264 standard is the H.264 standard. It is a subset of the entire bitstream syntax specified by the H.264 standard. A "level" corresponds to limitations on decoder resource consumption, eg, decoder memory and computations related to picture resolution, bit rate, and block processing rate. A profile may be signaled with a profile_idc (profile indicator) value, while a level may be signaled with a level_idc (level indicator) value.

[0048]H. The H.264 standard specifies that the performance of encoders and decoders depends on the values taken by syntax elements in the bitstream, e.g., the specified size of a decoded picture, within the limits imposed by the syntax of a given profile. We recognize that it may still require large fluctuations in H. The H.264 standard further recognizes that in many applications it is neither practical nor economical to implement a decoder capable of addressing all hypothetical uses of syntax within a particular profile. ing. Therefore, H. The H.264 standard defines a "level" as a specified set of constraints imposed on the values of syntax elements in a bitstream. These constraints can be simple restrictions on values. Alternatively, these constraints may take the form of constraints on arithmetic combinations of values (eg, picture width x picture height x number of pictures decoded per second). H. The H.264 standard further specifies that individual implementations may support different levels for each supported profile.

[0049] Profile compliant decoders typically support all features defined in the profile. For example, as a coding feature, B-picture coding is similar to H.264. H.264/AVC baseline profile, but not supported by H.264/AVC. Other profiles of H.264/AVC are supported. A level compliant decoder should be able to decode any bitstream that does not require resources beyond the limits defined in the level. Profile and level definitions can be useful for explainability. For example, during video transmission, profile and level definition pairs may be negotiated and agreed upon for the entire transmission session. More specifically, H. In H.264/AVC, the levels are a limit on the number of macroblocks that need to be processed, a decoded picture buffer (DPB) size, a coded picture buffer (CPB) size, a vertical motion vector range, and two consecutive It may define the maximum number of motion vectors per MB to be used and whether a B-block can have a sub-macroblock partition of less than 8×8 pixels. In this way, the decoder can determine whether it is able to properly decode the bitstream.

[0050] In the example of FIG. 1, encapsulation unit 30 of content creation device 20 includes an elementary stream comprising encoded video data from video encoder 28 and an elementary stream comprising encoded audio data from audio encoder 26. and receive. In some examples, video encoder 28 and audio encoder 26 may each include a packetizer for forming PES packets from encoded data. In other examples, video encoder 28 and audio encoder 26 may each interface with respective packetizers for forming PES packets from encoded data. In yet another example, encapsulation unit 30 may include a packetizer for forming PES packets from encoded audio data and encoded video data.

[0051] Video encoder 28 operates at various bitrates, as well as pixel resolutions, frame rates, compliance with various coding standards, compliance with various profiles and/or levels of profiles for various coding standards, ( For example, to generate different representations of multimedia content with different properties, such as representations having one or more views (for two-dimensional or three-dimensional playback), or other such properties. can encode video data for multimedia content in a variety of ways. A representation, as used in this disclosure, may comprise one of audio data, video data, text data (eg, for subtitles), or other such data. A representation may include elementary streams, such as audio elementary streams or video elementary streams. Each PES packet may contain a stream_id that identifies the elementary stream to which the PES packet belongs. The encapsulation unit 30 is responsible for assembling the elementary streams into different representations of video files (eg segments).

[0052] Encapsulation unit 30 receives PES packets for the elementary streams of the representation from audio encoder 26 and video encoder 28 and forms corresponding network abstraction layer (NAL) units from the PES packets. Coded video segments may be organized into NAL units that provide a “network-friendly” video presentation that addresses applications such as video telephony, storage, broadcasting, or streaming. NAL units may be classified into video coding layer (VCL) NAL units and non-VCL NAL units. A VCL unit may contain the core compression engine and may contain block, macroblock, and/or slice level data. Other NAL units may be non-VCL NAL units. In some examples, a coded picture during one temporal instance, typically presented as a primary coded picture, may be contained in an access unit that may contain one or more NAL units. .

[0053] Non-VCL NAL units may include parameter set NAL units and SEI NAL units, among others. Parameter sets may contain sequence-level header information (in sequence parameter sets (SPS)) and infrequently changing picture-level header information (in picture parameter sets (PPS)). With parameter sets (eg, PPS and SPS), infrequently changing information need not be repeated for each sequence or picture, thus coding efficiency may be improved. Additionally, the use of parameter sets may allow out-of-band transmission of important header information, avoiding the need for redundant transmissions for error resilience. In an example of out-of-band transmission, parameter set NAL units may be transmitted on a different channel than other NAL units, such as SEI NAL units.

[0054] Supplemental Enhancement Information (SEI) is information that is not required to decode the coded picture samples from the VCL NAL unit, but may aid processes related to decoding, display, error resilience, and other purposes. may contain SEI messages may be contained in non-VCL NAL units. SEI messages are a normative part of some standard specifications and are therefore not always mandatory for standard compliant decoder implementations. The SEI message can be a sequence level SEI message or a picture level SEI message. Some sequence level information may be included in SEI messages, such as the Scalability Information SEI message in the SVC example and the View Scalability Information SEI message in MVC. These exemplary SEI messages may, for example, convey information regarding the extraction of operating points and the properties of those operating points. Additionally, encapsulation unit 30 may form a manifest file, such as a Media Presentation Descriptor (MPD) that describes the characteristics of the presentation. Encapsulation unit 30 may format the MPD according to Extensible Markup Language (XML).

[0055] Encapsulation unit 30 may provide data for one or more representations of multimedia content to output interface 32 along with a manifest file (eg, MPD). Output interface 32 may be a universal serial bus (USB) interface, a network interface or interface for writing to storage media, such as a CD or DVD writer or burner, an interface to magnetic or flash storage media, or for storing or transmitting media data. may have other interfaces for Encapsulation unit 30 may provide data for each representation of the multimedia content to output interface 32, which may send the data to server device 60 via network transmission or storage media. In the example of FIG. 1, server device 60 includes storage medium 62 that stores various multimedia content 64, each including a respective manifest file 66 and one or more representations 68A-68N (representation 68). . In some examples, output interface 32 may also send data directly to network 74 .

[0056] In some examples, representation 68 may be separated into adaptive sets. That is, the various subsets of representation 68 may be codecs, profiles and levels, resolution, number of views, file formats for segments, text to be displayed using the representation and/or to be decoded and presented, for example, by a speaker. textual type information that may identify the language or other characteristics of the audio data to be used; camera angle information that may describe the scene camera angle or real-world camera perspective for presentation in the adaptation set; content about a particular audience; Each common set of characteristics may be included, such as rating information that describes suitability.

[0057] Manifest file 66 may contain data that indicates a subset of representations 68 that correspond to a particular adaptation set, as well as common characteristics for adaptation sets. Manifest file 66 may also include data representing individual characteristics for individual representations of the adaptation set, such as bitrate. In this way, the adaptation set may enable simplified network bandwidth adaptation. Expressions in the adaptation set may be indicated using child elements of the adaptation set element of manifest file 66 .

[0058] Server device 60 includes a request processing unit 70 and a network interface 72 . In some examples, server device 60 may include multiple network interfaces. Additionally, any or all of the features of server device 60 may be implemented on other devices of the content distribution network, such as routers, bridges, proxy devices, switches, or other devices. In some examples, intermediate devices of the content distribution network may include components that cache data for multimedia content 64 and substantially conform to components of server device 60 . Network interface 72 is generally configured to send and receive data over network 74 .

Request processing unit 70 is configured to receive network requests for data on storage medium 62 from client devices, such as client device 40 . For example, request processing unit 70 may use R.I. Fielding et al., RFC 2616, "Hypertext Transfer Protocol—HTTP/1.1", Network Working Group, IETF, June 1999, may implement the Hypertext Transfer Protocol (HTTP) version 1.1. That is, request processing unit 70 may be configured to receive HTTP GET requests or partial GET requests and provide data for multimedia content 64 in response to these requests. The request may specify a segment of representation 68 using, for example, the segment's URL. In some examples, the request may also specify one or more byte ranges of the segment, thus comprising a partial GET request. Request processing unit 70 may be further configured to service HTTP HEAD requests to provide header data for one segment of representation 68 . In either case, request processing unit 70 may be configured to process the request to provide the requested data to the requesting device, such as client device 40 .

[0060] Additionally or alternatively, request processing unit 70 may be configured to deliver media data via a broadcast or multicast protocol such as eMBMS. Content creation device 20 may create DASH segments and/or subsegments in substantially the same manner as described, but server device 60 may use eMBMS or another broadcast or multicast network transport protocol. , may deliver these segments or sub-segments. For example, request processing unit 70 may be configured to receive multicast group join requests from client devices 40 . That is, server device 60 may advertise the Internet Protocol (IP) address associated with the multicast group to client devices associated with particular media content (eg, live event broadcasts), including client device 40 . A client device 40 may submit a request to join the multicast group. This request may be propagated throughout the network 74, e.g., the routers that make up the network 74, so that the routers, such as client devices 40, subscribe to traffic destined for IP addresses associated with the multicast group. Pointed at the client device.

[0061] As shown in the example of Figure 1, multimedia content 64 includes a manifest file 66, which may correspond to a Media Presentation Description (MPD). Manifest file 66 may contain descriptions of different alternative representations 68 (e.g., video services with different qualities), which may include, for example, representation 68 codec information, profile values, level values, bitrates, and other descriptive properties. Client device 40 may retrieve the MPD of the media presentation to determine how segments of representation 68 should be accessed.

In particular, retrieval unit 52 may retrieve configuration data (not shown) of client device 40 to determine the decoding capabilities of video decoder 48 and the rendering capabilities of video output 44 . The configuration data may also include language preferences selected by the user of client device 40, one or more camera perspectives corresponding to depth preferences set by the user of client device 40, and/or Any or all of the selected rating preferences may be included. Retrieval unit 52 may, for example, comprise a web browser or media client configured to submit HTTP GET requests and partial GET requests. Retrieval unit 52 may correspond to software instructions executed by one or more processors or processing units (not shown) of client device 40 . In some examples, all or portions of the functionality described with respect to fetch unit 52 may be implemented in hardware, or a combination of hardware, software, and/or firmware, executing instructions for software or firmware. For this purpose, the required hardware can be provided.

[0063] Retrieval unit 52 may compare the decoding and rendering capabilities of client device 40 to characteristics of representation 68 indicated by information in manifest file 66. FIG. Retrieval unit 52 may first retrieve at least a portion of manifest file 66 to determine characteristics of representation 68 . For example, retrieval unit 52 may request a portion of manifest file 66 that describes characteristics of one or more adaptation sets. Retrieval unit 52 may select a subset (eg, an adaptation set) of representations 68 that has characteristics that can be satisfied by the coding and rendering capabilities of client device 40 . Retrieval unit 52 then determines the bitrates for the representations in the adaptation set, determines the currently available amount of network bandwidth, and determines one of the representations with bitrates that can be satisfied by the network bandwidth. You can pick a segment from

[0064] In general, higher bitrate renditions may result in higher quality video playback, and lower bitrate renditions may provide sufficient quality video playback when available network bandwidth is reduced. Thus, when the available network bandwidth is relatively high, retrieval unit 52 may retrieve data from relatively high bitrate representations, but when the available network bandwidth is low, retrieval unit 52 may retrieve data from relatively low bitrate representations. Data can be retrieved from the expression. In this manner, client device 40 may stream multimedia data over network 74 while also adapting to changing network bandwidth availability of network 74 .

[0065] Additionally or alternatively, retrieval unit 52 may be configured to receive data according to a broadcast or multicast network protocol such as eMBMS or IP multicast. In such examples, retrieval unit 52 may submit a request to join a multicast network group associated with particular media content. After joining the multicast group, retrieval unit 52 may receive the data of the multicast group without further requests issued to server device 60 or content creation device 20 . Retrieving unit 52 may submit a request to leave a multicast group when the data of the multicast group is no longer needed, for example, to stop playback or change the channel to a different multicast group.

[0066] Network interface 54 may receive and provide data for segments of the selected representation to retrieval unit 52 , which may provide the segments to decapsulation unit 50 . Decapsulation unit 50 decapsulates the elements of the video file into constituent PES streams and depacketizes the PES stream to retrieve the encoded data, e.g. The encoded data may be sent to either audio decoder 46 or video decoder 48, depending on whether the encoded data is part of an audio stream or a video stream. Audio decoder 46 decodes the encoded audio data and sends the decoded audio data to audio output 42, and video decoder 48 decodes the encoded video data and converts the decoded video data, which may include multiple views of the stream, into video. Send to output 44 .

[0067] Video encoder 28, video decoder 48, audio encoder 26, audio decoder 46, encapsulation unit 30, retrieval unit 52, and decapsulation unit 50 each include, when applicable, one or more microprocessors; Any of a variety of suitable processing circuits such as digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuits, software, hardware, firmware, or any combination thereof. It can be implemented as either Video encoder 28 and video decoder 48 may each be included in one or more encoders or decoders, any of which may be integrated as part of a combined video encoder/decoder (codec). Similarly, audio encoder 26 and audio decoder 46 may each be included in one or more encoders or decoders, any of which may be integrated as part of a composite codec. Devices including video encoder 28, video decoder 48, audio encoder 26, audio decoder 46, encapsulation unit 30, detachment unit 52, and/or decapsulation unit 50 may be integrated circuits, microprocessors, and/or cellular telephones, and the like. wireless communication devices.

[0068] Client device 40, server device 60, and/or content creation device 20 may be configured to operate in accordance with the techniques of this disclosure. By way of example, this disclosure describes these techniques with respect to client device 40 and server device 60 . However, it should be understood that content creation device 20 may be configured to perform these techniques instead of (or in addition to) server device 60 .

[0069] Encapsulation unit 30 comprises a NAL unit comprising a header identifying the program to which the NAL unit belongs, as well as a payload, eg, audio data, video data, or data describing the transport or program stream to which the NAL unit corresponds. can form For example, H. In H.264/AVC, a NAL unit contains a 1-byte header and a payload of varying size. A NAL unit containing video data in its payload may comprise video data of various granularity levels. For example, a NAL unit may comprise a block of video data, multiple blocks, a slice of video data, or an entire picture of video data. Encapsulation unit 30 may receive encoded video data from video encoder 28 in the form of PES packets of elementary streams. Encapsulation unit 30 may associate each elementary stream with a corresponding program.

[0070] Encapsulation unit 30 may also assemble an access unit from multiple NAL units. In general, an access unit may comprise a frame of video data as well as one or more NAL units for representing audio data corresponding to that frame when such audio data is available. An access unit generally includes all NAL units for one output time instance, eg all audio and video data for one time instance. For example, if each view has a frame rate of 20 frames per second (fps), each time instance may correspond to a time interval of 0.05 seconds. During this time interval, a particular frame of all views of the same access unit (same time instance) can be rendered simultaneously. In one example, an access unit may comprise a coded picture during one temporal instance, which may be presented as a primary coded picture.

[0071] Thus, an access unit may comprise all views corresponding to all audio and video frames of a common time instance, eg, time X. This disclosure also refers to the coded pictures of a particular view as "view components." That is, a view component may comprise coded pictures (or frames) for a particular view at a particular time. An access unit may thus be defined as comprising all view components of a common time instance. The decoding order of access units does not necessarily have to be the same as the output or display order.

[0072] A media presentation may include a media presentation description (MPD) that may include descriptions of different alternative representations (eg, video services with different qualities), which may include, for example, codec information and profile value and level value. MPD is an example of a manifest file, such as manifest file 66 . Client device 40 may retrieve the MPD of the media presentation to determine how to access movie fragments of various presentations. A movie fragment can be in a movie fragment box (moof box) of a video file.

[0073] Manifest file 66 (which may comprise, for example, an MPD) may advertise the availability of segments of representation 68. FIG. That is, the MPD may include information indicating the wall-clock time at which the first segment of one of representations 68 will become available, as well as information indicating the duration of the segments within representation 68 . In this manner, retrieval unit 52 of client device 40 may determine when each segment is available based on the start time as well as the duration of the segments preceding the particular segment.

[0074] After encapsulation unit 30 assembles the NAL units and/or access units into a video file based on the received data, encapsulation unit 30 passes the video file to output interface 32 for output. In some examples, encapsulation unit 30 either stores the video file locally or sends the video file to a remote server via output interface 32 rather than sending the video file directly to client device 40. can. Output interface 32 includes, for example, transmitters, transceivers, devices for writing data to computer readable media, such as optical drives, magnetic media drives (e.g., floppy drives), universal serial bus (USB) ports. , a network interface, or other output interface. Output interface 32 outputs the video file to a computer readable medium, such as a transmission signal, magnetic media, optical media, memory, flash drive, or other computer readable medium.

Network interface 54 may receive NAL units or access units via network 74 and provide NAL units or access units to decapsulation unit 50 via retrieval unit 52 . Decapsulation unit 50 decapsulates the elements of the video file into constituent PES streams and depacketizes the PES stream to retrieve the encoded data, e.g. The encoded data may be sent to either audio decoder 46 or video decoder 48, depending on whether the encoded data is part of an audio stream or a video stream. Audio decoder 46 decodes the encoded audio data and sends the decoded audio data to audio output 42, and video decoder 48 decodes the encoded video data and converts the decoded video data, which may include multiple views of the stream, into video. Send to output 44 .

[0076] In accordance with the techniques of this disclosure, server device 60 (or content creation device 20) may signal client device 40 a packet loss detection mechanism. For example, server device 60 may signal a packet loss detection mechanism over the control channel between server device 60 and client device 40 . The packet loss detection mechanism ensures that the segments are sent in chunks, that the Transport Object Identifier (TOI) is contiguous, or that the last packet of the object assigned to the TOI is set in the header of the last packet. It may include one or more of: having a specific flag, base URL, maximum latency, or synchronization points in the file delivery header.

[0077] Client device 40 then implements a signaled packet loss detection mechanism(s) to determine if packets in the broadcast or multicast stream from server device 60 have been lost. obtain. In response to determining that one or more packets have been lost, retrieval unit 52 generates one or more byte ranges specifying the byte range(s) of the segment containing the media data of the lost packets. A byte-range request (ie, an HTTP partial Get request) may be formed. Retrieval unit 52 may then submit these byte range requests to server device 60 to attempt to retrieve the lost media data via a unicast protocol, such as HTTP.

[0078] In some examples, server device 60 may signal an inter-packet timeout value. Client device 40 (and in particular retrieval unit 52) may use the inter-packet timeout value to determine if a packet has been lost, e.g. if not received within the timeout interval. The inter-packet timeout value may correspond to a nominal Common Media Application Format (CMAF) chunk duration. Server device 60 may signal the inter-packet timeout value, for example, in a file delivery table (FDT), enhanced FDT (eFDT), or in control plane services or session metadata. Accordingly, client device 40 may extract the inter-packet timeout value from this data.

[0079] Regardless of whether or not lost media data is received, retrieval unit 52 should be delivered to decapsulation unit 50, and ultimately to audio decoder 46 and/or video decoder 48. can form an appropriate media object to be rendered. Suitable media objects may generally be parseable and decodable by decapsulation unit 50 , audio decoder 46 , and video decoder 48 . A suitable media object may still be missing some of the media data of the lost packet(s), but may nevertheless be parsable and decodable.

[0080] FIG. 2 is a block diagram illustrating in more detail an exemplary set of components of the takeout unit 52 of FIG. In this example, retrieval unit 52 includes eMBMS middleware unit 100 , DASH client 110 and media application 112 .

[0081] In this example, the eMBMS middleware unit 100 further includes an eMBMS receiving unit 106, a cache 104, and a proxy server unit 102. In this example, the eMBMS receiving unit 106 is configured to receive data via eMBMS, eg according to FLUTE. That is, the eMBMS receiving unit 106 may receive files via broadcast from a server device 60, which may act as a Broadcast/Multicast Service Center (BM-SC), for example.

[0082] When the eMBMS middleware unit 100 receives data for a file, the eMBMS middleware unit may store the received data in cache 104. FIG. Cache 104 may comprise a computer-readable storage medium such as flash memory, hard disk, RAM, or any other suitable storage medium.

[0083] Proxy server unit 102 may act as a server for DASH client 110 . For example, proxy server unit 102 may provide DASH client 110 with an MPD file or other manifest file. The proxy server unit 102 may advertise availability times for segments in the MPD file, as well as hyperlinks from which the segments can be retrieved. These hyperlinks may include a local host address prefix corresponding to client device 40 (eg, 127.0.0.1 for IPv4). In this manner, DASH client 110 may request segments from proxy server unit 102 using HTTP GET requests or partial GET requests. For example, for a segment available from the link http://127.0.0.1/rep1/seg3, DASH client 110 makes a request for http://127.0.0.1/rep1/seg3. construct an HTTP GET request containing the request and submit the request to the proxy server unit 102 . Proxy server unit 102 may retrieve requested data from cache 104 and provide the data to DASH client 110 in response to such requests.

[0084] FIG. 3 is a conceptual diagram illustrating elements of exemplary multimedia content 120. As shown in FIG. Multimedia content 120 may correspond to multimedia content 64 ( FIG. 1 ) or other multimedia content stored on storage medium 62 . In the example of FIG. 3, multimedia content 120 includes a media presentation description (MPD) and multiple presentations 124A-124N (representation 124). Representation 124A includes optional header data 126 and segments 128A-128N (segment 128), and representation 124N includes optional header data 130 and segments 132A-132N (segment 132). The letter N is used for convenience to designate the last movie fragment in each of representations 124 . In some examples, there may be different numbers of movie fragments between representations 124 .

[0085] MPD 122 may comprise a data structure separate from representation 124; MPD 122 may correspond to manifest file 66 of FIG. Similarly, representation 124 may correspond to representation 68 in FIG. In general, MPD 122 provides coding and rendering characteristics, adaptation sets, profiles that MPD 122 supports, text type information, camera angle information, rating information, trick mode information (e.g., information indicating representations that include temporal subsequences), and/or or may include data generally describing characteristics of representation 124, such as information for retrieving remote time periods (eg, for targeted ad insertion into the media content being played).

[0086] Header data 126, when present, may describe characteristics of segment 128, such as the temporal location of a random access point (RAP, also called stream access point (SAP)), which random access point of segment 128 may be: Including a random access point, a byte offset to a random access point within segment 128 , a uniform resource locator (URL) of segment 128 , or other aspects of segment 128 . Header data 130, when present, may describe similar characteristics for segment 132. FIG. Additionally or alternatively, such features may be contained entirely within MPD 122 .

[0087] Segments 128, 132 include one or more coded video samples, each of which may include a frame or slice of video data. Each of the coded video samples of segment 128 may have similar characteristics, such as height, width, and bandwidth requirements. Such characteristics may be described by MPD 122 data, but such data are not shown in the example of FIG. MPD 122 may include characteristics described by 3GPP specifications, with the addition of any or all of the signaled information described in this disclosure.

[0088] Each of the segments 128, 132 may be associated with a unique uniform resource locator (URL). Accordingly, each of the segments 128, 132 may be independently retrievable using a streaming network protocol such as DASH. In this manner, a destination device such as client device 40 may use an HTTP GET request to retrieve segment 128 or 132 . In some examples, client device 40 may use an HTTP partial GET request to retrieve a particular byte range of segment 128 or 132 .

[0089] FIG. 4 is a block diagram illustrating elements of an exemplary video file 150 that may correspond to segments of a representation, such as one of segments 128, 132 of FIG. Each of the segments 128, 132 may contain data substantially conforming to the organization of data shown in the example of FIG. Video file 150 is sometimes said to encapsulate segments. As explained above, the ISO base media file format and its extension video files store data in a series of objects called "boxes". In the example of FIG. 4, the video file 150 includes a file type (FTYP) box 152, a movie (MOOV) box 154, a segment index (sidx) box 162, a movie fragment (MOOF) box 164, and a movie fragment random access box. (MFRA) box 166 . Although FIG. 4 represents an example video file, other media files may be other types of media data (e.g., , audio data, timed text data, etc.).

[0090] A file type (FTYP) box 152 generally describes the file type for the video file 150 . File type box 152 may contain data identifying a specification that describes the best use of video file 150 . File type box 152 may alternatively be placed before MOOV box 154 , movie fragment box 164 , and/or MFRA box 166 .

[0091] In some examples, a segment, such as video file 150, may include an MPD update box (not shown) before the FTYP box 152. The MPD update box may contain information to update the MPD as well as information indicating that the MPD corresponding to the presentation containing the video file 150 should be updated. For example, the MPD update box may give the URI or URL for the resource that should be used to update the MPD. As another example, the MPD update box may contain data for updating the MPD. In some examples, the MPD update box may immediately follow a segment type (STYP) box (not shown) for video file 150, where the STYP box indicates the segment type for video file 150. can be defined.

[0092] MOOV boxes 154, in the example of FIG. Generally, MVHD box 156 may describe general characteristics of video file 150 . For example, the MVHD box 156 contains data describing when the video file 150 was first created, when the video file 150 was last modified, the time scale for the video file 150, the duration of playback for the video file 150 , or other data that generally describes the video file 150 .

[0093] The TRAK box 158 may contain data about the tracks of the video file 150. FIG. A TRAK box 158 may include a Track Header (TKHD) box that describes characteristics of the track corresponding to the TRAK box 158 . In some examples, the TRAK box 158 may contain coded video pictures, and in other examples, the coded video pictures of the tracks in the movie fragments 164 may be referenced by data in the TRAK box 158 and/or the sidx box 162. can be included in

[0094] In some examples, the video file 150 may include more than one track. Therefore, MOOV box 154 may contain a number of TRAK boxes equal to the number of tracks in video file 150 . TRAK box 158 may describe characteristics of the corresponding track of video file 150 . For example, TRAK box 158 may describe temporal and/or spatial information for the corresponding track. When encapsulation unit 30 (FIG. 3) includes a parameter set track in a video file, such as video file 150, a TRAK box similar to TRAK box 158 of MOOV box 154 may describe the characteristics of the parameter set track. Encapsulation unit 30 may signal the presence of a sequence level SEI message in a parameter set track within a TRAK box that describes the parameter set track.

[0095] The MVEX box 160 may, for example, specify the properties of the corresponding movie fragment 164 to signal that the video file 150 contains the movie fragment 164 in addition to the video data contained within the MOOV box 154, if any. can be described. In the context of streaming video data, coded video pictures may be contained in movie fragments 164 rather than in MOOV boxes 154 . Therefore, all coded video samples may be contained in movie fragment 164 rather than in MOOV box 154 .

[0096] The MOOV box 154 may contain a number of MVEX boxes 160 equal to the number of movie fragments 164 in the video file 150. Each of MVEX boxes 160 may describe the properties of a corresponding one of movie fragments 164 . For example, each MVEX box may include a movie extension header box (MEHD) box that describes the duration for the corresponding one of movie fragments 164 .

[0097] As mentioned above, encapsulation unit 30 may store the sequence data set in video samples that do not contain the actual coded video data. A video sample may generally correspond to an access unit, which is a representation of a coded picture at a particular instance of time. In the context of AVC, a coded picture consists of one or more VCL NAL units containing information for building all pixels of an access unit and other associated non-VCL NAL units such as SEI messages. include. Thus, encapsulation unit 30 may include sequence data sets, which may include sequence level SEI messages in one of movie fragments 164 . Encapsulation unit 30 further detects the presence of the sequence data set and/or the sequence-level SEI message to one of movie fragments 164 in one of MVEX boxes 160 corresponding to one of movie fragments 164 . can be signaled as being in

[0098] SIDX box 162 is an optional element of video file 150 . That is, video files conforming to the 3GPP file format, or other such file formats, do not necessarily contain SIDX boxes 162 . According to an example 3GPP file format, SIDX boxes may be used to identify sub-segments of a segment (eg, a segment contained within video file 150). The 3GPP file format defines a subsegment as "one or more contiguous subsegments with corresponding Media Data Box(s) and Media Data Boxes containing data referenced by Movie Fragment boxes. A stand-alone set of movie fragment boxes must follow that movie fragment box and precede the next movie fragment box containing information about the same track." The 3GPP file format also states that a SIDX box "contains a sequence of references to sub-segments of the (sub-)segment documented by the box. The referenced sub-segments are adjacent in presentation time. Similarly, the segment Bytes referenced by an index box are always adjacent within a segment.The referenced size gives a count of the number of bytes in the referenced material.

[0099] SIDX box 162 generally provides information representing one or more sub-segments of a segment contained in video file 150. FIG. For example, such information may include playback times at which the sub-segments begin and/or end, byte offsets for the sub-segments, whether the sub-segments include (eg, start at) a Stream Access Point (SAP), the SAP's for the type (e.g., whether the SAP is an instantaneous decoder refresh (IDR) picture, a clean random access (CRA) picture, a broken link access (BLA) picture, etc.), during the subsegment (playback time and/or location of the SAP (in terms of byte offset), etc.

[0100] Movie fragment 164 may include one or more coded video pictures. In some examples, movie fragment 164 may include one or more groups of pictures (GOPs), each of which may include a number of coded video pictures, eg, frames or pictures. Additionally, as explained above, movie fragments 164 may include sequence data sets in some examples. Each of movie fragments 164 may include a movie fragment header box (MFHD, not shown in FIG. 4). The MFHD box may describe properties of the corresponding movie fragment, such as the sequence number for the movie fragment. Movie fragments 164 may be included in sequence number order in video file 150 .

[0101] MFRA box 166 may describe random access points within movie fragment 164 of video file 150. FIG. This may help perform trick modes, such as performing a seek to a particular time location (ie, playback time) within the segment encapsulated by the video file 150 . The MFRA box 166 is generally optional and need not be included in the video file in some examples. Similarly, a client device, such as client device 40 , does not necessarily need to refer to MFRA box 166 in order to properly decode and display the video data of video file 150 . MFRA box 166 includes a number of track fragment random access (TFRA) blocks equal to the number of tracks in video file 150, or in some examples equal to the number of media tracks (e.g., non-hinted tracks) in video file 150. ) box (not shown).

[0102] In some examples, movie fragment 164 may include one or more stream access points (SAPs), such as IDR pictures. Similarly, MFRA box 166 may provide an indication of location within SAP's video file 150 . Thus, temporal subsequences of video file 150 can be formed from the SAPs of video file 150 . A temporal subsequence may also include other pictures, such as SAP-dependent P-frames and/or B-frames. Frames and/or slices of a temporal subsequence may be organized into segments such that frames/slices of a temporal subsequence that depend on other frames/slices of the subsequence may be properly decoded. For example, in a hierarchical arrangement of data, data used for prediction of other data may also be included in the temporal subsequences.

[0103] FIG. 5 is a conceptual diagram illustrating an example system including a transmitter and a receiver in accordance with the techniques of this disclosure. The senders of FIG. 5 generally include a media encoder 200 , a CMAF/file format (FF) packager 202 , a DASH packager 204 , a ROUTE sender 206 and a CDN origin server 208 . Media encoder 200 encodes media data, such as audio or video data. Media encoder 200 may correspond to audio encoder 26 or video encoder 28 of FIG. CMAF/FF packager 202 and DASH packager 204 may correspond to encapsulation unit 30 of FIG. ROUTE sender 206 and CDN origin server 208 may correspond to request processing unit 70 and server device 60 of FIG.

[0104] The receiver of FIG. 5 generally includes a ROUTE receiver 210, a DASH client 212, a CMAF/FF parser 214, and a media decoder 216. ROUTE receiver 210 and DASH client 212 may correspond to retrieval unit 52 of FIG. CMAF/FF parser 214 may correspond to decapsulation unit 50 of FIG. Media decoder 216 may correspond to either or both of audio decoder 46 and video decoder 48 .

[0105] Media encoder 200 provides encoded media data to CMAF/FF packager 202, which converts the encoded media data according to a particular file format, such as CMAF and ISO BMFF or extensions thereof. Format to file. CMAF/FF packager 202 provides these files (eg, chunks) to DASH packager 204, which aggregates the files/chunks into DASH segments. DASH packager 204 may also create a manifest file, such as an MPD, containing data describing files/chunks/segments. Further, in accordance with the techniques of this disclosure, ROTUE sender 206 and/or CDN origin server 208 are configured to detect packet loss, for example, by ROUTE receiver 210 and/or DASH client 212. Or it may signal multiple packet loss detection mechanisms.

[0106] DASH packager 204 provides segments to ROUTE sender 206 and CDN origin server 208 along with MPDs. ROUTE sender 206 and CDN origin server 208 may correspond to server device 60 of FIG. 1 or CDN 220 of FIG. In general, ROUTE sender 206 may send media data to ROUTE receiver 210 according to the ROUTE in this example. In other examples, other file-based delivery protocols such as FLUTE may be used for broadcast or multicast. Additionally or alternatively, CDN origin server 208 may send media data to ROUTE receiver 210 and/or directly to DASH client 212, eg, according to HTTP.

[0107] The ROUTE receiver 210 may be implemented in middleware, such as the eMBMS middleware unit 100 of FIG. ROUTE receiver 210 may buffer received media data, for example, in cache 104 shown in FIG. DASH client 212 (which may correspond to DASH client 110 in FIG. 2) may retrieve cached media data from ROUTE receiver 210 using HTTP. Alternatively, DASH client 212 may retrieve media data directly from CDN origin server 208 according to HTTP, as described above.

[0108] Further, in accordance with the techniques of this disclosure, ROUTE receiver 210, for example, uses one or more signaling A modified packet loss detection mechanism may be used. In response, ROUTE receiver 210 may provide data to DASH client 212 indicating that packet loss has occurred. Additionally or alternatively, DASH client 212 may use one or more of the signaled packet loss detection mechanisms to determine that packet loss has occurred. In either case, the DASH client 212 forms a byte-range request, for example using an HTTP partial Get request, to retrieve the media data corresponding to the media data contained in the lost packet from the CDN origin server 208. can. In this manner, the receiver of FIG. 5 may generally receive media data via broadcast or multicast, but when some of the media data is lost, DASH client 212 uses unicast, e.g., HTTP. to retrieve lost media data.

[0109] This disclosure provides ROUTE sending to signal properties regarding how data is distributed, as well as routing from ROUTE receiver 210 to DASH client 212 if media data may be partially lost. Focus primarily on the interface to

[0110] The current Digital Video Broadcasting (DVB) Adaptive Bitrate (ABR) specification allows for multicast delivery in a low-latency mode, but the conventional mechanisms described above are for this low-latency mode. Not regulated, which results in their limitations. These limitations can cause unwanted stalls in playback, especially for low latency streams.

[0111] This disclosure recognizes several problems that can arise with conventional techniques, as described below.

[0112] The mechanisms in Sections 1.3 and 1.4 of the DVB ABR specification may operate to detect loss only after the nominal reception duration of all broadcast objects at the receiver side at the earliest. For example, if multicasting a DASH representation with a nominal segment duration of 2 seconds, this detection will occur, at the earliest, 2 seconds after the start of reception of the object. This can lead to significantly longer playback stalls due to packet loss.

[0113] The mechanisms in Section 1.2 of the DVB ABR specification can only operate after receipt of a subsequent packet (a packet received after the lost packet). Such reception is generally delayed for a long time, for example under bad channel conditions where packet reception stops suddenly and for long periods of time. Under this case, the receiver defaults to the mechanisms of Sections 1.3 and 1.4.

[0114] The mechanism in Section 1.2 of the DVB ABR specification is also not robust against out-of-order packet delivery.

[0115] The mechanism in Section 1.5 of the DVB ABR specification is defined only for full object reception and not for the low latency case.

[0116] Overall, therefore, according to Section 1.3 of the DVB ABR specification, using conventional mechanisms, the earliest detection of packet loss is typically no more than the nominal DASH segment duration. is done after

[0117] This disclosure describes techniques that may address the problems described above. Senders and receivers, such as source device 60 and client device 40, may signal data on control channels as follows.

1. Signal in the control channel that the segments are sent in chunks. When a chunk is completed at the sender, sending begins. This means that sending signaling can be abstracted to indicate that the interval between sending two packets is at most a certain duration (this signal may be added in the setup when the chunk duration is determined). The sending interval can be at most the chunk duration.

2. Signal in the control channel that the TOI is continuous.

3. Signaling on the control channel that the last packet of the object assigned to the TOI has a specific flag set in the packet header.

4. Based on the TOI of the received FLUTE/ROUTE object, a template base URL is signaled in the control channel that allows the generation of URLs where the same object can be requested in unicast over HTTP.

5. Signals the maximum latency for packets until they are given to the application client, even if the packets are partial.

6. ROUTE headers, i.e., signaling ISO BMFF synchronization points at the byte-range level, on chunk boundaries.

[0118] Client device 40, ROUTE receiver 210, or other receivers of FIG. 1 may utilize the signaled information to do the following.

1. Detecting the loss of one or several packets in delivery early enough by one or more of the signaling mechanisms.

2. The information in the ROUTE/FLUTE headers, namely TOI and start_offset, is used to generate the appropriate byte range request for the missing data.

3. Deliver the appropriate object to the application client (eg, DASH client 110 or media application 112 in FIG. 2) in time.

a. The gateway intercepts the transfer using, for example, HTTP 504 code, and the DASH player is adapted to handle such events. or,
b. A response is sent even if it is partial. The partial response contains the byte ranges of the objects that were correctly received as well as resynchronization points at the byte stream level.

[0119] An application (eg, media application 112 of FIG. 2) may take advantage of this information to present the best quality by using all available information received in time. Applications may accept partial data with resynchronization points.

[0120] Details of the chunk duration signaling may include the following. Early detection of transport packet loss may be enabled by inter-packet timeout signaling. Such timeouts may be set according to the nominal CMAF chunk duration of the stream being transported. Signaling can be done in multicast transport packet headers, or in multicast object metadata (FDT/eFDT), or in the control plane (service/session metadata). Signaling may take into account the maximum expected packet arrival jitter. If the signaling does not take this into account, the receiver implementation may use its experienced packet reception jitter. A receiver (eg, client device 40) may request data via unicast to ensure that playback is minimally stalled upon expiration of the inter-packet timeout described above. Since the receiver only determines the start of the byte range it needs to request, not how much data it needs, it can send the data via HTTP chunked mode. where it may request a reasonably large amount of data (eg nominal segment size) and may cancel reception when multicast reception resumes.

[0121] This solution may generally allow detection of packet loss within a duration of the order of the CMAF chunk duration, eg, 100-200 ms, whereas the state of the art does not allow for such detection. can take about 1-2 seconds. Such early detection may allow early repair attempts to be triggered, minimizing the duration of the stall in such cases.

[0122] In some examples, the DASH client 212 may make byte range requests to repair transport packet loss. After detecting packet loss as described above, DASH client 212 may form a unicast byte range request. Byte range determination of lost data was described above. A mechanism for retrieving the address of repair data is available at www. dvb. org/resources/restricted/members/documents/TM-IPI/ABR%20Multicast/TM-IPI3626r4_ABR-Multicast-taskforce---Unicast-repair-draft-specification. Documented in DVB Working Draft Section 9, TM-IPI3626r4: ABR Multicast Task Force - Unicast Repair Draft Specification, available in docx.

[0123] DASH client 212 and/or ROUTE receiver 210 may also handle packet repair failures. Ultimately, packet repair attempts may fail or not be completed in time. In this case, the multicast gateway may signal this problem to DASH client 212 . There are two possibilities for dealing with this situation.

1. Gateways interrupt transfers using, for example, HTTP 504 code, and DASH players are adapted to handle such events.

2. Additionally or alternatively, responses are sent even if the packet is partially received using the partial file signaling mechanism as described above. According to this mechanism, an application indicates its ability to accept partial files in its HTTP GET request. The partial response contains the byte ranges of the objects that were correctly received as well as resynchronization points at the byte stream level.

[0124] DASH client 212 and/or ROUTE receiver 210 may also handle transport object loss. One mechanism for handling complete loss of one or more transport objects and their associated metadata (eg, FDT) is to rely on the availability of unicast connections. A multicast gateway may be configured as an HTTP reverse proxy and may use a unicast connection, for example, to fetch a lost object when a request for that object is made by a player. The gateway may use the URL of the requested object and the unicast content location signaling as described above to retrieve the same content via unicast.

[0125] FIG. 6 is a flowchart illustrating an exemplary method, in accordance with the techniques of this disclosure. The method of FIG. 6 is described with respect to server device 60 and client device 40 of FIG. Other devices, such as the sender and receiver devices of FIG. 5, may be configured to implement this or similar methods.

[0126] Initially, server device 60 signals (250) one or more packet loss detection mechanisms. These mechanisms are that the segments are sent in chunks, that the Transport Object Identifier (TOI) is contiguous, or that the last packet of the object assigned to the TOI is set in the header of the last packet. It may include one or more of having a specific flag, a base URL, a maximum latency, or a synchronization point in the file delivery header. Client device 40 may also receive signaled data representing a packet loss detection mechanism (252).

[0127] Server device 60 may also send media data via a streaming protocol (254). For example, server device 60 may send media data via FLUTE, ROUTE, or other network or over-the-air (OTA) broadcast standards such as DVB. Client device 40 may then receive at least a portion of the media data (256). Client device 40 may also use packet loss detection mechanism(s) to determine whether one or more packets have been lost (258). If no packets were lost (“No” branch of 258), client device 40 may proceed to present 272 the media object, eg, audio and/or video data.

[0128] On the other hand, if at least one packet was lost (the "yes" branch of 258), client device 40 may retrieve the media data contained in the lost packet and the corresponding position of that packet's media data in the segment. and can be determined. For example, client device 40 may use a transport object identifier (TOI) and start_offset to determine the starting byte position of media data in a segment. Client device 40 may further determine the ending byte position of the media data of the lost packet, eg, using the signaled chunk duration. Client device 40 may then form a byte-range request for the lost media data of the lost packet, such as an HTTP partial Get request specifying the byte-range (260).

[0129] Client device 40 may then send a byte range request to server device 60 (262). In response to receiving (264) the byte range request, server device 60 may send (266) the requested byte range to client device 40, eg, according to a unicast protocol, such as HTTP. Client device 40 may then receive the requested byte range (268) and form an appropriate media object (270). Client device 40 may also present the media object (272).

[0130] Thus, the method of FIG. Contiguous, or the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet, the base URL, the maximum latency, or the synchronization point in the file delivery header. detecting loss of packets using at least one of the packet loss signaling mechanisms, the lost packets containing lost media data; using information in the file delivery header. to generate a byte range request for the missing media data; and delivering the appropriate media object to the media application.

[0131] The method of FIG. 6 also includes sending data indicating a packet loss signaling mechanism, and the packet loss signaling mechanism confirming that the segments are sent in chunks and that the transport object identifier (TOI) is contiguous. , or that the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet, the base URL, the maximum latency, or the synchronization point in the file delivery header. sending one or more packets containing media data via a broadcast or multicast protocol; and byte range requests indicating the media data of one of the lost packets via a unicast protocol. receiving from a client device over a byte range request; and sending a byte range of media data to a client device via a unicast protocol in response to a byte range request. .

[0132] Some techniques of this disclosure are summarized in the following examples.

[0133] Example 1: A method of retrieving media data, the method comprising: receiving data indicative of a packet loss signaling mechanism; , that the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet, the base URL, the maximum latency, or the synchronization point in the file delivery header. detecting loss of packets using at least one of the packet loss signaling mechanisms, the lost packets containing lost media data; using information in the file delivery header to detect loss; generating a byte range request for the media data received; and delivering the appropriate media object to the media application.

[0134] Example 2: The method of Example 1, wherein the file delivery header comprises a ROUTE header or a FLUTE header.

[0135] Example 3: The method of any of Examples 1 and 2, wherein the information in the file delivery header comprises a TOI and a start_offset value.

[0136] Example 4: The method of any of Examples 1-3, wherein the appropriate media object is a partial media object missing at least a portion of the lost media data.

[0137] Example 5: The method of any of Examples 1-4, wherein delivering the appropriate media object comprises delivering the appropriate media object by an expected delivery time for the media object.

[0138] Example 6: The method of any of Examples 1-5, wherein detecting packet loss comprises detecting packet loss using a signaled inter-packet timeout.

[0139] Example 7: The method of Example 6, wherein the inter-packet timeout is signaled according to a nominal CMAF chunk duration.

[0140] Example 8: Determining the chunk duration from multicast object metadata in the multicast transport packet header, File Delivery Table (FDT) or Enhanced FDT (EFDT), or in the control plane service or session metadata The method of any of Examples 6 and 7, further comprising:

[0141] Example 9: The method of any of Examples 6-8, further comprising determining whether the chunk duration signaling takes into account maximum expected packet arrival jitter.

[0142] Example 10: The method of Example 9, wherein the chunk duration signaling is determined according to experienced packet reception jitter when the chunk duration signaling does not consider the maximum expected packet arrival jitter.

[0143] Example 11: The method of any of Examples 6-10, wherein generating the byte-range request comprises generating the byte-range request after expiration of an inter-packet timeout.

[0144] Example 12: The method of any of Examples 1-11, further comprising submitting the byte range request via a unicast protocol.

[0145] Example 13: The method of Example 12, wherein the unicast protocol comprises HTTP.

[0146] Example 14: A device for retrieving media data, the device comprising one or more means for performing the method of any of Examples 1-13.

[0147] Example 15: The device of Example 14, wherein the one or more means comprises one or more processors implemented in circuits.

[0148] Example 16: The device of Example 14, wherein the device comprises at least one of an integrated circuit, a microprocessor, or a wireless communication device.

[0149] Example 17: A computer-readable storage medium storing instructions that, when executed, cause a processor to perform the method of any of Examples 1-13.

[0150] Example 18: A device for retrieving media data, the device comprising: means for receiving data indicative of a packet loss signaling mechanism; Contiguous, the last packet of the object assigned to the TOI has a specific flag set in the last packet's header, the base URL, the maximum latency, or the synchronization point in the file delivery header. means for detecting loss of packets using at least one of the packet loss signaling mechanisms, the lost packets containing lost media data; information in the file delivery header means for generating a byte range request for lost media data using a; and means for delivering the appropriate media object to the media application.

[0151] Example 19: A method of sending media data, the method sending data indicating a packet loss signaling mechanism, the data being sent in chunks in segments, the TOI being contiguous at least one of that the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet, the base URL, the maximum latency, or the synchronization point in the file delivery header. sending one or more packets containing media data via a broadcast or multicast protocol; and a byte range request indicating the media data of one of the lost packets via a unicast protocol. and sending the byte range media data to the client device via a unicast protocol in response to the byte range request.

[0152] Example 20: A device for transmitting media data, the device comprising one or more means for implementing the method of Example 19.

[0153] Example 21: The device of Example 20, wherein the device comprises at least one of an integrated circuit, a microprocessor, or a wireless communication device.

[0154] Example 22: A computer-readable storage medium storing instructions that, when executed, cause a processor to perform the method set forth in Example 19.

[0155] Example 23: A device for sending media data, the device comprising means for sending data indicating a packet loss signaling mechanism, the data indicating that segments are sent in chunks, TOI is contiguous, the last packet of the object assigned to the TOI has a specific flag set in the header of the last packet, the base URL, the maximum latency, or the synchronization point in the file delivery header. means for sending one or more packets containing media data via a broadcast or multicast protocol; and a byte range request indicating the media data of one of the lost packets, A device comprising: means for receiving from a client device via a unicast protocol; and means for sending byte-range media data to the client device via the unicast protocol in response to a byte-range request.

[0156] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code to be executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as a data storage medium or any medium that facilitates transfer of a computer program from one place to another, for example, according to a communication protocol. may include communication media including the medium of In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media is any use that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementation of the techniques described in this disclosure. possible medium. A computer program product may include a computer-readable medium.

[0157] By way of example and not limitation, such computer readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, instructions sent from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave Where applicable, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc refer to compact disc (CD), laser disc (disc), optical disc (disc), digital versatile disc ( DVDs), floppy disks and Blu-ray® discs, where a disk usually reproduces data magnetically and a disc stores data. Optically reproduced with a laser. Combinations of the above should also be included within the scope of computer-readable media.

[0158] Instructions may be implemented in one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. etc., can be executed by one or more processors. Accordingly, the term "processor" as used herein may refer to either the structure described above or any other structure suitable for implementing the techniques described herein. Moreover, in some aspects the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated into a composite codec. Also, the techniques may be fully implemented in one or more circuits or logic elements.

[0159] The techniques of this disclosure may be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (eg, chipsets). Although various components, modules or units have been described in this disclosure to emphasize functional aspects of devices configured to implement the disclosed techniques, such components, modules or units A unit does not necessarily require implementation by different hardware units. Rather, as described above, the various units are combined in a codec hardware unit, including one or more processors described above, along with suitable software and/or firmware. It can be provided by a collection of operable hardware units.

[0160] Various examples have been described. These and other examples are within the scope of the following claims.

Claims (41)

A method of retrieving media data comprising:
receiving data indicative of a packet loss signaling mechanism, the packet loss signaling mechanism indicating that segments are sent in chunks, that transport object identifiers (TOIs) are contiguous, or that objects assigned to TOIs are contiguous; at least one of a last packet having a particular flag set in the header of said last packet, a base URL, a maximum latency, or a synchronization point in a file delivery header;
detecting loss of packets using the at least one of the packet loss signaling mechanisms, the lost packets containing lost media data;
generating a byte range request for the lost media data using information in the file delivery header;
delivering the appropriate media object to the media application;
A method. 2. The method of claim 1, wherein detecting the loss of the packet comprises detecting the loss of the packet using a signaled inter-packet timeout value. 3. The method of claim 2, wherein the inter-packet timeout value is signaled according to a nominal Common Media Application Format (CMAF) chunk duration. 2. Further comprising determining chunk duration from multicast object metadata in a multicast transport packet header, File Delivery Table (FDT) or Extended FDT (EFDT), or in control plane service or session metadata. The method described in . 3. The method of claim 2, further comprising determining whether chunk duration signaling considers maximum expected packet arrival jitter. 6. The method of claim 5, determining the chunk duration signaling according to experienced packet reception jitter when the chunk duration signaling does not consider the maximum expected packet arrival jitter. 7. The method of claim 6, wherein generating the byte-range request comprises generating the byte-range request after expiration of the inter-packet timeout value. 2. The method of claim 1, wherein the file delivery header comprises a ROUTE header or a FLUTE header. 2. The method of claim 1, wherein the information in the file delivery header comprises a TOI and a start_offset value. 2. The method of claim 1, wherein the relevant media object is a partial media object missing at least part of the lost media data. 2. The method of claim 1, wherein delivering the appropriate media object comprises delivering the appropriate media object by an expected delivery time for the media object. 2. The method of claim 1, further comprising submitting the byte range request via a unicast protocol. 13. The method of claim 12, wherein said unicast protocol comprises HTTP. A device for retrieving media data, said device comprising:
a memory configured to store media data;
and one or more processors implemented in a circuit, the one or more processors comprising:
receiving data indicative of a packet loss signaling mechanism, the packet loss signaling mechanism indicating that segments are sent in chunks, that transport object identifiers (TOIs) are contiguous, or that objects assigned to TOIs are contiguous; at least one of a last packet having a particular flag set in the header of said last packet, a base URL, a maximum latency, or a synchronization point in a file delivery header;
detecting loss of packets using the at least one of the packet loss signaling mechanisms, the lost packets containing lost media data;
generating a byte range request for the lost media data using information in the file delivery header;
delivering the appropriate media object to the media application;
A device configured to 15. The device of claim 14, wherein the one or more processors are configured to detect the loss of the packet using a signaled inter-packet timeout value. 16. The device of claim 15, wherein the inter-packet timeout value is signaled according to a nominal Common Media Application Format (CMAF) chunk duration. The one or more processors determine chunk duration from multicast object metadata in a multicast transport packet header, File Delivery Table (FDT) or Enhanced FDT (EFDT), or in control plane service or session metadata. 16. The device of claim 15, further configured to. 16. The device of claim 15, wherein the one or more processors are further configured to determine whether chunk duration signaling considers maximum expected packet arrival jitter. 15. The device of Claim 14, wherein the file delivery header comprises a ROUTE header or a FLUTE header. 15. The device of Claim 14, wherein the information in the file delivery header comprises a TOI and a start_offset value. 15. The device of claim 14, wherein the one or more processors are further configured to submit the byte range request via a unicast protocol. The device is
integrated circuit,
microprocessors, or wireless communication devices,
15. The device of claim 14, comprising at least one of: A computer-readable storage medium storing instructions that, when executed, cause a processor to:
receiving data indicative of a packet loss signaling mechanism, the packet loss signaling mechanism indicating that segments are sent in chunks, that transport object identifiers (TOIs) are contiguous, or that objects assigned to TOIs are contiguous; at least one of a last packet having a particular flag set in the header of said last packet, a base URL, a maximum latency, or a synchronization point in a file delivery header;
detecting loss of packets using the at least one of the packet loss signaling mechanisms, the lost packets comprising lost media data;
generating a byte range request for the lost media data using information in the file delivery header;
delivering the appropriate media object to the media application;
A computer-readable storage medium that causes A device for retrieving media data, comprising:
means for receiving data indicating a packet loss signaling mechanism; at least one of a last packet of an object having a particular flag set in the header of said last packet, a base URL, a maximum latency, or a synchronization point in a file delivery header;
means for detecting loss of packets using said at least one of said packet loss signaling mechanisms, said lost packets comprising lost media data;
means for generating a byte range request for the lost media data using information in the file delivery header;
means for delivering appropriate media objects to media applications;
A device comprising: A method of transmitting media data, comprising:
sending data indicating a packet loss signaling mechanism, the packet loss signaling mechanism indicating that the segments are sent in chunks, that the Transport Object Identifier (TOI) is contiguous, or that the objects assigned to the TOI are contiguous; at least one of a last packet having a particular flag set in the header of said last packet, a base URL, a maximum latency, or a synchronization point in a file delivery header;
sending out one or more packets containing media data via a broadcast or multicast protocol;
receiving a byte range request from a client device via a unicast protocol indicating media data of one of the lost packets;
sending the byte range of the media data to the client device via the unicast protocol in response to the byte range request;
A method. 26. The method of claim 25, wherein sending the data indicating a packet loss signaling method further comprises sending data representing an inter-packet timeout value. 27. The method of claim 26, wherein sending the data representing the inter-packet timeout value comprises signaling the inter-packet timeout value according to a nominal Common Media Application Format (CMAF) chunk duration. Further comprising signaling data representing the chunk duration in multicast object metadata in a multicast transport packet header, File Delivery Table (FDT) or Enhanced FDT (EFDT), or in control plane service or session metadata. 27. The method of claim 26. 26. The method of Claim 25, wherein the file delivery header comprises a ROUTE header or a FLUTE header. 26. The method of claim 25, wherein said information in said file delivery header comprises a TOI and a start_offset value. 26. The method of claim 25, wherein said unicast protocol comprises HTTP. A device for transmitting media data,
a memory configured to store video data;
and one or more processors implemented in a circuit, the one or more processors comprising:
sending data indicating a packet loss signaling mechanism, the packet loss signaling mechanism indicating that the segments are sent in chunks, that the Transport Object Identifier (TOI) is contiguous, or that the objects assigned to the TOI are contiguous; at least one of a last packet having a particular flag set in the header of said last packet, a base URL, a maximum latency, or a synchronization point in a file delivery header;
sending out one or more packets containing media data via a broadcast or multicast protocol;
receiving a byte range request from a client device via a unicast protocol indicating media data of one of the lost packets;
sending the byte range of the media data to the client device via the unicast protocol in response to the byte range request;
A device configured to 33. The device of Claim 32, wherein the one or more processors are further configured to send data representing an inter-packet timeout value. 34. The device of claim 33, wherein the one or more processors are configured to signal the inter-packet timeout value according to a nominal Common Media Application Format (CMAF) chunk duration. The one or more processors specify the chunk duration in multicast object metadata in a multicast transport packet header, File Delivery Table (FDT) or Enhanced FDT (EFDT), or in control plane service or session metadata. 34. The device of claim 33, further configured to signal representative data. 33. The device of Claim 32, wherein the file delivery header comprises a ROUTE header or a FLUTE header. 33. The device of Claim 32, wherein the information in the file delivery header comprises a TOI and a start_offset value. 33. The device of claim 32, wherein said unicast protocol comprises HTTP. The device is
integrated circuit,
microprocessors, or wireless communication devices,
33. The device of claim 32, comprising at least one of: A computer-readable storage medium storing instructions that, when executed, cause a processor to:
sending data indicating a packet loss signaling mechanism, the packet loss signaling mechanism indicating that the segments are sent in chunks, that the Transport Object Identifier (TOI) is contiguous, or that the objects assigned to the TOI are contiguous; at least one of a last packet having a particular flag set in the header of said last packet, a base URL, a maximum latency, or a synchronization point in a file delivery header;
sending out one or more packets containing media data via a broadcast or multicast protocol;
receiving a byte range request from a client device via a unicast protocol indicating media data of one of the lost packets;
sending the byte range of the media data to the client device via the unicast protocol in response to the byte range request;
A computer-readable storage medium that causes A device for transmitting media data,
means for sending data indicating a packet loss signaling mechanism; at least one of a last packet of an object having a particular flag set in the header of said last packet, a base URL, a maximum latency, or a synchronization point in a file delivery header;
means for sending one or more packets containing media data via a broadcast or multicast protocol;
means for receiving a byte range request from a client device via a unicast protocol indicating media data of one of said packets lost;
means for sending the byte range of the media data to the client device via the unicast protocol in response to the byte range request;
A device comprising:

Images