CN118354166A

CN118354166A - HLS information generation method

Info

Publication number: CN118354166A
Application number: CN202410482910.1A
Authority: CN
Inventors: 郑振贵
Original assignee: Tianyi Cloud Technology Co Ltd
Current assignee: Tianyi Cloud Technology Co Ltd
Priority date: 2024-04-22
Filing date: 2024-04-22
Publication date: 2024-07-16

Abstract

The invention relates to a HLS information generation method, which belongs to the technical field of HLS streaming media, and comprises a processing method of an HLS server and a processing method of an HLS player when m3u8 is abnormal, data is lost and data is abnormal, wherein the processing method comprises the following steps: slicing streaming media audio and video data; acquiring audio and video information related to the slice, and packaging the audio and video information into MEPEG-TS format; encoding the related audio and video information into audio and video metadata and storing the audio and video metadata into MEPEG-TS extension fields; generating m3u8 information of HLS according to the audio and video information related to the plurality of slices; and encoding m3u8 information of the front slice and the rear slice to generate current slice linked list data, and storing the current slice linked list data into a slice MEPEG-TS extension field. The invention improves the fault tolerance of HLS when m3u8 is lost, and rapidly recovers the lost m3u8, thereby improving the fault recovery efficiency and the user experience; the method has strong universality and compatibility, can be widely applied to various scenes, and ensures the normal play of the HLS streaming media.

Description

HLS information generation method

Technical Field

The invention relates to the technical field of HLS streaming media, in particular to a HLS information generation method.

Background

HLS (HTTP Live Streaming) is a technique for streaming media, where the m3u8 file is an index file that lists URL addresses of individual clips of video and related information.

However, if the m3u8 file in HLS is lost, the normal transmission and playing of HLS streaming media will be greatly adversely affected, which mainly includes the following points:

1. Unable to play normally: the m3u8 file is an important component of the HLS streaming media, and the m3u8 file contains URL addresses and related information of each segmented video. If the m3u8 file is not available, the list and the position information of the segmented video cannot be obtained, so that the streaming media cannot be played normally.

2. Interruption of playing: the HLS streaming media needs to continuously acquire the fragmented video from the server for playing in the playing process. If the m3u8 file is lost, the latest fragment video list and the position information cannot be obtained, so that the playing process is interrupted.

3. The live broadcast cannot be performed: HLS is typically used for live streaming media, and the index information in the m3u8 file is updated in real time so that the viewer can watch the latest live content. If the m3u8 file is lost, the latest live content cannot be acquired, and the audience cannot watch the latest live.

Therefore, ensuring normal transmission of m3u8 files in HLS is important for normal playing of HLS streaming media. There is a need to develop a method capable of timely repairing or regenerating m3u8 files if m3u8 file loss in HLS is found, so as to solve the problems of difficulty and pain caused by HLS unable to play normally due to m3u8 file loss in the current stage.

Disclosure of Invention

In view of the above, the present invention aims to provide a HLS information generation method, which can resume HLS streaming media playback through slice files when m3u8 of HLS is abnormal; by encoding and storing the slice audio and video metadata and the m3u8 key information into an extension field in MEPEG-TS format, m3u8 information can be obtained again only by analyzing reserved slices when m3u8 is abnormal, thereby realizing resume of HLS streaming media play and effectively guaranteeing normal play of HLS streaming media.

The invention provides a HLS information generation method, which comprises the following steps: a processing method of an HLS server side and a processing method of an HLS playing side when m3u8 acquires abnormality, data loss and data abnormality;

The HLS server processing method comprises the following steps:

S11, slicing streaming media audio and video data according to fixed time length to obtain a plurality of audio and video slices;

s12, acquiring audio and video information related to the audio and video slice;

And extracting and recording the audio and video information related to each audio and video slice file, wherein the audio and video information mainly comprises video resolution, code rate, frame rate, video key frame index, audio sampling rate, audio and video encoder information and the like, and the audio and video information is used for generating related data in the subsequent steps. According to the embodiment of the invention, the Ffprobe tool is adopted to analyze and acquire the related audio and video information, and FFprobe is often used for analyzing the audio and video container format, the audio and video stream information, the audio and video packet, the audio and video frame and other information. The FFprobe tool can provide great help in transcoding and fault analysis of audio and video.

S13, packaging the audio and video data of the audio and video slice into MEPEG-TS format;

In order to ensure stability and compatibility of audio and video data in the transmission and storage processes, each audio and video slice file is packaged into MEPEG-TS (MPEG Transport Stream) format. The MEPEG-TS format is a packaging format widely applied to digital television, satellite broadcasting and network streaming media transmission, and can provide good transmission efficiency and compatibility. Preferably, the packaging in MEPEG-TS format can be performed using a third party open source tool FFmpeg.

S14, encoding the related audio and video information of the audio and video slice into audio and video metadata (metadata) of the audio and video slice according to a set format, and storing the audio and video metadata into MEPEG-TS extension fields of the audio and video slice;

s15, generating m3u8 information of HLS according to audio and video information related to a plurality of audio and video slices; the generation of the pattern of m3u8 information of HLS includes: generating an m3u8 play list according to a standard document of RFC 8216 about HLS with a window size of the set m3u8 play list;

HLS (HTTP Live Streaming) is an adaptive bitrate video streaming protocol based on HTTP, HLS was introduced by apple in 2009 to describe a set of tools and procedures for transmitting video and audio over the internet. HLS segments a video into small segments and organizes the segments into a playlist in the form of an m3u8 file. The player can dynamically select the most suitable segment to play according to network conditions and equipment performance so as to realize smooth video watching experience.

The M3U8 file is an HTTP-based streaming media file format, and is mainly used to describe URL addresses of multiple pieces of audio and video data, where the M3U8 file is used to divide the audio and video data into multiple pieces, and a user can download and play the pieces as required. Thus, M3U8 is more suitable for scenes that do not require real-time transmission, such as video on demand, short video, etc. The M3U8 file is characterized by flexibility and expandability, can be optimized for different network environments and devices, and supports encryption and playing after encryption. M3U8 files are often used to transmit and play audiovisual content over a network. Unlike traditional post download playback, the M3U8 file may be played directly on a browser or mobile device.

S16, coding m3u8 information of the front and rear audio/video slices to generate current slice linked list data, and storing the slice linked list data into MEPEG-TS expansion fields of the front and rear audio/video slices.

Specifically, the information of one or more pieces of the current slice is used as linked list data, MEPEG-TS extension fields can still be stored by adopting special segments, and meanwhile, the table_id of the special segments of the linked list data needs to be distinguished from the table_id of the special segments of the audio and video metadata.

Further, the processing method of the HLS playing end when m3u8 acquires an exception, data loss and data exception comprises the following steps:

s21, analyzing MEPEG-TS extension fields of the acquired audio and video slices;

s22, extracting slice chain table data from MEPEG-TS extension fields, and determining associated slices of a play list;

Specifically, according to the special section grammar of the slice chain table data, the information of the associated slice is obtained by analysis. And according to the number of m3u8 associated slices to be restored, analyzing the linked list data in the slice at the head or tail of the linked list again, and merging the linked list into the current linked list until the number of slices of the whole associated linked list meets the number of m3u8 to be restored.

S23, acquiring audio and video metadata of the associated slice;

And extracting the special segment data of the audio and video metadata of the audio and video slice, and analyzing to obtain the audio and video information of the audio and video slice according to the special segment grammar of the audio and video metadata.

S24, generating relevant labels of the required m3u8 according to the play list and the audio and video metadata of each relevant slice until the complete m3u8 file is restored.

Further, the method for generating m3u8 information of HLS according to the audio/video information related to the audio/video slices in the step S15 includes the following steps:

s151, generating a basic label of m3u8 according to a designated m3u8 version, wherein the basic label of m3u8 comprises: EXTM3U, EXT-X-VERSION; wherein EXTM U is a fixed m3U8 identifier, and the EXT-X-VERSION tag is used for marking the current VERSION of m3U 8;

S152, generating an m3u8 media slice type tag (MEDIA SEGMENT TAGS) according to audio and video information related to the audio and video slice; the m3u8 media slice class label includes: EXTINF, EXT-X-BYTERANGE, EXT-X-PROGRAM-DATE-TIME; wherein the EXTINF tag identifies a duration for which the URL specifies a media slice; the field can be filled with audio-video slice duration information; the subsequent URL of EXTINF is generated according to the specific slice storage position of the HLS service, the number of slices contained in the play list is set according to the window size, for example, the window size provided by the live HLS service is 5, and then the m3u8 play list contains 5 slice EXTINF labels in the latest time;

The EXT-X-BYTERANGE tag identifies the relevant data interval size of the media slice specified by the URL; the tag may be added when the play m3u8 playlist needs to indicate the video key frame position in the slice, and the tag may be generated using the video key frame index information obtained in step two.

The EXT-X-PROGRAM-DATE-TIME tag indicates the sampling TIME of the first sample segment using an absolute DATE/TIME; the generation may be performed using the slice time stamp recorded in step one.

S153, generating a playlist related label (PLAYLIST TAGS), the playlist related label comprising: EXT-X-STREAM-INF, EXT-X-TARGETDURATION, EXT-X-MEDIA-SEQUENCE;

Wherein, EXT-X-STREAM-INF label represents the attribute information of the media STREAM, the content includes: BANDWIDTH, RESOLUTION, codec; wherein, the BANDWIDTH is the audio/video code rate in the audio/video information related to the audio/video slice; RESOLUTION is a RESOLUTION wide high pixel value in the audio and video information related to the audio and video slice; the CODECS is encoder information in the audio and video information related to the audio and video slice;

The EXT-X-TARGETDURATION tag represents the maximum length of a slice in the playlist (in seconds) and is generated by selecting the playlist to contain the maximum slice length value in all slice information.

The EXT-X-MEDIA-SEQUENCE tag indicates the SEQUENCE number of the first URL fragment file of the playlist; each media segment URL has a unique integer sequence number.

Further, the method for analyzing MEPEG-TS extension fields of the acquired audio/video slice in the step S21 includes the following steps:

s211, analyzing TS data packets from MEPEG-TS extension fields of audio and video slices, wherein the length of each TS data packet is 188 bytes, and each TS data packet comprises a packet header and a payload;

S212, acquiring a PAT table (Program Association Table) and a PMT table (Program Map Table) from the TS data packet, wherein the PAT table contains PID values of the special segment;

S213, identifying TS data packets with PID values of the special segments, extracting and merging payloads of one or more TS data packets, and obtaining MEPEG-TS extension field data.

Further, the method for generating the relevant label of the required m3u8 in the step S24 includes the following steps:

s241, recovering the basic label of m3u8 according to the appointed m3u8 version; the m3u8 base tag includes: EXTM3U, EXT-X-VERSION; wherein EXTM U is a fixed m3U8 identifier, and the EXT-X-VERSION tag is used for marking the current VERSION of m3U 8;

S242, recovering m3u8 media slice type labels (MEDIA SEGMENT TAGS) according to the audio and video metadata of the audio and video slices; the m3u8 media slice class label includes: EXTINF, EXT-X-BYTERANGE, EXT-X-PROGRAM-DATE-TIME;

The EXTINF tag identifies the duration of the media slice specified by the URL thereafter; this field may be restored by a "duration" in the slice audio video metadata. The subsequent URL of extnf is generated according to the HLS service specific slice storage location, and the tag can be restored by "segment_info_length" and "segment_info_byte" in the slice chain table data.

The EXT-X-BYTERANGE tag identifies the relevant data interval size of the media slice specified by the URL thereafter, can be added when the m3u8 playlist needs to indicate the video key frame position in the slice, and can be restored by "keyframe _count", "keyframe _position" and "keyframe _size" in the audio-video metadata of the audio-video slice.

The EXT-X-PROGRAM-DATE-TIME tag indicates the sampling TIME of the first sample segment using an absolute DATE/TIME; the tag may be restored by "date_time" in the audio-video metadata of the audio-video slice.

S243, recovering a playlist related label (PLAYLIST TAGS) comprising:

The EXT-X-STREAM-INF tag indicates attribute information of a media STREAM, BANDWIDTH content of which can be restored by "video_data_rate" of the slice audio/video metadata "," audio_sample_rate ", RESOLUTION content of which can be restored by" heigh "," width "of the slice audio/video metadata, and CODES (encoder) content of which can be restored by" codes_info_length "of the slice audio/video metadata.

The EXT-X-TARGETDURATION tag indicates the maximum duration (in seconds) of a slice in the playlist, and is restored by selecting the playlist to contain the maximum "duration" value in all slice audio and video metadata.

The EXT-X-MEDIA-SEQUENCE tag indicates the SEQUENCE number of the playlist's first URL fragment file. The tag is restored by selecting the audio and video metadata "sequence" of the first slice of the playlist.

Further, the packet header in the step S211 includes sync bytes, PID (Packet Identifier), TSC (Transport Scrambling Control) information, and the payload includes audio/video data and PSI/SI table data.

Further, the MPEG-TS extension field of step S14 is defined by the MPEG-TS standard (MPEG-TS is a standardized audio and video transmission and storage format), and the fields utilized by the MPEG-TS extension include: a private segment, descriptor; the special segment is used for transmitting user-defined special data, and the descriptor is used for describing the characteristics and parameters of the audio and video stream and other data.

MPEG-TS (MPEG Transport Stream) is a standard data container format for transmitting and storing audio-video, program and system information protocol data. It is widely used in digital broadcasting systems such as DVB (Digital Video Broadcasting), ATSC (Advanced Television Systems Committee), IPTV (Internet Protocol Television), and the like. MPEG-TS is defined in MPEG-2 part 1 systems, also known as ISO/IEC standard 13818-1 or ITU-T recommendation.

Private section (private section) in MPEG-TS is a mechanism for transmitting user-defined private data. In the standard, a private section is defined as a special section for transmitting user-defined data. These private sections may be used to carry additional information defined by a particular application or vendor, such as particular metadata, additional descriptive information, etc. The use of dedicated segments allows users to add custom data in the MPEG-TS to meet specific application requirements. These custom data may be transmitted via dedicated segments and parsed and processed at the receiving end. By using dedicated segments, a user can flexibly extend the functionality of the MPEG-TS to accommodate different application scenarios.

Descriptors (descriptors) in MPEG-TS are used to describe the characteristics and parameters of audio-video streams and other data. In some cases custom descriptors may be added to extend the functionality of the MPEG-TS, e.g. to add specific coding parameters, identifiers, etc.

The use of custom extension fields should follow the specifications of the MPEG-TS standard and there may be compatibility issues between different applications and devices. Thus, when custom extension fields are used, it is necessary to ensure that the relevant devices and applications can properly parse and process these fields.

Preferably, the embodiment of the invention uses a special section in the MPEG-TS to save the audio and video metadata of the slice, wherein the special section consists of an 8-bit table_id field, a 1-bit section_syntax_indicator field, a 12-bit section_length field and some other fields. Wherein the section_length field indicates the total length of the dedicated section. A private data indicator (private_data_indicator) in the private section is used to identify whether the section contains private data. A value of 1 for the private data indicator indicates that the segment contains private data, the specific format and content of which is user-defined.

Further, the method for slicing the streaming media audio and video data in the step S11 according to the fixed duration includes:

The method comprises the steps of performing time cutting on streaming media audio and video data, cutting the streaming media audio and video data into a plurality of fragments according to a preset duration comprising any one of 10 seconds and 20 seconds, independently storing each fragment as audio and video slice data for subsequent processing, recording information such as a slice serial number, a slice starting time stamp, a slice duration and the like while slicing, and generating and using relevant fields in subsequent steps. To be compatible with different streaming audio and video formats, the slicing process may use third party open source tools, such as: libav, gstreamer, openCV, etc.

The present invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the HLS information generation method as described above.

The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed implements the steps of the HLS information generation method as described above.

Compared with the prior art, the invention has the beneficial effects that:

The HLS information generation method provided by the invention improves the fault tolerance of HLS streaming media when m3u8 is lost, reduces the threshold of a user for watching video and improves the user experience; by analyzing the slice file, the lost m3u8 is quickly recovered, and the failure recovery efficiency is improved; related information of the HLS streaming media is stored and transmitted through MEPEG-TS extension fields, so that the HLS streaming media has strong universality and compatibility, can be widely applied to various scenes, and effectively ensures normal play of the HLS streaming media.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention.

In the drawings:

fig. 1 is a schematic view of slicing and packaging streaming media data according to an embodiment of the present invention;

FIG. 2 is a schematic view of slice expansion data according to an embodiment of the present invention;

FIG. 3 is a diagram showing a syntax of an MPEG-TS specific segment according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a resume m3u8 playlist related slice in accordance with an embodiment of the present invention;

Fig. 5 is a flowchart of a processing method of an HLS server according to an embodiment of the present invention;

FIG. 6 is a flowchart of a processing method of an HLS playing end when m3u8 acquires an exception, data is lost, and data is abnormal according to an embodiment of the present invention;

FIG. 7 is a flowchart of a method for generating m3u8 information of HLS according to audio/video information related to a plurality of audio/video slices according to an embodiment of the present invention;

FIG. 8 is a flowchart of a method for parsing MEPEG-TS extension fields of an acquired audio-video slice according to an embodiment of the present invention;

FIG. 9 is a flow chart of a method of generating a desired m3u8 correlation tag according to an embodiment of the present invention;

Fig. 10 is a schematic diagram of a computer device according to an embodiment of the invention.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of systems and products consistent with some aspects of the present disclosure as detailed in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination" depending on the context.

Embodiments of the present invention are described in further detail below.

The embodiment of the invention provides an HLS information generation method, which aims to solve the problems of difficulty and pain caused by the fact that HLS cannot be normally played because the normal transmission and playing of HLS streaming media are adversely affected if m3u8 files in HLS are lost in the current stage.

Method embodiment

The embodiment of the invention provides a HLS information generation method, which comprises the following steps: a processing method of an HLS server side and a processing method of an HLS playing side when m3u8 acquires abnormality, data loss and data abnormality;

the processing method of the HLS server side includes the following steps (see fig. 5):

the method for slicing streaming media audio and video data according to fixed time length comprises the following steps:

in order to ensure stability and compatibility of audio and video data in the transmission and storage processes, each audio and video slice file is packaged into MEPEG-TS (MPEG Transport Stream) format as shown in fig. 1. The MEPEG-TS format is a packaging format widely applied to digital television, satellite broadcasting and network streaming media transmission, and can provide good transmission efficiency and compatibility. Preferably, the packaging in MEPEG-TS format (see FIG. 1) can be performed using a third party open source tool FFmpeg.

The MPEG-TS extension field is defined by the MPEG-TS standard (MPEG-TS is a standardized audio and video transmission and storage format), and the fields utilized by the MPEG-TS extension include: a private segment, descriptor; the special segment is used for transmitting user-defined special data, and the descriptor is used for describing the characteristics and parameters of the audio and video stream and other data.

Private section (private section) private section in MPEG-TS is a mechanism for transmitting user-defined private data. In the standard, a private section is defined as a special section for transmitting user-defined data. These private sections may be used to carry additional information defined by a particular application or vendor, such as particular metadata, additional descriptive information, etc. The use of dedicated segments allows users to add custom data in the MPEG-TS to meet specific application requirements. These custom data may be transmitted via dedicated segments and parsed and processed at the receiving end. By using dedicated segments, the user can flexibly extend the functionality of the MPEG-TS to accommodate different application scenarios (see fig. 3).

Descriptor (Descriptor) the Descriptor in MPEG-TS is used to describe the characteristics and parameters of the audio-video stream and other data. In some cases custom descriptors may be added to extend the functionality of the MPEG-TS, e.g. to add specific coding parameters, identifiers, etc.

Preferably, the embodiment of the invention uses a special section in the MPEG-TS to save the audio and video metadata of the slice, wherein the special section consists of an 8-bit table_id field, a 1-bit section_syntax_indicator field, a 12-bit section_length field and some other fields. Wherein the section_length field indicates the total length of the dedicated section. A private data indicator (private_data_indicator) in the private section is used to identify whether the section contains private data. A value of 1 for the private data indicator indicates that the segment contains private data, the specific format and content of which is user-defined. The format of the special segment of the slice audio and video metadata in the MPEG-TS according to the format standard requirements of the special segment is shown in table 1:

TABLE 1

The definition of the fields in table 1 is as follows:

sequence: this field identifies the serial number of the slice, and using the slice serial number recorded in step 11, the serial number value is continuously incremented from 0 and can be restarted when the maximum value is exceeded.

Date_time: this field identifies the generation time of the slice, and using the slice timestamp recorded in step one, a 10-bit Unix timestamp format or a string format may be used.

Duration: this field identifies the duration of the slice, using the slice duration information recorded in step one, in seconds.

Frame_rate: this field identifies the video frame rate of the slice, and uses the slice video information acquired in step two to indicate how many frames of video are in each second.

Video_data_rate: this field identifies the code rate of the slice video, and using the slice video information obtained in step two, the unit is kb/s, indicating how much kb the size of the video data is per second.

Audio_sample_rate: the field identifies the code rate of the sliced audio, and uses the sliced audio information obtained in step two, in kb/s, to indicate how much kb the video data is in size per second.

Heigh: this field identifies the resolution height of the slice video, and using the slice video information obtained in step two, the unit is pixels, indicating how many pixels the video resolution height is.

Width: this field identifies the resolution width of the slice video, and using the slice video information obtained in step two, the unit is pixels, indicating how many pixels the video resolution width is.

The content length of the media audio and video encoder information is identified by the field, and the slice audio and video information acquired in the second step is used, wherein the unit is bytes.

Codes_info_byte: the field identifies the specific content of the media audio/video encoder information, and the content can be represented by a character string with the length of codes_info_length by using the slice audio/video information acquired in the second step.

Keyframe _count: this field identifies the number of video key frames within the slice, using the slice video information obtained in step two.

Keyframe _position: this field identifies the offset position within the slice of each video key frame within the slice, using the slice video information obtained in step two, in bytes, the number of fields being dependent on keyframe _count.

Keyframe _size: this field identifies the size of each video key frame within the slice, using the slice video information obtained in step two, in bytes, the number of which depends on keyframe _count.

The method for generating m3u8 information of HLS according to audio/video information related to a plurality of audio/video slices, as shown in fig. 7, includes the following steps:

wherein, EXT-X-STREAM-INF label represents the attribute information of the media STREAM, the content includes: BANDWIDTH, RESOLUTION, CODECS; wherein, the BANDWIDTH is the audio/video code rate in the audio/video information related to the audio/video slice; RESOLUTION is a RESOLUTION wide high pixel value in the audio and video information related to the audio and video slice; the CODECS is encoder information in the audio and video information related to the audio and video slice;

S16, m3u8 information of the front and rear audio/video slices is encoded to generate current slice linked list data, and the slice linked list data is stored in MEPEG-TS expansion fields of the front and rear audio/video slices (see FIG. 2).

Specifically, the information of one or more pieces of the current slice is used as linked list data, MEPEG-TS extension fields can still be stored by adopting special segments, and meanwhile, the table_id of the special segments of the linked list data needs to be distinguished from the table_id of the special segments of the audio and video metadata. The format of the dedicated segment in the MPEG-TS of the slice link table data of this embodiment is as shown in table 2 according to the format standard requirements of the dedicated segment:

TABLE 2

The definition of the fields in table 2 is as follows:

The pre_segment_count field identifies the number of forward slices in the linked list, i.e., the number of slices in the linked list data that precede the current slice, e.g., the current slice number is 10 and the pre_segment_count is 2, which represents that the current slice linked list contains the slice information with the numbers 8 and 9. The forward slice information in the linked list is ordered from small to large according to the slice sequence number.

The next_segment_count field identifies the number of backward slices in the linked list, i.e. the number of slices located after the current slice in the linked list data, for example, the current slice number is 10, and the next_segment_count is 2, which represents that the current slice linked list contains the slice information with the numbers of 11 and 12. The ordering of the backward slice information in the linked list is ordered from small to large according to the slice sequence number.

The segment_info_length field identifies the length of the slice information content thereafter in bytes.

The segment_info_byte field identifies the specific content of the slice information, and the slice information should ensure that the slice can be located and acquired, for example, the URL download address of the slice, the file name of the slice, etc. can be directly stored, and the length is segment_info_length.

The processing method of the HLS playing end when m3u8 acquires an exception, data loss and data exception is shown in fig. 6, and includes the following steps:

The method for parsing MEPEG-TS extension fields of an acquired audio/video slice, as shown in FIG. 8, comprises the following steps:

the packet header contains synchronous bytes, PID (Packet Identifier) and TSC (Transport Scrambling Control) information, and the payload contains audio and video data and PSI/SI table data.

Specifically, according to the special section grammar of the slice chain table data, the information of the associated slice is obtained by analysis. And according to the number of m3u8 associated slices to be restored, analyzing the linked list data in the slice at the head or tail of the linked list again, and merging the linked list into the current linked list until the number of slices of the whole associated linked list meets the number of m3u8 to be restored (see fig. 4).

S23, acquiring audio and video metadata of the associated slice;

The method for generating the relevant label of the required m3u8, as shown in fig. 9, comprises the following steps:

S243, recovering a playlist related label (PLAYLIST TAGS) comprising:

EXT-X-STREAM-INF represents attribute information of a media STREAM, BANDWIDTH content of which can be restored by "video_data_rate" of slice audio and video metadata "," audio_sample_rate ", RESOLUTION content of which can be restored by" heigh "," width "of slice audio and video metadata, and CODES content of which can be restored by" codes_info_length "," codes_info_byte "of slice audio and video metadata.

EXT-X-TARGETDURATION represents the maximum length of a slice in the playlist (in seconds), and the tag is restored by selecting the playlist to contain the maximum "duration" value in all slice audio and video metadata.

EXT-X-MEDIA-SEQUENCE indicates the SEQUENCE number of the playlist's first URL fragment file. The tag is restored by selecting the audio and video metadata "sequence" of the first slice of the playlist.

The embodiment of the invention also provides a computer device, and fig. 10 is a schematic structural diagram of the computer device provided by the embodiment of the invention; referring to fig. 10 of the drawings, the computer apparatus includes: an input system 23, an output system 24, a memory 22, and a processor 21; the memory 22 is configured to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 are caused to implement the HLS information generation method as provided by the above-described embodiments; wherein the input system 23, output system 24, memory 22, and processor 21 may be connected by a bus or otherwise, as exemplified by a bus connection in fig. 10.

The memory 22 is used as a readable storage medium of a computing device, and can be used for storing a software program and a computer executable program, such as program instructions corresponding to the HLS information generation method according to the embodiment of the present invention; the memory 22 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the device, etc.; in addition, memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device; in some examples, memory 22 may further comprise memory located remotely from processor 21, which may be connected to the device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input system 23 is operable to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the device; output system 24 may include a display device such as a display screen.

The processor 21 executes various functional applications of the apparatus and data processing, i.e., implements the HLS information generation method described above, by running software programs, instructions, and modules stored in the memory 22.

The computer device provided by the above embodiment can be used for executing the HLS information generation method provided by the above embodiment, and has corresponding functions and beneficial effects.

Embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are for performing the HLS information generation method as provided by the above embodiments, the storage medium being any of various types of memory devices or storage devices, the storage medium comprising: mounting media such as CD-ROM, floppy disk or tape systems; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, lanbas (Rambus) RAM, etc.; nonvolatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memory or combinations thereof; in addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a second, different computer system, the second computer system being connected to the first computer system through a network (such as the internet); the second computer system may provide program instructions to the first computer for execution. Storage media includes two or more storage media that may reside in different locations (e.g., in different computer systems connected by a network). The storage medium may store program instructions (e.g., embodied as a computer program) executable by one or more processors.

Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the HLS information generation method described in the above embodiments, and may also perform the related operations in the HLS information generation method provided in any embodiment of the present invention.

While the applicant has described and illustrated the embodiments of the present invention in detail with reference to the drawings, it should be understood by those skilled in the art that the above embodiments are only preferred embodiments of the present invention, and the detailed description is only for the purpose of helping the reader to better understand the spirit of the present invention, and not to limit the scope of the present invention, but any improvements or modifications based on the spirit of the present invention should fall within the scope of the present invention.

Finally, it should be noted that the above embodiments are merely for illustrating the technical solution of the embodiments of the present invention, and are not limiting. Although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the invention, and any changes and substitutions that would be apparent to one skilled in the art are intended to be included within the scope of the present invention.

Claims

1. A method for generating HLS information, comprising: a processing method of an HLS server side and a processing method of an HLS playing side when m3u8 acquires abnormality, data loss and data abnormality;

The HLS server processing method comprises the following steps:

s14, encoding the related audio and video information of the audio and video slice into audio and video metadata of the audio and video slice according to a set format, and storing the audio and video metadata into MEPEG-TS extension fields of the audio and video slice;

2. The HLS information generation method according to claim 1, wherein the processing method of the HLS playback end when m3u8 acquires an exception, data loss, data exception comprises the steps of:

s23, acquiring audio and video metadata of the associated slice;

3. The HLS information generation method according to claim 1, wherein the method of generating m3u8 information of HLS from audio-video information related to a plurality of audio-video slices in step S15 includes the steps of:

s151, generating a basic label of m3u8 according to the appointed m3u8 version; the m3u8 base tag includes: EXTM3U, EXT-X-VERSION; wherein EXTM U is a fixed m3U8 identifier, and the EXT-X-VERSION tag is used for marking the current VERSION of m3U 8;

S152, generating an m3u8 media slice type tag according to audio and video information related to the audio and video slice; the m3u8 media slice class label includes: EXTINF, EXT-X-BYTERANGE, EXT-X-PROGRAM-DATE-TIME; wherein the EXTINF tag identifies a duration for which the URL specifies a media slice; the EXT-X-BYTERANGE tag identifies the relevant data interval size of the media slice specified by the URL; the EXT-X-PROGRAM-DATE-TIME tag indicates the sampling TIME of the first sample segment using an absolute DATE/TIME;

S153, generating a playlist related label; the playlist related label includes:

EXT-X-STREAM-INF、EXT-X-TARGETDURATION、

EXT-X-MEDIA-SEQUENCE；

the EXT-X-TARGETDURATION tag represents the maximum duration of a slice in the playlist, and the tag value is generated by selecting the maximum slice duration value in the playlist containing all slice information;

4. The HLS information generation method according to claim 2, wherein the method of parsing MEPEG-TS extension field of the acquired av slice in S21 step comprises the steps of:

s212, acquiring a PAT table and a PMT table from the TS data packet, wherein the PAT table comprises PID values of the special segments;

5. HLS information generation method according to claim 2, wherein the method of generating the relevant tag of the required m3u8 of step S24 comprises the steps of:

s242, recovering m3u8 media slice type labels according to the audio and video metadata of the audio and video slices; the m3u8 media slice class label includes: EXTINF, EXT-X-BYTERANGE, EXT-X-PROGRAM-DATE-TIME;

the EXTINF tag identifies the duration of the media slice specified by the URL thereafter;

the EXT-X-BYTERANGE tag identifies the relevant data interval size of the media slice specified by the URL;

the EXT-X-PROGRAM-DATE-TIME tag indicates the sampling TIME of the first sample segment using an absolute DATE/TIME;

S243, recovering a play list related label, wherein the play list related label comprises:

EXT-X-STREAM-INF represents attribute information of a media STREAM;

EXT-X-TARGETDURATION represents the maximum length of a slice in the playlist;

EXT-X-MEDIA-SEQUENCE indicates the SEQUENCE number of the playlist's first URL fragment file.

6. The HLS information generation method according to claim 4, wherein the header in the step S211 includes sync byte, PID, TSC information, and the payload includes audio/video data and PSI/SI table data.

7. The HLS information generation method of claim 1, wherein the MPEG-TS extension field of the S14 step is specified by the MPEG-TS standard, and the fields utilized by the MPEG-TS extension include: a private segment, descriptor; the special segment is used for transmitting user-defined special data, and the descriptor is used for describing the characteristics and parameters of the audio and video stream and other data.

8. The HLS information generation method of claim 6, wherein the method of slicing streaming audio and video data in step S11 according to a fixed time length comprises:

The method comprises the steps of performing time cutting on streaming media audio and video data, cutting the streaming media audio and video data into a plurality of fragments according to a preset duration comprising any one of 10 seconds and 20 seconds, independently storing each fragment as audio and video slice data for subsequent processing, recording information such as a slice serial number, a slice starting time stamp, a slice duration and the like while slicing, and generating and using relevant fields in subsequent steps.

9. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the HLS information generation method of any one of claims 1-8.

10. Computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the HLS information generation method as claimed in any one of claims 1-8 when the program is executed by the processor.