CN106791875B

CN106791875B - Video data decoding method, video data encoding method and related devices

Info

Publication number: CN106791875B
Application number: CN201611085152.1A
Authority: CN
Inventors: 张怡轩; 吴东昇; 秘谧
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2016-11-30
Filing date: 2016-11-30
Publication date: 2020-03-31
Anticipated expiration: 2036-11-30
Also published as: CN106791875A

Abstract

The embodiment of the invention discloses a video data decoding method, a video data encoding method and related equipment, wherein the method comprises the following steps: determining an initial playing frame in the video coding data, searching a data frame which is closest to the initial playing frame and is of a random access point type from the initial playing frame along a direction opposite to the decoding sequence of the video coding data, and determining the searched data frame as a target random access point; when the random access point type of the target random access point is a first random access point type which is preset based on the interframe coding frame, decoding the target random access point according to the intraframe random access point referred by the target random access point; and decoding the data frames one by one from the decoded target random access point along the same direction as the decoding sequence until the start playing frame is decoded. The invention can improve the access efficiency of random access while improving the video coding efficiency.

Description

Video data decoding method, video data encoding method and related devices

Technical Field

The present invention relates to the field of video encoding and decoding technologies, and in particular, to a video data decoding method, an encoding method, and a related device.

Background

In some surveillance videos, there are usually a large number of still or small moving areas for which it is easy to find a matching reference image within a previously or even long previously processed frame for the current video image. Therefore, when the encoder encodes the surveillance videos, the inter-frame coding frames should be used as much as possible to reduce the use of the intra-frame coding frames, so that the code rate can be greatly reduced to improve the video coding efficiency. The inter-frame coding frames obtained by the coding device can comprise P frames and B frames, namely the P frames and the B frames are generated based on an inter-frame compression algorithm, the P frames are forward prediction coding frames specifically, and the B frames are bidirectional prediction interpolation coding frames specifically; the intra-frame encoded by the encoder may include an I-frame, where the I-frame is generated based on an intra-frame compression algorithm, and the I-frame is specifically a completely encoded frame.

On the other hand, random access is an important requirement for encoded video streams. The current random access method is as follows: the method comprises the steps that an encoder sets an I frame in a video code stream as a random access point, the video code stream carrying the random access point is sent to a decoder, if the decoder detects that video playing is started from the position of a target frame (the target frame can be any data frame in the video code stream) in the received video code stream, the decoder can search a random access point (namely the I frame) which is located before the target frame and is closest to the target frame in the video code stream along the direction opposite to the decoding sequence, the data frames are decoded one by one from the closest random access point according to the decoding sequence until the target frame is decoded, and the decoder can send the decoded target frame to a player to play. Therefore, if the number of I frames in the video code stream is small, the number of inter-frame coding frames between the nearest random access point and the target frame may be large, that is, the decoder needs to spend much time to decode the target frame, thereby reducing the access efficiency of the random access.

The prior art provides a method for improving the access efficiency of random access, which improves the access efficiency of random access by increasing the number of I frames in a video code stream, but reduces the video coding efficiency at this time because the number of I frames is large.

Disclosure of Invention

The embodiment of the invention provides a video data decoding method, a video data encoding method and related equipment, which can improve the access efficiency of random access while improving the video encoding efficiency.

A first aspect of the present invention provides a video data decoding method, including:

determining an initial playing frame in the video coding data, searching a data frame which is closest to the initial playing frame and is of a random access point type from the initial playing frame along a direction opposite to the decoding sequence of the video coding data, and determining the searched data frame as a target random access point;

when the random access point type of the target random access point is a first random access point type which is preset based on the interframe coding frame, decoding the target random access point according to the intraframe random access point referred by the target random access point; the intra-frame random access point is a data frame of which the random access point type in the video coding data is a second random access point type based on the intra-frame coding frame;

and decoding the data frames one by one from the decoded target random access point along the same direction as the decoding sequence until the decoding is finished and the initial playing frame is played.

The inter-frame coding frame may be a B frame or a P frame, and the intra-frame coding frame may be an I frame. For example, after searching for a target random access point based on a P frame or a B frame, NALU header information of the data frame may be detected and decoded one by one from the position of the target random access point in a direction opposite to the decoding order of the video encoded data, an I frame-based random access point closest to the target random access point (the I frame-based random access point is a reference frame of the target random access point) may be searched for by a value of nal _ unit _ type in the NALU header information, the searched I frame-based random access point may be decoded, the target random access point may be directly decoded, and subsequent inter-frame encoded frames may be decoded one by one from the target random access point in the decoding order until a start play frame is decoded, and then the image data corresponding to the start play frame may be played. Because the intra-frame coding frame is used as a random access point, and part of inter-frame coding frames can also be used as random access points, the number of the random access points can be increased on the premise of not increasing the intra-frame coding frame, thereby improving the video coding efficiency and simultaneously improving the access efficiency of random access.

In a first possible implementation, determining a start playing frame in video encoding data, and searching for a data frame which is closest to the start playing frame and is of a random access point type from the start playing frame in a direction opposite to a decoding order of the video encoding data, and determining the searched data frame as a target random access point includes:

determining a first position of a starting playing frame in the video coding data;

detecting NALU header information of the data frames one by one in a direction opposite to a decoding order of the video encoded data from the first position; NALU header information includes the frame type of the data frame;

when the frame type of the data frame in the NALU header information is detected to be the random access point type, determining the data frame which is currently detected to be the random access point type as a target random access point;

the random access point type in the NALU header information is: a preset first random access point type based on inter-coded frames or a second random access point type based on intra-coded frames.

By presetting the first random access point type based on the interframe coding frame in the NALU header information, the P frame or the B frame can also be used as a random access point, and further the number of the random access points can be increased to improve the access efficiency of the random access.

With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner, the obtaining a first position of a start playing frame in the video coded data includes:

the method comprises the steps of obtaining a playing operation instruction for video coded data, searching a data frame at an initial playing position in the video coded data according to initial playing time information carried by the playing operation instruction to serve as an initial playing frame, and determining the initial playing position as a first position of the initial playing frame in the video coded data.

With reference to the first possible implementation manner of the first aspect, in a third possible implementation manner, decoding a target random access point according to an intra-frame random access point referred by the target random access point includes:

determining a second position of the target random access point in the video coding data, and acquiring a preset reference number for indicating the number of the random access points in the frame to be referred by the target random access point from NALU header information corresponding to the target random access point;

searching the intra-frame random access points required to be referenced by the target random access point one by one from the second position along the direction opposite to the decoding sequence according to the preset reference quantity; the number of the searched intra-frame random access points required to be referred by the target random access point is a preset reference number;

decoding the intra-frame random access point which is required to be referred by the searched target random access point to obtain a decoded intra-frame random access point;

and decoding the target random access point according to the decoded intra-frame random access point.

The process of searching for the random access points in the frame with the preset reference number may be: detecting and decoding NALU header information of the data frames one by one from the second position in a direction opposite to a decoding order of the video coded data, and searching for a preset reference number of I-frame-based random access points by a value of nal _ unit _ type in the NALU header information. For example, if the preset reference number is 1, searching for an intra-frame random access point closest to the target random access point before the target random access point; if the preset reference number is 2, searching the intra-frame random access point which is closest to the target random access point and is the second closest to the target random access point before the target random access point.

With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, if a decoding rule of the target random access point is the same as a P frame decoding rule, the preset reference number is at least one;

if the decoding rule of the target random access point is the same as the decoding rule of the B frame, the preset reference number is at least two.

If the NALU header information does not indicate the predetermined reference number, the predetermined reference number corresponding to the target random access point based on the P frame may be set to 1 by default, and the predetermined reference number corresponding to the target random access point based on the B frame may be set to 2 by default.

A second aspect of the present invention provides a video data encoding method, including:

encoding the original data to obtain video encoded data; the video coding data comprises a plurality of inter-frame coding frames and a plurality of intra-frame coding frames;

selecting at least one inter-frame coding frame from a plurality of inter-frame coding frames in video coding data, and setting the frame type of a data frame in NALU header information respectively corresponding to each inter-frame coding frame in the selected at least one inter-frame coding frame as a first random access point type based on the inter-frame coding frame;

and setting the frame type of the data frame in the NALU header information corresponding to each intra-coded frame in the plurality of intra-coded frames in the video coded data as a second random access point type based on the intra-coded frame.

By setting the value of nal _ unit _ type in NALU header information during video coding, some P frames or B frames can be set as random access points based on inter-frame coded frames, so that more random access points can be provided when decoding video coded data to improve the access efficiency of random access.

In a first possible implementation manner, selecting at least one inter-coded frame from a plurality of inter-coded frames in video coded data, and setting a frame type of a data frame in NALU header information respectively corresponding to each inter-coded frame in the selected at least one inter-coded frame as a first random access point type based on the inter-coded frame, specifically including:

setting the frame type of a data frame in the NALU header information corresponding to a first inter-coded frame in the video coding data as a first random access point type based on the inter-coded frame, and selecting one inter-coded frame at intervals of a preset number of frames from the first inter-coded frame to set the frame type of the data frame in the NALU header information corresponding to the selected inter-coded frame as the first random access point type based on the inter-coded frame.

With reference to the second aspect, in a second possible implementation manner, selecting at least one inter-coded frame from a plurality of inter-coded frames in video coded data, and setting a frame type of a data frame in NALU header information respectively corresponding to each inter-coded frame in the selected at least one inter-coded frame as a first random access point type based on the inter-coded frame, specifically including:

randomly selecting at least one interframe coding frame from a plurality of interframe coding frames in video coding data according to a preset proportion value, and setting the frame type of a data frame in NALU header information corresponding to each interframe coding frame in the selected at least one interframe coding frame as a first random access point type based on the interframe coding frame; the ratio of the number of the selected inter-coded frames to the total number of the inter-coded frames in the video coded data is equal to a preset proportional value.

With reference to the second aspect, in a third possible implementation manner, the method further includes:

and adding a preset reference number for indicating the number of the random access points in the frame to be referred by the random access points between frames in the NALU header information respectively corresponding to the random access points between frames.

A third aspect of the present invention provides a video data decoding apparatus having a function of realizing the behavior of the video data decoding apparatus in practice in the method of the first aspect described above. The functions may be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions.

In one possible design, the video data decoding apparatus includes a decoder configured to support the video data decoding apparatus to perform the corresponding functions in the method of the first aspect. The video data decoding apparatus may further include a memory for coupling with the decoder, which stores program instructions and data necessary for the video data decoding apparatus.

A fourth aspect of the present invention provides a video data encoding apparatus having a function of realizing the behavior of the video data encoding apparatus in practice in the method of the second aspect described above. The functions may be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions.

In one possible design, the video data encoding apparatus includes an encoder configured to support the video data encoding apparatus to perform the corresponding functions of the method of the second aspect. The video data encoding apparatus may also include a memory for coupling with the encoder that retains program instructions and data necessary for the video data encoding apparatus.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flow chart of a video data processing method according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of another video data processing method according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of video coded data according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of an apparatus for encoding video data according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an apparatus for decoding video data according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a deterministic search module according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a first decoding module according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of another video data encoding apparatus according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of another video data decoding apparatus according to an embodiment of the present invention.

Detailed Description

Embodiments of the present invention will be described below with reference to the accompanying drawings.

Referring to fig. 1, a flow chart of a video data encoding method according to an embodiment of the present invention is shown, where the method specifically describes a video encoding process, and the method may include:

s101, encoding original data to obtain video encoding data; the video encoding data comprises a plurality of inter-coded frames and a plurality of intra-coded frames;

specifically, the video sending end may include an encoder, and the encoder may encode the collected original video data to obtain the video encoded data. For example, the video sending end may be a video monitor, the video monitor collects image data (i.e. original video data) through a camera, and the video monitor may further encode the collected image data through an encoder to generate video encoded data, where the video encoded data may include a plurality of intra-frame encoded frames and a plurality of inter-frame encoded frames, for example, there are 100 data frames in the video encoded data, where 2 data frames are the intra-frame encoded frames, and 98 data frames are the inter-frame encoded frames. The inter-coded frame may be a B-frame or a P-frame, the intra-coded frame may be an I-frame, and the I-frame may be the first frame of each GOP (Group of Pictures).

S102, selecting at least one inter-frame coding frame from the inter-frame coding frames in the video coding data, and setting a frame type of a data frame in NALU (network abstraction Layer unit) header information corresponding to each inter-frame coding frame in the selected at least one inter-frame coding frame as a first random access point type based on the inter-frame coding frame;

specifically, after obtaining the inter-frame coding frames through coding, the video sending end may further select at least one inter-frame coding frame from the multiple inter-frame coding frames, and set a frame type of a data frame in NALU header information respectively corresponding to each inter-frame coding frame in the at least one selected inter-frame coding frame as a first random access point type based on the inter-frame coding frame, where an inter-frame coding frame with the frame type of the first random access point type is referred to as an inter-frame random access point hereinafter. And the reference frame of each interframe random access point is an I frame.

The process of setting the selected at least one inter-frame coding frame as the inter-frame random access point by the encoder may specifically be: setting the frame type of the data frame in the NALU header information corresponding to the first inter-frame coding frame in the video coding data as a first random access point type based on the inter-frame coding frame, and selecting one inter-frame coding frame at intervals of a preset number of frames from the first inter-frame coding frame so as to set the frame type of the data frame in the NALU header information corresponding to the selected inter-frame coding frame as the first random access point type based on the inter-frame coding frame. In step S101, the encoder may encode one inter-frame coding frame with the reference frame as the I frame every the preset number of frames from the first inter-frame coding frame, or may encode more inter-frame coding frames with the reference frame as the I frame on the basis of the inter-frame coding frame, so as to ensure that all the reference frames of the inter-frame random access points set subsequently are I frames. For example, between two I-frames, one of the inter-coded frames is selected as an inter-frame random access point every 20 data frames

Or, the process of setting the selected at least one inter-frame coding frame as the inter-frame random access point by the encoder may specifically be: randomly selecting at least one inter-frame coding frame from the inter-frame coding frames in the video coding data according to a preset proportion value, and setting the frame type of a data frame in NALU header information corresponding to each inter-frame coding frame in the selected at least one inter-frame coding frame as a first random access point type based on the inter-frame coding frame; the ratio of the number of the selected inter-frame coded frames to the total number of the inter-frame coded frames in the video coded data is equal to the preset proportional value. For example, if there are 102 data frames, 2 of which are I frames (i.e., random access points based on I frames) and the preset ratio value is 20% (i.e., it indicates that 20 inter-frame random access points are needed in 100 inter-frame encoded frames), the encoder may randomly select 20 frames from the remaining 100 inter-frame encoded frames to serve as inter-frame random access points (i.e., random access points based on P frames or random access points based on B frames), and the number of inter-frame encoded frames between every two adjacent inter-frame random access points may be different.

Further, the specific process of the encoder setting the frame type of the data frame in the NALU header information as the first random access point type based on the inter-frame coding frame may be: the encoder may set a value of nal _ unit _ type in NALU header information corresponding to the selected inter-coded frame to a value within a reserved value range (e.g., 14-18) or to a value within an unspecified value range (e.g., 24-31), and then determine a frame type corresponding to the set value as the first random access point type based on the inter-coded frame.

For inter-frame coding frames which are not set as inter-frame random access points, the value of nal _ unit _ type in the corresponding NALU header information is still set as a P-frame type or a B-frame type.

Optionally, the video sending end may further add a preset reference number used for indicating the number of intra-frame random access points to be referred by the inter-frame random access points in NALU header information respectively corresponding to the inter-frame random access points. The preset reference number is the number of random access points based on I frames referred by the random access points between frames. For example, if an I frame is used as a reference frame to encode a certain inter-frame coding frame in S101, when the inter-frame coding frame is set as an inter-frame random access point, the corresponding preset reference data may be set to 1 at the same time; if 2I frames are used as reference frames to encode a certain inter-frame encoded frame in S101, the corresponding preset reference data may be set to 2 at the same time when the inter-frame encoded frame is set as an inter-frame random access point. Further, if a certain P frame is set as an inter-frame random access point, the number of preset references corresponding to the P frame-based inter-frame random access point may be at least one; if a certain B frame is set as an inter-frame random access point, the number of preset references corresponding to the inter-frame random access point based on the B frame may be at least two. Since it is considered that if there is image coincidence between the inter-coded frame and the referred I frame, and a more accurate inter-coded frame can be decoded for the subsequent, the inter-coded frame can be encoded by referring to more I frames.

The NALU header information corresponding to the inter-frame random access point may be as shown in table 1 below:

TABLE 1

Here, If (nal _ unit _ type ═ PRAP | | | nal _ unit _ type ═ BRAP) in table 1 is used to indicate whether the frame is a random access point based on a P frame or a random access point based on a B frame (collectively referred to as an inter-frame random access point), the "PRAP" field is a value corresponding to a P-frame-based random access point type, and the "BRAP" field is a value corresponding to a B-frame-based random access point type. The reference _ num is a new syntax element in the NALU header, and the value of the reference _ num may represent the preset reference number. Alternatively, the "PRAP" field in the If (nal _ unit _ type ═ PRAP | | nal _ unit _ type ═ BRAP) statement may be replaced by using PRAP _1 (indicating that the preset reference number is 1) or PRAP _2 (indicating that the preset reference number is 2) or PRAP _3 (indicating that the preset reference number is 3) without setting the refer _ num. Alternatively, when the refer _ num does not need to be set, the preset reference number corresponding to the P-frame-based random access point may be set to 1 by default, and the preset reference number corresponding to the B-frame-based random access point may be set to 2 by default. Alternatively, If (nal _ unit _ type ═ PRAP | | | nal _ unit _ type ═ BRAP) statement may not be set in the NALU header information, that is, whether the corresponding frame is an intra-frame random access point (i.e., the I-frame based random access point) or an inter-frame random access point may be determined directly by detecting a value of nal _ unit _ type.

S103, setting the frame type of the data frame in the NALU header information corresponding to each intra-coded frame in the plurality of intra-coded frames in the video coded data as a second random access point type based on the intra-coded frame;

specifically, after obtaining the intra-coded frames through coding, the encoder may further set the frame type of the data frame in the NALU header information corresponding to each intra-coded frame to the second random access point type based on the intra-coded frame. For example, if the value of nal _ unit _ type in NALU header information of an I frame is set to 5, it indicates that the frame type of the I frame is the second random access point type based on the I frame, i.e., the I frame is an IDR (Instantaneous Decoding Refresh) frame. The data frame of the second random Access point type based on the I frame is the random Access point based on the I frame, the random Access point based on the I frame may include an IDR, BLA (key Link Access), CRA (clean random Access) frame, and the random Access point based on the I frame may also include other random Access points based on the I frame, which have different characteristics from the IDR, BLA, and CRA frames.

Optionally, the encoder may further send the video encoding data carrying the interframe random access point and the intraframe random access point to a video receiving end through a wired network or a wireless network, where the video receiving end may be a computer device in a monitoring room, and the video receiving end includes a decoder.

Further, please refer to fig. 2, which is a flowchart illustrating a video data decoding method according to an embodiment of the present invention, where the method specifically describes a video decoding process, and the method may include:

s201, determining an initial playing frame in video coded data, searching a data frame which is closest to the initial playing frame and is of a random access point type from the initial playing frame along a direction opposite to a decoding sequence of the video coded data, and determining the searched data frame as a target random access point;

specifically, after the step S103 in the embodiment corresponding to fig. 1, the video receiving end may receive the encoded video data sent by the encoder, and at this time, when the user selects to start playing the video from a position of a certain data frame in the encoded video data (e.g., clicks on a position of the certain data frame in the encoded video data), a decoder in the video receiving end may determine the selected data frame as a start playing frame. The video receiving end may further determine the position of the starting playing frame in the video coded data as a first position, and detect NALU header information of data frames one by one from the first position in a direction opposite to a decoding order of the video coded data; the NALU header information includes a frame type of the data frame; the specific process of detecting the NALU header information may be: and decoding the NALU header information, extracting a numerical value of the nal _ unit _ type from the decoded NALU header information, and analyzing a frame type corresponding to the numerical value of the nal _ unit _ type. When the frame type of the data frame in the NALU header information is detected to be a random access point type, determining the currently detected data frame which is the random access point type as a target random access point, namely, searching out the data frame which is closest to the initial playing frame and is the random access point type. Wherein the random access point type in the NALU header information is: a preset first random access point type based on inter-coded frames or a second random access point type based on intra-coded frames.

The specific process of acquiring the first position may be: acquiring a playing operation instruction for video coded data, searching a data frame at an initial playing position in the video coded data according to initial playing time information carried by the playing operation instruction to be used as an initial playing frame, and determining the initial playing position as a first position of the initial playing frame in the video coded data. For example, if the user pulls the playing progress bar of the video coded data to 3 minutes and 50.12 seconds, the data frame corresponding to 3 minutes and 50.12 seconds may be determined as the starting playing frame, and 3 minutes and 50.12 seconds may be recorded as the first position corresponding to the starting playing frame, or the number of data frames between the starting playing frame and the first data frame may be determined as the first position corresponding to the starting playing frame.

S202, when the random access point type of the target random access point is a preset first random access point type based on an interframe coding frame, decoding the target random access point according to an intraframe random access point referred by the target random access point;

specifically, when the random access point type of the target random access point is a preset first random access point type based on an inter-frame coding frame, that is, the target random access point is a random access point based on a P frame or a random access point based on a B frame, at this time, a second position of the target random access point in the video coded data is recorded (for example, the second position may be time information corresponding to the target random access point), and a preset reference number used for indicating the number of intra-frame random access points to which the target random access point needs to refer is obtained from NALU header information corresponding to the target random access point. The decoder further searches the intra-frame random access points required to be referenced by the target random access point one by one from the second position along the direction opposite to the decoding sequence according to the preset reference quantity; the number of the searched intra-frame random access points required to be referred by the target random access point is the preset reference number; the intra-frame random access point is a data frame in which the random access point type in the video coding data is a second random access point type based on an intra-frame coding frame. The process of searching for the intra-frame random access points of the preset reference number may be: and continuously detecting and decoding NALU header information of the data frames one by one from the second position along the direction opposite to the decoding sequence of the video coding data (the process does not need to decode the data frames and the corresponding frame header information), and searching the random access points based on the I frames with the preset reference number through the value of nal _ unit _ type in the NALU header information. For example, if the preset reference number is 1, searching for an intra-frame random access point closest to the target random access point before the target random access point; and if the preset reference number is 2, searching for the intra-frame random access point closest to the target random access point and the intra-frame random access point second closest to the target random access point before the target random access point.

The decoder can further decode the intra-frame random access points of the searched preset reference number, and as the intra-frame random access points are all random access points based on I frames, the intra-frame random access points which are required to be referred by the searched target random access point can be directly and completely decoded, and the decoded intra-frame random access points are cached as first reference frames; and decoding the target random access point at the second position according to the first reference frame. If the target random access point is a random access point based on a P frame, decoding the target random access point according to a decoding rule of the P frame; and if the target random access point is a random access point based on the B frame, decoding the target random access point according to a decoding rule of the B frame. The decoding rule of the P frame refers to a rule for decoding video data in the P frame in the HEVC (High efficiency video Coding, video compression standard), and the decoding rule of the B frame refers to a rule for decoding video data in the B frame in the HEVC standard, that is, image data corresponding to the P frame (or the B frame) can be obtained after the P frame (or the B frame) is decoded by the decoding rule of the P frame (or the B frame). That is, the P-frame-based random access point is different from the P-frame defined in the HEVC standard in that the NALU header information respectively corresponds to different points (specifically, the frame types in the NALU header information are different); the B-frame based random access point differs from the B-frame defined in the HEVC standard in that the NALU header information respectively corresponds to different points (specifically, the frame types in the NALU header information are different).

Wherein, if the decoding rule of the target random access point is the same as the P-frame decoding rule, the preset reference number corresponding to the target random access point may be at least one, and if the preset reference number is not indicated in the NALU header information, the preset reference number may be defaulted to 1; if the decoding rule of the target random access point is the same as the B-frame decoding rule, the number of the preset references corresponding to the target random access point may be at least two, and if the number of the preset references is not indicated in the NALU header information, the number of the preset references may be two by default, that is, 1 in the forward direction and 1 in the backward direction.

S203, starting to decode the data frames one by one from the decoded target random access point along the same direction as the decoding sequence until the initial playing frame is decoded;

specifically, when the decoder completes decoding of the target random access point, the video receiving end may decode data frames one by one from the second position in the same direction as the decoding sequence according to each data frame between the target random access point and the initial play frame and the second reference frame corresponding to the initial play frame, and when the initial play frame is decoded, the decoder may send the decoded initial play frame to the player so that the player plays the initial play frame.

Taking a data frame a between the target random access point and the start playing frame as an example, the second reference frame corresponding to the data frame a may be a data frame located between the target random access point and the start playing frame and before the data frame a, for example, the second reference frame is a previous data frame of the data frame a, that is, after the second reference frame is decoded, the data frame a may be decoded according to the decoded second reference frame. Or, the second reference frame is the first reference frame; or, the second reference frame is the target random access point. By setting the range of the second reference frame, the decoder can be ensured to decode the initial play frame smoothly.

Referring to fig. 3, the GOP in fig. 3 includes I frames, P frames, and B frames, where the I frame at a position a1 and the I frame at a position a5 are both random aps based on I frames, the B frame at a2 is a random ap based on B frames, the P frame at A3 is a random ap based on P frames, and the data frames at other positions are not random aps. When the initial playing frame is a P frame at a4, the decoder can search forward a random access point closest to the P frame at a4 through NALU header information, the searched random access point is a P-frame-based random access point at A3, and record the position of A3; further, the nearest random access point based on the I frame (assuming that the preset reference number is 1) is searched forward through the NALU header information, and the nearest random access point based on the I frame is the random access point located at the position of a1, so that the random access point located at a1 can be decoded, the random access point based on the P frame located at A3 is decoded according to the decoded random access point located at a1 and the position of A3, and then the P frame, the B frame and the P frame located at a4 are decoded one by one from the position of A3, and after the P frame located at a4 is decoded, the decoder can inform the player to start playing the video picture corresponding to the P frame located at a 4. While the random access rule of the prior art needs to decode data frames from the random access point located at a1 to the a4 position one by one, it can be seen that the embodiment of the present invention can omit the decoding of data frames between A3 and a1 (only NALU header information needs to be decoded), and therefore, the random access efficiency can be improved without increasing the number of I frames.

Optionally, when the random access point type of the target random access point is a second random access point type based on an intra-frame coding frame, the target random access point may be directly decoded, and data frames may be decoded one by one from the position of the target random access point according to a decoding sequence, until the start playing frame is decoded, the image data corresponding to the start playing frame may be played.

The embodiment of the invention can decode the target random access point according to the intraframe random access point referred by the target random access point when the random access point type of the target random access point is a preset first random access point type based on interframe coding frames by searching the data frame which is positioned before the initial playing frame and has the closest distance and is the type of the random access point and determining the searched data frame as the target random access point; the intra-frame random access point is a data frame of which the random access point type in the video coding data is a second random access point type based on the intra-frame coding frame; and then, decoding the data frames one by one from the decoded target random access point along the same direction as the decoding sequence until the start playing frame is decoded. Because the intra-frame coding frame is used as a random access point, and part of inter-frame coding frames can also be used as random access points, the number of the random access points can be increased on the premise of not increasing the intra-frame coding frame, thereby improving the video coding efficiency and simultaneously improving the access efficiency of random access.

Referring to fig. 4, which is a schematic structural diagram of a video data encoding apparatus 1 according to an embodiment of the present invention, the video data encoding apparatus 1 may include: the device comprises an encoding module 10, a first setting module 20, a second setting module 30 and a number setting module 40;

the encoding module 10 is configured to encode the original data to obtain video encoded data; the video encoding data comprises a plurality of inter-coded frames and a plurality of intra-coded frames;

the first setting module 20 is configured to select at least one inter-coded frame from the inter-coded frames in the video coded data, and set a frame type of a data frame in NALU header information respectively corresponding to each inter-coded frame in the selected at least one inter-coded frame as a first random access point type based on the inter-coded frame;

specifically, the first setting module 20 may be specifically configured to set a frame type of a data frame in the NALU header information corresponding to a first inter-coded frame in the video coded data to a first random access point type based on the inter-coded frame, and select one inter-coded frame every a preset number of frames from the first inter-coded frame, so as to set a frame type of the data frame in the NALU header information corresponding to the selected inter-coded frame to the first random access point type based on the inter-coded frame.

Alternatively, the first setting module 20 may be specifically configured to randomly select at least one inter-coded frame from the multiple inter-coded frames in the video coded data according to a preset ratio value, and set a frame type of a data frame in NALU header information respectively corresponding to each inter-coded frame in the at least one selected inter-coded frame as a first random access point type based on the inter-coded frame; the ratio of the number of the selected inter-frame coded frames to the total number of the inter-frame coded frames in the video coded data is equal to the preset proportional value.

The second setting module 30 is configured to set a frame type of a data frame in NALU header information corresponding to each of the plurality of intra-coded frames in the video coded data to a second random access point type based on the intra-coded frame.

The number setting module 40 is configured to add a preset reference number, which is used to indicate the number of intra-frame random access points to be referred to by the inter-frame random access points, to NALU header information respectively corresponding to the inter-frame random access points.

The specific functional implementation manners of the encoding module 10, the first setting module 20, the second setting module 30, and the number setting module 40 may refer to S101 to S103 in the embodiment corresponding to fig. 1, which is not described herein again.

Further, please refer to fig. 5, which is a schematic structural diagram of a video data decoding apparatus 2 according to an embodiment of the present invention, where the video data decoding apparatus 2 may include: determining a search module 50, a first decoding module 60, a second decoding module 70;

the determining and searching module 50 is configured to determine an initial playing frame in the video encoded data, search, from the initial playing frame, a data frame that is closest to the initial playing frame and is a random access point type along a direction opposite to a decoding order of the video encoded data, and determine the searched data frame as a target random access point;

the first decoding module 60 is configured to, when the random access point type of the target random access point is a preset first random access point type based on an inter-frame coding frame, decode the target random access point according to an intra-frame random access point referred by the target random access point; the intra-frame random access point is a data frame of which the random access point type in the video coding data is a second random access point type based on an intra-frame coding frame;

the second decoding module 70 is configured to decode data frames one by one from the decoded target random access point along a direction the same as the decoding sequence until the start playing frame is decoded;

specifically, the second decoding module 70 may be specifically configured to decode data frames one by one from the second position along the same direction as the decoding sequence according to each data frame between the target random access point and the initial play frame and a second reference frame corresponding to the initial play frame, and play image data corresponding to the initial play frame when the initial play frame is decoded; the second reference frame is a data frame between the target random access point and the starting playing frame; or, the second reference frame is the first reference frame; or, the second reference frame is the target random access point. Taking a data frame a between the target random access point and the initial play frame as an example, the second reference frame corresponding to the data frame a may be a data frame located between the target random access point and the initial play frame and before the data frame a, for example, the second reference frame is a previous data frame of the data frame a, that is, after the second reference frame is decoded, the data frame a may be decoded according to the decoded second reference frame. Or, the second reference frame is the first reference frame; or, the second reference frame is the target random access point. By setting the range of the second reference frame, the video receiving end can be ensured to decode the initial playing frame smoothly.

Further, please refer to fig. 6, which is a schematic structural diagram of a search determining module 50 according to an embodiment of the present invention, where the search determining module 50 may include: a position determination unit 501, a detection unit 502, a determination unit 503;

the position determining unit 501 is configured to determine a first position of the start playing frame in the video encoded data to which the start playing frame belongs;

specifically, the position determining unit 501 may be specifically configured to obtain a play operation instruction for video encoded data, search a data frame at an initial play position in the video encoded data according to initial play time information carried by the play operation instruction to serve as an initial play frame, and determine the initial play position as a first position of the initial play frame in the video encoded data.

The detecting unit 502 is configured to detect NALU header information of data frames one by one from the first position in a direction opposite to a decoding order of the video encoded data; the NALU header information includes a frame type of the data frame;

the determining unit 503 is configured to determine, when it is detected that the frame type of the data frame in the NALU header information is a random access point type, a currently detected data frame that is the random access point type as a target random access point;

The specific implementation functions of the position determining unit 501, the detecting unit 502, and the determining unit 503 may refer to S201 in the embodiment corresponding to fig. 2, which is not described herein again.

Further, please refer to fig. 7, which is a schematic structural diagram of a first decoding module 60 according to an embodiment of the present invention, where the first decoding module 60 may include: a determination acquisition unit 601, a reference frame search unit 602, a reference decoding unit 603, and a target decoding unit 604;

the determining and acquiring unit 601 is configured to determine a second position of the target random access point in the video coded data, and acquire a preset reference number, which is used to indicate the number of intra-frame random access points to be referred to by the target random access point, from NALU header information corresponding to the target random access point;

the reference frame searching unit 602 is configured to search, according to the preset reference number, intra-frame random access points that are required to be referred by the target random access point one by one from the second position along a direction opposite to the decoding order; the number of the searched intra-frame random access points required to be referred by the target random access point is the preset reference number;

the reference decoding unit 603 is configured to decode the intra-frame random access point that is needed to be referred to by the searched target random access point, so as to obtain a decoded intra-frame random access point;

the target decoding unit 604 is configured to decode the target random access point according to the decoded intra-frame random access point.

For specific implementation functions of the determination obtaining unit 601, the reference frame searching unit 602, the reference decoding unit 603, and the target decoding unit 604, reference may be made to S202 in the corresponding embodiment of fig. 2, which is not described herein again.

Fig. 8 is a schematic structural diagram of another video data encoding apparatus according to an embodiment of the present invention. As shown in fig. 8, the video data encoding apparatus 1000 may include a processor 1001, a communication interface 1002, and a memory 1003 (the number of the processors 1001 in the video data encoding apparatus 1000 may be one or more, and one processor 1001 is taken as an example in fig. 8). In some embodiments of the present invention, the processor 1001, the communication interface 1002 and the memory 1003 may be connected by a communication bus or other methods, wherein fig. 8 is taken as an example of the connection by the communication bus.

Wherein, the communication interface 1002 is used for communicating with a decoder;

the memory 1003 is used for storing programs;

the processor 1001 is configured to execute the program to implement

Encoding the original data to obtain video encoded data; the video encoding data comprises a plurality of inter-coded frames and a plurality of intra-coded frames;

selecting at least one inter-frame coding frame from the inter-frame coding frames in the video coding data, and setting the frame type of a data frame in NALU header information corresponding to each inter-frame coding frame in the selected at least one inter-frame coding frame as a first random access point type based on the inter-frame coding frame;

When the processor 1001 selects at least one inter-coded frame from the inter-coded frames in the encoded video data, and sets a frame type of a data frame in NALU header information corresponding to each inter-coded frame in the selected at least one inter-coded frame as a first random access point type based on the inter-coded frame, the processor is specifically configured to:

setting the frame type of the data frame in the NALU header information corresponding to the first inter-frame coding frame in the video coding data as a first random access point type based on the inter-frame coding frame, and selecting one inter-frame coding frame at intervals of a preset number of frames from the first inter-frame coding frame so as to set the frame type of the data frame in the NALU header information corresponding to the selected inter-frame coding frame as the first random access point type based on the inter-frame coding frame.

randomly selecting at least one inter-frame coding frame from the inter-frame coding frames in the video coding data according to a preset proportion value, and setting the frame type of a data frame in NALU header information corresponding to each inter-frame coding frame in the selected at least one inter-frame coding frame as a first random access point type based on the inter-frame coding frame; the ratio of the number of the selected inter-frame coded frames to the total number of the inter-frame coded frames in the video coded data is equal to the preset proportional value.

Wherein the processor 1001 is further configured to:

and adding a preset reference number used for indicating the number of the intra-frame random access points required to be referred to by the inter-frame random access points into the NALU header information respectively corresponding to the inter-frame random access points.

In the embodiment of the invention, the value of nal _ unit _ type in the NALU header information is set in the video coding process, and some P frames or B frames can be set as random access points based on interframe coding frames, so that more random access points can be provided when video coding data are decoded, and the access efficiency of random access is improved.

Fig. 9 is a schematic structural diagram of another video data decoding apparatus according to an embodiment of the present invention. As shown in fig. 9, the video data decoding apparatus 2000 may include a processor 2001, a communication interface 2002, and a memory 2003 (the number of the processors 2001 in the video data decoding apparatus 2000 may be one or more, and one processor 2001 is exemplified in fig. 9). In some embodiments of the invention, the processor 2001, the communication interface 2002 and the memory 2003 may be connected by a communication bus or other means, wherein fig. 9 is exemplified by being connected by a communication bus.

Wherein, the communication interface 2002 is used for communicating with the encoder;

the memory 2003 is used for storing programs;

the processor 2001 is used to execute the program to realize

Determining an initial playing frame in video coding data, searching a data frame which is closest to the initial playing frame and is of a random access point type from the initial playing frame along a direction opposite to a decoding sequence of the video coding data, and determining the searched data frame as a target random access point;

when the random access point type of the target random access point is a preset first random access point type based on an interframe coding frame, decoding the target random access point according to an intraframe random access point referred by the target random access point; the intra-frame random access point is a data frame of which the random access point type in the video coding data is a second random access point type based on an intra-frame coding frame;

and decoding the data frames one by one from the decoded target random access point along the same direction as the decoding sequence until the starting playing frame is decoded.

When determining a start playing frame in the encoded video data, searching for a data frame that is closest to the start playing frame and is of a random access point type from the start playing frame in a direction opposite to a decoding order of the encoded video data, and determining the searched data frame as a target random access point, the processor 2001 is specifically configured to:

detecting network abstraction layer unit NALU header information of data frames one by one from the first position in a direction opposite to a decoding order of the video encoded data; the NALU header information includes a frame type of the data frame;

when detecting that the frame type of the data frame in the NALU header information is a random access point type, determining the data frame currently detected as the random access point type as a target random access point;

When the processor 2001 executes to acquire the first position of the start playing frame in the video coded data, the processor is specifically configured to:

acquiring a playing operation instruction for video coded data, searching a data frame at an initial playing position in the video coded data according to initial playing time information carried by the playing operation instruction to be used as an initial playing frame, and determining the initial playing position as a first position of the initial playing frame in the video coded data.

When the processor 2001 decodes the target random access point according to the intra-frame random access point referred by the target random access point, specifically, the processor is configured to:

determining a second position of the target random access point in the video coded data, and acquiring a preset reference number used for indicating the number of intra-frame random access points required to be referred by the target random access point from NALU header information corresponding to the target random access point;

searching the intra-frame random access points required to be referenced by the target random access point one by one from the second position along the direction opposite to the decoding sequence according to the preset reference quantity; the number of the searched intra-frame random access points required to be referred by the target random access point is the preset reference number;

decoding the searched intra-frame random access point which needs to be referred by the target random access point to obtain a decoded intra-frame random access point;

If the decoding rule of the target random access point is the same as the decoding rule of the P frame, the preset reference number is at least one;

and if the decoding rule of the target random access point is the same as the decoding rule of the B frame, the preset reference number is at least two.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims

1. A method for decoding video data, comprising:

when the random access point type of the target random access point is a preset first random access point type based on an interframe coding frame, acquiring a preset reference number for indicating the number of intraframe random access points to be referred by the target random access point from network abstraction layer unit NALU header information corresponding to the target random access point;

searching the intra-frame random access points referred by the target random access point one by one from the target random access point along the direction opposite to the decoding sequence according to the preset reference number; the number of the searched intra-frame random access points referred by the target random access point is the preset reference number;

decoding the target random access point according to the intra-frame random access point referred by the target random access point; the intra-frame random access point is a data frame of which the random access point type in the video coding data is a second random access point type based on an intra-frame coding frame;

2. The method of claim 1, wherein the determining a starting play frame in the video encoding data and searching for a data frame closest to the starting play frame and of a random access point type starting from the starting play frame in a direction opposite to a decoding order of the video encoding data and determining the searched data frame as a target random access point comprises:

3. The method of claim 2, wherein said determining a first position of the start frame in the encoded video data comprises:

4. The method of claim 2, wherein the decoding the target random access point according to the intra-frame random access point referenced by the target random access point comprises:

5. The method of claim 4,

6. A method for encoding video data, comprising:

setting a frame type of a data frame in NALU header information corresponding to each of the plurality of intra-coded frames in the video coded data as a second random access point type based on the intra-coded frames;

7. The method according to claim 6, wherein the selecting at least one inter-coded frame from the inter-coded frames in the encoded video data, and setting a frame type of a data frame in NALU header information respectively corresponding to each inter-coded frame in the selected at least one inter-coded frame as a first random access point type based on the inter-coded frame, specifically comprises:

8. The method according to claim 6, wherein the selecting at least one inter-coded frame from the inter-coded frames in the encoded video data, and setting a frame type of a data frame in NALU header information respectively corresponding to each inter-coded frame in the selected at least one inter-coded frame as a first random access point type based on the inter-coded frame, specifically comprises:

9. An apparatus for decoding video data, comprising:

the device comprises a determining and searching module, a decoding module and a searching module, wherein the determining and searching module is used for determining an initial playing frame in video coded data, searching a data frame which is closest to the initial playing frame and is of a random access point type from the initial playing frame along a direction opposite to a decoding sequence of the video coded data, and determining the searched data frame as a target random access point;

a first decoding module, configured to, when the random access point type of the target random access point is a preset first random access point type based on an inter-frame coding frame, decode the target random access point according to an intra-frame random access point referred to by the target random access point; the intra-frame random access point is a data frame of which the random access point type in the video coding data is a second random access point type based on an intra-frame coding frame;

the second decoding module is used for decoding the data frames one by one from the decoded target random access point along the direction which is the same as the decoding sequence until the starting playing frame is decoded;

wherein the first decoding module comprises:

a determining and obtaining unit, configured to determine a second position of the target random access point in the video coded data, and obtain, from NALU header information corresponding to the target random access point, a preset reference number indicating a number of intra-frame random access points to which the target random access point needs to refer;

a reference frame searching unit, configured to search, according to the preset reference number, intra-frame random access points that are required to be referred by the target random access point one by one from the second position along a direction opposite to the decoding order; and the number of the searched intra-frame random access points required to be referred to by the target random access point is the preset reference number.

10. The apparatus of claim 9, wherein the determining a search module comprises:

the position determining unit is used for determining a first position of the starting playing frame in the video coding data;

a detection unit for detecting NALU header information of data frames one by one in a direction opposite to a decoding order of the video encoded data from the first position; the NALU header information includes a frame type of the data frame;

a determining unit, configured to determine, when it is detected that a frame type of a data frame in the NALU header information is a random access point type, a data frame currently detected as the random access point type as a target random access point;

11. The apparatus of claim 10,

the position determining unit is specifically configured to acquire a play operation instruction for video coded data, search a data frame at an initial play position in the video coded data according to initial play time information carried by the play operation instruction to serve as an initial play frame, and determine the initial play position as a first position of the initial play frame in the video coded data.

12. The apparatus of claim 10, wherein the first decoding module further comprises:

a reference decoding unit, configured to decode the intra-frame random access point to be referred to by the searched target random access point to obtain a decoded intra-frame random access point;

and the target decoding unit is used for decoding the target random access point according to the decoded intra-frame random access point.

13. The apparatus of claim 12,

14. An apparatus for encoding video data, comprising:

the encoding module is used for encoding the original data to obtain video encoding data; the video encoding data comprises a plurality of inter-coded frames and a plurality of intra-coded frames;

a first setting module, configured to select at least one inter-coded frame from the inter-coded frames in the video coded data, and set a frame type of a data frame in NALU header information respectively corresponding to each inter-coded frame in the selected at least one inter-coded frame as a first random access point type based on the inter-coded frame;

a second setting module, configured to set a frame type of a data frame in NALU header information corresponding to each of the plurality of intra-coded frames in the video coded data as a second random access point type based on the intra-coded frame;

further comprising:

and the quantity setting module is used for adding preset reference quantity for indicating the quantity of the intra-frame random access points required to be referred to by the inter-frame random access points into the NALU header information respectively corresponding to the inter-frame random access points.

15. The apparatus of claim 14,

the first setting module is specifically configured to set a frame type of a data frame in NALU header information corresponding to a first inter-coded frame in the video coded data to a first random access point type based on the inter-coded frame, and select one inter-coded frame at a preset number of frames from the first inter-coded frame at intervals, so as to set a frame type of a data frame in NALU header information corresponding to the selected inter-coded frame to the first random access point type based on the inter-coded frame.

16. The apparatus of claim 14,

the first setting module is specifically configured to randomly select at least one inter-frame coding frame from the multiple inter-frame coding frames in the video coding data according to a preset ratio value, and set a frame type of a data frame in NALU header information respectively corresponding to each inter-frame coding frame in the selected at least one inter-frame coding frame as a first random access point type based on the inter-frame coding frame; the ratio of the number of the selected inter-frame coded frames to the total number of the inter-frame coded frames in the video coded data is equal to the preset proportional value.

17. An apparatus for encoding video data, comprising a processor and a memory;

wherein the memory is used for storing programs;

wherein the processor is configured to execute the program to implement the method of any one of claims 6 to 8.

18. A video data decoding apparatus comprising a processor and a memory;

wherein the memory is used for storing programs;

wherein the processor is configured to execute the program to implement the method of any one of claims 1 to 5.

19. A computer-readable storage medium, characterized in that it stores a computer program which, when executed by hardware, is capable of implementing the method of any one of claims 1 to 5.

20. A computer-readable storage medium, characterized in that it stores a computer program which, when executed by hardware, is capable of implementing the method of any one of claims 6 to 8.