CN117749317A - A fountain code encoding and decoding method and device - Google Patents
A fountain code encoding and decoding method and device Download PDFInfo
- Publication number
- CN117749317A CN117749317A CN202211121539.3A CN202211121539A CN117749317A CN 117749317 A CN117749317 A CN 117749317A CN 202211121539 A CN202211121539 A CN 202211121539A CN 117749317 A CN117749317 A CN 117749317A
- Authority
- CN
- China
- Prior art keywords
- packet loss
- data packet
- loss rate
- frame
- time period
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
技术领域Technical field
本申请实施例涉及编解码技术领域,尤其涉及一种喷泉码编码、解码方法及装置。The embodiments of the present application relate to the field of encoding and decoding technology, and in particular to a fountain code encoding and decoding method and device.
背景技术Background technique
现有技术,有很多信道编码方案,比如差错控制编码(前向纠错编码等),该方式通常需要反馈重传、固定编码码率,因此该方式难以保证信息的可靠传输。因此,寻找适用不同信道状态的信道编码技术是十分迫切的。其中,喷泉码作为一种信道编码,具有编译复杂度低、开销小、信道容量高的优势。然而,目前对于喷泉码的研究,多集中于如何设计度分布,以达到优化编译码效率的目的。但现有技术没有考虑喷泉码与实际业务场景如何结合以发挥其优势。In the existing technology, there are many channel coding schemes, such as error control coding (forward error correction coding, etc.). This method usually requires feedback retransmission and fixed coding rate, so it is difficult to ensure reliable transmission of information. Therefore, it is very urgent to find channel coding technology suitable for different channel states. Among them, fountain code, as a channel code, has the advantages of low coding complexity, low overhead, and high channel capacity. However, current research on fountain codes mostly focuses on how to design degree distribution to achieve the purpose of optimizing coding and decoding efficiency. However, the existing technology does not consider how fountain codes can be combined with actual business scenarios to take advantage of them.
发明内容Contents of the invention
本申请提供一种喷泉码编码、解码方法及装置,以适配实际业务场景的需求。This application provides a fountain code encoding and decoding method and device to adapt to the needs of actual business scenarios.
第一方面,本申请提供一种喷泉码编码方法,该方法可通过喷泉码编码装置来执行,其中喷泉码编码装置可以为图像编码器、中央处理器(central processing unit,CPU)、图形处理器(graphics processing unit,GPU)等,本申请在此不具体限定。In a first aspect, the present application provides a fountain code encoding method, which can be executed by a fountain code encoding device, where the fountain code encoding device can be an image encoder, a central processing unit (CPU), or a graphics processor. (graphics processing unit, GPU), etc., this application is not specifically limited here.
喷泉码编码装置获取数据丢包率;根据数据丢包率调整图像组中编码帧的权重值,根据编码帧的权重值进行喷泉编码,得到编码数据包;将编码数据包传输至解码装置。The fountain code encoding device obtains the data packet loss rate; adjusts the weight value of the encoded frame in the image group according to the data packet loss rate, performs fountain encoding according to the weight value of the encoded frame, and obtains the encoded data packet; transmits the encoded data packet to the decoding device.
本申请中,喷泉码编码装置根据数据丢包率调整编码帧的权重值,之后基于度分布进行喷泉编码,可以适配流媒体的场景需求,保证流媒体数据的编码效率。In this application, the fountain code encoding device adjusts the weight value of the encoded frame according to the data packet loss rate, and then performs fountain encoding based on the degree distribution, which can adapt to the scene requirements of streaming media and ensure the coding efficiency of streaming media data.
在一种可选的方式中,数据丢包率通过以下参数中一种指示:In an optional manner, the data packet loss rate is indicated by one of the following parameters:
解码装置在预设时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;The ratio of the encoded data packets that the decoding device failed to parse within the preset time period to all the encoded data packets received;
第一比值与第二比值的差值,第一比值为解码装置在第一时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;第二比值为解码装置在第二时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;其中,第一时间段与第二时间段为相邻时间段。The difference between the first ratio and the second ratio. The first ratio is the ratio of the coded data packets that the decoding device failed to parse in the first time period to all the coded data packets received; the second ratio is the ratio of the coded data packets that the decoding device failed to parse in the second time period. The ratio of the coded data packets that were not successfully parsed within the segment to all the coded data packets received; where the first time period and the second time period are adjacent time periods.
通过该方式更易获取数据丢包率。This method makes it easier to obtain the data packet loss rate.
在一种可选的方式中,根据数据丢包率调整图像组中编码帧的权重值,包括:在预设时间段内,根据数据丢包率调整图像组中编码帧的权重值。In an optional method, adjusting the weight value of the encoded frame in the image group according to the data packet loss rate includes: adjusting the weight value of the encoded frame in the image group according to the data packet loss rate within a preset time period.
通过该方式可以提高关键帧的传输成功率,提高编码效率,并保证编码质量。This method can improve the transmission success rate of key frames, improve coding efficiency, and ensure coding quality.
在一种可选的方式中,编码帧包括以下中的至少一种:I帧、P帧以及B帧;I帧、P帧以及B帧的优先级依序降低;根据数据丢包率调整图像组中编码帧的权重值,包括:In an optional manner, the encoded frame includes at least one of the following: I frame, P frame and B frame; the priority of I frame, P frame and B frame decreases in order; the image is adjusted according to the data packet loss rate The weight value of the coded frames in the group, including:
在预设时间段内,数据丢包率大于丢包阈值时,按照优先级顺序依序调整图像组中编码帧的权重值,其中,编码帧的权重值之和为预设值,优先级高的编码帧的权重值大于优先级低的编码帧的权重值。Within the preset time period, when the data packet loss rate is greater than the packet loss threshold, the weight values of the encoded frames in the image group are adjusted in order of priority. Among them, the sum of the weight values of the encoded frames is the preset value, and the higher priority The weight value of the coded frame is greater than the weight value of the coded frame with lower priority.
通过该方式可以提高关键帧的传输成功率,提高编码效率,并保证编码质量。This method can improve the transmission success rate of key frames, improve coding efficiency, and ensure coding quality.
在一种可选的方式中,据丢包率大于丢包阈值时,按照优先级顺序依序调整图像组中编码帧的权重值,还包括:In an optional method, when the packet loss rate is greater than the packet loss threshold, the weight values of the encoded frames in the image group are adjusted sequentially in order of priority, which also includes:
获取第i个预设时间段内的第一数据丢包率,i为正整数;Obtain the first data packet loss rate in the i-th preset time period, i is a positive integer;
第一数据丢包率大于丢包阈值时,则按照第一策略调整编码帧的权重值,第一策略包括提高第一优先级的编码帧的权重值;When the first data packet loss rate is greater than the packet loss threshold, adjust the weight value of the encoded frame according to the first strategy, and the first strategy includes increasing the weight value of the encoded frame of the first priority;
获取第i+1个预设时间段内的第二数据丢包率;Obtain the second data packet loss rate within the i+1 preset time period;
第二数据丢包率小于第一数据丢包率时,按照第一策略调整第i+2个预设时间段内的编码帧的权重值;或,第二数据丢包率不小于第一数据丢包率时,按照第二策略调整第i+2个预设时间段内的编码帧的权重值,第二策略包括降低第一优先级的编码帧的权重值;When the second data packet loss rate is less than the first data packet loss rate, adjust the weight value of the encoded frame in the i+2 preset time period according to the first strategy; or, the second data packet loss rate is not less than the first data packet loss rate. When the packet loss rate is determined, adjust the weight value of the encoded frame in the i+2 preset time period according to the second strategy. The second strategy includes reducing the weight value of the encoded frame of the first priority;
获取第i+2个预设时间段内的第三数据丢包率;Obtain the third data packet loss rate within the i+2 preset time period;
第三数据丢包率小于第二数据丢包率时,按照第二策略调整第i+3个预设时间段内的编码帧的权重值,或,第三数据丢包率不小于第二数据丢包率时,按照第三策略调整第i+3个预设时间段内的编码帧的权重值,第三策略包括提高第二优先级的编码帧的权重值。When the third data packet loss rate is less than the second data packet loss rate, the weight value of the encoded frame in the i+3 preset time period is adjusted according to the second strategy, or the third data packet loss rate is not less than the second data packet loss rate. When the packet loss rate is reduced, the weight value of the encoded frame in the i+3 preset time period is adjusted according to the third strategy. The third strategy includes increasing the weight value of the encoded frame of the second priority.
通过该方式可以提高关键帧的传输成功率,提高编码效率,并保证编码质量。This method can improve the transmission success rate of key frames, improve coding efficiency, and ensure coding quality.
在一种可选的方式中,根据数据丢包率调整图像组中编码帧的权重值,还包括:若任一预设时间段内的数据丢包率小于丢包阈值,则不调整编码帧的权重值。In an optional method, adjusting the weight value of the encoded frame in the image group according to the data packet loss rate also includes: if the data packet loss rate in any preset time period is less than the packet loss threshold, the encoded frame will not be adjusted. weight value.
在一种可选的方式中,预设值为1。In an optional approach, the default value is 1.
在一种可选的方式中,根据数据丢包率调整图像组中编码帧的权重值,包括:In an optional method, the weight value of the encoded frame in the image group is adjusted according to the data packet loss rate, including:
将数据丢包率输入到权重调整模型中,利用以下算法中的一种或多种确定图像组中编码帧的权重值:蒙特卡洛算法、反向传播算法、牛顿梯度法、一维搜索算法。The data packet loss rate is input into the weight adjustment model and one or more of the following algorithms are used to determine the weight value of the encoded frame in the image group: Monte Carlo algorithm, back propagation algorithm, Newton gradient method, one-dimensional search algorithm .
通过该方式可以及时调整关键帧编码效率,提高译码成功率。In this way, the key frame coding efficiency can be adjusted in time and the decoding success rate can be improved.
在一种可选的方式中,编码数据包,包括:编码帧的位置、喷泉编码的度值。In an optional method, the encoded data packet includes: the position of the encoded frame and the degree value of the fountain encoding.
第二方面,本申请提供一种喷泉码解码方法,该方法可通过喷泉码解码装置来执行,其中喷泉码解码装置可以为图像解码器,本申请在此不具体限定。In a second aspect, the present application provides a fountain code decoding method, which can be executed by a fountain code decoding device, where the fountain code decoding device can be an image decoder, which is not specifically limited in this application.
喷泉码解码装置接收编码数据包;编码数据包是根据编码帧的权重值进行喷泉编码确定的;对编码数据包进行解析获取视频数据。The fountain code decoding device receives the encoded data packet; the encoded data packet is determined by fountain coding according to the weight value of the encoded frame; it analyzes the encoded data packet to obtain video data.
在一种可选的方式中,确定数据丢包率;In an optional manner, determine the data packet loss rate;
将数据丢包率反馈至编码装置。Feed back the data packet loss rate to the encoding device.
在一种可选的方式中,数据丢包率通过以下参数中一种指示:In an optional manner, the data packet loss rate is indicated by one of the following parameters:
解码装置在预设时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;The ratio of the encoded data packets that the decoding device failed to parse within the preset time period to all the encoded data packets received;
第一比值与第二比值的差值,第一比值为解码装置在第一时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;第二比值为解码装置在第二时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;其中,第一时间段与第二时间段为相邻时间段。The difference between the first ratio and the second ratio. The first ratio is the ratio of the coded data packets that the decoding device failed to parse in the first time period to all the coded data packets received; the second ratio is the ratio of the coded data packets that the decoding device failed to parse in the second time period. The ratio of the coded data packets that were not successfully parsed within the segment to all the coded data packets received; where the first time period and the second time period are adjacent time periods.
第三方面,本申请提供一种喷泉码编码装置,包括:收发单元和处理单元;收发单元,用于获取数据丢包率;处理单元,用于根据数据丢包率调整图像组中编码帧的权重值,根据编码帧的权重值进行喷泉编码,得到编码数据包;将编码数据包传输至解码装置。In a third aspect, the present application provides a fountain code encoding device, including: a transceiver unit and a processing unit; the transceiver unit is used to obtain the data packet loss rate; the processing unit is used to adjust the coding frame in the image group according to the data packet loss rate. Weight value, perform fountain encoding according to the weight value of the encoded frame, and obtain the encoded data packet; transmit the encoded data packet to the decoding device.
在一种可选的方式中,数据丢包率通过以下参数中一种指示:In an optional manner, the data packet loss rate is indicated by one of the following parameters:
解码装置在预设时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;The ratio of the encoded data packets that the decoding device failed to parse within the preset time period to all the encoded data packets received;
第一比值与第二比值的差值,第一比值为解码装置在第一时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;第二比值为解码装置在第二时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;其中,第一时间段与第二时间段为相邻时间段。The difference between the first ratio and the second ratio. The first ratio is the ratio of the coded data packets that the decoding device failed to parse in the first time period to all the coded data packets received; the second ratio is the ratio of the coded data packets that the decoding device failed to parse in the second time period. The ratio of the coded data packets that were not successfully parsed within the segment to all the coded data packets received; where the first time period and the second time period are adjacent time periods.
在一种可选的方式中,处理单元,具体用于:在预设时间段内,根据数据丢包率调整图像组中编码帧的权重值。In an optional manner, the processing unit is specifically configured to: adjust the weight value of the encoded frame in the image group according to the data packet loss rate within a preset time period.
在一种可选的方式中,编码帧包括以下中的至少一种:I帧、P帧以及B帧;I帧、P帧以及B帧的优先级依序降低;处理单元,具体用于:In an optional manner, the encoded frame includes at least one of the following: I frame, P frame and B frame; the priority of I frame, P frame and B frame decreases in order; a processing unit is specifically used for:
在预设时间段内,数据丢包率大于丢包阈值时,按照优先级顺序依序调整图像组中编码帧的权重值,其中,编码帧的权重值之和为预设值,优先级高的编码帧的权重值大于优先级低的编码帧的权重值。Within the preset time period, when the data packet loss rate is greater than the packet loss threshold, the weight values of the encoded frames in the image group are adjusted in order of priority. Among them, the sum of the weight values of the encoded frames is the preset value, and the higher priority The weight value of the coded frame is greater than the weight value of the coded frame with lower priority.
在一种可选的方式中,处理单元,还用于:In an optional manner, the processing unit is also used to:
获取第i个预设时间段内的第一数据丢包率,i为正整数;Obtain the first data packet loss rate in the i-th preset time period, i is a positive integer;
第一数据丢包率大于丢包阈值时,则按照第一策略调整编码帧的权重值,第一策略包括提高第一优先级的编码帧的权重值;When the first data packet loss rate is greater than the packet loss threshold, adjust the weight value of the encoded frame according to the first strategy, and the first strategy includes increasing the weight value of the encoded frame of the first priority;
获取第i+1个预设时间段内的第二数据丢包率;Obtain the second data packet loss rate within the i+1 preset time period;
第二数据丢包率小于第一数据丢包率时,按照第一策略调整第i+2个预设时间段内的编码帧的权重值;或,第二数据丢包率不小于第一数据丢包率时,按照第二策略调整第i+2个预设时间段内的编码帧的权重值,第二策略包括降低第一优先级的编码帧的权重值;When the second data packet loss rate is less than the first data packet loss rate, adjust the weight value of the encoded frame in the i+2 preset time period according to the first strategy; or, the second data packet loss rate is not less than the first data packet loss rate. When the packet loss rate is determined, adjust the weight value of the encoded frame in the i+2 preset time period according to the second strategy. The second strategy includes reducing the weight value of the encoded frame of the first priority;
获取第i+2个预设时间段内的第三数据丢包率;Obtain the third data packet loss rate within the i+2 preset time period;
第三数据丢包率小于第二数据丢包率时,按照第二策略调整第i+3个预设时间段内的编码帧的权重值,或,第三数据丢包率不小于第二数据丢包率时,按照第三策略调整第i+3个预设时间段内的编码帧的权重值,第三策略包括提高第二优先级的编码帧的权重值。When the third data packet loss rate is less than the second data packet loss rate, the weight value of the encoded frame in the i+3 preset time period is adjusted according to the second strategy, or the third data packet loss rate is not less than the second data packet loss rate. When the packet loss rate is reduced, the weight value of the encoded frame in the i+3 preset time period is adjusted according to the third strategy. The third strategy includes increasing the weight value of the encoded frame of the second priority.
在一种可选的方式中,处理单元,还用于:若任一预设时间段内的数据丢包率小于丢包阈值,则不调整编码帧的权重值。In an optional method, the processing unit is also configured to: if the data packet loss rate in any preset time period is less than the packet loss threshold, not adjust the weight value of the encoded frame.
在一种可选的方式中,预设值为1。In an optional approach, the default value is 1.
在一种可选的方式中,处理单元,具体用于:In an optional manner, the processing unit is specifically used to:
将数据丢包率输入到权重调整模型中,利用以下算法中的一种或多种确定图像组中编码帧的权重值:蒙特卡洛算法、反向传播算法、牛顿梯度法、一维搜索算法。The data packet loss rate is input into the weight adjustment model and one or more of the following algorithms are used to determine the weight value of the encoded frame in the image group: Monte Carlo algorithm, back propagation algorithm, Newton gradient method, one-dimensional search algorithm .
在一种可选的方式中,编码数据包,包括:编码帧的位置、喷泉编码的度值。In an optional method, the encoded data packet includes: the position of the encoded frame and the degree value of the fountain encoding.
第四方面,本申请提供一种喷泉码解码装置,包括:处理单元和收发单元;In the fourth aspect, this application provides a fountain code decoding device, including: a processing unit and a transceiver unit;
收发单元,用于接收编码数据包;编码数据包是根据编码帧的权重值进行喷泉编码确定的;处理单元,用于对编码数据包进行解析获取视频数据。The transceiver unit is used to receive the encoded data packet; the encoded data packet is determined by fountain coding according to the weight value of the encoded frame; the processing unit is used to parse the encoded data packet to obtain video data.
在一种可选的方式中,处理单元,用于确定数据丢包率;收发单元,用于将数据丢包率反馈至编码装置。In an optional method, the processing unit is used to determine the data packet loss rate; the transceiver unit is used to feed back the data packet loss rate to the encoding device.
在一种可选的方式中,数据丢包率通过以下参数中一种指示:In an optional manner, the data packet loss rate is indicated by one of the following parameters:
解码装置在预设时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;The ratio of the encoded data packets that the decoding device failed to parse within the preset time period to all the encoded data packets received;
第一比值与第二比值的差值,第一比值为解码装置在第一时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;第二比值为解码装置在第二时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;其中,第一时间段与第二时间段为相邻时间段。The difference between the first ratio and the second ratio. The first ratio is the ratio of the coded data packets that the decoding device failed to parse in the first time period to all the coded data packets received; the second ratio is the ratio of the coded data packets that the decoding device failed to parse in the second time period. The ratio of the coded data packets that were not successfully parsed within the segment to all the coded data packets received; where the first time period and the second time period are adjacent time periods.
第五方面,本申请实施例提供一种喷泉码编码装置包括:相互耦合的非易失性存储器和处理器,所述处理器调用存储在所述存储器中的程序代码以执行第一方面或者第一方面的任一设计所述的方法。In a fifth aspect, embodiments of the present application provide a fountain code encoding device including: a mutually coupled non-volatile memory and a processor, the processor calls the program code stored in the memory to execute the first aspect or the second aspect. One aspect of either design is the method described.
第六方面,本申请实施例提供一种喷泉码解码装置包括:相互耦合的非易失性存储器和处理器,所述处理器调用存储在所述存储器中的程序代码以执行第二方面或者第二方面的任一设计所述的方法。需要说明的是,处理器不执行编码操作。In a sixth aspect, embodiments of the present application provide a fountain code decoding device including: a mutually coupled non-volatile memory and a processor, and the processor calls the program code stored in the memory to execute the second aspect or the third aspect. Either design method described in the two aspects. It should be noted that the processor does not perform encoding operations.
第七方面,本申请实施例提供一种喷泉码处理系统,包括第五方面所述的喷泉码编码装置和第六方面所述的喷泉码解码装置。In a seventh aspect, embodiments of the present application provide a fountain code processing system, including the fountain code encoding device described in the fifth aspect and the fountain code decoding device described in the sixth aspect.
第八方面,本申请实施例提供一种计算机可读存储介质,所述计算机可读存储介质存储了程序代码,其中,所述程序代码包括用于执行第一方面或第二方面的任意一种方法的部分或全部步骤的指令。In an eighth aspect, embodiments of the present application provide a computer-readable storage medium that stores program code, wherein the program code includes any one of the methods for executing the first aspect or the second aspect. Instructions for some or all of the steps of a method.
第九方面,本申请实施例提供一种计算机程序产品,当所述计算机程序产品在计算机上运行时,使得所述计算机执行第一方面或第二方面的任意一种方法的部分或全部步骤。In a ninth aspect, embodiments of the present application provide a computer program product, which when the computer program product is run on a computer, causes the computer to execute part or all of the steps of any method of the first aspect or the second aspect.
应当理解的是,本申请的第二至第九方面的有益效果可以参见第一方面的相关描述,不再赘述。It should be understood that the beneficial effects of the second to ninth aspects of the present application can be referred to the relevant description of the first aspect, and will not be described again.
附图说明Description of drawings
图1示出了一种LT码编码和解码的示意图;Figure 1 shows a schematic diagram of LT code encoding and decoding;
图2示出了一种Raptor码编码的示意图;Figure 2 shows a schematic diagram of Raptor code encoding;
图3示出了一种图像组的结构示意图;Figure 3 shows a schematic structural diagram of an image group;
图4示出了本申请实施例提供的一种流媒体数据处理流程的示意图;Figure 4 shows a schematic diagram of a streaming media data processing flow provided by an embodiment of the present application;
图5示出了本申请实施例提供的一种喷泉码编码、解码方法的流程示意图;Figure 5 shows a schematic flow chart of a fountain code encoding and decoding method provided by an embodiment of the present application;
图6示出了本申请实施例提供的一种滑动窗口的示意图;Figure 6 shows a schematic diagram of a sliding window provided by an embodiment of the present application;
图7示出了本申请实施例提供的一种权重调整模型的示意图;Figure 7 shows a schematic diagram of a weight adjustment model provided by an embodiment of the present application;
图8示出了本申请实施例提供的一种喷泉码处理装置的结构示意图;Figure 8 shows a schematic structural diagram of a fountain code processing device provided by an embodiment of the present application;
图9示出了本申请实施例提供的一种喷泉码处理装置的结构示意图;Figure 9 shows a schematic structural diagram of a fountain code processing device provided by an embodiment of the present application;
图10示出了本申请实施例提供的一种喷泉码处理装置的结构示意图。Figure 10 shows a schematic structural diagram of a fountain code processing device provided by an embodiment of the present application.
具体实施方式Detailed ways
为了使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请作进一步地详细描述。方法实施例中的具体操作方法也可以应用于装置实施例或系统实施例中。其中,在本申请的描述中,除非另有说明,“多个”的含义是两个或两个以上。因此装置与方法的实施可以相互参见,重复之处不再赘述。In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be described in further detail below in conjunction with the accompanying drawings. The specific operation methods in the method embodiments can also be applied to the device embodiments or system embodiments. Among them, in the description of this application, unless otherwise stated, the meaning of "plurality" is two or more. Therefore, the implementation of the device and the method can be referred to each other, and repeated descriptions will not be repeated.
为了更好地说明本申请的方案,首先对本申请可能涉及的技术用语进行解释说明,具体如下:In order to better explain the solution of this application, first, the technical terms that may be involved in this application are explained, as follows:
1.现有的喷泉码包括两类:卢比变换码(luby transform codes,LT码)和速龙码(rapid tornado code,Raptor码)。LT码是喷泉码的第一次具体实现,是由Michael Luby提出的,后来Amin Shokrollahi对LT码做出了改进,提出了第二类喷泉码,即Raptor码。1. Existing fountain codes include two types: luby transform codes (LT codes) and rapid tornado codes (Raptor codes). LT code is the first specific implementation of fountain code, which was proposed by Michael Luby. Later, Amin Shokrollahi improved the LT code and proposed the second type of fountain code, namely Raptor code.
1)、LT码1), LT code
其中,LT码的编码过程可如图1中的(a)所示,待编码数据被分成等长的多个比特的数据包S1、S2…Sn,为每一个编码包C1、C2…Cn随机产生一个度值d,度值即为与其有关数据包的个数,比如C1度为3,C2度为1,在所有的数据包中,均匀地随机选取d个数据包,编码包即为d个输入数据包的异或的计算结果。Among them, the encoding process of LT code can be shown in (a) in Figure 1. The data to be encoded is divided into multiple bit data packets S 1 , S 2 ...Sn of equal length, and each coded packet C 1 , C 2 ...Cn randomly generates a degree value d. The degree value is the number of data packets related to it. For example, the C1 degree is 3 and the C2 degree is 1. Among all data packets, d data packets are randomly selected evenly and encoded. The packet is the calculation result of the XOR of d input data packets.
接收端在解码时必须知道每一个编码包的度值,以及具体由哪几个数据包异或产生。如果在某一时刻,度为1的包全消失了,但仍旧有一些源数据包尚未恢复,这样就会译码失败。度为1的包过多,也会导致冗余,效率不高。如图1中的(b)所示,找到度数为1的编码包,因为只与一个数据包相关,若没有度数为1的编码包,则译码过程结束。将恢复出的数据包与其他所有与此数据包相关的编码包进行异或计算,并将这些编码包的度数减1,这个过程可能会产生新的度数为1的编码包(解异或运算)。重复上述步骤,直至全部译出。如图1中(b),S2为度为1的编码包,可获取数据包C2,之后进行解异或运算恢复出全部的数据包。The receiving end must know the degree value of each encoded packet when decoding, and which data packets are specifically XORed. If at a certain moment, all packets with degree 1 disappear, but there are still some source data packets that have not been recovered, the decoding will fail. Too many packets with a degree of 1 will also lead to redundancy and low efficiency. As shown in (b) in Figure 1, find the encoding packet with degree 1, because it is only related to one data packet. If there is no encoding packet with degree 1, the decoding process ends. Perform an XOR calculation on the recovered data packet with all other encoding packets related to this data packet, and reduce the degree of these encoding packets by 1. This process may generate new encoding packets with a degree of 1 (exclusive OR operation ). Repeat the above steps until all are translated. As shown in (b) in Figure 1, S 2 is a coded packet with degree 1. Data packet C 2 can be obtained, and then the XOR operation is performed to recover all the data packets.
此外,理想情况下,一个度分布只会产生一个度为1的编码包,而每次一个数据包被恢复时,又恰好能够重新生成一个度为1的编码包。如此往复,始终保持一个度为1的编码包可以使用,直至n个数据包全部译出,译码过程结束。Luby提出了理想孤波分布(后来为了加强鲁棒性,提出鲁棒孤波分布),在此不具体示意度分布的形式。In addition, ideally, a degree distribution will only generate one coded packet with degree 1, and every time a data packet is recovered, it can regenerate exactly one coded packet with degree 1. This goes back and forth, always keeping a coded packet with degree 1 available, until all n data packets are decoded, and the decoding process ends. Luby proposed the ideal solitary wave distribution (later, in order to enhance the robustness, the robust solitary wave distribution was proposed), but the form of the degree distribution is not specified here.
2)、Raptor码2), Raptor code
Raptor码主要由两个编码过程组成:预编码和LT编码。如图2所示,待编码数据包S1、S2…S6,经过预编码得到预编码编码包C1、C2…C9,其中预编码编码包的数量多于待编码数据包,C7、C8、C9为冗余节点,之后对预编码编码包C1、C2…C9进行异或运算,确定编码数据。Raptor code mainly consists of two encoding processes: precoding and LT encoding. As shown in Figure 2, the data packets S 1 , S 2 ...S 6 to be encoded are precoded to obtain precoded coded packets C 1 , C 2 ...C 9 , in which the number of precoded coded packets is greater than the data packets to be coded. C 7 , C 8 , and C 9 are redundant nodes. Then, XOR operation is performed on the precoding encoding packets C 1 , C 2 ...C 9 to determine the encoding data.
过程1.预编码主要是有固定码率的传统纠删码,计算速度非常快,常见的是由高密度奇偶校验码与简单的正则低密度奇偶校验码级联而成(比如汉明码+低密度校验码)。Process 1. Precoding is mainly a traditional erasure code with a fixed code rate. The calculation speed is very fast. It is commonly composed of a high-density parity check code and a simple regular low-density parity check code cascaded (such as Hamming code + low density check code).
过程2.LT编码过程则变成预编码输出的中间编码块,过程遵循标准LT编码过程。Process 2. The LT encoding process becomes the intermediate encoding block of the precoding output, and the process follows the standard LT encoding process.
2.流媒体2. Streaming media
流媒体为一种将连续的多媒体数据(视频、图像、音频等)压缩后,分段通过网络传输,客户端接收到一部分连续数据后能够直接进行播放的技术。Streaming media is a technology that compresses continuous multimedia data (video, images, audio, etc.) and transmits it in segments through the network. The client can directly play the data after receiving a portion of the continuous data.
以H.265视频编码协议为例,现有编码协议的块编码判决采取遍历模式,如采用拉格朗日优化方法为每个LCU确定所有编码参数,主要包括CU划分模式、CU中PU和TU的划分、PU预测模式等。之后根据CU进行编码处理,首先对CU的原始像素数据(original pixel)进行预测(prediction),预测分为帧内预测(intra prediction)和帧间预测(interprediction)。在进行信源编码时,通常是基于图像组(group of picture,GoP)进行编码处理的,一个图像组中可能包括一个或多个编码帧,如:I帧、P帧以及B帧,其中,I帧为关键帧,后面的P帧或者B帧参考I帧进行编码处理。如图3所示为一个图像组的结构示意图,该图像组对应的时间间隔为30S,其编码帧的结构为IIPP…B,阴影填充代表I帧,非阴影填充代表P帧,黑色填充为B帧。Taking the H.265 video coding protocol as an example, the block coding decision of the existing coding protocol adopts the traversal mode. For example, the Lagrangian optimization method is used to determine all coding parameters for each LCU, which mainly includes the CU division mode, PU and TU in the CU. division, PU prediction mode, etc. Afterwards, the encoding process is performed according to the CU. First, the original pixel data (original pixel) of the CU is predicted. The prediction is divided into intra prediction (intra prediction) and inter prediction (interprediction). When encoding sources, encoding is usually performed based on a group of pictures (GoP). A group of pictures may include one or more encoded frames, such as I frames, P frames, and B frames, where, The I frame is a key frame, and subsequent P frames or B frames refer to the I frame for coding processing. Figure 3 shows a schematic structural diagram of an image group. The corresponding time interval of the image group is 30S. The structure of the encoded frame is IIPP...B. The shadow filling represents I frame, the non-shading filling represents P frame, and the black filling represents B. frame.
本申请中,“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B的情况,其中A,B可以是单数或者复数。以及,除非有相反的说明,本申请实施例提及“第一”、“第二”等序数词是用于对多个对象进行区分,不用于限定多个对象的顺序、时序、优先级或者重要程度。In this application, "and/or" describes the relationship between associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone. , where A and B can be singular or plural. And, unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of this application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or priority of multiple objects. Importance.
在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此,在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例,而是意味着“一个或多个但不是所有的实施例”,除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”,除非是以其他方式另外特别强调。Reference in this specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments", etc. appearing in different places in this specification are not necessarily References are made to the same embodiment, but rather to "one or more but not all embodiments" unless specifically stated otherwise. The terms “including,” “includes,” “having,” and variations thereof all mean “including but not limited to,” unless otherwise specifically emphasized.
目前已经存在一些不等差错保护喷泉码概念设计,不过仅存在于学术研究,没有商用技术。不等差错保护喷泉码通过赋予信息数据不同的参与编码权重,不同信息划分多个重要等级,实现对信息重要性的区分。发明人在研究喷泉码时,发现将喷泉码的编码方案,应用到流媒体中,有利于流媒体数据更好地编码和传输,其应用流程可如图4所示,图像数据(GRB数据或YUV数据)经过H.26X(X=3/4/5)编码后,经过数据整形(如划分成多个滑动窗口),通过喷泉码编码处理,打包发送到客户端。利用喷泉码进行编码的方案,除了应用于流媒体以外,还可以用于全向视频、增强现实(augmented reality,AR)/虚拟现实技术(virtual reality,VR)等媒体数据的传输,本申请在此不具体限定。There are already some conceptual designs of unequal error protection fountain codes, but they only exist in academic research and there is no commercial technology. The unequal error protection fountain code achieves the distinction of information importance by giving different participation coding weights to information data and dividing different information into multiple importance levels. When the inventor was studying the fountain code, he found that applying the encoding scheme of the fountain code to streaming media is beneficial to better encoding and transmission of streaming media data. The application process can be shown in Figure 4. Image data (GRB data or YUV data) is encoded by H.26X (X=3/4/5), then reshaped (such as divided into multiple sliding windows), encoded by fountain code, and packaged and sent to the client. In addition to being used in streaming media, the encoding scheme using fountain codes can also be used for the transmission of omnidirectional video, augmented reality (AR)/virtual reality (VR) and other media data. This application is in This is not specifically limited.
在一些流媒体场景下,部分数据是重要信息需要更多保护,例如MPEG流中,如果I帧独立于其他帧设计,P依赖I信息,B依赖前面I和P信息,故I帧需要更多保护,如何根据流媒体帧进行喷泉码的针对性适配非常重要。此外,人为设定的权重较为机械,当某些场景出现传输故障(花屏)或者质量降级(马赛克)等现象时,不能自适应进行帧权重调整。In some streaming media scenarios, some data are important information and require more protection. For example, in MPEG streams, if the I frame is designed independently of other frames, P depends on the I information, and B depends on the previous I and P information, so the I frame requires more Protection, how to perform targeted adaptation of fountain codes based on streaming media frames is very important. In addition, the artificially set weights are relatively mechanical. When certain scenes have transmission failures (blurred screens) or quality degradation (mosaic), frame weights cannot be adjusted adaptively.
目前主要的喷泉码技术存在不能针对场景进行权重调整或权重盲调的缺陷,特别是在一些流媒体场景下,部分数据是重要信息,需要更多保护。虽然现有的不等差错保护喷泉码技术,考虑了信息的重要性差别,但没有通过调整图像组中编码帧被编码位选中的机率或者权重来进行重要性区分进而进行喷泉码编码,不能实际保证对关键编码帧的传输保护。另外,如何进行针对性适配也存在着一定的机械性,多采用人为设定的方式来完成,如学界提到的不等差错保护编码(无实际应用案例)就存在一定的人为设定问题,不能针对场景自适应进行调整。The current main fountain code technology has the disadvantage that it cannot perform weight adjustment or blind weight adjustment according to the scene. Especially in some streaming media scenarios, some data are important information and require more protection. Although the existing unequal error protection fountain code technology takes into account the difference in importance of information, it does not adjust the probability or weight of the coded frame in the image group to be selected by the coded bit to distinguish the importance and then perform fountain code coding, which cannot be practical. Guaranteed transmission protection of key coded frames. In addition, there is also a certain degree of mechanics in how to carry out targeted adaptation, and it is often accomplished by artificial settings. For example, the unequal error protection encoding mentioned by the academic circle (no practical application case) has certain artificial setting problems. , cannot be adjusted for scene adaptation.
基于此,本申请提供一种喷泉码编码方法,以更好地适配流媒体的场景需求,可参照图5来执行,其中,编码装置可以理解为编码器、CPU、GPU等,解码装置可以理解为译码器等,执行如下:Based on this, this application provides a fountain code encoding method to better adapt to the scene requirements of streaming media. It can be executed with reference to Figure 5, where the encoding device can be understood as an encoder, CPU, GPU, etc., and the decoding device can Understood as a decoder, etc., the execution is as follows:
步骤501,编码装置获取数据丢包率。Step 501: The encoding device obtains the data packet loss rate.
在初次执行喷泉码编码的方案时,该数据丢包率可以为预设值。可选的,该数据丢包率可通过其他数据处理装置下发,如:服务器等,还可来自于解码装置,也即通过解码装置下发,图5中以解码装置通过步骤500,发送数据丢包率为例进行示意。When the fountain code encoding solution is first implemented, the data packet loss rate may be a preset value. Optionally, the data packet loss rate can be delivered through other data processing devices, such as servers, etc., or it can also come from the decoding device, that is, delivered through the decoding device. In Figure 5, the decoding device passes step 500 to send data. The packet loss rate is taken as an example.
其中,数据丢包率可通过以下参数中一种指示:Among them, the data packet loss rate can be indicated by one of the following parameters:
参数1、解码装置在预设时间段内未成功解析的编码数据包与接收的所有编码数据包的比值。Parameter 1. The ratio of the coded data packets that the decoding device failed to parse successfully within the preset time period to all the coded data packets received.
例如,预设时间段为10S,解码装置在10S内接收到10个编码数据包,成功解析8个编码数据包,未成功解析2(10-8)个编码数据包,那么数据丢包率可以为2/10,在此仅示例性描述,具体预设时间段设置为多少可根据实际应用进行设置,本申请在此不具体限定。For example, if the preset time period is 10S, the decoding device receives 10 encoded data packets within 10S, successfully parses 8 encoded data packets, and fails to parse 2 (10-8) encoded data packets, then the data packet loss rate can be It is 2/10. This is only an exemplary description. The specific preset time period can be set according to the actual application and is not specifically limited in this application.
参数2、第一比值与第二比值的差值,第一比值为解码装置在第一时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;第二比值为解码装置在第二时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;其中,第一时间段与第二时间段为相邻时间段。Parameter 2, the difference between the first ratio and the second ratio. The first ratio is the ratio of the encoded data packets that the decoding device failed to parse in the first time period to all the encoded data packets received; the second ratio is the ratio of the decoding device in the first time period. The ratio of coded data packets that were not successfully parsed to all coded data packets received in the second time period; where the first time period and the second time period are adjacent time periods.
例如,第一时间段为8:00~8:10,第二时间段为8:10~8:20,第一时间段内成功解析的编码数据包为6个,未成功解析的编码数据包为4个,接收的数据包为10个,第一比值为4/10,第二时间段内成功解析的编码数据包为18个,未成功解析的编码数据包为2个,接收的数据包为20个,第二比值为2/20,第一比值与第二比值的差值为0.3(4/10-2/20)。For example, the first time period is 8:00~8:10, the second time period is 8:10~8:20, the number of successfully parsed encoded data packets in the first time period is 6, and the number of unsuccessfully parsed encoded data packets is is 4, the number of received data packets is 10, the first ratio is 4/10, the number of successfully parsed encoded data packets in the second time period is 18, the number of unsuccessfully parsed encoded data packets is 2, and the number of received data packets is 20, the second ratio is 2/20, and the difference between the first ratio and the second ratio is 0.3 (4/10-2/20).
当然在实际应用时,解码装置可反馈参数1,也可反馈参数2,甚至还可将参数1和参数2进行加权运算后反馈至编码装置,本申请在此不具体限定,只要编码装置基于解码装置反馈的参数可以获取数据丢包率的信息即可。Of course, in practical applications, the decoding device can feed back parameter 1 or parameter 2, or even perform a weighted operation on parameter 1 and parameter 2 and then feed them back to the encoding device. This application is not specifically limited here, as long as the encoding device is based on decoding The parameters fed back by the device can be used to obtain information on the data packet loss rate.
步骤502,编码装置根据数据丢包率调整图像组中编码帧的权重值。Step 502: The encoding device adjusts the weight value of the encoded frame in the image group according to the data packet loss rate.
其中,图像组中编码帧的权重值不是一直不变的,可以在预设时间段内,基于数据丢包率进行调整,如预设时间段1内,数据丢包率增大,可提高编码帧的权重值,数据丢包率降低,可降低编码帧的权重值,具体如何调整可基于数据丢包率的大小灵活设置,相较于机械设置权重值以及固定的权重值而言,本申请基于数据丢包率调整编码帧的权重值可以提高编码效率。Among them, the weight value of the encoded frame in the image group is not always unchanged. It can be adjusted based on the data packet loss rate within the preset time period. For example, within the preset time period 1, the data packet loss rate increases, which can improve the encoding The weight value of the frame can reduce the data packet loss rate, which can reduce the weight value of the encoded frame. The specific adjustment can be flexibly set based on the size of the data packet loss rate. Compared with the mechanical setting of the weight value and the fixed weight value, this application Adjusting the weight value of the encoded frame based on the data packet loss rate can improve encoding efficiency.
示例性地,GoP的编码帧可能包括I帧、P帧以及B帧,且一个视频数据可能对应多个GoP,每个GoP的时间长度是相同的,且每个GoP中编码帧的格式是相同的,如视频数据1包括3个GoP,每个GoP的时间长度为10S,且对应的编码帧的格式为IBBPBBPBBPBB,I帧对应的权重值也即编码装置选择的机率可以为权重1,P帧对应的权重值也即编码装置选择的机率可以为权重2,B帧对应的权重值也即编码装置选择的机率可以为权重3,权重1可大于权重2,权重2大于权重3,本申请在此不具体限定。For example, the encoded frames of a GoP may include I frames, P frames, and B frames, and one video data may correspond to multiple GoPs. The time length of each GoP is the same, and the format of the encoded frames in each GoP is the same. , for example, video data 1 includes 3 GoPs, the time length of each GoP is 10S, and the format of the corresponding encoded frame is IBBPBBPBBPBB, the weight value corresponding to the I frame, that is, the probability of selection by the encoding device, can be weight 1, P frame The corresponding weight value, that is, the probability of selection by the encoding device, can be weight 2. The weight value corresponding to the B frame, that is, the probability of selection by the encoding device, can be weight 3. Weight 1 can be greater than weight 2, and weight 2 can be greater than weight 3. This application is This is not specifically limited.
可选的,编码装置将GoP划分成N个等长的滑动窗口;滑动窗口存在交叠;N≥1;N为正整数。根据数据丢包率调整滑动窗口中各编码帧的权重值。视频数据具有多个GoP,一般来说一个独立的视频,其GoP格式是固定的,由1个I帧和多个P或B帧构成(I/P/B、I/B或I/P)。举例来说,GoP可以由1个I帧,3个P帧,8个B帧构成对应的编码帧的格式为IBBPBBPBBPBB,编码装置可以将一个GOP分成多个滑动窗口,一个I帧可以被切分到多个滑动窗口,如将GoP的I帧分配到滑动窗口1、2、3后,I帧还有剩余的情况下,可将剩余部分分配到下一个滑动窗口4,B帧可以被切分到滑动窗口4、5、6等,B帧还有剩余的情况下,可将剩余部分分配到下一个滑动窗口。由于编码帧大小不定,所占滑动窗口数目也表现不一,被切分的窗口按照实际大小占优的帧进行类型标记,如滑动窗口1内的编码帧全来自I帧,其类型标记为I帧,滑动窗口4的编码帧来自I帧剩余部分和B帧部分,根据实际占据比例进行类型标记,若I帧占比不小于0.5,则滑动窗口4类型可标记为I帧,否则,标记为B帧。以此类推,定义不同滑动窗口的帧类型。之后,可根据数据丢包率调整预设时间内不同类型编码帧的权重值,如,调整I帧的权重值增加,凡是定义为I帧的滑动窗口被选中的概率均增加,那么I帧被保护的机会更大。Optionally, the encoding device divides the GoP into N sliding windows of equal length; the sliding windows overlap; N≥1; N is a positive integer. Adjust the weight value of each encoded frame in the sliding window according to the data packet loss rate. Video data has multiple GoPs. Generally speaking, an independent video has a fixed GoP format and consists of one I frame and multiple P or B frames (I/P/B, I/B or I/P). . For example, a GoP can be composed of 1 I frame, 3 P frames, and 8 B frames. The corresponding encoding frame format is IBBPBBPBBPBB. The encoding device can divide a GOP into multiple sliding windows, and an I frame can be segmented. to multiple sliding windows. For example, after allocating GoP I frames to sliding windows 1, 2, and 3, if there are still I frames left, the remaining parts can be allocated to the next sliding window 4, and the B frames can be divided. When reaching sliding windows 4, 5, 6, etc., if there are still B frames left, the remaining parts can be allocated to the next sliding window. Since the size of the encoding frame is variable, the number of sliding windows it occupies also varies. The segmented window is type-marked according to the actual size-dominated frame. For example, the encoding frames in sliding window 1 are all from I frames, and their type is marked I. Frame, the encoded frame of sliding window 4 comes from the remaining part of the I frame and the B frame part. The type is marked according to the actual occupancy proportion. If the I frame proportion is not less than 0.5, the sliding window 4 type can be marked as I frame. Otherwise, it is marked as B frame. By analogy, define the frame types of different sliding windows. After that, the weight values of different types of coded frames within the preset time can be adjusted according to the data packet loss rate. For example, if the weight value of the I frame is adjusted to increase, the probability of being selected for any sliding window defined as an I frame increases, then the I frame is selected. Better chance of protection.
如图6所示,GoP可以是1个I帧,3个P帧,8个B帧构成对应的编码帧的格式为IBBPBBPBBPBB,其中滑动窗口1~滑动窗口3对应均为I帧,滑动窗口4对应部分I帧和B帧,一个GOP可对应多个滑动窗口,在此仅以4个滑动窗口进行示例说明。可基于数据丢包率调整滑动窗口中编码帧的权重值。其中,不同滑动窗口中各编码帧的权重值是确定的,基于确定的权重值,有侧重的进行喷泉码编码可以保证编码帧的编码可靠性。此外,滑动窗口1可对应数据A,滑动窗口2可对应数据B,滑动窗口3对应数据C,滑动窗口4对应数据D,之后可基于度分布进行喷泉码编码,在此不对具体的度分布进行限定,可参考现有的度分布方式来理解,在此不赘述。As shown in Figure 6, GoP can be 1 I frame, 3 P frames, and 8 B frames. The format of the corresponding encoded frame is IBBPBBPBBPBB, where sliding window 1 to sliding window 3 are all I frames, and sliding window 4 Corresponding to some I frames and B frames, one GOP can correspond to multiple sliding windows. Here, only four sliding windows are used as an example. The weight value of the encoded frame in the sliding window can be adjusted based on the data packet loss rate. Among them, the weight value of each coding frame in different sliding windows is determined. Based on the determined weight value, focused fountain code encoding can ensure the coding reliability of the coding frame. In addition, sliding window 1 can correspond to data A, sliding window 2 can correspond to data B, sliding window 3 can correspond to data C, and sliding window 4 can correspond to data D. Afterwards, fountain code encoding can be performed based on the degree distribution. The specific degree distribution will not be discussed here. The limitation can be understood by referring to the existing degree distribution method, and will not be described in detail here.
本申请中,I帧、P帧以及B帧的优先级依序降低;在预设时间段内,数据丢包率大于丢包阈值(表明当前的编码策略不能满足实际场景的应用需求)时,按照优先级顺序依序调整编码帧的权重值,其中,编码帧的权重值之和为预设值(通常该预设值为1,但是在实际应用时,也写根据需求进行调整,如设置预设值为2等,本申请在此不具体限定),优先级高的编码帧的权重值大于优先级低的编码帧的权重值。In this application, the priorities of I frames, P frames, and B frames decrease in order; within the preset time period, when the data packet loss rate is greater than the packet loss threshold (indicating that the current encoding strategy cannot meet the application requirements of actual scenarios), Adjust the weight values of the encoded frames in sequence according to the priority order, where the sum of the weight values of the encoded frames is the default value (usually the default value is 1, but in actual applications, it can also be adjusted according to needs, such as setting The default value is 2, etc., which is not specifically limited in this application), and the weight value of a coded frame with a high priority is greater than the weight value of a coded frame with a low priority.
例如,上述图6中的GoP中包括I帧、P帧以及B帧,其中,I帧的优先级最高,B帧的优先级最低,在设置编码帧的权重时,可将I帧的权重值设置为权重1,P帧的权重值设置为权重2,B帧的权重值设置为权重3。在一种可能的实现方式中,权重1、权重2、权重3之间可以存在如下关系:权重1大于权重2大于权重3,权重1+2*权重2+6*权重3等于预设值。For example, the GoP in Figure 6 above includes I frames, P frames and B frames. Among them, the I frame has the highest priority and the B frame has the lowest priority. When setting the weight of the encoded frame, the weight value of the I frame can be Set to weight 1, the weight value of P frame is set to weight 2, and the weight value of B frame is set to weight 3. In a possible implementation, the following relationship may exist between weight 1, weight 2, and weight 3: weight 1 is greater than weight 2, greater than weight 3, and weight 1+2*weight 2+6*weight 3 is equal to the preset value.
具体地,数据丢包率大于丢包阈值时,按照优先级顺序依序调整编码帧的权重值,其中,各预设时间段内编码帧的格式是相同的,还包括:Specifically, when the data packet loss rate is greater than the packet loss threshold, the weight values of the encoded frames are adjusted in order of priority, where the format of the encoded frames in each preset time period is the same, and also includes:
获取第i个预设时间段内的第一数据丢包率;i为正整数;第一数据丢包率大于丢包阈值时,则按照第一策略调整第i+1个预设时间段内编码帧的权重值,第一策略包括提高第一优先级的编码帧的权重值;获取第i+1个预设时间段内的第二数据丢包率;第二数据丢包率小于第一数据丢包率时,说明按照第一策略调整编码帧的权重值的方向正确,因此继续按照第一策略调整第i+2个预设时间段内编码帧的权重值;或,第二数据丢包率不小于第一数据丢包率时,说明按照第一策略调整编码帧的权重值的方向错误,因此按照第二策略调整第i+2个预设时间段内编码帧的权重值,第二策略包括降低第一优先级的编码帧的权重值;获取第i+3个预设时间段内的第三数据丢包率;第三数据丢包率小于第二数据丢包率时,说明按照第二策略调整编码帧的权重值的方向正确,因此继续按照第二策略调整第i+3个预设时间段内编码帧的权重值;或,第三数据丢包率不小于第二数据丢包率时,说明按照第二策略调整编码帧的权重值的方向错误,因此按照第三策略调整第i+3个滑动窗口的编码帧的权重值,第三策略包括提高第二优先级的编码帧的权重值。Obtain the first data packet loss rate in the i-th preset time period; i is a positive integer; when the first data packet loss rate is greater than the packet loss threshold, adjust the i+1-th preset time period according to the first strategy The weight value of the encoded frame. The first strategy includes increasing the weight value of the encoded frame of the first priority; obtaining the second data packet loss rate within the i+1 preset time period; the second data packet loss rate is less than the first When the data packet loss rate is high, it means that the direction of adjusting the weight value of the encoded frame according to the first strategy is correct, so continue to adjust the weight value of the encoded frame in the i+2th preset time period according to the first strategy; or, the second data loss When the packet rate is not less than the first data packet loss rate, it means that the direction of adjusting the weight value of the encoded frame according to the first strategy is wrong. Therefore, the weight value of the encoded frame in the i+2 preset time period is adjusted according to the second strategy. The second strategy includes reducing the weight value of the first priority encoded frame; obtaining the third data packet loss rate within the i+3 preset time period; when the third data packet loss rate is less than the second data packet loss rate, indicate The direction of adjusting the weight value of the encoded frame according to the second strategy is correct, so continue to adjust the weight value of the encoded frame in the i+3 preset time period according to the second strategy; or, the third data packet loss rate is not less than the second data When the packet loss rate is high, it means that the direction of adjusting the weight value of the encoded frame according to the second strategy is wrong. Therefore, the weight value of the encoded frame of the i+3 sliding window is adjusted according to the third strategy. The third strategy includes increasing the second priority. The weight value of the encoded frame.
例如,编码装置确定7个预设时间段,各预设时间段内编码帧的格式包括I帧、P帧和B帧,丢包阈值为Y,预设时间段1内的数据丢包率大于Y,则提高预设时间段2中I帧的权重值(如增加0.1),并相应地降低预设时间段2中P帧的权重值或B帧的权重值,亦或者P帧的权重值以及B帧的权重值均降低,还或者仅降低B帧的权重值保持P帧的权重值不变,在此仅示例性说明,并不具体限定,以保证其中I帧的权重值、P帧的权重值和B帧的权重值相加为预设值1。获取预设时间段2的数据丢包率,如果数据丢包率降低,则确定该调整权重值的策略是正确的,在预设时间段3~预设时间段7中按照相同的权重调整策略进行权重的调整,但是若预设时间段3~预设时间段7任一预设时间段(假定该预设时间段为预设时间段5)的丢包率小于或等于Y,则退出权重调整策略(也即不调整编码帧的权重值),保持预设时间段5的权重调整策略调整预设时间段6和预设时间段7中编码帧的权重值,并进行喷泉码编码处理。For example, the encoding device determines 7 preset time periods. The format of the encoded frames in each preset time period includes I frame, P frame and B frame. The packet loss threshold is Y. The data packet loss rate in the preset time period 1 is greater than Y, then increase the weight value of the I frame in the preset time period 2 (for example, increase 0.1), and accordingly reduce the weight value of the P frame or the weight value of the B frame in the preset time period 2, or the weight value of the P frame And the weight value of the B frame is reduced, or only the weight value of the B frame is reduced and the weight value of the P frame remains unchanged. This is only an illustrative explanation and is not specifically limited to ensure that the weight value of the I frame and the P frame are The weight value of and the weight value of the B frame are added to the default value of 1. Obtain the data packet loss rate of preset time period 2. If the data packet loss rate decreases, it is determined that the strategy for adjusting the weight value is correct. Adjust the strategy according to the same weight in preset time period 3 to preset time period 7. Adjust the weight, but if the packet loss rate in any preset time period from preset time period 3 to preset time period 7 (assuming that the preset time period is preset time period 5) is less than or equal to Y, exit the weight Adjust the strategy (that is, do not adjust the weight value of the coded frame), maintain the weight adjustment strategy of the preset time period 5, adjust the weight value of the coded frame in the preset time period 6 and the preset time period 7, and perform fountain code encoding processing.
相反,若提高预设时间段2中I帧的权重值,并相应的降低预设时间段2中B帧的权重值,预设时间段2的数据丢包率没有降低,反而增加,则以相反的策略调整预设时间段3中编码帧的权重值,降低预设时间段3中I帧的权重值(如降低0.1),相应地提高预设时间段3中B帧的权重值,以保证其中I帧的权重值、P帧的权重值和B帧的权重值相加为预设值1。获取预设时间段3的数据丢包率,若预设时间段3的数据丢包率降低,则确定降低I帧的权重值的调整策略是正确的。On the contrary, if the weight value of the I frame in the preset time period 2 is increased, and the weight value of the B frame in the preset time period 2 is correspondingly reduced, the data packet loss rate in the preset time period 2 does not decrease, but increases, so as The opposite strategy adjusts the weight value of the encoded frame in the preset time period 3, reduces the weight value of the I frame in the preset time period 3 (for example, by 0.1), and accordingly increases the weight value of the B frame in the preset time period 3, so as to It is guaranteed that the weight value of the I frame, the weight value of the P frame and the weight value of the B frame add up to the preset value 1. Obtain the data packet loss rate in the preset time period 3. If the data packet loss rate in the preset time period 3 decreases, it is determined that the adjustment strategy of reducing the weight value of the I frame is correct.
若无论增大预设时间段3中I帧的权重值还是降低预设时间段3中I帧的权重值,数据丢包率的变化都不明显,那么则调整P帧的权重值,提高P帧的权重值(如增加0.05),相应地降低I帧的权重值或B帧的权重值,以保证I帧的权重值、P帧的权重值和B帧的权重值相加为预设值1。获取预设时间段4的数据丢包率,如果数据丢包率降低,则确定该调整权重值的策略是正确的,在预设时间段5~预设时间段7中按照相同的权重调整策略进行权重的调整,但是若后面任一预设时间段的丢包率小于或等于Y,则退出权重调整策略,保持最后一次调整的编码帧的权重值,进行喷泉码编码处理。If the change in data packet loss rate is not obvious no matter whether you increase the weight value of I frame in preset time period 3 or decrease the weight value of I frame in preset time period 3, then adjust the weight value of P frame to increase P The weight value of the frame (such as increasing 0.05), correspondingly reduce the weight value of the I frame or the weight value of the B frame to ensure that the weight value of the I frame, the weight value of the P frame and the weight value of the B frame add up to the preset value 1. Obtain the data packet loss rate in the preset time period 4. If the data packet loss rate decreases, it is determined that the strategy for adjusting the weight value is correct. Adjust the strategy according to the same weight in the preset time period 5 to 7. The weight is adjusted, but if the packet loss rate in any subsequent preset time period is less than or equal to Y, the weight adjustment strategy will be exited, the weight value of the last adjusted encoding frame will be maintained, and the fountain code encoding process will be performed.
综上,在本申请提供的一个喷泉码编码实施例中,可首先尝试提高预设时间段内I帧的权重值,看丢包率是否朝着变好的方向改变;若是,则继续提高后续的预设时间段中I帧的权重值,直到丢包率符合要求;否则,降低滑动窗口中I帧的权重值,看丢包率是否朝着变好的方向改变。如果无论提高还是降低I帧的权重值都无法使得丢包率产生明显改变,则尝试调整P帧和B帧的权重值。To sum up, in a fountain code encoding embodiment provided by this application, you can first try to increase the weight value of the I frame within the preset time period to see whether the packet loss rate changes in a better direction; if so, continue to increase the subsequent The weight value of I frames in the preset time period until the packet loss rate meets the requirements; otherwise, reduce the weight value of I frames in the sliding window to see if the packet loss rate changes in a better direction. If increasing or decreasing the weight value of I frames cannot significantly change the packet loss rate, try adjusting the weight values of P frames and B frames.
可选的,编码装置还可将数据丢包率输入到权重调整模型(其中,该权重调整模型可以为机器学习模型、深度学习模型,还可以为其他模型本申请在此不具体限定)中,利用以下算法中的一种或多种确定图像组中编码帧的权重值:蒙特卡洛算法、反向传播算法、牛顿梯度法、一维搜索算法。具体可如下:Optionally, the encoding device can also input the data packet loss rate into a weight adjustment model (wherein, the weight adjustment model can be a machine learning model, a deep learning model, or other models (this application is not specifically limited here)), Determine the weight value of the coded frame in the image group using one or more of the following algorithms: Monte Carlo algorithm, backpropagation algorithm, Newton gradient method, one-dimensional search algorithm. The details can be as follows:
(1)首先建立I/P/B帧权重与数据丢包率的权重调整模型,具体地,可定义一种深度神经网络,输入分别为当前带宽和信号强度序列,输出为I帧、P帧和B帧的权重值,神经网络根据当前带宽和信号强度变化选择合适的I帧、P帧、B帧权重值,参数的优化需要通过目标函数来评价,如图7所示。(1) First, establish a weight adjustment model for I/P/B frame weight and data packet loss rate. Specifically, a deep neural network can be defined. The input is the current bandwidth and signal strength sequence, and the output is I frame and P frame. and B frame weight values. The neural network selects appropriate I frame, P frame, and B frame weight values based on the current bandwidth and signal strength changes. The optimization of parameters needs to be evaluated through the objective function, as shown in Figure 7.
(2)建立目标函数充当参数优化情况衡量标准,以数据丢包率作为目标函数,进行最小化优化求解,可采用最小二乘法来求最优解。(2) Establish an objective function to serve as a measure of parameter optimization, and use the data packet loss rate as the objective function to perform a minimum optimization solution. The least squares method can be used to find the optimal solution.
(3)通过最优化方法(BP、牛顿梯度等)实现对最小数据丢包率的逼近,从而获取合适的I帧、P帧、B帧权重分配。(3) Use optimization methods (BP, Newton gradient, etc.) to achieve approximation to the minimum data packet loss rate, thereby obtaining appropriate weight distribution of I frames, P frames, and B frames.
步骤503,编码装置根据编码帧的权重值进行喷泉编码,得到编码数据包。Step 503: The encoding device performs fountain encoding according to the weight value of the encoded frame to obtain the encoded data packet.
具体地,编码装置确定编码帧的权重值后,可进行数据整形,在进行喷泉编码时,可利用编码帧的权重值,调整喷泉编码的度值分配,按照上述LT码或Raptor码编码的方式进行喷泉编码。Specifically, after the encoding device determines the weight value of the encoding frame, it can perform data shaping. When performing fountain encoding, the weight value of the encoding frame can be used to adjust the degree value distribution of the fountain encoding, according to the above-mentioned LT code or Raptor code encoding method. Do fountain coding.
例如,编码帧包括I帧、P帧以及B帧,其中,I帧的权重值为权重1,P帧的权重为权重2,B帧的权重为权重3,在经过数据整形后,采用LT码进行喷泉编码时,度值可以为参考现有的度分布方式来理解,在此不赘述。For example, the coded frames include I frames, P frames and B frames. Among them, the weight value of the I frame is weight 1, the weight value of the P frame is weight 2, and the weight value of the B frame is weight 3. After data shaping, the LT code is used When encoding fountains, the degree value can be understood by referring to the existing degree distribution method, which will not be described again here.
步骤504,编码装置将编码数据包传输至解码装置。Step 504: The encoding device transmits the encoded data packet to the decoding device.
其中,编码数据包,包括:编码帧的位置、喷泉编码的度值,以便解码装置可以顺利解析编码数据包。Among them, the encoded data packet includes: the position of the encoded frame and the degree value of the fountain encoding, so that the decoding device can successfully parse the encoded data packet.
步骤505,解码装置对编码数据包进行解析获取视频数据。Step 505: The decoding device analyzes the encoded data packet to obtain video data.
通过该方式进行喷泉码的编码译码处理可以将喷泉码更好地应用到流媒体中,保证流媒体中数据编解码效率,同时可以给用户带来良好地业务体验。Encoding and decoding fountain codes in this way can better apply fountain codes to streaming media, ensure the efficiency of data encoding and decoding in streaming media, and at the same time provide users with a good business experience.
上述主要从设备交互的角度对本申请实施例提供的方案进行了介绍。可以理解的是,为了实现上述功能,各个设备可以包括执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,本申请的实施例能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。The above mainly introduces the solutions provided by the embodiments of the present application from the perspective of device interaction. It can be understood that, in order to implement the above functions, each device may include a corresponding hardware structure and/or software module to perform each function. Those skilled in the art should easily realize that, with the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.
本申请实施例可以根据上述方法示例对设备进行功能单元的划分,例如,可以对应各个功能划分各个功能单元,也可以将两个或两个以上的功能集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。Embodiments of the present application can divide the device into functional units according to the above method examples. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.
在采用集成的单元的情况下,图8示出了本申请实施例中所涉及的喷泉码处理装置的可能的示例性框图。如图8所示,喷泉码处理装置800可以包括:处理单元801和收发单元802。处理单元801用于对喷泉码处理装置800的动作进行控制管理。收发单元802用于支持喷泉码处理装置800与其他设备的通信。可选地,收发单元802可以包括接收单元和/或发送单元,分别用于执行接收和发送操作。可选的,喷泉码处理装置800还可以包括存储单元,用于存储喷泉码处理装置800的程序代码和/或数据。所述收发单元可以称为输入输出单元、通信单元等,所述收发单元可以是收发器;所述处理单元可以是处理器。当图像处理装置是通信设备中的模块(如,芯片)时,所述收发单元可以是输入输出接口、输入输出电路或输入输出管脚等,也可以称为接口、通信接口或接口电路等;所述处理单元可以是处理器、处理电路或逻辑电路等。具体地,该图像处理装置可以为上述的喷泉码编码装置或喷泉码解码装置等。In the case of using an integrated unit, FIG. 8 shows a possible exemplary block diagram of the fountain code processing device involved in the embodiment of the present application. As shown in FIG. 8 , the fountain code processing device 800 may include: a processing unit 801 and a transceiver unit 802 . The processing unit 801 is used to control and manage the actions of the fountain code processing device 800. The transceiver unit 802 is used to support communication between the fountain code processing device 800 and other devices. Optionally, the transceiver unit 802 may include a receiving unit and/or a sending unit, respectively configured to perform receiving and sending operations. Optionally, the fountain code processing device 800 may also include a storage unit for storing program codes and/or data of the fountain code processing device 800 . The transceiver unit may be called an input-output unit, a communication unit, etc., the transceiver unit may be a transceiver, and the processing unit may be a processor. When the image processing device is a module (such as a chip) in a communication device, the transceiver unit may be an input-output interface, an input-output circuit, or an input-output pin, etc., and may also be called an interface, a communication interface, or an interface circuit, etc.; The processing unit may be a processor, a processing circuit or a logic circuit, etc. Specifically, the image processing device may be the above-mentioned fountain code encoding device or fountain code decoding device.
在一个实施例中,收发单元802,用于获取数据丢包率;处理单元801,用于根据数据丢包率调整图像组中编码帧的权重值,根据编码帧的权重值进行喷泉编码,得到编码数据包;将编码数据包传输至解码装置。In one embodiment, the transceiver unit 802 is used to obtain the data packet loss rate; the processing unit 801 is used to adjust the weight value of the coded frame in the image group according to the data packet loss rate, and perform fountain coding according to the weight value of the coded frame, to obtain Encode the data packet; transmit the encoded data packet to the decoding device.
在一种可选的方式中,数据丢包率通过以下参数中一种指示:In an optional manner, the data packet loss rate is indicated by one of the following parameters:
解码装置在预设时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;The ratio of the encoded data packets that the decoding device failed to parse within the preset time period to all the encoded data packets received;
第一比值与第二比值的差值,第一比值为解码装置在第一时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;第二比值为解码装置在第二时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;其中,第一时间段与第二时间段为相邻时间段。The difference between the first ratio and the second ratio. The first ratio is the ratio of the coded data packets that the decoding device failed to parse in the first time period to all the coded data packets received; the second ratio is the ratio of the coded data packets that the decoding device failed to parse in the second time period. The ratio of the coded data packets that were not successfully parsed within the segment to all the coded data packets received; where the first time period and the second time period are adjacent time periods.
在一种可选的方式中,处理单元801,具体用于:在预设时间段内,根据数据丢包率调整图像组中编码帧的权重值。In an optional manner, the processing unit 801 is specifically configured to adjust the weight value of the encoded frame in the image group according to the data packet loss rate within a preset time period.
在一种可选的方式中,编码帧包括以下中的至少一种:I帧、P帧以及B帧;I帧、P帧以及B帧的优先级依序降低;处理单元801,具体用于:In an optional manner, the encoded frame includes at least one of the following: I frame, P frame and B frame; the priority of I frame, P frame and B frame decreases in order; the processing unit 801 is specifically used for :
在预设时间段内,数据丢包率大于丢包阈值时,按照优先级顺序依序调整图像组中编码帧的权重值,其中,编码帧的权重值之和为预设值,优先级高的编码帧的权重值大于优先级低的编码帧的权重值。Within the preset time period, when the data packet loss rate is greater than the packet loss threshold, the weight values of the encoded frames in the image group are adjusted in order of priority. Among them, the sum of the weight values of the encoded frames is the preset value, and the higher priority The weight value of the coded frame is greater than the weight value of the coded frame with lower priority.
在一种可选的方式中,处理单元801,还用于:In an optional way, the processing unit 801 is also used to:
获取第i个预设时间段内的第一数据丢包率,i为正整数;Obtain the first data packet loss rate in the i-th preset time period, i is a positive integer;
第一数据丢包率大于丢包阈值时,则按照第一策略调整编码帧的权重值,第一策略包括提高第一优先级的编码帧的权重值;When the first data packet loss rate is greater than the packet loss threshold, adjust the weight value of the encoded frame according to the first strategy, and the first strategy includes increasing the weight value of the encoded frame of the first priority;
获取第i+1个预设时间段内的第二数据丢包率;Obtain the second data packet loss rate within the i+1 preset time period;
第二数据丢包率小于第一数据丢包率时,按照第一策略调整第i+2个预设时间段内的编码帧的权重值;或,第二数据丢包率不小于第一数据丢包率时,按照第二策略调整第i+2个预设时间段内的编码帧的权重值,第二策略包括降低第一优先级的编码帧的权重值;When the second data packet loss rate is less than the first data packet loss rate, adjust the weight value of the encoded frame in the i+2 preset time period according to the first strategy; or, the second data packet loss rate is not less than the first data packet loss rate. When the packet loss rate is determined, adjust the weight value of the encoded frame in the i+2 preset time period according to the second strategy. The second strategy includes reducing the weight value of the encoded frame of the first priority;
获取第i+2个预设时间段内的第三数据丢包率;Obtain the third data packet loss rate within the i+2 preset time period;
第三数据丢包率小于第二数据丢包率时,按照第二策略调整第i+3个预设时间段内的编码帧的权重值,或,第三数据丢包率不小于第二数据丢包率时,按照第三策略调整第i+3个预设时间段内的编码帧的权重值,第三策略包括提高第二优先级的编码帧的权重值。When the third data packet loss rate is less than the second data packet loss rate, the weight value of the encoded frame in the i+3 preset time period is adjusted according to the second strategy, or the third data packet loss rate is not less than the second data packet loss rate. When the packet loss rate is reduced, the weight value of the encoded frame in the i+3 preset time period is adjusted according to the third strategy. The third strategy includes increasing the weight value of the encoded frame of the second priority.
在一种可选的方式中,处理单元801,还用于:若任一预设时间段内的数据丢包率小于丢包阈值,则不调整编码帧的权重值。In an optional manner, the processing unit 801 is also configured to: if the data packet loss rate in any preset time period is less than the packet loss threshold, not adjust the weight value of the encoded frame.
在一种可选的方式中,预设值为1。In an optional approach, the default value is 1.
在一种可选的方式中,处理单元801,具体用于:In an optional manner, the processing unit 801 is specifically used for:
将数据丢包率输入到权重调整模型中,利用以下算法中的一种或多种确定图像组中编码帧的权重值:蒙特卡洛算法、反向传播算法、牛顿梯度法、一维搜索算法。The data packet loss rate is input into the weight adjustment model and one or more of the following algorithms are used to determine the weight value of the encoded frame in the image group: Monte Carlo algorithm, back propagation algorithm, Newton gradient method, one-dimensional search algorithm .
在一种可选的方式中,编码数据包,包括:编码帧的位置、喷泉编码的度值。In an optional method, the encoded data packet includes: the position of the encoded frame and the degree value of the fountain encoding.
在一个实施例中,收发单元802,用于接收编码数据包;编码数据包是根据编码帧的权重值进行喷泉编码确定的;处理单元801,用于对编码数据包进行解析获取视频数据。In one embodiment, the transceiver unit 802 is used to receive encoded data packets; the encoded data packets are determined by fountain coding according to the weight value of the encoded frame; the processing unit 801 is used to parse the encoded data packets to obtain video data.
在一种可选的方式中,处理单元801,用于确定数据丢包率;收发单元,用于将数据丢包率反馈至编码装置。In an optional manner, the processing unit 801 is used to determine the data packet loss rate; the transceiver unit is used to feed back the data packet loss rate to the encoding device.
在一种可选的方式中,数据丢包率通过以下参数中一种指示:In an optional manner, the data packet loss rate is indicated by one of the following parameters:
解码装置在预设时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;The ratio of the encoded data packets that the decoding device failed to parse within the preset time period to all the encoded data packets received;
第一比值与第二比值的差值,第一比值为解码装置在第一时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;第二比值为解码装置在第二时间段内未成功解析的编码数据包与接收的所有编码数据包的比值;其中,第一时间段与第二时间段为相邻时间段。The difference between the first ratio and the second ratio. The first ratio is the ratio of the coded data packets that the decoding device failed to parse in the first time period to all the coded data packets received; the second ratio is the ratio of the coded data packets that the decoding device failed to parse in the second time period. The ratio of the coded data packets that were not successfully parsed within the segment to all the coded data packets received; where the first time period and the second time period are adjacent time periods.
如图9所示,为本申请还提供的一种喷泉码处理装置900。喷泉码处理装置900可以是芯片或芯片系统。该喷泉码处理装置可以位于上述任一方法实施例所涉及的设备中,例如图像编码装置或图像解码装置等,以执行该设备所对应的动作。As shown in FIG. 9 , a fountain code processing device 900 is also provided in this application. The fountain code processing device 900 may be a chip or a chip system. The fountain code processing device may be located in the device involved in any of the above method embodiments, such as an image encoding device or an image decoding device, to perform actions corresponding to the device.
可选的,芯片系统可以由芯片构成,也可以包含芯片和其他分立器件。Optionally, the chip system may be composed of chips, or may include chips and other discrete devices.
喷泉码处理装置900包括处理器910。The fountain code processing device 900 includes a processor 910 .
处理器910,用于执行存储器920中存储的计算机程序,以实现上述任一方法实施例中各个设备的动作。The processor 910 is configured to execute the computer program stored in the memory 920 to implement the actions of each device in any of the above method embodiments.
喷泉码处理装置900还可以包括存储器920,用于存储计算机程序。The fountain code processing device 900 may also include a memory 920 for storing a computer program.
可选地,存储器920和处理器910之间耦合。耦合是装置、单元或模块之间的间接耦合或通信连接,可以是电性,机械或其它的形式,用于装置、单元或模块之间的信息交互。可选的,存储器920与处理器910集成在一起。Optionally, memory 920 and processor 910 are coupled. Coupling is an indirect coupling or communication connection between devices, units or modules, which can be in electrical, mechanical or other forms, and is used for information interaction between devices, units or modules. Optionally, the memory 920 is integrated with the processor 910 .
其中,处理器910和存储器920均可以为一个或多个,不予限制。There can be one or more processors 910 and memories 920 without limitation.
可选的,在实际应用中,喷泉码处理装置900中可以包括收发器930,也可不包括收发器930,图中以虚线框来示意,喷泉码处理装置900可以通过收发器930和其它设备进行信息交互。收发器930可以是电路、总线、收发器或者其它任意可以用于进行信息交互的装置。Optionally, in practical applications, the fountain code processing device 900 may or may not include the transceiver 930 , as shown by a dotted box in the figure. The fountain code processing device 900 may perform processing through the transceiver 930 and other devices. Information exchange. The transceiver 930 may be a circuit, a bus, a transceiver, or any other device that may be used for information exchange.
本申请实施例中不限定上述收发器930、处理器910以及存储器920之间的具体连接介质。本申请实施例在图9中以存储器920、处理器910以及收发器930之间通过总线连接,总线在图9中以粗线表示,其它部件之间的连接方式,仅是进行示意性说明,并不引以为限。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示,图6中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。在本申请实施例中,处理器可以是通用处理器、数字信号处理器、专用集成电路、现场可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件,可以实施或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成,或者用处理器中的硬件及软件模块组合执行完成。The specific connection medium between the above-mentioned transceiver 930, processor 910 and memory 920 is not limited in the embodiment of the present application. In the embodiment of the present application, the memory 920, the processor 910 and the transceiver 930 are connected through a bus in Figure 9. The bus is represented by a thick line in Figure 9. The connection methods between other components are only schematically explained. It is not limited. The bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in Figure 6, but it does not mean that there is only one bus or one type of bus. In the embodiment of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or Execute each method, step and logical block diagram disclosed in the embodiment of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented by a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.
在本申请实施例中,存储器可以是非易失性存储器,比如硬盘(hard disk drive,HDD)或固态硬盘(solid-state drive,SSD)等,还可以是易失性存储器(volatilememory),例如随机存取存储器(random-access memory,RAM)。存储器还可以是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质,但不限于此。本申请实施例中的存储器还可以是电路或者其它任意能够实施存储功能的装置,用于存储计算机程序、程序指令和/或数据。In the embodiment of the present application, the memory may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or it may be a volatile memory (volatile memory), such as a random access memory. Access memory (random-access memory, RAM). Memory may also be, but is not limited to, any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory in the embodiment of the present application can also be a circuit or any other device capable of performing a storage function, used to store computer programs, program instructions and/or data.
基于以上实施例,参见图10,本申请实施例还提供另一种喷泉码处理装置1000,包括:接口电路1010和逻辑电路1020;接口电路1010,可以理解为输入输出接口,可用于执行上述任一方法实施例中各个设备的收发步骤;逻辑电路1020可用于运行代码或指令以执行上述任一实施例中各个设备执行的方法,不再赘述。Based on the above embodiments, referring to Figure 10, the embodiment of the present application also provides another fountain code processing device 1000, including: an interface circuit 1010 and a logic circuit 1020; the interface circuit 1010 can be understood as an input and output interface, and can be used to perform any of the above. Transmitting and receiving steps of each device in a method embodiment; the logic circuit 1020 can be used to run codes or instructions to perform the method performed by each device in any of the above embodiments, which will not be described again.
基于以上实施例,本申请实施例还提供一种计算机可读存储介质,该计算机可读存储介质存储有指令,当所述指令被执行时,使上述任一方法实施例中各个设备执行的方法被实施,例如,使得图5所示实施例中喷泉码编码装置和喷泉码解码装置执行的方法被实施。该计算机可读存储介质可以包括:U盘、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。Based on the above embodiments, embodiments of the present application also provide a computer-readable storage medium that stores instructions. When the instructions are executed, each device in any of the above method embodiments is executed. For example, the method performed by the fountain code encoding device and the fountain code decoding device in the embodiment shown in FIG. 5 is implemented. The computer-readable storage medium may include: U disk, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk and other various media that can store program codes.
基于以上实施例,本申请实施例提供一种喷泉码处理系统,该通信系统包括上述任一方法实施例中提及的喷泉码编码装置和喷泉码解码装置,可用于执行上述任一方法实施例中各个设备执行的方法。Based on the above embodiments, embodiments of the present application provide a fountain code processing system. The communication system includes the fountain code encoding device and the fountain code decoding device mentioned in any of the above method embodiments, and can be used to execute any of the above method embodiments. Methods executed by each device.
本领域内的技术人员应明白,本申请的实施例可提供为方法、系统、或计算机程序产品。因此,本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、紧凑型光盘只读储存器(compact disc read-only memory,CD-ROM)、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art will understand that embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present application may employ computer-usable storage media (including but not limited to magnetic disk storage, compact disc read-only memory (CD-ROM)) containing computer-usable program code therein. , optical storage, etc.).
本申请是参照根据本申请的方法、装置(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理装置的处理器以产生一个机器,使得通过计算机或其他可编程数据处理装置的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce a A device for realizing the functions specified in one process or multiple processes of the flowchart and/or one block or multiple blocks of the block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理装置以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction apparatus, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.
这些计算机程序指令也可装载到计算机或其他可编程图像处理装置上,使得在计算机或其他可编程装置上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程装置上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions may also be loaded onto a computer or other programmable image processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.
Claims (15)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211121539.3A CN117749317A (en) | 2022-09-15 | 2022-09-15 | A fountain code encoding and decoding method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211121539.3A CN117749317A (en) | 2022-09-15 | 2022-09-15 | A fountain code encoding and decoding method and device |
Publications (1)
Publication Number | Publication Date |
---|---|
CN117749317A true CN117749317A (en) | 2024-03-22 |
Family
ID=90251334
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211121539.3A Pending CN117749317A (en) | 2022-09-15 | 2022-09-15 | A fountain code encoding and decoding method and device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117749317A (en) |
-
2022
- 2022-09-15 CN CN202211121539.3A patent/CN117749317A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4980567B2 (en) | Multimedia server with simple adaptation to dynamic network loss conditions | |
WO2019119950A1 (en) | Video coding processing method and apparatus, and application having video coding function | |
US7095729B2 (en) | Method for multimedia communication over packet channels | |
US8233539B2 (en) | Method and apparatus for transmitting packet-based image frame | |
US9209836B2 (en) | Methods and systems for minimizing decoding delay in distributed video coding | |
US20090103635A1 (en) | System and method of unequal error protection with hybrid arq/fec for video streaming over wireless local area networks | |
CN105471545B (en) | Data packet processing method and device | |
US8621320B2 (en) | Per-image forward error correction | |
JP5706144B2 (en) | Method and apparatus for transmitting scalable video according to priority | |
JP7638011B2 (en) | MEDIUM CONTENT BASED SELF-ADAPTIVE SYSTEM CODE FEC ENCODING AND DECODING METHOD, APPARATUS, SYSTEM AND MEDIUM | |
CN110876066A (en) | Adaptive forward error correction method, apparatus, medium and device | |
CN115037416B (en) | Data forward error correction processing method, device, electronic equipment and storage medium | |
CN106488243A (en) | A kind of many description screen content method for video coding | |
CN104871546A (en) | Enhanced video streaming with application layer forward error correction | |
CN112866746A (en) | Multi-path streaming cloud game control method, device, equipment and storage medium | |
CN111385055B (en) | Data transmission method and device | |
WO2017161124A1 (en) | System for video streaming using delay-aware fountain codes | |
CN113890683B (en) | A hierarchical video transmission method based on online fountain code with unequal error protection | |
CN108667557A (en) | An Adaptive FEC Coding Matrix Design Method Based on Media Content | |
CN114900698A (en) | Forward error correction-based video transmission method, apparatus and computer storage medium | |
CN117749317A (en) | A fountain code encoding and decoding method and device | |
CN113037437A (en) | Data transmission method and device | |
CN111245566B (en) | Method and device for preventing packet loss of unreliable network, storage medium and electronic equipment | |
CN113301387B (en) | Data encoding and decoding method, related equipment and system | |
WO2015101280A1 (en) | Channel code rate allocation method and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |