JP4649615B2 - Video encoding / decoding device, video encoding / decoding method, and program thereof - Google Patents

Description

  The present invention relates to a low-delay video encoding and decoding device, and more particularly to a technique for constructing a video encoding / decoding system that achieves both low delay, high image quality, and high compression rate.

  Coexistence of high image quality and high compression rate in video encoding has already reached a high level. For example, even MPEG-2 encoding standardized more than 10 years ago, HDTV digital broadcasting has a high compression ratio of about 1/100 with slight image quality degradation that is hardly perceptible to non-experts. Is realized. Since the standardization of MPEG-2, the compression rate has been improved. H.263, MPEG-4, and recently standardized H.264. In H.264, it is said that a compression ratio more than twice that of MPEG-2 can be obtained.

  As described above, although the improvement of the compression rate is remarkably advanced, the reduction of the delay time under the condition of the constant transmission bit rate (CBR), which is important in communication and broadcasting that requires real time, Since 2 there has been no substantial advancement. As a result, video phones with low demands on image quality achieve a low delay of 100 ms or less, but sacrifice the delay time in applications that require both high image quality and a high compression rate, such as digital TV broadcasting. Inevitably, the delay required for video encoding and decoding is as long as 1 second or more even in live sports broadcasts that require real-time performance.

  Of course, MPEG-2 also considers delay reduction, and two low-delay mode frame drops and intra slices are prepared. The former frame drop is a picture in which most of the pictures (frames or fields) intra-coded pictures (hereinafter referred to as I [intra] pictures) or intra-picture coded macroblocks (intra macroblocks) can be said to be the main cause of delay. If the amount of generated code exceeds a predetermined value, some of the subsequent pictures in the encoding order are omitted (dropped) to satisfy the CBR (constant bit rate) condition. is there. When this mode is applied, two or more pictures may be dropped continuously, resulting in a video that appears to be temporarily frozen, a sense of incongruity is great, and is not suitable for demanding use of image quality.

  In contrast, intra slices do not periodically insert I pictures, which are the main cause of increased delay, but move between cascaded P pictures (inter-picture coded pictures in which motion compensation is performed only in forward prediction) ( A partial slice in a P picture (usually moved cyclically from top to bottom) (a group of specific macroblocks in the picture, in MPEG-2, typically one to several rows of horizontal strip macros) The arrangement of block macroblocks is a sequence of 1 to several vertical strips of macroblocks (called an intra column in MPEG-2) or rectangular macroblocks). This is a method (intra-slice refresh method) that realizes refresh in a distributed manner by coding all as intra macroblocks. . This method has the advantage that the delay due to refresh can be reduced without substantially degrading the image quality.

  FIG. 1 shows a picture sequence structure (PPP structure) of PPPP ... on the left side and a picture sequence structure of BBPBBP ... (picture sequence structure having a P picture period of BBP) (BBP structure) on the right side. The picture sequence in the display order shows that the intra slice moves cyclically from top to bottom. Intra-slices are performed by sequentially setting each picture constituting an image as an I picture, P picture, and B picture (inter-picture coded pictures using forward prediction, backward prediction, and bidirectional prediction in combination with motion compensation). Code having a conventional standard configuration corresponding to the IBBPBBPBBP... Structure (IBBP structure) or the IBPBPBP... Structure (IBP structure). It also has an advantage that can be easily realized in the encoding device (FIG. 2) and the decoding device (FIG. 3).

  However, this intra slice refresh method is also combined with the low delay mode of the PPP structure and has only been applied to a part of the application range of video coding such as videophone and surveillance. Although the BBP or BP structure is more advantageous than the PPP structure in terms of image quality, it is not explicitly defined as a standardization target (it can be realized within the standardization range of MPEG-2). This is due to the following problems in the coding method of the BBP structure or the BPBPBP... Structure (BP structure), and it is not only possible to obtain a sufficient delay reduction effect, but also to meet strict image quality requirements for broadcasting and storage applications. It is thought that there was not.

(1) Since the intra slice must be inserted into the P picture as much as it cannot be inserted into the intermediate B picture, the code amount of the intra slice occupying the P picture increases. Does not get smaller. For this reason, the effect of reducing the delay due to the decrease in the code amount of the P picture is small compared to the PPP structure.
(2) The sum of both the delay when the pictures are rearranged from the input order to the encoding order and the delay due to having to encode the entire picture as an I picture at the time of a scene change is large. The reduction effect is hidden.
(3) The movement between pictures at the intra-slice insertion position, which is indispensable for reliably obtaining the refresh effect, is noticeable directly or as flickering.
However, with respect to the problem (3), a method of aligning the quantization roughness of the intra slice portion and the peripheral portion based on the result of the pre-analysis (for example, refer to Patent Document 1), and the compression rate of the intra slice portion A method (for example, see Patent Document 2) that suppresses the temporary increase in the amount of generated code has been proposed.

  However, since the pre-analysis used in the former does not always work well under low delay conditions, the code generation amount may temporarily become excessive in the intra slice part, and as a result, the code generation amount constraint is observed. Therefore, the compression rate of the immediately following macroblock has to be extremely increased, and conversely, the intra slice portion can be conspicuous. In the latter case, the refreshed part is blurred from the periphery, which may be conspicuous, and is not suitable for high-quality video encoding.

  Instead of applying the intra slice to the BBP or BP structure, not only the entire refresh picture of the IBBP or IBP structure is encoded as a P picture, but also the locally restored P picture is re-encoded as an I picture and re-encoded. A method (see Patent Document 3) for reducing delay by postponing data has been proposed. However, this method has a problem that image quality deterioration that cannot be ignored is caused by re-encoding, although there is no movement of the intra slice as in the intra slice refresh method, and this does not stand out. In addition, the effect of reducing the delay caused by a scene change cannot be expected.

JP 7-95564 A JP 2005-124041 A JP 2004-147306 A

In realizing a low-delay video encoding / decoding device using the intra-slice refresh method, the above-mentioned problems (1) to (3) are solved, in other words, a BP or BBP structure having excellent image quality. It is a problem to be solved by the present invention to achieve a low delay equivalent to that of the PPP structure and to make the movement of an intra slice less noticeable.
That is, according to the present invention, in the current mainstream low-delay video encoding / decoding method in which the intra slice insertion position is moved for each picture, flicker caused by the movement of the intra slice is made inconspicuous, and a B picture is used. In this case, an object is to realize a low delay equivalent to the case where no B picture is used.

The present invention reduces the delay from encoding to decoding by the following four means.
(1) The encoding apparatus encodes the refresh target slice as an intra slice and also performs inter-picture encoding as a part of the P picture, and sends the P picture encoded data in the encoding order.
(2) In the encoding apparatus, P-picture encoded data and B-picture code are encoded in the P-picture cycle so that the intra-slice encoded data for refreshing is encoded in every P picture period so that the increase in the code amount does not cause an increase in delay. Sent during the transmission of the digitized data or after the completion of the transmission.
(3) The decoding apparatus decodes and reproduces the P picture without intra slice from the P picture encoded data sent in the encoding order, and the restored P picture (restored P picture) is critical for delay reduction. This is used as a reference image for extracting a prediction block during decoding and reproduction of a B picture (see the middle part of FIG. 4). In FIG. 4, each rectangle indicates a restored picture decoded and reproduced from encoded data generated by the encoding apparatus of the present invention. The upper part shows the restored intra slice obtained by decoding and reproducing the refresh target slice from the encoded data of the intra slice, and the middle part shows the restored picture decoded and reproduced from the inter-picture encoded data including the refresh target part. The lower row shows only the slice to be refreshed in the middle restored P picture. It shows the restoration P pictures including an intra-slice substituted with tiger slices. In this figure, arrows between picture indicates the direction of the reference toward the referenced picture from the reference original picture.).
(4) In the decoding apparatus, a restored P picture including an intra slice that can be decoded and reproduced only after receiving intra-slice encoded data that arrives late is used as a reference picture when restoring a P picture without an intra slice in order to realize a refresh function. (Refer to the arrow from the restored P picture including the lower intra slice in FIG. 4 to the restored P picture without the intra slice in the middle stage).
The effects of these four means are as follows.

In the conventional intra slice refresh method, an intra slice having a large amount of code must be sent in the coding order, which causes an increase in delay. In the coding apparatus of the present invention, the coding order according to the coding order is the same. Since the intra slice portion that must be sent is replaced with the encoded data of the P slice having a small code amount, the area of the refresh target slice increases as the number of inserted B pictures increases, and the code amount of the encoded data of the intra slice Even if increases, it does not cause an increase in delay.
In addition, even under low delay conditions, a sufficient amount of code can be provided for encoding the slice to be refreshed as an intra slice, so that encoding with higher image quality than conventional intra slices is possible.

Further, when the P picture including the refresh target slice is not referred to by the subsequent P picture in the coding order in the scene change, the intra slice encoded data of the refresh target slice that is useless is transmitted to the decoding device. , And the amount of code corresponding thereto is allocated to encoding of an intra picture that must be inserted immediately after a scene change. As a result, the image quality of an intra picture immediately after a scene change, which has been difficult to be deteriorated because a sufficient amount of code cannot be given under low delay conditions, can be greatly improved.
On the other hand, in the decoding apparatus of the present invention, it is necessary to restore the P picture without intra slice from the P picture encoded data sent in the encoding order including the refresh target slice portion, and display it first. Since it is used as a reference picture of a picture, even if arrival of intra slice encoded data is delayed due to a large code amount, display of a B picture is not delayed.

  When decoding and reproducing the subsequent P picture in the coding order following the decoding and reproduction of the B picture, reference is made so that the restored intra slice that is decoded and reproduced from the intra slice encoded data that arrives later enters the refresh target slice portion. Therefore, the reference image when decoding and reproducing the subsequent P picture is a restored P picture including an intra slice, and a refresh that is the same as the conventional intra slice refresh method is realized. Furthermore, since the P picture that is decoded and reproduced first for B picture decoding and reproduction does not include an intra slice and is not easily affected by refreshing, the movement of the intra slice is conspicuous when used for display (see the middle part of FIG. 4). This can alleviate the disadvantage of the conventional intra-slice refresh method.

According to the present invention, not only the drawback of the conventional intra-slice refresh method that the trace of the cyclic movement between pictures at the local refresh position is easily noticeable, but also the cause of the refresh under the condition of inserting the B picture. Can significantly reduce the delay. In fact, under the condition that NTSC TV video is encoded with a frame structure and a sequence structure of BBPBBP ..., a delay of about 40 milliseconds is reduced with a still image that is prominent in image quality degradation compared to a conventional intra slice. Is confirmed by simulation.
In addition, post-transmission of intra-slice encoded data can make it impossible to send most of the intra-slice encoded data when a scene change is detected. The intra-slice encoded data can be discarded without being transmitted. The discarded code amount can be turned to an I picture that cannot be inserted immediately after the scene change, and the delay at the time of the scene change can be reduced accordingly. Since the delay reduction amount corresponding to this is usually 40 ms or more, when the delay is determined due to the restriction of image quality maintenance at the time of a scene change, the delay is reduced by 40 ms or more.

  Furthermore, the ability to post-translate intra-slice encoded data means that n-pass encoding can be realized with respect to encoding of the intra-slice portion, so that the image quality of the intra-slice portion is greatly improved over the conventional intra-slice refresh method. Is also possible. Of course, in the present invention, since the inter-picture coding is also performed on the refresh target slice, extra code is generated accordingly, and as a result, the image quality is deteriorated as compared with the conventional intra slice refresh method even under the low delay coding condition. Things can happen. However, if the present invention and the conventional intra-slice refresh method are adaptively switched and used according to changes in video characteristics, this image quality degradation problem can be avoided.

As described above, the encoding device and the decoding device of the present invention can reduce the delay time to the limit without deterioration in image quality, and are low-delay encoding for a BPBP ... or BBPBBP ... picture sequence. , It can support trick play (smooth fast-forward and fast-reverse at low speed magnification) when playing stored streams, and can be applied widely from broadcasting applications to next-generation video communications including mobile phones. .
Furthermore, the reduction in code amount fluctuation realized by sending intra-slice coded data between P pictures and B pictures or after that is achieved through a communication path where bandwidth is not guaranteed, such as best effort. It is also useful as an encoding / decoding device that realizes stable video communication for a network such as the Internet.

  The present invention is implemented as a video encoding device, a video decoding device, or a video encoding / decoding device. However, in order for the invention to be effective, they must be combined with each other via a communication channel or a broadcast network. An embodiment of the present invention will be described below in detail with reference to the drawings, but the technical scope of the present invention is not limited by the following embodiment. The technical scope of the present invention extends to an equivalent range.

  FIG. 5 shows a block diagram of a first embodiment of the encoding apparatus of the present invention. In this figure, reference numeral 5 denotes a control unit for controlling other units based on a synchronization signal obtained from an input video signal. In addition, 10 is an inter-picture coding unit that performs P and B type inter-picture coding and local decoding using motion compensation (evaluates the amount of code generated by inter-picture coding and intra coding in units of macroblocks). In addition, a macroblock that is determined to have a smaller amount of code generated by intra coding has a function of performing intra coding instead of inter picture coding), and 110 is a refresh target that is inserted in a form that circulates between P pictures. An intra-slice encoding unit that performs coding as an intra-slice and local decoding thereof on a slice, 20 is a buffer memory that temporarily holds inter-picture encoded data, and 21 is intra-slice encoded data of a refresh target slice Is temporarily stored in the buffer memory, 30 is inter-picture encoded data and Header adding function multiplexer for multiplexing the slice coded data, 22 is a buffer memory for temporarily storing the generated coded data. 23 is an intra slice memory for storing an intra slice restored by local decoding, 31 is a restored picture of inter-picture encoded data from the inter-picture coding unit 10 or a restored intra slice from the intra slice memory 23 A multiplexer 80 is selected and sent to the frame memory, and 80 is a frame memory for storing P and B restored pictures.

The operation of the encoding apparatus according to the first embodiment will be described below. As shown in FIG. 5, the picture sequence to be encoded is input to the inter-picture encoding unit 10 as a video signal (a signal composed of a luminance signal, a color difference signal, a synchronization signal, etc.) developed in the raster scanning order. The
The inter-picture coding unit 10 reads (refers to) a P or B restored picture in the frame memory 80, performs motion compensation using a prediction block cut out from the reference picture, and thereby all pictures in the input picture sequence On the other hand, inter-picture encoding is performed in the order of BPBP... Or BBPBBP... And encoded data obtained thereby is output to the buffer memory 20 after adding information such as a header. At this time, the inter-picture coding is performed so as to satisfy the target value of the code generation amount for each coding unit (for example, a unit such as a macro block, a slice, and a picture) designated by the control unit 5 with the control signal. In addition, decoding / reproduction processing is performed at the same time, and the obtained restored image (locally decoded image) is stored in the frame memory 80 via the multiplexer 31 so that it can be used as a reference image for subsequent P and B picture encoding. To do. Further, the quantization step size and code generation amount are fed back to the control unit 5 for each coding unit.

  The intra slice coding unit 110 takes out a refresh target slice for moving the target part for each P picture, performs intra coding so that the control unit 5 satisfies the target value of the code generation amount specified by the control signal, Information such as a header is added to the encoded data obtained as a result and output to the buffer memory 21. Further, the intra-slice encoded data of the refresh target slice is locally decoded and reproduced (restored) and output to the intra-slice memory 23. Further, like the inter-frame coding unit 10, the quantization step size and the code generation amount for each coding unit are fed back to the control unit 5.

The restored slice in the intra slice memory 23 is stored in the frame memory 80 after the completion of the reference of the restored P picture without intra slice for decoding and reproduction of the B picture until the start of encoding of the next P picture. The corresponding P picture refresh target part is overwritten, and the restored P picture including the same intra slice as in the conventional method is decoded and reproduced in the frame memory 80. Accordingly, in the B picture coding, the restored P picture including an intra slice in the forward direction and the restored P picture without an intra slice in the backward direction are used as reference images, respectively. As in the intra slice refresh method, a restored P picture including an intra slice located in the front in the coding order is used as a reference image.
Transfer of the encoded data encoded by the inter-picture encoding unit 10 and the intra-slice encoding unit 110 to the buffer memory 22 is performed by switching the multiplexer 30 to the buffer memory 20 side every P picture cycle first. This process is performed on the inter-picture encoded data in the order of the encoded data and the B picture encoded data, and then the multiplexer 30 is switched to the buffer memory 21 side to perform the intra slice encoded data. However, if the rate of generation of inter-picture encoded data becomes low during the process and the buffer memory 22 is about to underflow, the multiplexer 30 is temporarily switched to the buffer memory 21 side, and the amount necessary to prevent underflow. The intra slice encoded data is extracted from the buffer memory 21, added with header information for identification, etc., and transferred to the buffer memory 22. From the buffer memory 22, the data is output to the outside at a predetermined bit rate.

  The control unit 5 generates an appropriate control signal so that the operation of each unit described above is performed, and also generates a quantization step size and code generation fed back from the inter-picture coding unit 10 and the intra slice coding unit 110. Based on the amount and a predetermined target encoding rate, the target code generation amount of the encoding unit in the encoding of the next P, B picture and intra slice is calculated, and the inter-picture encoding unit 10 And the intra slice encoding unit 110.

As is clear from the above description of the operation, the present invention post-sends intra-slice encoded data of a refresh target slice having a large code amount, and thus has the advantage that inter-picture encoded data can be sent earlier. . If the decoding apparatus side restores the P picture using only the inter-picture coded data that can be received earlier, and uses it as a reference picture for the subsequent B picture restoration, the intra slice coded data that arrives later Since the B picture can be restored without using any of the above, the restoration of the B picture is accelerated. As a result, the restored image can be displayed earlier, and the delay from encoding to decoding can be greatly reduced.
Since the amount of code of intra-slice encoded data is less likely to affect the delay, not only can a sufficient amount of code be provided for encoding the slice to be refreshed as an intra-slice, but also the intra-slice encoded data is encoded. Therefore, it is not necessary to transmit at the same time as encoding, so that an n-pass encoding technique can be used, and stable intra encoding with little variation in the quantization step size can be realized in an intra slice. Therefore, there is an advantage that image quality degradation can be suppressed to a minimum even under low-delay encoding conditions.

Further, even if the amount of inter-picture encoded data is extremely reduced and the output buffer memory 22 is likely to underflow in a still image or a still image with little motion, the multiplexer 30 is temporarily set. By switching to the buffer memory 21 and intra slice encoded data can be flowed, there is an advantage that useless stuffing insertion can be minimized even under low delay encoding conditions.
Furthermore, when a scene change occurs within the P picture cycle of the next stage and refresh in units of slices becomes meaningless, transfer of unnecessary intra slice encoded data that does not contribute to compression is interrupted when a scene change is detected There is also an advantage that can be canceled. For this reason, compared to the conventional intra slice refresh method in which intra-slice coded data has already been sent at the time of the scene change detection, the amount of interruption or suspension to the intra picture that must be inserted immediately after the scene change. Thus, it is possible to significantly reduce image quality deterioration immediately after a scene change.
In this embodiment, the inter-picture coding unit 10 and the intra-slice coding unit 110 are configured as separate units. However, intra-coding in units of macroblocks incorporated in the inter-picture coding unit 10 is used. If the processing capability of the local decoding function is increased and an output function to the intra slice memory is added, the function of the intra slice coding unit can be realized simultaneously. The inter-picture coding unit 10 and the intra slice coding unit 110 can also be integrated as an inter-picture / intra-slice encoding unit. In this embodiment, the frame memory 80, the buffer memories 20 to 22, and the intra slice memory 23 are configured as separate memories, but can be allocated to different areas on one memory. In this case, the multiplexer function is realized as an access area switching function. Integration of these encoding units and memories makes it possible to reduce the hardware scale of the entire encoding apparatus.

  FIG. 6 shows a configuration diagram of an encoding apparatus according to the second embodiment of the present invention. The only difference from the first embodiment is that the frame memory 80 is moved between the inter-picture coding unit and the multiplexer 31. Hereinafter, the difference in operation from the first embodiment will be described.

In the first embodiment, the restored intra slice in the intra slice memory 23 is overwritten on the refresh target part, so that a restored P picture including the intra slice is actually generated in the frame memory 80 and read out. It was. On the other hand, in the second embodiment, when the restored P picture is read from the frame memory, the multiplexer 31 is switched as necessary so that the restored intra slice of the intra slice memory 23 can be read. In addition, a restored P picture including an intra slice is read out.
Specifically, in the encoding of the B picture, the restored P picture without intra slice in the frame memory is read out as a reference picture without switching the multiplexer 31 in the forward and backward directions, while the P picture In the encoding, only the refresh target slice portion switches the multiplexer 31 to the intra slice memory side and reads the restored intra slice as a reference image. The control unit 5 switches the multiplexer 31 in accordance with the movement of the intra slice portion, and generates and supplies necessary addresses and read control signals to the frame memory 80 and the intra slice memory 23.

The second embodiment has an advantage that the transfer of the restored intra slice for overwriting the refresh slice portion from the intra slice memory 23 to the frame memory 80 becomes unnecessary. In addition, it is possible to refer to a restored P picture without intra slices in both the forward and backward directions when decoding and reproducing a B picture, which is effective for making the movement of the refresh target slice portion inconspicuous in the display image on the decoding device side. There is an advantage that can be realized without separately storing a copy of a P picture without an intra slice or performing extra transfer.
FIG. 7 shows a configuration diagram of a decoding apparatus according to the third embodiment of the present invention that receives the encoded data generated in the first and second embodiments, and decodes and reproduces the encoded video. ing. In this figure, 35 is a multiplexer, 50 is extracting and analyzing additional information such as a header from the encoded data sequence, and based on the result, the encoded data sequence is converted into an inter-picture encoded data sequence and intra-slice encoded data. A data separation unit for separating into columns, 62 is a buffer memory for buffering an input encoded data sequence, 60 is a buffer memory for receiving and buffering an inter-picture encoded data sequence separated by the data separation unit 50, and 61 is also Buffer memory that receives and buffers the intra slice encoded data separated by the data separation unit 50, 63 is an intra slice memory that stores the decoded intra slice, and 81 is a frame memory that stores the restored P picture, 200 Is buffer memory 6 An inter-picture encoded data decoding unit that reads out inter-picture encoded data from and decodes and reproduces the inter-picture encoded data, 210 reads out intra-slice encoded data from the buffer memory 61, and decodes and reproduces the intra-slice encoded unit; Reference numeral 90 denotes a video signal converting unit that converts P and B restored pictures in the frame memory 81 into video signals in raster scanning order and outputs them. Reference numeral 55 denotes a control unit for controlling each unit, and generates a control signal (including an address signal for the memory) for each unit.

The operation of the decoding apparatus according to the third embodiment will be described below from the start of decoding. The operation of this decoding apparatus is as follows: the time when the apparatus starts up normally after power-on and begins to take in the encoded data input sent before the power-on, or the encoding apparatus sent from the encoding apparatus side Start from the point when the beginning of the data arrives for the first time. In either case, the encoded data separation unit 50 extracts and analyzes additional information such as a header from the encoded data input, separates it into inter-picture encoded data and intra-slice encoded data, and stores the former in the buffer memory 60. The latter is output to the buffer memory 61. At this time, additional information that is unnecessary in the subsequent processing is removed, and encoded data in units of pictures or slices can be continuously read out from the decoding units 200 and 210.
The inter-picture decoding unit 200 starts reading from the point in time when effective inter-picture encoded data can be read in the inter-picture encoded data buffer, and the pictures necessary for decoding the encoded data (forward or backward in display order). Decoding and reproduction of inter-picture coded data is executed while referring to a restored image of a picture located behind) as necessary. The picture restored by this decoding and reproduction is temporarily stored in the restored picture storage frame memory 81, and thereafter used as a reference picture for decoding and reproducing the picture from the encoded data.

Each restored picture in the frame memory 81 is output to an external display device in the display order via the video signal conversion unit 90. The restored picture that has become unnecessary after the output for display and the reference for decoding and reproduction is overwritten with the newly created restored picture, and is deleted from the frame memory 81.
Similarly to the inter-picture encoded data decoding unit 200, the intra slice decoding unit 210 also starts reading from the point in time when effective intra-slice encoded data can be read from the buffer memory 61, and performs decoding and reproduction operations. Start. The restored intra slice decoded and reproduced in this way is stored in the intra slice memory 63. The restored intra slice in the intra slice memory 63 overwrites the refresh target slice portion of the restored P picture in the frame memory, which is the restored image to be cut out at the time of encoding. However, the overwriting timing is from the end of the reference for decoding and reproducing the B picture to the restored P picture to be overwritten until the restoration of the next P picture starts.
As a result, for decoding and reproduction of a P picture, decoding and reproduction of encoded data of an intra slice is referred to for decoding and reproducing a B picture, while maintaining the restriction of refresh implementation by the intra slice that refers to a P picture including an intra slice. Will be delayed until the end. Therefore, even if the reception of the encoded data of the intra slice is delayed, the progress of the decoding / reproduction of the B picture is not affected. Therefore, unlike the conventional method, an increase in the code amount of the intra slice does not cause a delay increase, and the encoding device side can give a sufficient code amount to the intra slice. Even under decoding conditions, refresh can be realized with minimal image quality degradation.

In the present embodiment, the inter-picture encoded data decoding unit 200 and the intra-slice decoding unit 210 are configured as separate units, but the macroblock incorporated in the inter-picture encoded data decoding unit 200 If the processing capability of the intra decoding function in units is increased and the output function to the intra slice memory is added, the function of the intra slice decoding unit can be realized at the same time, and the inter-picture decoding unit 200 and the intra slice decoding are realized. Unit 210 may be integrated as an inter-picture encoded data / intra slice decoding unit.
In this embodiment, the frame memory 81, the buffer memories 60 to 62, and the intra slice memory 63 are configured as separate memories, but can be allocated to different areas on one memory. In this case, the multiplexer function is realized as an access area switching function. By integrating these decoding units and memories, the hardware scale of the entire encoding apparatus can be reduced.

FIG. 8 shows a configuration diagram of a decoding apparatus according to the fourth embodiment of the present invention. The difference from the third embodiment is that when the frame memory 81 moves between the inter-picture decoding unit and the multiplexer 35 and refers to the area of the slice to be refreshed, the reference source memory is stored in the frame memory 81 or the intra-frame. It is only a point that can be selected by any one of the slice memories 63. The difference in operation from the third embodiment will be described below.
In the third embodiment, a restored P picture including an intra slice is actually configured in the frame memory 81 by overwriting the restored intra slice in the intra slice memory 63 to the refresh target part, and then the reference is made. I was trying to do it. On the other hand, in the present embodiment, when referring to the P picture, the restored intra slice of the intra slice memory 63 can be referred to by switching the multiplexer 35 as means for selecting the reference source as necessary. Not only a restored P picture without an intra slice but also a restored P picture including an intra slice can be referred to.
Specifically, for the B picture, the restored P picture without intra slice is cut out from the frame memory 81 as a reference image while the multiplexer 35 is tilted forward and backward to the frame memory 81 side, and decoding reproduction is performed. The result is stored in the frame memory 81.

On the other hand, the P picture is decoded and reproduced by referring to the restored P picture including the intra slice as follows, and stored in the frame memory 81 without the intra slice. For reference to the restored P picture including the intra slice, the multiplexer 35 is switched so that only the refresh target slice portion is selected on the intra slice memory 63 side and the other portion is selected on the frame memory 81 side. By doing so, the reference to the refresh target slice area is cut out from the restored intra slice, and the reference to other areas is cut out from the restored P picture without the intra slice. The restored pictures of P and B are output to an external display device via the video signal conversion unit 90 in the display order as in the third embodiment.
The control unit 55 generates and supplies a switching control signal for the multiplexer 35 and addresses and read control signals necessary for the frame memory 81 and the intra slice memory 63 in conjunction with the movement of the intra slice portion.

  The fourth embodiment has an advantage that the transfer of the restored intra slice for overwriting the refresh slice portion from the intra slice memory 63 to the frame memory 81 becomes unnecessary. In addition, a reference picture of a B picture that is effective for making the movement of the refresh target slice part inconspicuous in the display image is used as a restored P picture without an intra slice in both the forward and backward directions. And there is an advantage that can be realized without extra transfer. Further, unlike the third embodiment, since the restored intra slice is not overwritten on the restored P picture in the frame memory 81, the restored intra slice is overwritten to output the restored P picture without the intra slice for display. There is an advantage that the restriction that must be done before is eliminated.

As described in the above embodiments, on the encoding device side, in addition to performing inter-picture encoding by motion compensation based on forward prediction for the P picture as a whole, and transmitting encoded data in the encoding order, The refresh target slice portion whose insertion position is moved for each P picture is also encoded as an intra slice. Then, in the transmission of the intra slice encoded data, in the encoding for each P picture period, the interval between the transmission of the P picture encoded data and the B picture encoded data so that the increase in the code amount does not cause an increase in delay. Alternatively, it is performed after sending them.
On the other hand, the decoding device side decodes and reproduces a P picture without an intra slice from P picture encoded data sent in the encoding order, and delays the restored P picture (restored P picture without an intra slice). This is used as a reference image for extracting a prediction block when decoding and reproducing a B picture that is critical for reduction. A restored P picture including an intra slice that cannot be decoded and reproduced without intra-slice encoded data that arrives late is used as a reference image when restoring a P picture without an intra slice in order to realize a refresh function.

  In the above description, as shown in FIGS. 5 to 8, the method of realizing the video encoding device and the video decoding device of the present invention in hardware by combining individual units or memories with each other has been shown. However, recent advances in microprocessor technology have made significant progress, enabling individual units to be implemented as software running on the microprocessor. The memory can be similarly mounted on the main memory of the microprocessor. In other words, part or all of the present invention can be implemented as computer software. In this case, each unit is realized as a subroutine, a function, or an object.

  The present invention is not limited to MPEG-2 and MPEG-4 encoding schemes as long as it is an encoding scheme in which a refresh target slice is provided in a P picture. 261-H. The present invention can be applied to various encoding methods such as the H.263 encoding method. As a B picture, a bi-predicted inter-picture coded picture that generates a prediction block from an interpolated image of a block cut out from two restored images positioned forward or backward instead of conventional bi-directional prediction is used. The present invention can be similarly applied to an encoding method such as H.264. Further, as described in the first and second embodiments, the restored P picture used as a reference image for extracting a motion compensation prediction block at the time of B picture encoding is, for example, an intra slice for the forward direction. A backward P picture displayed later in time may be a restored P picture without an intra slice, or both may be restored P pictures without an intra slice. In addition, it is not always necessary to provide a refresh target slice for every P picture, and there may be a P picture without a refresh target slice.

It is the figure which showed a mode that an intra slice was cyclically inserted in the picture sequence of a PPP structure and a BBP structure. It is a block diagram of the video encoding apparatus of the conventional standard structure. It is a block diagram of the video decoding apparatus of the conventional standard structure. It is a figure for demonstrating the means to solve a subject. It is a block diagram of the video coding apparatus of 1st Example of this invention. It is a block diagram of the video coding apparatus of 2nd Example of this invention. It is a block diagram of the video coding apparatus of the 3rd Example of this invention. It is a block diagram of the video decoding apparatus of the 4th Example of this invention.

Explanation of symbols

5,55 Control unit 10 Inter picture coding unit 20, 21, 22, 60, 61, 62 Buffer memory 23, 63 Intra slice memory 30, 31 Multiplexer 50 Data separation unit 80, 81 Frame memory 90 Video signal unit 110 Intra Slice encoding unit 200 Inter-picture encoded data decoding unit 210 Intra slice decoding unit

Claims (6)

In a video encoding apparatus that realizes periodic refresh by moving the position in a picture of a refresh target slice provided in a P picture between cascaded P pictures,
Means for inter-coding the entire picture as a P picture without an intra slice and generating P picture encoded data regardless of whether or not a refresh target slice is provided in the P picture at the time of P picture encoding;
Means for intra-coding the portion of the refresh target slice as an intra slice and generating intra slice encoded data when providing the refresh target slice in the P picture;
At the time of encoding a B picture, at least one reference image among a plurality of reference images from which a motion compensation prediction block is extracted is inter-picture encoded using a restored P picture without intra-slice that is obtained by decoding the P picture encoded data. Means for generating B picture encoded data;
The inter-picture encoded P-picture encoded data and B-picture encoded data are transmitted in the encoding order, and the intra-slice encoded data of the refresh target slice is converted into the P-picture encoded data and the B-picture encoded A video encoding apparatus comprising: means for transmitting data during transmission or after completion of the transmission. In a video decoding apparatus that decodes data encoded by a video encoding method that realizes periodic refresh by moving the position in a picture of a refresh target slice provided in a P picture between cascaded P pictures,
P picture encoded data obtained by encoding the entire picture as a P picture without an intra slice, and a restored P picture without an intra slice are used for at least one reference picture among a plurality of reference pictures for extracting a motion compensation prediction block. Means for receiving encoded B picture data and intra slice encoded data obtained by encoding the refresh target slice portion as an intra slice when a refresh target slice is provided in the P picture; ,
Means for decoding the received P picture encoded data and generating and storing a restored P picture without intra slice;
Using at least one reference picture among a plurality of reference pictures from which the predicted block for motion compensation is extracted from the received B picture coded data, using the restored P picture without intra slice obtained by decoding the P picture coded data Means for decoding and generating a restored B picture;
Means for decoding the received intra slice encoded data and restoring the intra slice;
Means for replacing the refresh target slice portion of the restored P picture without the intra slice with the restored intra slice;
And means for making a reference image from which a motion compensation prediction block is cut out when decoding subsequent P picture encoded data with the restored P picture in which the refresh target slice portion is replaced with the intra slice. Video decoding device. In a video encoding method for realizing periodic refresh by moving a position in a picture of a refresh target slice provided in a P picture between cascaded P pictures,
A process of inter-picture encoding the entire picture as a P picture without an intra slice and generating P picture encoded data regardless of whether or not a refresh target slice is provided in the P picture at the time of P picture encoding;
When providing a refresh target slice in the P picture, intra-encoding the portion of the refresh target slice as an intra slice, and generating intra slice encoded data;
At the time of encoding a B picture, at least one reference image among a plurality of reference images from which a motion compensation prediction block is extracted is inter-picture encoded using a restored P picture without intra-slice that is obtained by decoding the P picture encoded data. Generating B-picture encoded data;
The inter-picture encoded P-picture encoded data and B-picture encoded data are transmitted in the encoding order, and the intra-slice encoded data of the refresh target slice is converted into the P-picture encoded data and the B-picture encoded A video encoding method characterized by having a process of transmitting data during transmission or after completion of the transmission. In a video decoding method for decoding data encoded by a video encoding method that realizes periodic refresh by moving a position in a picture of a refresh target slice provided in a P picture between cascaded P pictures,
P picture encoded data obtained by encoding the entire picture as a P picture without an intra slice, and a restored P picture without an intra slice are used for at least one reference picture among a plurality of reference pictures for extracting a motion compensation prediction block. A process of receiving encoded B picture data and intra slice encoded data obtained by encoding the refresh target slice portion as an intra slice when the refresh target slice is provided in the P picture; ,
Decoding the received P picture encoded data to generate and store a restored P picture without intra slice;
Using at least one reference picture among a plurality of reference pictures from which the predicted block for motion compensation is extracted from the received B picture coded data, using the restored P picture without intra slice obtained by decoding the P picture coded data Decoding and generating a restored B picture;
Decoding the received intra slice encoded data and restoring the intra slice;
A step of replacing the refresh target slice part of the restored P picture without the intra slice with the restored intra slice;
And a process of using a restored P picture in which the refresh target slice portion is replaced with the intra slice as a reference image from which a motion compensation prediction block is cut out when decoding subsequent P picture encoded data. Video decoding method. A video encoding program for causing a computer to execute video encoding for realizing periodic refresh by moving a position in a picture of a refresh target slice provided in a P picture between cascaded P pictures,
The computer,
Means for inter-coding the entire picture as a P picture without an intra slice and generating P picture encoded data regardless of whether or not a refresh target slice is provided in the P picture at the time of P picture encoding;
Means for intra-coding the portion of the refresh target slice as an intra slice and generating intra slice encoded data when providing the refresh target slice in the P picture;
At the time of encoding a B picture, at least one reference image among a plurality of reference images from which a motion compensation prediction block is extracted is inter-picture encoded using a restored P picture without intra-slice that is obtained by decoding the P picture encoded data. Means for generating B picture encoded data;
The inter-picture encoded P-picture encoded data and B-picture encoded data are transmitted in the encoding order, and the intra-slice encoded data of the refresh target slice is converted into the P-picture encoded data and the B-picture encoded As a means for sending data between sending data or after sending them,
Video encoding program to make it function. By moving the position in the picture of the refresh target slice provided in the P picture between cascaded P pictures, video decoding for decoding data encoded by the video encoding method that realizes periodic refresh is performed on the computer. A video decoding program for executing,
The computer,
P picture encoded data obtained by encoding the entire picture as a P picture without an intra slice, and a restored P picture without an intra slice are used for at least one reference picture among a plurality of reference pictures for extracting a motion compensation prediction block. Means for receiving encoded B picture data and intra slice encoded data obtained by encoding the refresh target slice portion as an intra slice when a refresh target slice is provided in the P picture; ,
Means for decoding the received P picture encoded data and generating and storing a restored P picture without intra slice;
Using at least one reference picture among a plurality of reference pictures from which the predicted block for motion compensation is extracted from the received B picture coded data, using the restored P picture without intra slice obtained by decoding the P picture coded data Means for decoding and generating a restored B picture;
Means for decoding the received intra slice encoded data and restoring the intra slice;
Means for replacing the refresh target slice portion of the restored P picture without the intra slice with the restored intra slice;
As means for making a reference image from which a motion compensation prediction block is cut out when decoding subsequent P picture encoded data with a restored P picture in which the refresh target slice portion is replaced with the intra slice,
Video decoding program to make it function.

Images