JP6348562B2 - Decoding device and decoding method - Google Patents

Description

  The present invention relates to a video encoding device for encoding video, a video encoding program, a video encoding method, a video decoding device for decoding encoded video, a decoding program, a decoding method, The present invention relates to a medium on which an encoded video encoded stream is recorded and a method for transmitting encoded video data.

  As a method for recording and transmitting large volume video and audio information as compressed digital data, an encoding method such as the MPEG (Moving Picture Experts Group) method has been established, and the MPEG-1 standard and the MPEG-2 standard. It is an international standard encoding method such as MPEG-4 standard. In addition, as a method to further improve the compression rate, Joint Video Team (JVT) of ISO / IEC MPEG & ITU-T VCEG: “Text of International Standard of Joint Video Specification”, ITU-T Rec. H.264 | ISO / H. described in IEC 14496-10 Advanced Video Coding, (December, 2003). The H.264 / AVC (Advanced Video Coding) standard is defined. These systems are adopted as encoding systems in digital satellite broadcasting, DVDs, mobile phones, digital cameras, and the like, and the range of use is now expanding and becoming familiar.

  Of these encoding schemes, H.264. In the H.264 / AVC format, in addition to the techniques of motion compensation prediction and arithmetic transform coding using Discrete Cosine Transform (DCT), which have been used in MPEG and the like, Intra-coded prediction coding and The so-called technology is adopted. This is a technique for predicting a signal level from a coded adjacent block in an intra frame without using information of other pictures.

  FIG. 3 is an explanatory diagram of intra prediction encoding. Here, as an example, a case will be described in which prediction is performed in block units of 4 × 4 pixel size in a raster scan method in which scanning is sequentially performed from the upper left to the lower right of the screen. The signal level of the pixel 304 of the prediction block 303 (block to be predicted) is the boundary pixel 302 of the adjacent block 301 that has already been encoded (pixels adjacent to the prediction block among the pixels of the adjacent block 301. However, the prediction block For the adjacent block on the upper right side of (1), it is predicted from the signal level of the pixel in the lowermost row), and the difference (residual component) between the predicted value and the actual value is subjected to arithmetic transform coding by DCT or the like. At this time, the pixel 305 of the adjacent block which is not yet encoded cannot be used for prediction. There are a plurality of prediction directions (that is, encoding modes) 306, prediction is performed for all directions, the residual components are compared, the direction with the smallest residual component is selected, and the residual component is the arithmetic transform code. It becomes. Here, the direction “0: DC” is a mode in which an average value of signal levels of all boundary pixels is a predicted value.

  Even when prediction is performed in units of blocks each having a size of 16 × 16 pixels, the prediction direction is limited, but intra prediction encoding can be performed in the same manner.

  As described above, in intra prediction encoding, pixels of adjacent blocks that are not yet encoded cannot be used. For this reason, a difference occurs in prediction accuracy depending on the prediction direction 306. For example, when a block is scanned by a raster scan method that sequentially scans from the upper left to the lower right of the screen, the prediction accuracy from the upper left to the lower right direction increases, but the prediction accuracy from the upper right to the lower left direction decreases. In other words, it is easy to predict an image in which pixels with the same signal level continue from the upper left to the lower right (lower-right image), so the compression rate can be increased, but the signal level is equal from the upper right to the lower left. Since it is difficult to predict an image with a continuous pixel (upwardly rising image), the compression rate is low.

  Further, in the conventional intra prediction coding, since prediction is performed using only boundary pixels, there is a problem that continuous changes in the signal level inside the block cannot be predicted with sufficient accuracy. Usually, the signal level of the pixel changes gently. However, in the conventional intra prediction coding, prediction is performed by extending the boundary pixel at the block boundary (that is, the pixel in the predicted block is predicted to be the same as the boundary pixel). For this reason, as the pixel is farther from the boundary pixel, the residual component becomes larger and the amount of code after arithmetic conversion increases.

  The present invention has been made in view of the above problems, and before performing intra prediction, the input image is inverted in the vertical direction or the horizontal direction, or rotated to an arbitrary angle to thereby change the scan direction. It is an object of the present invention to provide a video encoding device capable of always performing intra prediction in a direction with high prediction accuracy without change, and a video decoding device corresponding to the video encoding device.

  In addition, the present invention provides a video encoding device that realizes intra prediction with high accuracy by a predetermined interpolation formula using not only the boundary pixels but also the pixels inside the block at the time of intra prediction, and corresponding to this. An object is to provide a video decoding device.

  According to the present invention, in a decoding device that decodes a signal of an encoded image, a determination unit that determines whether or not a direction of the image has been converted, and a decoding unit that decodes the image There is provided an image conversion unit that performs conversion on the decoded image to restore the determined conversion.

  Therefore, according to the present invention, it is possible to provide a video decoding apparatus corresponding to a video encoding apparatus in which the compression rate is increased by always performing intra prediction in a direction with high prediction accuracy without changing the scan direction.

  Embodiments of the present invention and advantages of the present invention will be described in detail below with reference to the accompanying drawings.

It is a block diagram explaining the hardware constitutions of the video coding apparatus of the 1st Embodiment of this invention. It is a block diagram explaining the function of the video coding apparatus of the 1st real form of this invention. It is explanatory drawing of intra prediction encoding. It is explanatory drawing of the procedure of the video encoding of the 1st Embodiment of this invention. It is a block diagram explaining the hardware constitutions of the video decoding apparatus of the 2nd Embodiment of this invention. It is a block diagram explaining the function of the video decoding apparatus of the 2nd Embodiment of this invention. It is explanatory drawing of the procedure of the video decoding of the 2nd Embodiment of this invention. It is explanatory drawing of the intra prediction method of the 3rd Embodiment of this invention. It is a block diagram explaining the function of the intra estimation part to which the intra prediction method of the 3rd Embodiment of this invention is applied. It is explanatory drawing of the procedure of the intra prediction of the 3rd Embodiment of this invention. It is explanatory drawing of the data recording medium of the 4th Embodiment of this invention. It is explanatory drawing of the packet of the 5th Embodiment of this invention.

  FIG. 1 is a block diagram illustrating a hardware configuration of a video encoding apparatus according to the first embodiment of this invention.

  The video encoding apparatus 101 according to the first embodiment includes a processor 102, a memory 103, an input interface (input I / F) 104, and an output interface (output I / F) 106 that are connected so as to communicate with each other. The input I / F 104 is connected to the input device 105. The output I / F 106 is connected to the output device 107.

  The processor 102 is a processor that performs video encoding processing according to the present invention, executes a program stored in the memory 103, encodes data received from the input I / F 104, and sends the data to the output I / F 106.

  The memory 103 stores a program executed by the processor 102. In addition, data to be processed by the processor 102 is temporarily stored.

  The video encoding device 101 may be provided with a plurality of processors 102 and memories 103. For example, a dedicated processor that executes only a part of the program for performing the video encoding process of the present invention may be provided. A plurality of dedicated processors that perform the same processing may be provided.

  The processor 102 and memory 103 may be implemented on a single chip.

  The input I / F 104 is an interface that receives video data processed by the processor 102 from the input device 105.

  The input device 105 is a device that inputs a video signal processed by the video encoding device 101 to the input I / F 104, and is, for example, a video camera or a TV tuner. In this case, the input I / F 104 is, for example, a video capture card. The input device 105 may be a storage device that stores unencoded video data. In this case, the input I / F 104 is, for example, a SCSI interface.

  The output I / F 106 is an interface that transmits data encoded by the processor 102 to the output device 107.

  The output device 107 is an output destination of the data encoded by the video encoding device 101, and is, for example, a storage device that stores the encoded data. In this case, the output I / F 106 is, for example, a SCSI interface. The output device 107 may be a computer device connected to the output I / F 106 via a LAN, an IP network, or the like (not shown). In this case, the output I / F 106 is a network interface. The output device 107 may be an image receiving device connected to the output I / F 106 via a telephone communication network (not shown). In this case, the output I / F 106 is a telephone signal transmitter. The output device 107 may be a digital TV receiver. In this case, the output I / F 106 is a digital TV signal transmitter.

  The video encoding device 101 may include a plurality of input I / Fs 104 and output I / Fs 106, and different types of input devices and output devices may be connected to the respective input I / Fs 104 and output I / Fs 106. For example, the video encoding device 101 may include two output I / Fs 106, one of which is connected to a hard disk device and the other is connected to a magneto-optical disk device. Further, a hard disk device may be connected to one side, and a computer device may be connected to the other side via a LAN or the like.

  FIG. 2 is a block diagram illustrating functions of the video encoding device 101 according to the first embodiment of this invention.

  The video encoding apparatus 101 includes an original image memory 201, an image conversion unit 202, an encoding unit 203, a mode control unit 213, and a mode selection unit 214. Here, the original image memory 201 is a partial area of the memory 103, and the image conversion unit 202, the mode control unit 213, and the mode selection unit 214 are programs stored in the memory 103 and executed by the processor 102. . In addition, the encoding unit 203 is a program executed by the processor 102, such as a motion prediction unit 204, an intra prediction unit 205, an arithmetic conversion unit 206, a quantization unit 207, a coefficient encoding unit 208, an inverse quantization unit 209, and an inverse quantization unit 209. It comprises an arithmetic conversion unit 210 and a decoded image memory 211 and a predicted image memory 212 which are partial areas of the memory 103.

  Next, functions of each unit of the video encoding device 101 will be described along a procedure for encoding an image.

  The original image memory 201 temporarily buffers an original image to be encoded. Next, the image conversion unit 202 performs conversion processing on all or part of the frame of the image acquired from the original image memory 201. Here, the part of the frame may be, for example, a macro block or a block obtained by dividing the frame into a predetermined size, or may be a predetermined rectangular area. In general, encoding is performed on a macroblock basis.

  The image conversion unit 202 may divide the data obtained by performing the conversion process on the entire frame into units of macro blocks and transmit the data to the encoding unit 203. Alternatively, the image conversion unit 202 may perform the conversion process after dividing the frame into units of macro blocks. You may go. Information on whether or not conversion processing has been performed is transmitted to the mode control unit 213.

  The conversion process performed by the image conversion unit 202 is a process of converting the direction of the image. The process of changing the direction of the image is, for example, a line-symmetrical transformation or a point-symmetrical transformation such as a symmetrical transformation that inverts the frame left and right, a vertically symmetrical transformation that inverts the frame up and down, and a rotational transformation that rotates the frame. is there. Hereinafter, as an example, a case will be described in which the case where the entire image is subjected to left-right symmetric conversion is compared with the case where no conversion is performed, and the smaller code amount is selected.

  The encoding unit 203 acquires from the image conversion unit 202 an image that has undergone left-right symmetry conversion and an image that has not been converted, and sequentially encodes each image. Also, a plurality of encoding units 203 may be provided, and two encoding units 203 may encode each image in parallel. When a plurality of encoding units 203 are provided, the video encoding apparatus 101 is provided with a plurality of dedicated processors 102 that execute only the program of the encoding unit 203.

  The motion prediction unit 204 performs inter-frame prediction on the image acquired from the image conversion unit 202 using the images in the predicted image memory 212 and the decoded image memory 211, and transmits information such as a motion vector to the mode control unit 213. Then, the residual component of the encoded block obtained by the prediction is transmitted to the arithmetic conversion unit 206.

  The intra prediction unit 205 performs intra prediction on the image acquired from the image conversion unit 202 using the image in the decoded image memory 211, transmits mode information and the like to the mode control unit 213, and is obtained by prediction. The residual component of the encoded block is transmitted to the arithmetic conversion unit 206.

  The arithmetic conversion unit 206, the quantization unit 207, and the coefficient encoding unit 208 are the same as those of the conventional encoding device, and perform DCT operation, quantization of conversion coefficients, conversion of coefficients to codes, and the like, respectively. . Also, the inverse quantization unit 209 and the inverse arithmetic conversion unit 210 are the same as those of the conventional encoding device, and respectively return the encoded data to the image information by inversely transforming the encoded data, and the decoded image memory 211 and the prediction Store in the image memory 212.

  The mode control unit 213 manages the encoding mode of the entire image (frame) and the macroblock being encoded. With respect to the entire image, information on an encoding process for an image that has undergone conversion processing and an encoding process for an image that has not been subjected to conversion processing are held. That is, information (motion vector, reference frame information, etc.), intra coding mode (intra prediction direction) information, etc., which is a reference for motion prediction, for each of the cases where conversion processing is performed and not performed. Hold. Regarding the macro block, information about whether the current macro block is intra-encoded or inter-frame predictively encoded, and intra-encoding mode, motion vector, reference frame information and the like related thereto are held. The mode control unit 213 transmits these pieces of information to the mode selection unit 214.

  The mode selection unit 214 configures and outputs the encoded data of the image from the encoded data regarding the entire image and the macroblock being encoded and the information of the encoding mode. The entire code amount when encoded by performing the conversion process by the image conversion unit 202 is compared with the code amount when encoded without performing the conversion process, and the smaller code is output as a stream. In addition, a flag indicating whether or not conversion processing has been performed and, if a partial area of the image is converted, position information of the area is added to the stream as data.

  The flag indicating whether or not the conversion process has been performed may be one bit each for the left-right symmetric conversion and the vertical symmetric conversion. In the case of rotation conversion, if the number is incremented by 1 every 90 degrees clockwise, 360 degrees can be expressed with 2 bits.

  FIG. 4 is an explanatory diagram of a video encoding procedure according to the first embodiment of this invention.

  First, the image conversion unit 202 determines the encoding method (step 401). That is, it is determined whether conversion processing is performed on the entire frame or conversion is performed in units of macroblocks. Hereinafter, as an example, a case where the entire frame is converted will be described. The process that performs the conversion process proceeds to step 402, and the process that does not perform the conversion process proceeds to step 404.

  In step 402, the image conversion unit 202 converts the input image. That is, for each frame, left-right symmetry conversion, vertical symmetry conversion, rotation conversion, and the like are performed. Then, it progresses to step 403.

  In step 403 and step 404, the encoding unit 203 encodes the image. This encoding is as described in FIG.

  Next, the mode selection unit 214 performs code amount comparison and mode determination (step 405). That is, as described with reference to FIG. 2, the code amount obtained by encoding is compared between the process that performs the conversion process and the process that does not perform the conversion process, and determines that data with a small code amount is output.

  Next, the mode selection unit 214 outputs a code and a flag (step 406). As described in FIG. 2, according to the determination in step 405, a flag indicating whether or not conversion processing has been performed, the position information of the region when a partial region of the image is converted, and the selected code Output the digitized data as a stream. Thus, the encoding process ends.

  FIG. 5 is a block diagram illustrating the hardware configuration of the video decoding apparatus according to the second embodiment of the present invention.

  The video decoding apparatus 501 according to the second embodiment includes a processor 502, a memory 503, an input interface (input I / F) 504, and an output interface (output I / F) 506 that are connected so as to communicate with each other. The input I / F 504 is connected to the input device 505. The output I / F 506 is connected to the output device 507.

  The processor 502 is a processor that performs video decoding processing according to the present invention, executes a program stored in the memory 503, decodes data received from the input I / F 504, and sends the data to the output I / F 506.

  The memory 503 stores a program executed by the processor 502. In addition, data to be processed by the processor 502 is temporarily stored.

  The video decoding apparatus 501 may be provided with a plurality of processors 502 and memories 503. For example, a dedicated processor that executes only a part of the program that performs the video decoding process of the present invention may be provided.

  The processor 502 and memory 503 may be implemented on a single chip.

  The input I / F 504 is an interface that receives encoded data to be processed by the processor 102 from the input device 505.

  The input device 505 is a device that inputs encoded data to be processed by the video decoding device 501 to the input I / F 504. For example, the input device 505 is a storage device that stores the encoded data. In this case, the input I / F 504 is, for example, a SCSI interface. The input device 505 may be a computer device connected to the input I / F 504 via a LAN, an IP network, or the like (not shown). In this case, the input I / F 504 is a network interface. The input device 505 may be a video data transmission device connected to the input I / F 504 via a telephone communication network (not shown). In this case, the input I / F 504 is a telephone signal receiver. The input device 505 may be a digital TV broadcast station. In this case, the input I / F 504 is a digital TV tuner.

  The output I / F 506 is an interface that transmits data decoded by the processor 502 to the output device 507.

  The output device 507 is an output destination of the data decoded by the video decoding device 501 and is, for example, a display that outputs video. In this case, the output I / F 506 is, for example, a video card. The output device 507 is a storage device that records the decoded data as a stream. In this case, the output I / F 506 is, for example, a SCSI interface.

  The video decoding device 501 may include a plurality of input I / Fs 504 and output I / Fs 506, and different types of input devices and output devices may be connected to the respective input I / Fs 504 and output I / Fs 506. For example, the video encoding device 101 may include two input I / Fs 504, one of which is connected to a hard disk device and the other is connected to an optical disk device. Further, a hard disk device may be connected to one side, and a computer device may be connected to the other side via a LAN or the like.

  FIG. 6 is a block diagram illustrating functions of the video decoding apparatus 501 according to the second embodiment of this invention.

  The video decoding apparatus 501 includes a stream analysis unit 601, a mode determination unit 602, a decoding unit 603, an image conversion unit 610, and a decoded image memory 611. Here, the stream analysis unit 601, the mode determination unit 602, the decoding unit 603, and the image conversion unit 610 are programs stored in the memory 503 and executed by the processor 502, and the decoded image memory 611 is stored in the memory 503. Part area. Also, the decoding unit 603 is a program executed by the processor 502, a motion prediction unit 604, an intra prediction unit 605, a coefficient analysis unit 606, an inverse quantization unit 607, an inverse arithmetic operation unit 608, and one of the memories 503. A prediction image memory 609 which is a region of a part.

  The decoding apparatus 501 according to the second embodiment can decode the stream encoded by the video encoding apparatus 101 according to the first embodiment. Next, the function of each unit of the video decoding apparatus 501 will be described along a procedure for decoding an encoded stream.

  The stream analysis unit 601 analyzes input encoded stream data, and transmits flag and data information to the mode determination unit 602. The stream analysis unit 601 analyzes the stream data and flag created by the encoding apparatus 101.

  Next, the mode determination unit 602 controls modes related to motion prediction, intra prediction, and image conversion based on the information analyzed by the stream analysis unit 601. When a stream is flagged indicating that conversion processing such as inversion and rotation has been performed on the image at the time of encoding, information on the type of conversion processing performed (for example, symmetrical conversion) And the like) is transmitted to the image conversion unit 610.

  The motion prediction unit 604 performs inter-frame prediction using information such as motion vectors transmitted from the mode determination unit 602 and images in the prediction image memory 609 and the decoded image memory 611, and transmits the prediction information to the coefficient analysis unit 606. To do.

  The intra prediction unit 605 performs intra prediction using information such as the intra coding mode transmitted from the mode determination unit 602 and the image in the decoded image memory 611, and transmits the prediction information to the coefficient analysis unit 606.

  The coefficient analysis unit 606, the inverse quantization unit 607, and the inverse arithmetic conversion unit 608 are the same as those of the conventional decoding device, and each of them combines prediction information and coefficient information, inverse quantization of transform coefficients, DCT calculation, etc. I do.

  The image conversion unit 610 performs conversion processing on the whole or part of the decoded image frame in accordance with the information transmitted from the mode determination unit 602. That is, a process for returning the conversion process performed by the image conversion unit 202 of the encoding apparatus 101 in FIGS. 1 and 2 is performed.

  The decoded image memory 611 stores the decoded image after the conversion processing by the image conversion unit 610, transmits the decoded image to the output device 504, and displays the decoded image on a screen or outputs it to a stream.

  FIG. 7 is an explanatory diagram of a video decoding procedure according to the second embodiment of this invention.

  First, the stream analysis unit 601 and the mode determination unit 602 perform stream and flag analysis (step 701). Next, the decoding unit 603 decodes the image (step 702). Next, the image conversion unit 610 converts the decoded image and stores it in the decoded image memory 611 (step 703). Finally, the image stored in the decoded image memory 611 is output for display or stream recording (step 704). This completes the decoding process. Since the contents of the processing of each of the above parts are as described in FIG. 6, detailed description thereof will be omitted.

  The video encoding apparatus 101 according to the first embodiment of the present invention and the video decoding apparatus 501 according to the second embodiment described above may be implemented as the same hardware. In this case, each program described in FIGS. 2 and 6 is stored in the memory 103 (or the memory 503), and each area described in FIGS. 2 and 6 is secured.

  According to the first and second embodiments of the present invention described above, the amount of code after encoding is reduced by always performing intra prediction in a direction with high prediction accuracy without changing the scan direction (that is, It is possible to provide a video encoding device (with a high compression rate) and a video decoding device corresponding thereto. As a result, compared with the conventional intra prediction, the code amount is reduced by about 10% at the maximum. However, since the code amount changes according to the contents of the original image, it is not always a constant reduction amount.

  FIG. 8 is an explanatory diagram of the intra prediction method according to the third embodiment of this invention.

  The present embodiment is a method of using not only the signal level of the boundary pixel but also the signal level of the entire pixel of the adjacent block 801 that has already been encoded when predicting the signal level of the pixel of the prediction block 802. It is used in the intra prediction units 205 and 605 in FIGS. Here, “prediction block” refers to a block that is to be encoded and has not been encoded yet, and “adjacent block” refers to an encoded block that is adjacent to the prediction block.

  The direction of prediction is as indicated by 306 in FIG. In this embodiment, for the sake of simplicity, only predictions for some directions are described, but in actual prediction, prediction is performed for all prediction directions 306, and the residual component is the smallest direction. Is selected. Here, prediction refers to a difference (residual) between an actual signal level of each pixel of the prediction block 802 and a prediction value calculated from a signal level of a pixel of an adjacent block used for prediction in encoding. This is a procedure for encoding (component). On the other hand, in decoding, it means a procedure for obtaining a decoded image by adding a decoded residual component to a predicted value calculated in the same manner. The encoding procedure will be described below.

  The conventional intra prediction method is a prediction method using only boundary pixels. For example, when predicting in the vertical direction, the prediction is performed in the downward direction using the boundary pixel (the pixel in the lowest row) of the adjacent block 801 directly above the prediction block 802. That is, the prediction block 802 is divided into four columns, and the signal level value of the boundary pixel in contact with the column including the pixel is subtracted from the signal level value of each pixel. Similarly, when predicting in the horizontal direction, the prediction is performed in the right direction using the boundary pixel (the pixel in the rightmost column) of the adjacent block on the left side of the prediction block. That is, the prediction block 802 is divided into four rows, and the signal level value of the boundary pixel in contact with the left of the row including the pixel is subtracted from the signal level value of each pixel. That is, in any of the above predictions, the pixel level of the prediction block is predicted to be the same as the signal level of the boundary pixel, and the difference (residual component) between the predicted value and the actual value is calculated. calculate.

  On the other hand, the intra prediction method according to the third embodiment of the present invention is a prediction method based on composite pixels that uses not only boundary pixels but also pixels inside adjacent blocks. Here, the composite pixel refers to a plurality of blocks used for predicting the pixel of the prediction block. For example, when predicting in the horizontal direction, it is one row (803) of an adjacent block adjacent to the left side of the prediction target row. In this case, a prediction value is calculated from the value of the composite pixel 803 by an interpolation formula, and the value of the pixel in one row of the adjacent prediction block is predicted. For this prediction, for example, a Newton forward interpolation formula is used. For example, the number n of the composite pixel of the adjacent block and the prediction target pixel of the prediction block is set to 0 to 7, the signal level of the pixel n is set to yn, and n = 0, 1, 2, 3 (encoded) to n = The case of obtaining 4, 5, 6, 7 (unencoded) will be described. At this time, the signal level predicted value yn of the pixel n is calculated by Expression (1).

  The residual component is obtained by subtracting the predicted value calculated by Equation (1) from the pixel value of the predicted block. For example, when n = 7, the residual component of the pixel 7 is a value obtained by subtracting y7 from the actual value of the pixel 7. Similarly, when predicting in an oblique direction, the number n (n = 0 to 7) is assigned to the pixel of the prediction block to be predicted and the pixel of the adjacent block that is the basis of prediction, and the equation ( A predicted value is calculated according to 1) (805). When the number of pixels to be predicted is less than 4, calculation is performed with the range of n limited. For example, when the number of pixels to be predicted is 3, n = 0 to 6 is set (806).

  Equation (1) is an example of a mathematical formula that is applied when the block size is 4 × 4 pixels, but the prediction method according to the present embodiment is for other block sizes (for example, 16 × 16 pixels). Can also be applied. In general, when predicting the signal level values of the pixels m + 1 to n from the known signal level values of the pixels 0 to m for the pixels numbered from 0 to n, the predicted value yn is expressed by the equation (2). ). Here, nCj is a binomial coefficient.

  Next, the residual by the conventional prediction method is compared with the residual by the prediction method of the present embodiment. The signal level 808 by the conventional prediction method is the same as the signal level of the pixel 3 (boundary pixel). On the other hand, the signal level 809 according to the prediction method of the present embodiment is a value calculated by Expression (1).

  For example, when the signal levels of the pixels 0 to 3 change at a constant rate, the actual signal levels 810 of the pixels 4 to 7 often change at a rate close thereto. That is, the actual signal level 810 is often closer to the signal level 809 according to the prediction method of the present embodiment than the signal level 808 according to the conventional prediction method. As a result, the residual 812 according to the prediction method of the present embodiment is smaller than the residual 811 according to the conventional prediction method, and the code amount is reduced. In Expressions (1) and (2), the signal levels of all known pixels are used for prediction, but prediction can also be performed using the signal levels of some pixels.

  Note that a least square method may be applied to this embodiment. In this case, a linear expression that is estimated by a least square method from a known signal level is used. For example, similarly to the above, based on the known signal level yn of the pixel n when n = 0, 1, 2, 3, the signal level yn of the pixel n when n = 4, 5, 6, 7 is obtained. If so, equation (3) is used. Here, m is the number of pixels serving as a basis for prediction, and in this example, m = 4.

  FIG. 9 is a block diagram illustrating functions of the intra prediction unit 205 to which the intra prediction method according to the third embodiment of the present invention is applied.

  The intra prediction unit 205 according to the present embodiment uses the intra block coding mode of the prediction block (that is, the prediction direction 306 in FIG. 3 above) based on the signal level information 907 of the adjacent block that has already been encoded and the current image. ) Is performed, prediction processing is performed, and information 906 on the encoding mode and residual component is transmitted to the mode control unit 213 and the arithmetic conversion unit 206.

  The intra prediction unit 205 includes an intra prediction control unit 901 and a plurality of prediction units corresponding to the respective encoding modes. The plurality of prediction units are classified into those that perform the prediction using the conventional boundary pixels and those that perform the prediction using the composite pixel of the present embodiment, each of which is shown in the prediction direction 306 of FIG. Includes those that make predictions in each direction.

  For the sake of simplicity, FIG. 9 illustrates a vertical boundary pixel mode prediction unit 902 that predicts in the vertical direction (“0: Vertical” in FIG. 2) using boundary pixels, and a horizontal direction (FIG. 2, “1: Horizontal”), a horizontal boundary pixel mode prediction unit 903 that predicts in a vertical direction using a composite pixel, a horizontal composite pixel mode prediction unit 904 that predicts in a vertical direction using a composite pixel, and a horizontal prediction using a composite pixel. Only the directional composite pixel mode prediction unit 905 is shown, but actually, a prediction unit that uses boundary pixels and a prediction unit that uses composite pixels are provided for each of the other prediction directions 306 in FIG.

  The intra prediction control unit 901 manages an intra prediction method. That is, it checks whether a block adjacent to the prediction block is usable, and controls information for intra prediction.

  Each mode prediction unit 902 to 905 predicts the signal level of the pixel of the prediction block from the signal level of the pixel of the adjacent block, and calculates a residual component from the signal level of the actual pixel. Next, the mode information with the smallest calculated residual component is transmitted to the mode control unit 213, and the residual component calculated in that mode is transmitted to the arithmetic conversion unit 206.

  The vertical boundary pixel mode prediction unit 902 predicts downward using the boundary pixel of the adjacent block above the prediction block. The horizontal boundary pixel mode prediction unit 903 performs prediction in the right direction using the boundary pixel of the adjacent block on the left side of the prediction block.

  The vertical direction composite pixel mode prediction unit 904 and the horizontal direction composite pixel mode prediction unit 905 perform intra prediction using the composite pixel of the present embodiment. The vertical composite pixel mode prediction unit 904 uses a composite pixel (that is, a boundary pixel and an internal pixel) of an adjacent block on the upper side of the prediction block, and predicts a signal level prediction value using Equation (1) or Equation (2). And a residual component with the actual signal level is obtained. The horizontal composite pixel mode prediction unit 905 calculates a prediction value of the signal level according to the formula (1) or the formula (2) using the composite pixel of the adjacent block on the left side of the prediction block, and the residual of the actual signal level. Find the difference component.

  FIG. 10 is an explanatory diagram of an intra prediction procedure according to the third embodiment of this invention.

  First, the intra prediction control unit 901 checks the intra prediction mode (step 1001). That is, it is determined whether or not the pixels of the adjacent block can be used, and the pixels to be used for prediction are specified in the applicable prediction direction 306 and each prediction direction 306 based on the determination result.

  Next, the mode prediction units 902 to 905 perform intra prediction for the applicable prediction direction 306. In FIG. 10, as an example, prediction using a conventional boundary pixel and prediction using a composite pixel of the present embodiment are performed for each of the vertical direction and the horizontal direction (steps 1002 to 1005).

  Next, an optimal intra prediction mode is selected from the prediction results of steps 1002 to 1005 (step 1006). As a result of the prediction in steps 1002 to 1005, a total value for each mode of the residual component for each pixel is calculated. Since the code amount after encoding can be reduced as the mode having a smaller total value, the mode having the smallest total value of residual components is selected as the optimum mode. Next, the selected residual component is transmitted to the arithmetic conversion unit 206, and information on the selected mode is transmitted to the mode control unit 213. In order to make a more accurate determination, the mode selection unit 214 may select a mode having the smallest total value of residual components after the arithmetic conversion by the arithmetic conversion unit 206 as the optimum mode. This completes the intra prediction process.

  The mode selection unit 214 adds a flag indicating the selected optimum mode to the output code stream.

  In FIG. 10, only the prediction in the vertical direction and the horizontal direction is described for the sake of simplicity, but actually, prediction is performed for all prediction directions determined by the intra prediction control unit 901 to be applicable, The optimum mode can be selected from the result.

  9 and 10 describe the video encoding apparatus 101, the intra prediction method of the present embodiment can be similarly applied to the video decoding apparatus 501. In this case, the intra prediction unit 605 includes a composite pixel mode prediction unit for each prediction direction 306 in addition to the conventional boundary pixel mode prediction unit, and performs intra prediction according to the prediction mode of the input encoded data. Thus, the video decoding apparatus 501 corresponding to the intra prediction method of the present embodiment can be realized.

  That is, the mode determination unit 602 refers to the flag attached to the stream of the image to be decoded, and determines the intra prediction mode selected when the image is encoded. Next, the intra prediction unit 605 performs prediction according to the determined intra prediction mode, and decodes the image. For example, if the image to be decoded is encoded in the boundary pixel mode in the right direction, the signal level of the pixel of the prediction block is predicted to be the same as the boundary pixel of the adjacent block on the left side of the prediction block. And decrypt. Further, for example, when the image to be decoded is encoded in the composite pixel mode in the right direction, the signal level of the pixel of the prediction block is expressed in the composite pixel of the adjacent block on the left side of the prediction block ( 1) is applied to calculate a prediction value and decode it.

  According to the third embodiment of the present invention described above, the amount of code after encoding is reduced by performing highly accurate intra prediction using composite pixels (pixels in a block) (that is, the compression rate). (High) video encoding device and video decoding device corresponding to this can be provided.

  FIG. 11 is an explanatory diagram of a data recording medium according to the fourth embodiment of this invention.

  The data recording medium 1101 is a recording medium when the output device 107 or the input device 505 is a storage device, and is, for example, a magnetic disk. The encoded data created by the video encoding apparatus 101 according to the first embodiment of the present invention is recorded as a data string 1102 on the data recording medium 1101. The data string 1102 is recorded as an encoded stream according to a certain grammar. Hereinafter, H.C. An example of the H.264 / AVC standard will be described.

  H. In H.264 / AVC, a stream includes a sequence parameter set 1103, a picture parameter set 1104, and slices 1105, 1106, and 1107. Hereinafter, a case where one image (frame) is stored in one slice will be described as an example.

  In a stream in which conversion processing such as left-right symmetric conversion, up-down symmetric conversion, or rotation conversion is performed on the entire frame, a slice header 1108 is recorded at the head of the slice 1105, and the slice header 1108 includes a conversion process. Information 1109 such as a flag to be stored is stored. As contents of this information, a flag indicating whether or not conversion processing has been performed and, when a partial area of the image is converted, position information of the partial area and the like are stored. A flag indicating whether or not conversion processing has been performed may be one bit each for left-right symmetric conversion and up-down symmetric conversion. In the case of rotation conversion, if the number is incremented by 1 every 90 degrees clockwise, 360 degrees can be expressed with 2 bits. The position information of the area subjected to the conversion process is information such as an abscissa, ordinate, width, and height in the image. Further, when the conversion process is performed for each macroblock, these pieces of information can be stored in a portion where a macroblock flag is recorded instead of the slice header.

  Further, when the intra prediction method according to the third embodiment of the present invention is applied, A new mode using composite pixels is added to the H.264 / AVC standard encoded stream. In this case, a numerical value indicating a new mode is added to information 1109 such as a conventional flag and stored. As a new mode expression format, a bit indicating that prediction is performed using composite pixels may be added, or a numerical value obtained when mode prediction is performed using composite pixels may be assigned to each prediction direction. Good.

  FIG. 12 is an explanatory diagram of a packet according to the fifth embodiment of this invention.

  FIG. 12 is a diagram illustrating, as an example, a procedure in which the output I / F 106 of the video encoding device 101 generates an IP packet to be transmitted to the IP network from the data string 1102 in FIG.

  First, the data string 1102 is divided into a predetermined size, TCP headers 1204 to 1206 are attached, and TCP segments 1201 to 1203 are generated. FIG. 12 shows an example in which one segment is generated from one slice. At this time, the slice header 1108 included in each slice is also included in the segment. As described in FIG. 11, information 1109 such as a flag related to the conversion process is stored in the slice header 1108. Although not described in FIG. 12, segments are similarly generated for slices 1107 and later.

  Next, each segment is divided into a predetermined size, an IP header is added, and an IP packet is generated. For example, the segment 1202 is divided into a predetermined size, IP headers 1209 and 1210 are added, and IP packets 1207 and 1208 are generated. Here, only IP packets 1207 and 1208 generated from a part of the segment 1202 are shown, but an IP packet is generated in the same manner for the entire segment 1202. Although not described in FIG. 12, the other segments 1201 and the like are divided in the same manner to generate an IP packet.

  The IP packet 1209 and the like are generated by, for example, the output I / F 106 of the video encoding device 101 and transmitted to the output device 107 via an IP network (not shown).

  Further, the IP packet 1209 and the like include a storage device in which the data string 1102 is stored and an output interface that generates and transmits the IP packet 1209 and the like from the data string 1102 and has no video encoding function ( (Not shown) may be transmitted.

  Similarly to FIG. 12, the video encoding device 101 or the packet transmission device may generate a wireless packet from the data string 1102 and transmit the wireless packet to the output device 107 via the wireless packet communication network.

  In addition, this invention is H.264. The present invention is not limited to H.264 / AVC and can be applied to video encoding devices and video decoding devices based on various standards.

  Typical examples of the viewpoint of the present invention not described in the claims include the following.

(1) In an encoding device that encodes an image signal,
An image conversion unit (202) for converting the direction of the image;
One or a plurality of encoding units (203) that respectively encode the pre-conversion image whose direction has not been converted and the post-conversion image whose direction has been converted;
The code amount of the encoded pre-conversion image and the code amount of the encoded post-conversion image are compared, a code with a small code amount is selected, the selected code, and the selected code An encoding device comprising: a mode selection unit (214) that outputs an identifier indicating a code type.

(2) The image conversion unit (202) performs a vertical symmetric mode for inverting the image up and down, a symmetric mode for inverting the image left and right, and / or a rotation mode for rotating the image.
The encoding unit (203) encodes the pre-conversion image and the post-conversion image subjected to the conversion in each mode,
The mode selection unit (214) compares the code amount of the encoded pre-conversion image with the code amount of the encoded post-conversion image of each mode, and selects a code with a small code amount. The encoding apparatus according to (1), wherein the selected code and an identifier indicating a type of the selected code are output.

(3) In a decoding device for decoding a signal of an encoded image,
A mode determination unit (602) for determining whether the direction of the image has been converted;
A decoding unit (603) for decoding the image;
An image conversion unit (610) that performs conversion for returning the determined conversion to the original image with respect to the decoded image.

  (4) The mode determination unit (602) is any one of a vertical symmetric mode for inverting the image up and down, a symmetric mode for inverting the image left and right, and / or a rotation mode for rotating the image. (3). The decoding device according to (3).

(5) In a program for causing a processor (102) to execute processing for encoding an image signal,
A first procedure for converting the direction of the image;
A second procedure for encoding a pre-conversion image that has not been transformed;
A third procedure for encoding the converted image having the direction converted;
The code amount encoded in the second procedure is compared with the code amount encoded in the third procedure, a code with a small code amount is selected, and the selected code and the selected code are selected. And a fourth procedure for outputting an identifier indicating the type of the code.

(6) In the first procedure, a vertical symmetry mode for flipping the image up and down, a left-right symmetry mode for flipping the image left and right, and / or a rotation mode for rotating the image are converted.
The third procedure encodes the converted image that has been converted in each mode,
The fourth procedure compares the code amount of the encoded pre-conversion image with the code amount of the encoded image after conversion in each of the modes, selects a code with a small code amount, and The program according to (5), wherein the selected code and an identifier indicating the type of the selected code are output.

(7) In a program for causing a processor (502) to execute a process of decoding a signal of an encoded image,
A first procedure for determining whether the direction of the image has been converted;
A second procedure for decoding the image;
And a third procedure for performing a conversion to restore the determined conversion to the decoded image.

  (8) The first procedure is performed by any one of a vertical symmetry mode for flipping the image up and down, a left-right symmetry mode for flipping the image left and right, and / or a rotation mode conversion for rotating the image. (7) The program according to (7), wherein it is determined whether or not it has been broken.

(9) A method of encoding an image signal,
A first procedure for converting the direction of the image;
A second procedure for encoding a pre-conversion image that has not been transformed;
A third procedure for encoding the converted image having the direction converted;
The code amount encoded in the second procedure is compared with the code amount encoded in the third procedure, a code with a small code amount is selected, and the selected code and the selected code are selected. And a fourth procedure for outputting an identifier indicating the type of the code.

(10) The first procedure performs a conversion of a vertically symmetric mode for flipping the image up and down, a left-right symmetric mode for flipping the image left and right, and / or a rotation mode for rotating the image.
The third procedure encodes the converted image that has been converted in each mode,
The fourth procedure compares the code amount of the encoded pre-conversion image with the code amount of the encoded image after conversion in each of the modes, selects a code with a small code amount, and The method according to (9), wherein the selected code and an identifier indicating the type of the selected code are output.

(11) A method for decoding a signal of an encoded image,
A first procedure for determining whether the direction of the image has been converted;
A second procedure for decoding the image;
And a third procedure for performing a transformation on the decoded image to reverse the determined transformation.

  (12) The first procedure is performed by any one of a vertical symmetry mode in which the image is inverted up and down, a left-right symmetry mode in which the image is inverted left and right, and / or a rotation mode in which the image is rotated. (11) The method according to (11), wherein it is determined whether or not it has been broken.

(13) In a recording medium on which an encoded image is recorded,
Indicates that the image has been converted into a vertically symmetric mode that flips the image up and down, a left-right symmetric mode that flips the image left and right, and / or a rotation mode that rotates the image when the image is encoded. An identifier is recorded together with the encoded image.

(14) A transmission method for transmitting an encoded image,
The encoded image and a vertically symmetric mode for inverting the image up and down when encoding the image, a symmetric mode for inverting the image left and right, and / or a rotation mode for rotating the image. A transmission method comprising: transmitting a packet including an identifier indicating that any of the conversions has been performed.

(15) In an encoding device for encoding an image signal,
A first difference between information on the first pixel in the prediction region of the image and a prediction value calculated by applying a forward interpolation formula to information on a plurality of second pixels in the prediction direction from the first pixel And an encoding unit (203) for encoding the first difference of the prediction region;
The code amount of the first difference in the prediction area calculated for a plurality of prediction directions is compared, the code with the smallest code amount is selected, and the selected code and the selected code are either And a mode selection unit (214) that outputs an identifier indicating whether the difference is the first difference calculated for the prediction direction.

(16) The encoding unit (203) includes a second difference between information on a first pixel in the prediction region of the image and information on one third pixel in the prediction direction from the first pixel. And encoding the second difference of the prediction region,
The mode selection unit (214) compares the code amounts of the first and second differences of the prediction regions calculated for a plurality of prediction directions, selects a code with the smallest code amount, and selects the selection And the identifier indicating which of the prediction directions the first code or the second difference calculated for which prediction direction is selected is output. Encoding device.

(17) In a decoding device for decoding a signal of an encoded image,
A mode determination unit (602) for determining in which prediction direction the signal of the encoded image is predicted;
A decoding unit (603) that calculates a prediction value by applying a forward interpolation formula to information of a plurality of pixels in the prediction direction, and decodes the encoded first difference. Decoding device to perform.

(18) The mode determination unit (602) may determine that the encoded image signal has a first difference based on a prediction value calculated by applying the forward interpolation formula to information of a plurality of pixels. It is determined whether it is encoded or the second difference based on information of one pixel is encoded,
When the encoded image signal is obtained by encoding the first difference, the decoding unit (603) applies the forward interpolation formula to the information of the plurality of pixels in the prediction direction. Apply to calculate a prediction value, decode the first encoded difference, and if the encoded image signal is the second difference encoded, the prediction direction The decoding apparatus according to (17), wherein prediction based on information of the one pixel is performed, and the encoded second difference is decoded.

(19) In a program for causing a processor (102) to execute processing for encoding an image signal,
A first difference between information on the first pixel in the prediction region of the image and a prediction value calculated by applying a forward interpolation formula to information on a plurality of second pixels in the prediction direction from the first pixel And a first procedure for encoding the first difference of the prediction region;
The code amount of the first difference in the prediction area calculated for a plurality of prediction directions is compared, the code with the smallest code amount is selected, and the selected code and the selected code are either And a second procedure for outputting an identifier indicating whether the difference is the first difference calculated for the prediction direction.

(20) The first procedure calculates a second difference between information on a first pixel in the prediction region of the image and information on one third pixel in the prediction direction from the first pixel. And encoding the second difference of the prediction region,
The second procedure compares the code amounts of the first and second differences of the prediction regions calculated for a plurality of the prediction directions, selects a code with the smallest code amount, and selects the selected The program according to (19), characterized by outputting a code and an identifier indicating which of the prediction directions the selected code is the first or second difference calculated for.

(21) In a program for causing a processor (502) to execute a process of decoding a signal of an encoded image,
A first procedure for determining in which prediction direction the signal of the encoded image is predicted;
And a second procedure for calculating a prediction value by applying a forward interpolation formula to information on a plurality of pixels in the prediction direction and decoding the encoded first difference. .

(22) In the first procedure, the signal of the encoded image is encoded with a first difference based on a prediction value calculated by applying the forward interpolation formula to information of a plurality of pixels. Whether the second difference based on the information of one pixel is encoded or not,
The second procedure applies the forward interpolation formula to the information of the plurality of pixels in the prediction direction when the signal of the encoded image is the one in which the first difference is encoded. A prediction value is calculated, the encoded first difference is decoded, and when the encoded image signal is obtained by encoding the second difference, the prediction direction is The program according to (21), wherein prediction based on information of one pixel is performed, and the encoded second difference is decoded.

(23) A method of encoding an image signal,
A first difference between information on the first pixel in the prediction region of the image and a prediction value calculated by applying a forward interpolation formula to information on a plurality of second pixels in the prediction direction from the first pixel And a first procedure for encoding the first difference of the prediction region;
The code amount of the first difference in the prediction area calculated for a plurality of prediction directions is compared, the code with the smallest code amount is selected, and the selected code and the selected code are either And a second procedure for outputting an identifier indicating whether the difference is the first difference calculated for the prediction direction.

(24) The first procedure calculates a second difference between information on a first pixel in the prediction region of the image and information on one third pixel in the prediction direction from the first pixel. And encoding the second difference of the prediction region,
The second procedure compares the code amounts of the first and second differences of the prediction regions calculated for a plurality of the prediction directions, selects a code with the smallest code amount, and selects the selected The method according to (23), characterized by outputting a code and an identifier indicating which of the prediction directions the selected code is the first or second difference calculated for.

(25) A method for decoding a signal of an encoded image, comprising:
A first procedure for determining in which prediction direction the signal of the encoded image is predicted;
And a second procedure of calculating a prediction value by applying a forward interpolation formula to information of a plurality of pixels in the prediction direction, and decoding the encoded first difference. .

(26) In the first procedure, the signal of the encoded image is encoded with a first difference based on a predicted value calculated by applying the forward interpolation formula to information of a plurality of pixels. Whether the second difference based on the information of one pixel is encoded or not,
The second procedure applies the forward interpolation formula to the information of the plurality of pixels in the prediction direction when the signal of the encoded image is the one in which the first difference is encoded. A prediction value is calculated, the encoded first difference is decoded, and when the encoded image signal is obtained by encoding the second difference, the prediction direction is The method according to (25), wherein prediction is performed based on information of one pixel, and the encoded second difference is decoded.

(27) In a recording medium on which an encoded image is recorded,
When encoding the image, the first difference based on the information of one pixel is encoded, or prediction is made by applying a forward interpolation formula to the information of a plurality of pixels and the information of the plurality of pixels A recording medium, wherein a value is calculated, and an identifier indicating whether or not the second difference based on the predicted value is encoded is recorded together with the encoded image.

(28) A transmission method for transmitting an encoded image,
The encoded image and the first difference based on the information of one pixel when the image is encoded are encoded, or the information of a plurality of pixels and the information of the plurality of pixels A transmission method characterized in that a predicted value is calculated by applying a forward interpolation formula, and a packet including an identifier indicating whether or not the second difference based on the predicted value is encoded is transmitted.

(29) In a decoding device for decoding a signal of an encoded image,
A mode determination unit (602) for determining whether or not the direction of the image has been converted based on an identifier sent together with the image;
A decoding unit (603) for decoding the image;
An image conversion unit (610) that performs conversion for returning the determined conversion to the original image with respect to the decoded image.

  Therefore, according to the present invention, it is possible to provide a video encoding device in which the compression rate is increased by always performing intra prediction in a direction with high prediction accuracy without changing the scan direction, and a video decoding device corresponding thereto. Can do.

  Further, according to the present invention, it is possible to provide a video encoding device that increases the compression rate by performing intra prediction with high accuracy using pixels in the block, and a video decoding device corresponding to this.

  The present invention can be used for recording and transmission of image data, and contributes to a reduction in recording capacity and an increase in transmission speed by improving the compression rate and reducing the amount of data. For example, the present invention can be used for a video recorder and a video player using a hard disk or a DVD. In addition, the present invention can be used for an image distribution service using a wired or wireless communication network including a mobile phone and a television broadcast. Further, the present invention can be used for a video phone, a video conference system, and the like.

Claims (2)

In a decoding device for decoding a signal of an intra-coded image,
The first information sent together with the signal of the image indicates whether a partial area of the image has been converted in a vertically symmetric mode that is inverted in the up-down direction or converted in a left-right symmetric mode that is inverted in the left-right direction. A determination unit for determining;
A decoded image generation unit that performs a conversion process corresponding to the conversion determined by the determination unit after the intra prediction process and generates a decoded image;
The first information includes information regarding the position of the inverted region,
The first information designates an area larger than a block unit as an inverted area ,
The intra prediction process performs intra prediction using the prediction direction indicated in the second information sent together with the signal of the image, but in the inverted region designated by the first information. A decoding device in which the prediction direction indicated by the second information in the intra prediction process is not reversed even in the intra prediction process . In a decoding method for decoding an intra-coded image signal,
The first information sent together with the signal of the image indicates whether a partial area of the image has been converted in a vertically symmetric mode that is inverted in the up-down direction or converted in a left-right symmetric mode that is inverted in the left-right direction. A determination step for determining;
A decoded image generating step for performing a conversion process corresponding to the conversion determined in the determination step after the intra prediction process and generating a decoded image, and
The first information includes information regarding the position of the inverted region,
The first information designates an area larger than a block unit as an inverted area ,
The intra prediction process performs intra prediction using the prediction direction indicated in the second information sent together with the signal of the image, but in the inverted region designated by the first information. A decoding method in which the prediction direction indicated by the second information in the intra prediction process is not reversed even in the intra prediction process .

Images