JP6480790B2 - Image determination apparatus, encoding apparatus, and program - Google Patents

Description

  The present invention relates to an image determination apparatus, an encoding apparatus, and a program for determining whether or not each block into which an image is divided is screen content.

  As a video encoding method, ITU-T H.264 is used. H.264 / MPEG-4 AVC and ITU-T H.264. H.265 / MPEG-H HEVC is known, and in these encoding methods, a frame image is divided into blocks, orthogonal transform processing is performed for each block, each orthogonal transform coefficient is quantized, and entropy coding is performed. Apply. For blocks that are encoded without using inter prediction (inter-frame prediction, inter-frame prediction), a predicted image is generated from the pixel values of the encoded adjacent blocks, and the difference from the predicted image is encoded. Intra prediction (intra-screen prediction, intra-frame prediction) is employed (see, for example, Non-Patent Document 1). In intra prediction, prediction is performed by selecting an optimal one of preset prediction directions, and information on a prediction mode indicating the selected prediction direction is transmitted.

  By the way, in general moving images for broadcast use, the ratio of natural video is high, and screen contents such as artificial CG and character information (telop) are often used in an added manner. When screen content is used for all or part of an original image, it is known that encoding efficiency can be improved by changing the encoding processing content of a corresponding block (for example, Non-Patent Document 2). reference). For example, in screen content, since the prediction error signal includes many high-frequency edges, orthogonal transform processing is not suitable. Therefore, there is a technique in which orthogonal transform processing is skipped in the 4 × 4 transform region and pixel values are directly quantized. It is known (see, for example, Patent Document 1).

  Further, it is known that, when screen content is encoded, encoding efficiency can be improved by skipping the deblocking filter processing of the color difference signal (see, for example, Non-Patent Document 3).

International Publication No. 2013/187060

Supervised by Satoshi Okubo, “Impress Standard Textbook Series H.265 / HEVC Textbook”, Impress Japan Co., Ltd., October 21, 2013 Rajan Joshi, “JCTVC-R1014 Screen content coding test model 2 (SCM 2)”, ITU, October 16, 2014 Ohji Nakagami, “JCTVC-S0044 Chroma deblocking filter control for SCC”, ITU, October 7, 2014

  However, the conventional encoding technique has a problem that it does not visually determine whether or not screen content is included in the original image. Therefore, when trying all combinations of processing modes for improving the encoding efficiency, it takes a long time for encoding.

  An object of the present invention made in view of such circumstances is to provide an image determination device, an encoding device, and a program capable of determining whether or not screen content is included in an original image.

  In order to solve the above-described problems, an image determination apparatus according to the present invention is an image determination apparatus that determines whether or not each block obtained by dividing an image is screen content, and has an intra prediction mode for luminance signals and color difference signals. Both are DC predictions, a block in which some pixel values of the luminance signal and the DC prediction value of the luminance signal match, and a part of the pixel value of the color difference signal and the DC prediction value of the color difference signal match is screen content. When a pixel value comparison unit that is determined as a candidate and blocks determined as screen content candidates by the pixel value comparison unit are adjacent to each other, it is determined that the plurality of adjacent blocks are screen content, and the determination result is shown. And a screen content determination unit that outputs determination information.

  Furthermore, in the image determination device according to the present invention, the screen content determination unit is adjacent to each other when blocks determined as screen content candidates by the pixel value comparison unit are adjacent to each other and the DC predicted values are also equal. A plurality of blocks are determined to be screen contents.

  Furthermore, in the image determination device according to the present invention, the pixel value comparison unit sets the partial pixel values as pixel values at four corners of the block.

  Furthermore, in the image determination apparatus according to the present invention, it is determined that one or more blocks surrounded by the block determined to be screen content by the screen content determination unit are all screen content, and the determination information is corrected. It further comprises a determination correction unit.

  In order to solve the above problems, an encoding apparatus according to the present invention includes the image determination apparatus, and changes the encoding process according to the determination information.

  In order to solve the above problems, a program according to the present invention causes a computer to function as the image determination device.

  In order to solve the above problems, a program according to the present invention causes a computer to function as the encoding device.

  According to the image determination apparatus of the present invention, it is possible to determine whether or not screen content such as artificial CG is included in a part of an original image. Also, according to the encoding apparatus of the present invention, the encoding image quality can be improved without decreasing the encoding speed by changing the encoding processing mode depending on whether or not the content is screen content. .

It is a block diagram which shows the structural example of the image determination apparatus which concerns on the 1st Embodiment of this invention. The process of the screen content determination part in the image determination apparatus which concerns on the 1st Embodiment of this invention is demonstrated. It is a flowchart which shows the operation example of the image determination apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the image determination apparatus which concerns on the 2nd Embodiment of this invention. The process of the determination correction part in the image determination apparatus which concerns on the 2nd Embodiment of this invention is demonstrated. It is a block diagram which shows the structural example of the encoding apparatus which concerns on the 3rd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  Most screen content is designed to draw a color frame such as a square or round shape on the background of the character to make it easier to see. In many cases, gradation and the like are not added so that it is difficult to see the characters to be added, so that the luminance signal and the color difference signal in the color frame portion are constant, which makes it difficult to appear in a natural image. In the present invention, in consideration of this feature, it is considered that there is a high possibility that all pixel values are equal in a block in which some pixel values are equal, such as the four corners. In addition, the prediction mode of intra prediction of blocks having the same all pixel values (hereinafter referred to as “intra prediction mode”) is DC prediction, and similar blocks continuously occupy a certain area. From these conditions, it is determined whether or not the screen content is combined.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of an image determination apparatus according to the first embodiment of the present invention. In the example illustrated in FIG. 1, the image determination device 1 includes a prediction mode determination unit 11, a pixel value measurement unit 12, a pixel value comparison unit 13, and a screen content determination unit 14.

  The image determination apparatus 1 is incorporated in an encoding apparatus that performs intra prediction, and inputs an original image on which intra prediction is performed and intra prediction information. Then, it is determined whether or not the original image on which intra prediction is performed is screen content in units of blocks obtained by dividing the original image, and determination information indicating the determination result is output to the outside. The encoding apparatus decomposes the original image into a luminance signal and a color difference signal, and performs intra prediction using the luminance signal and the color difference signal, respectively.

  The intra prediction information includes information on the prediction mode selected in the intra prediction. The intra prediction mode is, for example, ITU-T H.264. In H.264 / MPEG-4 AVC, nine types of prediction in eight directions and DC prediction are defined. In addition, ITU-T H.I. In H.265 / MPEG-H HEVC, 35 types of prediction in 33 directions, DC prediction, and average approximate prediction are defined. When the prediction mode is DC prediction, all pixels of the target block are predicted by the average value (DC prediction value) of the encoded pixels adjacent to the target block, but the intra prediction information also includes the DC prediction value. . The intra prediction information also includes block position information in order to be able to associate which block the prediction mode is the prediction mode.

  The prediction mode determination unit 11 inputs intra prediction information, and when the intra prediction mode of both the luminance signal and the color difference signal of the block is DC prediction, the DC determination information indicating that the prediction mode is DC prediction is a pixel. In addition to outputting to the value measuring unit 12, the DC predicted values of the luminance signal and the color difference signal are output to the pixel value comparing unit 13.

  The pixel value measurement unit 12 inputs the luminance signal and color difference signal of the original image and the DC determination information determined by the prediction mode determination unit 11, the intra prediction mode of the luminance signal is DC prediction, and the color difference signal For a block whose intra prediction mode is DC prediction, a part of pixel values of the luminance signal of the block and a part of pixel values of the color difference signal of the block are acquired and output to the pixel value comparison unit 13.

  Here, the partial pixel values acquired by the pixel value measuring unit 12 are arbitrary pixel values of the block, but it is particularly preferable to set the pixel values at the four corners of the block. The reason for this is that when a very gentle gradation portion (for example, blue sky) is determined to be DC prediction at the time of intra prediction, the pixel value comparison unit 13 causes the pixel at the four corners that are farthest away from the DC prediction value in the block. This is because the difference becomes remarkable by comparing the values, and the gradation image and the screen content can be identified.

  The pixel value comparison unit 13 compares the DC predicted value of the luminance signal input from the prediction mode determination unit 11 with the pixel value of the luminance signal input from the pixel value measurement unit 12 and inputs from the prediction mode determination unit 11. The DC predicted value of the chrominance signal to be compared is compared with the pixel value of the chrominance signal input from the pixel value measuring unit 12. Then, a block in which the DC prediction value and the pixel value of the luminance signal match and the DC prediction value and the pixel value of the color difference signal match is determined as a screen content candidate. Then, candidate information indicating whether each block is a screen content candidate is output to the screen content determination unit 14. In the present embodiment, this candidate information is a 1-bit flag (candidate flag), the value when it is determined as a screen content candidate is 1, and the value when it is not determined as a screen content candidate is 0.

  In addition, the pixel value comparison unit 13 outputs the DC prediction value input from the prediction mode determination unit 11 to the screen content determination unit 14. The predicted DC value may be output from the prediction mode determination unit 11 to the screen content determination unit 14.

  The screen content determination unit 14 acquires block position information from the intra prediction information in order to identify adjacent blocks. In the first determination method, when the blocks having the candidate flag of 1 are adjacent to each other in the luminance signal and color difference signal of each block, the plurality of adjacent blocks are determined to be screen contents, and the determination result indicating the determination result Output information. That is, when a block having a candidate flag of 1 exists alone, the block is not determined as screen content. In this embodiment, this determination information is a 1-bit flag (determination flag), a value when it is determined as screen content is 1, and a value when it is not determined as screen content is 0.

  In the second determination method, the screen content determination unit 14 uses a plurality of adjacent blocks when the blocks having the candidate flag of 1 are adjacent to each other and the DC prediction values are equal in the luminance signal and the color difference signal of each block. Is determined as screen content, and determination information indicating the determination result is output. Compared with the first determination method, the second determination method can further improve the determination accuracy of the screen content.

  FIG. 2 is a diagram for explaining the processing of the second determination method of the screen content determination unit 14 described above. As shown in FIG. 2A, the candidate flag for block a is 1 and the DC prediction value is 110, the candidate flag for block b is 1 and the DC prediction value is 110, and the candidate flag for block c is 0. The DC prediction value is 110, the candidate flag of block d is 1 and the DC prediction value is 110, the candidate flag of block e is 0 and the DC prediction value is 110, and the candidate flag of block f is 1 and the DC prediction value is 110 The value is 105, the candidate flag for block g is 0 and the DC prediction value is 105, the candidate flag for block h is 1 and the DC prediction value is 105, the candidate flag for block i is 0 and the DC prediction value is 82, and the candidate flag of block j is 0 and the DC prediction value is 82, the blocks d and h are not regarded as screen contents because the candidate flag of the adjacent block is 0. , The decision flag is set to 0. Moreover, although the candidate flag of the adjacent flag is 1 in the blocks a, b, and f, the DC prediction value of the block f is not the same as the DC prediction value of the blocks a and b. Therefore, as shown in FIG. 2B, the determination flag is set to 1 only for the blocks a and b, and the determination flag of the block f is set to 0. In FIG. 2B, blocks with the determination flag of 1 are indicated by hatching.

  FIG. 3 is a flowchart illustrating an operation example of the image determination apparatus 1 described above. First, the image determination apparatus 1 determines the intra prediction mode of each block (target block) by the prediction mode determination unit 11 (step S101), and when the intra prediction mode of the luminance signal and the color difference signal is not DC prediction (step S102). -No), it is determined that the block is a natural image (not screen content), and the determination flag is set to 0 (step S103).

  When both the intra prediction modes of the luminance signal and the color difference signal are DC prediction (step S102—Yes), the pixel value comparison unit 13 stores the DC prediction value of the luminance signal and the DC prediction value of the color difference signal (step S102). S104). Next, the pixel value comparison unit 13 matches the DC predicted value of the luminance signal with the pixel value of a part of the luminance signal (four corners of the block in this example), and the DC predicted value of the color difference signal and a part of the color difference signal ( In this example, it is determined whether or not the pixel values at the four corners of the block match (step S105). If the determination result in step S105 is No, it is determined that the block is a natural image, and the determination flag is set to 0 (step S103). If the determination result in step S105 is Yes, 1 is set in the candidate flag (step S106).

  Next, the screen content determination unit 14 determines whether the candidate flag of the adjacent block is 1 and the DC prediction values match for the block with the candidate flag of 1 (step S107). If the determination result in step S107 is No, it is determined that the block is a natural image, and the determination flag is set to 0 (step S103). On the other hand, if the determination result in step S107 is Yes, adjacent blocks are regarded as screen contents and merged, and the determination flag between adjacent blocks is set to 1 (step S108).

  As described above, in the image determination device 1, both the intra prediction modes of the luminance signal and the color difference signal are DC prediction by the pixel value comparison unit 13, and the pixel value of a part of the luminance signal and the DC prediction value of the luminance signal are , And a block in which a part of pixel values of the color difference signal and a DC prediction value of the color difference signal match is determined as a screen content candidate, and the screen content determination unit 14 determines that the block is a screen content candidate. When the determined blocks are adjacent to each other, it is determined that the plurality of adjacent blocks are screen contents, and determination information indicating the determination result is output. With this configuration, the image determination apparatus 1 can determine whether the content is a screen content or a natural image for each block of the image.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. FIG. 4 is a block diagram illustrating a configuration example of the image determination device 2 according to the second embodiment. In the example illustrated in FIG. 4, the image determination device 2 includes a prediction mode determination unit 11, a pixel value measurement unit 12, a pixel value comparison unit 13, a screen content determination unit 14, and a determination correction unit 15. The image determination device 2 according to the second embodiment is different from the image determination device 1 according to the first embodiment in that a determination correction unit 15 is further provided. Since other configurations are the same as those of the first embodiment, the same reference numerals are given and description thereof is omitted as appropriate.

  The determination correction unit 15 inputs the determination information from the screen content determination unit 14. Then, the screen content determination unit 14 determines that one or more blocks surrounded by the blocks determined to be screen content are all screen content, corrects the determination information, and outputs final determination information.

  FIG. 5 is a diagram for explaining the processing of the determination correction unit 15. Assume that the determination flag of each block input from the screen content determination unit 14 is shown in FIG. In this case, there is a block surrounded by blocks whose determination flag is 1. Therefore, as shown in FIG. 5B, the determination correction unit 15 determines that the block surrounded by the block having the determination flag of 1 is the screen content, and corrects the determination flag of the block from 0 to 1. In FIG. 5, blocks with a determination flag of 1 are indicated by hatching.

  As described above, the image determination apparatus 2 uses all the blocks surrounded by the blocks determined by the screen content determination unit 14 as the screen content by the determination correction unit 15 (blocks having a determination flag of 1) as the screen content. Judge that there is. Since the screen content is highly likely to be composed of continuous blocks, the determination accuracy of the screen content can be further improved as compared with the image determination device 1 by correcting the determination information by the determination correction unit 15.

  Here, a computer can be preferably used to cause the image determination apparatus 1 or 2 to function as described above, and such a computer can store a program describing processing contents for realizing each function of the image determination apparatus 1 or 2. It can be realized by storing the program in a storage unit of the computer and reading and executing the program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

(Third embodiment)
Next, a third embodiment of the present invention will be described. FIG. 6 is a block diagram showing a configuration example of an encoding apparatus according to the third embodiment of the present invention. In the example illustrated in FIG. 6, the encoding device 20 includes the image determination device 1 or 2 described above, the subtraction unit 21, the orthogonal transformation unit 22, the quantization unit 23, the inverse quantization unit 24, and the inverse orthogonal transformation. Unit 25, addition unit 26, deblocking filter 27, memory 28, intra prediction unit 29, motion compensation unit 30, changeover switch 31, and variable length coding unit 32. The encoding device 20 is a conventional ITU-T H.264 standard. H.264 / MPEG-4 AVC and ITU-T H.264. Compared with an encoding device using an encoding method such as H.265 / MPEG-H HEVC, the image determination device 1 or 2 further includes an orthogonal transform unit 22, a quantization unit 23, an inverse quantization unit 24, and an inverse orthogonal. The conversion unit 25, the deblocking filter 27, and the variable length coding unit 32 are different in that they operate by inputting determination information from the image determination apparatus 1 or 2.

  As described above, the image determination apparatus 1 or 2 receives the original image on which intra prediction is performed and the intra prediction mode generated by the intra prediction unit 29, and the generated determination information is subjected to orthogonal transformation unit 22 and quantization unit 23. , Output to the inverse quantization unit 24, the inverse orthogonal transform unit 25, the deblocking filter 27, and the variable length coding unit 32. In FIG. 6, signal lines for transmitting the determination information are indicated by dotted lines.

  The subtraction unit 21 calculates a difference between the original image (luminance signal and color difference signal) and a prediction image (luminance prediction image and color difference prediction image) input from the intra prediction unit 29 or the motion compensation unit 30 via the changeover switch 31. The difference image is calculated and output to the orthogonal transform unit 22.

  The orthogonal transform unit 22 performs orthogonal transform processing on the difference image input from the subtraction unit 21 in units of blocks, and outputs the obtained orthogonal transform coefficients to the quantization unit 23. However, when the determination information input from the image determination apparatus 1 or 2 indicates that the block is screen content, the orthogonal transform unit 22 does not perform orthogonal transform processing.

  The quantization unit 23 selects a quantization table for the orthogonal transform coefficient input from the orthogonal transform unit 22 and performs a quantization process. The quantized orthogonal transform coefficient is converted into an inverse quantization unit 24 and a variable length code. To the conversion unit 32. However, when the determination information input from the image determination apparatus 1 or 2 indicates that the block is screen content, orthogonal transform processing is not performed, but the quantization unit 23 does not perform orthogonal transform coefficient bits. It corresponds to the number. Therefore, before the quantization process, the left shift process of the bits is performed so that the orthogonal transform coefficient and the norm (size) are matched.

  The inverse quantization unit 24 performs inverse quantization processing on the quantized orthogonal transform coefficient input from the quantization unit 23 and outputs the obtained orthogonal transform coefficient to the inverse orthogonal transform unit 25. However, if the determination information input from the image determination device 1 or 2 indicates that the block is screen content, the bit is left-shifted before the quantization process. After performing the quantization process, a bit right shift process is performed to restore the number of bits.

  The inverse orthogonal transform unit 25 performs an inverse orthogonal transform (for example, IDCT; Inverse Discrete Cosine Transform) process on the orthogonal transform coefficient input from the inverse quantization unit 24 and outputs an inversely orthogonally transformed image to the addition unit 26. To do. However, when the determination information input from the image determination apparatus 1 or 2 indicates that the block is screen content, the inverse orthogonal transform unit 25 does not perform the inverse orthogonal transform process.

  The adding unit 26 performs an inverse orthogonal transform image input from the inverse orthogonal transform unit 25 and a predicted image (luminance prediction image or color difference prediction image) input from the intra prediction unit 29 or the motion compensation unit 30 via the changeover switch 31. ) Is added to generate a local decoded image, and the generated local decoded image is output to the deblocking filter 27.

  The deblocking filter 27 adaptively performs filtering processing according to the block boundary strength in order to remove block noise generated at the block boundary due to encoding, and outputs the result to the memory 28. However, when the determination information input from the image determination apparatus 1 or 2 indicates that the block is screen content, the filtering process is not performed on the color difference signal.

  The intra prediction unit 29 refers to the locally decoded image stored in the memory 28, selects an intra prediction mode, and generates an intra predicted image. Then, the generated intra prediction image is output to the subtraction unit 21 and the addition unit 26 via the changeover switch 31, and the intra prediction mode is output to the image determination device 1 or 2 and the variable length encoding unit 32.

  The motion compensation unit 30 refers to the past and / or future frames stored in the memory 28 with respect to the original image, generates a motion vector by motion estimation, and sends the generated motion vector to the variable length encoding unit 32. Output. In addition, the motion compensation unit 30 performs motion compensation on the decoded image stored in the memory 28 using the generated motion vector to generate an inter-predicted image, and switches the generated inter-predicted image to the changeover switch 31. To the subtractor 21 and the adder 26.

  The changeover switch 31 switches between the intra prediction image input from the intra prediction unit 29 and the inter prediction image input from the motion compensation unit 30, and outputs the result to the subtraction unit 21 and the addition unit 26.

  The variable length encoding unit 32 scans the quantized orthogonal transform coefficient input from the quantization unit 23 to perform variable length encoding processing, and outputs the obtained encoded data to the outside. Further, the variable length coding unit 32 has variable lengths in the prediction mode input from the intra prediction unit 29, the motion vector information input from the motion compensation unit 30, and the determination flag input from the image determination device 1 or 2. Encoding processing is performed, and the obtained encoded data is output to the outside.

  Thus, since the encoding device 20 includes the image determination device 1 or 2, the encoding process can be changed according to the determination information, and thus the encoding image quality of the moving image including the screen content can be set to the encoding speed. It becomes possible to improve without lowering.

  Here, a computer can be suitably used to cause the above-described encoding device 20 to function, and such a computer can store a program describing processing contents for realizing each function of the encoding device 20 in the computer. It can be realized by storing the program in a storage unit and reading and executing the program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks described in the embodiments can be combined into one, or one constituent block can be divided.

DESCRIPTION OF SYMBOLS 1, 2 Image determination apparatus 11 Prediction mode determination part 12 Pixel value measurement part 13 Pixel value comparison part 14 Screen content determination part 15 Determination correction part 20 Encoding apparatus 21 Subtraction part 22 Orthogonal transformation part 23 Quantization part 24 Inverse quantization part 25 Inverse Orthogonal Transformer 26 Adder 27 Deblocking Filter 28 Memory 29 Intra Prediction Unit 30 Motion Compensation Unit 31 Changeover Switch 32 Variable Length Coding Unit

Claims (7)

An image determination device that determines whether or not each block into which an image is divided is screen content,
The intra prediction modes of the luminance signal and the chrominance signal are both DC prediction, and the pixel value of a part of the luminance signal matches the DC prediction value of the luminance signal, and the partial pixel value of the chrominance signal and the DC of the chrominance signal A pixel value comparison unit that determines a block with a predicted value as a screen content candidate;
A screen content determination unit for determining, when blocks determined as screen content candidates by the pixel value comparison unit are adjacent, that the plurality of adjacent blocks are screen content and outputting determination information indicating a determination result; ,
An image determination apparatus comprising:   The screen content determination unit determines that the plurality of adjacent blocks are screen content when the blocks determined as screen content candidates by the pixel value comparison unit are adjacent to each other and the DC predicted values are also equal. The image determination apparatus according to claim 1, wherein:   The image determination apparatus according to claim 1, wherein the pixel value comparison unit sets the partial pixel values as pixel values at four corners of the block.   The image processing apparatus further includes a determination correction unit that determines that one or more blocks surrounded by blocks determined to be screen content by the screen content determination unit are all screen content and corrects the determination information. The image determination apparatus according to any one of claims 1 to 3. An image determination device according to any one of claims 1 to 4, comprising:
An encoding apparatus, wherein an encoding process is changed according to the determination information.   The program for functioning a computer as an image determination apparatus as described in any one of Claim 1 to 4. A program for causing a computer to function as the encoding device according to claim 5.

Images