JP2004088795A - Image signal preparation device and method, and image signal reproducing device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image signal preparation device and a method, and image signal reproducing device and its method suitable for use in the case of recording a motion image signal in a record medium for instance such as a magnet-optical disk or a magnetic tape and the like and reproducing it for presenting on a display and the like, or transmitting the dynamic image signal from a transmitter to a recipient through a transmission path such as a video conference system, a videophone system and a broadcasting instrument and the like and receiving and displaying it in the recipient. <P>SOLUTION: Data Y1 or Y4 of the luminance block of a considered macroblock, a master slice that consists of the data Cb5" and Cr6" of the color difference block of the lowest resolution, a slave slice 1 that consists of the data Cb5', Cr6', Cb7' and Cr8' of the color difference block of the medium resolution of a corresponded macroblock, and a slave slice 2 that consists of the data Cb5, Cr6, Cb7, Cr8, Cb9, Cr10, Cb11 and Cr12 of the color difference block of the highest resolution of the corresponded macro block are serially arranged in the image signal preparation device. <P>COPYRIGHT: (C)2004,JPO

Description

The present invention records a moving image signal on a recording medium such as a magneto-optical disk or a magnetic tape, and reproduces the moving image signal to display it on a display or the like. The image signal is transmitted from the transmission side to the reception side via the transmission path, and the reception side receives and receives the image signal. About the method.

For example, in a system for transmitting a moving image signal to a remote place, such as a video conference system and a video telephone system, in order to efficiently use a transmission path, a line correlation or an inter-frame correlation of a video signal is used. The image signal is compressed and encoded.

If the line correlation is used, the image signal can be compressed by, for example, DCT (discrete cosine transform) processing.

In addition, if the inter-frame correlation is used, the image signal can be further compressed and encoded. For example, as shown in FIG. 14, when frame images PC1, PC2, and PC3 are generated at times t1, t2, and t3, respectively, the difference between the image signals of frame images PC1 and PC2 is calculated to generate PC12. Further, a difference between the frame images PC2 and PC3 is calculated to generate a PC23. Normally, the images of temporally adjacent frames do not have such a large change, and when the difference between them is calculated, the difference signal has a small value. Therefore, if the difference signal is encoded, the code amount can be compressed.

However, transmitting only the differential signal cannot restore the original image. Therefore, the image of each frame is one of three types of pictures, i-pictures, P-pictures, and B-pictures, and the image signal is compression-coded.

That is, as shown in FIG. 15, for example, image signals of 17 frames from frame F1 to frame F17 are set as a group of pictures and are set as one unit of processing. Then, the image signal of the first frame F1 is encoded as an I picture, the second frame F2 is processed as a B picture, and the third frame F3 is processed as a P picture. Hereinafter, the fourth and subsequent frames F4 to F17 are alternately processed as B pictures or P pictures.

As an I-picture image signal, the image signal for one frame is transmitted as it is. On the other hand, as an image signal of a P picture, basically, as shown in FIG. 15A, a difference from an image signal of an I picture or a P picture which precedes it is transmitted. Further, as an image signal of a B picture, basically, as shown in FIG. 15B, a difference from an average value of both temporally preceding and succeeding frames is obtained, and the difference is encoded. Become

FIG. 16 shows the principle of a method for encoding a moving image signal in this manner. As shown in the figure, since the first frame F1 is processed as an I picture, it is directly transmitted as transmission data F1X to the transmission path (intra-picture encoding). On the other hand, since the second frame F2 is processed as a B picture, the difference between the temporally preceding frame F1 and the average value of the temporally succeeding frame F3 is calculated, and the difference is calculated. It is transmitted as transmission data F2X.

However, there are four types of processing as a B picture, if described in more detail. The first process is to transmit the data of the original frame F2 as it is as the transmission data F2X (SP1) (intra coding), and is the same process as in the case of the I picture. The second process is to calculate a difference from the temporally later frame F3 and transmit the difference (SP2) (backward prediction coding). The third process is to transmit the difference (SP3) from the temporally preceding frame F1 (forward prediction coding). Further, the fourth processing is to generate a difference (SP4) between the average value of the temporally preceding frame F1 and the average value of the following frame F3, and transmit this as transmission data F2X (bidirectional predictive coding). .

の う ち Among these four methods, the method that minimizes the transmission data is adopted.

When transmitting the difference data, a motion vector x1 (a motion vector between frames F1 and F2) with an image (predicted image) of a frame whose difference is to be calculated (in the case of forward prediction) or x2 (Motion vector between frames F3 and F2) (for backward prediction) or both x1 and x2 (for bidirectional prediction) are transmitted along with the difference data.

Further, a frame F3 of a P picture is calculated using a temporally preceding frame F1 as a predicted image, a difference signal (SP3) from the frame, and a motion vector x3, and the resultant is transmitted as transmission data F3X (forward prediction). Coding). Alternatively, the data of the original frame F3 is transmitted as it is as the transmission data F3X (SP1) (intra coding). Which method is used, as in the case of the B picture, is selected so that the transmission data is smaller.

FIG. 17 shows a configuration example of a device that encodes and transmits a moving image signal based on the above-described principle and decodes the encoded signal. The encoding device 1 encodes an input video signal and transmits the encoded video signal to a recording medium 3 as a transmission path. The decoding device 2 reproduces a signal recorded on the recording medium 3, and decodes and outputs the signal.

In the encoding device 1, an input video signal is input to a preprocessing circuit 11, where a luminance signal and a chrominance signal (a color difference signal in this example) are separated, and are separated by A / D converters 12 and 13, respectively. A / D conversion is performed. The video signal converted into a digital signal by A / D conversion by the A / D converters 12 and 13 is supplied to the frame memory 14 and stored therein. The frame memory 14 stores the luminance signal in the luminance signal frame memory 15 and the chrominance signal in the chrominance signal frame memory 16, respectively.

The format conversion circuit 17 converts the frame format signal stored in the frame memory 14 into a block format signal. That is, as shown in FIG. 18, the video signal stored in the frame memory 14 is data of a frame format in which V lines of H dots are collected per line. The format conversion circuit 17 divides the signal of one frame into M slices in units of 16 lines. Then, each slice is divided into M macroblocks. Each macro block is composed of a luminance signal corresponding to 16 × 16 pixels (dots), and this luminance signal is further divided into blocks Y [1] to Y [4] in units of 8 × 8 dots. You. The luminance signal of 16 × 16 dots corresponds to a Cb signal of 8 × 8 dots and a Cr signal of 8 × 8 dots.

デ ー タ The data thus converted into the block format is supplied from the format conversion circuit 17 to the encoder 18, where the data is encoded. The details will be described later with reference to FIG.

The signal encoded by the encoder 18 is output to the transmission path as a bit stream, and is recorded on the recording medium 3, for example.

(4) The data reproduced from the recording medium 3 is supplied to the decoder 31 of the decoding device 2 and is decoded. The details of the decoder 31 will be described later with reference to FIG.

The data decoded by the decoder 31 is input to the format conversion circuit 32, and is converted from the block format to the frame format. Then, the luminance signal in the frame format is supplied to and stored in the luminance signal frame memory 34 of the frame memory 33, and the color difference signal is supplied to and stored in the color difference signal frame memory 35. The luminance signal and the chrominance signal read from the luminance signal frame memory 34 and the chrominance signal frame memory 35 are D / A converted by D / A converters 36 and 37, respectively, supplied to a post-processing circuit 38, and synthesized. . Then, the data is output and displayed on a display such as a CRT (not shown).

Next, a configuration example of the encoder 18 will be described with reference to FIG.

画像 Image data to be encoded is input to the motion vector detection circuit 50 in macroblock units. The motion vector detection circuit 50 processes the image data of each frame as an I picture, a P picture, or a B picture according to a predetermined sequence set in advance. Whether the image of each frame that is sequentially input is processed as one of I, P, and B pictures is determined in advance (for example, as illustrated in FIG. 15, the image is configured by frames F1 to F17). The group of pictures are processed as I, B, P, B, P,... B, P).

Image data of a frame (for example, frame F1) processed as an I picture is transferred from the motion vector detection circuit 50 to the front original image section 51a of the frame memory 51, stored, and processed as a B picture (for example, frame F2). Is transferred and stored in the original image unit 51b, and the image data of a frame (for example, frame F3) processed as a P picture is transferred and stored in the rear original image unit 51c.

Further, at the next timing, when an image of a frame to be further processed as a B picture (frame F4) or a P picture (frame F5) is input, the first P picture stored in the rear original image section 51c until then is input. The image data of (frame F3) is transferred to the front original image section 51a, the image data of the next B picture (frame F4) is stored (overwritten) in the original image section 51b, and the next P picture (frame F5) Is stored (overwritten) in the rear original image section 51c. Such an operation is sequentially repeated.

The signal of each picture stored in the frame memory 51 is read therefrom, and the prediction mode switching circuit 52 performs frame prediction mode processing or field prediction mode processing. Further, under the control of the prediction determination circuit 54, the calculation unit 53 performs calculation of intra prediction, forward prediction, backward prediction, or bidirectional prediction. Which of these processes is to be performed is determined in accordance with the prediction error signal (the difference between the reference image to be processed and the predicted image corresponding thereto). For this reason, the motion vector detection circuit 50 generates a sum of absolute values (or a sum of squares) of the prediction error signal used for this determination.

Here, the frame prediction mode and the field prediction mode in the prediction mode switching circuit 52 will be described.

When the frame prediction mode is set, the prediction mode switching circuit 52 sends the four luminance blocks Y [1] to Y [4] supplied from the motion vector detection circuit 50 to the subsequent operation unit 53 as they are. Output. That is, in this case, as shown in FIG. 20A, the data of the line of the odd field and the data of the line of the even field are mixed in each luminance block. In this frame prediction mode, prediction is performed in units of four luminance blocks (macroblocks), and one motion vector corresponds to four luminance blocks.

On the other hand, in the field prediction mode, the prediction mode switching circuit 52 converts the signal input from the motion vector detection circuit 50 with the configuration shown in FIG. Among the luminance blocks, the luminance blocks Y [1] and Y [2] are constituted by, for example, only dots of the odd field lines, and the other two luminance blocks Y [3] and Y [4] The data is constituted by the data of the lines of the even-numbered fields and output to the calculation unit 53. In this case, one motion vector corresponds to two luminance blocks Y [1] and Y [2], and the other two luminance blocks Y [3] and Y [4]. Thus, another one motion vector is corresponded.

The motion vector detection circuit 50 outputs the sum of absolute values of the prediction errors in the frame prediction mode and the sum of the absolute values of the prediction errors in the field prediction mode to the prediction mode switching circuit 52. The prediction mode switching circuit 52 compares the absolute values of the prediction errors in the frame prediction mode and the field prediction mode, performs a process corresponding to the prediction mode having a small value, and outputs the data to the calculation unit 53.

However, such a process is actually performed by the motion vector detection circuit 50. That is, the motion vector detection circuit 50 outputs a signal having a configuration corresponding to the determined mode to the prediction mode switching circuit 52, and the prediction mode switching circuit 52 outputs the signal as it is to the subsequent operation unit 53.

In the frame prediction mode, the color difference signal is supplied to the arithmetic unit 53 in a state where the data of the lines of the odd fields and the data of the lines of the even fields are mixed as shown in FIG. In the case of the field prediction mode, as shown in FIG. 20B, the upper half (4 lines) of each of the chrominance blocks Cb and Cr corresponds to the odd field corresponding to the luminance block Y [1] and Y [2]. The lower half (4 lines) is a color difference signal of an even field corresponding to the luminance blocks Y [3] and Y [4].

In addition, the motion vector detection circuit 50 determines the absolute value of the prediction error for determining whether to perform intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in the prediction determination circuit 54 as follows. Generate a sum of values.

That is, as the sum of the absolute values of the prediction errors of the intra-picture prediction, the sum Σ | Aij | of the sum 信号 Aij | of the sum 信号 Aij of the signal Aij of the macroblock of the reference picture and the sum │Aij│ of the absolute value of the signal Aij of the macroblock Find the difference between The sum of absolute values | Aij−Bij | of the difference Aij−Bij | between the signal Aij of the macroblock of the reference image and the signal Bij of the macroblock of the predicted image is used as the absolute value sum of the prediction error of the forward prediction. Bij |. In addition, the absolute value sum of the prediction error between the backward prediction and the bidirectional prediction is obtained in the same manner as in the forward prediction (by changing the predicted image to a different predicted image from that in the forward prediction).

和 The sum of these absolute values is supplied to the prediction determination circuit 54. The prediction determination circuit 54 selects the smallest absolute value sum of the prediction errors of the forward prediction, the backward prediction, and the bidirectional prediction as the absolute value sum of the prediction errors of the inter prediction. Furthermore, the absolute value sum of the prediction error of the inter prediction and the absolute value sum of the prediction error of the intra prediction are compared, and the smaller one is selected, and the mode corresponding to the selected absolute value sum is set as the prediction mode. select. That is, if the sum of the absolute values of the prediction errors of the intra prediction is smaller, the intra prediction mode is set. If the sum of the absolute values of the prediction errors in the inter prediction is smaller, the mode having the smallest absolute value sum is set among the forward prediction, backward prediction, and bidirectional prediction modes.

As described above, the motion vector detecting circuit 50 converts the signal of the macroblock of the reference image into the prediction mode switching circuit 52 in a configuration corresponding to the mode selected by the prediction mode switching circuit 52 among the frame or field prediction modes. And a motion vector between the predicted image and the reference image corresponding to the prediction mode selected by the prediction determination circuit 54 among the four prediction modes, and the variable-length encoding circuit 58 Is output to the motion compensation circuit 64. As described above, as the motion vector, the motion vector having the smallest absolute value sum of the corresponding prediction errors is selected.

The prediction determination circuit 54 sets the intra-frame (image) prediction mode (mode in which motion compensation is not performed) as the prediction mode when the motion vector detection circuit 50 is reading the image data of the I picture from the front original image section 51a. Then, the switch 53d of the calculation unit 53 is switched to the contact a side. As a result, the image data of the I picture is input to the DCT mode switching circuit 55.

The DCT mode switching circuit 55 converts the data of the four luminance blocks into a state where the lines of the odd field and the lines of the even field are mixed (frame DCT mode), as shown in FIG. Alternatively, the signal is output to the DCT circuit 56 in one of the separated states (field DCT mode).

That is, the DCT mode switching circuit 55 compares the coding efficiency in the case where the DCT processing is performed in a mixed state with the data of the odd field and the even field with the coding efficiency in the case where the DCT processing is performed in the separated state. Choose a good mode.

For example, as shown in FIG. 21A, the input signal has a configuration in which the lines of the odd field and the even field are mixed, and the difference between the signal of the line of the odd field and the signal of the line of the even field adjacent vertically. And calculate the sum (or sum of squares) of the absolute values. As shown in FIG. 21B, the input signal has a structure in which the lines of the odd field and the even field are separated from each other, and the signal difference between the lines of the odd field adjacent vertically and the line of the even field. The difference between the signals is calculated, and the sum (or sum of squares) of the respective absolute values is obtained. Further, the DCT mode corresponding to the smaller value is set by comparing the two (the sum of absolute values). That is, if the former is smaller, the frame DCT mode is set, and if the latter is smaller, the field DCT mode is set.

{Circle around (4)} The data having the configuration corresponding to the selected DCT mode is output to the DCT circuit 56, and the DCT flag indicating the selected DCT mode is output to the variable length coding circuit 58 and the motion compensation circuit 64.

As is apparent from a comparison between the prediction mode (FIG. 20) in the prediction mode switching circuit 52 and the DCT mode (FIG. 21) in the DCT mode switching circuit 55, the data structure of each mode of the luminance block is substantially the same. Are identical.

When the prediction mode switching circuit 52 selects the frame prediction mode (mode in which odd lines and even lines are mixed), the DCT mode switching circuit 55 also performs frame DCT mode (mode in which odd lines and even lines are mixed). If the field prediction mode (mode in which the data of the odd field and the data of the even field are separated) is selected in the prediction mode switching circuit 52, the DCT mode switching circuit 55 selects the field DCT mode ( (The mode in which the data of the odd field and the data of the even field are separated) is likely to be selected.

However, this is not always the case. In the prediction mode switching circuit 52, the mode is determined so that the sum of absolute values of the prediction errors is small, and in the DCT mode switching circuit 55, the coding efficiency is good. The mode is determined so that

The I-picture image data output from the DCT mode switching circuit 55 is input to the DCT circuit 56, where it is subjected to DCT (Discrete Cosine Transform) processing and converted into DCT coefficients. This DCT coefficient is input to the quantization circuit 57 and the transmission buffer 59
After being quantized in a quantization step corresponding to the data accumulation amount (buffer accumulation amount) of the data, the data is input to the variable-length encoding circuit 58.

The variable-length encoding circuit 58 converts the image data (in this case, I-picture data) supplied from the quantization circuit 57 in accordance with the quantization step (scale) supplied from the quantization circuit 57, for example. The data is converted into a variable length code such as a Huffman code and output to the transmission buffer 59.

The variable length encoding circuit 58 also determines whether a quantization step (scale) by the quantization circuit 57 or a prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction) by the prediction determination circuit 54. Mode), a motion vector from the motion vector detection circuit 50, a prediction flag (a flag indicating whether the frame prediction mode or the field prediction mode is set) from the prediction mode switching circuit 52, and a DCT output from the DCT mode switching circuit 55. A flag (a flag indicating whether the frame DCT mode or the field DCT mode is set) is input, and these are also subjected to variable length coding.

The transmission buffer 59 temporarily stores the input data, and outputs data corresponding to the storage amount to the quantization circuit 57.

(4) When the remaining data amount increases to the allowable upper limit, the transmission buffer 59 reduces the data amount of the quantized data by increasing the quantization scale of the quantization circuit 57 by the quantization control signal. Conversely, when the remaining data amount decreases to the allowable lower limit, the transmission buffer 59 reduces the data amount of the quantized data by reducing the quantization scale of the quantization circuit 57 by the quantization control signal. Increase. In this way, overflow or underflow of the transmission buffer 59 is prevented.

The data stored in the transmission buffer 59 is read out at a predetermined timing, output to a transmission path, and recorded on the recording medium 3, for example.

On the other hand, the I-picture data output from the quantization circuit 57 is input to the inverse quantization circuit 60 and is inversely quantized in accordance with the quantization step supplied from the quantization circuit 57. The output of the inverse quantization circuit 60 is input to an IDCT (inverse DCT) circuit 61, subjected to inverse DCT processing, supplied to a forward prediction image section 63a of a frame memory 63 via an arithmetic unit 62, and stored.

When the motion vector detection circuit 50 processes the sequentially input image data of each frame as, for example, pictures of I, B, P, B, P, B,. After processing the image data as an I picture, before processing the image of the next input frame as a B picture, the image data of the next input frame is further processed as a P picture. This is because a B picture involves backward prediction and cannot be decoded unless a P picture as a backward prediction image is prepared first.

Then, after the processing of the I picture, the motion vector detection circuit 50 starts processing the image data of the P picture stored in the rear original image section 51c. Then, as in the case described above, the absolute value sum of the inter-frame difference (prediction error) in macroblock units is supplied from the motion vector detection circuit 50 to the prediction mode switching circuit 52 and the prediction determination circuit 54. The prediction mode switching circuit 52 and the prediction determination circuit 54 perform frame / field prediction mode, intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in accordance with the sum of absolute values of prediction errors of macroblocks of the P picture. Set the prediction mode for.

When the intra-frame prediction mode is set, the calculation unit 53 switches the switch 53d to the contact a as described above. Therefore, this data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59, similarly to the I picture data. The data is supplied to the backward prediction image section 63b of the frame memory 63 via the inverse quantization circuit 60, the IDCT circuit 61, and the calculator 62, and stored therein.

In the forward prediction mode, the switch 53d is switched to the contact b, and the image data (in this case, the I picture) stored in the forward prediction image section 63a of the frame memory 63 is read out. Thus, motion compensation is performed in accordance with the motion vector output from the motion vector detection circuit 50. That is, when the setting of the forward prediction mode is instructed by the prediction determination circuit 54, the motion compensation circuit 64 sets the read address of the forward prediction image section 63a to the position of the macroblock that the motion vector detection circuit 50 is currently outputting. Data is read out from the corresponding position by a distance corresponding to the motion vector, and predicted image data is generated.

The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53a. The arithmetic unit 53a subtracts the predicted image data corresponding to the macroblock supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and calculates the difference (prediction error). ) Is output. This difference data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59. The difference data is locally decoded by the inverse quantization circuit 60 and the IDCT circuit 61, and is input to the arithmetic unit 62.

演算 The same data as the predicted image data supplied to the calculator 53a is also supplied to the calculator 62a. The calculator 62 adds the prediction image data output from the motion compensation circuit 64 to the difference data output from the IDCT circuit 61. As a result, image data of the original (decoded) P picture is obtained. The P-picture image data is supplied to and stored in the backward prediction image section 63b of the frame memory 63.

After the I-picture and P-picture data are stored in the forward prediction image section 63a and the backward prediction image section 63b, the motion vector detection circuit 50 executes the processing of the B picture. The prediction mode switching circuit 52 and the prediction determination circuit 54 set the frame / field mode according to the magnitude of the sum of the absolute values of the inter-frame differences in macroblock units. Set to one of the forward prediction mode, the backward prediction mode, and the bidirectional prediction mode.

As described above, in the intra-frame prediction mode or the forward prediction mode, the switch 53d is switched to the contact point a or b. At this time, the same processing as in the case of the P picture is performed, and the data is transmitted.

On the other hand, when the backward prediction mode or the bidirectional prediction mode is set, the switch 53d is switched to the contact point c or d.

In the backward prediction mode in which the switch 53d is switched to the contact point c, the image data (in this case, the image of the P picture) stored in the backward prediction image section 63b is read out. Motion compensation is performed corresponding to the motion vector output from the vector detection circuit 50. That is, when the setting of the backward prediction mode is instructed by the prediction determination circuit 54, the motion compensation circuit 64 sets the read address of the backward prediction image unit 63b to the position of the macroblock that the motion vector detection circuit 50 is currently outputting. Data is read out from the corresponding position by a distance corresponding to the motion vector, and predicted image data is generated.

The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53b. The arithmetic unit 53b subtracts the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference. This difference data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.

In the bidirectional prediction mode in which the switch 53d is switched to the contact d, the image data (in this case, the I-picture image) stored in the forward prediction image section 63a and the image data stored in the backward prediction image section 63b. Image data (in this case, an image of a P picture) is read out, and the motion compensation circuit 64 performs motion compensation corresponding to the motion vector output from the motion vector detection circuit 50. That is, when the setting of the bidirectional prediction mode is instructed by the prediction determination circuit 54, the motion vector detection circuit 50 outputs the read addresses of the forward predicted image section 63a and the backward predicted image section 63b. The data is read out from the position corresponding to the position of the macro block in which the data is shifted by an amount corresponding to the motion vector (in this case, two motion vectors for the forward predicted image and the backward predicted image), and predicted image data is generated. I do.

The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53c. The arithmetic unit 53c subtracts the average value of the prediction image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the motion vector detection circuit 50, and outputs the difference. This difference data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.

The image of the B picture is not stored in the frame memory 63 because it is not regarded as a predicted image of another image.

In the frame memory 63, the forward prediction image section 63a and the backward prediction image section 63b are switched between banks as necessary, and the one stored in one or the other with respect to a predetermined reference image is replaced with the forward prediction image section 63a. It can be switched and output as a predicted image or a backward predicted image.

In the above description, the description has been made centering on the luminance block. However, the chrominance block is also processed and transmitted in units of macroblocks shown in FIGS. As a motion vector for processing a chrominance block, a motion vector obtained by halving the motion vector of the corresponding luminance block in the vertical and horizontal directions is used.

<< Next, FIG. 22 is a block diagram showing a configuration of an example of the decoder 31 of FIG. The encoded image data transmitted via the transmission path (recording medium 3) is received by a receiving circuit (not shown) or reproduced by a reproducing device, temporarily stored in a receiving buffer 81, and then decoded by a decoding circuit 90. Is supplied to the variable length decoding circuit 82. The variable length decoding circuit 82 performs variable length decoding on the data supplied from the reception buffer 81, and outputs the motion vector, the prediction mode, the prediction flag and the DCT flag to the motion compensation circuit 87, and the quantization step to the inverse quantization circuit. 83, and outputs the decoded image data to the inverse quantization circuit 83.

The inverse quantization circuit 83 inversely quantizes the image data supplied from the variable length decoding circuit 82 in accordance with the quantization step also supplied from the variable length decoding circuit 82, and outputs the result to the IDCT circuit 84. The data (DCT coefficient) output from the inverse quantization circuit 83 is subjected to inverse DCT processing in the IDCT circuit 84 and supplied to the arithmetic unit 85.

When the image data supplied from the IDCT circuit 84 is I-picture data, the data is output from the arithmetic unit 85 and a predicted image of image data (P or B-picture data) to be input to the arithmetic unit 85 later. For data generation, the data is supplied to and stored in the forward prediction image section 86a of the frame memory 86. This data is output to the format conversion circuit 32 (FIG. 17).

When the image data supplied from the IDCT circuit 84 is P-picture data in which image data one frame before the P-picture is used as predicted image data and is data in the forward prediction mode, the forward predicted image portion 86a of the frame memory 86 Is read out, and the motion compensation circuit 87 performs motion compensation corresponding to the motion vector output from the variable length decoding circuit 82. The arithmetic unit 85 adds the image data (difference data) supplied from the IDCT circuit 84 and outputs the result. The added data, that is, the decoded P picture data is transmitted to the rear of the frame memory 86 in order to generate predicted image data of the image data (B picture or P picture data) input later to the arithmetic unit 85. It is supplied to and stored in the prediction image section 86b.

Even in the case of P-picture data, the data in the intra-prediction mode is stored in the backward prediction image unit 86b without being processed by the arithmetic unit 85, as in the case of the I-picture data.

Since this P picture is an image to be displayed next to the next B picture, it is not yet output to the format conversion circuit 32 at this time (as described above, the P picture input after the B picture is Processed and transmitted before the B picture).

When the image data supplied from the IDCT circuit 84 is B picture data, it is stored in the forward prediction image section 86a of the frame memory 86 corresponding to the prediction mode supplied from the variable length decoding circuit 82. I-picture image data (in the case of forward prediction mode), P-picture image data stored in the backward prediction image section 86b (in the case of backward prediction mode), or both image data (in the case of bidirectional prediction mode) Are read out, and the motion compensation circuit 87 performs motion compensation corresponding to the motion vector output from the variable length decoding circuit 82 to generate a predicted image. However, when motion compensation is not required (in the case of the intra-picture prediction mode), a predicted picture is not generated.

デ ー タ The data subjected to the motion compensation by the motion compensation circuit 87 in this way is added to the output of the IDCT circuit 84 in the arithmetic unit 85. This addition output is output to the format conversion circuit 32.

However, the added output is B-picture data, and is not stored in the frame memory 86 because it is not used for generating a predicted image of another image.

After the image of the 画像 B picture is output, the image data of the P picture stored in the backward prediction image section 86b is read and supplied to the arithmetic unit 85 via the motion compensation circuit 87. However, at this time, no motion compensation is performed.

Although the decoder 31 does not show circuits corresponding to the prediction mode switching circuit 52 and the DCT mode switching circuit 55 in the encoder 18 shown in FIG. 19, processing corresponding to these circuits, that is, odd field and even number The motion compensation circuit 87 executes a process of returning the configuration in which the signal of the line of the field is separated to the original mixed configuration as necessary.

In the above, the processing of the luminance signal has been described, but the processing of the chrominance signal is similarly performed. However, in this case, the motion vector used for the luminance signal is halved in the vertical and horizontal directions.

As described above, in the conventional image signal encoding and decoding methods, the resolution of the color difference signal is fixed to one type, and the color difference signals of a plurality of types of resolutions cannot be transmitted.

The present invention has been made in view of such a situation, and it is an object of the present invention to efficiently transmit color difference signals of a plurality of resolutions and easily obtain images of colors of a plurality of resolutions in real time. It is.

An image signal generation device according to the present invention is an image signal generation device that generates an image signal encoded in units of a predetermined macroblock, and generates an encoded luminance signal by encoding a luminance signal component of a macroblock. Encoding a color signal component of a macroblock at a first resolution to generate a first encoded color signal and outputting a first slice data signal including an encoded luminance signal and a first encoded color signal A first image encoding unit that converts a color signal component having a second resolution different from the first resolution of the macroblock into a first encoded color signal encoded by the first image encoding unit. A second image encoding unit that encodes using the decoded signal to generate a second encoded color signal, and outputs a second slice data signal including the second encoded color signal; Output from the image encoding means One slice data signal and the second slice data signal output from the second image encoding means are adjacent to each other so as to be sequentially read out during decoding processing and one of them is selected. And a combining means for arranging and combining at a position.

A second predictive image generating unit configured to generate a first predictive image signal for a color signal component of a second resolution based on a signal obtained by decoding the first coded color signal; Storing means for storing a second coded color signal relating to the coded image signal; and a second resolution of the second resolution based on the decoded signal of the second coded color signal stored in the storing means. A second predicted image generating means for generating a second predicted image signal for the color signal component, a prediction error signal when using the first predicted image signal generated by the first predicted image generating means, The second prediction image signal generated by the second prediction image generation means is compared with a prediction error signal when the second prediction image signal is used, and based on the comparison result, the first prediction image signal and the second prediction image signal are compared. To select one of Means for encoding a color signal component of the second resolution using the one of the first predicted image signal and the second predicted image signal selected by the selecting means, thereby forming a second encoded color signal. And a second coded color signal generating means for generating and outputting a second slice data signal including the second coded color signal.

The combining means can further multiplex and combine the result of the selection by the selecting means together with the first slice data signal and the second slice data signal.

The second image encoding unit includes a storage unit configured to store a second encoded color signal related to the encoded image signal, a signal obtained by decoding the first encoded color signal, and a second unit stored in the storage unit. And a signal synthesizing unit for performing predetermined weighting on the signal obtained by decoding the coded color signal of No. 2 and synthesizing the signal by a predetermined weight. It is possible to generate a second encoded color signal by encoding based on the signal, and to output a second slice data signal including the second encoded color signal.

In the second image encoding means, the signal synthesizing means includes a signal obtained by decoding the first encoded color signal with a plurality of weighting coefficients having different magnitudes, and a second encoded color signal stored in the storage means. A plurality of decoded signals are generated by weighting and synthesizing a signal obtained by decoding the coded color signal, and a prediction error signal is calculated for each of the plurality of weighting coefficients when a second encoded color signal is generated based on the decoded signal. And comparing the decoded signals to select which decoded signal is used for encoding the second encoded color signal.

The combining means can further multiplex and combine the weighting coefficients together with the first slice data signal and the second slice data signal.

An image signal generation method according to the present invention is an image signal generation method for generating an image signal encoded in units of a predetermined macroblock, and generates an encoded luminance signal by encoding a luminance signal component of a macroblock. Encoding a color signal component of a macroblock at a first resolution to generate a first encoded color signal and outputting a first slice data signal including an encoded luminance signal and a first encoded color signal A first image encoding step, and a color signal component having a second resolution different from the first resolution of the macroblock are converted into a first encoded color encoded by the processing of the first image encoding step. A second image encoding step of encoding the decoded signal to generate a second encoded color signal and outputting a second slice data signal including the second encoded color signal; First image coding The first slice data signal output by the processing of the step and the second slice data signal output by the processing of the second image encoding step are sequentially read out at the time of decoding processing, and either one of them is read out. And a combining step of arranging and combining them at positions close to each other so that is selected.

The second image encoding step includes a first predicted image generation step of generating a first predicted image signal for a color signal component of a second resolution based on a signal obtained by decoding the first encoded color signal. And a storage step of storing a second encoded color signal related to the encoded image signal; and a second encoding color signal stored in the storage step, based on the decoded second encoded color signal. A second prediction image generation step of generating a second prediction image signal for the color signal component of the resolution, and a prediction error when using the first prediction image signal generated by the processing of the first prediction image generation step The signal is compared with a prediction error signal in the case of using the second predicted image signal generated by the process of the second predicted image generation step, and based on the comparison result, the first predicted image signal and A selection step of selecting one of the two predicted image signals, and a color signal component of the second resolution is selected by the processing of the selection step of the first predicted image signal and the second predicted image signal Generating a second encoded color signal by performing encoding using the encoded signal, and outputting a second slice data signal including the second encoded color signal. You can do so.

In the combining step, the result of the selection processing in the selecting step can be further multiplex-combined together with the first slice data signal and the second slice data signal.

The second image encoding step includes: a storage step of storing a second encoded color signal relating to the encoded image signal; a signal obtained by decoding the first encoded color signal; And a signal synthesizing step of performing predetermined weighting on a signal obtained by decoding the second encoded color signal and synthesizing the color signal component of the second resolution by the signal synthesizing step. It is possible to generate a second encoded color signal by encoding based on the decoded signal, and output a second slice data signal including the second encoded color signal.

In the second image encoding step, the signal synthesizing step includes a signal obtained by decoding the first encoded color signal with a plurality of weighting coefficients having different magnitudes, and a second encoding stored by the processing of the storing step. A signal obtained by decoding a color signal is weighted and synthesized to generate a plurality of decoded signals, and for each of a plurality of weighting coefficients, a prediction error signal generated when a second encoded color signal is generated based on the decoded signal. The method may further include a selecting step of calculating and comparing the respective decoded signals to select which decoded signal is used for encoding the second encoded color signal.

In the combining step, the weighting coefficients can be further multiplexed together with the first slice data signal and the second slice data signal and output.

In the image signal generating apparatus and method according to the present invention, the image signal is encoded on a macroblock basis to generate an image signal. More specifically, the luminance signal component of the macroblock is encoded to generate an encoded luminance signal, and the color signal component of the macroblock is encoded at a first resolution to generate a first encoded color signal. Then, a first slice data signal including the encoded luminance signal and the first encoded color signal is output. Further, a color signal component having a second resolution higher than the first resolution of the macroblock is obtained by using a signal obtained by decoding the first encoded color signal encoded by the processing of the first image encoding step. Encoding is performed to generate a second encoded color signal, and a second slice data signal including the second encoded color signal is output. Then, the outputted first slice data signal and the outputted second slice data signal are arranged at positions close to each other such that they are sequentially read out during decoding processing and one of them is selected. And synthesized.

An image signal reproducing apparatus according to the present invention is an image signal encoded in a predetermined macroblock unit, wherein an encoded luminance in which a luminance signal component of a predetermined macroblock of the encoded image signal is encoded. A first slice data signal including a signal and a first encoded color signal in which a color signal component of a macroblock is encoded at a first resolution; and a second resolution different from the first resolution of the macroblock. And a second slice data signal including a second encoded chrominance signal encoded using a signal obtained by decoding the first encoded chrominance signal are arranged at positions close to each other. An image signal reproducing apparatus for reproducing an image signal supplied thereto, sequentially reproducing a first slice data signal and a second slice data signal corresponding to a macro block, and reproducing the reproduced first slice data signal. Separating means for separating the data signal from the second slice data signal, decoding the encoded luminance signal from the first slice data signal to reproduce the luminance signal component, and decoding the first encoded color signal. A first image decoding unit for reproducing a first color signal component having a first resolution, and a second slice data signal using the first color signal component decoded by the first image decoding unit. A second image decoding means for decoding a second encoded color signal from the second color signal to reproduce a second color signal component of a second resolution, and a first color signal reproduced by the first image decoding means And a selecting unit that selects a color signal component used in a macroblock decoding process from the components and the second color signal component reproduced by the second image decoding unit.

The second image decoding unit includes a first predicted image generation unit configured to generate a first predicted image signal corresponding to the second encoded color signal based on the first color signal component; Storage means for storing the second color signal component, and a second predicted image for generating a second predicted image signal for the second encoded color signal based on the second color signal component stored in the storage means Generating means, selecting one of a first predicted image signal generated by the first predicted image generating means and a second predicted image signal generated by the second predicted image generating means; The predicted image selecting means and the second coded color signal are decoded by using the one of the first predicted image signal and the second predicted image signal selected by the predicted image selecting means to obtain a second color signal. And second color signal reproducing means for reproducing a signal component. Door can be.

The image signal including the first slice data signal and the second slice data signal is further multiplexed with a command instructing which of the first predicted image signal and the second predicted image signal to use. The predicted image selection means is synthesized and can select one of the first predicted image signal and the second predicted image signal based on the instruction.

The second image decoding unit includes a first predicted image generation unit that generates a first predicted image signal for the second encoded color signal based on the first color signal component, and a second predicted image signal that has already been decoded. Storage means for storing the color signal component of the second color signal component, and second predicted image generation for generating a second predicted image signal for the second encoded color signal based on the second color signal component stored in the storage means Means, a predicted image synthesizing means for outputting a synthesized signal obtained by performing predetermined weighting on the first predicted image signal and the second predicted image signal, and outputting the second encoded color signal to the synthesized signal. And a second color signal reproducing means for reproducing the second color signal component by decoding using the second color signal component.

The image signal including the first slice data signal and the second slice data signal is further multiplex-synthesized with a weighting coefficient used in the process of synthesizing the predicted image signal by the predicted image synthesizing unit. , The first predicted image signal and the second predicted image signal are weighted and synthesized based on the weighting coefficient, and a synthesized signal can be output.

An image signal reproducing method according to the present invention is an image signal coded in a predetermined macroblock unit, wherein a coded luminance in which a luminance signal component of a predetermined macroblock in the coded image signal is coded. A first slice data signal including a signal and a first encoded color signal in which a color signal component of a macroblock is encoded at a first resolution; and a second resolution different from the first resolution of the macroblock. And a second slice data signal including a second encoded chrominance signal encoded using a signal obtained by decoding the first encoded chrominance signal are arranged at positions close to each other. An image signal reproducing method for reproducing the supplied image signal, wherein a first slice data signal and a second slice data signal corresponding to a macro block are sequentially reproduced, and the reproduced first slice data signal is reproduced. Separating an encoded luminance signal from the first slice data signal to reproduce a luminance signal component, and decoding a first encoded color signal from the first slice data signal. A first image decoding step of reproducing a first color signal component having a first resolution, and a second slice using the first color signal component decoded by the processing of the first image decoding step. A second image decoding step of decoding a second encoded color signal from the data signal to reproduce a second color signal component of a second resolution; and a second image decoding step reproduced by the processing of the first image decoding step. A selection step of selecting a color signal component used for macroblock decoding processing from among the first color signal component and the second color signal component reproduced by the processing of the second image decoding step. It is characterized by.

The second image decoding step includes a first predicted image generation step of generating a first predicted image signal for the second encoded color signal based on the first color signal component; And a second prediction for generating a second predicted image signal for the second encoded color signal based on the second color signal stored in the storing step of storing the second color signal component. Any of the image generation step, the first predicted image signal generated by the processing of the first predicted image generation step, and the second predicted image signal generated by the processing of the second predicted image generation step A predicted image selecting step of selecting one of the two, and a second coded color signal is determined by using one of the first predicted image signal and the second predicted image signal which is selected by the processing of the predicted image selecting step. Decrypt and second It can be made to contain a second color signal reproducing step of reproducing the signal components.

The image signal including the first slice data signal and the second slice data signal is further multiplexed with a command instructing which of the first predicted image signal and the second predicted image signal to use. The predicted image selection step is a step of selecting one of the first predicted image signal and the second predicted image signal based on the instruction.

The second image decoding step includes a first predicted image generation step of generating a first predicted image signal for the second encoded color signal based on the first color signal component, and a second predicted image signal that has already been decoded. And a second prediction for generating a second predicted image signal for the second encoded color signal based on the second color signal component stored by the processing of the storage step. An image generating step, a predicted image synthesizing step of outputting a synthesized signal obtained by performing predetermined weighting on the first predicted image signal and the second predicted image signal, and outputting the second encoded color signal, And a second color signal reproduction step of reproducing the second color signal component by decoding using the synthesized signal.

The image signal including the first slice data signal and the second slice data signal is further multiplex-synthesized with a weighting coefficient used in the process of synthesizing the predicted image signal in the predicted image synthesizing step. , The first predicted image signal and the second predicted image signal are weighted and synthesized based on the weighting coefficient, and a synthesized signal can be output.

In the image signal reproducing apparatus and method according to the present invention, an encoded luminance signal obtained by encoding a luminance signal component of a predetermined macroblock in an encoded image signal, and a color signal component of the macroblock are set to a first signal. A first slice data signal including a first encoded color signal encoded at a resolution and a second encoded color encoded using a signal obtained by decoding the first encoded color signal The second slice data signal including the signal is arranged at a position close to each other, and the provided image signal is reproduced. More specifically, a first slice data signal and a second slice data signal corresponding to a macroblock are sequentially reproduced, and the reproduced first slice data signal and the reproduced second slice data signal are separated. . From the first slice data signal, an encoded luminance signal is decoded to reproduce a luminance signal component, and a first encoded chrominance signal is decoded to reproduce a first color signal component of a first resolution. You. On the other hand, the second encoded color signal is decoded from the second slice data signal using the decoded first color signal component, and the second color signal component of the second resolution is reproduced. . Then, a color signal component used for macroblock decoding processing is selected from the reproduced first color signal component and second color signal component.

As described above, according to the present invention, in an apparatus for decoding an image corresponding to a low-resolution color signal, only a circuit for processing a high-resolution color signal component is added. It is possible to easily obtain a color image and a high-resolution color image in real time.

(5) Embodiments of the present invention will be described below. The correspondence between constituent elements described in the claims and specific examples in the embodiments of the present invention is as follows. This description is for confirming that a specific example supporting the invention described in the claims is described in the embodiment of the invention. Therefore, even if there is a specific example which is described in the embodiment of the invention but is not described here as corresponding to the configuration requirement, the fact that the specific example is It does not mean that it does not correspond to the requirement. Conversely, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

Further, this description does not mean that the invention corresponding to the specific examples described in the embodiments of the invention is all described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of the invention not described in the claims of this application, that is, It does not deny the existence of the invention added by the amendment.

According to the present invention, an image signal generating device is provided. This image signal generation device (for example, the image signal generation device in FIG. 10 or FIG. 12) is an image signal generation device that generates an image signal encoded in a predetermined macroblock unit, and includes a luminance signal component of a macroblock. (Eg, Y1 to Y4 in FIG. 10 or FIG. 12) to generate an encoded luminance signal (eg, <Y1>, <Y2>, <Y3>, <Y4> in FIG. 10 or FIG. 12). Encoding the color signal components of the macroblock at a first resolution to generate a first encoded color signal (for example, encoding Cb5 ″ and Cr6 ″ in FIG. 10 or 12 to <cb5 ''> And <cr6 ″>), and outputs a first slice data signal (for example, a master slice in FIG. 8) including the encoded luminance signal and the first encoded chrominance signal. 1 image encoding means (for example, the circuit 100 of FIG. 10 or FIG. 12) and the macro The color signal components (for example, Cb5 ', Cr6', Cb7 ', Cr8' in FIG. 10 or FIG. 12) of the block having a second resolution different from the first resolution are converted by the first image encoding means. The encoded first chrominance signal is encoded using a decoded signal (for example, a signal from the arithmetic unit 62 of the circuit 100) to form a second encoded chrominance signal (for example, FIG. 10 or FIG. 10). 12 <Cb5 ′>, <Cr6 ′>, <Cb7 ′>, <Cr8 ′>), and generates a second slice data signal (for example, the slave slice of FIG. 8) including the second encoded color signal. (1) a second image encoding means (for example, the circuit 101 in FIG. 10 or FIG. 12), the first image encoding means, the first slice data signal output from the first image encoding means, And the second slice data signal output from the image encoding means of (1) is sequentially read at the time of decoding processing. A synthesizing unit (for example, the synthesizing circuit 105 in FIG. 10) that arranges them at positions close to each other so as to select one of them (for example, as shown in FIG. 8) so that one of them is selected. And characterized in that:

The second image encoding means (for example, the circuit 101 in FIG. 10) of the image signal generating apparatus, based on the signal obtained by decoding the first encoded color signal, outputs the color of the second resolution. First predicted image generation means (for example, the upsampling circuit 111 in FIG. 10) for generating a first predicted image signal for a signal component, and the second encoded color signal for an encoded image signal is stored. Based on a signal obtained by decoding the second encoded color signal stored in the storage unit (for example, the chroma frame memory 174 of FIG. 10) and the second encoded color signal stored in the storage unit, the color signal component of the second resolution is obtained. A second predicted image generating means (for example, the motion compensation circuit 175 in FIG. 10) for generating a second predicted image signal and the first predicted image signal generated by the first predicted image generating means are used. Is compared with the prediction error signal when the second predicted image signal generated by the second predicted image generation means is used, and based on the result of the comparison, A selection unit (for example, a selection circuit 176 in FIG. 10) for selecting one of the first predicted image signal and the second predicted image signal, and the color signal component of the second resolution The second coded color signal is generated by encoding using one of the first predicted image signal and the second predicted image signal, which is selected by the selection unit, to generate the second coded color signal. And a second coded color signal generating means (for example, the arithmetic unit 112 to the variable length coding circuit 115 in FIG. 10) for outputting the second slice data signal.

The synthesizing means (for example, the circuit 105 in FIG. 10) of the image signal generating apparatus outputs the result of the selection by the selecting means (for example, FIG. 10) together with the first slice data signal and the second slice data signal. Space / Time flag) can be further multiplex-combined.

The second image encoding means (for example, the circuit 101 in FIG. 12) of the image signal generating device stores the second encoded color signal related to the encoded image signal (for example, FIG. 12). Chroma frame memory 174), a signal obtained by decoding the first coded color signal, and a signal obtained by decoding the second coded color signal stored in the storage means, respectively, are given predetermined weights. And a signal synthesizing unit for performing and synthesizing (for example, the up-sampling circuit 111, the motion compensation circuit 175, the weighting circuit 191, the weighting circuit 192, and the arithmetic unit 193 in FIG. 12), and the second resolution color The signal component is encoded based on the decoded signal weighted and synthesized by the signal synthesis unit to generate the second encoded color signal, and a second slice including the second encoded color signal is generated. Can output data signal.

In the second image encoding unit of the image signal generation device, the signal synthesizing unit includes a plurality of weighting coefficients (for example, coefficients (1-W) in FIG. In FIG. 12, the coefficient W in FIG. 12 is 1, 3/4, 2/4, 1/4, 0, and 0, 1/4, 2/4, 3/4, 1). A signal obtained by decoding an encoded color signal and a signal obtained by decoding the second encoded color signal stored in the storage unit are weighted and combined to generate a plurality of decoded signals, and a plurality of weighted coefficients are respectively generated. , A prediction error signal when the second coded color signal is generated based on the decoded signal is calculated and compared, and any decoded signal is used for coding the second coded color signal. Further, there is further provided a selecting means (for example, a computing unit 193 in FIG. 12) for selecting whether the user Door can be.

The synthesizing means (for example, the circuit 105 in FIG. 12) of the image signal generating apparatus includes the first slice data signal and the second slice data signal together with the weighting factor (for example, Weighting Factor: W in FIG. 10). ) Can be further multiplexed and output.

According to the present invention, an image signal generating method is provided. This image signal generation method (for example, the image signal generation method of the image signal generation device of FIG. 10 or FIG. 12) is an image signal generation method of generating an image signal encoded in units of a predetermined macroblock, The luminance signal components of the block (for example, Y1 to Y4 in FIG. 10 or FIG. 12) are encoded, and the encoded luminance signals (for example, <Y1>, <Y2>, <Y3>, <Y4> in FIG. 10 or FIG. 12). ), And encodes the color signal component of the macroblock at a first resolution to generate a first encoded color signal (for example, Cb5 ″ and Cr6 ″ in FIG. 10 or FIG. 12). Encoding to generate <cb5 ″> and <cr6 ″>), and a first slice data signal (for example, the master slice of FIG. 8) including the encoded luminance signal and the first encoded chrominance signal. 1) (for example, the circuit of FIG. 10 or FIG. 12) 100) and a color signal component of a second resolution different from the first resolution of the macroblock (for example, Cb5 ', Cr6', Cb7 ', Cr8' in FIG. 10 or FIG. 12). ) Is encoded by using a signal (for example, a signal from the arithmetic unit 62 of the circuit 100) obtained by decoding the first encoded color signal encoded by the processing of the first image encoding step. Generate a second coded color signal (for example, <Cb5 ′>, <Cr6 ′>, <Cb7 ′>, <Cr8 ′> in FIG. 10 or FIG. 12) and include the second coded color signal. A second image encoding step (for example, a step corresponding to the circuit 101 in FIG. 10 or 12) for outputting a second slice data signal (for example, slave slice 1 in FIG. 8), and the first image code The first slice data signal output by the processing of the And the second slice data signal output by the processing of the second image encoding step is arranged in a position close to each other so as to be sequentially read out during the decoding processing and one of them is selected. (For example, a step corresponding to the processing of the combining circuit 105 in FIG. 10) for combining (for example, arranged as shown in FIG. 8).

The second image encoding step (for example, a step corresponding to the processing of the circuit 101 in FIG. 10) of the image signal generation method includes the step of: A first predicted image generation step of generating a first predicted image signal for the color signal component of the second resolution (for example, a step corresponding to the processing of the upsampling circuit 111 in FIG. 10), and an encoded image A storing step (for example, a step corresponding to the processing of the chroma frame memory 174 in FIG. 10) of storing the second encoded color signal relating to the signal, and a step of storing the second encoded color signal stored in the storage unit. A second predicted image generating step of generating a second predicted image signal for the color signal component of the second resolution based on the decoded signal (for example, FIG. Corresponding to the processing of the compensation circuit 175), a prediction error signal when the first predicted image signal generated by the processing of the first predicted image generation step is used, and the second predicted image Comparing a second prediction image signal generated by the processing in the generation step with a prediction error signal when the second prediction image signal is used, and comparing the first prediction image signal and the second prediction image based on a result of the comparison; A selection step of selecting one of the signals (for example, a step corresponding to the processing of the selection circuit 176 in FIG. 10), and the color signal component of the second resolution is combined with the first predicted image signal. The second coded color signal is generated by encoding using the one of the second predicted image signals selected by the process of the selecting step, and the second coded color signal including the second coded color signal is generated. Second encoded color signal generating step of outputting the slice data signal (e.g., steps corresponding to the processing of the arithmetic unit 112 to variable-length coding circuit 115 in FIG. 10) may include a.

The synthesizing step (for example, the processing corresponding to the circuit 105 in FIG. 10) of the image signal generating method includes the first slice data signal and the second slice data signal together with the result of the selection processing in the selecting step ( For example, the Space / Time flag in FIG. 10 can be further multiplex-combined.

The second image encoding step (e.g., a step corresponding to the processing of the circuit 101 in FIG. 12) of the image signal generation method includes storing the second encoded color signal related to an encoded image signal. A step (for example, a step corresponding to the processing of the chroma frame memory 174 in FIG. 12), a signal obtained by decoding the first coded color signal, and the second coded color stored by the processing of the storing step. A signal synthesizing step (eg, the up-sampling circuit 111, the motion compensation circuit 175, the weighting circuit 191, the weighting circuit 192, and the arithmetic unit 193 in FIG. Corresponding to the respective processes of the above), and the color signal component of the second resolution is superimposed by the process of the signal synthesizing step. Assigned by encoding based on the combined decoded signal to generate the second encoded color signals, can output the second slice data signal including the second encoded color signals.

In the second image encoding step of the image signal generating apparatus, the signal synthesizing step includes a plurality of weighting coefficients (for example, coefficients (1-W) in FIG. In FIG. 12, the coefficient W in FIG. 12 is 1, 3/4, 2/4, 1/4, 0, and 0, 1/4, 2/4, 3/4, 1). A signal obtained by decoding an encoded color signal and a signal obtained by decoding the second encoded color signal stored in the storage unit are weighted and combined to generate a plurality of decoded signals, and a plurality of weighted coefficients are respectively generated. , A prediction error signal when the second coded color signal is generated based on the decoded signal is calculated and compared, and any decoded signal is used for coding the second coded color signal. A selection step (for example, of the arithmetic unit 193 in FIG. 12) It may further comprise the step) corresponding sense.

In the image signal generating method, the synthesizing step (for example, processing corresponding to the circuit 105 in FIG. 12) includes the weighting coefficient (for example, in FIG. 10) together with the first slice data signal and the second slice data signal. Weighting Factor (W) can be further multiplexed and output.

According to the present invention, an image signal reproducing device is provided. This image signal reproducing apparatus (for example, the image signal reproducing apparatus shown in FIG. 11 or FIG. 13) is an image signal encoded in a predetermined macro block unit, and a predetermined macro of the encoded image signal. An encoded luminance signal (for example, <Y1>, <Y2>, <Y3>, <Y4 in FIG. 10 or FIG. 12) in which the luminance signal components of the block (for example, Y1 to Y4 in FIG. 10 or 12) are encoded. >), And a first encoded color signal (for example, Cb5 ″ and Cr6 ″ in FIG. 10 or FIG. 12) in which the color signal components of the macroblock are encoded at a first resolution. <cb5 ″> and <cr6 ″>), and a color signal having a second resolution different from the first resolution of the macroblock (for example, the master slice in FIG. 8). The components (for example, Cb5 ', Cr6', Cb7 ', Cr8' in FIG. 10 or FIG. 12) are the first codes A second encoded color signal (for example, <Cb5 ′>, <Cr6 in FIG. 10 or FIG. 12) encoded using a signal obtained by decoding a color signal (for example, a signal from the arithmetic unit 62 of the circuit 100). '>, <Cb7'>, <Cr8 '>) and a second slice data signal (for example, slave slice 1 in FIG. 8) are arranged at positions close to each other (for example, shown in FIG. 8). And an image signal reproducing apparatus for reproducing the supplied image signal.

Specifically, the image signal reproducing apparatus sequentially reproduces the first slice data signal and the second slice data signal corresponding to the macroblock, and reproduces the reproduced first slice data signal and the second slice data signal. Separating means (for example, the separating circuit 150 of FIG. 11 or 13) for separating the two slice data signals, and decoding the encoded luminance signal from the first slice data signal to reproduce the luminance signal component A first image decoding unit (for example, the circuit 161 of FIG. 11 or 13) that decodes the first encoded color signal and reproduces a first color signal component of the first resolution; The second coded color signal is decoded from the second slice data signal using the first color signal component decoded by the first image decoding means, and the second encoded color signal is decoded at the second resolution. A second image decoding means (for example, the circuit 162 in FIG. 11 or FIG. 13) for reproducing the color signal component of the first color signal component, the first color signal component reproduced by the first image decoding means, Selecting means (for example, the selecting circuit 164 in FIG. 11 or FIG. 13) for selecting, from the second color signal components reproduced by the image decoding means, a color signal component used for decoding the macroblock. And characterized in that:

The second image decoding means (for example, the circuit 162 in FIG. 11) of the image signal reproducing apparatus, based on the first color signal component, generates a first predicted image for the second encoded color signal. First predictive image generating means for generating a signal (for example, the upsampling circuit 151 in FIG. 11) and storing means for storing the already decoded second color signal component (for example, the chroma frame memory 181 in FIG. 11) ) And a second predicted image generating means (for example, FIG. 3) for generating a second predicted image signal for the second encoded color signal based on the second color signal component stored in the storing means. 11 motion compensation circuits 182), the first predicted image signal generated by the first predicted image generating means, and the second predicted image signal generated by the second predicted image generating means. Out of A predictive image selecting means (for example, a selection circuit 183 in FIG. 11) for selecting one of the first and second predictive image signals; Second color signal reproducing means (for example, the variable length decoding circuit 152 to the arithmetic unit 155 in FIG. 11) for decoding using the one selected by the predicted image selecting means and reproducing the second color signal component. And can have.

In this image signal reproducing device, the image signal including the first slice data signal and the second slice data signal includes the first predicted image signal and the second predicted image signal. (For example, Space / Time flag in FIG. 11) is further multiplex-synthesized, and the predicted image selecting means outputs the first predicted image signal and the first predicted image signal based on the instruction. One of the second predicted image signal and the second predicted image signal can be selected.

The second image decoding means (for example, the circuit 162 in FIG. 13) of the image signal reproducing apparatus, based on the first color signal component, generates a first predicted image signal for the second encoded color signal. (For example, the up-sampling circuit 151 in FIG. 13) and storage means for storing the already decoded second color signal component (for example, the chroma frame memory 181 in FIG. 13). And a second predicted image generating means for generating a second predicted image signal for the second encoded color signal based on the second color signal component stored in the storing means (for example, FIG. Motion compensation circuit 182), and a predicted image synthesizing unit (for example, FIG. 13) that outputs a synthesized signal obtained by performing weighting on the first predicted image signal and the second predicted image signal, respectively, and synthesizing them. And a second color for reproducing the second color signal component by decoding the second encoded color signal using the composite signal. Signal reproducing means (for example, the variable length decoding circuit 152 to the arithmetic unit 155 in FIG. 13) can be provided.

In this image signal reproducing apparatus, the image signal including the first slice data signal and the second slice data signal is provided with a weighting coefficient ( For example, the weighting factor (W) of FIG. 13 is further multiplex-combined, and the predicted image combining unit weights and combines the first predicted image signal and the second predicted image signal based on the weighting coefficient. Thus, the composite signal can be output.

According to the present invention, an image signal reproducing method is provided. This image signal reproducing method (for example, the image signal reproducing method of the image signal reproducing apparatus of FIG. 11 or FIG. 13) is an image signal encoded in a predetermined macroblock unit, and The luminance signal components (for example, Y1 to Y4 in FIG. 10 or FIG. 12) of the predetermined macroblock are encoded luminance signals (for example, <Y1>, <Y2>, <Y2 in FIG. 10 or FIG. 12). Y3>, <Y4>) and a first encoded color signal (for example, Cb5 ″ and Cr6 ″ in FIG. 10 or FIG. 12) in which the color signal components of the macroblock are encoded at a first resolution. A first slice data signal (for example, a master slice in FIG. 8) including <cb5 ″> and <cr6 ″>) encoded with a second resolution different from the first resolution of the macroblock. (For example, Cb5 ', Cr6', Cb7 ', Cr8' in FIG. 10 or FIG. 12) , A second encoded color signal (for example, as shown in FIG. 10 or FIG. 12) encoded using a signal obtained by decoding the first encoded color signal (for example, a signal from the arithmetic unit 62 of the circuit 100). A second slice data signal (for example, slave slice 1 in FIG. 8) including <Cb5 ′>, <Cr6 ′>, <Cb7 ′>, <Cr8 ′>) is arranged at a position close to each other ( For example, there is an image signal reproducing method for reproducing the supplied image signal (disposed as shown in FIG. 8).

Specifically, the image signal reproducing method sequentially reproduces the first slice data signal and the second slice data signal corresponding to the macroblock, and reproduces the reproduced first slice data signal and the second slice data signal. A separating step (for example, a step corresponding to the processing of the separating circuit 150 in FIG. 11 or FIG. 13) for separating the two slice data signals, and decoding the encoded luminance signal from the first slice data signal. A first image decoding step of reproducing the luminance signal component and decoding the first encoded color signal to reproduce a first color signal component of the first resolution (for example, FIG. 11 or FIG. 13); Using the first color signal component decoded by the processing of the first image decoding step), and performing the second slice decoding using the first color signal component decoded by the processing of the first image decoding step. A second image decoding step of decoding the second coded color signal from the data signal and reproducing a second color signal component of the second resolution (for example, the processing of the circuit 162 in FIG. 11 or FIG. 13). ), The first color signal component reproduced by the processing of the first image decoding step, and the second color signal component reproduced by the processing of the second image decoding step. And a selecting step (for example, a step corresponding to the selecting circuit 164 in FIG. 11 or FIG. 13) of selecting a color signal component used in the decoding processing of the macro block.

The second image decoding step (e.g., a step corresponding to the processing of the circuit 162 in FIG. 11) of the image signal reproducing method includes the step of performing the second encoded color signal based on the first color signal component. A first predicted image signal generating step (eg, a step corresponding to the processing of the up-sampling circuit 151 in FIG. 11) for generating a first predicted image signal for the second color signal component and the second color signal component already decoded A storing step (for example, a step corresponding to the processing of the chroma frame memory 181 in FIG. 11) and the second encoded color signal based on the second color signal component stored by the processing of the storing step A second predicted image generating step of generating a second predicted image signal (for example, a step corresponding to the motion compensation circuit 182 in FIG. 11); Prediction image selection for selecting one of the first predicted image signal generated by the processing of the step and the second predicted image signal generated by the processing of the second predicted image generation step A step (for example, a step corresponding to the processing of the selection circuit 183 in FIG. 11), and converting the second encoded color signal into the predicted image of the first predicted image signal and the second predicted image signal A second color signal reproducing step of decoding using the one selected by the processing of the selecting step to reproduce the second color signal component (for example, the processing of the variable length decoding circuit 152 to the arithmetic unit 155 in FIG. 11) Corresponding to the above).

In this image signal reproducing method, the image signal including the first slice data signal and the second slice data signal includes the first predicted image signal and the second predicted image signal. (For example, Space / Time flag in FIG. 11) is further multiplex-combined, and the predicted image selecting step includes the first predicted image signal and the first predicted image signal based on the instruction. One of the second predicted image signal and the second predicted image signal can be selected.

In the image signal reproducing method, the second image decoding step (for example, a step corresponding to the processing of the circuit 162 in FIG. 13) is performed on the second encoded color signal based on the first color signal component. A first predicted image generation step for generating a first predicted image signal (for example, a step corresponding to the processing of the upsampling circuit 151 in FIG. 13), and storage for storing the already decoded second color signal component A second predicted image signal for the second encoded color signal based on a step (for example, the chroma frame memory 181 in FIG. 13) and the second color signal component stored by the processing of the storing step. A second predicted image generating step (for example, a step corresponding to the processing of the motion compensation circuit 182 in FIG. 13), and the first predicted image signal and the second predicted image A predicted image synthesizing step (for example, a step corresponding to the processing of the weighting circuit 201, the weighting circuit 202, and the arithmetic unit 203 in FIG. 13) for outputting a synthesized signal synthesized by performing predetermined weighting on the image signal and A second chrominance signal reproducing step of decoding the second coded chrominance signal using the synthesized signal to reproduce the second chrominance signal component (for example, the variable length decoding circuits 152 to 152 in FIG. 13). Step corresponding to the processing of the arithmetic unit 155).

In this image signal reproducing method, the image signal including the first slice data signal and the second slice data signal is provided with a weighting coefficient (for example, , And Weighting Factor (W) in FIG. 13 are further multiplex-combined, and in the predicted image combining step, the first predicted image signal and the second predicted image signal are weighted and combined based on the weighting coefficients. Thus, the composite signal can be output.

Hereinafter, the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the overall configuration of an image signal encoding device and a decoding device according to the present invention, and portions corresponding to those in the conventional case shown in FIG. 17 are denoted by the same reference numerals. In the present embodiment, the A / D conversion timing (sampling timing) of the color difference signal output by the pre-processing circuit 11 in the A / D converter 300 is different from that in the A / D converter 13 in FIG. I have. As a result, the chrominance signal frame memory 301, the format conversion circuit 302, and the encoder (image signal generation device) 303, which process the chrominance signal output from the A / D converter 300 at the subsequent stage, are configured differently from the conventional case. I have.

Furthermore, in the decoding device 2, the decoder (image signal reproducing device) 401, the format conversion circuit 402, the color difference signal frame memory 403, and the D / A converter 404 are the decoder 31, the format conversion circuit shown in FIG. 32, a color difference signal frame memory 35, and a D / A converter 37.

Other configurations are the same as those in FIG.

(2) In the A / D converter 300 of the present invention, sampling is performed as shown in FIG. That is, assuming that the sampling points of the luminance signal in the A / D converter 12 are indicated by circles in the figure, the sampling points of the color difference signals performed by the A / D converter 300 are indicated by crosses in the figure. As shown in the figure, the sampling point of the luminance signal corresponds to 1: 1 (4: 4: 4 sampling).

The color difference signal sampled by the A / D converter 300 is supplied to the color difference signal frame memory 301 and stored. The color difference signal read from the color difference signal frame memory 301 is input to the format conversion circuit 302 and down-sampled.

That is, the format conversion circuit 302 incorporates, for example, a down-sampling circuit as shown in FIG. 3, and converts the color difference signal sampled by the A / D converter 300 at a ratio of 4: 4: 4 into a low-pass filter 131. Then, the data of each line is thinned every other line by the thinning circuit 132. As a result, 4: 2: 2 sampling is performed as shown in FIG. That is, in this case, in each line, one color difference signal corresponds to two luminance signals.

The format conversion circuit 302 down-samples the color difference signal sampled at the ratio of 4: 2: 2 in this manner by the built-in down-sampling circuit in the same manner as shown in FIG. 2C. A color difference signal of a sampling ratio of 4: 2: 0 is generated. In this case, since all the color difference signals of 4: 2: 2 sampling are thinned out every other line, one color difference signal corresponds to four luminance signals.

In the above description, the sampling ratio is changed by simply thinning out the data. However, it is also possible to perform sub-sampling by, for example, averaging the color difference signals at a plurality of predetermined positions. For example, by averaging the four color difference signals shown in FIG. 2A, it is possible to obtain the color difference signals indicated by crosses in parentheses in FIG. 2C.

The format conversion circuit 302 blocks the color difference signals generated in this manner into the signals of different resolution levels together with the corresponding luminance signal data. As a result, as shown in FIG. 4, a configuration of three types of macroblocks of 4: 4: 4, 4: 2: 2, or 4: 2: 0 becomes possible.

As shown in FIG. 4A, the 4: 4: 4 macro block includes four luminance blocks Y1 to Y4, corresponding Cb color difference blocks Cb5, Cb7, Cb9, Cb11, and a Cr color difference block. It is composed of Cr6, Cr8, Cr10 and Cr12. On the other hand, as shown in FIG. 4B, in the 4: 2: 2 macro block, the luminance block is the same as that in the 4: 4: 4 macro block shown in FIG. 4A. However, the color difference block Cb is composed of Cb5 'and Cb7'. The color difference block Cr is composed of Cr6 ′ and Cr8 ′. Further, as shown in FIG. 4C, in the 4: 2: 0 macroblock, the luminance block is the same as in the 4: 4: 4 macroblock, but the color difference block Cb is Cb5 ″. , And the color difference block Cr is composed of Cr6 ″.

Note that the number attached to the code of each block indicates the order of transmission in the case of transmitting that data within each macroblock. Also, 'indicates that it is down-sampled data, and' indicates that down-sampling has been performed twice. Therefore, for example, Cb5 'directly down-samples Cb5. (This is generated by down-sampling Cb5 and Cb9 as described above).

The reason why the transmission order of the chrominance data of the 4: 2: 2 macroblock in FIG. 4B is not such that Cb5 ′ is transmitted after Cb5 ′, as shown in FIG. This is to make it correspond to the transmission order in the 2: 0 macro block. That is, in the macroblock shown in FIG. 4C, Cr6 "is transmitted after Cb5". Therefore, even in the 4: 2: 2 macroblock shown in FIG. 4B, Cr6 'is transmitted next to Cb5'.

Similarly, the transmission order in the 4: 4: 4 macroblock shown in FIG. 4A is determined so as to correspond to the transmission order of the 4: 2: 2 macroblock shown in FIG. 4B. . By doing so, it is possible to perform processing by a common circuit in the encoder, regardless of which ratio of macroblocks has been transmitted.

However, in the present embodiment, these three types of macroblocks are not transmitted to the encoder 303, but four luminance blocks Y1 to Y4 and two chrominance blocks Cb5 ″, shown in FIG. Among the blocks forming the 4: 2: 0 macroblock composed of Cr6 ″ and the 4: 2: 2 macroblock shown in FIG. 4B, the chrominance blocks Cb5 ′, Cb7 ′, And the chrominance blocks Cr6 'and Cr8' and the chrominance blocks Cb5, Cb7, Cb9 and Cb11 excluding the luminance block, and the chrominance blocks Cr6, Cr8 and Cr10 of the 4: 4: 4 macroblock shown in FIG. 4A. , Cr12 are transmitted to the encoder 303.

The encoder 303 is configured, for example, as shown in FIG. Note that the motion vector detection circuit 50, the frame memory 51, the prediction mode switching circuit 52, the prediction determination circuit 54, the DCT mode switching circuit 55, the transmission buffer 59, and the like in FIG. 19 are omitted from FIG. The connection is similarly made in the embodiment.

In the embodiment of FIG. 5, the frame memory 63 is a luma (luminance signal) frame memory 63L and a chroma (color difference signal) frame memory 63C, and the motion compensation circuit 64 is a motion compensation circuit 64L and a motion compensation circuit. Although the image data is divided into 64C and is displayed in the example of FIG. 19, this is integrally shown in the example of FIG. 19, and also in the apparatus of FIG. 2) are prepared for the signal (signal) and for the chroma (color difference signal).

Further, in the embodiment of FIG. 5, the luma frame memory 63L and the chroma frame memory 63C each have a forward prediction image portion and a backward prediction image portion inside similarly to the case shown in FIG. is there.

That is, in the embodiment of FIG. 5, the circuit 100 composed of the motion vector detection circuit 50 to the motion compensation circuit 64 has basically the same configuration as the case of FIG.

This circuit 100 processes a color difference signal having the lowest resolution when focusing on the color difference signal. In this embodiment, as a circuit for processing a color difference signal having a higher resolution than the color difference signal in the circuit 100, an up-sampling circuit 111, a computing unit 112, a DCT circuit 113, a quantization circuit 114, a variable length A circuit 101 including an encoding circuit 115 is provided. The circuit 102 for processing a color difference signal having a higher resolution than the color difference signal in the circuit 101 includes an inverse quantization circuit 121, an IDCT circuit 122, an arithmetic unit 123, an upsampling circuit 124, an arithmetic unit 125, a DCT circuit 126, A circuit 127 and a variable length encoding circuit 128 are provided.

The circuit 102 receives a color difference signal having the highest resolution, and the circuit 101 receives a color difference signal having a low resolution, which is obtained by down-sampling the color difference signal input to the circuit 102 by the down sampling circuit 103. The circuit 100 is configured to receive a color difference signal having the lowest resolution obtained by further down-sampling the color difference signal input to the circuit 101 by the down sampling circuit 104.

5 The downsampling circuits 103 and 104 shown in FIG. 5 are built in the format conversion circuit 302 in FIG. Then, a chrominance block having the highest resolution, which is generated so as to form a 4: 4: 4 macroblock, is input to the circuit 102, and the macroblock is downsampled by the downsampling circuit 103 to 4: 2: 2. Are input to the circuit 101. The chrominance blocks forming the 4: 2: 2 macroblock are further down-sampled by the downsampling circuit 104, and the chrominance blocks forming the 4: 2: 0 macroblock are combined with the luminance blocks into 4: 2: The macroblock of 0 is input to the circuit 100 in units.

Since the processing in the circuit 100 is the same as that described with reference to FIG. 19, the description thereof is omitted. However, if the description of the processing order of the luminance block and the chrominance block is added, the luminance blocks Y1 to Y4 are sequentially input first, so that these data are transferred to the frame memory 51 via the motion vector detection circuit 50. Is written to the frame memory for the luminance block. Similarly, the chrominance block data is written to the chrominance block frame memory of the frame memory 51 via the motion vector detection circuit 50 (FIG. 19).

Then, the data of the luminance blocks Y1 to Y4 are read from the frame memory 51, and the prediction mode switching circuit 52, the arithmetic unit 53, the DCT circuit 56, the quantization circuit 57, the inverse quantization circuit 60, the IDCT circuit 61, and the arithmetic unit After being processed by the Luma frame memory 63L and the motion compensation circuit 64L, they are output via the variable length coding circuit 58 and the transmission buffer 59.

The data of the chrominance block is basically processed in the same manner as the data of the luminance block, but the data of the chrominance block output from the calculator 62 is supplied to the chroma frame memory 63C and stored therein. . Then, in the motion compensation circuit 64C, motion compensation is performed using the motion vectors obtained by shortening the motion vectors in the luminance blocks Y1 to Y4 by に in the horizontal direction and the vertical direction, respectively.

{Circle around (1)}, the circuit 100 supplies a signal of a group including the luminance blocks Y1, Y2, Y3, and Y4 and the color difference blocks Cb5 ″ and Cr6 ″ to the synthesis circuit 105.

On the other hand, the data of the chrominance block converted into the 4: 2: 2 macroblock format by the downsampling circuit 103 is supplied to the arithmetic unit 112 of the circuit 101. The arithmetic unit 112 also includes data obtained by upsampling the data of the lower-resolution chrominance block, which is output from the arithmetic unit 62 of the circuit 100, by the upsampling circuit 111 twice (spatially) in the vertical direction. It is supplied as a prediction error signal.

This up-sampling circuit 111 can be constituted by an interpolation circuit 141, for example, as shown in FIG. For example, as shown in FIG. 7, the interpolation circuit 141 adds (averages) the chrominance data of the line having no chrominance data after halving the values of the chrominance data located above and below the line, respectively. Can be generated. Since the band is limited when down-sampling is performed by the down-sampling circuit 104, the spatial frequency is not expanded by the up-sampling, but the resolution can be doubled.

デ ー タ Thus, the data of the chrominance block generated by the upsampling circuit 111 is subtracted from the chrominance data output from the downsampling circuit 103 as a predicted image signal, and the difference is generated. This difference includes a vertical high-frequency component because the up-sampling circuit 111 performs up-sampling by two times in the vertical direction. This output of the arithmetic unit 112 is subjected to DCT processing by the DCT circuit 113, then quantized by the quantization circuit 114, and is subjected to variable-length encoding by the variable-length encoding circuit 115. Then, although not shown, it is supplied to the synthesizing circuit 105 via a transmission buffer. Thereby, a signal of a group of color difference blocks Cb5 ', Cr6', Cb7 ', Cr8' having a higher resolution than the color difference blocks Cb5 ", Cr6" output from the circuit 100 is generated.

On the other hand, in the circuit 102, the data output from the quantization circuit 114 of the circuit 101 is inversely quantized by the inverse quantization circuit 121, which is further subjected to IDCT processing by the IDCT circuit 122 and output to the arithmetic unit 123. The prediction error signal output from the upsampling circuit 111 and used in the circuit 101 is supplied to the arithmetic unit 123. The arithmetic unit 123 locally decodes the color difference signal in the circuit 101 by adding the prediction error signal output from the upsampling circuit 111 and the signal output from the IDCT circuit 122.

{Circle around (2)} The signal output from the arithmetic unit 123 is upsampled twice in the horizontal direction by the upsampling circuit 124 and output to the arithmetic unit 125. The arithmetic unit 125 subtracts the signal output from the up-sampling circuit 124 as a prediction error signal from the data of the chrominance block in the 4: 4: 4 macroblock format supplied from the format conversion circuit 302. Thus, the difference data includes a high frequency component in the horizontal direction.

出力 The output of the arithmetic unit 125 is subjected to DCT processing by the DCT circuit 126, quantized by the quantization circuit 127, and then subjected to variable length coding in the variable length coding circuit 128. Then, the signal is output to the synthesis circuit 105 via a transmission buffer (not shown). Thereby, data of the group of the color difference blocks Cb5, Cr6, Cb7, Cr8, Cb9, Cr10, Cb11, and Cr12 having the highest resolution is obtained.

In this way, the synthesizing circuit 105 outputs the data of the group constituted by the luminance blocks Y1 to Y4 output from the circuit 100, the data Cb5 ″ and Cr6 ″ of the chrominance blocks having the lowest resolution, and the intermediate data output from the circuit 101. Group of data consisting of color difference block data Cb5 ', Cr6', Cb7 ', Cr8' of the same resolution, and the color difference block data Cb5, Cr6, Cb7, Cr8, Cb9, Cr10, Cb11, Cr12 of the highest resolution. The data of the group constituted by is synthesized.

At the time of this synthesis, the synthesis circuit 105 arranges headers H1 to H3 at the head of the data of the three groups, respectively, as shown in FIG. As a result, a master slice composed of the header H1 and Y1, Y2, Y3, Y4, Cb5 ″, Cr6 ″, a slave slice 1 composed of the headers H2, Cb5 ′, Cr6 ′, Cb7 ′, Cr8 ′, and a header A bit stream composed of slave slices 2 composed of H3, Cb5, Cr6, Cb7, Cr8, Cb9, Cr10, Cb11, and Cr12 is supplied to the transmission path and recorded on the recording medium 3.

After the data of the master slice of one frame of data is transmitted, the data of the slave slice 1 for one frame is transmitted next, and then the data of the slave slice 2 for one frame is transmitted. Is also theoretically possible. However, this makes it difficult to obtain a high-resolution color image in real time. Therefore, as shown in FIG. 8, the master slice, the slave slice 1, and the slave slice are sequentially transmitted. preferable.

デ ー タ The data recorded on the recording medium 3 is reproduced from the recording medium 3 according to the format shown in FIG. 8 and is input to the decoder 401 of the decoding device 2 in FIG.

This decoder 401 is configured, for example, as shown in FIG. In FIG. 9, the portions corresponding to the case shown in FIG. 22 are denoted by the same reference numerals. In this embodiment, the data supplied from the recording medium 3 (transmission path) is supplied to the reception buffer 81, and once stored, supplied to the separation circuit 150, where the luminance block and the chrominance block having the lowest resolution are grouped. , The data of the group of the color difference blocks of the intermediate resolution, and the data of the group of the color difference blocks of the highest resolution are supplied to the circuits 161, 162 or 163, respectively.

Circuit 161 has substantially the same configuration as decoding circuit 90 shown in FIG. In the circuit 161, a luma frame memory 86L and a chroma frame memory 86C are shown as the frame memory 86, and a motion compensation circuit 87L and a motion compensation circuit 87C are shown as the motion compensation circuit 87. Although not shown in FIG. 90, these circuits are built in. Further, the luma frame memory 86L and the chroma frame memory 86C of FIG. 9 do not show the forward prediction image section and the backward prediction image section shown in FIG. 22, but both have these built-in. is there.

Therefore, the circuit 161 performs the same processing as the case described with reference to FIG. Only the relationship between the luma frame memory 86L and the chroma frame memory 86C will be described. The luma frame memory 86L stores the luminance block data output from the arithmetic unit 85. Then, the motion compensation for the luminance signal is performed by the motion compensation circuit 87L, and is output to the calculator 85. On the other hand, the chroma frame memory 86C stores data relating to the color difference blocks. Then, the motion compensation circuit 87C performs motion compensation on the data read from the chroma frame memory 86C using the motion vector obtained by halving the motion vector used in the motion compensation circuit 87L in the horizontal direction and the vertical direction, respectively. Output to

In this way, the data of the 4: 2: 0 macroblock composed of the four luminance blocks Y1 to Y4 and the blocks Cb5 ″ and Cr6 ″ of the lowest resolution color difference signal from the circuit 161 is sent to the selection circuit 164. Is output.

On the other hand, the data of the chrominance block having the intermediate resolution separated by the separation circuit 150 is subjected to variable-length decoding in the variable-length decoding circuit 152 and inversely quantized in the inverse quantization circuit 153. Then, after being subjected to IDCT processing in the IDCT circuit 154, it is input to the arithmetic unit 155.

(5) The lower-resolution color difference block data output from the calculator 85 of the circuit 161 is vertically up-sampled by the up-sampling circuit 151 and supplied to the calculator 155. That is, this signal corresponds to the predicted image signal generated by the upsampling circuit 111 of the circuit 101 in FIG. Therefore, the data output from the IDCT circuit 154 and the prediction error signal output from the up-sampling circuit 151 are added by the arithmetic unit 155, so that the blocks Cb5 ', Cr6', Cb7 ', Cr8 'is obtained. This color difference signal is supplied to the selection circuit 164.

Further, the data of the chrominance blocks Cb5, Cr6, Cb7, Cr8, Cb9, Cr10, Cb11, and Cr12 having the highest resolution and separated by the separation circuit 150 are supplied to the variable length decoding circuit 157 of the circuit 163, where the data is variable. It is long decoded. The signal output from the variable length decoding circuit 157 is inversely quantized by the inverse quantization circuit 158, subjected to IDCT processing by the IDCT circuit 159, and then input to the arithmetic unit 160.

{Circle around (1)} The color difference signal of the intermediate resolution output from the calculator 155 of the circuit 162 is horizontally up-sampled by the up-sampling circuit 156 and supplied to the calculator 160 as a prediction error signal. The arithmetic unit 160 adds the prediction error signal to the output of the IDCT circuit 159, decodes the highest-resolution color difference signals Cb5, Cr6, Cb7, Cr8, Cb9, Cr10, Cb11, and Cr12, and outputs the decoded signals to the selection circuit 164. I do.

The selection circuit 164 is included in the format conversion circuit 402 in FIG. The selection circuit 164 selects a luminance signal and selects one of three color difference signals having different resolutions in response to a command from a user. The luminance signal is supplied to the luminance signal frame memory 34, and the chrominance signal is supplied to the chrominance signal frame memory 403. The luminance signal read from the luminance signal frame memory 34 is D / A-converted by the D / A converter 36 and then supplied to the post-processing circuit 38. The color difference signal read from the color difference signal frame memory 403 is D / A converted by a D / A converter 404 and then supplied to the post-processing circuit 38. The clock of the D / A converter 404 is changed according to the selected color difference signal.

Therefore, the user can arbitrarily select one of the three hierarchical resolutions and display it on a display or the like as necessary.

FIG. 10 shows a second embodiment of the encoder 303. In this embodiment, the circuit 102 for processing a color difference signal of the highest resolution in the first embodiment (FIG. 5) is omitted, a circuit 101 for processing a color difference signal of an intermediate resolution, and a color difference signal of the lowest resolution. And a circuit 100 for processing the luminance signal. Among them, the circuit 100 has the same configuration as that in FIG.

On the other hand, the circuit 101 includes, in addition to an arithmetic unit 112, a DCT circuit 113, a quantization circuit 114, and a variable length encoding circuit 115, an inverse quantization circuit 171, an IDCT circuit 172, an arithmetic unit 173, a chroma frame memory 174, a motion compensation A circuit 175 and a selection circuit 176 are provided.

That is, in this embodiment, the operation of the circuit 100 is the same as that in FIG.

In the circuit 101, the method of generating the predicted image signal is different from that in FIG. That is, in this embodiment, similarly to the case of the embodiment of FIG. 5, the locally decoded chrominance signal output from the arithmetic unit 62 of the circuit 100 is up-sampled by the up-sampling circuit 111 in the vertical direction. As a result, a first predicted image signal is generated.

{Circle around (4)} The signal output from the quantization circuit 114 is inversely quantized by the inverse quantization circuit 171, subjected to IDCT processing by the IDCT circuit 172, and then input to the arithmetic unit 173. The prediction image signal selected by the selection circuit 176 is input to the arithmetic unit 173.

The arithmetic unit 173 adds the predicted image signal and the signal output from the IDCT circuit 172, and performs local decoding. The decoded chrominance signal is supplied to the chroma frame memory 174 and stored. The color difference signal stored in the chroma frame memory 174 is subjected to motion compensation in the motion compensation circuit 175 using a motion vector obtained by halving the motion vector in the motion compensation circuit 64L in the vertical direction. Is supplied as a second predicted image signal.

The selection circuit 176 includes a prediction error signal when using the first predicted image signal output from the upsampling circuit 111 and a prediction error signal when using the second predicted image signal output from the motion compensation circuit 175. And selects a predicted image signal corresponding to a small prediction error signal. Then, the selected predicted image signal is supplied to the arithmetic unit 173 to be used for local decoding as described above, and is also supplied to the arithmetic unit 112 to be supplied to the intermediate unit from the format conversion circuit 302. It is used as a prediction image signal for encoding a color difference signal of resolution.

As described above, in this embodiment, the circuit 101 applies the up-sampling circuit 111 (spatial filter) including the interpolation circuit 141 (FIG. 6) to a decoded image of a color difference signal having a low resolution. , A predicted image having the same resolution as the high-resolution (intermediate resolution) color difference signal is generated, and the high-resolution (intermediate resolution) color difference signal is locally decoded to generate a predicted image. Then, of the two predicted images, the one with better prediction efficiency is adaptively selected. This makes it possible to compress data more efficiently.

In this embodiment, the selection circuit 176 indicates which one of the first prediction image signal output by the upsampling circuit 111 and the second prediction image signal output by the motion compensation circuit 175 has been selected. A space (when the former is selected) / time (when the latter is selected) flag is output, and this is multiplexed and synthesized in the synthesizing circuit 105 together with the data output by the circuits 100 and 101 and transmitted.

FIG. 11 shows an embodiment of a decoder 401 for decoding data encoded by the encoder 303 shown in FIG. 10 and decoding. In the embodiment of FIG. 11, parts corresponding to those of the embodiment shown in FIG. 9 are denoted by the same reference numerals. In this embodiment, the circuit 163 for processing the color difference signal of the highest resolution in FIG. 9 is omitted, the circuit 162 for processing the color difference signal of the intermediate resolution, and the circuit 161 for processing the color difference signal and the luminance signal of the low resolution. It consists of: The configuration of the circuit 161 is similar to that in FIG.

The circuit 162 includes a chroma frame memory 181, a motion compensation circuit 182, and a selection circuit 183 in addition to the upsampling circuit 151, the variable length decoding circuit 152, the inverse quantization circuit 153, the IDCT circuit 154, and the arithmetic unit 155. Have been.

The decoded intermediate-resolution color difference signal output from the arithmetic unit 155 is supplied to the chroma frame memory 181 and stored therein. Then, motion compensation is performed by the motion compensation circuit 182 using the motion vector obtained by halving the motion vector in the motion compensation circuit 87C in the vertical direction, and is supplied to the selection circuit 183 as a second predicted image signal in the time axis direction. .

The selection circuit 183 also vertically upsamples the lower resolution color difference signal output from the arithmetic unit 85 of the circuit 161 by the upsampling circuit 151 and expands the color difference signal to the resolution of the intermediate resolution color difference signal. One predicted image signal is supplied.

The separation circuit 150 detects the space / time flag from the signal supplied from the reception buffer 81, and outputs this to the selection circuit 183. The selection circuit 183 selects the first predicted image signal output by the upsampling circuit 151 when the space flag is detected, and the second predicted image signal output by the motion compensation circuit 182 when the time flag is detected. A signal is selected, and the selected predicted image signal is output to the arithmetic unit 155. As a result, the color difference signal of the intermediate resolution is adaptively decoded.

FIG. 12 shows a third embodiment of the encoder 303. In this embodiment, the configuration of the circuit 101 is a slightly improved configuration of the circuit 101 of the second embodiment shown in FIG. In this circuit 101, the second predicted image signal output from the motion compensation circuit 175 is supplied to a calculator 193 after being multiplied by a weighting coefficient W by a weighting circuit 191. The first predicted image signal output from the up-sampling circuit 111 is multiplied by a weighting coefficient (1-W) by a weighting circuit 192, and is then supplied to a computing unit 193. The calculator 193 adds the weighted predicted image signals supplied from the weighting circuits 191 and 192.

For example, when 0, 1/4, 2/4, 3/4, and 1 are set by the weighting circuit 191 as the weighting coefficient W, the weighting circuit 192 determines that the weighting coefficient (1-W) is 1, 3 / 4, 2/4, 1/4, 0 are set. The weighting circuits 191 and 192 multiply the input predicted image signals by five types of coefficients, respectively, and output five types of predicted image signals to the calculator 193. The computing unit 193 generates five types of predicted image signals by adding the corresponding ones of the five types of weighted predicted image signals. Then, a prediction error signal in the case of adopting each of the five types is generated, a signal having the smallest prediction error signal is selected as a final prediction error signal, and a prediction image signal corresponding to the selected prediction error signal is calculated. To the devices 112 and 173.

This allows more efficient compression.

In this embodiment, the arithmetic unit 193 outputs the finally selected weighting coefficient W to the synthesis circuit 105. The combining circuit 105 multiplexes the weighting coefficient W with other color difference signals and outputs the result.

FIG. 13 shows a configuration example of the decoder 401 when decoding a signal encoded by the encoder 303 shown in FIG. The embodiment shown in FIG. 13 has basically the same configuration as the embodiment shown in FIG. However, the configuration of the circuit 162 is slightly improved from the case of FIG.

In the embodiment of FIG. 13, the predicted image signal output from the motion compensation circuit 182 is weighted by the coefficient W in the weighting circuit 201 and then supplied to the arithmetic unit 203. The prediction image signal output from the upsampling circuit 151 is weighted by the coefficient (1−W) by the weighting circuit 202 and then supplied to the arithmetic unit 203. The weighting coefficients W in the weighting circuits 201 and 202 correspond to the weighting coefficients of the weighting circuits 191 and 192 in FIG.

Therefore, the arithmetic unit 203 outputs the five types of weighted second predicted image signals output from the weighting circuit 201 and the five types of weighted first predicted image signals output from the weighting circuit 202. Are added together. Then, the separation circuit 150 selects a signal corresponding to the weighting coefficient W separated from the signal supplied from the reception buffer 81 from the added predicted image signals. Then, the selected predicted image signal is input to the arithmetic unit 155, and is used as a predicted image signal of a color difference signal having an intermediate resolution.

In the above-described embodiment, band division is performed by DCT to orthogonally transform data of a block of n × n (n = 8 in the above embodiment). However, for example, QMF is used. Subband division can also be performed. Further, the present invention can be applied to a case where octave division is performed by wave red transform, or a case where encoding is performed by performing predetermined conversion or division on input two-dimensional image data.

Furthermore, the coded audio signal and the synchronization signal are multiplexed with the coded video signal bit stream, a code for error correction is added, and a predetermined modulation is applied. The laser light can be modulated and recorded as pits or marks on the disk. Further, a stamper is formed using this disk as a master disk, and a large number of duplicate disks (for example, optical disks) can be formed from the stamper. In this case, the decoder will reproduce data from this duplicate disc.

1 is a block diagram illustrating a configuration of an embodiment of an image signal encoding device and a decoding device according to the present invention. FIG. 2 is a diagram illustrating a sampling format of a color difference signal in a format conversion circuit 302 in FIG. 1. FIG. 6 is a block diagram illustrating a configuration example of downsampling circuits 103 and 104 in FIG. 5. FIG. 3 is a diagram illustrating a configuration of a macro block. FIG. 2 is a block diagram illustrating a configuration of a first example of an encoder 303 in FIG. 1. FIG. 6 is a block diagram illustrating a configuration example of upsampling circuits 111 and 124 in FIG. 5. FIG. 7 is a diagram illustrating an interpolation operation of the interpolation circuit 141 in FIG. 6. FIG. 2 is a diagram illustrating a recording format of a recording medium 3 in FIG. 1. FIG. 2 is a block diagram illustrating a configuration of a first example of the decoder 401 in FIG. 1. FIG. 4 is a block diagram illustrating a configuration of a second embodiment of the encoder 303 in FIG. 1. FIG. 4 is a block diagram illustrating a configuration of a second embodiment of the decoder 401 in FIG. 1. FIG. 11 is a block diagram illustrating a configuration of a third embodiment of the encoder 303 in FIG. 1. FIG. 11 is a block diagram illustrating a configuration of a third embodiment of the decoder 401 in FIG. 1. FIG. 3 is a diagram illustrating the principle of high-efficiency coding. FIG. 4 is a diagram for explaining types of pictures when compressing image data. FIG. 3 is a diagram illustrating a principle of encoding a moving image signal. FIG. 10 is a block diagram illustrating a configuration example of a conventional image signal encoding device and a conventional decoding device. FIG. 18 is a diagram illustrating an operation of format conversion of a format conversion circuit 17 in FIG. 17. 18 is a block diagram illustrating a configuration example of an encoder 18 in FIG. FIG. 20 is a diagram for explaining the operation of the prediction mode switching circuit 52 of FIG. FIG. 20 is a diagram illustrating the operation of the DCT mode switching circuit 55 of FIG. FIG. 18 is a block diagram illustrating a configuration example of a decoder 31 in FIG. 17.

Explanation of reference numerals

1 encoding device, {2} decoding device, {3} recording medium, {12, 13} A / D converter, {14} frame memory, {15} luminance signal frame memory, {16} color difference signal frame memory, {17} format conversion circuit, {18} encoder, {31} decoder, 32 format conversion circuit, {33} frame memory, {34} luminance signal frame memory, {35} chrominance signal frame memory, {36, 37} D / A converter, {50} motion vector detection circuit, {51} frame memory, {52} prediction mode switching circuit, {53} arithmetic unit, 54 prediction decision circuit, 55 DCT mode switching circuit, 56 DCT circuit, 57 quantization circuit, 58 variable length coding circuit, 59 transmission buffer, 60 inverse quantization circuit, 61 IDCT Path, {62} arithmetic unit, {63} frame memory, {64} motion compensation circuit, {81} reception buffer, {82} variable length decoding circuit, {83} inverse quantization circuit, {84} IDCT circuit, {85} computing unit, {86} frame memory, {87} motion compensation circuit, {100} , 101, 102 circuits, {103, 104} downsampling circuits, {111, 124} upsampling circuits, {131} low-pass filters, {132} thinning circuits, {141} interpolating circuits

Claims (22)

In an image signal generation device that generates an image signal encoded in units of a predetermined macro block,
Encoding a luminance signal component of the macroblock to generate an encoded luminance signal; encoding the color signal component of the macroblock at a first resolution to generate a first encoded color signal; First image encoding means for outputting a first slice data signal including a signal and the first encoded color signal;
A color signal component having a second resolution different from the first resolution of the macroblock is used as a signal obtained by decoding the first encoded color signal encoded by the first image encoding unit. A second image encoding unit that generates a second encoded color signal by performing encoding and outputs a second slice data signal including the second encoded color signal;
The first slice data signal output from the first image encoding unit and the second slice data signal output from the second image encoding unit are sequentially read during decoding processing. And a synthesizing means for synthesizing and arranging them at positions close to each other so that one of them is output and one of them is selected. The second image encoding means includes:
First predicted image generation means for generating a first predicted image signal for the color signal component of the second resolution based on the signal obtained by decoding the first encoded color signal;
Storage means for storing the second encoded color signal relating to an encoded image signal;
A second predicted image generation for generating a second predicted image signal for the color signal component of the second resolution based on a signal obtained by decoding the second encoded color signal stored in the storage unit; Means,
A prediction error signal when the first predicted image signal generated by the first predicted image generating means is used and a second predicted image signal generated by the second predicted image generating means are used. Selecting means for comparing one of the first predicted image signal and the second predicted image signal based on a result of the comparison,
Encoding the color signal component of the second resolution using the one of the first predicted image signal and the second predicted image signal selected by the selecting unit, and performing the second encoding 2. The image according to claim 1, further comprising: a second encoded color signal generating unit that generates a color signal and outputs the second slice data signal including the second encoded color signal. 3. Signal generator. The image signal generating apparatus according to claim 2, wherein the synthesizing unit further multiplexes and synthesizes a result of the selection by the selecting unit together with the first slice data signal and the second slice data signal. The second image encoding means includes:
Storage means for storing the second encoded color signal relating to an encoded image signal;
A signal synthesizing unit that performs a predetermined weighting on the signal obtained by decoding the first encoded color signal and the signal obtained by decoding the second encoded color signal stored in the storage unit, and synthesizes the signals. Has,
The color signal component of the second resolution is encoded based on the decoded signal weighted and synthesized by the signal synthesizing means to generate the second encoded color signal, and includes the second encoded color signal. The image signal generation device according to claim 1, wherein the image signal generation device outputs a second slice data signal. In the second image encoding means,
The signal synthesizing unit includes a signal obtained by decoding the first encoded color signal with a plurality of weighting coefficients having different sizes, and a signal obtained by decoding the second encoded color signal stored in the storage unit. Are weighted and synthesized to generate a plurality of decoded signals,
For each of the plurality of weighting coefficients, a prediction error signal in the case where the second encoded color signal is generated based on the decoded signal is calculated and compared, and any decoded signal is subjected to the second encoding. The image signal generation device according to claim 4, further comprising a selection unit that selects whether to use the color signal encoding. The image signal generating apparatus according to claim 4, wherein the combining unit further multiplexes the weighting coefficient together with the first slice data signal and the second slice data signal. In an image signal generating method for generating an image signal encoded in a predetermined macroblock unit,
Encoding a luminance signal component of the macroblock to generate an encoded luminance signal; encoding the color signal component of the macroblock at a first resolution to generate a first encoded color signal; A first image encoding step of outputting a first slice data signal including a signal and the first encoded color signal;
A signal obtained by decoding the first encoded color signal obtained by encoding a color signal component of the macroblock having a second resolution different from the first resolution by the processing of the first image encoding step A second image encoding step of generating a second encoded chrominance signal by using the second coded color signal, and outputting a second slice data signal including the second coded chrominance signal;
The first slice data signal output by the processing of the first image encoding step and the second slice data signal output by the processing of the second image encoding step are decoded. A synthesizing step of arranging the images at positions close to each other so that they are sequentially read out and one of them is selected. The second image encoding step includes:
A first predicted image generation step of generating a first predicted image signal for the color signal component of the second resolution based on the signal obtained by decoding the first encoded color signal;
A storage step of storing the second encoded chrominance signal for an encoded image signal;
A second prediction that generates a second predicted image signal for the color signal component of the second resolution based on a signal obtained by decoding the second encoded color signal stored by the processing of the storage step; An image generation step;
A prediction error signal when using the first predicted image signal generated by the processing of the first predicted image generation step, and the second prediction generated by the processing of the second predicted image generation step A selection step of comparing a prediction error signal in a case where an image signal is used, and selecting one of the first prediction image signal and the second prediction image signal based on a result of the comparison; ,
The color signal component of the second resolution is encoded using the one of the first predicted image signal and the second predicted image signal selected by the processing of the selection step, and the second predicted image signal and the second predicted image signal are encoded. Generating an encoded color signal, and outputting the second slice data signal including the second encoded color signal, a second encoded color signal generating step. Image signal generation method. The image signal generating method according to claim 8, wherein the synthesizing step further multiplexes and synthesizes a result of the selection processing in the selecting step together with the first slice data signal and the second slice data signal. . The second image encoding step includes:
A storage step of storing the second encoded chrominance signal for an encoded image signal;
A signal combining unit that performs a predetermined weighting on a signal obtained by decoding the first encoded color signal and a signal obtained by decoding the second encoded color signal stored by the processing of the storing step, and synthesizes the signals. Steps and
Encoding the color signal component of the second resolution based on the decoded signal weighted and synthesized by the processing of the signal synthesizing step to generate a second encoded color signal; The method according to claim 7, further comprising: outputting a second slice data signal including the following. In the second image encoding step,
The signal synthesizing step decodes the signal obtained by decoding the first encoded color signal with a plurality of weighting coefficients having different magnitudes and the second encoded color signal stored by the processing of the storing step. Generates a plurality of decoded signals by weighting and combining the signal and
For each of the plurality of weighting coefficients, a prediction error signal in the case where the second encoded color signal is generated based on the decoded signal is calculated and compared, and any decoded signal is subjected to the second encoding. The image signal generating method according to claim 10, further comprising a selecting step of selecting whether to use for encoding of the color signal. The image signal generation method according to claim 10, wherein, in the combining step, the weighting coefficients are further multiplex-synthesized together with the first slice data signal and the second slice data signal. An image signal encoded in a predetermined macroblock unit,
An encoded luminance signal in which a luminance signal component of a predetermined macroblock is encoded in the encoded image signal, and a first code in which a color signal component of the macroblock is encoded at a first resolution A first slice data signal including a simulated color signal;
A color signal component having a second resolution different from the first resolution of the macroblock is converted into a second encoded color signal encoded using a signal obtained by decoding the first encoded color signal. And the second slice data signal includes
In an image signal reproducing device that reproduces the image signal that has been supplied and arranged at a position close to each other,
Separation means for sequentially reproducing the first slice data signal and the second slice data signal corresponding to the macroblock, and separating the reproduced first slice data signal and the reproduced second slice data signal When,
Decoding the encoded luminance signal from the first slice data signal to reproduce the luminance signal component, and decoding the first encoded color signal to decode the first color signal of the first resolution; First image decoding means for reproducing the component;
The second coded color signal is decoded from the second slice data signal using the first color signal component decoded by the first image decoding means, and the second encoded color signal is decoded at the second resolution. Second image decoding means for reproducing two color signal components;
Of the first color signal component reproduced by the first image decoding means and the second color signal component reproduced by the second image decoding means, the macroblock decoding processing is performed. And a selecting means for selecting a color signal component to be used. The second image decoding means includes:
First predicted image generation means for generating a first predicted image signal for the second encoded color signal based on the first color signal component;
Storage means for storing the already decoded second color signal component;
A second predicted image generation unit that generates a second predicted image signal for the second encoded color signal based on the second color signal component stored in the storage unit;
One of the first predicted image signal generated by the first predicted image generation unit and the second predicted image signal generated by the second predicted image generation unit is selected. Predictive image selecting means;
The second coded color signal is decoded using the one of the first predicted image signal and the second predicted image signal selected by the predicted image selecting means, and the second color signal is decoded. 14. The image signal reproducing apparatus according to claim 13, further comprising: second color signal reproducing means for reproducing a component. The image signal including the first slice data signal and the second slice data signal indicates which of the first predicted image signal and the second predicted image signal to use. Instructions are further multiply synthesized,
The image according to claim 14, wherein the predicted image selecting unit selects one of the first predicted image signal and the second predicted image signal based on the command. Signal reproduction device. The second image decoding means includes:
First predicted image generation means for generating a first predicted image signal for the second encoded color signal based on the first color signal component;
Storage means for storing the already decoded second color signal component;
A second predicted image generation unit that generates a second predicted image signal for the second encoded color signal based on the second color signal component stored in the storage unit;
Predictive image combining means for outputting a combined signal obtained by combining the first predicted image signal and the second predicted image signal by performing predetermined weighting, and
14. A second chrominance signal reproducing means for decoding the second coded chrominance signal by using the composite signal to reproduce the second chrominance signal component. Image signal reproducing device. The image signal including the first slice data signal and the second slice data signal is further multiplex-synthesized with a weighting coefficient used in a synthesis process of a predicted image signal by the predicted image synthesis unit,
The predictive image synthesizing unit weights and synthesizes the first predictive image signal and the second predictive image signal based on the weighting coefficient, and outputs the synthetic signal. An image signal reproducing apparatus according to claim 1. An image signal encoded in a predetermined macroblock unit,
An encoded luminance signal obtained by encoding a luminance signal component of a predetermined macroblock in the encoded image signal, and a first code obtained by encoding a color signal component of the macroblock at a first resolution A first slice data signal including a simulated color signal;
A color signal component having a second resolution different from the first resolution of the macroblock is converted into a second encoded color signal encoded using a signal obtained by decoding the first encoded color signal. And the second slice data signal includes
An image signal reproducing method for reproducing the image signal that has been supplied and arranged at a position close to each other,
A separating step of sequentially reproducing the first slice data signal and the second slice data signal corresponding to the macroblock and separating the reproduced first slice data signal and the reproduced second slice data signal When,
Decoding the encoded luminance signal from the first slice data signal to reproduce the luminance signal component, and decoding the first encoded color signal to decode the first color signal of the first resolution; A first image decoding step for reconstructing the components;
Decoding the second coded color signal from the second slice data signal using the first color signal component decoded by the processing of the first image decoding step; A second image decoding step of reproducing a second color signal component of
Of the first color signal component reproduced by the processing of the first image decoding step and the second color signal component reproduced by the processing of the second image decoding step, A selecting step of selecting a color signal component used in the decoding process. The second image decoding step includes:
A first predicted image generation step of generating a first predicted image signal for the second encoded color signal based on the first color signal component;
A storing step of storing the already decoded second color signal component;
A second predicted image generating step of generating a second predicted image signal for the second encoded color signal based on the second color signal component stored by the processing of the storing step;
Any of the first predicted image signal generated by the processing of the first predicted image generation step and the second predicted image signal generated by the processing of the second predicted image generation step A predicted image selecting step of selecting one,
The second encoded color signal is decoded using the one of the first predicted image signal and the second predicted image signal selected by the processing of the predicted image selection step, and the second encoded color signal is decoded by the second predicted color signal. 20. The image signal reproducing method according to claim 18, further comprising: a second color signal reproducing step of reproducing a color signal component. The image signal including the first slice data signal and the second slice data signal indicates which of the first predicted image signal and the second predicted image signal to use. Instructions are further multiply synthesized,
20. The image according to claim 19, wherein the predicted image selecting step selects one of the first predicted image signal and the second predicted image signal based on the command. Signal playback method. The second image decoding step includes:
A first predicted image generation step of generating a first predicted image signal for the second encoded color signal based on the first color signal component;
A storing step of storing the already decoded second color signal component;
A second predicted image generating step of generating a second predicted image signal for the second encoded color signal based on the second color signal component stored by the processing of the storing step;
A predicted image synthesizing step of outputting a synthesized signal obtained by performing predetermined weighting on the first predicted image signal and the second predicted image signal, respectively,
19. A second color signal reproducing step of decoding the second encoded color signal using the composite signal to reproduce the second color signal component. Image signal reproduction method. The image signal including the first slice data signal and the second slice data signal is further multiplex-synthesized with a weighting coefficient used in a synthesis process of a predicted image signal in the predicted image synthesis step,
22. The prediction image synthesizing step, wherein the first prediction image signal and the second prediction image signal are weighted and synthesized based on the weighting coefficient, and the synthesized signal is output. The image signal reproducing method described in the above.

Images