JPH06290262A - Processor for image codec - Google Patents

Abstract

PURPOSE:To provide the processor of a simple constitution for the image CODEC by regarding a microblock, consisting of plural blocks composed of mXn image data respectively, as one process unit, employing 'multiple data stream control by single instruction stream' processing adaptively to field image processing and frame image processing, and improving the parallelism of processors. CONSTITUTION:Mutually adjacent element processors 11-18 which are connected by data transfer lines 42-45 are provided corresponding to microblocks and a coefficient memory 51 which gives a coefficient to those element processors in common is provided. Further, a common adding circuit 41 which adds arithmetic results is provided at the output of the element processors 11-14 which perform the field processing and process image data in microblock units are inputted to the element processors 11-18 through buses 31-34; and the respective element processors perform pipeline processing in three stages, i.e., data input stage, calculation stage, and data output stage.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a central processing unit (processor) in a computer system used for, for example, numerical calculation, image processing, graphics processing, etc., and particularly to video signal processing such as an image codec. The present invention relates to a suitable signal processing device.

[0002]

2. Description of the Related Art First, the concept of image macroblocks and blocks in image codec processing will be described. Here, a (4: 4: 2) signal based on the CCIR601 format is taken as an example. The image codec processing in the present invention means CCITT H.264. 261 recommendation and MPEG image compression encoding / decompression decoding standard represented by (image data motion compensation + discrete cosine transform (DCT) processing) based on a macroblock as an image encoding process and It means a decryption process.

One image frame includes a luminance component (Y component) having a size of 720 × 480 pixels and two kinds of color difference components sub-sampled in the horizontal direction having a size of 360 × 480 pixels, that is, a first color difference component (Cr Component) and the second color difference component (Cr component). This image frame
The luminance component is divided into square (rectangular) regions of 16 × 16 pixels, and each of the two types of color difference components is 8 × 1.
Divide into a rectangular area of 6 pixels. 1 in this luminance component
A square area of 6 × 16 pixels and a rectangular area of 8 × 16 pixels in two kinds of color difference components corresponding to the area positionally are collectively called a macro block. A square area of 8 × 8 pixels is called a block regardless of the luminance component and the color difference component. Therefore, as shown in FIG. 6, one macroblock consists of a total of 8 blocks of 4 blocks of luminance (Y) component and 4 blocks of color difference component (2 blocks of Cr component, 2 blocks of Cb component).

As shown in FIG. 7, one macroblock is used.
Conventional image codec processor 1 as a processing unit
Is composed of an input data memory 2, an arithmetic unit 3, and an output data memory 4, and an "input stage" for inputting a macroblock to be image codec processed, and an image codec process using the arithmetic unit 3 " Pipeline processing is performed with a three-stage configuration including a “calculation stage” and an “output stage” that outputs a macroblock after image codec processing. The respective stages are connected by a data memory having a double buffer structure (a structure having two memory banks and switching for each processing unit) called an input data memory 2 and an output data memory 4.

The pipeline processing in the conventional image codec processor 1 using macro blocks as one processing unit will be briefly described below. The coefficient k described below
Is a positive integer. A. When the (2k) th macroblock is in the calculation stage A1. Input stage (2k + 1) th macroblock is the first memory bank 5 (bank 0) of the input data memory 2 shown in FIG.
Write in. A2. Calculation stage The second memory bank 6 (bank 1) of the input data memory 2 is loaded with the 2k-th macroblock, the image codec processing is performed in the arithmetic unit 3, and then the first memory bank of the output data memory 4 is used. 7 (bank 0)
Write in. A3. Output stage Outputs the (2k-1) th image codec-processed macroblock from the second memory bank 8 (bank 1) of the output data memory 4.

B. When the (2k + 1) th macroblock is in the calculation stage B1. The (2k + 2) th input stage macroblock is written in the second memory bank 6 (bank 1) of the input data memory 2. B2. Calculation stage The (2k + 1) th macro block is fetched from the first memory bank 5 (bank 0) of the input data memory 2, the image codec processing is performed in the arithmetic unit 3, and then the second data block of the output data memory 4 is processed. Memory bank 8
Write to (bank 1). B3. Output stage Outputs the 2kth macroblock after the image codec processing from the first memory bank (bank 0) of the output data memory 4.

Since the conventional image codec processor has the double buffer data memory provided between the stages as described above, there is no problem even if the operating frequencies of the stages are different. Further, since there is no competition for access to the data memory between the stages, the stages can operate completely in parallel.

[0008]

However, in the configuration of the image codec processor described above, the arithmetic unit 3 is used.
However, there is a problem that the number of ports of the data memory is increased and the interconnection network between the arithmetic unit and the data memory is complicated when the parallelization is performed to increase the speed. For example,
Assuming that the number of data inputs of one arithmetic unit is two,
Considering a processor in which four arithmetic units operate in four parallels, in order to supply arbitrary data of a macroblock to all four arithmetic units every clock cycle,
An 8-port multiport memory is required for each memory bank of the input data memory. Also, regarding the mutual coupling network between the arithmetic unit and the data memory, each of the eight ports needs to supply data to an arbitrary input terminal of the arithmetic unit, which is a very complicated 8x8 crossbar network. It is necessary to construct an interconnection network with a large amount of hardware. This is because all the data of the macro block is stored in one memory bank of the data memory.

Therefore, an approach is conceivable in which the arithmetic units are parallelized without increasing the number of data memory ports. In this approach, the data memory is divided into blocks corresponding to the blocks existing in the macroblock, and an arithmetic unit is provided corresponding to each data memory. Furthermore, data transfer to a data memory other than the corresponding data memory is prohibited for all arithmetic units. For example, considering the (4: 2: 2) signal illustrated in FIG.
Since the macro block consists of eight blocks, the data memory is divided into eight, and eight arithmetic units are provided corresponding to the respective data memories. In the case of adopting this configuration, assuming that the number of data inputs of one arithmetic unit is two, a multiport memory having at most two ports per one memory bank of the input data memory will suffice, and between the arithmetic unit and the data memory. The mutual connection network of is also simplified.

However, in the image codec processing, there are cases where element processing in which a data dependency exists across blocks is necessary. That is, there is an element process that requires not only the data in the block but also the data in the adjacent block. As an example, motion vector detection at the time of encoding, and field / frame adaptive discrete cosine transform (DCT) in MPEG2
The processing will be described in order.

First, the motion vector detection of image data will be described. When the block matching full search algorithm is adopted in the motion vector detection, the following calculation is performed on the luminance component y _{i, j} (0 ≦ i <16, 0 ≦ j <16) of the macroblock.

[0012]

[Equation 1]

Note that c _{i, j} in the above equation means the data of the candidate block (for details of the motion vector detection processing, the applicant of the present application filed earlier, October 2, 1992).
8th application, the specification of "arithmetic circuit" Japanese Patent Application No. 4-311163 and drawings are described). As can be seen from Equation 1 above, the luminance component of the macroblock (4 blocks)
One sum of absolute differences is obtained by using all the data of. Therefore, the motion vector detection process is an element process in which there is a data dependency relationship between blocks. In addition, even in various mode determination processes at the time of encoding, there is a data dependency relationship that spans such blocks.

Next, frame / field adaptive discrete cosine transform (DCT) processing in MPEG2 will be described. In the frame / field adaptive DCT in MPEG2, the frame D
Adaptively switch between CT and field DCT. At this time, as shown in FIG. 8, in the case of the frame DCT, the configuration is the same as that of the block shown in FIG. 6, but as shown in FIG. 9, in the case of the field DCT, data is alternately provided in the vertical direction of the macroblock. Extract and form blocks. That is, in the field DCT, there is a data dependency that extends over two blocks in the vertical direction in FIG. 6 (for example, block 0 and block 1).

As described above, in the image codec processing, there is an element processing in which there is a data dependency relationship between blocks, so it is difficult to divide the data memory into blocks and provide the arithmetic units in blocks. Met. Further, in the above-described configuration of the conventional image codec processor, since different data memories are used for input and output, memory banks for arithmetic processing exist independently for input and output. The data memory capacity has increased. If the memory capacity of one memory bank is m, a total memory capacity of 4 m is required.

[0016]

In order to solve the above-mentioned problems, the present invention uses an input stage, a calculation stage, and a macro block as one processing unit in image codec processing.
In a processor that performs pipeline processing with a three-stage configuration of output stages, an element processor provided corresponding to each block that constitutes a macroblock has an arithmetic unit and a data memory of a double buffer configuration, and further adjoins each other. There is provided a circuit for adding operation results of the element processors, and a circuit for enabling data transfer between data memories of adjacent element processors.

Therefore, according to the present invention, a macroblock consisting of a plurality of blocks, each of which is composed of mxn image data, is used as one processing unit, and signal processing over the image data of a plurality of blocks and within one block are carried out. "Single instruction stream / multiple data stream: SIMD" that adaptively performs signal processing on image data and multiple data stream control processing with a single instruction stream
In the control type image codec processor, a plurality of pairs of adjacent element processors, which are connected by a data transfer path between adjacent element processors, are provided corresponding to the macroblocks, and a coefficient is shared by the plurality of element processors. A common coefficient memory to be provided is provided, and a common adder circuit that adds these operation results to the outputs of a plurality of pairs of element processors that perform signal processing across the image data of the plurality of blocks is provided. An image codec processor is provided, wherein processed image data is input to the element processor.

[0018] Preferably, each of the element processors is configured to pipeline three stages consisting of a data input stage, a calculation stage, and a data output stage. Specifically, each of the element processors has an I / O buffer having three banks corresponding to three stages including the data input stage, the calculation stage, and the data output stage;
A first data memory having at least two banks arranged in parallel that are at least alternately operable, and a second data memory having at least two banks arranged in parallel that are at least alternately operable; These I / O buffers, first
And an interconnecting network interconnecting the second data memory and an arithmetic unit connected to the interconnecting network.

Preferably, the arithmetic unit comprises a 2-input / 2-output extended arithmetic logical operation processing unit for performing addition, subtraction, various logical operations, magnitude comparison, difference absolute value operation and butterfly operation, and the coefficient memory. A multiplication unit that multiplies the output of the extended arithmetic logic operation processing unit, and an accumulating unit that cumulatively processes the multiplication result. Further preferably, a pipeline register is provided at a stage subsequent to the extended arithmetic logic operation processing unit, a pipeline register is provided at a stage subsequent to the multiplication unit, and a pipeline register is provided at a stage subsequent to the accumulating unit, and the data input stage is provided. Pipeline processing corresponding to three stages including a calculation stage and a data output stage is performed.

Specifically, the signal processing over the image data of the plurality of blocks is field image signal processing, and the signal processing for the image data in the one block is frame image processing.

Further, according to the present invention, in the image codec processor in which the macroblock is one processing unit in the image codec processing, a plurality of element processors are provided for each block forming the macroblock, and the image in each macroblock is There is provided an image codec processor characterized in that codec processing is performed in parallel by "single instruction stream / multiple data stream: SIMD" control using the plurality of element processors.

Further, according to the present invention, in the image codec processor which performs pipeline processing in a three-stage configuration of an input stage, a calculation stage and an output stage with a macroblock as one processing unit in the image codec processing,
An element processor provided corresponding to each block forming the macro block has an arithmetic unit and a data memory having a double buffer structure, and image codec processing in each block is performed by using a plurality of element processors to obtain a “single instruction stream”.・ Multiple data streams: SIM
An image codec processor is provided which is characterized in that it is operated in parallel by "D" control.

Further, according to the present invention, in the image codec processing, the input stage is set with the macroblock as one processing unit,
In an image codec processor that performs pipeline processing with a three-stage configuration including a calculation stage and an output stage, an arithmetic unit and a data memory having a double buffer configuration are provided in an element processor provided corresponding to each block forming the macroblock. In addition, a circuit for adding operation results of adjacent element processors is provided, and image codec processing in each block is performed in parallel by a "single instruction stream / multiple data stream: SIMD" control using a plurality of element processors. An image codec processor is provided which is characterized by performing.

Further, according to the present invention, in the image codec processor for performing pipeline processing in a three-stage configuration of an input stage, a calculation stage, and an output stage with a macroblock as one processing unit in the image codec processing,
An element processor provided corresponding to each block forming the macro block has an arithmetic unit and a data memory having a double buffer structure, and further has a circuit for adding the arithmetic results of the adjacent element processors, , Enables data transfer between the data memories of adjacent element processors, and performs image codec processing in each block in parallel by using "single instruction stream / multiple data stream: SIMD" control using a plurality of element processors. An image codec processor is provided.

[0025]

In the element processor provided corresponding to each block constituting the macroblock, the operation unit and the data memory having the double buffer structure are provided, and further, the operation results of the adjacent element processors are added. Further, by enabling data transfer between the data memories of the adjacent element processors, the image codec processing in each block is performed by using a plurality of element processors, “single instruction stream / multiple data stream: S
It can be done in parallel by "IMD" control.

It is only necessary to provide one adder circuit in common and one coefficient memory in common.

[0027]

Embodiments of the image codec processor of the present invention will now be described in detail with reference to the drawings. The image codec processor according to the embodiment of the present invention has a plurality of arithmetic units such as an arithmetic logic operation unit (ALU), a multiplier, and an accumulator, and these arithmetic units have a single instruction stream. SIMD (Single Instruction stream Multip) that processes multiple data in parallel by
le Data stream) ”type processor. In addition,
"Single instruction stream / multiple data streams: SIM
For "D" control, see Yamauchi, et al,
"Architure and Implement
station of a Highly Parallel
elSingle: Chip Video DSP ",
IEEE TRANSACTIONS AND SYS
TEMS FOR VIDEO TECHNOLOG
Y, VOL. 2, JUNE 1992, pp. 207
See -220.

Further, in the arithmetic unit of this processor, arithmetic units can be connected in pipeline,
It also performs pipeline arithmetic processing. That is, the image codec processor of the present invention, as shown in FIG. 1, has a macroblock input terminal 21, a macroblock output terminal 22,
It has a macro block input / output terminal 23 of the frame memory, a macro block input terminal 24 of the frame memory, and further has an input data bus 31 connected to these terminals,
Output data bus 32 and data buses 33 and 34
Further has. Further, a plurality of image codec processors, eight element processors (PE) 11 in this example, are connected to each other via these buses 31 to 34.
.About.18, one adder circuit 41 for adding the results of the four element processors 11 to 14, and one coefficient memory 51 for applying the coefficient to the block input terminal 81 (FIG. 2) of each element processor. The element processors 11 to 18
In addition to being connected to each other by the buses 31 to 34, adjacent element processors, that is, PE0 and PE1, PE2 and PE3, PE4 and PE5, PE6 and PE7 are connected to each other.

The overall structure of the image codec processor according to the first embodiment of the present invention will be described below.
The respective configurations of the arithmetic unit and the data memory will be described. 1. Overall Configuration FIG. 1 is an overall configuration diagram of an image codec processor as one embodiment of the present invention. This image codec processor is provided with eight element processors 11 to 18 (hereinafter, generally 1k, but may be expressed as k = 1 to 8) corresponding to each block of the macroblock shown in FIG. Has been. FIG. 2 shows the internal configuration of the element processor. k
The th element processor includes an arithmetic unit 6k, an input / output (I / O) buffer 91, and a first frame buffer 0.
(92), the second frame buffer 1 (93), and the data memory 7k including the working buffer 94.
And an interconnection network 9 connecting these buffers 91 to 94
5 and a selector 111.

The operation of the image codec processor of the present invention will be described below. First, the macroblock to be processed by the image codec receives image data one by one from the macroblock input terminal 21 shown in FIG. At this time, each block of the macro block is stored via the input data bus 31 in the data memory 7k of the kth element processor to which the same element processor number as the own block number is attached. That is, block 0 is the first
Data memory 71 of the element processor 11 (PE0) of
The block 1 is the second element processor 12 (PE1).
Data memory 72, hereafter, similarly, block 7 is the eighth
Is stored in the data memory 78 of the element processor 18 (PE7). At the same time, the macro blocks of the past frame and the future frame, which are necessary when performing the motion compensation of the image data, are processed by the data bus 3 similarly to the above input operation.
The data is stored in the data memory 7k of the k-th element processor from the macroblock input / output terminal 23 or the input terminal 24 of the frame memory via 3, 34. This input operation differs depending on the prediction mode of the macroblock, for example, forward prediction or bidirectional prediction. That is, if the macroblock that is the target of the image codec process does not perform the motion compensation of the image data, this input operation is not performed. Also, when performing motion compensation for forward prediction or backward prediction,
Only the macroblock input terminal 24 of the frame memory is used to input only the macroblock of either the past or the future frame. When performing bidirectional prediction motion compensation, the macroblock input / output terminal 23 of the frame memory and the macroblock input terminal 24 of the frame memory are both used to input the macroblocks of both frames.

In parallel with these input operations, in the arithmetic unit 6k of the kth element processor PE, an image codec such as discrete cosine transform (DCT) or quantization is performed by "single instruction stream / multiple data stream: SIMD" control. The element processing of is executed in parallel. In addition,
As described in the above-mentioned document, the “single instruction stream / multiple data stream: SIMD” control means
It is a method of controlling multiple (multiple) data flows with a single instruction. In addition, element processing of all image codecs is performed by "single instruction stream / multiple data stream: SI
Since the "MD" control is performed, the coefficient memory 51 shown in FIG. 1 is shared by all the element processors 11 to 18, and the coefficient memory 51 does not have to be provided in each of the element processors 11 to 18. Further, in parallel with these input operation and calculation operation, the macro block after the image codec processing is output from the macro block output terminal 22 one data at a time.
At this time, each block of the macro block is output from the data memory 7k of the element processor having the same element processor number as the own block number to the macro block output terminal 22 via the output data bus 32. That is, the block 0 is from the data memory 71 of the element processor 11 (PE0), and the block 1 is from the element processor 12 (PE
From the data memory 72 of 1), similarly, the block 7 is the data memory 78 of the element processor 18 (PE7).
Is output to the macroblock output terminal 22 via the output data bus 32.

At the same time, when the macroblock after the image codec processing is required for compensating the image data motion of another macroblock, the kth element processor via the data bus 33 as in the above output operation. Is output to the macro block input / output terminal 23 of the frame memory from the data memory 7k. In this image codec processing, the macro block input / output terminal 23 of the frame memory is
There is no need for simultaneous input and output of macroblocks in.

Next, the adder circuit 41 in FIG. 1 will be described. The inter-block data dependency such as the image data motion vector detection and the mode determination processing described in the above-mentioned “Problems to be solved by the invention” can be solved if all the calculation results obtained for each block can be added. For example, considering image data motion vector detection, the sum of absolute differences may be obtained for each block of the luminance component (4 blocks) of the macroblock, and the sum of these four absolute differences may be added last. For this purpose, the adder circuit 41 is provided at the outputs of the four element processors 11 to 14 which store the luminance components of the macroblocks. This adder circuit 41
Any configuration may be used as long as all the calculation results can be added. The addition result is written in the data register (not shown) of the control circuit.

Finally, the data transfer paths 42 to 45 between the adjacent element processors in FIG. 1 will be described. In the field DCT processing described in the above-mentioned "Problems to be solved by the invention", the data dependence that extends over two blocks in the vertical direction is caused by the two adjacent element processors.
For example, it can be solved by exchanging 32 pieces of data of half of the 8 × 8 block between PE0 and PE1. For this purpose, transfer paths 42 to 45 are provided to enable data exchange with the data memory of the adjacent element processor during the field DCT / inverse DCT (IDCT) processing. At the time of the field DCT / IDCT processing, it is sufficient to exchange the data of half the blocks using these transfer paths 42 to 45 in advance and then execute the DCT / IDCT processing.

Arrangement of Arithmetic Unit FIG. 3 shows the internal arrangement of the arithmetic unit according to one embodiment of the present invention. This arithmetic unit includes a first selector 131,
The second selector 132, the extended AL (EALU) 121,
Two pipeline registers 141 and 142, third selector 133, multiplier 122, pipeline register 1
43, a fourth selector 134, a fifth selector 135,
Accumulator 123 with shift function, pipeline register 14
It has four and six selectors 136 and a pipeline memory 124.

Configuration of Extended ALU 121 The extended ALU 121 includes a positive / negative inverter 301 and an adder 30.
2, subtractor 303, logical operation unit 304, positive / negative decision unit 30
5 and data selectors 301 and 307 are connected as shown. The data selector 13 shown in FIG.
The selected output data of 1 is applied as X, and the selected output data of the data selector 132 is applied as Y.
The logical operation unit 304 performs logical operations such as negation, logical sum, logical product, and exclusive logical sum. The positive / negative inverter 301 inverts the polarity of the input data X and applies it to the data selector 306. The adder 302 receives the input data Y when the polarity-inverted data: -X is output from the data selector 306.
The polarity-inverted data: -X is added, and (Y-X) is output. Further, the adder 302 includes the data selector 30.
When the input data X is output from 6, the input data Y
And the addition result (Y + X) of the input data X is output. The subtractor 303 calculates (X-Y). Logical operation unit 30
4 performs a logical operation on the data X and the data Y. The plus / minus determiner 305 determines whether the input data is positive or negative.

The extended ALU 121 described above is a normal ALU.
In addition to the functions of addition, subtraction, and logical operation, which are the functions of (1) and (2), magnitude comparison operation, difference absolute value operation, and butterfly operation are provided as extended functions. These functions will be described below. (A) Addition In the adder 302, the data X and Y input to the input terminals 311 and 312 are added. In this case, the data selector 306 is selected so that the input data X is applied to the adder 320 without being inverted. The addition result (X + Y) is output from the data selector 307. The addition result A is applied to the pipeline register 141 shown in FIG. 3 via the output terminal 313. (B) Subtraction The subtractor 303 subtracts Y from the data X input to the input terminals 311 and 312. This subtraction result B is
It is applied to the pipeline register 142 shown in FIG. 3 through the output terminal 314. (C) Logical operation In the logical operation unit 304, the negation, logical sum, logical product of the data X and Y input to the input terminals 311 and 312,
A logical operation such as an exclusive OR is performed and the result is output to the pipeline register 141 via the data selector 307 and the output terminal 313. (D) Size comparison: min (X, Y), max (X, Y) data X and Y input to the input terminals 311 and 312
With respect to the positive / negative inverter 301, the adder 302, and the subtracter 3
03, the positive / negative determination unit 305 is used to compare the magnitude. In this case, the data selector 306 is set so that the data (−X) whose polarity is inverted by the positive / negative inverter 301 is input to the adder 302. The positive / negative determination unit 305 includes an adder 302.
From (Y−X) and (X−Y) from the subtractor 303, and the positive / negative determination unit 305 determines (a) as the minimum value min (X, Y), when (Y−X) ≧ 0, X When (X−Y)> 0, Y (b) As the maximum value max (X, Y), when (Y−X) ≧ 0, when Y (X−Y)> 0, X is set to the data selector 307. And output through the output terminal 313. However, the minimum value and the maximum value cannot be output at the same time. (E) Difference absolute value calculation: / XY / Data X and Y input to input terminals 311 and 312
With respect to the positive / negative inverter 301, the adder 302, and the subtracter 3
03 and the positive / negative determiner 305 are used to perform the difference absolute value calculation. In this case, in the data selector 306, the data (−X) whose polarity is inverted by the positive / negative inverter 301 is added by the adder 302.
Is set to be input to. (Y−X) is input from the adder 302 and (X−Y) is input from the subtractor 303 to the positive / negative determination unit 305, and the positive / negative determination unit 305 determines (Y−X) when (Y−X) ≧ 0. ) When (XY)> 0, (XY) is output via the data selector 307 and the output terminal 313. (F) Butterfly operation Data X and Y input to input terminals 311 and 312
With respect to, the butterfly operation is performed using the adder 302 and the subtractor 303. In this case, the data selector 3
06 is set so that the input data X is input to the adder 302. The data selector 307 outputs the addition result (X + Y) of the adder 302 to the output terminal 313, and the subtracter 30
The subtraction result (X−Y) of 3 is output to the output terminal 314.

Next, the outline of pipeline processing in the arithmetic unit shown in FIG. 3 will be described with reference to FIG. As a simple operation example, the extended ALU 121 performs the addition operation in the first step (stage), and the multiplier 122 performs the second operation.
Accumulator with shift function 12
3 performs accumulation in the third step. And
It is assumed that the operation of each of these steps is performed within one clock cycle. Since the pipeline registers 141 and 142 are provided after the expanded ALU 121, the pipeline register 143 is provided after the multiplier 122, and the pipeline register 144 is provided after the shift function accumulator 123.
In the (k-2) clock cycle, the extended ALU 12
1, the addition result is stored in the pipeline register 141, and in the (k−1) clock cycle, the multiplier 1 using the pipeline register 141 storing the addition result in the extended ALU 121 is used.
22 performs the multiplication and saves it in the pipeline register 143, and the extended ALU 121 performs new addition and saves it in the pipeline register 141. In k clock cycles, the pipeline in the multiplier 122 in the above (k−1) clock cycle. Using the multiplication result stored in the register 143, cumulative operation is performed in the accumulator with shift function 123 and the pipeline register 1
The addition result stored in the pipeline register 141 stored in the extended register ALU 121 in the (k-1) clock cycle
In the pipeline register 141, the multiplication is performed and the result is stored in the pipeline register 143. Further, a new addition is performed in the extended ALU 121 and the result is stored in the pipeline register 141. Hereinafter, similarly, in the same clock cycle, addition, multiplication and accumulation are simultaneously performed. Thus, in the arithmetic unit, addition, multiplication and accumulation are performed in parallel in order.

This arithmetic unit is constructed by the applicant of the present invention at the same time as the "processing adaptive arithmetic pipeline structure".
It has the same configuration as the arithmetic unit shown in (1) except for the following points, and is also suitable for the element processing of the image codec. The difference is that in the present invention, a pipeline memory 124 is provided in the arithmetic unit, and a discrete cosine transform / discrete inverse cosine transform (DC) of one 8 × 8 block is provided.
That is, the T / IDCT) processing is performed using only one arithmetic unit. As a result, element processing of all image codecs can be realized by "single instruction stream / multiple data stream: SIMD" control.

According to the analysis by the inventor of the present application, it is found that the frequency of the element processing for continuously performing multiplication or the element processing for obtaining the sum of multiplication results is small in the image codec. To eliminate wiring and reduce the wiring between arithmetic units. In the present invention, element processing of all image codecs is performed by "single instruction stream ...
Since it can be realized by "multiple data stream: SIMD" control, the coefficient memory 51 is shared by all the element processors as shown in FIG.

Structure of Data Memory FIG. 2 shows the structure of the data memory according to one embodiment of the present invention. The data memory includes an input / output (I / O) buffer 91,
The first frame buffer 0 (92), the second frame buffer 1 (93), the working buffer 94, and
It comprises an interconnection network 95. Each will be described below. (A) I / O buffer 91 The I / O buffer 91 includes banks 0, 1, 2 (101, 1).
No. 02, 103), and each has a memory capacity capable of storing at least one 8 × 8 block. A larger memory capacity may be required depending on the image codec processing. This I / O
The buffer 91 is one in which the number of memory banks is reduced by using the input data memory and the output data memory described in the section "Prior Art" in common. That is, the data input and output buffers in the calculation stage are combined into one.

The operation of the I / O buffer 91 will be described below. In the I / O buffer 91, a memory bank for arithmetic processing, an input memory bank, and an output memory bank are switched as follows for each macroblock which is a processing unit of arithmetic. Here, the memory bank for arithmetic processing is connected to the interconnection network 95, the memory bank for input is connected to the input data bus 31, and the memory bank for output is connected to the output data bus 32.

[0043]

[Table 1] Table 1 1.3kth Macro Block Processing Memory bank for arithmetic processing: Bank 0 (101) Memory bank for input: Bank 1 (102) Memory bank for output: Bank 2 (103) 2. When processing the (3k + 1) th macroblock Memory bank for arithmetic processing: Bank 1 (102) Memory bank for input: Bank 2 (103) Memory bank for output: Bank 0 (101) 3. When processing the (3k + 1) th macroblock Memory bank for arithmetic processing: Bank 2 (103) Memory bank for input: Bank 0 (101) Memory bank for output: Bank 1 (102)

Since the I / O buffer 91 operates as described above, there is no problem even if the operating frequencies of the input stage, calculation stage and output stage are different. Further, since there is no competition for access to the I / O buffer 91 between the stages, the stages can operate completely in parallel.

(B) First frame buffer 0 (92)
And the second frame buffer 1 (93) The first frame buffer 0 (92) and the second frame buffer 1 (93) are respectively banks 0, 1 (10).
4: 105, 106: 107), each of which has a memory capacity capable of storing at least one 8 × 8 block. A larger memory capacity may be required depending on the image codec processing. The first frame buffer 0 (92) and the second frame buffer 1 (93) store macroblocks of past frames and future frames that are necessary when performing motion compensation of image data. These macroblocks are input from the macroblock input / output terminal 23 or the macroblock input terminal 24 of the frame memory via the data buses 33 and 34. At this time, since the frame buffer 0 (92) and the frame buffer 1 (93) have a double buffer structure, there is no problem even if the operating frequencies of the input stage and the calculation stage are different.

Further, since there is no competition for access to the frame buffer between the stages, the input stage and the calculation stage can operate completely in parallel. Furthermore, the first frame buffer 0 (92) stores a macroblock that has undergone the image codec processing when the macroblock is required for performing image data motion compensation of another macroblock. Further, the data is output to the macro block input / output terminal 23 of the frame memory via the data bus 33. At this time, the first frame buffer 0 (92)
Has a double-buffer structure, there is no problem even if the operating frequencies of the calculation stage and the output stage are different. Further, since there is no competition for accessing the frame buffer between the stages, the calculation stage and the output stage can operate in completely parallel.

(C) Working Buffer 94 The working buffer 94 is a buffer for storing intermediate calculation results, and has a memory capacity capable of storing at least one 8 × 8 block. The working buffer 94 may require a larger memory capacity depending on the image codec processing.

(D) Mutual Connection Network 95 The mutual connection network 95 is the above-mentioned four types of buffers, that is, I / I.
O buffer 91, first frame buffer 0 (92),
This is a network that connects the second frame buffer 1 (93) and the working buffer 94 to the arithmetic unit 6k. The mutual connection network 95 may have any configuration, but at a minimum, two pieces of data can be supplied from the arbitrary buffers 91 to 94 to the arithmetic unit 6k every clock cycle, and one piece of data can be calculated at the same time. Unit 6k
Can be stored in any of the buffers 91 to 94. Further, at a minimum, one data is supplied from the arbitrary buffers 91 to 94 to the arithmetic unit 6k every one clock cycle, and at the same time, two data can be stored from the arithmetic unit 6k to the arbitrary buffers 91 to 94. . In the element processing of the image codec, every clock cycle,
The same memory bank of the first frame buffer 0 (92) and the second frame buffer 1 (93) (bank 0,
2) is supplied from the
No two data are stored in the same memory bank of the frame buffer 0 (92) and the second frame buffer 1 (93). Therefore, the first frame buffer 0 (92) and the second frame buffer 1 (9
All the memory banks of 3) may have a single port configuration. On the other hand, each memory bank of the I / O buffer 91 and the working buffer 94 have at least a 2-port multiport configuration.

Although the image codec processor as one embodiment of the present invention has been described above, the embodiment of the present invention is not limited to the above-described embodiment, and other configurations for performing the same processing and processing as above may be used. can do.

[0050]

According to the present invention, a circuit for adding operation results of adjacent element processors is provided, and data transfer is possible between the data memories of adjacent element processors PE, so that blocks between image codecs can be transferred. It is possible to realize a process in which there is a data dependency relationship. As a result, it becomes possible to provide an element processor having a double buffer type data memory for each block. As a result, the number of ports of the data memory and the parallelism can be increased without complicating the mutual connection network, and the processing performance can be improved.

Further, according to the present invention, in the I / O buffer in the data memory, the memory banks for arithmetic processing, which are separately provided in the input data memory and the output data memory in the conventional configuration, can be used in common. It is sufficient to have data memory for three memory banks. Therefore, if the data memory capacity of one memory bank is m, the total is 3
Since the data memory capacity is m, the data memory capacity can be reduced to 3/4 compared to the conventional configuration.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an image codec processor as an embodiment of the present invention.

FIG. 2 is a configuration diagram of an element processor shown in FIG.

FIG. 3 is a configuration diagram of an arithmetic unit shown in FIG.

FIG. 4 is a configuration diagram of an extended ALU shown in FIG.

5 is a graph showing pipeline processing in the arithmetic unit shown in FIG.

FIG. 6 is based on the CCIR601 format (4:
FIG. 2 is a diagram showing the concept of macroblocks and blocks in a 2: 2) signal.

FIG. 7 is a diagram showing a configuration of a conventional image codec processor.

FIG. 8 is a diagram illustrating macroblocks (only luminance components) during frame DCT processing.

FIG. 9 is a diagram showing macroblocks (only luminance components) during field DCT processing.

[Explanation of symbols]

1k (11-18) ··· Element processor (PE) of the embodiment of the present invention 21 ·· Macroblock input terminal 22 · · Macroblock output terminal 23 · · Macroblock input / output terminal 24 of frame memory · · Macro block input terminal 31 ··· Input data bus 32 · · Output data bus 33, 34 · · Data bus 41 · · Adder circuit 42 to 45 · · Data transfer path between adjacent element processors 51 · · Coefficient memory 6k (61) -68) -Operation unit 7k (71-78) -Data memory 81-Block input terminal 82-Block output terminal 83-Frame memory block input / output terminal 84-Frame memory block input terminal 91- I / O buffer 92 first frame buffer 0 93 second frame buffer 19 Working buffer 95 Mutual interconnection network 101 I / O buffer bank 0 102 I / O buffer bank 1 103 I / O buffer bank 2 104 Frame buffer 0 bank 0 105 -Bank 1 of frame buffer 0 106-Bank 0 of frame buffer 1 107-Bank 1 of frame buffer 1 111-Data selector 121-Extended logical operation unit (EALU) 122-Multiplier 123-Shift Accumulator with function 124 ··· Pipeline memory 131 to 136 ·· Data selector 141 to 144 · · Pipeline register

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Office reference number FI technical display location H04N 7/133 Z

Claims (16)

[Claims] 1. A signal processing for processing image data of a plurality of blocks and a signal processing for image data in one block, with a macro block composed of a plurality of blocks each of which is composed of mxn image data as one processing unit. In a processor for a "single instruction stream / multiple data stream: SIMD" control type image codec that adaptively controls multiple data streams with a single instruction stream, each of them is connected by a data transfer path between adjacent element processors. A plurality of pairs of element processors adjacent to each other are provided corresponding to the macroblocks, and a common coefficient memory that provides a coefficient commonly to the plurality of element processors is provided, and signal processing across image data of the plurality of blocks is performed. The results of these operations on the outputs of multiple pairs of element processors that A common addition circuit for adding provided, the image codec processor, wherein the processed image data of the macro block is input to the element processor. 2. The processor for an image codec according to claim 1, wherein each of the element processors is configured to pipeline three stages including a data input stage, a calculation stage, and a data output stage. 3. Each of the element processors can operate at least alternately with an I / O buffer having three banks corresponding to the three stages including the data input stage, the calculation stage, and the data output stage. A first data memory having two banks arranged in parallel, a second data memory having at least two banks arranged in parallel that can operate alternately, and I / O buffers thereof 4. The image codec processor according to claim 3, further comprising: an interconnection network interconnecting the first and second data memories, and an arithmetic unit connected to the interconnection network. 4. A two-input / two-output extended arithmetic logic operation processing unit, wherein the operation unit performs addition, subtraction, various logic operations, magnitude comparison, difference absolute value operation and butterfly operation, and a coefficient from the coefficient memory. 4. The image codec processor according to claim 3, further comprising: a multiplication unit that multiplies the output of the extended arithmetic logic operation processing unit, and an accumulation unit that accumulates the multiplication result. 5. A pipeline register is provided after the extended arithmetic logic operation processing unit, a pipeline register is provided after the multiplication unit, and a pipeline register is provided after the accumulation unit, and the data input stage and the calculation are provided. The image codec processor according to claim 4, which performs pipeline processing corresponding to three stages including a stage and a data output stage. 6. The image according to claim 5, wherein the signal processing over the image data of the plurality of blocks is field image signal processing, and the signal processing for the image data in the one block is frame image processing. Codec processor. 7. A macroblock is set to 1 by image codec processing.
In the image codec processor as a processing unit, a plurality of element processors are provided for each block forming the macroblock, and the image codec processing in each macroblock is performed by using the plurality of element processors as a “single instruction stream. Multiple data streams: SIM
An image codec processor characterized in that it is performed in parallel by "D" control. 8. A macro block is set to 1 by image codec processing.
In an image codec processor that performs pipeline processing with a three-stage configuration including an input stage, a calculation stage, and an output stage as a processing unit, an arithmetic unit and a double buffer are provided in an element processor provided corresponding to each block forming the macroblock. The image codec processing in each block is performed by using a plurality of element processors having a data memory having a configuration of “single instruction stream / multiple data stream: SIM
An image codec processor characterized in that it is performed in parallel by "D" control. 9. An image codec processor for performing pipeline processing in a three-stage configuration of an input stage, a calculation stage, and an output stage with a macroblock as one processing unit for image codec processing, and corresponding to each block forming the macroblock. The element processor provided has an arithmetic unit and a data memory of a double buffer structure, and further has a circuit for adding the arithmetic results of the adjacent element processors, and the image codec processing in each block is performed by a plurality of element processors. An image codec processor characterized by performing "single instruction stream / multiple data stream: SIMD" control in parallel using. 10. An image codec processor for performing pipeline processing in a three-stage configuration of an input stage, a calculation stage, and an output stage with a macroblock as one processing unit for image codec processing, and corresponding to each block constituting the macroblock. The element processor provided has an arithmetic unit and a data memory having a double buffer structure, and further has a circuit for adding the arithmetic results of the adjacent element processors, and between the data memories of the adjacent element processors. An image codec processor characterized by enabling data transfer and performing image codec processing in each block in parallel by "single instruction stream / multiple data stream: SIMD" control using a plurality of element processors. 11. The image codec processor according to claim 7, wherein each of said element processors has three stages corresponding to said data input stage, calculation stage, and data output stage. An I / O buffer having a bank, a first data memory having at least two banks arranged in parallel that can operate alternately, and at least two banks arranged in parallel that can operate alternately Processor for image codec having a second data memory having an I / O buffer, an interconnection network interconnecting the first and second data memories, and an arithmetic unit connected to the interconnection network . 12. An arithmetic arithmetic operation processing unit of 2 inputs and 2 outputs, wherein the arithmetic unit performs addition, subtraction, various logical operations, magnitude comparison, difference absolute value arithmetic and butterfly arithmetic, and a coefficient from the coefficient memory. The image codec processor according to claim 11, further comprising: a multiplication unit that multiplies the output of the extended arithmetic logic operation processing unit, and an accumulation unit that accumulates the multiplication result. 13. A pipeline register is provided after the extended arithmetic logic operation processing unit, a pipeline register is provided after the multiplication unit, and a pipeline register is provided after the accumulation unit. The data input stage and the calculation are provided. 13. The image codec processor according to claim 12, which performs pipeline processing corresponding to three stages including a stage and a data output stage. 14. The image codec processor uses a macroblock consisting of a plurality of blocks each of which is composed of mxn image data as one processing unit, and performs signal processing over a plurality of blocks of image data and one block. Adaptively with the signal processing of the image data of
Controlling multiple data streams with a single instruction stream "Single instruction stream / multiple data stream: S
14. The image codec processor according to claim 13, which performs "IMD" control. 15. The image according to claim 14, wherein the signal processing over the image data of the plurality of blocks is field image signal processing, and the signal processing for the image data in the one block is frame image processing. Codec processor. 16. A pair of adjacent element processors are connected by a data transfer path between adjacent element processors,
A common coefficient memory provided corresponding to the macroblocks and commonly providing coefficients to the plurality of element processors is provided, and is output to a plurality of pairs of element processors that perform signal processing across the image data of the plurality of blocks. 16. The image codec processor according to claim 15, further comprising a common adder circuit for adding these calculation results, wherein the processed image data in macro block units is input to the element processor.