JP4625903B2 - Image processor - Google Patents

Description

  The present invention relates to an image processing processor specialized in image processing and filter calculation.

  In recent years, transmission / reception of moving images via a communication network and storage of moving images in storage media are widely performed. In general, since moving images have a large amount of information, when moving images are transmitted using a communication channel with a limited transmission bit rate or when moving images are stored on a storage medium with a limited storage capacity, Techniques for encoding and decoding images are indispensable. As moving picture encoding / decoding methods, there are MPEG (Moving Picture Experts Group) and H.26X which are being standardized by ISO / IEC. These encode or decode a plurality of consecutive frames constituting a moving image, and reduce the redundancy by using temporal correlation and spatial correlation of the moving image. This is a technique for encoding with a reduced amount of information and decoding the encoded moving image back to the original moving image.

Such encoding / decoding technology has been applied to portable video systems with built-in microcomputers and information terminal devices for mobile phones, but recently, since high-resolution moving images are used, the information terminal devices are limited. There is a demand for a motion detection processor for high-resolution video encoding processing that operates with low power consumption due to battery capacity.
Most of the power consumption of the entire moving image encoding process is occupied by the motion detection process. In order to realize low power consumption of the entire moving image encoding process, it is most effective to reduce the power consumption of the motion detection process.

Most video encoding processes employ block-based (window-based) image processing as motion detection processing. There are two types of current general block-based image processing.
One is single block matching calculation processing. The single block matching calculation process is an operation for processing a random block in an image frame to be processed. A conceptual diagram of the single block matching calculation process is shown in FIG. In FIG. 1A, three search blocks (101 to 103) exist at random positions in the screen frame 100 and perform calculation processing on pixel data. In the single block matching calculation process, since there is no correlation in the context of the search block, there is no pixel data reusability with the block matching calculation process in the next step.
As a search algorithm using this single block matching calculation process, for example, there are a few steps such as initial value search, prediction vector search, and three-step search.

The other is continuous block matching calculation processing. The continuous block matching calculation process is an operation for processing continuous blocks in the image frame to be processed. A conceptual diagram of continuous block matching calculation processing is shown in FIG. In FIG. 1-2, three search blocks (101 to 103) exist in positions where some overlap in the screen frame 100, and perform pixel data processing. In the continuous block matching calculation process, the reproducibility of the pixel data with the block matching calculation process in the next step is high because the context of the search block has a strong correlation.
As a search algorithm using this continuous block matching calculation process, there are, for example, a full search method, a sub-sampling search method, a one-dimensional search, a one-step search, and the like.

  Almost all algorithms for motion detection processing in moving image encoding processing are combinations of the above search algorithms. In the above search algorithms, there are suitable processor architectures.

  In the single block matching calculation processing described above, since there is no reusability of pixel data, a SIMD (Single Instruction Multiple Data Stream) type processor architecture specialized for high-speed calculation performance is suitable. As an SIMD type processor, for example, Patent Document 1 is known. The SIMD type processor is characterized in that it can calculate at a very high speed instead of reusing pixel data. However, on the other hand, since the reusability of the pixel data is low, the frequency of accessing the pixel data to the cache memory is very large, and the power consumption reduction effect is small.

  In the continuous block matching calculation processing, a systolic array processor architecture capable of reusing pixel data is suitable. For example, Patent Document 2 to Patent Document 3 are known as systolic array type processors. The systolic array type processor is characterized in that pixel data is highly reusable and the frequency of accessing pixel data to the cache memory is reduced, so that the effect of reducing power consumption is extremely high. However, on the other hand, high-speed calculation performance is low.

JP-A-8-63452 JP 2000-293510 A JP 2002-175283 A

  The conventional motion detection processing processor has either a SIMD type or a systolic array type architecture, and has a problem that it may not be optimal depending on a search algorithm.

  In view of the above problems, the present invention can switch between SIMD type and systolic array type configurations according to the search algorithm in motion detection processing, and has an optimum architecture configuration depending on the search algorithm performing motion detection processing. An object is to provide an image processor to be realized.

  In order to achieve the above object, an image processing processor according to claim 1 of the present invention is a motion detection processing processor including a control unit and a data path unit, and the control unit receives a control signal according to an external command. Generating an address continuously for the original image memory and the reference image memory, generating a switching signal for the data path unit, the data path unit including the original image memory, A reference image memory, a plurality of arithmetic circuits, and a switching circuit that switches the parallel number of the arithmetic circuits and the input data of some arithmetic circuits to the output data of other arithmetic circuits according to the switching signal, The configuration is characterized in that the configuration of the SIMD type and the systolic array type can be switched by the search algorithm of the detection process.

  With the above configuration, both SIMD type and systolic array type can be switched according to the search algorithm for motion detection processing, so low power consumption can be achieved by switching the optimal architecture with an algorithm with different pixel data reusability. Can do.

  Next, an image processor according to claim 2 of the present invention is the image processor according to claim 1, wherein the switching signal is a signal of the arithmetic circuit when performing single block matching arithmetic processing. Maximize the number of parallels to have a SIMD type configuration, and when performing continuous block matching computation processing, reduce the number of parallel computation circuits and switch the input data of some computation circuits to the output data of other computation circuits It is characterized by having a systolic array type configuration.

  With the above configuration, when performing single block matching arithmetic processing, SIMD type improves high-speed arithmetic performance, and when performing continuous block matching arithmetic processing, systolic array type enhances data reusability and low power consumption. Can be achieved.

  Next, according to a third aspect of the present invention, in the image processing processor according to the first aspect, the arithmetic circuit inputs the pixel data of the original image and the pixel data of the reference image to input the absolute difference. The circuit is configured to output a value.

  With the above configuration, H. In the 26X motion detection processing processor, the efficiency of the integer pixel search algorithm can be improved.

  Next, an image processor according to claim 4 of the present invention is the image processor according to claim 1, wherein the arithmetic circuit inputs a plurality of pixel data of an original image and a plurality of pixel data of a reference image. Thus, the circuit is configured to output a sum of absolute differences.

  With the above configuration, the efficiency of the integer pixel search algorithm can be increased similarly. Further, it is possible to improve efficiency by processing a plurality of pixel data as operation units.

  Next, an image processor according to claim 5 of the present invention is the image processor according to any one of claims 1 to 4, wherein the original image memory and / or the reference image memory is a two-read port memory. It is characterized by being configured.

  With the above-described configuration, it is possible to deal with a plurality of block matching calculation processes including a single block matching calculation process and a continuous block matching calculation process with high memory access efficiency.

  According to the image processor according to the present invention, it is possible to switch between the SIMD type and the systolic array type according to the search algorithm in the motion detection process, and the optimum architecture configuration according to the search algorithm performing the motion detection process. By realizing the above, low power consumption can be achieved.

  According to the image processor of the present invention, low power consumption is achieved in moving image data processing, and it can be easily incorporated into various information terminal devices and information systems that require low power consumption. System design is possible. Further, even a small terminal such as a mobile phone can perform high-resolution moving image encoding / decoding processing, and the use of the portable information terminal device is variously expanded.

Hereinafter, embodiments of the image processor of the present invention will be described in detail with reference to the drawings.
First, FIG. 2 shows the architecture of a general SIMD type processor and a systolic array type processor. FIG. 2A shows a general SIMD type processor architecture. Pixel data and the like are read from the random access memory 10, and eight arithmetic circuits (1a to 1h) perform processing in parallel.
FIG. 2B shows a general systolic array type processor architecture. After reading pixel data and the like from the random access memory 10 and performing processing in parallel by the four arithmetic circuits (2a to 2d), the pixel data and the like are shifted to the four arithmetic circuits (2e to 2h) for calculation. It will be reused.

The SIMD type processor configured as shown in FIG. 2 (A) is capable of performing very high-speed operations instead of reusing pixel data, and is therefore optimal for single block matching calculation processing without pixel data reusability. It is.
In addition, the systolic array type processor configured as shown in FIG. 2B has high reusability of pixel data, and the frequency of accessing pixel data to the cache memory is reduced. Therefore, the effect of reducing power consumption is extremely high. Therefore, it is optimal for continuous block matching calculation processing.

FIG. 3 shows the architecture of the image processor of the present invention. The architecture of the image processor of the present invention can be switched between SIMD type and systolic array type architecture according to the search algorithm in the motion detection process, as shown in FIG. Is provided.
The switching circuit 20 switches the data path such as pixel data read from the memory according to the search algorithm in the motion detection process.

In FIG. 3B, the switching circuit 20 constructs a data path indicated by a dotted line, and the four arithmetic circuits PE (3a to 3d) and the four arithmetic circuits PE (3e to 3h) perform arithmetic processing in parallel. This is equivalent to the architecture of a SIMD type processor.
In FIG. 3 (3), the switching circuit 20 constructs a data path indicated by a dotted line, and the four arithmetic circuits PE (3a to 3d) and the four arithmetic circuits PE (3e to 3h) are connected in cascade. Performing arithmetic processing is equivalent to the architecture of a systolic array processor.
In FIG. 3, the number of arithmetic circuits PE is four, but it goes without saying that the number can be freely changed. In FIG. 3, the number of stages connected to the systolic array cascade is 2, but it goes without saying that the number of stages can be freely changed.

  Next, FIG. 4 shows a schematic block diagram of the image processor of the present invention. The image processor according to the present invention includes a control unit 11, an address generation unit 12, an original image memory 13, a reference image memory 14, and a data path unit 15. The control unit 11 generates a control signal and a switching signal based on a command written from outside. This control signal is sent to the address generation unit 12 or the data path unit 15. The switching signal is sent to the data path unit 15. The address generation unit 12 continuously supplies addresses to the original image memory 13 and the reference image memory 14. The data path unit 15 reads pixel data from the original image memory 13 and the reference image memory 14, performs predetermined arithmetic processing, and delivers the processing result to the control unit 11.

The data path unit 15 includes a plurality of arithmetic circuits, reads pixel data from the original image memory 13 and the reference image memory 14, and operates in accordance with a switching signal sent from the control unit 11. And the input data of some arithmetic circuits are switched to the output data of other arithmetic circuits.
In the control unit 11, the arithmetic circuit of the data path unit 15 is switched between the SIMD type and the systolic array type in accordance with a search algorithm for motion detection processing.
In the search algorithm using single block matching calculation processing such as initial value search, prediction vector search, and three-step search such as three-step search, the control unit 11 is switched so that the calculation circuit of the data path unit 15 is SIMD type. A signal is sent.
On the other hand, in a search algorithm using a continuous block matching calculation process such as a full search method, a sub-sampling search method, a one-dimensional search, or a three-step search, the calculation circuit of the data path unit 15 is made to be a systolic array type. A switching signal is sent from the control unit 11.

  In the following examples, H.P. H.264 baseline profile integer pixel precision A more specific description will be given using a motion detection processing processor as an example.

Integer pixel accuracy In the motion detection processing processor, the search block is a block of 8 pixels × 8 pixels. FIG. 5 shows a search block 30 of 64 pixels (8 pixels × 8 pixels). In the first embodiment, for convenience of explanation, it is assumed that a motion detection process is performed by searching for a block (hatched portion in FIG. 6) of two pixels (one row and two rows) of eight pixels (one row). .
FIG. 6 shows a case where this motion detection processing is performed with the architecture of an image processing processor having a SIMD type configuration, and FIG. 7 shows a case where it is performed with the architecture of an image processing processor having a systolic array type configuration.

Here, the arithmetic circuit A1 (41) and the arithmetic circuit A2 (42) are both arithmetic circuits that input an original image and a reference image of 8-pixel data and output a sum of absolute differences between them.
The input lines and output lines of the arithmetic circuit A1 (41) and the arithmetic circuit A2 (42) will be described. The input lines are 64 data signal lines for capturing 8 pixel data from the original image memory 13 (8 pixels × 8 bits / pixel), 64 data signal lines for capturing 8 pixel data from the reference image memory 14, and control. A control line for taking in a load signal of the data signal line from the unit 11 is provided. The output line includes a data signal line for sending out the result data of the sum of absolute differences between the original image of 8 pixel data and the reference image, and, in the case of a systolic array type, the 8 pixel data of the reference image is output to the next stage arithmetic circuit. Are provided with 64 data signal lines to be transmitted.
In the arithmetic circuit A1 (41) and the arithmetic circuit A2 (42), eight arithmetic units for processing one pixel data are arranged in parallel and are used as an 8-way SIMD type processor.
The arithmetic circuit B (50) is an arithmetic circuit that inputs the sum of absolute differences of the arithmetic circuits A1 (41) and A2 (42) and outputs the result data of the sum of absolute differences of two rows of 8 pixel data. is there.

In FIG. 6, the arithmetic circuit A1 (41) and the arithmetic circuit A2 (42) of the 8-way SIMD processor are subjected to data processing in parallel, and both arithmetic circuits as a whole are used as a 16-way SIMD processor.
On the other hand, in FIG. 7, the arithmetic circuit A1 (41) and the arithmetic circuit A2 (42) of the 8-way SIMD processor are connected in series, and the output data of the arithmetic circuit A1 (41) is input to the arithmetic circuit A2 (42). Processing is performed as data. The arithmetic circuit A1 (41) of the 8-way SIMD processor and the arithmetic circuit A2 (42) of the 8-way SIMD processor are used as the systolic array processor.

  FIG. 8 shows the architecture of the image processor of the first embodiment. The architecture of the image processing processor according to the first embodiment is a motion detection processing processor including a control unit 11 and a data path unit 15. The control unit 11 generates a control signal in accordance with an external command, and generates an original image memory. 13 and the reference image memory 14 are successively generated via the address generation unit 15 and a switching signal is generated for the data path unit 15. The data path unit 15 includes an arithmetic circuit A (41) according to the original image memory 13, the reference image memory 14, a plurality of arithmetic circuits A (41, 42), and a switching signal sent from the control unit 11. , 42) and a switching circuit 21 for switching input data of some arithmetic circuits A (42) to output data of other arithmetic circuits A (41).

Next, a method for switching between the SIMD type and the systolic array type in the architecture of the image processor of the first embodiment will be described with reference to FIGS. 9 and 10.
FIG. 9 shows the architecture (SIMD type configuration) of the image processor of the first embodiment. A switching signal (1) is sent from the control unit 11 to the switching circuit 21, and a data signal line as shown by a dotted line in the switching circuit 21 in FIG. 9 is constructed.
As a result, the arithmetic circuit A1 (41) and the arithmetic circuit A2 (42) receive the pixel data from the original image memory 13 and the reference image memory 14, and each process them in parallel.

FIG. 10 shows the architecture (system configuration of a systolic array) of the image processor according to the first embodiment. A switching signal (0) is sent from the control unit 11 to the switching circuit 21, and a data signal line as shown by a dotted line in the switching circuit 21 in FIG. 10 is constructed.
As a result, the arithmetic circuit A1 (41) takes in the pixel data from the original image memory 13 and the reference image memory 14, and the arithmetic circuit A2 (42) takes in the output data of the arithmetic circuit A1 (41). Will be.

  Here, it is assumed that the original image memory 13 and the reference image memory 14 are 2-read port memories having 64 output data signal lines. In the case of the architecture of the SIMD type image processor as shown in FIG. 9, 128 output data signal lines (64 × 2) are occupied, but the architecture of the systolic array type image processor as shown in FIG. In this case, it occupies 50% of the 128 output data signal lines (64 lines × 2). One of the two read ports will be occupied. Therefore, the memory can be shared by other arithmetic processing units, and the motion detection processing algorithm can be efficiently executed.

The architecture of the image processor of the first embodiment can be switched between the SIMD type and the systolic array type according to the search algorithm in the motion detection process by the configuration as described above.
In the search algorithm using single block matching calculation processing such as initial value search, prediction vector search, and three-step search such as three-step search, there is no correlation of search block context. There is no reusability of pixel data. Therefore, instead of reusing the pixel data, the SIMD processor is configured so that the calculation can be performed at a very high speed, and the number of calculation processing cycles is reduced to improve the power consumption reduction effect.

  In addition, in a search algorithm using continuous block matching calculation processing such as a full search method, a sub-sampling search method, a one-dimensional search, or a three-step search, there is a strong correlation in the context of the search block. The reusability of pixel data with the matching calculation process is high. Therefore, the reusability of the pixel data is high, the frequency of accessing the pixel data to the cache memory is reduced, and the configuration of a systolic array type processor that increases the power consumption reduction effect is obtained.

In the first embodiment, two lines of 8 pixel data, an original image and a reference image of 16 pixel data are taken in, the difference absolute value thereof is calculated, and then the sum of all the difference absolute values is calculated. The circuit for switching between the SIMD type and the systolic array type in the output data path unit has been described.
Next, in Example 2, the 8 pixel data for 4 rows, the original image and the reference image of 32 pixel data are taken in, the difference absolute value thereof is calculated, and then the sum of all the difference absolute values is calculated. A circuit for switching between the SIMD type and the systolic array type in the data path unit for outputting the results will be described.

  FIG. 11 shows the architecture of the image processor of the second embodiment. The architecture of the image processor according to the second embodiment is a motion detection processing processor including a control unit 11 and a data path unit 15. The control unit 11 generates a control signal in accordance with an external command, and generates an original image memory. 13 and the reference image memory 14 are continuously generated through the address generation unit 15, and a switching signal is generated for the switching circuit (21, 22, 23) in the data path unit 15.

  The data path unit 15 corresponds to the original image memory 13, the reference image memory 14, the plurality of arithmetic circuits A 1 to A 4 (41, 42, 43, 44), and a switching signal sent from the control unit 11. The parallel number of the arithmetic circuits A1 to A4 is switched. That is, the switching circuit 21 switches whether the 16 arithmetic units of the arithmetic circuit A1 (41) and the arithmetic circuit A2 (42) are used as SIMD type processors or systolic array type processors. Further, the switching circuit 22 switches whether the 16 arithmetic units of the arithmetic circuit A2 (42) and the arithmetic circuit A3 (43) are used as SIMD type processors or systolic array type processors. In addition, the switching circuit 23 switches whether the 16 arithmetic units of the arithmetic circuit A3 (43) and the arithmetic circuit A4 (44) are used as SIMD type processors or systolic array type processors.

  According to the image processor of the present invention, low power consumption is achieved in data processing of high-resolution moving images, and it can be easily incorporated into various systems that require low power consumption. Is possible.

Conceptual diagram of single block matching operation processing Conceptual diagram of continuous block matching operation processing (A) shows a general SIMD type processor architecture, and (B) shows a general systolic array type processor architecture. 1 shows the architecture of an image processor of the present invention. 1 shows a schematic block diagram of an image processor of the present invention. A search block of 64 pixels (8 pixels × 8 pixels) is shown. 2 shows an example of an architecture of an image processing processor having a SIMD type configuration. 1 shows an example of an architecture of an image processing processor having a systolic array configuration. 1 shows an architecture of an image processor according to a first embodiment. 1 shows an architecture (SIMD type configuration) of an image processor according to a first embodiment. 1 shows an architecture (a systolic array type configuration) of an image processor according to a first embodiment. 2 shows an architecture of an image processor according to a second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Random access memory 11 Control part 12 Address generation part 13 Original image memory 14 Reference image memory 15 Data path part 20, 21, 22, 23 Switching circuit 30 Pixel (8 pixel x 8 pixel) block 41 Arithmetic circuit A1
42 Arithmetic Circuit A2
43 arithmetic circuit A3
44 arithmetic circuit A4
50 arithmetic circuit B
100 Screen frame 101, 102, 103 Search block PE Processing element (Processor Element)

Claims (6)

A motion detection processor comprising a control unit and a data path unit,
The control unit generates a control signal according to an external command, continuously generates addresses for the original image memory and the reference image memory, and generates a switching signal for the data path unit;
The data path unit includes the original image memory, the reference image memory, a plurality of arithmetic circuits, and the parallel number of the arithmetic circuits and input data of some arithmetic circuits according to the switching signal. A switching circuit for switching to output data of the arithmetic circuit;
An image processor capable of switching between a SIMD type and a systolic array type according to a search algorithm for motion detection processing.   The switching signal has a SIMD type configuration by maximizing the parallel number of the arithmetic circuits when performing single block matching arithmetic processing, and the parallel number of the arithmetic circuits when performing continuous block matching arithmetic processing. 2. The image processor according to claim 1, wherein the image processing processor is configured to be a systolic array type by switching input data of some arithmetic circuits to output data of other arithmetic circuits.   2. The image processor according to claim 1, wherein the arithmetic circuit is a circuit that inputs pixel data of an original image and pixel data of a reference image and outputs an absolute difference value.   2. The circuit according to claim 1, wherein the arithmetic circuit is a circuit that inputs a plurality of pixel data of an original image and a plurality of pixel data of a reference image and outputs a sum of absolute differences. Image processor.   5. The image processor according to claim 1, wherein the original image memory and / or the reference image memory is a two-read port memory. 6. An information terminal device equipped with the image processor according to claim 1.