JP2647377B2 - Image processing device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that performs high-speed processing and parallel processing of image data using a control technique of an image memory.

[Prior Art] Generally, when processing an image at high speed, a method of performing processing by software is used as processing by a computer. However, as image data becomes enormous, speeding up is required. There are two methods for speeding up,
One is a serial processing type hardware called a pipeline system, and the other is a parallel processing type in which a plurality of processors are provided. In the former case, the clock frequency of the processing is increased due to the high-speed processing of the image data, and there is a limit. On the other hand, the latter can increase the speed as much as possible by increasing the number of processors placed in parallel. Extremely speaking, it is one of the technologies that are currently attracting attention because it is possible to obtain the maximum speed by placing processors for the number of pixels.

By the way, at this time, communication processing between the pixels becomes important, and it is necessary to perform processing while performing mutual communication.
In such a parallel processing system, it is impossible to have processors as many as the number of pixels when handling high-resolution data. For example, when dealing with an image in which A4 is read at 16 pixels / mm (pe1), the number of pixels is about 16 M pixels (pixels), and it can be said that it is impossible to have this many processors simultaneously.

[Problems to be Solved by the Invention] The present invention provides high-speed and high-quality image processing, such as filtering, in which a target pixel is processed using its peripheral pixels by parallel processing of a small number of processors. The present invention provides an image processing apparatus for performing the above.

[Means for Solving the Problem] In order to solve this problem, an image processing device of the present invention processes an attention pixel using its surrounding pixels (in the embodiment, FIG. 36, FIG. 38, wherein each processor element simultaneously processes each area of the two-dimensionally divided image.
A processor unit composed of elements (corresponding to 1 to 4, or 1,1 to 3, 3 in the embodiment), wherein the processor element overlaps a boundary portion with an adjacent region in both two-dimensional directions. (Corresponding to FIG. 35, FIG. 37 (a), FIG. 39, page 62, line 7 to page 72, line 11 in the embodiment) to process the pixel of interest at the boundary. (In the embodiment, FIG. 37 (b), FIG. 40,
(Corresponding to page 72, line 12, to page 75, line 3).

Example An example of the present invention will be described below.

The configuration of the image processing apparatus of this embodiment includes an image memory 1 for one page, a processor unit 2 and a peripheral unit 3 such as an input / output device. FIG. 1 shows the principle configuration of only the basic part. A processor unit 2 is connected to an image memory 1. The nxm image data at an arbitrary position on the image memory 1 is transferred to a processor unit 2 comprising an array of nxm processor elements 2a.
After the high-speed processing, the image data is returned to the image memory 1 again. n
Each processing in the array of xm processor elements 2a is a so-called parallel processing architecture in which the processing is performed simultaneously. 9 (a) and 9 (b) show another configuration. In FIG. 9 (a), under the control of the control circuit 94,
The image data from the input side image memory is subjected to predetermined processing in parallel by a plurality of pixels in a processor unit 92 composed of a plurality of processor elements, and is stored in an output side image memory 93. On the other hand, FIG. 9 (b) shows a configuration in which the image memory 91 or 93, the processor unit 92, the input device 96 and the output device are connected by a common bus.

 Hereinafter, the image memory 1 will be described in detail.

For the sake of simplicity, assume that the image size is 1024 x 1024 pixels,
We will proceed with an image memory having bit / pixel data.
Changing the image size only requires extending the architecture of the present embodiment. Also, processor unit 2 is 4 ×
4 and a total of 16 processor elements 2a.

FIG. 2 is a diagram showing the configuration of the image memory 1. If the image is composed of 1024 x 1024 pixels as shown in the figure,
If this is divided into 4 × 4 units, 256 × 256 total 64
It is divided into K (= 65536) blocks. Now, this is rearranged in units of 4 × 4 pixels as shown in FIG.
Assume that there are K (each pixel has 8-bit length data). Therefore, the address space of the memory is 4 × 4 × 64K
Is a three-dimensional address specification. Assuming that one memory chip handles one 64K pixel in 4 × 4, 64K
In this address space, a memory chip is required in which each address is 8 bits deep. This requires a memory chip with a capacity of 512K bits (= 64K bytes). In this embodiment, two 256K bits of dynamic RAM (D-RAM) are used in combination. That is, 64K of 256K bit D-RAM
A 64K × 8 bit is used by using two of a × 4 bit configuration. These two memory chips will hereinafter be referred to as memory elements 1a.

The above image memory 1 corresponds to 4 × 4 matrix.
Is composed of 16 memory elements 1a. FIG. 4 shows the configuration of such a 4 × 4 memory element 1a.
Each memory element 1a receives a row address and a column address, and inputs / outputs image data of one pixel of 4 × 4 pixels in a 64K address space. Row address generator 4 and column address generator 5
Gives an address to each 4 × 4 memory element 1a. If the memory element 1a is a D-RAM that gives a row address and a column address in a time-sharing manner, only one address generator is required.
At this time, time division switching control between the row address and the column address is required.

By giving each address from such an address generator, a 4 × 4 pixel memory element
1a can be read / written. That is, image data for 4 × 4 pixels can be driven simultaneously by one address designation. Therefore, the data line
It is assumed that an 8-bit data line directly exits from each memory element 1a.

Assuming that data having a row address of A (0 ≦ A ≦ 255) and a column address of B (0 ≦ B ≦ 255) is called from the image memory 1, the image data is the second data.
The 8-bit image data of 4 × 4 pixels corresponding to the address (A, B) in the figure is read.

Further, simultaneous access of a plurality of pixels will be generalized and described.

FIG. 10 shows one page of the image as it is, and as shown in FIG.
It is divided by the block of pixels and is made to correspond to k × l memory elements 1a as shown in FIG. Blocks of k × l pixels are numbered from the end as (0,0), (0,1), (0,2), (0,3), and k × l pixels as shown in FIG. Corresponds to the memory unit 1 including the memory elements 1a. Thirteenth
The figure shows the memory unit 1 two-dimensionally. Further, since the memory size to be accessed is a unit of the block size of k × l pixels, even if a block R of k × l pixels at an arbitrary position is accessed, k × l memory blocks are accessed.
All the elements 1a are accessed, and one address is accessed for each memory element 1a.

As described above, the image data of k × l adjacent pixels at an arbitrary position in the image are accessed at a time, read, and then processed by the processor unit 2. The image data processed by the processor unit 2 is again k ′ ×
It is possible to access and write an arbitrary position with a block size of l 'pixels. Here, the following description will be made on the assumption that k ′ = k, l ′ = 1

Supplementary explanation of the above-mentioned memory access of only k '× l' pixels will be described. If the processing in the processor unit 2 is a spatial filter processing or the like, the access on the write side is more than the block size k × l accessed on the read side. Block size may be reduced. Generally, the block size k '× l' on the writing side is 1 × 1.
There are many processes that become Also, when the processing in the processor unit 2 is image reduction processing, the block size accessed on the write side becomes smaller than the block size k × l accessed on the read side.

In general, for the block size k '× l' on the write side, the minimum integer satisfying k'≥αk, l'≥βl is k ', l' when the vertical and horizontal reduction ratios are α and β. If the read and write memories have the same memory configuration or the same k × 1 memory configuration, and the processing as in the above two examples is performed, the configuration size k × l of the memory unit 1 on the writing side is smaller. Writing must be performed to the size k ′ × l ′. In this case, it is necessary not to access all k × l memory elements 1a, but to mask the memory elements 1a that do not correspond to writing so as not to access them. However, in the image memory 1 composed of k × 1 memory elements 1a, the data that can be accessed and read at one time is a maximum of k × l of adjacent image data, but is smaller than the adjacent image data. The image data of k '× l' can be freely accessed by performing the masking. Simultaneous access to only k ′ × l ′ by masking can be easily performed by operating the chip enable of the memory element 1a.

Next, step by step, in the embodiment of the memory access of a predetermined pixel at an arbitrary position, the memory unit configuration is 4 × 4.
And the case of k × 1 will be described, and the control of the chip enable for masking will also be described.

First, an example in which the block size k × l is 4 × 4 will be described.

FIG. 5 is an enlarged view of a part of FIG. A process for reading image data of an arbitrary 4.times.4 block S from the image memory 1, processing the image data in the processor unit 2, and then transferring the image data to an arbitrary 4.times.4 block T will be described. The 4 × 4 grid on FIGS. 5 and 6 is a grid that divides 16 4 × 4 memory elements 1a. Aa, A are temporarily stored in these 16 memory elements 1a.
Name them b,…, Ba, Bb,… Ca,… Dc, Dd. First, when a 4 × 4 block S is read, (N, M) is given as (row address, column address) to the memory element Dd among the 16 memory elements 1a.
(N, M + 1) is stored in the memory elements Db, Dc, Dd, and (N + 1, M) is stored in the memory elements Ad, Bd, Cd.
The elements are given (N + 1, M + 1). This is generated by the row address generator 4 and column address generator 5 described above. When the position of the end point u of the 4 × 4 block S is determined, the horizontal and vertical position addresses are divided by 4, and the remaining numbers n and m are divided into the memory elements Aa to Dd. It is clear that the row address and column address to be attached are uniquely determined. Assuming that the position address of u is u (Y, X), Y = 4N + n (n = 0,1,2,3) X = 4M + m (m = 0,1,2,3) For example, address generators 4,5 Then, the information of M and N and m,
n information into a lookup table, etc.
A configuration that outputs an address given to the elements Aa to Dd is also conceivable. At this time, it is clear from the above description that the output is one of M, N, M + 1, and N + 1. Utilizing this property, as shown in FIG. 7, n or m is input to the lookup table, 0 and 1 are output according to this value, and the address N to be given to the memory elements Aa to Dd is inputted. Alternatively, it is sufficient to control whether M is incremented or not. The row address generator 4 uses n and N,
In the column address generator 5, m and M are used.

In this way, as described above, the addresses are given to the 16 4 × 4 memory elements from the address generators 4 and 5, and 16 data can be obtained simultaneously.

These 16 data are transferred to the 4.times.4 block T shown in FIG. 5 again without any processing or any processing in the processor unit 2. However, the image data read from the 16 memory elements Aa to Dd are not always transferred to the same memory elements Aa to Dd. 4x in Fig. 5
When the four memory blocks S are transferred to the 4 × 4 memory blocks T, the data read from the memory element Aa in the 4 × 4 memory blocks S must be transferred to the memory element Dc. Must.

When the 4 × 4 memory blocks S and T have their end points u and v at arbitrary positions (Y, X) and (Y ′, X ′), the 16 memory blocks Aa to Dd Which memory element among the memory elements Aa to Dd should be written with the read data described above will be described.

 As shown in FIG. 5, Y = 4N + n (n = 0, 1, 2, 3) X = 4M + m (m = 0, 1, 2, 3) Y '= 4P + p (p = 0, 1, 2, 3) X '= 4Q + q (q = 0,1,2,3), pn = 4y' + y (y '=-1,0 y = 0,1,2,3) ... qm = 4x '+ X (x' =-1,0 x = 0,1,2,3) ... x and y are obtained.

First, a row array A consisting of (Aa, Ab, Ac, Ad) is shifted rightward by x
Rotate once. This is named a row array A '.
Similarly, row arrays B, C, and D rotated rightward x times are named row arrays B ', C', and D '.

Next, an array (ABCD) 'composed of row arrays A', B ', C', and D '
Is rotated downward y times.

In the case of FIG. 5, since it is clear from FIG. 5 that n, m, p, and q are 3, 3, 2, 1, y ′ = − 1, y = 3, x ′ = −
1, x = 2 is obtained. Therefore, the following matrix is obtained from the above description.

When rotated to the right twice, the row array A '= (Ac, Ad, Aa, Ab) B' = (Bc, Bd, Ba, Bb) C '= (Cc, Cd, Ca, Cb) D' = (Dc, Dd, Da, Db) When rotated downward three times, (Bc, Bd, Ba, Bb) (Cc, Cd, Ca, Cb) (Dc, Dd, Da, Db) (Ac, Ad, Aa, Ab)… When this matrix is compared with the basic array below, Aa, Ab, Ac, Ad Ba, Bb, Bc, Bd Ca, Cb, Cc, Cd Da, Db, Dc, Dd… Basic Array The basic array is simply a two-dimensional array of read data from the memory elements Aa to Dd arranged in order from left to right and from top to bottom. A matrix consists of data to be written to the memory elements Aa to Dd arranged in order. This corresponds to a two-dimensional array. That is, as an example, the data read from the memory element Aa is written in the fourth row and the third column in the array. When this is referenced to the basic array, Dc is in the fourth row and third column, so that it is sufficient that the read data of the memory element Aa is written in the memory element Dc.

As a supplementary explanation, it is easy to notice that the read data of the memory element Aa shown in FIG. 5 should be written to the position of Dc. However, the displacement from the position of Aa to the position of Dc is caused by the change from the position address u to the position address v. Equal to displacement. Further, since the configuration of the memory element 1a is 4 × 4, the remainder obtained by dividing both the horizontal and vertical positions by 4 can be considered as the displacement x, y of the memory element. For example, if the displacement of u, v is a multiple of 4, the displacement x, y becomes 0, and the data read from a certain memory element is written to the same memory element after the processing is performed. .

The hardware implementation of the above processing will be briefly described. FIG. 8 shows that the data simultaneously read from the memory element 10 consisting of 4 × 4 16 memory elements 1a is processed by the processor unit 2 and the data is x-rotated by the x-displacement rotator 81 by four elements each. Rotate the number of times. Thereafter, rotation is performed by the number of y by the y displacement rotator 82, and each is written to the memory elements 1a of Aa to Ad, Ba to Bd, Ca to Cd, and Da to Dd.

It is needless to say that the y-displacement rotator 82 can be composed of exactly the same four elements as the x-displacement rotator 81 since the input is four-element data. Also, the rotator may have the same number of bits as the depth of the memory data, and it is needless to say that the rotator may use the same number of bits as the depth of the memory data. or,
It can be easily inferred that the rotator can use a shift register, a barrel shifter, or the like.

Considering this more generalized, the memory block is k
When the size is × 1, the configuration of the memory unit 10 is also k × 1. In this case, k ×
When 1 memory block S is processed by the processor unit 2 and then transferred to a k × 1 memory block T at an arbitrary position, Y = kN + n (n = 0, 1,..., k−1) X = lM + m (m = 0,1, ..., l-1) (N, M, P, Q is 0,1,2,3 ...) Y '= kP + p (p = 0,1, ..., k-1) X '= LQ + q (q = 0, 1,..., Q−1) where the position address of the end point of S is (Y, X), and the position address of the end point of T is (Y ′, X ′) (10) n, m, p, q are obtained, and pn = Ky '+ y (y' = 1, 0, y = 0, 1, 2, 3,..., k-1) qm = lx '+ x (x '= -1,0, x = 0,1,2,3, ..., l-1) (11) Using x and y, for example, an x displacement rotator 81 and a y displacement rotator as shown in FIG. What is necessary is just to process using 82. In this case, the x displacement rotator 81 has l inputs and can shift from 0 to l-1. The y displacement rotator 82 has k inputs and can shift from 0 to k-1. In addition, since each of the k inputs of the y displacement rotator 82 has one element, the rotator of one input element has one element.

As shown in FIG. 10, the access control of the memory element for simultaneous access of the aforementioned k'.times.l 'blocks will be described.

The position address of the end point i of the block k ′ × l ′ is (f,
g). When reading a memory to be accessed in accordance with the above equation (10), f and g are substituted for Y and X, and when writing to a memory to be accessed, f and g are substituted for Y 'and X'. Substituting the result into equation (11) yields y, x,
The embodiment shown in FIGS. 7 and 8 can be applied to a generalized version of k × l.

At this time, of the k × l memory elements,
Only k'.times.l 'memory elements are chip enabled. The chip to be enabled is given by the formula (1) if only the position address of (f, g) at the end point i of k ′ × l ′ is determined.
0), n, m or p, q is uniquely determined, and k ′ × l ′ memory elements to be accessed are also uniquely determined.

By the way, in the memory configuration composed of k × 1 memory elements as described above, the read access side simultaneously accesses k ′ × l ′ blocks and the write side simultaneously accesses k ″ × l ″ blocks. (Where 0 ≦ k ″ ≦ k, 0 ≦ l ″ ≦ l), but this is also the same as described above. An embodiment of the control of the chip enable given to the memory element in this case will be described.
It is shown in Figure 14.

When the position addresses of the end points of the blocks of k ′ × l ′ and k ″ × l ″ are (Y, X) and (Y ′, X ′), n, m and p, q are obtained from equation (10). . These n, m and p, q are input to the data input of the selector. Further, a read / write signal R / W for memory access is input as a selection control signal of the selector, and n and m are selectively output at the time of reading, and p and q at the time of writing.
Is selected and output.

Similarly, the block size, k ', l' and k ", l" are also input to the selector, and the R / W signal is input as a selection control signal. At the time of reading, k ', l' is selectively output, and at the time of writing, k ", l" is selectively output. By the way, the memory elements to be accessed are n, m, k ', l' on the read side,
It is obvious that if k ″, l ″, p, q on the write side is determined, these data output from the selector are input to the lookup table, and k × l memory elements And outputs a signal for controlling the memory to be accessed.

By the way, if the image memory 1 before and after processing by the processor unit 2 is another memory, and its memory configuration is k × l and K × L, respectively, two look-up tables are stored as shown in FIG. It is easy to guess what should be used. In this case, the lookup table 151 and the lookup table 152 are tables having different contents.

There is no problem even if k = K and l = L. With the above-described configuration, it is possible to partially mask the memory elements to be accessed, instead of all k × 1 memory elements. The configuration of the memory element of k × l may be set to the size of k × l required at the maximum.

Next, how to access memory elements to process all image data corresponding to the entire previous screen, that is, a method of scanning all memory data will be described.

For example, the position address of the end point u of an adjacent k × l block to be accessed, that is, the number when counting from 0 in the vertical direction from the end, and the number when counting from 0 in the horizontal direction from the end The method of accessing the memory when Y and X are determined when X is set has been described above. Next, an embodiment will be described in which order X and Y are scanned to process all images.

(First Example) A method of scanning by increasing / decreasing the position addresses Y and X of image data for accessing a k × 1 memory element by integer multiples of k and l, for example, first setting Y and X to 0 Is set, and X is sequentially increased by l. After X is increased to the end point in the horizontal direction, X is reset to 0, Y is increased by k, and X is increased by 1 again. This is sequentially repeated to scan the entire screen or a part of the screen. This is temporarily referred to as a first sequential scan method.

(Second example) In addition, X and Y are not sequentially increased or decreased as described above, but successive k × l blocks throughout the entire image are accessed in an intermittent manner. , Y is a first random scan method when the displacement is an integral multiple of kl.

(Third example) A method of scanning by increasing and decreasing the position addresses Y and X of image data for accessing a k × 1 memory element by integers, for example, first, sets Y and X to 0, and sets X to X Increment one by one. After increasing X to the end point in the horizontal direction, X is reset to 0 again, Y is increased by 1, and X is incremented by one. This is sequentially repeated to scan the entire screen or a part of the screen.
This is temporarily referred to as a second sequential scan system. In this case, the same memory data is accessed many times.

(Fourth Example) Also, without increasing or decreasing X and Y in a sequential manner as described above, k × l blocks throughout the entire image screen are accessed in a discrete manner, and this is executed for all X and Y. . Or, for all X and Y of a continuous part of the whole screen. When it is random, it is tentatively named a second random scan method.

(Fifth example) In a memory configuration having k × 1 memory elements, when the memory block to be accessed is k ′ × l ′, the address Y is located at (1 ≦ k ′ ≦ k, 1 ≦ l ′ ≦ l). , X is incremented or decremented by an integer multiple of k ', l', and this is sequentially repeated to scan the entire screen. This is distinguished from the first sequential scan method, and is referred to as a block-wise sequential scan method.

(Sixth Example) Also, without successively increasing and decreasing X and Y as in (Fifth Example), successive k '× l' blocks throughout the entire image screen are accessed in a discrete manner. Y,
When X is a displacement of an integral multiple of k '× l', this is tentatively named a blockwise random scan method.

(Seventh example) Regarding the k × l memory configuration of the memory element, the one that scans sequentially, for example, the one that changes X and Y every arbitrary number d ′ and f ′ to scan, is simply sequenced. This is referred to as the "Shirskyan method".

(Eighth Example) In the case of random scanning in (Seventh Example) or
Even in the case of Example), when memory access is not performed for all combinations of X and Y, it is simply called a random scan method.

As described above, there are a number of scan methods that can be considered. Separately, there are two types of memory access: read-side memory access, and read-side memory access and write-side memory access. They do not always match.

In this scan method, if the read side is determined, X 'and Y' to be accessed on the write side are determined by the processing contents of the processor unit 2. Further, the scan method on the light side may be determined first. In this case, the scan on the read side is determined by the processing content.

Also, the block size k ', l' to be accessed on the read side and the write side may be different, and the size of the memory element configuration k × l may be different.

[Density Conversion, Color Correction, Mask Calculation] In the case of processing in density conversion, color correction, and the like, processing performed by a method employing the first sequential scan method on both the read side and the write side will be described in further detail.

The color conversion is a process of converting color information into other predetermined color information when image data has specific color information.

The mask operation is to output only a specific part of the image data as it is as shown in FIG. 20 and to output image data (for example, a white background) corresponding to the base under the other conditions. As a base, data of a gray or colorless ground may be output in addition to a white ground. FIG. 20A shows data indicating an area to be masked, FIG. 20B shows an image to be masked, and FIG. 20C shows an output result thereof.

In the density conversion, for example, each processor element in the processor unit may operate according to the flowchart shown in FIG. Here, the output value V _out corresponding to the input value V _in, for _{example, V out = 1/64 ·} V in 2 ... (12) are defined in such as the 4 × 4 pieces of 8-bit length of data Each processor element corresponds to one processor element, and a processor unit composed of a total of 16 processor elements has 4 × 4 elements corresponding to the addresses (A, B) in FIG. Eight-bit data is input. The respective processor elements operate in parallel and output output data. Therefore, 4 × 4 8 accessed at once
Bit-length data is processed by the processor unit at a time and output at a time, enabling high-speed image processing.

Each processor element performs processing such as density conversion, color conversion, and mask calculation.

In the density conversion, for example, an input value is changed to a corresponding output value in accordance with an input density-output density correspondence as shown in FIG. 19 and output. As a result, the contrast of the image can be enhanced, and the tone of light and dark can be changed. (12) Execute the operation according to the expression.

In the color conversion, for example, each processor element in the processor unit may operate according to the flowchart shown in FIG. Here, the color information is represented by, for example, a combination of R, G, and B.
And B, respectively, and are held with a data length of 8 bits each. In this case, the corresponding R, G
And B image memory cells correspond to one and the same processor element. Specific R, G, B of these
Is registered in advance as a designated color, and the changed color information is also registered in advance as a combination of R, G, and B values. Of course, a plurality of designated colors may be used.

The mask operation means that, for example, each processor element in the processor unit operates in accordance with the flowchart shown in FIG. Here, the mask information is bit data held in a memory constituted by a bit map memory corresponding to each cell of the image memory, and is data indicating whether or not the cell is within the mask. Also in this case, one identical processor element corresponds to the corresponding cell in the mask memory and the image memory.

If communication can be performed between the processor elements in the processor unit, processing such as spatial filtering, recognition, compression, and decoding in the processor unit can be executed.

Second Embodiment k × l for Accessing k × l Data Simultaneously
A second embodiment of allocating image data to the memory elements will be described. FIG. 16 is a diagram showing a state in which the upper part of one image is replaced with data, which is divided into equal parts in the horizontal direction and equal parts in the vertical direction. At this time, for explanation of the area divided into k × l, (0,
(0,1),... (0, l),..., (K, l), each of these areas is assigned to each memory element as shown in FIG. The method of allocation is as follows.
Address, and the next image data is assigned to address 1 of each memory element.
When all the lines have been allocated, the second line is similarly allocated from left to right, and all the image data are allocated. Then, for all k × 1 memory elements, the row address generator 4 shown in FIG.
And when the addresses given by the column address generator 5 are all the same, as shown by the hatched portion in FIG.
You can access discrete image data at once.

With such a configuration, a certain address is designated to read the image memory 1, and the processor unit 2
After receiving the processing in, there is a possibility that data can be written without changing the address when writing to the k × 1 memory elements 1a. For example, as shown in FIG. 16, when the area is composed of K × L pixel data, one part of one screen of the image is an integral multiple of L in the horizontal direction.
In the case of performing processing such as movement or transfer of a displacement of an integral multiple of K in the vertical direction, the read address and the write address may be the same. For this reason, the load related to address control such as the row address generator 4 and the column address generator 5 is extremely reduced.

This movement and transfer processing is processed in the processor unit 2. As shown in FIG. 16, k × l image data and also image data over the entire screen are input to the processor unit 2 as shown by hatched broken lines in FIG. Has an integral multiple of L and K, so that k × l in the processor unit 2
It is sufficient to convert and transfer individual data, and sequentially execute processing from 0 for all addresses of the memory element. As a result, processing on the entire screen can be performed.

In this embodiment, k × l memory configurations are changed to, for example, 1 × l, k
By allocating one horizontal line or one vertical line in one screen of an image to each memory unit in a configuration such as × 1, the processing in the processor unit 2 can perform a histogram operation or a one-dimensional Fourier processing for one line of the image. It can be inferred that it can be applied to various image processing such as conversion. Further, at the time of simultaneous access of a plurality of pixels, there is no limitation on which data in one screen is allocated to which address of which memory element.

An application example of this memory configuration will be described.
The case of movement or the like is described above, but the rotation will be described. For the sake of simplicity, the configuration of k × l memory elements by k × l division is assumed to be k = 1, and
The description is made as follows. Also, one divided area K × L is K
= L. In this case, when one image screen is rotated by + 90 °, −90 °, and + 180 ° around the center of the image, the processor unit alternately exchanges 1 × 1 data to + 90 ° and −90 °. performed on l ^2/4 pieces of l × l the Roteshiyon four data having a phase of 90 ° increments with respect to the center data l × l in the process.

For example, rotation is performed on four pieces of diagonal data. The rotation of 180 °, carried out for l ^2/2 pieces of Roteshiyon two data with a phase-by 180 ° with respect to the center. By this operation, rough rotation is performed for one image screen. This is a rotation from l × l areas to the areas from FIG. 24 to FIG. Actually, even in the same area, rotation of + 90 °, −90 °, + 180 °, and the like must be performed.

Unless a rotation operation is performed in this area, the rotation within one image screen is not completed. The rotation operation in this area will be described. This operation is a process of converting an address at the time of reading a memory element to an address at the time of writing, and performs address conversion as shown in Table 1.

FIG. 18 is a block diagram specifically illustrating this operation.
In the figure, the selection signal changes according to the rotation angle, and the selector 181,
182, 183, and 184 are input as selection control signals.

When the normal rotation angle is 0 °, the outputs of the row address generator 4 and the column address generator 5 are supplied to the row and column addresses of the memory element as they are via selectors 181, 183 and selectors 182, 184, respectively. When the rotation angles are 90 ° and −90 °, the outputs a and b of the selectors 181 and 182 output the column address and the row address, respectively.

When the rotation angles are 0 ° and 180 °, row addresses and column addresses are output from the outputs a and b of the selectors 181 and 182, respectively. Furthermore, when the rotation angle is 90 ° and 180 °,
The selector 184 operates to select the output of 186, and the selector 183 operates to select the output of 185 when the rotation angle is −90 ° and 180 °. The arithmetic units 185 and 186 output a value obtained by subtracting the input data from the length L of one side of the area. With such a configuration, the processing shown in Table 1 is performed, and the rotation is performed over the entire screen of one image.

Next, how to access the memory element to process all image data corresponding to the entire screen, that is,
A method of scanning all memory data will be described.

For example, the position address of the end point u of the adjacent k × l block to be accessed, that is, the number when counting from 0 in the vertical direction from the end, is Y, and when the number is counted from 0 in the horizontal direction from the end. The method of accessing the memory when Y and X are fixed when the number is X has already been described. Next, an embodiment will be described in which order X and Y are scanned and all images are processed.

An example of the scan of the address given to the memory element will be described. Each area when one screen is divided into l × l areas corresponds to each memory element. Can be obtained by giving the same address to all the memory elements and incrementing the address sequentially from 0. memory·
The element address has a column address and a row address.
Increment the column from 0 to the last address. Thereafter, after the row address is incremented, the column is incremented again from 0 to the final address. This is repeated to access all of the memory elements.

By the way, a more detailed example of the image rotation processing will be described below.

To simultaneously access multiple blocks of the original image, input data from each block in parallel, and process and output this in parallel, rotate the image by 0 °, 90 °, 180 °, and 270 °. A method of outputting the data will be described.

Figure 24 shows the original picture divided into blocks,
The figure shows a state in which an area of pixels × 256 pixels is divided by a block composed of 4 pixels × 4 pixels.

FIG. 25 shows a state in which the original picture is rotated 90 ° counterclockwise in units of blocks. FIG. 26 shows a state in which each pixel in the block shown in FIG. 25 is rotated by 90 ° counterclockwise in units of pixels.

FIGS. 27 and 28 are diagrams showing the positional relationship of each pixel in the block in each block of the original picture shown in FIG. In FIGS. 24 and 25, the positional relationship of each pixel in each block is the same. If this is represented in FIG. 27, each pixel is rotated 90 ° counterclockwise in each block.
The rotated state is represented in FIG.

To obtain an image with the original picture rotated 90 ° counterclockwise,
The original picture represented in Fig. 25 is treated as a block-to-block relationship as shown in Fig. 26, and the pixels in each block are
This can be realized by changing the relationship from the diagram to the relationship shown in FIG.

FIG. 29 is a block diagram of an example of a circuit configuration for performing the series of processes. Reference numerals 1601 and 1602 denote image memories that hold the respective pixels of the original image shown in FIGS. 24 to 28, 1601 denotes an input side memory, and 1602 denotes an output side memory. Also, 1603
Is an arithmetic circuit composed of a processing circuit corresponding to each pixel data read in parallel. 1 for each block
Each of the processing circuits has data input from each block, and outputs the processed data to the corresponding block on the output side. Reference numeral 1604 denotes a block data selection circuit, the details of which are shown in FIG. Reference numeral 1605 denotes a circuit for outputting an address indicating which position (in-block address) in each block of the input side memory is to be accessed. Reference numeral 1606 denotes a circuit for outputting an address indicating which position (in-block address) in each block of the memory on the output side is to be accessed. A control circuit 1607 controls 1604, 1605, and 1606 in accordance with the rotation angle.

The block data selection circuit 1604 is constituted by selectors shown in FIG. 30 corresponding to the number of blocks (64 in this example). Each selector corresponds to one block of the output side memory, and the i-th block of the output side memory.
The selector corresponding to the block in the row, j-th column (hereinafter referred to as (i, j)) includes (i, j),
(M−j + 1, i), (mi−1, m−j + 1), (j, m−
Data from four blocks (j + 1) are input. m means that the input and output memories to be symmetric are both m blocks × m blocks.
= 8. Also, 1 ≦ i ≦ m and 1 ≦ j ≦ m.

The control circuit 1607 controls 1604 so that each selector outputs (i, j) if the original image is rotated by 0 ° in the counterclockwise direction, and (J, m−j + 1) if it is 90 °. , 180 °, (m−j + 1, M−j + 1), and 270 °, (m−j + 1)
+1 and i). Thereby, the conversion corresponding to FIG. 24 to FIG. 25 is realized.

The control circuit 1607 controls the addresses in the blocks 1605 and 1606. In the block, there are n pixels multiplied by n pixels, and when outputting to the pixel at the position (k, l) in the block, the address in the block on the input side is obtained by dividing the original picture by a half clock. (K, l) to output by rotating in the direction 0 °, and (l, n−k + 1) in the case of 90 °, 18
If it is 0 °, it is controlled so as to be (n−k + 1, n−l + 1), and if it is 270 °, it is controlled so as to be (n−l + 1, k). (In this embodiment, n = 4.) This means that 1605 is constituted by a look-up table, for example, as shown in FIG.
This can be realized by using a count-up counter as shown in FIG. 1605 and 1606 are opposite to FIG. 32.
Alternatively, of course, the configuration shown in FIG. 31 may be changed to 1606. As a result, pixel conversion corresponding to FIGS. 25 to 26 (or FIGS. 27 to 28) is realized.

Also, instead of FIG. 29, a block area selection circuit may be arranged on the output side of the arithmetic circuit as shown in FIG. In this case, the output from each processing circuit of the arithmetic circuit is
The operation is exactly the same as in the case of FIG. 29 except that four signals are input to each selector of the block area selection circuit.

As described above, according to the present embodiment, high-speed processing is possible by simultaneously taking in data of a plurality of pixels at positions corresponding to rotation, and rotation operation can be realized with a small-scale circuit.

Further, a supplementary explanation will be added to the embodiment described later.

Now, when the addresses to be given to the k × 1 memory elements are sequentially scanned from address 0 to the last address as described above, the image data of each area corresponding to the area of 1 / k × l of the entire screen is k. Output is performed from each of the × 1 memory elements. If attention is paid to only one memory element, an image corresponding to one area is sequentially scanned and read in the horizontal and vertical directions. If this is applied to processing of an image which is sequentially scanned in the horizontal and vertical directions over one screen of a conventional image, a processing speed of k × l times can be obtained. In addition, since the image area to be handled is small, for example, the line buffer is small. For this reason, the processor unit 2 can be easily configured as an array processor or the like. A detailed description is provided below.

As described above, the pixel data from the k × l memory elements are sequentially input to the processor unit 2 for performing the spatial filtering while sequentially scanning the memory. Although the processor unit 2 will be described later, each of the processor elements, which are constituent elements thereof, forms a buffer for a plurality of horizontal lines of a rectangular area corresponding to each of the above-described k × l divided areas. After a built-in and two-dimensional spatial filtering process is performed, the data is sequentially written to the writing side by the same scan as the reading side. Hereinafter, the embodiment will be described in detail.

FIGS. 34 to 37 show one screen as 2 × for easy explanation.
The memory elements are divided into two, and the memory elements have four 2 × 2 configurations. In this case, as shown in FIG. 34, the image memory on the read side is read so that the horizontal scan and the vertical scan are repeated in each section at the same time, but the images are allocated to the memories as described above. Therefore, the memory address to be accessed at this time increments the vertical address from 0 to the last address every time the horizontal address is repeated from 0 to the last address for all the memory addresses.

FIG. 36 shows a memory element R-1, R-2, R-3, R-4 constituting an image memory on the read side in the embodiment shown in FIG. Are stored.

Incidentally, the same address is given to the memory elements R-1 to R-4 by an address generator which can be controlled by a control circuit, and the address given to the image memory on the read side is scanned as described above. . The image data read from the memory elements R-1 to R-4 are input to the processor elements 1 to 4, respectively. The processor elements 1 to 4 constituting the processor unit receive necessary control signals from the control circuit and instruct processing contents. The spatial filtering is performed by each of the processor elements 1 to 4. Since each of the processor elements 1 to 4 has a line buffer, a horizontal address is supplied from an address generator. Alternatively, it is generated inside the processor elements 1-4.

The output data that has been subjected to the spatial filter processing by the processor elements 1 to 4 is written to the memory elements W-1 to W-4 that constitute the write-side image memory. At this time, the same address is given to the memory elements W-1 to W-4 constituting the write-side image memory, but an address delayed from the address given to the read-side memory is given. This delay amount corresponds to the delay amount of the pixel of interest in a line buffer or the like provided for performing spatial filtering in the processor unit.

By the way, with the configuration described above, it is possible to perform the spatial filter processing on almost all the space on the entire screen. However, the hatched portion and the gray portion as shown in FIG. 35 are the peripheral portions of each region. As a result, the two-dimensional processing mask of the spatial filter protrudes and is not processed. In order to interpolate this unprocessed area, the control circuit causes the address generator to generate a memory address corresponding to the shaded area, the gray area, and the periphery thereof, and sequentially captures image data from the read-side image memory. The spatial filter operation is performed by the CPU in the control circuit. The operation result is written to the write-side image memory, and its address is controlled by the control circuit to provide an address of the write-side image memory, similarly to the read-side memory. In the above embodiment, the processing in the spatial filter by the CPU in the control circuit takes some processing time, but the ratio of the shaded area and the gray area shown in FIG. 35 to the front of the screen is very low, and the effect on the processing time is small. Almost negligible.

An embodiment in which the arithmetic processing by the CPU is not performed will be described below.

FIG. 38 is a simplified block diagram of the present embodiment, which is almost the same as the previous embodiment except that the entire screen is divided into 3 × 3 areas and a data control circuit is provided. This embodiment will be described below in detail focusing only on the differences.

Further, in this embodiment, one screen is divided into 3 × 3 areas, and a case where the screen is divided into m × n areas (hereinafter referred to as a general case) will also be described. (M and n are positive integers).

FIGS. 37 (a) and 37 (b) show how to scan the entire screen divided into 3 × 3.

For each of the divided areas, an extended area indicated by a broken line is assumed as shown in the figure, and the output of the extended area is input to each processor element. This extended area is, for example, an unprocessed part after the spatial filter processing as shown in the first embodiment, particularly the area (1,1) and the area (1,1).
2), (2,1), and (2,2) are expanded to cover the unprocessed portion around the boundary. Also, for an arbitrary region (k, l) (where 1 ≦ k ≦ m, 1 ≦ l ≦ n), the region (k, l) and the regions (k, l + 1), (k + 1, l),
The unprocessed portion around the boundary of (k + 1, l + 1) is extended so that it can be covered even by spatial filtering.

For example, in a case where a 3 × 3 smoothing process is performed by a processor unit, one pixel is left unprocessed at the right end in each region (k, l). Further, in the adjacent area (k, l + 1), one pixel is not processed at the left end.
l) to the right. Similarly, it is necessary to extend two pixels downward.

The expanded regions (1, 1) to are expanded while scanning as shown in FIGS. 37 (a) and (b).
(3,3) is input to the processor elements (1,1) to (3,3). Here, the address given to the memory element will be briefly described. The control circuit causes the address generator to increment the horizontal address sequentially from 0 to the maximum address. Thereafter, the value is incremented from 0 to the address of the area extended rightward. If the pixel is extended by two pixels in the right direction, the address is output as 0 after the address is incremented to 0, 1 again. Next, the vertical address is incremented from 0, and the scanning of the horizontal address is repeated. The vertical address is incremented again, and the scanning of the horizontal address is repeated. After repeating this, the vertical address becomes the maximum address, and if the scanning of the horizontal address is repeated, then the vertical address is set to 0, and this is repeated up to the address of the area extended downward. At this time, each extended area (1,1)-(3,
Outputs up to 3) are from processor elements (1,1) to (3,
The data control circuit relays the data in order to correspond to 3).

Next, the data control circuit will be described. The data control circuit is as shown in FIG. M × n for general case
It is composed of 4to1 selectors.

The output of the selector is the processor element (1,1)
Output to memory elements (1,1), (1,
2), (2,1) and (2,2) are the inputs, and each selector outputs one of the four inputs according to the selection control signal from the control circuit. By the way, when the output of the selector is output to the processor element (k, l), the input is the memory element (k, l), (k, l + 1), (k + 1,
l), (k + 1, l + 1). However, if k + 1 and l + 1 exceed m and n respectively for the area division m and n due to arbitrary k and l, in particular, input nothing or dummy data to the input area section side. input.

Next, a selector selection control signal output from the control circuit will be described. As described above, the control circuit controls the address generator to increment the horizontal address from 0 to the final address and then scan from 0 to the address of the extended area, that is, while the horizontal address overlaps the area of (k, l + 1). At time, the output (k, l + 1) is selectively output. Also, the vertical address is 0
After incrementing from 0 to the final address, while scanning from 0 to the address of the expanded area (for example, 1 in 3 × 3 spatial filter processing), that is, the area expansion of (k, l) is (k + 1, l) If they overlap the area, (k + 1,
l) Selectively output. In the two cases, that is, when the expanded area overlaps (k + 1, l + 1),
If the control circuit supplements the data control circuit between the processor unit and the write-side image memory so as to select the output of the memory element (k + 1, l + 1),
The circuit configuration is the same as that in Fig. 39, but the input when the selector output is output to the memory element (k, l) is
(K, l), (k, l-1), (k-1, l), (k-1, l-1)
Becomes In addition, the extended area as shown in FIG.
l) does not cover the unprocessed portions of the three regions left, bottom, and lower left, but instead of the upper, lower, left,
When the area is expanded so as to cover only the right unprocessed portion, the data control circuit between the processor unit and the write-side image memory becomes unnecessary. In this case, the data control circuit between the image memory on the read side and the processor unit sends (k, l), (k) to the output (k, l) of the selector.
−1, l−1), (k−1, l), (k−1, l + 1), (k, l−
1), (k, l + 1), (k + 1, l-1), (k + 1, l),
The only requirement is that (k + 1, l + 1) be input.
Can be inferred more easily. To further explain, it is necessary that data of nine areas included in the extended area, that is, data of nine memory elements be selected by the selector and input to the processor element (k, l) in the processor unit. Yes, requires a data control circuit, but controls k × 1 selectors.

Next, the data output by the data control circuit is input to the processor elements constituting the processor unit, but the processor elements (1, 1) to
(3, 3) has a line buffer for several lines of the expanded area, and after performing a spatial filter process, passes through a data control circuit to form a memory for configuring a write-side image memory.
Output to elements (1,1) to (3,3). This data control circuit is also the same as that described above, and performs correction when the result of the spatial filter processing is in the area of the extended portion. At this time, the address given from the address generator to each memory element of the write-side image memory is the same. However,
A delay of an amount delayed by the processor element from the address given to the read side image memory is performed. In addition, even if spatial filtering is performed on the expanded area at this time, the spatial filtering is not performed on the periphery of the expanded area.
For example, in the case of processing such as 3 × 3 smoothing, the output area becomes smaller by one pixel width on the left, right, upper and lower sides than the expanded area. The memory for the write-side image is enabled only when the data in the area subjected to the spatial filter processing is output.

In the processor element, when the spatial filter processing is performed, the width of the output area becomes smaller at the top, bottom, left and right, and returns to the same size as the (k, l) area. Since it corresponds to the element (k, l), a data control circuit between the processor unit and the write-side image memory becomes unnecessary.

Next, the processor elements constituting the processor unit will be described in detail.

FIG. 40 is an example of a 3 × 3 smoothed space filter.
Image data input from the read-side memory element to the processor element via the data control circuit is input to the latch 2604 and the line buffer 2601. The output of line buffer 2601 is latch 2607 and line buffer.
Input to 2602. The output of the line buffer 2602 is input to a latch 2610. The line buffers 2601 and 2602 are supplied with the address given by the counter 2603 or the address given to the address generator (not shown) as shown in FIG. The counter 2603 receives an image transfer clock from the control circuit for every other horizontal pixel in the extension area, counts up the address, and inputs a clear signal from the control circuit for each horizontal line to clear the address. The line buffer can write and read input / output signals simultaneously.

For this operation, the latches 2604, 2607 and 2610 receive pixel data at a certain position on three consecutive horizontal lines.
The latches 2604 to 2612 are constituted by D-type flip-flops. The outputs of the latches 2604, 2607, 2610 are sequentially changed by the latches 2605, 260 by the image transfer clock from the control circuit.
It is output to the latches 2606, 2609, 2612 via 8,2611.

At this time, the outputs of the latches 2604 to 2612 are consecutive 3 × 3 pixels in one screen of the image. This output is input to an adder 2613, and after taking the sum of all pixels, it is input to a divider 2614 and multiplied by 1/9, that is, a 3 × 3 smoothing process is performed and the write side memory element Alternatively, the data is output to a data control circuit connected to the write-side memory element.
Although the second embodiment of the present invention has been described above, it goes without saying that the processor unit constituting the processor unit can be constituted by a processor such as a CPU.

In both embodiments, the periphery of one screen, that is, the 35th
It goes without saying that the spatial filtering of the gray part shown in the figure cannot be performed.

As described above, since one screen of an image is divided and divided into m × n areas, the image area covered by one memory element is reduced, and the corresponding spatial filter processing processor has a small area line buffer. Only need to have a multi-line buffer inside
It is now possible to use LSi-based processors. or,
CP that uses CPU as processor or has internal RAM
U and others are now available. Since the spatial filter processing is simultaneously performed on a plurality of m × n regions at the same time, the execution time of the processing is extremely reduced.

As described above, according to the present embodiment, by adopting a configuration of m × n memory elements, m × n pixels can be accessed and processed at the same time, and high-speed image processing can be performed.

In addition, a method of distributing m × n pixels in one screen to m × n memory elements so that they can be accessed simultaneously, and a method in which m × n pixels scattered in one screen can be accessed simultaneously. By adopting a configuration such as a method of distributing to memory elements, processing suitable for image processing such as two-dimensional filter processing of a continuous small space in the former case and processing such as movement in the latter case becomes easier To perform memory access.

Further, according to the present embodiment, the configuration of k × 1 memory elements makes it possible to mask all the memory elements without accessing them. That is, when performing processing such as a spatial filter, the processing result is 1 ×
Although there are many processes that result in a small block size such as 1 or the like, it is possible to perform a process in which the size of the memory element configuration and the size of the image to be accessed are different.

[Effect of the Invention] According to the present invention, a pixel of interest is processed using its peripheral pixels by parallel processing of a finite number of small processors.
For example, it is possible to provide an image processing apparatus that performs high-speed and high-quality image processing such as filtering.

[Brief description of the drawings]

FIG. 1 is a diagram showing the configuration of an image processing apparatus according to the present embodiment. FIG. 2 is a diagram showing one image corresponding to the address of a memory element. FIG. 3 is a memory comprising 4 × 4 memory elements. FIG. 4 is a diagram of a memory and an address generator provided thereto, FIG. 5 is a diagram showing one part of an image, FIG. 6 is a diagram showing memory allocation of one image, and FIG. 7 is a memory. FIG. 8 shows a block diagram of pixel data control, FIGS. 9 (a) and 9 (b) show a configuration of another image processing apparatus of the present embodiment, and FIG. 10 shows an image. FIG. 11 shows one screen, FIG. 11 shows k × 1 memory elements, FIG. 12 and FIG. 13 show one memory element, FIG. 14 and FIG. Diagram showing a control circuit for element access, FIG. 16 is a diagram showing one image screen, and FIG. FIG. 18 illustrates a memory element, FIG. 18 illustrates an address conversion circuit, FIG. 19 illustrates a density conversion, FIG. 20 illustrates a mask calculation, and FIG. 21 illustrates a density conversion. FIG. 22 is a flowchart illustrating a procedure of color conversion, FIG. 23 is a flowchart illustrating a procedure of mask calculation, FIG. 24 is a diagram illustrating an image data array before a rotation process, FIG. The figure shows a state in which rotation is performed within the range of a pixel block. FIG. 26 shows a state in which rotation is performed in units of pixels in a block. FIG. 27 shows the rotation processing before rotation processing and in units of pixel blocks. FIG. 28 is a diagram showing a pixel arrangement in a block after the pixel block. FIG. 28 is a diagram showing a pixel arrangement in the block after a rotation process is performed in the pixel block. Example diagram, Fig. 30 is block data selection circuit FIG. 31 is a diagram showing an example of realizing an intra-block address conversion circuit. FIG. 32 is a diagram showing an intra-block address generation circuit before conversion. FIG. 33 is another block configuration example of a circuit for performing a rotation process. , FIG. 34 is a diagram showing an area division and a scan on an image, FIG. 35 is a diagram showing a portion that is not subjected to spatial filter processing, FIG. 36 is a diagram showing a configuration of an image processing apparatus that performs spatial filter processing, FIG. (A) is a diagram showing the expansion and scanning of the area division, FIG. 37 (b) is a view showing a part of the divided area, and FIG. 38 shows another configuration of the image processing apparatus for performing the spatial filter processing. FIG. 39 is a diagram showing a data control circuit, and FIG. 40 is a diagram showing a processor element. In the figure, 1 ... image memory, 1a, 1b ... memory element,
2 Processor unit 2a Processor element 3 Peripheral unit 4 Row address generator 5
... a column address generator, 91 ... an input side image memory, 92 ... a processor unit, 93 ... an output side image memory, 94 ... a control circuit, 95 ... an input device, 96 ... an output device.

Continuation of the front page (72) Inventor Naoto Kawamura 3-30-2 Shimomaruko, Ota-ku, Tokyo Inside Canon Inc. (56) References JP-A-61-16369 (JP, A) JP-A-61-55783 ( JP, A)

Claims (1)

(57) [Claims] 1. An image processing apparatus for processing a pixel of interest using its peripheral pixels, wherein each processor element comprises a plurality of processor elements for simultaneously processing each area of a two-dimensionally divided image. An image processing apparatus, comprising: a processor unit comprising: a processor unit configured to access a boundary portion between adjacent regions in two-dimensionally overlapping directions and process a target pixel in the boundary portion.