JP6554373B2 - Data processor - Google Patents

Description

  The present invention relates to data processing.

  As described in Patent Documents 1 to 3, various data processing apparatuses have been proposed conventionally.

JP-A-8-328941 JP, 2013-186624, A JP, 2013-186705, A

  With respect to data processing devices, improvement in processing speed is desired.

  Therefore, the present invention has been made in view of the above points, and an object thereof is to provide a technique capable of improving the processing speed of a data processing device.

To solve the above problems, an aspect of the data processing apparatus according to the present invention includes a storage unit, a storage control unit for controlling said storage unit, and a processing unit, the storage control unit, by the processing unit Processing target reading processing for reading processing target data to be processed from the storage unit; writing processing for writing processed data which is the processing target data processed by the processing unit into the storage unit; and stored in the storage unit The output target data, which is the processed data to be output, is read out from the storage unit, and the storage control unit executes the processing target readout which is the execution request for the processing target readout process. A processing target read request output unit that outputs a request, a write request output unit that outputs a write request that is an execution request for the write processing, and the output target read When an output target read request output unit that outputs an output target read request that is an execution request for processing and the processing target read request, the write request, and the output target read request do not conflict, the processing target read request, When processing corresponding to an input request among the write request and the output target read request is performed in the storage unit, and at least two requests of the process target read request, the write request, and the output target read request conflict the, the selected one of the at least two request by arbitrating least two requirements, possess the arbitration unit which performs in the storage unit the processing corresponding to the selected request, the storage unit 1 In the port memory, the storage control unit writes the processed data into the storage unit. If the row is a storage area where the processing target data used is read in the generation of the processed data, and writes the processed data.

Another embodiment of a data processing apparatus according to the present invention includes a storage unit, a storage control unit for controlling said storage unit, and a processing unit, the storage control unit, a processing target to be processed by the processing unit A processing target reading process for reading data from the storage unit and a writing process for writing processed data, which is the processing target data processed by the processing unit, into the storage unit are executed, and the storage control unit performs the processing A process target read request output unit that outputs a process target read request that is an execution request of a target read process, a write request output unit that outputs a write request that is an execution request of the write process, the process target read request and the write If the requests do not conflict, the storage unit performs processing corresponding to the input request among the processing target read request and the write request. When the processing target read request and the write request conflict, the processing target read request and the write request are arbitrated to select one of the processing target read request and the write request, and according to the selected request The output control unit further includes: an arbitration unit that performs processing in the storage unit; and an output control unit that controls an output of the processed data output from the processing unit to the outside of the data processing apparatus. Outputs the processed data to be output, which has not been written to the storage unit, to the outside of the data processing unit , the storage unit is a one-port memory, and the storage control unit performs the processing When executing the writing process of writing data to the storage unit, the processing target data used in generating the processed data is read. In the storage area, and writes the processed data.

  Further, in one aspect of the data processing apparatus according to the present invention, processing priorities are assigned to the processing target reading request, the writing request, and the output target reading request, and the arbitration unit is configured to execute the processing target When at least two requests of the read request, the write request, and the output target read request conflict, the at least two requests are arbitrated based on the priority of the at least two requests.

  Further, in one aspect of the data processing device according to the present invention, processing priorities are assigned to the processing target reading request and the writing request, and the arbitration unit is configured to execute the processing target reading request and the writing request. , The processing target read request and the write request are arbitrated based on the priority of the processing target read request and the write request.

  In the aspect of the data processing apparatus according to the present invention, the priority can be changed.

  Further, in one aspect of the data processing device according to the present invention, the storage control unit reads first processed data, which is the processed data written in the storage unit in the writing process, as the processing target data. The processing target reading process and the writing process of writing second processed data obtained by the processing unit processing the first processed data into the storage unit are executed.

Further, in one aspect of the data processing device according to the present invention, a first buffer for temporarily storing the processing target data read from the storage unit is further provided, and the processing unit is configured to execute the processing from the first buffer. reading the processed data, processes the read the processed data, in the first buffer, the processing target data read by said processing unit is erased, the processing target read request output unit, the first When one buffer is empty, the processing target read request is output.

  Further, in one aspect of the data processing device according to the present invention, a second buffer for temporarily storing the processed data output from the processing unit is further provided, and the storage control unit further includes, in the writing process, The processed data stored in the second buffer is read, the read processed data is written to the storage unit, and the processed data written to the storage unit is erased in the second buffer, The write request output unit outputs the write request when the second buffer is full.

  Further, in one aspect of the data processing device according to the present invention, the output target data output from the data processing device is temporarily stored in a third buffer outside the data processing device; The read output target data is erased, and the output target read request output unit outputs the output target read request based on the availability of the third buffer.

  Processing speed can be improved.

It is a figure which shows the structure of an image processing apparatus. It is a figure which shows the structure of an image processing circuit. It is a figure for demonstrating the operation | movement of a coordinate transformation circuit. It is a figure which shows the structure of a coordinate transformation circuit. It is a figure which shows a mode that multiple types of rotation parameters are memorize | stored in the memory | storage circuit. It is a figure which shows an example of specific information. It is a figure for demonstrating the operation | movement of a coordinate transformation circuit. It is a figure for demonstrating a rotation parameter. It is a figure for demonstrating the operation | movement of a processing circuit. It is a figure for demonstrating operation | movement of a processing circuit. It is a figure which shows the structure of a processing circuit. It is a figure for demonstrating the operation | movement of an image processing apparatus. It is a figure showing composition of an image processing system. It is a figure which shows an example of a captured image typically. It is a figure which shows the structure of the determination apparatus. It is a figure which shows the structure of a feature image generation part. It is a flowchart which shows operation | movement of a determination apparatus. It is a figure which shows an example of a candy area | region typically. It is a figure which shows an example of a background image typically. It is a figure which shows an example of a background difference image typically. It is a figure which shows an example of an external shape template typically. It is a figure for demonstrating template matching. It is a figure for demonstrating the extraction process of a sweets area | region. It is a figure which shows an example of an edge image typically. It is a figure for demonstrating the production | generation method of a synthesized image. It is a figure which shows the structure of the modification of a coordinate transformation circuit. It is a figure for demonstrating operation | movement of the modification of a coordinate transformation circuit. It is a figure which shows the structure of the modification of a coordinate transformation circuit. It is a figure which shows the structure of the modification of a processing circuit. It is a figure which shows the modification of a conversion parameter list | wrist. It is a figure which shows the modification of specific information. It is a figure for demonstrating operation | movement of the modification of an image processing circuit. It is a figure which shows the structure of the modification of a coordinate transformation circuit. It is a figure which shows the structure of the modification of a processing circuit. It is a figure which shows the structure of the modification of a processing circuit.

<Overall configuration of image processing apparatus>
FIG. 1 is a diagram showing the configuration of the image processing apparatus 1. The image processing apparatus 1 is an apparatus that performs various processes on an image to be processed, and includes a central processing unit (CPU) 2, a system memory 3, and an image processing circuit 4. The processing target image is, for example, a captured image captured by a camera or an image obtained by performing a predetermined process on the captured image. The CPU 2, the system memory 3 and the image processing circuit 4 are mutually connected via, for example, a bus 5.

  The CPU 2 is a kind of processor and operates using software (program). The CPU 2 centrally manages the overall operation of the image processing apparatus 1 by controlling other components of the image processing apparatus 1. The CPU 2 executes various programs (various software programs) in the system memory 3 to realize various functions in the image processing apparatus 1.

  The system memory 3 is configured by, for example, a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory). The system memory 3 stores an image to be processed and various programs.

  The image processing circuit 4 reads the processing target image from the system memory 3 and performs processing on the read processing target image. The image processing circuit 4 performs, for example, coordinate conversion of the image on the processing target image. Then, the image processing circuit 4 writes the image obtained by the processing on the processing target image into the system memory 3. The image processing circuit 4 is, for example, a circuit that operates without using software, and is different from a processor that operates using software. In other words, the image processing circuit 4 is hardware that does not require software for realizing its functions. Hereinafter, the processing target image is referred to as “target image 110”.

<Configuration of image processing circuit>
FIG. 2 is a diagram showing a configuration of the image processing circuit 4. As shown in FIG. 2, the image processing circuit 4 includes a DMA (Direct Memory Access) controller 10, an input frame buffer 11, a coordinate conversion circuit 12, a storage circuit 13, and a processing circuit 14. The image processing circuit 4 can access the system memory 3 without the CPU 2 by the action of the DMA controller 10. Thereby, data transfer is performed between the image processing circuit 4 and the system memory 3 without going through the CPU 2.

  The DMA controller 10 can read data from the system memory 3. The DMA controller 10 can write data to the system memory 3. The DMA controller 10 has a buffer 100 capable of storing a plurality of pixel data. The buffer 100 is configured by, for example, a FIFO (first in first out) buffer.

  The input frame buffer 11 can store the target image 110 (one target image 110) for one frame. That is, the input frame buffer 11 can store image data for one frame. The DMA controller 10 writes the target image 110 in the system memory 3 in the input frame buffer 11. Specifically, the DMA controller 10 reads a plurality of pixel data of the target image 110 from the system memory 3 and temporarily stores it in the buffer 100. Then, the DMA controller 10 writes a plurality of pixel data in the buffer 100 into the input frame buffer 11. By repeating this processing a plurality of times, the target image 110 for one frame (image data of the target image 110 for one frame) is written to the input frame buffer 11. The DMA controller 10 sequentially reads out a plurality of target images 110 stored in the system memory 3 and sequentially stores them in the input frame buffer 11.

  Also, the DMA controller 10 writes the output image output from the processing circuit 14 into the system memory 3. Specifically, the DMA controller 10 temporarily stores the plurality of pixel data of the output image output from the processing circuit 14 in the buffer 100. Then, the DMA controller 10 writes a plurality of pixel data in the buffer 100 to the system memory 3. By repeating this process a plurality of times, an output image for one frame (image data of an output image for one frame) is written in the system memory 3.

  The coordinate conversion circuit 12 performs image coordinate conversion on the target image 110 stored in the input frame buffer 11. The target image 110 that has undergone coordinate transformation is input to the processing circuit 14. When a new target image 110 is stored in the input frame buffer 11, the coordinate conversion circuit 12 performs image coordinate conversion on the new target image 110 in the input frame buffer 11. The storage circuit 13 stores various parameters used in the coordinate conversion circuit 12. Hereinafter, the target image 110 on which the coordinate conversion has been performed may be referred to as a “post-conversion target image 110”. Moreover, simply speaking "target image 110" means the target image 110 for which coordinate conversion has not been performed yet.

  The processing circuit 14 performs a predetermined process on the post-conversion target image 110 obtained by the coordinate conversion circuit 12. For example, the processing circuit 14 combines a plurality of post-conversion target images 110 obtained by the coordinate conversion circuit 12 and outputs the combined image obtained thereby to the DMA controller 10 as an output image.

<Coordinate transformation circuit>
Here, the outline of the operation of the coordinate conversion circuit 12 is first described, and then the configuration and detailed operation of the coordinate conversion circuit 12 are described.

<Outline of operation of coordinate transformation circuit>
The coordinate conversion circuit 12 performs a plurality of types of coordinate conversion using one target image 110 in the input frame buffer 11. Specifically, as shown in FIG. 3, the coordinate conversion circuit 12 separately performs a plurality of types of coordinate conversion on one target image 110 in the input frame buffer 11 to convert a plurality of images. The processed image 115 is generated. The processed image 115 is the converted target image 110. Hereinafter, a process of generating a plurality of processed images 115 by performing each of a plurality of types of coordinate conversion separately on one target image 110 is referred to as an “individual conversion process”.

  In this example, the coordinate conversion circuit 12 performs coordinate conversion on the target image 110 to perform affine conversion of the image. Specifically, the coordinate conversion circuit 12 performs rotation coordinate conversion on the target image 110 to rotate the image. Thereby, in the coordinate conversion circuit 12, the target image 110 is rotated, and the rotated target image 110 is obtained as the processed image 115 (the converted target image 110). Here, the rotation of the image means rotation of the subject shown in the image, and even if the image is rotated, the outer shape of the image does not change. Hereinafter, the rotated target image 110 may be simply referred to as a “rotated image”.

  The coordinate conversion circuit 12 uses a single target image 110 in the input frame buffer 11 to perform a plurality of types of rotation coordinate conversion for performing a plurality of types of rotation of the image. In this example, the coordinate conversion circuit 12 separately performs each of a plurality of types of coordinate conversion for rotation on one target image 110 in the input frame buffer 11, and rotates a plurality of rotations having different rotation angles. Generate an image. Each time the target image 110 is written in the input frame buffer 11, the coordinate conversion circuit 12 separately performs plural types of rotational coordinate conversion on the target image 110 in the input frame buffer 11. Hereinafter, in the coordinate conversion circuit 12, a plurality of rotation images of a group of rotation angles different from one another, which are generated based on one target image 110, may be referred to as "rotation image group". Also, performing each of a plurality of types of rotation coordinate transformations separately on one target image 110 may be referred to as “rotated image group generation processing”. The rotational image group generation process is a type of individual conversion process.

<Details of coordinate conversion circuit>
FIG. 4 is a diagram showing a configuration of the coordinate conversion circuit 12. As shown in FIG. 4, the coordinate conversion circuit 12 includes a conversion circuit 20 that performs coordinate conversion, and a control circuit 21 that controls the storage circuit 13. The coordinate conversion circuit 12 is hardware that operates without using software, unlike, for example, a CPU or the like.

  As shown in FIG. 4, the storage circuit 13 stores a conversion parameter list 130 used in coordinate conversion. The conversion parameter list 130 includes a plurality of types of conversion parameters 131 necessary for a plurality of types of coordinate conversion. The conversion parameter list 130 is written into the storage circuit 13 by the CPU 2 as will be described later.

  The conversion circuit 20 performs a plurality of types of rotation coordinate conversion on the single target image 110 in the input frame buffer 11 to generate a rotation image group. The conversion parameter list 130 includes a rotation conversion parameter 131 (hereinafter also referred to as “rotation parameter 132”) necessary for the rotation coordinate conversion.

  FIG. 5 is a diagram showing an example of how the conversion parameter list 130 is stored in the storage circuit 13. As shown in FIG. 5, the conversion parameter list 130 includes a plurality of types of rotation parameters 132 necessary for a plurality of types of rotation coordinate conversion for respectively performing a plurality of types of rotation of the image. The conversion parameter list 130 includes, for example, 360 types of rotation parameters 132 for rotating the image from 1 ° to 360 °. In the storage circuit 13, the rotation parameter 132 with a smaller rotation angle is stored in a storage area with a smaller address. Accordingly, when viewing the 360 types of rotation parameters 132 stored in the storage circuit 13 in ascending order of addresses of the storage areas in which they are stored, the 360 types of rotation parameters 132 are arranged in ascending order of rotation angle. It is.

  The conversion circuit 20 performs rotation coordinate conversion on the target image 110 based on the rotation parameter 132 in the storage circuit 13. Thereby, the target image 110 is rotated by a rotation angle corresponding to the rotation parameter 132. The conversion circuit 20 separately performs each of the plurality of types of rotational coordinate conversion on one target image 110 in the input frame buffer 11 using the plurality of types of rotational parameters 132 in the storage circuit 13. Thus, a plurality of rotated images having different rotation angles are generated. In the present example, for example, the target image 110 is rotated clockwise around its center of gravity.

  The control circuit 21 sequentially reads a plurality of types of rotation parameters 132 from the storage circuit 13. The rotation parameter 132 read from the storage circuit 13 is input to the conversion circuit 20. As a result, a plurality of types of rotation parameters 132 in the storage circuit 13 are sequentially input to the conversion circuit 20 by the operation of the control circuit 21. The conversion circuit 20 rotates the target image 110 in the input frame buffer 11 based on the input rotation parameter 132 each time the rotation parameter 132 is input, and outputs the rotation image obtained thereby to the processing circuit 14. Output.

  The control circuit 21 has a register 22. The register 22 identifies a plurality of types of use target rotation parameters 132 used in the conversion circuit 20 among a plurality of types of rotation parameters (in this example, 360 types of rotation parameters 132) constituting the conversion parameter list 130. Specific information 220 is stored. The control circuit 21 sequentially reads out the plurality of types of use target rotation parameters 132 from the storage circuit 13 based on the specific information 220 in the register 22. The specific information 220 is set in the register 22 by the CPU 2 as described later.

  FIG. 6 shows an example of the identification information 220. As shown in FIG. As shown in FIG. 6, the specific information 220 includes, for example, the number of used parameters 221, the use start position 222, and the use interval 223. In this example, the coordinate conversion circuit 12 uses a plurality of types of use target rotation parameters 132 in the order of smaller corresponding rotation angles, and converts one type of rotation coordinate conversion of different rotation angles to one another The image 110 is continuously performed.

  The number of used parameters 221 indicates the number of types of rotation parameters 132 used in the rotation image group generation process. In this example, since one type of rotation parameter is used for the target image 110 and one coordinate conversion is performed, the number of used parameters 221 indicates the number of coordinate conversions for one target image 110. It can be said that In the case where the number of used parameters 221 indicates, for example, "180", the conversion circuit 20 separately performs 180 types of coordinate conversion for rotation on one target image 110, and the rotation angles are different from each other. 180 rotated images are continuously generated.

  The use start position 222 indicates the storage position of the first use target rotation parameter 132 used in the rotated image group generation process, that is, the address of the storage area in which the first use target rotation parameter 132 is stored. . In the example of FIG. 5, when the use start position 222 indicates, for example, “0001h”, in the rotational image group generation process, a rotation parameter 132 for rotating the target image 110 by 2 ° is first used.

  When the rotation coordinate conversion is performed at a certain timing, the use interval 223 is separated from the rotation parameter 132 used in the previous rotation coordinate conversion in a plurality of types of rotation parameters 132 arranged in descending order of rotation angle. It shows whether the rotation parameter 132 is used. For example, when the use interval 223 indicates “2”, when the rotation coordinate conversion is performed at a certain timing, the rotation parameter 132 used in the previous rotation coordinate conversion (for example, the target image 110 is rotated by 2 °). A rotation parameter 132 (for example, a rotation parameter 132 for rotating the target image 110 by 4 °) is used.

  As described above, in the rotated image group generation processing, when one target image 110 in the input frame buffer 11 is rotated by 2 ° from 2 ° to 360 °, the number of used parameters 221 becomes “180” and the use starts. The position 222 is “0001h” and the use interval 223 is “2”. In this case, under control of the storage circuit 13 by the control circuit 21, from the storage circuit 13, a rotation parameter 132 with a rotation angle of 2 °, a rotation parameter 132 with a rotation angle of 4 °, and a rotation parameter 132 with a rotation angle of 6 ° ... The rotation parameters 132 with a rotation angle of 360 ° are sequentially read out and sequentially input to the conversion circuit 20. As a result, as shown in FIG. 7, the conversion circuit 20 sequentially generates 180 rotation images 120 whose rotation angles differ by 2 °.

  The conversion circuit 20 outputs the rotated images 120 one by one to the processing circuit 14. Then, the conversion circuit 20 outputs a plurality of pixels constituting the rotation image 120 to the processing circuit 14 one by one. That is, the conversion circuit 20 outputs a plurality of pixel data constituting the image data of the rotated image 120 to the processing circuit 14 one pixel at a time. Each time the conversion circuit 20 generates pixel data of the rotated image 120, the conversion circuit 20 outputs the generated pixel data.

  As described above, in the coordinate conversion circuit 12, since the plural types of rotation targets 132 to be used are specified by the specific information 220, it is determined by the specific information 220 what kind of coordinate conversion is to be performed by the coordinate conversion circuit 12. can do.

  For example, in the example of FIG. 5, when “360” is set to the number of used parameters 221, “0000h” is set to the use start position 222, and “1” is set to the use interval 223, 1 in the input frame buffer 11 The target image 110 is rotated by 1 ° from 1 ° to 360 ° to generate 360 rotated images 120.

  In addition, the number of used parameters 221 is set to "36", the use start position 222 is set to the address of the storage area where the rotation parameter 132 with a rotation angle of 10 ° is stored, and the use interval 223 is set to "10" Then, one target image 110 in the input frame buffer 11 is rotated by 10 ° from 10 ° to 360 °, and 36 rotated images 120 are generated.

  In addition, the number of used parameters 221 is set to "18", the use start position 222 is set to the address of the storage area where the rotation parameter 132 having a rotation angle of 270 ° is stored, and the use interval 223 is set to "5" Then, one target image 110 in the input frame buffer 11 is rotated by 5 ° from 270 ° to 360 ° to generate 18 rotated images 120.

  In addition, since the plurality of types of use target rotation parameters 132 are specified by the specific information 220, the specific information 220 used by the coordinate conversion circuit 12 differs among the plurality of image processing apparatuses 1, thereby Different coordinate transformations can be performed between the image processing apparatuses 1.

  Further, a plurality of types of specific information 220 may be used in one image processing apparatus 1. In this case, the coordinate conversion circuit 12 uses the first specific information 220 when performing coordinate conversion on a certain target image 110, and performs coordinate conversion on another target image 110. The second identification information 220 different from the one identification information 220 is used. Accordingly, the coordinate conversion circuit 12 can perform different coordinate conversions on the plurality of target images 110.

<Example of rotation parameters>
In coordinate conversion of the target image 110, as a pixel at a certain coordinate Z in the converted target image 110, a pixel at a coordinate CZ corresponding to the certain coordinate Z in the target image 110 is adopted. In other words, in coordinate conversion of the target image 110, pixel data (pixel value) at a certain coordinate Z in the target image 110 after conversion, pixel data at a coordinate CZ corresponding to the certain coordinate Z in the target image 110 Pixel value) is adopted. In the target image 110, when there is no pixel at a coordinate CZ corresponding to a certain coordinate Z in the converted target image 110, pixel data at the corresponding coordinate CZ is a plurality of pixel data around the corresponding coordinate CZ. It is obtained by interpolation using pixel data of a plurality of pixels present at coordinates PCZ. For example, bilinear interpolation or bicubic interpolation is used for this pixel interpolation.

  FIG. 8 is a diagram for explaining an example of the rotation parameter 132. In this example, the rotation parameter 132 includes four coordinates A0, B0, and B4 in the target image 110 corresponding to the coordinates A1, B1, C1, and D1 of the four corner pixels in the rotated image (converted corresponding image). C0 and D0 are included. In other words, the rotation parameter 132 includes the coordinates A0, B0, C0, and D0 of the original pixel in the target image 110 for the four corner pixels in the rotated image.

  When the rotation parameter 132 is input from the storage circuit 13, the conversion circuit 20 includes a plurality of images constituting the converted target image 110 based on the four coordinates A 0, B 0, C 0, D 0 included in the input rotation parameter 132. A plurality of coordinates in the target image 110 respectively corresponding to the coordinates of the pixel of. Then, the conversion circuit 20 determines pixel data at each of the plurality of coordinates obtained with respect to the target image 110 based on the pixel data of the plurality of pixels constituting the target image 110. At this time, pixel interpolation is used as needed. Thereafter, the conversion circuit 20 employs pixel data at corresponding coordinates in the target image 110 as pixel data at each coordinate in the converted target image 110. Thereby, in the conversion circuit 20, the target image 110 is rotated based on the input rotation parameter 132, and a rotation image is generated. That is, the conversion circuit 20 generates image data of a rotated image. The image data of the rotated image generated by the conversion circuit 20 is input to the processing circuit 14.

  Note that image distortion such as barrel distortion included in the target image 110 can be corrected by performing predetermined coordinate transformation on the target image 110. Therefore, the conversion parameter 131 corresponding to both the rotation coordinate conversion and the distortion correction coordinate conversion is stored in the storage circuit 13 as the conversion parameter 131, and the coordinate conversion circuit 12 is based on the conversion parameter 131 in the storage circuit 13. Coordinate conversion for rotation and distortion correction may be simultaneously performed on the target image 110. Patent Documents 2 and 3 described above disclose barrel distortion correction techniques.

<About processing circuit>
Here, the outline of the operation of the processing circuit 14 is first described, and then the configuration and detailed operation of the processing circuit 14 are described.

<Outline of processing circuit operation>
As shown in FIG. 9, the processing circuit 14 combines a plurality of rotated images 120 constituting the rotated image group 121 generated by the coordinate conversion circuit 12 and uses a synthesized image 140 obtained thereby as an output image. Output to the DMA controller 10. The processing circuit 14 has a frame memory capable of storing an image for one frame, and generates a composite image 140 using the frame memory.

  FIG. 10 is a diagram for explaining the outline of the operation of the processing circuit 14. The processing circuit 14 generates a composite image 140 by sequentially integrating the N rotated images 120 constituting the rotated image group 121.

  As illustrated in FIG. 10, the processing circuit 14 includes a rotated image 120-1 that is first generated by the conversion circuit 20 in the rotated image group 121 (a rotated image having a rotation angle of 2 ° in the example of FIG. 7 described above). ) Is stored in the frame memory.

  The processing circuit 14 uses the rotation image 120-2 (the rotation image having a rotation angle of 4 ° in the example of FIG. 7) generated second by the conversion circuit 20 in the rotation image group, as the first in the frame memory. Is added to the rotated image 120-1 of 1 to generate a one-time integrated image 150-1. The processing circuit stores the once integrated image 150-1 in the frame memory.

  The processing circuit 14 uses the rotation image 120-3 (the rotation image having a rotation angle of 6 ° in the example of FIG. 7 described above) generated third by the conversion circuit 20 in the rotation image group as 1 in the frame memory. The accumulated image 150-1 is added twice to generate the accumulated image 150-2 twice. The processing circuit 14 stores the twice-integrated image 150-2 in the frame memory.

  Thereafter, the processing circuit 14 operates in the same manner, and the processing circuit 14 detects that the rotation image 120-N generated last by the conversion circuit 20 in the rotation image group (the rotation angle is 360 ° in the example of FIG. 7 above). Is added to the (N-2) -times accumulated image 150- (N-2) in the frame memory to generate (N-1) -times accumulated images 150- (N-1). This (N-1) -time integrated image 150- (N-1) becomes the composite image 140. The processing circuit 14 stores the composite image 140 in a frame memory. The composite image 140 in the frame memory is output to the DMA controller 10. Thereafter, when it is not necessary to particularly distinguish the one-time accumulated image 150-1, the two-time accumulated image 150-2,. It is called "integrated image 150".

<Details of processing circuit>
FIG. 11 is a diagram showing the configuration of the processing circuit 14. As shown in FIG. 11, the processing circuit 14 includes a frame memory 40, a storage control unit 41, an addition processing unit 42, a write buffer 43, a read buffer 44, and an output control unit 45. The processing circuit 14 is hardware that operates without using software, unlike, for example, a CPU or the like. Therefore, the storage control unit 41, the addition processing unit 42, and the output control unit 45 can be referred to as the storage control circuit 41, the addition processing circuit 42, and the output control circuit 45, respectively. The processing circuit 14 is a type of data processing apparatus that processes image data.

  The frame memory 40 is a kind of storage circuit, and is composed of, for example, an SRAM. The frame memory 40 can store, for example, an image for one frame (one image). That is, the frame memory 40 can store one frame of image data. The frame memory 40 may be capable of storing images for a plurality of frames.

  The output control unit 45 receives the data read from the frame memory 40, and controls the output of the data to the DMA controller 10. The output control unit 45 has, as an operation mode, an output mode for outputting input data to the DMA controller 10, and a non-output mode for not outputting input data to the DMA controller 10. The output control unit 45 basically operates in the non-output mode, and operates in the output mode only when the output target data is read from the frame memory 40. As a result, the output target data read from the frame memory 40 is output from the output control unit 45 to the DMA controller 10.

  The storage control unit 41 controls the frame memory 40, the read buffer 44, the write buffer 43, and the output control unit 45. Data is written to the frame memory 40 or data is read from the frame memory 40 by the storage control unit 41 controlling the frame memory 40. The output control unit 45 controls output of the data read from the frame memory 40 to the DMA controller 10 according to the control by the storage control unit 41. The storage control unit 41 will be described in detail later.

  The addition processing unit 42 adds the rotated image generated by the conversion circuit 20 to the addition target image read from the frame memory 40 under the control of the storage control unit 41. That is, the addition processing unit 42 adds the image data of the rotated image generated by the conversion circuit 20 to the image data of the image to be added which is read from the frame memory 40. As shown in FIG. 10 described above, the image to be added for the second rotation image 120-2 in the rotation image group is the first rotation image 120-1, and the m-th (m is a variable, 3 ≦ 0). The image to be added with respect to the rotated image 120-m with m ≦ N) is the (m−2) times accumulated image 150− (m−2).

  Here, one and the other of the two image data to be added will be referred to as "first image data" and "second image data", respectively. Then, each of a plurality of pixel data constituting the first image data is referred to as “first pixel data”, and each of a plurality of pixel data constituting the second image data is referred to as “second pixel data”. The addition processing unit 42 sets the first pixel data and the second image data at the same position (coordinates) as the first pixel data, for each of the plurality of first pixel data forming the first image data. By adding the two pixel data, the first image data and the second image data are added.

  In this example, the coordinate conversion circuit 12 outputs a plurality of pixel data constituting the image data of the rotated image 120 pixel by pixel as described above. When the pixel data of the first rotated image 120-1 is input as the input pixel data 125, the addition processing unit 42 writes the input pixel data 125 to the light buffer 43 as it is without performing the addition process. The light buffer 43 can store, for example, pixel data for four pixels. When pixel data for four pixels is stored in the light buffer 43, pixel data for four pixels in the light buffer 43 is written at one time to the frame memory 40 by control of the frame memory 40 by the storage control unit 41. Thus, pixel data is written to the frame memory 40 in units of four pixels. That is, pixel data for four pixels in the write buffer 43 is written by one write access to the frame memory 40. In the frame memory 40, pixel data for four pixels in the write buffer 43 is written to the storage area of the same address. When the data in the write buffer 43 is written to the frame memory 40, the written data is erased from the write buffer 43.

  By control of the frame memory 40 by the storage control unit 41, pixel data is read from the frame memory 40 in units of four pixels. That is, pixel data for four pixels stored in the storage area of the same address is read from the frame memory 40 by one read access to the frame memory 40. The pixel data for four pixels read out at one time from the frame memory 40 is written to the read buffer 44.

  When the input pixel data 125 is the l-th (l is a variable, 2 ≦ l ≦ N) pixel data of the rotated image 120-1, the addition processing unit 42 receives the pixel data 165 to be added from the read buffer 44. read out. Then, the addition processing unit 42 adds the input pixel data 125 to the pixel data 165 read from the read buffer 44. Thereby, the addition processing unit 42 obtains the pixel data 155 of the (l-1) -time accumulated image 150- (l-1). The addition processing unit 42 stores the pixel data 155 of the (l−1) -time accumulated image 150-(l−1) in the light buffer 43. Pixel data in the write buffer 43 is stored in the frame memory 40. The addition processing unit 42 performs the same processing each time pixel data is input from the coordinate conversion circuit 12. As a result, the frame memory 40 stores the image data of the composite image 140 ((N−1) times accumulated image 150-(N−1)). When data is read from the read buffer 44, the read data is erased from the read buffer 44.

  As described above, the addition processing unit 42 adds the pixel data of the l-th rotated image 120 to the pixel data 165 to be added in the read buffer 44, and adds the (l-1) -time accumulated image 150-( l-1) pixel data 155 is generated. For example, when the pixel data of the second rotated image 120-2 is the input pixel data 125, the pixel data 165 to be added for the input pixel data 125 is the input pixel data in the first rotated image 120-1. It becomes pixel data at the same position as 125. When the pixel data of the third rotated image 120-3 is the input pixel data 125, the pixel data 165 to be added to the input pixel data 125 is the input pixel data in the one-time integration image 150-1. It becomes pixel data at the same position as 125. The pixel data 165 to be added with respect to the input pixel data 125 is stored in the read buffer 44 when the input pixel data 125 is input to the addition processing unit 42 under the control of the frame memory 40 by the storage control unit 41.

  In this example, the frame memory 40 is, for example, a 1-port memory. Therefore, when data is read from frame memory 40, data can not be written to frame memory 40.

  In the processing circuit 14, read modify write is performed on the frame memory 40. Specifically, after the pixel data stored in the storage area at a certain address of the frame memory 40 is read from the frame memory 40, the addition processing unit 42 executes an addition process. Then, the pixel data subjected to the addition processing is written back to the storage area of the same address as the given address. In other words, when the pixel data 155 of the integrated image 150 in the write buffer 43 is written to the frame memory 40, the pixel data 165 to be added used for generating the pixel data 155 in the addition processing unit 42 is The pixel data 155 is stored in the read storage area.

  In this example, pixel data for four pixels are read from the frame memory 40 at one time and stored in the read buffer 44. In addition, pixel data for four pixels in the write buffer 43 is written to the frame memory 40 at one time. In the processing circuit 14, pixel data for four pixels stored in a storage area at a certain address of the frame memory 40 is read from the frame memory 40 and temporarily stored in the read buffer 44. The addition process is executed at. Then, the pixel data for the four pixels subjected to the addition process is temporarily stored in the write buffer 43 and then written back to the storage area having the same address as the certain address.

  As described above, in this example, the frame memory 40 is a one-port memory, and the read modify write is performed on the frame memory 40. Therefore, the circuit scale of the frame memory 40 can be reduced.

  The image data of the composite image 140 in the frame memory 40 is output from the output control unit 45 to the DMA controller 10. When the pixel data (output target data) 145 of the composite image 140 is read from the frame memory 40 by control of the frame memory 40 by the storage control unit 41, the output control unit 45 performs DMA control on the read pixel data 145 Output to 10. Since pixel data for four pixels are read out from the frame memory 40 at one time, the output control unit 45 uses the DMA controller 10 for the pixel data 145 for four pixels of the composite image 140 read from the frame memory 40. Output to The pixel data 145 of the composite image 140 read from the frame memory 40 is not stored in the read buffer 44 due to control of the read buffer 44 by the storage control unit 41.

  The DMA controller 10 temporarily stores the pixel data input from the processing circuit 14 in the buffer 100. The buffer 100 can store pixel data for a plurality of pixels. When the buffer 100 is full, the DMA controller 10 transfers pixel data for a plurality of pixels in the buffer 100 to the system memory 3. As a result, the system memory 3 stores the composite image 140 (image data of the composite image 140) generated by the image processing circuit 4. In the DMA controller 10, when the data in the buffer 100 is transferred to the system memory 3, the transferred data is erased from the buffer 100. Note that the DMA controller 10 may transfer the data in the buffer 100 to the system memory 3 when the amount of data in the buffer 100 exceeds a predetermined value.

<Details of storage controller>
The storage control unit 41 executes a first reading process (also referred to as “process target reading process”) for reading from the frame memory 40 the pixel data 165 to be added that is processed by the addition processing unit 42. Further, the storage control unit 41 performs a writing process of writing the pixel data 155 of the integrated image 150 generated by the addition processing unit 42 into the frame memory 40. It can be said that the pixel data 155 of the integrated image 150 is the pixel data 165 processed by the addition processing unit 42. Then, the storage control unit 41 executes a second reading process (also referred to as “output target reading process”) for reading from the frame memory 40 the pixel data 145 of the composite image 140, which is output target data.

  The storage control unit 41 includes a first read request output unit 411, a write request output unit 412, a second read request output unit 413, and an arbitration unit 410. The first read request output unit 411 outputs a first read request (also referred to as a “processing target read request”) that is an execution request for the first read process. The write request output unit 412 outputs a write request that is an execution request for the write request. The second read request output unit 413 outputs a second read request (also referred to as “output target read request”) that is a request for executing the second read process.

  The first read request output unit 411 outputs a first read request signal indicating a first read request. The first read request signal includes the address of the storage area in the frame memory 40 where the pixel data 165 to be read is stored.

  The write request output unit 412 outputs a write request signal indicating a write request. The write request signal includes an address of a storage area in the frame memory 40 in which the pixel data 155 to be written is written.

  The second read request output unit 411 outputs a second read request signal indicating a second read request. The second read request signal includes the address of the storage area in the frame memory 40 in which the pixel data 145 to be read is stored.

  The first read request output unit 411, the write request output unit 412, and the second read request output unit 413 output the first read request, the write request, and the second read request based on mutually independent criteria.

  For example, when the read buffer 44 is empty, the first read request output unit 411 outputs a first read request (first read request signal). For example, when the write buffer 43 is full, the write request output unit 412 outputs a write request (write request signal).

  The second read request output unit 413 outputs a second read request (second read request signal) based on, for example, the availability of the buffer 100 of the DMA controller 10. For example, the second read request output unit 413 outputs a second read request when the buffer 100 is empty. Alternatively, the second read request output unit 413 outputs the second read request when the free space of the buffer 100 is equal to or more than a predetermined amount (for example, when the amount of pixel data of four pixels is equal to or more). In other words, the second read request output unit 413 outputs the second read request when the amount of data that can be stored in the free storage area in the buffer 100 is equal to or more than the predetermined amount.

  However, when the output target data, that is, the pixel data 145 of the composite image 140 is not stored in the frame memory 40, the storage control unit 41 can not read the output target data from the frame memory 2. Therefore, when the pixel data 145 of the composite image 140 is stored for four or more pixels in the frame memory 40 and the buffer 100 is empty, the second read request output unit 413 outputs the second read request. Output. Alternatively, if the second read request output unit 413 stores four or more pixel data 145 of the composite image 140 in the frame memory 40 and the free space of the buffer 100 is a predetermined amount or more, Output a second read request.

  As described above, the first read request output unit 411, the write request output unit 412, and the second read request output unit 413 respectively output the first read request, the write request, and the second read request based on the mutually independent criteria. Thus, at least two requests of the first read request, the write request and the second read request may conflict. That is, requests may be simultaneously output from at least two of the first read request output unit 411, the write request output unit 412, and the second read request output unit 413.

  Therefore, the storage control unit 41 is provided with an arbitration unit 410 that arbitrates at least two conflicting requests when at least two requests of the first read request, the write request, and the second read request conflict.

  When the first read request, the write request, and the second read request do not conflict, the arbitration unit 410 performs processing corresponding to the input request on the frame memory 2. On the other hand, when at least two requests, the first read request, the write request, and the second read request, conflict, the arbitration unit 410 arbitrates between at least two conflicting requests, thereby adjusting one of the two requests. Choose Then, the arbitrating unit 410 performs processing corresponding to the selected request on the frame memory 2.

  In this example, processing priorities are assigned to the first read request, the write request, and the second read request. For example, the highest priority is divided for the first read request, the second highest priority is assigned to the write request, and the lowest priority is assigned to the second read request. Then, the arbitrating unit 410 arbitrates the at least two requests based on the priorities of at least two competing requests. In the arbitration unit 410, a request having the highest priority is selected from at least two competing requests, and processing according to the selected request is performed. Therefore, for example, when the three requests of the first read request, the write request, and the second read request conflict, the first read request is selected. In addition, when the two requests of the write request and the second read request conflict, the write request is selected.

  When the arbitration unit 410 performs processing according to the first read request, the arbitration unit 410 outputs the address included in the first read request signal input from the first read request output unit 411 and the read signal to the frame memory 40. Do. As a result, the pixel data for four pixels is read from the storage area of the address included in the first read request signal in the frame memory 40 and written to the read buffer 44. When the arbitration unit 410 performs processing in response to the first read request, the output control unit 45 operates in the non-output mode, so that data read from the frame memory 40 is output from the output control unit 45. I will not.

  The arbitration unit 410 reads the pixel data for four pixels from the write buffer 43 and inputs the pixel data to the frame memory 40 and performs the write request signal input from the write request output unit 412 when performing processing according to the write request. And the write signal are output to the frame memory 40. As a result, the pixel data for four pixels output from the write buffer 43 is written to the storage area of the address included in the write request signal in the frame memory 40.

  When the arbitration unit 410 performs processing according to the second read request, the arbitration unit 410 outputs the address included in the second read request signal input from the second read request output unit 413 and the read signal to the frame memory 40. At the same time, the operation mode of the output control unit 45 is changed from the non-output mode to the output mode. Thus, pixel data for four pixels is read out from the storage area of the address included in the second read request signal in the frame memory 40, and the read pixel data for four pixels is output from the output control unit 45 by DMA. It is output to the controller 10.

  In the above example, the processing time in the coordinate conversion circuit 12 is relatively long, and the pixel data of the rotated image 120 cannot be obtained immediately in the coordinate conversion circuit 12, and as a result, the pixel data 145 (output of the composite image 140) Assuming a situation in which it is difficult to obtain target data), the priority of the first read request and the write request necessary for generating the pixel data 145 of the composite image 140 is set to improve the processing speed of the entire image processing apparatus 1. , Higher than the priority of the second read request. However, the priority assignment method for the first read request, the write request, and the second read request is not limited to this.

  For example, when the utilization of the bus 5 is always high and it is difficult to transfer data from the DMA controller 10 to the system memory 3 and as a result, it is difficult to make space in the buffer 100 of the DMA controller 10. The output target data (pixel data 145 of the composite image 140) is likely to be accumulated. In this case, if the priority of the first read request and the write request is higher than the priority of the second read request, the output target data in the frame memory 40 may not be output from the processing circuit 14 indefinitely. Therefore, in such a case, the priority of the second read request necessary for reading the output target data from the frame memory 40 is made higher than the priority of the first read request and the write request. As a result, the processing speed of the entire image processing apparatus 1 can be improved.

  As described above, in the storage control unit 41, when at least two of the first read request, the write request, and the second read request are in contention, the at least two requests are arbitrated. Therefore, the first read request output unit 411, the write request output unit 412 and the second read request output unit 413 can operate independently of each other. Therefore, it is possible to suppress the processing at a specific output unit among the first read request output unit 411, the write request output unit 412, and the second read request output unit 413 from becoming a bottleneck. As a result, the processing speed of the entire processing circuit 14 can be improved.

  Note that the priority of the process may be changeable. For example, the image processing apparatus 1 may be configured to allow the user to change the priority of processing. Further, the processing circuit 14 may change the priority of processing in accordance with the operation state of the image processing apparatus 1. For example, a circuit for obtaining the usage rate of the bus 5 may be provided in the image processing apparatus 1, and the processing circuit 14 may change the priority of processing based on the usage rate of the bus 5 obtained by the circuit. Specifically, the processing circuit 14 makes the priority of the second read request higher than the priority of the first read request and the write request when the usage rate of the bus 5 is larger than the predetermined value. When the usage rate of 5 is less than or equal to the predetermined value, the priority of the first read request and the write request is made higher than the priority of the second read request. Thus, the processing speed of the entire processing circuit 14 can be further improved by dynamically changing the processing priority.

<About a series of operations from reading the target image 110 to generating a composite image>
FIG. 12 is a flowchart showing the operation of the image processing apparatus 1. When the power of the image processing apparatus 1 is turned on, the CPU 2 performs initial setting of the image processing apparatus 1. In initial setting of the image processing apparatus 1, initial setting for the image processing circuit 4 is performed (step s0). The image processing circuit 4 is provided with various setting registers (not shown). In the initial setting, the CPU 2 sets data in various setting registers in the image processing circuit 4. For example, the CPU 2 sets the address of the storage area where the target image 110 is stored in the system memory 3 in the first setting register, or stores in the system memory 3 the write destination of the composite image 140 obtained by the image processing circuit 4 The area address is set in the second setting register, and the image size of the target image 110 processed by the image processing circuit 4 is set in the third setting register. Further, in the initial setting of the image processing circuit 4, the CPU 2 sets the conversion parameter list 130 in the storage circuit 13 of the image processing circuit 4. As a result, the conversion parameter list 130 is stored in the storage circuit 13. For example, the CPU 2 writes the conversion parameter list 130 in the storage circuit 13 by executing a program in which the conversion parameter list 130 is described in the system memory 3. Note that the CPU 2 may create the conversion parameter list 130 and write the created conversion parameter list 130 into the storage circuit 13 at the time of initial setting.

  After the initial setting is completed, the CPU 2 sets the image processing circuit 4 regarding the first target image 110 (step s1). Specifically, the CPU 2 sets specific information 220 used in processing for the first target image 110 in the register 22 included in the control circuit 21 of the image processing circuit 4.

  In the image processing circuit 4, when the specific information 220 is set in the register 22, the DMA controller 10 counts the first target image 110 from the system memory 3 based on the address set in the first setting register described above. Is written to the input frame buffer 11 (step s1-1). The coordinate conversion circuit 12 reads the target rotation parameter 132 to be used first from the storage circuit 13 based on the specific information 220 in the register 22, and based on the read target rotation parameter 132, the inside of the input frame buffer 11. The first rotation coordinate transformation is performed on the target image 110 (step s1-1). Thereby, the first rotated image 120-1 is obtained. The first rotated image 120-1 is stored in the frame memory 40 of the processing circuit 14.

  When the coordinate conversion circuit 12 generates the first rotation image 120-1, the coordinate conversion circuit 12 reads out the use target rotation parameter 132 to be used second from the storage circuit 13 based on the specific information 220, and uses it as the read use target rotation parameter 132. The second rotation coordinate conversion is performed on the target image 110 in the input frame buffer 11 (step s1-2). From this, the second rotated image 120-2 is obtained. On the other hand, the processing circuit 14 adds the second rotated image 120-2 to the first rotated image 120-1 to generate a one-time integrated image 150-1 (step s1-2).

  Here, the coordinate conversion circuit 12 outputs the generated pixel data each time the pixel data of the rotated image 120 is generated. When pixel data is input from the coordinate conversion circuit 12, the processing circuit 14 adds the input pixel data to the pixel data to be added that has been read from the read buffer 44. Therefore, the image processing circuit 4 does not generate the integrated image 150-1 once after the rotation image 120-2 is generated, but generates the rotation image 120-2 in the coordinate conversion circuit 12, The process of generating the one-time accumulated image 150-1 in the processing circuit 14 is performed in parallel. That is, in the image processing circuit 4, the process of generating the l-th rotated image 120-1 in the coordinate conversion circuit 12 and the process of generating the (l−1) -time integrated image 150-(l−1) in the processing circuit 14. And run in parallel.

  When the coordinate conversion circuit 12 generates the second rotation image 120-2, it reads the use target rotation parameter 132 to be used third from the storage circuit 13 based on the specific information 220 and reads the use target rotation parameter 132. The third rotation coordinate conversion is performed on the target image 110 in the input frame buffer 11 based on the above. Thus, the third rotated image 120-3 is obtained. On the other hand, the processing circuit 14 adds the third rotated image 120-3 to the once accumulated image 150-1 to generate a twice accumulated image 150-2.

  Thereafter, the image processing circuit 4 operates in the same manner, and the coordinate conversion circuit 12 generates the Nth rotated image 120-N (step s1-N). On the other hand, the processing circuit 14 adds the N-th rotated image 120-3 to the (N-2) -time accumulated image 150- (N-2) to obtain the (N-1) -time accumulated image 150-. 2, that is, a composite image 140 is generated (step s1-N). Then, the processing circuit 14 outputs the composite image 140 to the DMA controller 10 (step s1-N). The DMA controller 10 writes the composite image 140 input from the processing circuit 14 in the storage area of the address set in the above-mentioned second setting register in the system memory 3 (step s1-N). As described above, since the first read request output unit 411, the write request output unit 412, and the second read request output unit 413 operate independently of one another, the addition processing in the addition processing unit 42, the output control unit 45 It is executed in parallel with the output processing in. Therefore, the processing circuit 14 does not output pixel data of the composite image 140 after all pixel data making up the composite image 140 have been generated, but generates pixel data of the composite image 140 and the composite image The process of outputting 140 pixel data is performed in parallel.

  When the writing of the composite image 140 into the system memory 3 is completed, the image processing circuit 4 notifies the CPU 2 of the completion of the processing on the first target image 110 (step s1-e). . The completion notification from the image processing circuit 4 to the CPU 2 uses, for example, the interrupt function of the CPU 2. When the CPU 2 receives the completion notification, the CPU 2 sets the second target image 110 in the image processing circuit 4 (step s2). Specifically, the CPU 2 sets the specific information 220 used in the process for the second target image 110 in the register 22 of the control circuit 21 of the image processing circuit 4. The image processing circuit 4 operates in the same manner, generates a composite image 140 based on the second target image 110, and writes the composite image 140 in the system memory 3. Thereafter, the image processing apparatus 1 performs the same process on the third and subsequent target images 110.

  The specific information 220 used in the process for the p-th (p is a variable, p 、 2) target image 110 is the specific information 220 used in the process for the (p−1) -th target image 110. If the CPU 2 receives the completion notification of the (p-1) th target image 110, the CPU 2 sets the register 22 with the specific information 220 used in the processing for the pth target image 110. Instead, the image processing circuit 4 may be instructed to start processing on the pth target image 110.

  As described above, the coordinate conversion circuit 12 continuously performs a plurality of types of coordinate conversion using one target image 110 based on a plurality of types of conversion parameters 131. That is, the coordinate conversion circuit 12 performs the plurality of types of coordinate conversion using one target image 110 without exchanging data with the CPU 2 based on the plurality of types of conversion parameters 131. Therefore, each time the coordinate conversion circuit 12 performs coordinate conversion, the processing time for a plurality of types of coordinate conversion can be reduced as compared with the case where the conversion parameter 131 used in the coordinate conversion is received from the CPU 2.

  In this example, the coordinate conversion circuit 12 continuously generates a plurality of rotated images 120 having different rotation angles based on a single target image 110 based on a plurality of types of rotation parameters 132. Therefore, each time the coordinate conversion circuit 12 generates the rotation image 120, it is necessary to generate a plurality of rotation images 120 as compared to the case where the rotation parameter 132 used in the generation of the rotation image 120 is received from the CPU 2. Processing time can be reduced.

  Further, the processing load of CPU 2 is performed by performing coordinate conversion of a plurality of types using a single target image 110 without exchanging data with CPU 2 based on a plurality of types of conversion parameters 131. Can be reduced. Further, when the CPU 2 and the image processing circuit 4 are connected by the bus 5 as in this example, the bandwidth of the bus 5 can be reduced.

  Further, in this example, since the CPU 2 sets the conversion parameter list 130 in the storage circuit 13 of the image processing circuit 4 at the time of initial setting of the image processing apparatus 1, the coordinate conversion circuit 12 performs the initial setting. The conversion parameter 131 to be used can be read from the storage circuit 13. Therefore, the coordinate conversion circuit 12 does not need to receive the conversion parameter 131 to be used from the CPU 2 every time the conversion parameter 131 is used. Thus, the processing time in the coordinate conversion circuit 12 is reduced.

  Note that the conversion parameter list 130 of the image processing apparatus 1 may include only the conversion parameters 131 used by the image processing apparatus 1. For example, when the coordinate conversion circuit 12 rotates the target image 110 by 2 ° from 2 ° to 360 ° for every target image 110, the rotation angle is 360 ° from the rotation parameter 132 whose rotation angle is 1 °. Among the rotation parameters 132 of the above, only the rotation parameter 132 having an even rotation angle may be included in the conversion parameter list 130.

  Further, a conversion parameter list 130 common to a plurality of image processing apparatuses 1 that perform coordinate conversion different from each other on the target image 110 may be stored in the plurality of image processing apparatuses 1. For example, the first image processing device 1 performs a process of rotating the target image 110 by 1 ° from 1 ° to 90 °, and the second image processing device 1 performs the processing of the target image 110 by 1 ° from 91 ° to 180 °. The third image processing apparatus 1 rotates the object image 110 by 1 ° from 181 ° to 270 °, and the fourth image processing device 1 rotates the object image 110 from 271 ° to 360 °. Let us consider a case where a process of rotating by 1 ° is performed. In this case, the conversion parameter list 130 common to the first to fourth image processing apparatuses 1 includes 360 kinds of rotation parameters 132 for rotating the image from 1 ° to 360 ° as in the above example. Include. As a result, it is not necessary to prepare the conversion parameter list 130 for each image processing apparatus 1, so that the erroneous conversion parameter list 130 can be prevented from being stored in the image processing apparatus 1.

  In the above example, the processing circuit 14 outputs the composite image 140 to the DMA controller 10. However, even if the converted target image 110 obtained by the coordinate conversion circuit 12, ie, the rotation image 120, is output. good. In this case, the processing circuit 14 does not require the addition processing unit 42, the read buffer 44, and the first read request output unit 411. Then, the pixel data of the rotated image 120 output from the coordinate conversion circuit 12 is temporarily stored in the write buffer 43, and the pixel data of the rotated image 120 in the write buffer 43 is written into the frame memory 40. The pixel data of the rotated image 120 in the frame memory 40 is output from the output control unit 45 to the DMA controller 10, whereby the rotated image 120 is input to the DMA controller 10.

  Also, the conversion parameter list 130 includes, in place of or in addition to the rotation parameter 132, conversion parameters 131 other than the rotation parameter 132, that is, conversion parameters 131 necessary for coordinate conversion other than rotation coordinate conversion. It may be.

  For example, the conversion parameter list 130 may include a conversion parameter for enlargement (hereinafter, referred to as “enlargement parameter”) necessary in enlargement coordinate conversion for performing image enlargement. In this case, the conversion parameter list 130 may include a plurality of types of enlargement parameters necessary for the plurality of types of enlargement coordinate conversions having different enlargement ratios. The coordinate conversion circuit 12 can enlarge the target image 110 at an enlargement rate corresponding to the enlargement parameter by performing enlargement coordinate transformation on the target image 110 based on the enlargement parameter. For example, when the coordinate conversion circuit 12 performs the coordinate conversion for enlargement on the target image 110 based on the enlargement parameter of the enlargement ratio of 1.1, the target image 110 enlarged to 1.1 times is obtained. Here, the enlargement of the image means enlargement of the subject shown in the image, and not an increase in the number of pixels in the vertical and horizontal directions of the image. When the conversion parameter list 130 includes the rotation parameter and the enlargement parameter, the coordinate conversion circuit 12 generates a rotated image obtained by rotating the object image 110 and an enlarged image obtained by enlarging the object image 110. And can be generated. Similar to the rotation parameter, the enlargement parameter includes, for example, the coordinates of the four pixels of the mapping source in the target image 110 with respect to the four corner pixels in the enlarged image.

  In addition, the conversion parameter list 130 may include a conversion parameter for reduction (hereinafter referred to as “reduction parameter”) necessary in coordinate conversion for reduction for image reduction. In this case, the conversion parameter list 130 may include a plurality of types of reduction parameters necessary for each of a plurality of types of reduction coordinate conversions having different reduction ratios. The coordinate conversion circuit 12 can reduce the target image 110 at a reduction rate according to the reduction parameter by performing a reduction coordinate conversion on the target image 110 based on the reduction parameter. For example, when the coordinate conversion circuit 12 performs coordinate conversion for reduction on the target image 110 based on a reduction parameter of reduction ratio 0.9, a target image 110 reduced to 0.9 times is obtained. Here, the reduction of the image is the reduction of the subject appearing in the image, not the reduction of the number of vertical and horizontal pixels of the image. When the conversion parameter list 130 includes a rotation parameter and a reduction parameter, the coordinate conversion circuit 12 generates a rotated image obtained by rotating the object image 110 and a reduced image obtained by reducing the object image 110. And can be generated. Similar to the rotation parameter, the reduction parameter includes, for example, the coordinates of the four pixels that are the mapping source in the target image 110 for the four corner pixels in the reduced image.

  In addition, the conversion parameter list 130 may include a conversion parameter for parallel movement (hereinafter referred to as “translation parameter”) necessary for parallel coordinate conversion for parallel movement of an image. In this case, the conversion parameter list 130 may include a plurality of types of parallel movement parameters necessary for the plurality of types of parallel movement coordinate conversion in which the movement directions are different from each other. Further, the conversion parameter list 130 may include a plurality of types of parallel movement parameters required for a plurality of types of parallel movement coordinate conversion with different amounts of movement. The coordinate conversion circuit 12 performs coordinate conversion for parallel movement on the target image 110 based on the parallel movement parameter, thereby moving the target image 110 in the movement direction according to the parallel movement parameter according to the parallel movement parameter. The parallel movement can be performed by the movement amount. For example, when the coordinate conversion circuit 12 performs coordinate conversion for translation on the target image 110 based on the translation parameter with the movement direction “right” and the movement amount “1 pixel”, it is parallel to the right by one pixel. A moved target image 110 is obtained. Here, the parallel movement of the image is a parallel movement of the subject appearing in the image. When the conversion parameter list 130 includes the rotation parameter and the translation parameter, the coordinate conversion circuit 12 is obtained by translating the target image 110 and the rotation image obtained by rotating the target image 110. A translational image can be generated. Similar to the rotation parameter, the translation parameter includes, for example, the coordinates of the four pixels of the mapping source in the target image 110 for the four corner pixels in the translation image.

<Usage example of image processing apparatus>
The image processing apparatus 1 can be used for various purposes. Hereinafter, as an example, a case where the image processing apparatus 1 is used in the image processing system 500 which determines whether an object appearing in an image corresponds to a comparison target will be described.

<Configuration of image processing system>
FIG. 13 shows the configuration of the image processing system 500. As shown in FIG. The image processing system 500 is a system that uses image processing to determine whether an object corresponds to a comparison target. For example, the image processing system 500 is introduced into a production line in which a circular candy such as a cookie or chocolate is manufactured in a factory. The image processing system 500 determines whether or not the confectionery manufactured on the production line in actual operation corresponds to the confectionery manufactured correctly. In other words, the image processing system 500 determines whether the confectionery produced on the production line in operation is correctly manufactured, that is, the quality of the confectionery. As a result, the image processing system 500 can detect the collapse and chipping of the candy pattern. In the image processing system 500 of this example, the object is a candy, and the comparison object to be compared with the object is a correctly produced candy (goods candy). The object may be something other than sweets. The outer shape of the object may be other than a circle.

  As shown in FIG. 13, the image processing system 500 includes the CPU 2, the system memory 3 and the image processing circuit 4 described above, an imaging device 6, and a DSP (Digital Signal Processor) 7. These components are connected to each other by a bus 5.

  The imaging device 6 images the candy manufactured on the manufacturing line, and generates a captured image 510 in which the candy is reflected. The captured image 510 is stored in the system memory 3. In the production line, the confectionery is conveyed by the conveyance conveyor, and the imaging device 6 images the confectionery being conveyed. A plurality of manufactured sweets move in a line on the conveyer in an irregular direction. In this example, the captured image 510 generated by the imaging device 6 is a color image, but may be a grayscale image.

  FIG. 14 is a diagram schematically illustrating an example of a captured image 510 obtained by the imaging device 6. In the captured image 510 shown in FIG. 14, a correctly manufactured candy 600, that is, a non-defective candy 600 is shown. Further, in the captured image 510, a transport conveyor 601 which is a background is shown. On one main surface 600a of the non-defective candy 600, for example, a pattern of alphabet “A” is shown. The pattern of the non-defective candy 600 may be other than this. Moreover, the pattern may be attached | subjected to the both main surfaces of the good confectionery 600.

  The sweets 600 move on the conveyer 601 with the one main surface 600 a upward. On the conveyor 601, a plurality of sweets 600 move in a line. Then, the imaging device 6 images the candy 600 from the one main surface 600 a side of the candy 600. Therefore, the picked-up image 510 shows the one main surface 600a of the candy 600.

  In the image processing system 500, the CPU 2, the system memory 3, the image processing circuit 4 and the DSP 7 determine whether or not the candy 600 shown in the captured image 510 obtained by the imaging device 6 corresponds to that produced correctly. A determination device 8 that determines whether or not the candy 600 is correctly manufactured and outputs the determination result is configured. The determination result obtained by the determination device 8 is input to a control device provided in the factory that manages the operation of the manufacturing line. The control device performs various operations based on the input determination result. Hereinafter, the process in which the determination device 8 determines whether the confectionery product 600 shown in the captured image 510 is correctly manufactured is referred to as “confectionery determination process”.

  The DSP 7 executes a program stored in the system memory 3. When the CPU 2 and the DSP 7 execute the program in the system memory 3, various functional blocks are formed in the determination device 8.

  FIG. 15 is a diagram showing the configuration of the determination device 8. As illustrated in FIG. 15, the determination device 8 includes a conversion unit 530, a feature image generation unit 531, and a determination unit 532. The conversion unit 530 converts the captured image 510 obtained by the imaging device 6 from a color image to a grayscale image, and outputs the converted captured image 510 as a captured image 511.

  The feature image generation unit 531 generates a feature image 520 indicating the feature of the candy 600 based on the captured image 511 on which the candy 600 appears. Based on the characteristic image 520 generated by the characteristic image generation unit 531, the determination unit 532 determines whether the candy 600 shown in the captured image 511 is correct, and outputs the determination result 521. The determination unit 532 compares the feature image 520 generated from the captured image 511 by the feature image generation unit 531 with the template feature image 522 indicating the feature of the non-defective candy 600, and based on the comparison result, the captured image It is determined whether or not the candy 600 shown in 511 is correct. The template feature image 522 is stored in the system memory 3.

  FIG. 16 is a diagram illustrating a configuration of the feature image generation unit 531. As illustrated in FIG. 16, the feature image generation unit 531 includes a first image generation unit 540 and a second image generation unit 550. The first image generation unit 540 generates a first image 524 showing the candy 600 based on the captured image 511. The second image generation unit 550 synthesizes a plurality of rotated images obtained by rotating the first image 524 generated by the first image generation unit 540, and outputs a synthesized image obtained thereby as a feature image 520.

  The first image generation unit 540 includes an extraction unit 541 and an edge image generation unit 542. The extraction unit 541 extracts the candy area 523 in which the candy 600 appears from the captured image 511. The edge image generation unit 542 performs edge detection on the candy region 523 extracted by the extraction unit 541 and generates an edge image as the first image 524.

  In this example, the image processing circuit 4 functions as the second image generation unit 550. Also, the DSP 7 functions as a conversion unit 530, a first image generation unit 540, and a determination unit 532. The first image 524 (edge image) generated by the first image generation unit 540 corresponds to the above-described target image 110 processed by the image processing circuit 4, and the characteristic image 520 generated by the second image generation unit 550. (Composite image) corresponds to the composite image 140 generated by the image processing circuit 4.

  When the image processing system 500 is turned on, the CPU 2 performs initial setting of the image processing system 500. In the initial setting of the image processing system 500, the initial setting of the image processing circuit 4 is performed. In the initial setting of the image processing circuit 4, as described above, the CPU 2 sets data in various setting registers in the image processing circuit 4.

  When the initial setting of the image processing system 500 is completed, the imaging device 6 starts imaging and performs imaging at a predetermined frame rate. The captured images 510 that are sequentially generated by the imaging device 6 are stored in the system memory 3. The determination device 8 reads the captured image 510 from the system memory 3 and performs a candy determination process based on the read captured image 510.

<About the flow of candy determination processing>
Next, a series of operations of the determination device 8 when the determination device 8 performs the candy determination process while the production line is in actual operation will be described. FIG. 17 is a flowchart showing the operation.

  As shown in FIG. 17, in step s11, the conversion unit 530 (DSP 7) reads the captured image 510 from the system memory 3 and converts the read captured image 510 from a color image to a grayscale image, thereby obtaining the image The captured image 511 thus obtained is stored in the system memory 3 as the determination target image 511.

  Next, in step s12, the extraction unit 541 (DSP 7) reads the determination target image 511 from the system memory 3, and extracts the candy area 523 in which the candy 600 appears from the read determination target image 511. Next, in step s13, the edge image generation unit 542 (DSP 7) performs edge detection on the candy area 523 extracted by the extraction unit 541 to obtain an edge image as a first image 524 indicating the candy 600. Generate Then, the edge image generation unit 542 (DSP 7) stores the generated edge image in the system memory 3.

  Next, in step s14, the second image generation unit 550 (image processing circuit 4) reads out the edge image from the system memory 3, and rotates a plurality of rotated images (rotated image 120) obtained by rotating the read edge image. A composite image (composite image 140) synthesized is generated as a feature image 520. The second image generation unit 550 stores the generated feature image 520 in the system memory 3.

  Next, in step s15, the determination unit 532 (DSP 7) reads the feature image 520 and the template feature image 522 from the system memory 3. Then, the determination unit 532 compares the read feature image 520 and the template feature image 522, and determines whether the candy 600 shown in the determination target image 511 is correct based on the comparison result. In other words, based on the comparison result of the characteristic image 520 and the template characteristic image 522, the determination unit 532 determines whether or not the candy 600 shown in the captured image 510 generated by the imaging device 6 is correct. . Then, in step s16, the determination unit 532 writes the determination result 521 in step s15 to the system memory 3. The determination result 521 in the system memory 3 is input to a control device provided in the factory. If the candy 600 reflected in the captured image 510 is not manufactured correctly, in other words, if the candy 600 reflected in the captured image 510 does not correspond to a non-defective product, for example, the control device outputs a warning sound from a speaker. Issue a warning to the outside by displaying warning information on the display, etc.

  Thereafter, the determination device 8 executes step s1 to read the captured image 510 generated next by the imaging device 6 from the system memory 3, and the captured image 511 obtained from the read captured image 510 is a new determination target image Steps 511 to s12 are executed. Thereafter, the determination device 8 performs the same operation.

<Detailed description of each component>
Hereinafter, operations of the extraction unit 541, the edge image generation unit 542, the second image generation unit 550, and the determination unit 532 will be described in more detail.

<About the extraction unit>
FIG. 18 is a diagram schematically illustrating an example of the candy region 523 that is extracted from the captured image 511 by the extraction unit 541. There are various methods for extracting the candy area 523 from the captured image 511.

  For example, there is a first extraction method utilizing the fact that the outer shape of the candy 600 is circular. In the first extraction method, first, edge detection is performed on the captured image 511 to generate an edge image. As a method of generating an edge image, for example, the Sobel method, Laplacian method, Canny method or the like is used. Next, a circular area is extracted from the generated edge image. For example, Hough transform is used as a method of extracting a circular area. Then, the circular area in the captured image 511 present at the same position as the position of the circular area in the edge image is taken as the candy area 523.

  Another method is a second extraction method of extracting the candy area 523 from the captured image 511 using the background subtraction method and the labeling. In the second extraction method, first, a background difference image indicating a difference between the captured image 511 and a background image (an image showing only the background of the captured image 511) is generated, and the generated background difference image is binarized. Ru. Then, labeling such as 4-connection is performed on the binary background difference image. Then, a partial area in the captured image 511 that exists at the same position as the position of the connected area (independent area) obtained as a result of labeling in the binary background difference image is set as the candy area 523.

  In this example, the extraction unit 541 extracts the candy area 523 from the captured image 511 by a method different from the above two methods. The operation of the extraction unit 541 will be described below. The extraction unit 541 may extract the candy area 523 from the captured image 511 using any one of the above two methods.

  First, the extraction unit 541 generates a background difference image indicating the difference between the captured image 511 and the background image 560 (an image showing only the background of the captured image 511), and binarizes the generated background difference image. FIG. 19 is a view schematically showing the background image 560, and FIG. 20 is a view schematically showing the binary background difference image 561. As shown in FIG. In the binary image schematically shown in FIG. 20 and the drawings to be described later, an area where the pixel value is “1” (high luminance area) is shown in black, and an area where the pixel value is “0” (low luminance) The region is shown in white. The background image 560 is stored in the system memory 3 in advance.

  Next, the extraction unit 541 performs template matching on the binary background difference image 561 using the binary outer shape template 562 indicating the outer shape of the sweet 600. That is, the extraction unit 541 specifies where an area similar to the external shape template 562 exists in the background difference image 561. In other words, the extraction unit 541 specifies where in the background difference image 561 there is an area that matches the outline of the candy 600 indicated by the outline template 562. FIG. 21 schematically shows the outer shape template 562. The outline template 562 is stored in the system memory 3 in advance.

  In template matching, the extraction unit 541 moves the outline template 562 little by little in the raster scan direction on the background difference image 561 as shown in FIG. In other words, the extraction unit 541 raster scans the outline template 562 on the background difference image 561. At this time, the extraction unit 541 generates an AND image of the outer shape template 562 and the partial region of the background difference image 561 overlapping the outer shape template 562 at each position of the outer shape template 562. Thereby, a plurality of binary AND images are generated. Then, the extraction unit 541 selects the background difference image 561 of the outer shape template 562 used to generate the AND image with the largest number of pixels (high luminance pixels) having a pixel value of “1” among the plurality of generated AND images. Identify the top position. This position is a position where an area similar to the outline template 562 exists in the background difference image 561. And the extraction part 541 extracts the partial area | region 511a in the captured image 511 which exists in the same position as the specified position as FIG. In other words, when the outer shape template 562 is arranged in the captured image 511 at the same position as the specified position, the extraction unit 541 uses the partial region 511a in the captured image 511 that overlaps the outer shape template 562 as the candy region 523. Extract. At this time, in the partial area 511a, the confectionery area 523 in which the pixel value of each pixel outside the circle indicated by the outer shape template 562 on the partial area 511a is zero may be used. The candy area 523 extracted by the extraction unit 541 is a gray scale image.

<About the edge image generator>
The edge image generation unit 542 performs edge detection on the candy area 523 extracted by the extraction unit 541 using, for example, the Sobel method, Laplacian method, Canny method or the like to generate an edge image 565. The edge image generation unit 542 uses, for example, the Sobel method whose processing is light. The edge image 565 is a binary image. FIG. 24 is a diagram schematically showing the edge image 565.

<About the second image generation unit>
The second image generation unit 550 generates a combined image 570 by combining a plurality of rotated images 565 a obtained by rotating the edge image 565 generated by the edge image generating unit 542 as the feature image 520. The rotated image 565a corresponds to the above-described rotated image 120, and the composite image 570 corresponds to the above-described composite image 140.

  FIG. 25 is a diagram corresponding to FIG. 9 described above. The second image generation unit 550 separately performs a plurality of types of rotation coordinate transformations on the edge image 565 (target image 110) in the same manner as the coordinate transformation circuit 12 of the image processing circuit 4 described above. Thus, as shown in FIG. 25, a plurality of rotation images 565a (a plurality of rotation images 120) having different rotation angles are generated. Then, in the same manner as the processing circuit 14 of the image processing circuit 4, the second image generation unit 550 synthesizes the plurality of generated rotational images 565a to generate a synthesized image 570 (the synthesized image 140). The second image generation unit 550 uses the generated composite image 570 as a feature image 520 indicating the feature of the candy 600 shown in the captured image 511.

  As described above, the imaging device 6 images the candy 600 conveyed in an irregular direction, and generates a captured image 510 on which the candy 600 is displayed. Therefore, the circumferential direction of the candy 600 shown in the captured image 511 generated by the conversion unit 530 of the determination device 8 is not always the same. In other words, the circumferential direction of the candy 600 reflected in the captured image 511 is not constant, and the attitude of the candy 600 reflected in a certain captured image 511 is the orientation of the candy 600 reflected in another captured image 511. There is a possibility that the posture may be rotated around the rotation axis along the thickness direction passing through the center of gravity (center) of both main surfaces of 600. Therefore, the direction of the circumferential direction of the candy 600 reflected in the candy area 523 is not constant. On the other hand, the composite image 570 is a composite of a plurality of rotated images 565a obtained by rotating the edge image 565 showing the candy 600 shown in the captured image 511. Therefore, even if the direction in the circumferential direction of the candy 600 shown in the captured image 511 is not constant, the composite image 570 obtained from the captured image 511 hardly changes if the candy 600 is a non-defective item. Therefore, it can be said that the composite image 570 is a feature image 520 showing the characteristics of the candy 600 that is not easily affected by the circumferential direction of the candy 600 shown in the captured image 511.

<About the judgment unit>
The determination unit 532 compares the composite image 570 (feature image 520) generated by the second image generation unit 550 with the template feature image 522 indicating the feature of the non-defective candy 600, and based on the comparison result, It is determined whether the sweets 600 shown in the captured image 511 are correctly manufactured. As the template characteristic image 522, a characteristic image 520 (a composite image 570) indicating the characteristic of the good product 600 generated by the characteristic image generation unit 531 in the same manner as described above from the captured image 511 on which the good product 600 is captured. ) Is used. When the production line is not in operation, in order to obtain the template characteristic image 522, good-quality sweets 600 are input to the production line. In the image processing system 500, a picked-up image 511 in which the inputted good confectionery 600 is reflected is generated. This captured image 511 is referred to as a “standard image 511”. In the image processing system 500, the feature image generation unit 531 generates, based on the standard image 511, a feature image 520 (a composite image 570) indicating the features of the non-defective candy 600 shown in the standard image 511. When this feature image 520 is called a “standard feature image 520”, the standard feature image 520 becomes the template feature image 522. A template feature image 522 generated when the production line is not actually operating is stored in the system memory 3. Thereafter, in order to distinguish the characteristic image 520 (the characteristic image 520 generated from the captured image 511 on which the candy 600 to be subjected to the quality judgment is shown) generated while the production line is in operation, from the standard characteristic image 520 It may be called "feature image 520".

  The determination unit 532 compares the composite image 570 (target feature image 520) and the template feature image 522, for example, to obtain a similarity (difference) between them. The determination unit 532 uses, for example, a sum of absolute difference (SAD) as a value indicating the degree of similarity. A large SAD means low similarity, and a small SAD means high similarity. Other values such as SSD (Sum of Squared Difference) or NCC (Normalized Correlation Coffiecient) may be used as a value indicating the degree of similarity.

  When the degree of similarity between the composite image 570 and the template characteristic image 522 is high, the determination unit 532 determines that the candy 600 shown in the captured image 511 is correctly manufactured, and the degree of similarity is low. In the case, it is determined that the candy 600 shown in the captured image 511 is not correctly manufactured. Specifically, when the SAD between the composite image 570 and the template characteristic image 522 is equal to or less than the threshold value, the determination unit 532 determines that the candy 600 shown in the captured image 511 is correctly manufactured. If it is determined that the SAD is greater than the threshold value, it is determined that the candy 600 shown in the captured image 511 is not correctly manufactured. Then, the determination unit 532 stores the determination result 521 in the system memory 3.

  In the above example, the binary edge image 565 is used as the first image 524 indicating the candy 600 shown in the captured image 511, but the candy region 523 of the grayscale image is used as the first image 524. It is good. In this case, a combined image obtained by combining a plurality of rotated images obtained by rotating the candy region 523 is used as the feature image 520.

  In the above example, the composite image 570 is the feature image 520, but a part of the composite image 570 may be the feature image 520. In this case, a part of the standard feature image 520 is a template feature image 522.

  As described above, in the image processing system 500, at least a part of the combined image 570 obtained by combining the plurality of rotated images obtained by rotating the first image 524 indicating the sweet 600 is used as the feature image 520. The characteristic image 520 is unlikely to be affected by the circumferential direction of the candy 600 shown in the captured image 511. Therefore, even if the circumferential direction of the candy 600 in the captured image 511 is not constant, whether the candy 600 in the captured image 511 is correctly manufactured using the characteristic image 520 Can be determined more accurately. Therefore, the determination accuracy of the sweets determination process is improved.

  In this example, since the binary edge image 565 is the first image 524, the brightness of the imaging region in the imaging device 6 is compared to the case where the candy region 523 of the grayscale image is the first image 524. It is possible to suppress the first image 524 from being affected by the change in height. Therefore, it can suppress that the characteristic image 520 receives to the influence of the brightness change of the imaging area where the candy 600 is imaged, As a result, the precision of the quality determination of the candy 600 improves.

  In this example, since the extraction unit 541 extracts the candy region 523 from the captured image 511 by using template matching, the first extraction method described above using the Hough transform or labeling is used. Compared with the second extraction method, the extraction process is simplified.

  In this example, the determination unit 532 compares the feature image 520 generated from the captured image 511 with the template feature image 522 using SAD or the like, and based on the comparison result, the confectionery reflected in the captured image 511 Since it is determined whether 600 is a non-defective product, the determination process is simplified.

  In the above-mentioned example, although image processing system 500 containing image processing device 1 was introduced in a manufacturing line which manufactures sweets, it may be introduced in other devices or various places.

  For example, the image processing system 500 may be introduced into a vending machine into which money having a pattern on its surface is inserted. In this case, the money inserted into the vending machine is imaged by the imaging device 6. In a vending machine, for example, money moves while rotating on a rail. The imaging device 6 images a rotating coin. The feature image generation unit 531 generates a feature image indicating the feature of the money based on the captured image 511 showing the money. Based on the feature image generated by the feature image generator 531, the determination unit 532 determines whether or not the money shown in the captured image 511 corresponds to the correct one. Since the pattern on the money surface can be handled in the same manner as the pattern on the candy surface, the image processing system 500 determines whether or not the money shown in the captured image 511 corresponds to the correct one in the same manner as described above. can do. Thereby, the authenticity of the money thrown into the vending machine can be determined.

  In addition, the image processing system 500 may be introduced to a manufacturing line that assembles a product in a factory. More specifically, the image processing system 500 may be introduced into, for example, a manufacturing line for mounting a plurality of components on a substrate. In this case, the imaging device 6 captures an image of a substrate on which a plurality of components are mounted. The feature image generation unit 531 generates a feature image indicating the feature of the substrate based on the captured image 511 in which the substrate on which the plurality of components are mounted is captured. Based on the characteristic image generated by the characteristic image generation unit 531, the determination unit 532 determines whether the substrate on which the plurality of components are mounted, which is shown in the captured image 511, corresponds to the substrate on which the plurality of components are correctly mounted. That is, it is determined whether or not a plurality of components are correctly mounted on the board. Since a plurality of parts on the substrate can be treated in the same manner as the pattern on the coin surface, in the same manner as described above, the image processing system 500 determines whether the plurality of parts are correctly mounted on the substrate Can be determined. As a result, it is possible to detect a mounting error of a part, a mounting omission of a part, and the like.

  In addition, the image processing system 500 may be installed in a sushi restaurant. In a sushi restaurant, a price according to the pattern of the plate on which the product is placed may be set as the price of the product such as sushi. And in a conveyor belt sushi restaurant, the sum total of the price according to the pattern of each dish which the customer took in hand may be calculated as food and drink charges. The image processing system 500 introduced to such a spin-roll sushi store specifies the pattern of the plate picked up by the customer by image processing.

  Specifically, when a customer picks a plate from a rotating lane on which a plurality of plates carrying sushi or the like is mounted, the plate is imaged by the imaging device 6. The feature image generation unit 531 generates a feature image indicating the feature of the dish based on the captured image 511 in which the dish is reflected.

  Here, a pattern is given to the peripheral part of the plate, and a product such as sushi is placed on the central part (the part without the pattern) of the plate. Since the central portion of the edge image 565 generated by the first image generation unit 540 of the characteristic image generation unit 531 reflects a product and does not indicate the feature of the dish, the first image generation unit 540 The portion of the edge image 565 other than the central portion is taken as a first image 524 showing a plate. The second image generation unit 550 of the feature image generation unit 531 uses at least a part of a composite image obtained by combining a plurality of rotated images obtained by rotating the first image 524 as a feature image.

  The determination unit 532 determines, based on the feature image generated by the feature image generation unit 531, whether or not the plate reflected in the captured image 511 corresponds to a plate having a predetermined pattern. That is, the determination unit 532 determines whether or not the pattern of the dish shown in the captured image 511 matches the predetermined pattern based on the feature amount. Since the pattern on the dish surface can be handled in the same manner as the pattern on the money surface, in the same manner as described above, the image processing system 500 determines whether the dish displayed in the captured image 511 corresponds to a dish having a predetermined pattern. It can be determined whether or not. The determination unit 532 determines, for each of the plurality of types of patterns attached to the plurality of types of dishes used in the sushi restaurant, whether or not the pattern matches the pattern of the dishes shown in the captured image 511. . Thereby, the pattern of the dish reflected in the captured image 511 is specified. Therefore, the price according to the pattern of the dish reflected in the captured image 511 can be specified automatically. Therefore, it is possible to automatically calculate the total price according to the pattern of each dish taken by the customer, that is, the food and beverage charges.

  Further, the image processing system 500 may be introduced into a distribution center. In a distribution center, a sticker with a mailing address or the like may be affixed to a package such as cardboard. The image processing system 500 determines whether the sticker affixed to the package corresponds to the correct sticker, that is, whether the package is affixed the correct sticker. In this case, the sticker attached to the luggage is imaged by the imaging device 6. The feature image generation unit 531 generates a feature image indicating the feature of the seal based on the captured image 511 in which the seal is reflected. Based on the feature image generated by the feature image generation unit 531, the determination unit 532 determines whether the seal shown in the captured image 511 corresponds to the correct seal, that is, the seal shown in the captured image 511 is correct. It is determined whether or not. Since the characters on the seal surface can be handled in the same manner as the pattern on the money surface, the image processing system 500 can determine whether the correct seal is affixed to the package in the same manner as described above. . In this way, it is possible to detect a sticking error or the like of the seal.

  The second image generation unit 550 (image processing circuit 4) changes the number of rotated images (value of N) used when generating the composite image 570 according to the operation status of the image processing system 500. Also good. For example, the case where the conveyance speed of the sweets in a manufacturing line changes according to a time zone etc. is assumed. In such a case, when the conveyance speed of the sweets is high and it is necessary to obtain the determination result 521 immediately, the second image generation unit 550 gives priority to the processing time over the determination accuracy, and the composite image is generated. The number of rotated images used when generating 570 is reduced. For example, the second image generation unit 550 rotates the edge image 565 by 10 ° from 10 ° to 360 ° to generate 36 rotated images 565a. Then, the second image generation unit 550 generates a combined image 570 by combining the generated 36 rotated images 565a. On the other hand, when the conveyance speed of sweets on the manufacturing line is low and it is not necessary to obtain the determination result 521 immediately, the second image generation unit 550 gives priority to the determination accuracy over the processing time, and combines them. The number of rotated images used in generating the image 570 is increased. For example, the second image generation unit 550 rotates the edge image 565 by 1 ° from 0 ° to 360 ° to generate 360 rotated images 565a. Then, the second image generation unit 550 generates a combined image 570 by combining the generated 360 rotated images 565a.

  In the production line, the product may move on the conveyor without changing its posture. When such a product is picked up by the imaging device 6, the posture of the product shown in the captured image 510 is constant, but the position of the product shown in the left and right direction may vary. For example, in one captured image 510, a product is displayed on the left, in another captured image 510, the product is in the center, and in another captured image 510, the product may be displayed on the right.

  In such a case, a plurality of types of parallel movement parameters are included in the conversion parameter list 130, and the captured image 511 is input to the second image generation unit 550 as the target image 110, not the edge image 565. Then, the second image generation unit 550 separately performs each of a plurality of types of parallel movement coordinate transformation on the captured image 511 based on a plurality of types of parallel movement parameters, and obtains a plurality of parallel movement images. Generate For example, the second image generation unit 550 generates a plurality of right parallel movement images in which the captured image 511 is moved by two pixels in the right direction and moves the edge image 565 by two pixels in the left direction. Generate multiple left translation images. Then, the second image generation unit 550 generates a composite image by combining a plurality of parallel displacement images including a plurality of right parallel displacement images and a plurality of left translational images. The determination unit 532 determines whether or not the product shown in the captured image 510 is correctly manufactured using the composite image as a feature image. Thus, even if the position of the product in the lateral direction of the captured image 510 varies, the image processing system 500 appropriately determines whether the product in the captured image 510 is properly manufactured. can do.

  In addition, depending on the environment in which the image processing system 500 is introduced, the size of the object shown in the captured image 510 may vary. For example, when the actual size of the object varies, or when the distance between the object and the imaging lens of the imaging device 6 varies, the size of the object reflected in the captured image 510 may vary.

  In such a case, the conversion parameter list 130 includes a plurality of types of enlargement parameters and a plurality of types of reduction parameters. Then, the second image generation unit 550 separately performs each of the plurality of types of enlargement coordinate transformations on the edge image (the first image 524) based on the plurality of types of enlargement parameters, thereby enlarging the plurality of sheets. Generate an image. Furthermore, the second image generation unit 550 separately performs a plurality of types of reduction coordinate transformations on the edge image based on the plurality of types of reduction parameters to generate a plurality of reduced images. Then, the second image generation unit 550 combines a plurality of images composed of a plurality of enlarged images and a plurality of reduced images to generate a composite image. The determination unit 532 determines whether the target shown in the captured image 510 corresponds to a comparison target (for example, a non-defective product), using the composite image as a feature image. Thereby, even if the size of the object appearing in the captured image 510 varies, the image processing system 500 appropriately determines whether the object appearing in the captured image 510 corresponds to a comparison target. be able to.

  When a plurality of types of products are produced in the same production line, the conversion parameters of the plurality of types of conversion parameters used in the second image generation unit 550 (image processing circuit 4) are generated according to the types of products. The set may be changed. For example, when a certain type of product is manufactured on the manufacturing line, the second image generation unit 550 uses a plurality of types of rotation parameters included in the conversion parameter list 130 to generate a plurality of target images 110. Perform each kind of rotation coordinate transformation separately. On the other hand, when another type of product is manufactured on the manufacturing line, the second image generation unit 550 uses the plurality of types of translation parameters included in the conversion parameter list 130 to generate the target image 110. On the other hand, each of the plural types of coordinate conversion for parallel movement is performed separately.

<Various modifications>
Various modified examples of the following image processing circuit 4 will be described.

<First Modification>
FIG. 26 is a diagram showing the configuration of the coordinate conversion circuit 12 according to the present modification. The coordinate conversion circuit 12 according to this modification can adjust the position of a specific subject (hereinafter referred to as “specific subject”) among the subjects shown in the rotated image 120. Hereinafter, the coordinate conversion circuit 12 according to this modification will be described focusing on differences from the coordinate conversion circuit 12 shown in FIG. 4 described above.

  The coordinate conversion circuit 12 according to this modification includes an addition circuit 25. The adder circuit 25 adds the parameter offset 230 to the conversion parameter 131 read from the storage circuit 13 by the control circuit 21. Then, the addition circuit 25 outputs the conversion parameter 131 added with the parameter offset 230 to the conversion circuit 20.

  The parameter offset 230 is stored in the register 22 of the control circuit 21. The parameter offset 230 indicates, for example, the amount of deviation of the position of the center of gravity of the specific subject in the target image 110 with respect to the position of the center of gravity of the target image 110 (the intersection of diagonals in the target image 110 of quadrilateral). When the image processing circuit 4 functions as the second image generation unit 550 of the above-described image processing system 500 that performs the candy determination process, the specific subject is, for example, the candy 600 shown in the captured image 511. In this case, the center of gravity of the specific subject is the center of gravity of the candy 600, that is, the center of the circular candy 600.

  The parameter offset 230 is a displacement amount xoff of the position of the center of gravity of the specific subject in the target image 110 with respect to the position of the center of gravity of the target image 110 in the x direction (for example, the horizontal direction of the image) An offset amount yoff of the position of the center of gravity of the specific subject in the target image 110 in the y direction (for example, the vertical direction of the image) is included. Hereinafter, the shift amount xoff is referred to as “x offset xoff”. The shift amount yoff is referred to as “y offset yoff”.

  Here, the coordinates of the center of gravity of the target image 110 are (x1, y1), and the coordinates of the center of gravity of the specific subject in the target image 110 are (x2, y2). At this time, x offset xoff = x2-x1 and y offset yoff = y2-y1.

  When the addition circuit 25 adds the parameter offset 230 to the conversion parameter 131, the addition circuit 25 adds the x offset xoff of the parameter offset 230 to each of the four coordinates constituting the conversion parameter 131. Then, the adding circuit 25 adds the y offset yoff of the parameter offset 230 to the y coordinate of each of the four coordinates constituting the conversion parameter 131.

  The conversion circuit 20 performs coordinate conversion on the target image 110 based on the conversion parameter 131 to which the parameter offset 230 is added. When the conversion parameter 131 is the rotation parameter 132, the conversion circuit 20 performs rotation coordinate conversion on the target image 110 based on the rotation parameter 132 to which the parameter offset 230 is added. Thereby, even if the position of the center of gravity of the specific subject in the target image 110 is deviated from the position of the center of gravity of the target image 110, the specific subject in the rotational image 120 generated based on the target image 110 The position of the center of gravity coincides with the position of the center of gravity of the rotated image 120. In the case where the image processing circuit 4 is used in the image processing system 500 that performs the candy determination process, the position of the center of gravity of the candy in the edge image 565 is shifted from the position of the center of gravity of the edge image 565 Even if it exists, the position of the gravity center of the confectionery in the rotation image 565a generated based on the edge image 565 coincides with the position of the gravity center of the rotation image 565a.

  The parameter offset 230 is set by the CPU 2 together with the specific information 220 when the CPU 2 makes settings for the target image 110 with respect to the image processing circuit 4 (such as steps s1 and s2 in FIG. 12 described above). Set to When setting the target image 110 with respect to the image processing circuit 4, the CPU 2 reads from the system memory 3 the identification information 220 and the parameter offset 230 corresponding to the identification image 110, and The parameter offset 230 is set in the register 22. The control circuit 21 outputs the parameter offset 230 in the register 22 to the addition circuit 25 and reads the conversion parameter 131 from the storage circuit 13 based on the specific information 220 in the register 22. The addition circuit 25 adds the parameter offset 230 to the conversion parameter 131 read from the storage circuit 13 and outputs the conversion parameter 131 to which the parameter offset 230 has been added to the conversion circuit 20.

  When the image processing circuit 4 is used in the image processing system 500 that performs the sweets determination process, the parameter offset 230 is generated by, for example, the DSP 7. The DSP 7 functioning as the first image generation unit 540 can generate the parameter offset 230 when generating the edge image 565 (first image 524) based on the captured image 511. The DSP 7 stores the parameter offset 230 obtained when generating the edge image 565 (target image 510) in the system memory 3 in association with the edge image 565.

  Thus, in the coordinate conversion circuit 12 according to the present modification, the conversion circuit 20 performs coordinate conversion on the target image 110 based on the conversion parameter 131 to which the parameter offset 230 is added. Therefore, by adjusting the parameter offset 230 for each target image 110, it is possible to adjust the position of the specific subject in the processed image 115 (rotation image 120) obtained by the conversion circuit 20. In the image processing system 500 that performs the candy determination process, when the position of the candy in the rotation image 565a varies, the composite image 570 varies, and as a result, the feature image 520 varies. If the feature image 520 is dispersed, even the feature image 520 generated based on the captured image 510 on which the correctly manufactured sweets are reflected may be significantly different from the template feature image 522. As a result, the determination accuracy in the determination unit 532 may be reduced. Therefore, as in this modified example, by matching the position of the center of gravity of the candy in the rotated image 565a with the position of the center of gravity of the rotated image 565a, the variation in the position of the candy in the rotated image 565a is reduced, As a result, the determination accuracy in the determination unit 532 can be improved.

<Second Modification>
In the present modification, as shown in FIG. 27, the image processing circuit 4 superimposes multiple types of coordinate transformation on one target image 110 in the input frame buffer 11 to generate one processed image. Generate 215. Hereinafter, a process of generating a single processed image 215 by overlapping a plurality of types of coordinate conversion on one target image 110 may be referred to as “overlap conversion process”.

  FIG. 28 is a diagram showing the configuration of the coordinate conversion circuit 12 according to this modification. FIG. 29 is a diagram showing a configuration of the processing circuit 14 according to this modification. As illustrated in FIG. 29, the processing circuit 14 according to the present modification does not include the addition processing unit 42 as compared with the processing circuit 14 illustrated in FIG. 11 described above. In the processing circuit 14, the pixel data output from the coordinate conversion circuit 12 is written into the write buffer 43. Further, the pixel data in the read buffer 44 is output to the coordinate conversion circuit 12.

  As shown in FIG. 28, the coordinate conversion circuit 12 according to the present modification further includes a selection circuit 23 as compared with the coordinate conversion circuit 12 shown in FIG. 4 described above. The selection circuit 23 selects either the output of the input frame buffer 11 or the output of the processing circuit 14 based on the control signal CNT1 output from the control circuit 21 and connects the selected output to the conversion circuit 20. In the present modification, when the control signal CNT1 is in the first state (for example, when it indicates "1"), the selection circuit 23 selects the output of the input frame buffer 11. When the selection circuit 23 selects the output of the input frame buffer 11, the conversion circuit 20 can access the target image 110 in the input frame buffer 11. On the other hand, when the control signal CNT1 is in the second state (for example, when it indicates "0"), the selection circuit 23 selects the output of the processing circuit 14. When the selection circuit 23 selects the output of the processing circuit 14, the conversion circuit 20 is connected to the read buffer 44 of the processing circuit 14 and can read pixel data from the read buffer 44.

  FIG. 30 is a diagram showing an example of the conversion parameter list 130 according to this modification. In the conversion parameter list 130 according to this modification, a unique parameter number (identification information) is associated with each of a plurality of types of conversion parameters 131. Assuming that the conversion parameter list 130 includes P types of conversion parameters 131 (P ≧ 2), in the example of FIG. Are associated with each other.

  Further, in the example of FIG. 30, the conversion parameter list 130 includes a plurality of types of rotation parameters 132, a plurality of types of enlargement parameters 133, a plurality of types of reduction parameters 134, and a plurality of types of parallel movement parameters 135. There is. Specifically, the conversion parameter list 130 includes 360 kinds of rotation parameters 132 for rotating the image from 1 ° to 360 °. The conversion parameter list 130 also includes an enlargement parameter 133 necessary for enlargement coordinate conversion for enlarging the image by 1.1 times, and an enlargement necessary for enlargement coordinate conversion for enlarging the image by 1.2 times. Parameters 133 and the like are included. The conversion parameter list 130 includes a reduction parameter 134 necessary for coordinate conversion for reduction for reducing the image by 0.9 times, and a reduction necessary for coordinate conversion for reduction for reducing the image to 0.8 times. Parameters 134 and the like are included. Then, in the conversion parameter list 130, a parallel movement parameter 135 necessary for parallel movement coordinate conversion for parallel movement of the image by one pixel in the right direction, and for parallel movement of the image by two pixels in the right direction. A translation parameter 135 and the like necessary for coordinate conversion for translation are included.

  FIG. 31 is a diagram showing an example of the specific information 220 according to this modification. As shown in FIG. 31, the identification information 220 according to the present modification includes a look-up table (LUT) 240 and reference mode information 250.

  A plurality of parameter numbers are described in the LUT 240. The reference mode information 250 is information indicating how to refer to a plurality of parameter numbers described in the LUT 240. The control circuit 21 refers to the parameter numbers in the LUT 240 one by one in accordance with the reference mode information 250, and reads the conversion parameter 131 corresponding to the referenced parameter number each time the parameter number is referenced. The read conversion parameter 131 is input to the conversion circuit 20.

  The reference mode information 250 includes the number of times of reference 251, the reference start position 252, and the reference interval 253. The number of references 251 indicates the number of references to the parameter number in the LUT 240. The reference start position 252 indicates the position of the parameter number to be referred to first in the LUT 240. The reference interval 253 indicates how many conversion parameters 131 apart from the previously referenced conversion parameter 131 are to be referred to in the LUT 240 when referring to the conversion parameter 131 at a certain timing.

  The control circuit 21 sequentially refers to a plurality of parameter numbers in the LUT 240 from the top side to the tail side of the LUT 240. When coordinate conversion of M (≧ 2) types is performed on the target image 110 in the coordinate conversion circuit 12 in an overlapping manner, the control circuit 21 first determines the parameter number of the position indicated by the reference start position 252 in the LUT 240. Refer to Then, the control circuit 21 reads out the conversion parameter 131 corresponding to the referred parameter number from the storage circuit 13 and inputs it to the conversion circuit 20. Next, in the LUT 240, the control circuit 21 refers to the parameter number that is separated from the parameter number that was referred to last time by the number indicated by the reference interval 253 toward the end. Then, the control circuit 21 reads the conversion parameter 131 corresponding to the referenced parameter number from the storage circuit 13 and inputs it to the conversion circuit 20. Thereafter, the control circuit 21 operates in the same manner, and refers to the parameter number the number of times (M times) indicated by the number of times of reference 251. Thus, among the plurality of types of conversion parameters 131 stored in the storage circuit 13, M types of use target conversion parameters 131 used by the conversion circuit 20 for coordinate conversion on the target image 110 in the input frame buffer 11 are converted in order Input to the circuit 20.

  In the example of FIGS. 30 and 31, for example, when the reference frequency 251 indicates “3 times”, the reference start position 252 indicates “head”, and the reference interval 253 indicates “1”, the control circuit 21 includes a plurality of LUTs 240. Among the parameter numbers, No. 1, No. 361 and no. 400 is referred to in order. As a result, the converter circuit 20 receives no. 1 corresponding to the rotation parameter 132 for rotating the image by 1 °; No. 3 corresponding to U.S. Pat. A translation parameter 135 corresponding to 400 for translating the image by one pixel in the right direction is sequentially input.

<Operation of image processing circuit>
FIG. 32 is a diagram for explaining the operation of the image processing circuit 4 according to the present modification. When the target image 110 is written in the input frame buffer 11, the control circuit 21 sets the control signal CNT 1 to the first state and reads the conversion parameter 131 used first from the storage circuit 13 to the conversion circuit 20. Output. The conversion circuit 20 performs coordinate conversion on the target image 110 in the input frame buffer 11 based on the input conversion parameter 131 to generate a first intermediate image 160-1. When the conversion circuit 20 generates the first intermediate image 160-1, the control circuit 21 sets the control signal CNT1 to the second state. Thereby, the read buffer 44 of the processing circuit 14 is connected to the conversion circuit 20.

  The conversion circuit 20 writes the pixel data of the first intermediate image 160-1 to the light buffer 43 of the processing circuit 14. Pixel data in the write buffer 43 is written to the frame memory 40. Thereby, the first intermediate image 160-1 is stored in the frame memory 40.

  The processing circuit 14 writes the pixel data of the first intermediate image 160-1 in the frame memory 40 to the read buffer 44. The conversion circuit 20 reads the first intermediate image 160-1 pixel data in the read buffer 44. As a result, the first intermediate image 160-1 is input to the conversion circuit 20 from the processing circuit 14. The conversion circuit 20 performs coordinate conversion on the first intermediate image 160-1 based on the second used conversion parameter 131 read from the storage circuit 13 by the control circuit 21, and the second intermediate image 160-1. An image 160-2 is generated. The conversion circuit 20 writes the pixel data of the second intermediate image 160-2 to the light buffer 43 of the processing circuit 14. Pixel data in the write buffer 43 is written into the frame memory 40. Thereby, the second intermediate image 160-2 is stored in the frame memory 40.

  The processing circuit 14 writes the pixel data of the second intermediate image 160-2 in the frame memory 40 to the read buffer 44. The conversion circuit 20 reads the pixel data of the second intermediate image 160-2 in the read buffer 44. Thereby, the second intermediate image 160-2 is input to the conversion circuit 20 from the processing circuit 14. The conversion circuit 20 performs coordinate conversion on the second intermediate image 160-2 on the basis of the third used conversion parameter 131 read from the storage circuit 13 by the control circuit 21 to obtain the third intermediate image 160-2. An image 160-3 is generated. The conversion circuit 20 writes the pixel data of the third intermediate image 160-3 to the light buffer 43 of the processing circuit 14. Pixel data in the write buffer 43 is written to the frame memory 40. Thereby, the third intermediate image 160-3 is stored in the frame memory 40.

  Thereafter, the image processing circuit 4 operates in the same manner, and the conversion circuit 20 is read from the storage circuit 13 by the control circuit 21 for the (M-1) -th intermediate image 160- (M-1). Coordinate transformation is performed based on the transformation parameter 131 used in the Mth to generate an Mth intermediate image 160-M. The Mth intermediate image 160 -M is written to the frame memory 40 as the processed image 215.

  In the example of FIGS. 30 and 31, for example, when the reference count 251 indicates “3”, the reference start position 252 indicates “first”, and the reference interval 253 indicates “1”, the conversion circuit 20 is informed as described above. The rotation parameter 132 for rotating the image by 1 °, the enlargement parameter 133 for enlarging the image by 1.1 times, and the translation parameter 135 for translating the image by one pixel in the right direction Are entered in order. Therefore, in this case, the conversion circuit 20 performs rotation coordinate conversion for rotating the image by 1 ° with respect to one target image 110, and enlargement coordinate conversion for enlarging the image by 1.1 times. And parallel coordinate transformation for translating the image to the right by one pixel in this order. That is, the conversion circuit 20 rotates one target image 110 by 1 °, enlarges the rotation image obtained thereby by 1.1 times, and enlarges the obtained enlargement image by one pixel in the right direction. By moving in parallel, one processed image 215 is generated.

  The output control unit 45 of the processing circuit 14 outputs the processed image 215 as an output image to the DMA controller 10 as described above. When the storage control unit 41 controls the frame memory 40 to read out the pixel data for four pixels of the processed image 215 from the frame memory 40, the output control unit 45 performs DMA processing on the read pixel data for the four pixels. Output to the controller 10. The DMA controller 10 writes the processed image 215 in the system memory 3. Each time the conversion circuit 20 generates pixel data, the conversion circuit 20 writes the generated pixel data to the write buffer 43, and the write request output unit 412 and the second read request output unit 413 operate independently. Therefore, after the processed image 215 is generated, the processed image 215 is not output to the DMA controller 10, but the process of generating pixel data of the processed image 215 in the conversion circuit 20 and the processed image in the output control unit 45 The process of outputting pixel data 215 is performed in parallel.

  As described above, the coordinate conversion circuit 12 according to the present modification superimposes a plurality of types of coordinate conversion on a single target image 110 without exchanging data with the CPU 2 based on a plurality of types of conversion parameters 131. Do. Therefore, in the same way as described above, each time the coordinate conversion circuit 12 performs coordinate conversion, the processing time of a plurality of types of coordinate conversion is reduced as compared with the case where the conversion parameter 131 used in the coordinate conversion is received from the CPU 2. be able to.

  Further, as described above, in the storage control unit 41, when at least two requests of the first read request, the write request, and the second read request are in conflict, the at least two requests are arbitrated. Therefore, the first read request output unit 411, the write request output unit 412 and the second read request output unit 413 can operate independently of each other. Therefore, it is possible to suppress the processing at a certain output unit among the first read request output unit 411, the write request output unit 412, and the second read request output unit 413 from becoming a bottleneck. As a result, the processing speed of the entire processing circuit 14 can be improved.

  Further, in this modification, the specific information 220 includes a LUT 240 in which a parameter number corresponding to the conversion parameter 131 is described, and reference mode information 250 indicating how to refer to a plurality of parameter numbers described in the LUT 240. It consists of For this reason, the target conversion parameter 131 used in the conversion circuit 20 is specified by the parameter number. Therefore, regardless of how a plurality of types of conversion parameters 131 are stored in the memory circuit 13, the target conversion parameters 131 used in the conversion circuit 20 are freely specified from the plurality of types of conversion parameters 131. be able to.

  For example, as shown in FIG. 5 described above, it is assumed that a plurality of types of rotation parameters 132 are stored in the storage circuit 13. In this case, as shown in FIG. 6 described above, when the specific information 220 includes the number of used parameters 221, the use start position 222, and the use interval 223, the image is rotated by 1 ° according to the specific information 220. It is not possible to specify only the rotation parameter 132 for rotation, the rotation parameter 132 for rotating the image 2 °, the rotation parameter 132 for rotating the image 4 °, and the rotation parameter 132 for rotating the image 7 °. On the other hand, in the present modification, in the LUT 240, the parameter number corresponding to the rotation parameter 132 for rotating the image by 1 °, the parameter number corresponding to the rotation parameter 132 for rotating the image by 2 °, and the image By specifying information 220 by describing parameter numbers corresponding to the rotation parameter 132 for rotating the image by 4 ° and parameter numbers corresponding to the rotation parameter 132 for rotating the image by 7 °. It is possible to specify only

  The specific information 220 includes the LUT 240 and the reference mode information 250, but may include only the LUT 240. In this case, the control circuit 21 refers to a plurality of parameter numbers in the LUT 240 sequentially from the top of the LUT 240, for example. Even in such a case, since the conversion target parameter 131 to be used in the conversion circuit 20 is designated by the parameter number, in the storage circuit 13, a plurality of types of conversion parameters 131 are stored. However, the target conversion parameter 131 used in the conversion circuit 20 can be freely specified from the plurality of types of conversion parameters 131.

  Further, in the image processing circuit 4 according to the present modification, the specific information 220 shown in FIG. 6 described above may be used.

<Third Modification>
The image processing circuit 4 according to the present modification includes, as operation modes, an individual conversion processing mode for performing individual conversion processing on the target image 110, and an overlap conversion processing mode for performing overlap conversion processing on the target image 110. ing. 33 and 34 are diagrams respectively showing configurations of the coordinate conversion circuit 12 and the processing circuit 14 according to the present modification.

  As shown in FIG. 33, the coordinate conversion circuit 12 according to the present modification has a configuration similar to that of the coordinate conversion circuit 12 shown in FIG. 28 described above. In the modification, the register 22 of the control circuit 21 stores operation mode information 260 indicating an operation mode in which the image processing circuit 4 should operate. The operation mode information 260 is set in the register 22 by the CPU 2, for example. When the CPU 2 sets the specific information 220 in the register 22, the CPU 2 also sets the operation mode information in the register 22.

  The control circuit 21 controls the control signal CNT1 based on the operation mode information 260. Further, the control circuit 21 outputs a control signal CNT2 for controlling a selection circuit 46 described later included in the processing circuit 14. The control circuit 21 controls the control signal CNT2 based on the operation mode information 260.

  As shown in FIG. 34, the processing circuit 14 according to the present modification further includes a selection circuit 46 in comparison with the processing circuit 14 shown in FIG. 11 described above. The selection circuit 46 selects one of the output of the conversion circuit 20 and the output of the addition processing unit 42 based on the control signal CNT2, and connects the selected output to the write buffer 43. In the present modification, when the control signal CNT2 is in the first state (for example, a state indicating “1”), the selection circuit 46 selects the output of the addition processing unit 42 and connects it to the write buffer 43. When the selection circuit 46 selects the output of the addition processing unit 42, the pixel data output from the addition processing unit 42 is written in the write buffer 43. On the other hand, when the control signal CNT2 is in the second state (for example, a state indicating “0”), the selection circuit 46 selects the output of the conversion circuit 20 and connects it to the write buffer 43. When the selection circuit 46 selects the output of the conversion circuit 20, the pixel data output from the conversion circuit 20 is written into the write buffer 43.

<Operation mode setting>
When the operation mode information 260 indicates the individual conversion processing mode, the control circuit 21 sets both the control signals CNT1 and CNT2 to the first state. As a result, the operation mode of the image processing circuit 4 according to the present modification is the individual conversion processing mode, and the configuration thereof is equivalent to the configuration of the image processing circuit 4 shown in FIGS. The operation of the image processing circuit 4 in the individual conversion processing mode is the same as the operation of the image processing circuit 4 shown in FIGS.

  On the other hand, when the operation mode information 260 indicates the overlap conversion processing mode, the control circuit 21 sets the control signal CNT2 to the second state. As a result, the operation mode of the image processing circuit 4 according to the present modification is the overlap conversion processing mode, and the configuration is equivalent to the configuration of the image processing circuit 4 according to the second modification shown in FIGS. Become. The operation of the image processing circuit 4 in the overlap conversion processing mode is the same as the operation of the image processing circuit 4 according to the second modification described above.

  After the image processing apparatus 1 including the image processing circuit 4 according to the present modification is activated, the CPU 2 sets the q-th (q is a variable, q ≧ 1) target image 110 to the image processing circuit 4. When performing the above (steps s1, s2, etc. in FIG. 12 described above), the CPU 2 performs the specific information 220 used in the process for the qth target image 110 and the process for the qth target image 110. The operation mode information 260 indicating the operation mode of the image processing circuit 4 is set in the register 22 included in the control circuit 21 of the image processing circuit 4. In the image processing circuit 4, when the specific information 220 and the operation mode information 260 are set in the register 22, the DMA controller 10 reads the qth target image 110 from the system memory 3 and writes it to the input frame buffer 11. In the coordinate conversion circuit 12, the control circuit 21 controls the control signals CNT1 and CNT2 to set the operation mode of the image processing circuit 4 to the operation mode indicated by the operation mode information 260 in the register 22. The image processing circuit 4 performs processing on the qth target image 110 in the input frame buffer 11 according to the operation mode.

  When setting the (q + 1) -th target image 110 with respect to the image processing circuit 4, the CPU 2 similarly performs specification information 220 used in the processing on the (q + 1) -th target image 110. The operation mode information 260 indicating the operation mode of the image processing circuit 4 in the process for the (q + 1) th target image 110 is set in the register 22 of the control circuit 21 of the image processing circuit 4.

  As described above, since the image processing circuit 4 according to the present modification includes the individual conversion processing mode and the overlap conversion processing mode as the operation mode, the individual conversion processing and the overlap conversion processing in one image processing apparatus 1 are performed. Can do both. Alternatively, in two image processing apparatuses 1 having the same configuration, only one individual conversion process can be performed in one image processing apparatus 1 and only an overlap conversion process can be performed in the other image processing apparatus 1.

<Fourth Modification>
FIG. 35 is a diagram showing the configuration of the processing circuit 14 of the image processing circuit 4 according to the present modification. As shown in FIG. 35, the processing circuit 14 according to the present modification does not include the second read request output unit 413 as compared to the processing circuit 14 shown in FIG. 11 described above. Therefore, the arbitration unit 410 arbitrates between the first read request output from the first read request output unit 411 and the write request output from the write request output unit 412 to either the first read request or the write request. Either of these is selected, and processing according to the selected request is performed.

  Further, in the processing circuit 14 according to the present modification, the output control circuit 45 does not output pixel data of the composite image 140 read from the frame memory 40, but pixel data of the composite image 140 in the light buffer 43. Output When the pixel data for four pixels of the composite image 140 is stored in the write buffer 43, the output control circuit 45 reads the pixel data for four pixels from the write buffer 43 and outputs it to the DMA controller 10. In the present modification, when the pixel data of the composite image 140 is stored in the write buffer 43, the write request output unit 412 does not output the write request exceptionally even if the write buffer 43 becomes full.

  As described above, in the processing circuit 14 according to the present modification, when the first read request and the write request are in conflict, the two requests are arbitrated, so the first read request output unit 411 and the write request output unit 412 can operate independently of each other. Therefore, it is possible to suppress the processing at one of the first read request output unit 411 and the write request output unit 412 from becoming a bottleneck. As a result, the processing speed of the entire processing circuit 14 can be improved.

  Further, the output control unit 45 outputs pixel data of the composite image 140 that has not been written in the frame memory 40 to the DMA controller 10. Therefore, compared with the case where the output control unit 45 outputs the pixel data of the composite image 140 read from the frame memory 40, the DMA controller 10 immediately outputs the output target data output from the addition processing unit 42. Can receive. Thus, it is possible to reduce the time until the output target data is generated and then written to the system memory 3.

<Other Modifications>
In the above example, the coordinate conversion circuit 12 performs a plurality of types of coordinate conversion using one target image 110. However, even if only one type of coordinate conversion is performed on one target image 110 good. The coordinate conversion circuit 12 may repeatedly perform one type of coordinate conversion on one target image 110 repeatedly. For example, in the second modification described above, the coordinate conversion circuit 12 performs rotation coordinates of 2 ° with respect to one target image 110 based on the rotation parameter 132 for rotating the image by 2 °. The conversion is performed 90 times. Thus, an image obtained by rotating the target image 110 by 180 ° is obtained as the processed image 215.

  In addition, all or part of the functions of the coordinate conversion circuit 12 may be realized by a processor (such as a CPU or DSP). Also, all or part of the functions of the processing circuit 14 may be realized by a processor.

  The image processing circuit 4 may not include the processing circuit 14. In this case, data output from the coordinate conversion circuit 12 is written to the system memory 3 through the DMA controller 10. The processing circuit 14 may process data other than image data. Further, instead of the addition processing unit 42, a processing unit that performs processing other than addition processing may be provided.

  As described above, although the image processing apparatus 1 and the image processing system 500 have been described in detail, the above description is an exemplification in all aspects, and the present invention is not limited thereto. The various modifications described above can be applied in combination as long as they do not contradict each other. And it is understood that the countless modification which is not illustrated can be assumed without deviating from the scope of the present invention.

12 coordinate conversion circuit 14 processing circuit 40 frame memory 41 storage control unit 42 addition processing unit 43 write buffer 44 read buffer 45 output control unit 100 buffer 410 arbitration unit 411 first read request output unit 412 write request output unit 413 second read request Output unit

Claims (9)

A data processing device,
A storage unit,
A storage control unit that controls the storage unit ;
A processing unit and <br/>
The storage control unit
A processing target reading process of reading processing target data processed by the processing unit from the storage unit;
A writing process of writing processed data, which is the processing target data processed by the processing unit, into the storage unit;
An output target read process for reading out output target data, which is the processed data to be output, stored in the storage unit, from the storage unit, and
The storage control unit
A process target read request output unit that outputs a process target read request that is an execution request for the process target read process;
A write request output unit that outputs a write request that is an execution request for the write process;
An output target read request output unit that outputs an output target read request that is an execution request for the output target read process;
When the processing target read request, the write request, and the output target read request do not conflict, the processing corresponding to the input request among the processing target read request, the write request, and the output target read request is stored in the memory , And when at least two of the processing target read request, the write request, and the output target read request are in conflict, one of the at least two requests is arbitrated by arbitrating the at least two requests. An arbitration unit for performing processing according to the selected and selected request in the storage unit;
The storage unit is a 1-port memory,
The storage control unit, when executing the write process of writing the processed data to the storage unit, in the storage area where the processing target data used in the generation of the processed data is read, A data processing device that writes processed data. A data processing device,
A storage unit,
A storage control unit for controlling the storage unit ;
A processing unit and <br/>
The storage control unit
A processing target reading process of reading processing target data processed by the processing unit from the storage unit;
A write process for writing processed data, which is the processing target data processed by the processing unit, into the storage unit;
The storage control unit
A process target read request output unit that outputs a process target read request that is an execution request for the process target read process;
A write request output unit that outputs a write request that is an execution request for the write process;
When the processing target read request and the write request do not conflict, the processing target read request and the write request are processed in the storage unit according to the input request, and the processing target read request and the write request are performed. In the case where the requests conflict with each other, the processing target read request and the write request are arbitrated to select one of the processing target read request and the write request, and the processing according to the selected request is performed to the storage unit Have a department,
An output control unit for controlling output of the processed data output from the processing unit to the outside of the data processing device;
The output control unit outputs the processed data to be output, which is not written in the storage unit, to the outside of the data processing unit,
The storage unit is a 1-port memory,
The storage control unit, when executing the write process of writing the processed data to the storage unit, in the storage area where the processing target data used in the generation of the processed data is read, A data processing device that writes processed data. The data processing apparatus according to claim 1, wherein
A processing priority is assigned to the process target read request, the write request, and the output target read request,
When the at least two requests of the process target read request, the write request, and the output target read request conflict, the arbitration unit determines the at least two requests based on the priorities of the at least two requests. A data processing device that arbitrates. The data processing apparatus according to claim 2, wherein
A processing priority is assigned to the processing target read request and the write request,
The arbitration unit arbitrates the processing target read request and the write request based on the priority of the processing target read request and the write request when the processing target read request and the write request conflict with each other. , Data processing unit. A data processing apparatus according to any one of claims 3 and 4.
The data processing device, wherein the priority is changeable. The data processing apparatus according to any one of claims 1 to 5, wherein
The storage control unit
The processing target read process for reading out the first processed data, which is the processed data written in the storage unit in the writing process, as the processing target data;
A data processing apparatus, which executes the writing process of writing second processed data obtained by the processing unit processing the first processed data in the storage unit. A data processing apparatus according to any one of claims 1 to 6, wherein
It further comprises a first buffer for temporarily storing the processing target data read from the storage unit,
The processing unit reads the processing target data from the first buffer, and processes the read processing target data.
In the first buffer, the processing target data read by the processing unit is erased;
The processing target read request output unit outputs the processing target read request when the first buffer is empty. The data processing apparatus according to any one of claims 1 to 7, wherein
It further comprises a second buffer for temporarily storing the processed data output from the processing unit,
The storage control unit reads the processed data stored in the second buffer in the write process, and writes the read processed data to the storage unit.
In the second buffer, the processed data written to the storage unit is erased;
The data processing apparatus, wherein the write request output unit outputs the write request when the second buffer is full. The data processing apparatus according to claim 1, wherein
The output target data output from the data processing device is temporarily stored in a third buffer outside the data processing device,
In the third buffer, the read output target data is erased;
The output target read request output unit outputs the output target read request based on an empty state of the third buffer.

Images