JPH08272981A - Image processor - Google Patents

Abstract

PURPOSE: To process a delay execution system image processing pipeline by using a memory efficiently on the whole by performing the image processing immediately by a process execution control means when an image processing means stores the whole image in the memory temporarily. CONSTITUTION: The image processor 11 consists of a process storage means 13 which stores image process instructions given from a host application 16, the image processing means 14 which processes an image stored in an image storage means 12 according to the image process instructions stored in the storage means 13, and the control means 15 which controls the execution of this image process. And, the control means 15 consists of a detecting means 21 which detects whether or not a given image process instruction requires a page buffer, a memory securing means 22 which secures the page buffer by using a system call of an OS when necessary, a process start means 234 which starts the actual image process by the processing means 14, and a postprocess means 14 which releases used resources that become unnecessary after the image process ends.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing system in which a plurality of primitive image processing modules are prepared to provide various image processing functions by combining them. These image processing functions are used in various systems that handle digital images, such as a document editor, a drawing tool, an image transmission device and a printer. For example, when performing image processing on an operating system such as UNIX (registered trademark) or DOS,
A system call is used to request free memory from the system in order to secure the memory area necessary for processing. When the memory area is used up, the used memory area is released so that a later process or another process can use the area. To save memory resources, it is important to release unnecessary memory areas to the system as soon as possible.

Such an image processing function is often provided as an image processing library file, and in that case, the function is used by linking with an application program which wants to use the image processing function. INDUSTRIAL APPLICABILITY The present invention can be used for effective use of memory without impairing the advantages of the conventional method, particularly when performing image processing on a system that requires or releases such memory.

[0003]

2. Description of the Related Art In a conventional image processing system, a plurality of primitive image processing modules are disclosed in, for example, Document D. S.
Dyer, "A Dataflow Toolkit
for Visualization ”, IEEE Co
mputer Graphics and Appli
situations 10 (4), July 1990, p.
p. 62 ln. As described in 3-44, pipelined based on a dataflow model such that the output of one image processing module becomes the input of the next image processing module, or: Document MIT X Conso.
rium, “X Image Extension-
Protocol Reference Manual
l ”, Public Review Draft, De
Cember 23, 1992, Version 4.
12, pp. As described in 5-1 and 5-2,
For example, there is a method in which a module group configured to form a graph structure such as a directed closed circuit graph (DAG) and configured as a pipeline or a graph structure is executed as one image processing module. Such an image processing method will be called a pipeline method. Further, the configured module group will be referred to as a pipeline, and the primitive image processing as a structural unit will be referred to as a job.

In such an image processing system, if sufficient primitive modules are prepared, it is possible to develop necessary functions without individually developing a number of highly specialized modules. There is an advantage that a general-purpose image processing function can be provided without increasing the cost.

However, in the pipeline system, for example, the document "Solaris XIL 1.0 Im" is used.
aging Libraray Program
r's Guide ”, February 199
3, pp. As described in H.297, when processing one image data using a plurality of image processing modules, each job is executed sequentially,
In order to hold the intermediate result, it is necessary to temporarily hold the entire image data in the memory. If we call a memory for holding such whole image data a page buffer, the number of accesses to the page buffer will increase in proportion to the number of jobs that make up the pipeline, and a large amount of memory space is usually required. Therefore, a large amount of access to the page buffer consumes a large amount of CPU power and memory and significantly reduces the execution speed.

Since there is no guarantee that the output result will be used after image processing is performed in this way, for example, when an error occurs in a job newly added to the pipeline, or a job using the output result. However, there is a problem in that all the computations that have been performed by consuming a large amount of computer resources until then will be wasted.

In order to solve such a problem, for example, the document MIT X Consortium, "X Im.
age Extension-Protocol Re
ference Manual ", Public Re
view draft, December 23, 19
92, Version 4.12, pp. 5-3 and the document "Solaris XIL 1.0 Imaging"
LibrayProgrammer's Gui
de ", February 1993, pp. 298-
As described in U.S.A. 299, no actual image processing is performed at the stage of constructing a pipeline or a graph structure, and only relatively inexpensive processing such as storing image processing instructions or initializing modules is performed. ,
There is a method in which the actual processing is performed after the designated stage or the required stage. Such a pipeline method will be called a delayed execution method.

In the delayed execution method, image processing performed in each job and access to an intermediate buffer such as a page buffer are performed only when instructed or necessary, so that a large number of computers are required. A simple error such as a parameter defect or a memory shortage can be detected before the resource-consuming image processing is started, and the waste is reduced. Further, as described in JP-A-5-207266, the execution order of the image processing jobs stored before the image processing is started is changed, or the image processing sequence in the pipeline is detected. In this case, by changing the image processing method such as replacing the job with a more optimized job, there is an advantage that processing can be performed at higher speed and a higher quality output result can be obtained.

[0009]

However, in the delayed execution method, when a plurality of jobs holding the internal page buffer are connected to the pipeline, initialization is performed at the time of connection to secure the page buffer. The problem is that a large number of page buffers that are not used at the same time are secured, which consumes an extremely large amount of memory compared to the method in which execution of one job is completed and execution of the next job is started. Have. Alternatively,
Although the problem of consuming a lot of memory if the page buffer is secured at runtime without being initialized at the time of connection and released each time the page buffer is used up, error detection can be done early. The advantage of the delayed execution method that can be performed is lost. Examples of jobs that hold such an internal page buffer include coordinate conversion processing such as rotation and affine transformation, and image analysis processing such as Fourier transformation.

Even if a large number of unnecessary page buffers are secured, a large-capacity memory must be prepared as a system resource for normal operation, which leads to an increase in system cost. In addition, in the method of allocating a page buffer at the time of execution, if an attempt to allocate a page buffer fails after consuming a large amount of computer resources and performing a lot of image processing, the processing cannot be continued until then. All calculations are wasted.

FIG. 20 shows an example of image processing according to the prior art. If the image processing apparatus is given an image processing command and an image to be processed by the host application and outputs the image of the processing result, the host application first gives the image processing command to the image processing apparatus, and this is stored in the processing storage means. To be done. Taking the image processing apparatus of the delayed execution system described above as an example, for example, an image processing command from the host application is stored in the processing storage means as an image processing job in step 81. Further, when the next image processing command is given, it is added to the processing storage means as an image processing job to be executed next. Similarly, these steps are repeated until the addition of the image processing job is completed (step 84), and after all the image processing jobs are added to the processing storage means, the image processing is executed in step 85. Resources such as buffer memory for storing intermediate results may be required during processing, so if resources are secured for processing in this way,
In step 86, resources that are no longer needed after the processing is released are released.

At this time, for example, by initializing the internal state of the image processing means and securing resources such as a memory necessary for processing (step 82), an error due to a parameter defect or a memory shortage can be detected early. can do. Normally, digital image processing requires time because it requires access to a large amount of data, and if an error due to initialization failure is detected after sequentially executing a plurality of image processing jobs (step 83), a large amount of calculation is performed until then. The processing that consumes resources and executes is wasted.

On the other hand, FIG. 21 shows an example of an image processing apparatus according to the prior art which also has a processing execution control means. Similarly, the image processing job is stored in the processing storage means according to an instruction from the host application (step 91).
At this time, if the host application gives an instruction to execute the process immediately (step 92) or if the given image processing job is a job that outputs a value such as a histogram (step 93), the addition of the image processing job is completed. Even before the execution, the image processing job given up to that point is executed (step 94). The instruction for immediate execution of the process may be defined, for example, by providing a procedure interface for instructing the addition of a job with a flag indicating whether or not the immediate execution is performed, and if this flag is set, the immediate execution is performed. Further, in order to determine whether or not the job is a job that outputs a value, for example, is it a job that stores all job types of the image processing unit in advance in a table and outputs a value for each item of the job type? You just need to prepare a flag that indicates whether or not it is correct. In this case, for example, each time an image processing job is added, it is determined from the job type whether the job outputs a value by referring to the table.

However, when the page buffer is secured in the initialization of the image processing job that requires the page buffer for 90 ° rotation or the mirror image in the vertical direction to internally hold the input image, for example. If there are a plurality of jobs that secure such a page buffer in the image processing jobs held in the processing storage means, each of them has a page buffer, and a very large amount of memory resources are required at the same time. Moreover, for example, when a job having each page buffer does not output image data until the input page buffer is filled, the third page is filled until the second page buffer is filled with the processing result data. Since no data is sent to the buffer, it is useless to secure the third and subsequent page buffers from the beginning.

The present invention has been made in view of the above problems of the conventional system. An image processing pipeline of a delayed execution system composed of a plurality of image processing jobs is provided while solving these problems as much as possible. It is an object of the present invention to provide a method of efficiently performing processing using a memory.

[0016]

In the first invention,
For the conventional delayed execution type image processing pipeline,
A page buffer detection means having a delayed execution control unit for controlling delayed execution, and detecting whether a job to which the delayed execution control unit is connected requires a page buffer and determining whether to execute the delayed execution. And the page buffer securing means that actually secures the page buffer when it is determined not to execute the delayed execution, and the image processing start that executes the image processing connected to the preceding stage of the pipeline after securing the page buffer. It is different in that it includes a means and a post-processing means for releasing resources reserved for use in the image processing of the first stage after the image processing of the first stage is completed.

In the image processing system of the present invention, when a job holding a page buffer is connected to the pipeline, the image processing of the preceding stage is performed immediately after the page buffer is secured. The result is stored in the reserved page buffer, and various computer resources such as a memory and a file reserved for use in the preceding job can be released because the processing is completed.

Next, when a job that holds the page buffer once again is connected to the pipeline, image processing is performed immediately after securing the page buffer, and the result is stored in a new page buffer. . At this time, the page buffer previously secured belongs to the preceding job of this job and can be released because all the processing has been completed.

When the pipeline has a DAG structure and is accompanied by branching, it may be impossible to release all the computer resources of the preceding job even if the image processing is completed in the job holding the page buffer. For example, this is a case where there is a job that branches before the job that holds the page buffer and the branch job itself holds the page buffer. In such a case, if the resources of the branch job are released when the image processing of the job holding this page buffer is completed, it is necessary when the job connected to the other branch destination requests an image from the branch job. Resource, in this case the page buffer, has been released and cannot continue.

Therefore, in such a case, when the job holding the page buffer transmits a resource release request to the preceding stage, the branch job stores the fact that the processing on the branch path is completed and branches. Resources are not released in the job. However, since the processing should have been completed, the resource release request is transmitted to the preceding stage of the branch job. For this reason, the image processing means forming the branch job has a processing end confirmation means corresponding to each branch path. Alternatively, the end of all jobs may be determined by simply providing a counter that stores the number of branch paths and counts the number of paths that have completed processing or have not yet completed processing, and compare with this counter. Alternatively, a process monitoring unit for monitoring the progress of the process may be separately provided, and the process monitoring unit may have a process end confirmation unit.

When it is confirmed that the processing has been completed for all branch paths from the branch job, resources in the branch job including the page buffer held by the branch job are released.

When a job holding a page buffer is connected to the pipeline in this way, image processing is performed immediately after the page buffer is reserved, and the computer resources reserved for use in the job in the preceding stage. By releasing the above, even if the delayed execution method is used when other jobs are connected, only at most two page buffers are simultaneously held in the entire pipeline.

When the pipeline has a DAG structure and is accompanied by a branch, when the job holding the page buffer sends a resource release request to the preceding stage, the branch job finishes the processing on the branch path. When it is confirmed that processing has been completed for all branch paths from the branch job, other resources are released by releasing the resources in the branch job including the page buffer held by the branch job. Even if the delayed execution method is used when connecting, the entire pipeline simultaneously secures the minimum required page buffer. Therefore, in contrast to the delay execution method that used a large amount of memory resources by securing a wasteful page buffer that was not used at the same time, this method does not impair the advantage of the delay execution method that early detection of errors is possible. Since it consumes only a minimum amount of memory, it can be expected to save memory resources. This leads to cost reduction of the entire system.

In the second aspect of the invention, a memory allocation control unit for controlling the allocation of the page buffer is provided for the image processing pipeline of the conventional delay execution system, and the memory allocation control unit is connected. Detecting means for detecting whether or not the job requires a page buffer, memory securing means for actually securing the page buffer at the time of initialization or execution of the image processing job, and the image of the preceding stage after the image processing of the preceding stage is completed. Includes post-processing means for releasing resources reserved for use in processing.

In the image processing method of the present invention, when a job that holds a page buffer is connected to the pipeline, the page buffer is not always secured immediately. The page buffer is secured by the memory securing unit at the time of normal execution, but may be secured at the time of initialization depending on the image processing job connected until then. Further, when the image processing of the preceding stage of a certain image processing job is completed, various computer resources such as memory and files secured for use in the preceding job are released since the processing is completed. Alternatively, in the other image processing method of the present invention, the page buffer is normally reserved at the time of initialization, but it is not always reserved separately for each image processing job, and a sharable page buffer is shared. Saves overall memory usage.

When the image processing pipeline has a simple list structure, the page buffer is reserved at the time of initialization when the number of already reserved page buffers is one or less. The image processing job executed after the second side is secured cannot be executed until all the image processing jobs using the first side are completed. Therefore, if the page buffer memory used is released after the completion of all image processing jobs that use the first side when the image processing job is executed, not at initialization, and then the buffers for the third side and thereafter are secured, At the same time, it is possible to save the page buffers that normally consume a large amount of memory resources without securing page buffers for three or more planes.

Further, when the image processing job using the page buffer of the third side is executed, the page buffer of the first side is unnecessary, and the page buffer of the second side is executed when the image processing job using the page buffer of the fourth side is executed. As such, the even-numbered jobs and the odd-numbered jobs that use the page buffers do not use the page buffers at the same time. That is, even-numbered jobs and odd-numbered jobs that use the page buffer may use the same page buffer. Therefore, even if two page buffers are prepared and used alternately, the page buffers can be saved without securing three or more page buffers at the same time. In this case, it is necessary to allocate a page buffer of a size required by the job that uses the most memory among the image processing jobs that use the page buffers. Therefore, each time the page buffer is allocated, it is confirmed whether the size of the page buffer is sufficient, and if it is insufficient, the memory of the required size is reallocated. The image processing job to which the relevant page buffer has been allocated up to that point uses less memory than the page buffer capacity after re-allocation, so there is no problem in operation. Furthermore, since no more page buffer memory is used at the same time in the entire image processing pipeline, no error due to page buffer memory allocation failure occurs at the time of execution. In other words, the memory shortage error can be detected at the time of initialization, which leads to early detection of the error.

When the pipeline has a DAG structure and is accompanied by branching, it may not be possible to release all the computer resources of the preceding job even if the image processing is completed in the job holding the page buffer. For example, there is a job that branches before the job that holds the page buffer, and this branch job itself holds the page buffer. In such a case, if the resources of the branch job are released when the image processing of the job holding this page buffer is completed, it is necessary when the job connected to another branch destination requests an image from the branch job. Resource, in this case the page buffer, has been released and cannot continue. Therefore, in such a case, when the job holding the page buffer sends a resource release request to the preceding stage, the branch job stores the fact that the processing on the branch path is completed, and the resource in the branch job is stored. Will not be released. However, since the processing should have been completed, the resource release request is transmitted to the preceding stage of the branch job. For this reason, the image processing means forming the branch job has a processing end confirmation means corresponding to each branch path. Alternatively, it is possible to determine the end of all jobs by simply providing a counter that stores the number of branch paths and counts the number of paths for which processing has been completed or has not yet completed, and to compare with this. Alternatively, a processing monitoring unit for monitoring the progress of processing is separately provided,
The process monitoring unit may have a process end confirmation unit. When it is confirmed that the processing has been completed for all branch paths from the branch job, the resources including the page buffer held by the branch job are released.

When a job using a page buffer is connected to a pipeline in this way, an image processing pipe forming a particularly simple list structure is controlled by appropriately controlling allocation and release of the page buffer. The line holds at most two page buffers at the same time.

Further, by alternately using the page buffers and sharing them among a plurality of image processing jobs, all the page buffers can be allocated at the time of initialization, and error detection due to insufficient memory can be performed early. Further, when the image processing pipeline has a DAG structure and is accompanied by a branch, when the job holding the page buffer sends a request for releasing the resource to the preceding stage, the branch job finishes the processing on the branch path. By memorizing that, when it is confirmed that the processing has been completed for all the branch paths from the branch job, by releasing the resources in the branch job including the page buffer held by the branch job,
Even if the delayed execution method is used when connecting other jobs, the minimum required page buffer is secured at the same time in the entire pipeline. Therefore, the conventional method consumes only the minimum required memory, whereas the conventional method uses a large amount of memory resources by securing a wasteful page buffer that is not used at the same time. Since this advantage can be utilized without impairing the advantage of the delayed execution format that early detection of errors is possible, not only memory resources but also CP
It can be expected to save U resources. These lead to cost reduction of the entire system.

[0031]

【Example】

[Embodiment 1] The present invention operates on a hardware configuration as shown in FIG. 1, for example. For example, image data is input from the scanner 308 or the like and stored in the information storage memory 304. For example, the operator gives an image processing command through the input device 305 such as a keyboard or a mouse while referring to the image data displayed on the display device 307. The given image processing command is stored in the information storage memory 304,
The image processing apparatus 303 performs processing in accordance with the stored image processing command and outputs it to the printer 309 or the like. Image processing instructions and input images provided by a page description language such as PostScript (registered trademark) are
The image is processed and printed out by the image processing apparatus 303. Reference numeral 301 denotes a computer such as a personal computer or a workstation.

Further, the present invention operates on an image processing system of a scanner or a printer as shown in FIGS. 2 and 3, for example.

FIG. 2 is a block diagram for explaining the detailed arrangement of the scanner. The optical system driving circuit 36 moves the original 30 or the lens 31 and the CCD sensor 32 in the direction perpendicular to the reading direction of the CCD sensor 32, and the original 3
The entire 0 image is read. The image read by the CCD sensor 32 is analog-digital converted by the A / D converter 33 and temporarily stored in the buffer memory 34.
The image data stored in the buffer memory 34 is synchronized and encoded via the image data transmission circuit 35 and transmitted to the image data reception circuit 40. The control circuit 17 is
Controls the entire scanner. Reference numeral 38 is an interface circuit.

FIG. 3 is a block diagram for explaining the detailed arrangement of the printer. The image data transmitted from the image data transmitting circuit 25 is received and converted by the image data receiving circuit 40, and synchronized by the synchronizing circuit 47. A laser driver 41 and a laser oscillator 42 output laser light. Further, the synchronizing circuit 47 controls the motor driving circuit 48 to start and stop the motor 46 that rotates the polygon mirror 43. Further, the laser oscillator 42 and the polygon mirror 43 are synchronized with the laser light detected by the beam detector 45 by the synchronizing circuit 47 to form a latent image on the photoconductor 44. In this embodiment, it is assumed that an electrophotographic system is used as the image forming means. An image forming method that combines a laser and an electrophotographic method is already known, and therefore a detailed description thereof is omitted here. The synchronization circuit 47 is controlled by a control circuit 49 which controls the entire printer. The control circuit 49 further controls the paper feed system driving circuit 50 and also the interface circuit 38.
A series of control is performed in response to the command of.

4 and 5 show examples of the image processing pipeline. In the example of FIG. 4, the image processing job 61 reads an image file on the external storage device 306 such as a hard disk. The output is rotated by the image processing job 62, sharpened by the image processing job 63, and finally written into the image file by the image processing job 64. In the example of FIG. 5, the image processing pipeline does not have a simple list structure but a DAG structure, that is, a directed acyclic graph structure. Similarly, after the image processing job 71 reads the image file and the image processing job 72 performs color adjustment, the output is branched into two,
One of them is written into an image file by the image processing job 73, and the other one is subjected to resolution conversion by the image processing job 74 and then CRT by the image processing job 75.
Is output to the display device 307.

The image processing apparatus 11 according to the present invention, as shown in FIG. 6, has a processing storage means 13 for storing an image processing command given from a host application 16, an image storage means 12 for storing image data, and an image storage. Image processing means 14 for performing image processing on the image stored in the means 12 in accordance with the image processing command stored in the processing storage means 13.
And a control means 15 for controlling the execution of this image processing. In addition, the control means 15, as shown in FIG.
Detecting means 21 for detecting whether or not a given image processing instruction requires a page buffer, memory securing means 22 for securing a page buffer using an OS system call or the like as necessary, and an image The processing unit 14 includes a process starting unit 23 that starts actual image processing, and a post-processing unit 14 that releases used resources that have become unnecessary after the image processing is completed.

FIG. 8 shows an embodiment of the image processing job according to the present invention. FIG. 8 illustrates the flow of a portion 91 for adding the image processing job of FIG. 21, which is an example of the conventional technique. The remaining part is similar to that of FIG.

First, when an image processing job is given, the detecting means 21 determines whether the image processing job requires a page buffer. For example, all the job types of the image processing means are stored in a table in advance, and a flag indicating whether or not the job requires a page buffer is provided for each item of the job type. It is sufficient to refer to the table and determine whether the job requires a page buffer. If the page buffer is not needed, the step of adding the image processing job is finished as it is (step 1).
01). When the page buffer is required, the memory securing unit 22 secures the required page buffer (Step 102), and the process activation unit 23 executes the image processing job that has been added by then (Step 10).
3) The result is saved in the secured page buffer (step 104). If the results are saved in the page buffer, the post-processing means 24 has been reserved for use by these image processing jobs, since the image processing jobs that have been added up to that point have been completed at this point. Released resources (step 105).

At this time, for example, even if the image processing job added until then holds the page buffer, the resources are released when the saving to the new page buffer is completed, and the page buffer is reserved first. Page buffer is released. Similarly, even if a job requiring a page buffer is added after this, a new page buffer is secured at that point and the image processing up to that point is executed immediately, so the page buffer secured by the current job is It becomes unnecessary and is released.

[Embodiment 2] This embodiment describes the post-processing means 24 in the present invention. The image processing job provided by the host application 16 and stored in the processing storage unit 13 releases resources such as a file handler and a memory used in the job to the system by the post-processing unit 24 when the actual image processing is completed. . When the image processing job has a simple list structure, there is no branching of the processing such that the output of a certain image processing job becomes the input of a plurality of image processing jobs. The resources used for the sequential processing may be released according to the post-processing request. FIG. 9 is an example showing the flow of such post-processing. For example, the control means 15
Judges whether or not another image processing job exists before the image processing job currently being processed (step 121), and if there is another image processing job, requests the previous processing for post processing (step 122). Finally, in either case, the used resources used by the image processing job which is currently processing are released (step 123).

However, when the image processing job stored in the processing storage means 13 has a DAG structure and processing is branched, even if a post-processing request is received from one of the branching paths, the other branching path is processed. Since the processing is not always completed, the post-processing cannot be performed immediately. Figure 10
Is a job 112 with such a branch in the present invention.
Is an example of post-processing when is stored in the processing storage unit 13. The direction of the solid line arrow indicates the flow of image data. The branch job 112 has a flag 116 corresponding to each of the branch paths 113 to 115. FIG.
It is an example of the flow of the post-processing of such a branched job in the present invention. When the post-processing request is transmitted from a certain branch path, the post-processing is not immediately started in the branch job 112, but a flag corresponding to the branch path for which the post-processing request is made is set (step 131). Next, it is checked whether all the flags are set (step 132), and if all the flags are set, the normal post-processing shown in FIG. 9 is performed (step 133). If all the flags are not set, the process is terminated as it is and waits for the next post-processing request.

If post-processing requests are issued from all the branch paths 113 to 115, it can be seen that none of the branch paths requires any more input image data. Processing can be performed.

[Third Embodiment] As another embodiment of the branch job, there is also a method of providing a post-processing request counter instead of having a flag corresponding to each branch path. The counter is initialized to 0. When a branch job receives a post-processing request from a branch path, the counter is incremented, and if the counter reaches the total number of branch paths, the post-processing is performed. If the counter has not reached the total number of branch paths, the processing is terminated and waits for the next post-processing request.

If the counter has reached the total number of branch paths, it means that post-processing requests have been issued from all the branch paths, and it is known that none of the branch paths requires any more input image data. Jobs and jobs preceding them can be post-processed.

[Fourth Embodiment] Next, a fourth embodiment of the present invention will be described. The parts corresponding to those of the above-described embodiments are designated by the same reference numerals, and the duplicated description will be omitted.

Image processing apparatus 11 in the fourth embodiment
Is a processing storage means 13 for storing an image processing command given from the host application 16 as shown in FIG.
An image storage means 12 for storing image data, an image processing means 14 for performing image processing on an image stored in the image storage means 12 in accordance with an image processing command stored in the processing storage means, and for executing this image processing. It comprises a memory allocation control means 16 for controlling allocation of required memory. In addition, the memory allocation control means 16 is configured as shown in FIG.
As shown in FIG. 3, a detecting means 21 for detecting whether or not a given image processing instruction requires a page buffer, and if necessary, a page buffer is secured by using, for example, an OS system call or the like. Memory securing means 22
After the image processing, the post-processing unit 24 releases the used resources that are no longer needed.

FIG. 14 shows an embodiment of the image processing apparatus according to the present invention. FIG. 14 shows an embodiment of the prior art shown in FIG.
The flow of step 82 for initializing an image processing job of 0 is described.

First, when an image processing job is given, the detecting means 21 determines whether the image processing job requires a page buffer (step 181). For example, all the job types of the image processing means are stored in a table in advance, and a flag indicating whether or not the job requires a page buffer is provided for each item of the job type. It is sufficient to refer to the table and determine whether the job requires a page buffer. If the page buffer is not required, the initialization step relating to the page buffer is omitted and other initialization is performed (step 184). If the page buffer is required, it is inspected whether the already reserved page buffer is one side or less (step 18).
2) Memory securing means 22 only when the number of pages is one or less
Secure a page buffer of the required capacity (Step 1
83). At step 184, initialization other than initialization related to the page buffer is performed. For example, since it is not necessary to secure a page buffer in an image processing job such as color space conversion, if processing is performed in scan line units and the result is passed to the job of the next stage, the buffer memory for one line is used. Should be secured.

The present invention is particularly effective when it is unavoidable that a plurality of page buffers are required in such an image processing system that normally allocates a buffer memory for one line. Alternatively, in a process such as a convolution filter using a 3 × 3 kernel, data of one line before and after the target pixel is also required to obtain a processing result of a certain scan line, and therefore a band buffer for three lines is used. Secure. Alternatively, in a job in which an output pixel value is determined by referring to a table for speeding up processing such as application of a tone reproduction curve, a memory for a lookup table is secured and the table is initialized.

Consider an example in which this is applied to an image processing instruction sequence as shown in FIG. Here, it is assumed that the image storage instructions are stored in the processing storage means 13 in the order of the mirror image in the main scanning direction, the color adjustment, the 90 degree rotation, and the Fourier transform. Among these image processing commands, the mirror image, the 90-degree rotation, and the Fourier transform each require a page buffer for temporarily storing the entire surface of the input image due to the nature of the processing. First, when the image processing job 1101 that executes a mirror image is initialized, it is checked in step 181 whether the detection unit 21 needs a page buffer, and since the page buffer is required, the process proceeds to step 182. In step 182, the number of page buffers that have been secured so far is inspected, and since no surface has been secured yet, the process proceeds to step 183. The number of page buffers secured so far can be easily checked by preparing a counter, for example. In step 183, the memory securing unit 22 allocates a page buffer 1109 having a sufficient size to hold the input image to the image processing job 1101. Next, when the image processing job 1102 that executes color adjustment is initialized, it is determined that the page buffer is not used in step 181, and the step 183 of allocating the page buffer is not executed. Other initializations such as initialization of the up table are performed. Next, when the image processing job 1103 that rotates 90 degrees is initialized, it is determined in step 181 that a page buffer is necessary and step 182.
Going to step 182, since only one page buffer 1109 has been secured so far in step 182, the process proceeds to step 183 and the page buffer 1110 is secured in step 183. Next, when the image processing job 1104 for performing the Fourier transform is initialized, step 1
At 81, it is determined that a page buffer is necessary, and the process proceeds to step 182. At step 182, since the two page buffers 1109 and 1110 have already been reserved, the process proceeds to step 184 and the image processing job 1
The page buffer 1111 for 104 is not secured at initialization.

FIG. 16 illustrates a flow when image processing is executed after each image processing job corresponding to the image processing command stored in the processing storage means 13 of the image processing apparatus according to the present invention is initialized. Is. First, in step 1001, it is determined whether the job uses the page buffer. If it is used, the process proceeds to step 1002, and if it is not used, the process proceeds to step 1008. Step 10
In 02, it is determined whether a page buffer is reserved for this job at the time of initialization. Like the job 1101 and the job 1103 in the previous example in FIG.
If the page buffer is secured at the time of initialization, the process proceeds to step 1008, and if the page buffer is not secured as in job 1104, step 100 is performed.
Go to 3. In step 1003, a necessary page buffer is secured. If the page buffer cannot be secured, error processing is performed in step 1004 and the processing is interrupted. If not, the process proceeds to step 1005 to perform the image processing up to the preceding stage of the image processing job,
In step 1006, the processing result is saved in the page buffer secured in step 1003. At this point, all the image processing up to the previous stage has been completed, so in step 1007, the resources used in the image processing jobs up to the previous stage are released.
In step 1008, image processing is performed using the input image data stored in the page buffer in step 1006, and it is determined in step 1009 whether processed data of all scan lines has been output. If all the data has been output, the process proceeds to step 1010 to release the used resources. This may also include the page buffer used in step 1006. Otherwise, the resource is not released and the processing is terminated to prepare for the next output request.

Steps 1106, 1107 and 1 in FIG.
Reference numeral 108 indicates the usage status of the page buffer during execution.
At the end of initialization, as in step 1105, the page buffer 1109 is assigned to the image processing job 1101 that uses the page buffer first, and the page buffer 1110 is assigned to the job 1103 that uses the page buffer second. When the job 1101 is executed, the page buffer 110
The input data is read in 9 and the result is job 110
Passed to 2. Image processing job 1102 for color adjustment
Is a job that does not perform coordinate conversion or reference to neighboring pixels, does not require a page buffer and outputs the processing result to the next job 11.
Can be passed to 03. Next, in the image processing job 1103 that performs 90-degree rotation using the page buffer,
Since the last scan line of the input image is necessary to output the scan line of the input image, the entire image data is read into the page buffer 1110 (step 1106). At this point, the page buffer allocated for the job 1101 preceding the job 1103. Job 1109 is released because it is no longer needed, and the page buffer is needed next.
A new page buffer 1111 is allocated for 04 (step 1107). Since the job 1104 that outputs the Fourier transform image also needs the input image in the page buffer, the output of the job 1103 is read into the allocated page buffer 1111 (step 110).
8). As described above, in the image processing apparatus of the present invention, only two page buffers are secured at each step (1105 to 1108) of image processing, and the memory used can be saved.

[Embodiment 5] This embodiment describes another embodiment of page buffer allocation in the present invention. FIG. 17 is a flow chart at the time of initialization in the present invention, which shows the details of the step 82 in FIG. 20 showing the image processing embodiment according to the prior art similar to FIG. First step 1
At 21, it is checked whether the image processing job provided by the detection means 21 uses a page buffer. Step 12 if a page buffer is needed
2; otherwise, proceed to step 127.
In step 122, it is determined whether the image processing job is an odd-numbered or even-numbered image processing job using the page buffer. In this case, for example, a counter may be prepared in the memory allocation control means 16 and the counter may be incremented every time a job using the page buffer is initialized, and it may be determined whether the counter value is an odd number or an even number. If it is odd, the first page buffer is allocated in step 123, and if it is even, the second page buffer is allocated in step 124. In step 125, it is checked whether the size of the page buffer allocated in step 123 or step 124 is sufficient for the image processing job. If it is insufficient, the process proceeds to step 126 to reallocate the required page buffer. For this, for example, the ANSI C standard library function realloc () or the like can be used. When reallocating, the size of the allocated region may be changed or a different region may be allocated while the region remains the same, but if the page buffer area after allocation is different from the previous page buffer area, The previous page buffer area is released. If the page buffer capacity checked in step 124 is sufficient, step 127
Proceed to. In step 127, initialization other than initialization relating to the page buffer is performed, such as securing a line buffer and initializing internal variables.

FIG. 18 shows an example of the image processing pipeline initialized in this way. Here, each job of the mirror image 141, the color adjustment 142, the 90-degree rotation 143, and the histogram normalization 144 is connected in series as in the example shown in FIG. Among these, three image processing jobs that use the page buffer are a mirror image 141, a 90-degree rotation 143, and a histogram normalization 144. When initializing the entire image processing pipeline, first, two empty page buffer areas are reserved, and the first page buffer 1
25 and the second page buffer 126. First, when the image processing job 141 for obtaining a mirror image is initialized, it is determined in step 121 that a page buffer is required, and the process proceeds to step 122. In step 132, since this job is the first job using the page buffer, it is found that it is an odd-numbered job, and the process proceeds to step 123. In step 123, the first page buffer which is secured in advance with the size 0 is allocated. However, the size of the page buffer required is 0
Since it is larger, step 126 reallocates the required size. If the image processing job 142 for color adjustment is then initialized, it is checked in step 121 that the page buffer is not required, and step 127 is executed.
Proceed to and only the initialization except for the page buffer is performed. Next, an image processing job 143 for rotating 90 degrees
Is initialized, the page buffer is allocated similarly to the initialization of the image processing job 141, but since it is the second job to use the page buffer, it is determined to be an even number in step 122 and is determined in step 124. The difference is that a second page buffer is allocated. Next, when the image processing job 144 for normalizing the histogram is initialized, the initialization is performed in the same manner as the initialization of the image processing job 141. At this time, if the size of the page buffer required by the image processing job 144 is larger than the size of the page buffer required by the image processing job 141, the size of the first page buffer is reallocated. It is corrected by.

FIG. 19 shows the processing flow of each image processing job when the actual image processing is executed after the initialization of the image processing job in this way.
First, at step 131, image processing of a type corresponding to each job is performed. In step 132 it is checked whether all scan lines of the output image have been output. When all scan lines have been output, the process proceeds to step 133. If not, the process ends. After step 133, the processing of the image processing job has been completed, so the post-processing unit 23 performs post-processing. First, in step 133, resources other than the page buffer are released. For example, if a memory is allocated to the table area holding the histogram data for normalizing the histogram, this area is released. Next, in step 134, it is checked whether all other image processing jobs sharing the page buffer are finished. For example, in the example of the image processing pipeline shown in FIG. 18, when the post-processing of the image processing job 141 is performed, the image processing job 144 has not finished processing, and therefore the page buffer 145 cannot be released.
However, when the image processing job 143 ends, there is no other job that uses the page buffer 146, so the area of the page buffer 146 can be released immediately.
Normally, image processing jobs are completed from the beginning of the image processing pipeline, so for the odd-numbered and even-numbered jobs that use the page buffer, the ones that belong to the last stage of the pipeline release the page buffer. Will be done. Whether all jobs sharing the page buffer in step 134 have ended is determined by whether the job belongs to the last stage of the pipeline in the odd-numbered job and the even-numbered job using the page buffer, respectively. May be stored and the determination may be made by referring to it.

As described above, in the image processing method according to the present invention, even when a job utilizing a plurality of page buffers is connected to the image processing pipeline, especially at most 2 if the pipeline has a simple list structure. The system's memory resources can be effectively used by securing only one page buffer. Further, since the page buffer memory can be allocated all at the time of initialization, the advantage of the delayed execution method that early detection of an error is not impaired.

[0057]

As described above, in the image processing apparatus of the present invention, particularly in the image processing pipeline without branching like the list structure, only two page buffers are reserved at the same time, which is a wasteful memory. The processing can be performed efficiently without consuming.

Further, even in the image processing pipeline having a branch, only the minimum required page buffer is secured at the same time, and efficient processing can be performed without wasting memory.

Further, since the page buffer is secured when the image processing is initialized, it is possible to detect an error due to a memory shortage at an early stage.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of a hardware configuration in which an image processing apparatus of the present invention operates.

FIG. 2 is a configuration diagram of an embodiment of a scanner in which the image processing apparatus of the present invention operates.

FIG. 3 is a configuration diagram of an embodiment of a printer in which the image processing apparatus of the present invention operates.

FIG. 4 is a configuration example of an image processing pipeline implemented by the image processing apparatus of the present invention.

FIG. 5 is a configuration example of an image processing pipeline with branching executed by the image processing apparatus of the present invention.

FIG. 6 is a block diagram of an embodiment of an image processing apparatus according to the present invention.

FIG. 7 is a block diagram of an embodiment of the image processing apparatus of the present invention.

FIG. 8 is a flow chart of an embodiment showing an addition method of an image processing job in the image processing apparatus of the present invention.

FIG. 9 is a flowchart illustrating an example of post-processing according to the related art.

FIG. 10 is a configuration diagram of an embodiment of a branch job in the image processing apparatus of the present invention.

FIG. 11 is a flowchart showing an example of post-processing in the image processing apparatus of the present invention.

FIG. 12 is a block diagram of an embodiment of an image processing apparatus according to the present invention.

FIG. 13 is a block diagram of another embodiment of the image processing apparatus of the present invention.

FIG. 14 is a flowchart of an embodiment showing an initialization method of an image processing job in the image processing apparatus of the present invention.

FIG. 15 is a flowchart of an embodiment showing an example of using a page buffer in the image processing apparatus of the present invention.

FIG. 16 is a flowchart of an embodiment showing an execution method of an image processing job in the image processing apparatus of the present invention.

FIG. 17 is a flowchart of an embodiment showing an initialization method of an image processing job in the image processing apparatus of the present invention.

FIG. 18 is a flowchart of an embodiment showing a usage example of a page buffer in the image processing apparatus of the present invention.

FIG. 19 is a flowchart of an embodiment showing an image processing job execution method in the image processing apparatus of the present invention.

FIG. 20 is a flowchart showing an example of pipeline type image processing according to a conventional technique.

FIG. 21 is a flowchart showing an example of image processing of a delay execution system according to a conventional technique.

[Explanation of symbols]

11 ... Image processing device, 12 ... Image storage means, 13 ... Processing storage means, 14 ... Image processing means, 15 ... Control means, 16
... memory allocation control means, 21 ... detection means, 22 ... memory securing means, 23 ... process starting means, 24 ... post-processing means, 30 ... original, 31 ... lens, 32 ... CCD sensor,
33 ... A / D converter, 34 ... Buffer memory, 35 ... Image data transmitting circuit, 36 ... Optical system drive circuit, 37 ... Control circuit, 38 ... Interface, 40 ... Image data receiving circuit, 41 ... Laser driver, 42 ... Laser oscillator, 4
3 ... Polygon mirror, 44 ... Photoconductor, 45 ... Beam detector, 46 ... Motor, 47 ... Synchronous circuit, 48 ... Motor drive circuit, 49 ... Control circuit, 50 ... Paper feed system drive circuit,
61-64 ... Image processing jobs, 71-75 ... Image processing jobs, 302 ... Central processing unit, 303 ... Image processing device, 304 ... Information storage memory, 305 ... Input device, 30
6 ... External storage device, 307 ... Display device, 308 ... Scanner, 309 ... Printer

Claims (10)

[Claims] 1. An image storage means for storing an image, a processing storage means for inputting and storing information for processing an image stored in the image storage means, and an image processing stored in the processing storage means. An image processing apparatus comprising image processing means for processing an image according to information, and processing execution control means for controlling the timing of executing image processing according to the type of image processing command input to the processing storage means, The image processing apparatus is characterized in that the execution control means immediately performs the image processing when the image processing means for executing the input image processing instruction temporarily stores the entire image in the memory. 2. The processing execution control means detects the input image processing instruction by detecting means for detecting whether the image processing means for executing the input image processing instruction temporarily stores the entire image in the memory. When it is detected that the image processing means to be executed temporarily stores the entire surface of the image in the memory, the memory securing means for securing the memory for storing the image, and the memory after securing the memory for storing the image, It is characterized by including processing starting means for executing the image processing stored in the processing storage means, and post-processing means for releasing the resources reserved for use in the image processing when the image processing is completed. The image processing apparatus according to item 1. 3. The image processing means has a branching means for branching an output of a certain image processing into an input of a plurality of image processings, and the branching means is requested by the post-processing means to release resources. The image processing apparatus according to claim 2, wherein the resources are not released immediately at all times, and the resources are released when the release requests for all the branch destinations are completed. 4. The branching means has flags for all branch destinations, sets the flags in response to a resource release request, and releases the resources when all the flags are set. Item 3. The image processing device according to item 3. 5. The branching means has a counter related to a resource release request, increments the counter each time there is a resource release request, and releases the resource when the value of the counter reaches the number of branch destinations. The image processing apparatus according to claim 3, wherein: 6. An image storage means for storing an image, a processing storage means for inputting and storing information for processing an image stored in the image storage means, and an image processing stored in the processing storage means. An image including image processing means for processing an image in accordance with information, and memory allocation control means for controlling memory allocation required when the image processing means executes the image processing command input to the processing storage means. In the processing device, the memory allocation control means delays the allocation of the memory for storing the image until the processing is executed when the image processing means for executing the input image processing instruction temporarily stores the entire image in the memory. An image processing device characterized by the above. 7. The memory allocation control means executes the input image processing command when the image processing means initializes the image processing means according to the processing parameters and the characteristics of the input image. If there is only one image processing command for temporarily storing the entire image surface before the image processing command detected by the detection means, whether or not the entire surface is temporarily stored in the memory is detected. 7. The image processing apparatus according to claim 6, wherein a memory for storing the entire surface of the image is allocated instead of the buffer memory for one line when the image is initialized according to the processing parameters and the characteristics of the input image. 8. The memory allocation control means detects whether or not the image processing means for executing an input image processing command when executing the input image processing instruction temporarily stores the entire image in the memory, and temporarily stores the entire image. In the case of storing in the memory, a memory for temporarily storing the entire surface of the image is secured by the memory securing means, and the memory is substituted for the already allocated buffer memory for one line, Output image data required to fill the memory is requested to the image processing unit in the previous stage, and after the memory is filled with image data, the post-processing unit assigns it to the image processing command in the stage before the image processing command. 8. The image processing apparatus according to claim 7, wherein a memory for temporarily storing the stored image is released. 9. The image processing means has a memory for temporarily storing a maximum of two image entire surfaces at the same time, and the memory allocation control means controls the image processing means by the detecting means for processing parameters and input images. At the time of initialization according to the characteristics, it is detected whether the image processing means for executing the input image processing command at the time of executing the input image processing command temporarily stores the entire surface of the image in the memory, and 0 or an even number of images in the preceding stage. There is an image processing instruction that temporarily stores the entire surface in memory. An image processing instruction that temporarily stores the entire surface in memory shares the memory that temporarily stores the entire first image, and temporarily stores an odd number of images in the previous stage. 8. The image processing instruction for temporarily storing the entire image surface in which there is an image processing instruction for temporarily storing in the memory shares the memory for temporarily storing the second image entire surface. Image processing device. 10. The image processing instruction, when the memory allocation control means has an image processing instruction for temporarily storing two or more whole images in a memory before the image processing instruction detected by the detecting means. If there is an image processing command for temporarily storing the entire surface of an even number of images in the preceding stage, the same memory as the memory allocated to the image processing command for temporarily storing the entire surface of the first image is allocated, and If there is an image processing instruction that temporarily stores the entire surface of an odd number of images in the previous stage, allocate the same memory as the memory assigned to the image processing instruction that temporarily stores the entire surface of the second image,
10. The image processing apparatus according to claim 9, wherein the required amount of memory is reallocated when the respective memories do not reach the required capacity.