JP6278108B2 - Image processing apparatus, image sensor, and image processing method - Google Patents

Description

  The present invention relates to a technique for identifying or recognizing a model from an input image.

  In the FA (Factory Automation) field, a sensor device called an image sensor (visual sensor) is widely used to measure and monitor a measurement object (hereinafter also referred to as “work”) flowing through a line. The image sensor is composed of a camera and an image processing apparatus, detects a workpiece in an image by matching processing with a pre-registered teacher object (hereinafter also referred to as “model” or “pattern”), and extracts necessary information.・ Has a function to perform measurement. The output of the image sensor is used for various purposes such as workpiece identification, inspection, and sorting.

  By the way, as a recent trend in the FA field, there are an increasing number of production forms in which a plurality of types of workpieces are mixed and flowed on one line, and different processing is applied to each type. For example, in an assorted product boxing line, different types of workpieces are randomly flowed on a conveyor, picked up by type with a pickup robot, and boxed in the correct position. In addition, in a high-mix low-volume production line, series products with slightly different specifications such as shape, color, and size are manufactured on the same line, and there are cases where processing methods and inspection details differ depending on the specifications.

  In such a line in which a plurality of types of workpieces are mixed (hereinafter also referred to as “mixed flow line”), it is necessary to recognize (and detect / search) a plurality of models from the image. When performing object recognition for one model, a model template is generated by reducing the model image to reduce processing time, use memory capacity, accuracy, etc., and the input image is reduced to the same reduction ratio. It is known that search processing is performed (Patent Document 1). This method is particularly effective when the registration model is large. For example, the reduction rate may be determined so that the number of feature points included in the model becomes a predetermined number, or may be determined so that the area of the registration region becomes a predetermined size.

  In the mixed flow line, since a plurality of models are recognized, it is necessary to repeat the search process as many times as the number of models. Therefore, there is a problem that the processing time increases in proportion to the increase in the number of models.

When performing object detection using a multi-resolution image (pyramid image) for one model, the reduction ratio b of the template and the reduction ratio a of the pyramid image are set to b = a ^N (N is an integer of 2 or more). The method of setting to is known (Patent Document 2). According to this method, the number of pyramid images and the number of templates used for collation can be reduced, so that detection processing can be performed at high speed. However, this method cannot be applied when the types of objects to be detected are different.

JP 2010-67246 A JP 2006-202184 A

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing technique capable of identifying or recognizing a plurality of models at high speed from an input image.

  In order to achieve the above object, in the present invention, it is determined that the reduction ratios of the respective models are the same as much as possible. By reducing the number of times, the processing speed is increased.

  Specifically, an image processing apparatus according to the present invention is an image processing apparatus that identifies or recognizes a plurality of models from an input image, and stores a model template obtained by reducing the model image for the plurality of models. A plurality of storage units; a reduction unit that reduces the input image at a reduction ratio of each model; and a search unit that searches for a position of the model in the input image based on the reduced input image and the model template. For at least two of the models, a model template obtained by reducing a model image at the same reduction ratio is stored in the storage unit, and for the at least two models, processing by the reduction unit Are shared.

  Since the image reduction process is a process for the entire image, it is a process with a relatively large amount of calculation. In particular, the amount of calculation increases as the resolution of the input image increases. In the present invention, it is possible to reduce the total amount of calculation by making the model reduction rate the same and sharing the image reduction process for the models with the same reduction rate. In addition to the image reduction process, when a smoothing process, a feature amount calculation process, or the like is performed as a pre-process, these processes can be performed in common.

  An image processing apparatus according to the present invention includes a model image input unit that receives input of model images for a plurality of models, a temporary reduction rate determination unit that determines a provisional reduction rate for each of the plurality of model images, and the plurality of models Based on the provisional reduction rate of the image, a reduction rate determination unit that determines a reduction rate of each model image, and generates a model template by reducing each of the plurality of model images at a corresponding reduction rate, It is also preferable to further include a model template generation unit that associates and stores the model template in the storage unit.

  It is preferable that the temporary reduction rate determination unit determines a suitable reduction rate as the temporary reduction rate in light of a predetermined standard for each model image. The higher the reduction ratio, the faster the processing can be achieved. On the other hand, if the reduction ratio is too high, the accuracy of identification or recognition is reduced. Therefore, an appropriate reduction ratio considering the trade-off between processing speed and accuracy exists for each model image. The provisional reduction ratio is a preferable reduction ratio in this sense. For example, the provisional reduction ratio can be determined according to a standard such that the number of feature points included in the model is a predetermined number or a standard that the area of the model image is a predetermined size. it can.

  In order to obtain the actual reduction ratio from the provisional reduction ratio, for example, the following can be performed. That is, the reduction rate determination unit can group the plurality of model images so that the variation in the temporary reduction rate within the group is equal to or less than a predetermined threshold, and can determine the reduction rate for each group. The variation in the provisional reduction ratio within the group can be represented by, for example, the standard deviation or variance of the provisional reduction ratio. This grouping process (clustering process) can be performed using any clustering algorithm such as the k-means method. In addition, this grouping process may be a process of performing grouping by brute force and finding a classification that satisfies the above criteria. At this time, the reduction ratio applied to the model images in each group can be determined as, for example, the average value, median value, or mode value of the provisional reduction ratios of the model images in the group.

  As another method for obtaining the actual reduction rate from the provisional reduction rate, for example, there is the following method. That is, the reduction ratio determination unit stores a plurality of predetermined reduction ratios, and among the plurality of predetermined reduction ratios, the reduction ratio corresponding to the provisional reduction ratio of the model image is determined as the model image. Can be determined as the reduction ratio. The reduction rate corresponding to the temporary reduction rate is preferably set to a reduction rate closest to the temporary reduction rate among a plurality of predetermined reduction rates. Here, the standard of proximity can be set as appropriate, and for example, a simple difference or a square difference can be adopted. The reduction ratio corresponding to the provisional reduction ratio is preferably set to a minimum reduction ratio equal to or higher than the provisional reduction ratio or a maximum reduction ratio equal to or lower than the provisional reduction ratio among a plurality of predetermined reduction ratios.

  The present invention can be understood as an image processing apparatus having at least a part of the above configuration. The present invention can also be understood as an image sensor having a camera for photographing an object and an image processing device.

  The present invention can also be understood as an image processing apparatus for generating a model template. That is, one aspect of the present invention is an image processing apparatus that generates a model template for a plurality of models, a model image input unit that receives model images for a plurality of models, and provisional reduction for each of the plurality of model images Each of the plurality of model images corresponds to a provisional reduction rate determination unit that determines a rate, a reduction rate determination unit that determines a reduction rate of each model image based on the provisional reduction rate of the plurality of model images, and It has a model template generation unit that generates a model template by reducing with a reduction rate, and an output unit that outputs the generated model template together with a corresponding reduction rate. As described above, an actual reduction ratio determination method can employ a method based on grouping based on a provisional reduction ratio or a method of selecting a corresponding reduction ratio from predetermined reduction ratios.

  Further, the present invention can also be understood as an image processing method including at least a part of the above processing, or a program for causing a computer to execute such a method, or a computer-readable recording medium in which the program is temporarily stored. it can. Each of the above configurations and processes can be combined with each other as long as no technical contradiction occurs.

  According to the present invention, a plurality of models can be identified or recognized from an input image at high speed.

The figure which shows the whole structure of an image sensor. The figure which shows the hardware constitutions of an image sensor. The figure which shows the function structure in connection with kind discrimination | determination of an image identification apparatus. The flowchart which shows the flow of the kind discrimination | determination process in operation mode. The flowchart which shows the detail of the search process in step S101 of FIG. The figure explaining the effect of the search process by this embodiment. The flowchart which shows the flow of the model registration process in registration mode. The flowchart which shows the detail of the grouping process in step S303 of FIG. The figure explaining grouping process. The flowchart which shows the flow of the modification of the model registration process in registration mode.

  The present invention relates to an image identification technique for extracting an image that best matches an input image from a plurality of registered images (model images) registered in advance by template matching when an input image is given. is there. This technology can be applied to object discrimination in an image sensor for FA, computer vision, machine vision, and the like, or a similar image search for detecting an image similar to a query image from a group of images in an image database. In the embodiment described below, as one of preferred applications of the present invention, the present invention is implemented in an FA image sensor that detects and discriminates each workpiece in a mixed flow line in which a plurality of types of workpieces flow. An example will be described.

(Image sensor)
With reference to FIG. 1, an overall configuration and application scene of an image sensor according to an embodiment of the present invention will be described.
The image sensor 1 is a system that is installed in a production line or the like and performs type discrimination of the workpiece 2 using an input image obtained by imaging a product (work 2). In addition to the type discrimination, the image sensor 1 can be mounted with various image processing functions as necessary, such as edge detection, scratch / dirt detection, and area / length / centroid measurement.

  As shown in FIG. 1, a plurality of types of workpieces 2 are mixed on the conveyor 3. The image sensor 1 captures an image periodically from the camera 11 or at the timing of a trigger signal sent from the PLC 4, and processes such as detection and type determination of each workpiece 2 included in the image by an image processing device (image identification device) 10. And the result is displayed on the display 12 or output to an external device (PLC 4 or the like). A PLC (Programmable Logic Controller) 4 is a device that controls a manufacturing apparatus (not shown) such as the image sensor 1, the conveyor 3, and the robot.

(Hardware configuration of image sensor)
The hardware configuration of the image sensor 1 will be described with reference to FIG. The image sensor 1 generally includes a camera 11 and an image processing device 10.

  The camera 11 is a device for capturing an image of the workpiece 2 into the image processing apparatus 10. For example, a complementary metal-oxide-semiconductor (CMOS) camera or a charge-coupled device (CCD) camera can be preferably used. The format of the input image (resolution, color / monochrome, still image / moving image, gradation, data format, etc.) is arbitrary, and may be appropriately selected according to the type of workpiece 2 and the purpose of sensing. When a special image (such as an X-ray image or a thermo image) other than a visible light image or an image with depth (distance) information is used for inspection, a camera that matches the image may be used.

  The image processing apparatus 10 includes a CPU (Central Processing Unit) 110, a main memory 112 and a hard disk 114 as a storage unit, a camera interface 116, an input interface 118, a display controller 120, a PLC interface 122, and a communication interface. 124 and a data reader / writer 126. These units are connected to each other via a bus 128 so that data communication is possible.

  The camera interface 116 is a part that mediates data transmission between the CPU 110 and the camera 11, and has an image buffer 116 a for temporarily storing image data from the camera 11. The input interface 118 mediates data transmission between the CPU 110 and an input unit (mouse 13, keyboard, touch panel, jog controller, etc.). The display controller 120 is connected to a display 12 such as a liquid crystal monitor, and controls display on the display 12. The PLC interface 122 mediates data transmission between the CPU 110 and the PLC 4. The communication interface 124 mediates data transmission between the CPU 110 and a console (or a personal computer or a server device). The data reader / writer 126 mediates data transmission between the CPU 110 and the memory card 14 as a storage medium.

  The image processing apparatus 10 can be configured by a computer having a general-purpose architecture, and the CPU 110 reads and executes a program (instruction code) stored in the hard disk 114 or the memory card 14 to provide various functions. . Such a program is distributed while being stored in a computer-readable recording medium such as the memory card 14 or an optical disk.

  When a general-purpose personal computer is used as the image processing apparatus 10, in addition to an application program for providing an object discrimination function described in the present embodiment, an OS (operation for providing a basic function of the computer) -System) may be installed. In this case, the program according to the present embodiment may realize a target function by using a program module provided by the OS. Note that the program according to the present embodiment may be provided as a single application program, or may be provided as a module incorporated in a part of another program. Further, a part or all of the functions may be replaced with a dedicated logic circuit.

(Functional configuration of image processing apparatus)
FIG. 3 shows a functional configuration related to type discrimination (object discrimination) provided by the image processing apparatus. The image processing apparatus 10 includes an image input unit 130, a search unit 131, a storage unit 137, an identification unit 138, an output unit 139, a model image input unit 140, and a model template generation unit 141 as functions related to type discrimination. . These functional blocks are realized by the CPU 110 of the image processing apparatus 10 executing a computer program.

  The storage unit 137 stores the model template generated by the model template generation unit 141. The model template is generated by subjecting the model image input from the model image input unit 140 to image reduction processing at a predetermined reduction rate and extracting a feature amount from the reduced image. In the present embodiment, since a coarse / fine search (pyramid search) is performed, a plurality of model templates having a reduction ratio increased stepwise for one model image are generated and stored. In the present specification, the lowest reduction ratio among the reduction ratios when generating the model template is referred to as the reduction ratio of the model template.

  The storage unit 137 stores the model template 137a and the reduction rate 137b of the model template in association with each other. In the storage unit 137, the number of model templates corresponding to the type of object to be discriminated is stored. Note that the model template generation unit 141 determines the reduction rate so that the reduction rates of the models are as identical as possible. Therefore, there are at least two (one set) model templates having the same reduction ratio in the storage unit 137.

(Operation of image processing device)
The image processing apparatus 10 captures the image of the workpiece 2 flowing on the conveyor 3 and executes an “operation mode” in which processing such as search (collation) and type determination of the workpiece 2 is performed, and prior to the operation mode, the image processing apparatus 10 It has a “registration mode” for registering models. The user can arbitrarily switch the mode. The model template 137a in the storage unit 137 is generated by the model template generation unit 141 in the registration mode.

  Hereinafter, the operation in the “operation mode” will be described on the assumption that the generation of the model template 137a has already been completed, and then the model registration operation in the “registration mode” will be described.

(1) Operation Mode The operation of each functional block in the operation mode and the overall flow of the type determination process will be described with reference to the flowchart of FIG.

  When the trigger signal from the PLC 4 is input, the image input unit 130 captures an image from the camera 11 (step S100). Next, the search part 131 detects each workpiece | work 2 from an input image (step S101). Details of the search (collation) processing in step S101 will be described with reference to FIG. The search unit 131 acquires the reduction rate 137b of the model template 137a stored in the storage unit 137, and repeatedly executes the following steps S201 to S203 for each reduction rate. The processing in steps S201 to S203 is preprocessing for search processing.

  In step S201, the image reduction unit 132 performs an image reduction process with the same reduction rate as the reduction rate of the model template on the input image. In step S202, the smoothing processing unit 133 applies a smoothing filter to the reduced image to remove noise. In step S203, the luminance gradient image generation unit 134 generates a luminance gradient image from the smoothed image. The luminance gradient image generation unit 134 extracts feature points such as edges and corners from the luminance gradient image based on the luminance gradient. In this embodiment, since the coarse / fine search is performed, the processing from image reduction to luminance gradient image generation is executed a plurality of times with the reduction rate being increased stepwise.

  The processing in steps S204 to S206 is processing for actually searching for a model from the input image, and is repeatedly executed for each of the plurality of models. In step S204, the coarse / fine search unit 135 searches for a model using the coarse / fine search method. That is, first, a low-resolution image is roughly searched, a rough position is specified, and then detailed positioning is performed using the high-resolution image. Any number of hierarchies may be used in the coarse / fine search. According to such a density search, positioning can be performed with high speed and high accuracy. Next, in step S205, the neighborhood exclusion unit 136 performs neighborhood exclusion processing. In the search process of step S204, a plurality of detection candidates may appear as a result of the search. In such a case, detection candidates with similar positions and orientations are regarded as the same, and only one of them is left and the others are excluded, thereby omitting the processing of duplicate detection candidates. This is the neighborhood exclusion process. In step S206, the search unit 131 outputs the position of the model in the input image obtained in this way. With the above processing, the positions of all models registered in the storage unit 137 are searched from the input image.

  Returning to the flowchart of FIG. In step S102, the identification unit 138 analyzes the workpiece detected by the search process and determines the type of workpiece. When similar models are detected, it may be determined in the identification process that the workpiece matches a plurality of models. The identification unit 138 determines which of the plurality of models the detected workpiece best matches. Various methods can be adopted as this determination method. For example, a model having the highest similarity with the workpiece may be determined as the type of workpiece. Alternatively, the feature amount extracted from the detected workpiece may be input to a discriminator that identifies the model type, and the type of the workpiece may be determined.

  By repeating the process of step S102 for all the workpieces detected in step S101, the type of each workpiece can be determined. The determination result is output to the display 12 or the PLC 4 by the output unit 139 (step S103).

The effect of this method will be described with reference to FIG. FIG. 6A is a diagram for schematically explaining the processing of this method, and FIG. 6B is a diagram for schematically explaining the processing of the conventional method (comparative example). In the conventional method, it is necessary to execute the image reduction process, the smoothing process, and the luminance gradient image generation process as many times as the number of models. On the other hand, according to this method, the image reduction process, the smoothing process, and the luminance gradient image generation process can be shared when the reduction rate of the model template is the same (dotted line portion). Therefore, it is not necessary to repeat the above processing for the number of models, and the processing speed can be increased. Since these processes are processes for the entire image, the amount of calculation is relatively large. Therefore, effective speeding up can be achieved by reducing the number of executions. For example, as shown in FIG. 6, consider the processing time when the number of models is five and the reduction ratios of all models are the same. It is assumed that the processing time required for the search is 1 second as a whole, and half of the processing time is occupied by preprocessing such as image reduction processing, smoothing processing, and luminance gradient image generation processing.
Then, in the conventional method, since it is necessary to execute all search processes for each model, the processing time is 1 second × 5 types = 5 seconds. On the other hand, in this method, since the pre-processing needs to be performed only once, 0.5 seconds + 0.5 seconds × 5 types = 3 seconds. That is, according to this method, the processing time can be reduced by 40% compared to the conventional method. The example of FIG. 6 shows an example in which the reduction ratios of all models can be made the same, and the maximum effect can be obtained. However, if at least two models have the same reduction ratio, the preprocessing is common. As a result, the processing time can be shortened compared to the conventional example.

(2) Registration Mode Next, a model registration process in the registration mode, particularly a method for determining a model reduction rate will be described with reference to the flowchart of FIG. Note that the processing shown in FIG. 7 is executed, for example, when the image sensor 1 is newly installed, or when the type of work flowing on the line is changed.

  First, in step S <b> 301, an input of a model image to be registered is received from the model image input unit 140. For example, the model image input unit 140 captures, as a model image, an image of a region portion designated by the user from images of a plurality of types of objects (models) photographed by the camera 11 under the same conditions as in operation.

  Next, the temporary reduction rate determination unit 141a of the model template generation unit 141 obtains a suitable reduction rate (provisional reduction rate) for each model image. If the reduction ratio is increased, the processing speed can be increased, but the identification accuracy is lowered. On the other hand, if the reduction ratio is lowered, the identification accuracy is improved, but the effect of increasing the processing speed is limited. Therefore, the provisional reduction rate determination unit 141a obtains a reduction rate appropriate for the model image in accordance with a predetermined standard in consideration of a trade-off between speed and accuracy. For example, the provisional reduction ratio determination unit 141a may determine the provisional reduction ratio so that the number of feature amounts included in the model becomes a predetermined number, or provisionally so that the area of the registration area of the model becomes a predetermined size. The reduction rate may be determined. In general, the provisional reduction rate obtained in this manner is different for each model.

  As described above, although the optimum reduction rate differs for each model, the model template generation unit 141 makes the reduction rate of each model as identical as possible within a range in which the processing speed and the identification accuracy are not reduced as much as possible. Specifically, in step S303, the reduction rate determination unit 141b of the model template generation unit 141 performs grouping of models so that the variation in the provisional reduction rate of each model in the group becomes sufficiently small. The grouping process can be performed using a clustering algorithm such as the k-means method, but can be realized by the following simple method.

  Details of the grouping process in step S303 will be described with reference to the flowchart of FIG. First, the model template generation unit 141 sorts a plurality of models so that the provisional reduction ratios are in ascending order (S401). Then, the variable G representing the group number is initialized to 1 (S402). The following steps S403 to S408 are repeatedly executed for a plurality of models. At this time, processing is performed in order from a model with a small provisional reduction ratio.

  First, among the unprocessed models, the model having the smallest provisional reduction ratio is added to the group G (S403), and the standard deviation of the provisional reduction ratio is calculated for the group after the addition (S404). If the calculated standard deviation is within the threshold (S405—YES), the addition of the model to the group G is permitted. That is, if the variation in the temporary reduction ratio is within an allowable range, the addition of the model to the group G is permitted. On the other hand, if the standard deviation calculated in step S404 is larger than the threshold value (S405-NO), the model is deleted from the group G without allowing the model to be added to the group G (S406). This model will be included in the new group. That is, the variable G representing the group number is incremented by 1 (S407), and this model is added to the group G (after update) (S408).

  By repeatedly executing the processes in steps S403 to S408 for all models, the grouping in which the variation (standard deviation) in the provisional reduction ratio within the group is finally within the threshold is completed.

  With reference to FIG. 9, the grouping process of FIG. 8 will be exemplarily described. Here, there are seven types of models A to G, and the respective provisional reduction ratios are A: 0.3, B: 0.33, C: 0.42, D: 0.65, E: 0, 67, F. : 0.71, G: 0.75. Further, the standard deviation threshold is set to 0.05.

  First, the model A having the smallest provisional reduction rate is added to the group 1. Next, the standard deviation of the provisional reduction ratio in group 1 when model B is added to group 1 is calculated. As shown in FIG. 9A, since this standard deviation is 0.021 and is within the threshold value, it can be determined that the model B may be included in the group 1. Next, the standard deviation of the provisional reduction ratio in group 1 when model C is added to group 1 is calculated. As shown in FIG. 9B, this standard deviation is 0.062, which exceeds the threshold value. Therefore, it is determined that the model C is not included in the group 1. Group 1 is determined to include only model A and model B, and model C is included in the new group 2. Next, the standard deviation of the provisional reduction ratio in group 2 when model D is added to group 2 is calculated. As shown in FIG. 9C, this standard deviation is 0.16, which exceeds the threshold value, so that it is determined that the model D is not included in the group 2. That is, the group 2 is determined to include only the model C, and the model D is included in the new group 3. Thereafter, in the same manner as described above, a model with a small provisional reduction ratio is added to the group in order, and it is determined whether or not the standard deviation of the provisional reduction ratio within the group is within a threshold. In this example, when the models E, F, and G are added to the group 3, as shown in FIGS. 9D, 9E, and 9F, the standard deviation is within the threshold value. Therefore, group 3 is determined as including models D, E, F, and G. As described above, the seven models A to G are grouped into three groups (A, B), (C), and (D, E, F, G).

  In the flowchart of FIG. 8, processing is performed in order from a model with a small provisional reduction ratio, but processing may be performed in order from a model with a large provisional reduction ratio. Alternatively, the processing may be started from the largest and smallest provisional reduction ratios. The method is not particularly limited as long as it is grouped so that the standard deviation of the provisional reduction rate within the group is within the threshold value. Further, although the standard deviation is adopted as a measure of variation, variance may be adopted, or a difference between the maximum value and the minimum value may be adopted.

  Returning to the flowchart of FIG. When the grouping is completed, in step S304, the reduction rate determination unit 141b determines a representative reduction rate for each group. Here, the average value of the provisional reduction ratio in the group is set as the representative reduction ratio of the group. For example, in the example of FIG. 9, as shown in FIG. 9F, the representative reduction ratio of the group 1 consisting of the models A and B is 0.315, and the representative reduction ratio of the group 2 consisting of only the model C is 0.420. Thus, the representative reduction ratio of the group 3 including the models D, E, F, and G is 0.695. Note that the representative reduction ratio is not the average value of the provisional reduction ratios in the group, but may be the median value or the mode value.

  In step S305, the model template generation unit 141 generates, for each model, a model template by reducing the model image at the representative reduction rate of the group to which the model belongs. In this embodiment, since the identification is performed using the feature value of the brightness gradient, a brightness gradient image is generated from the reduced model image as the model template, and the feature value is calculated and used as the model template. Further, in this embodiment, since a coarse / fine search is performed, a model template is generated based on a model image obtained by reducing the model image at a reduction rate higher than the representative reduction rate.

  In step S306, the model template generation unit 141 outputs the generated model template together with the reduction ratio to the storage unit 137. Thereby, the model template 137a and the reduction ratio 137b are stored in the storage unit 137 in association with each other.

  As described above, a plurality of models can have the same reduction rate within a range in which a deviation from the optimum reduction rate (provisional reduction rate) for each model can be allowed.

  According to the present embodiment, since the reduction ratio at the time of model registration is the same for a plurality of models, it is possible to increase the speed by making common pre-processing such as input image reduction processing and feature amount extraction processing during operation. In addition, when the model reduction rate is the same, the variation in the provisional reduction rate within the group is made small (so that it is within the threshold), so the impact of degradation of identification accuracy is minimized. Can be suppressed.

  If a model image is added later, the above registration process may be executed again. In the registration process, an image before reduction is required for all model images. Therefore, in order to be able to add a model image later, it is preferable to save the model image before reduction after the registration process.

(Modification of model registration process (reduction rate determination process))
In the above description, the reduction rate determination of each model at the time of model registration is performed by grouping the models based on the provisional reduction rate. Model registration processing, particularly reduction rate determination processing, can also be realized by methods other than those described above. For example, as a modification of the model registration process, a plurality of predetermined reduction ratios are stored, and any one of the predetermined reduction ratios is determined based on the temporary reduction ratio of the model. , Can be determined as the reduction rate of the model. Hereinafter, the model registration process according to this example will be described with reference to FIG.

  First, in step S401, an input of a model image to be registered is received from the model image input unit 140. Next, the model template generation unit 141 repeats the processes of steps S402 to S405 below for each model image. In step S402, the provisional reduction rate determination unit 141a of the model template generation unit 141 obtains a suitable reduction rate (provisional reduction rate) of the model image. This process is the same as the process described above.

  Next, the reduction rate determination unit 141b determines a reduction rate closest to the temporary reduction rate among a plurality of predetermined reduction rates as the reduction rate of the model image. For example, assume that a plurality of predetermined reduction ratios are (1.0, 0.707, 0.5, 0.354, 0.25). In such a case, if the temporary reduction ratio is 0.3, 0.25 is selected as the actual reduction ratio. Similarly, the reduction ratio 0.354 is selected for the temporary reduction ratios 0.33 and 0.42, and the reduction ratio 0 is set for the temporary reduction ratios 0.65, 0.67, 0.71, and 0.75. .707 is selected.

If a plurality of reduction ratios are determined in advance, the reduction ratio determination process can be obtained by the following condition judgment. In the following, r represents the provisional reduction ratio, and R represents the actual reduction ratio.
If 0.854 ≦ r, then R = 1
If 0.604 ≦ r <0.854, R = 0.707
If 0.427 ≦ r <0.604, then R = 0.5
If 0.302 ≦ r <0.427, then R = 0.354
If r <0.302, then R = 0.250

  In step S404, the model template generation unit 141 generates a model template by reducing the model image at the determined reduction rate. In step S405, the model template generation unit 141 outputs the generated model template to the storage unit 137 together with the reduction ratio.

  Even in this case, the reduction ratios of a plurality of models can be made the same within a range in which deviation from the optimum reduction ratio (provisional reduction ratio) for each model can be allowed. In this method, even if a new model is added after the registration process is performed once, the existing model template is not changed. Therefore, the process at the time of model addition is simplified. Also, there is an advantage that it is not necessary to store the model image after creating the model template.

  In step S403, a model closest to the provisional reduction ratio among a plurality of predetermined reduction ratios is determined as the model reduction ratio. In the above, a simple difference is used as the distance measure. For example, even if a reduction ratio closest to the provisional reduction ratio is selected based on an arbitrary distance measure such as a square difference, a logarithmic difference, or a square root difference. Good. Moreover, you may make it select other than the nearest reduction rate. For example, the smallest reduction ratio greater than or equal to the provisional reduction ratio may be determined as the model reduction ratio, or the largest reduction ratio less than or equal to the provisional reduction ratio may be determined as the model reduction ratio. .

  Further, in the above description, five reduction ratios between 1 and 0.25 are adopted as a plurality of predetermined reduction ratios, but the number and reduction ratio values can be changed as appropriate. . For example, a value smaller than 0.25 can be adopted. In the above example, the reduction ratio is determined to be a geometric sequence (common ratio 0.707), but it is not necessarily required to be a geometric sequence.

(Other variations)
The configuration of the above-described embodiment is merely a specific example of the present invention, and is not intended to limit the scope of the present invention. The present invention can take various specific configurations without departing from the technical idea thereof. For example, in the above-described embodiment, an example in which the present invention is applied to an object discrimination apparatus that discriminates a plurality of similar object types has been described, but the scope of application of the present invention is not limited to this. The present invention is applicable to any image processing apparatus that identifies or recognizes a plurality of models from an input image, and the plurality of models do not need to be similar.

  Further, the present invention does not limit a specific method of template matching. For example, in the identification processing, identification may be performed based on feature amounts such as edges, corners, luminance gradient directions, luminance gradient direction histograms, or identification may be performed using pixel values themselves such as luminance and color. Further, any algorithm for speeding up such as coarse / fine search can be used. The present invention can be applied to any of existing methods of template matching. Although it is necessary to change the model template creation process according to the template matching to be employed, those methods can be easily understood by those skilled in the art.

  In the above description of the embodiment, the model template is created in the image processing apparatus that performs the identification process. However, the apparatus that creates the model template and the apparatus that uses the created model template may be different. Absent. The present invention can be regarded as an image identification device (image processing device) for identifying an image using a model template obtained by reducing a model image at the same reduction ratio for at least two models, It can also be regarded as a model template creation device (image processing device) that creates a model template by reducing the model image at the same reduction ratio for at least two models.

  In addition to the above-described embodiment, the present invention performs template matching on an image recognition apparatus that registers an abnormal state as a model image and inspects an object, or on a specific area of an input image without performing a search. Or an image recognition apparatus that performs template matching on the entire input image having the same size as the model image.

1: Image sensor, 2: Workpiece, 3: Conveyor, 4: PLC
DESCRIPTION OF SYMBOLS 10: Image processing apparatus, 11: Camera, 12: Display 130: Image input part, 131: Search part, 137: Storage part 140: Model image input part, 141: Model template production | generation part

Claims (17)

An image processing apparatus for identifying or recognizing a plurality of models from an input image,
A storage unit for storing a model template obtained by reducing a model image for a plurality of models,
A model image input unit that accepts input of model images for a plurality of models;
For each of the plurality of model images, a provisional reduction ratio determination unit that determines a provisional reduction ratio;
A reduction rate determination unit that determines a reduction rate of each model image based on the provisional reduction rates of the plurality of model images;
A model template generating unit that generates a model template by reducing each of the plurality of model images at a corresponding reduction rate, and stores the model template in the storage unit in association with the reduction rate;
A reduction unit that reduces the input image at a reduction ratio of each model;
A search unit that searches for the position of the model in the input image based on the reduced input image and the model template;
Have
For at least two models of the plurality of models, a model template obtained by reducing the model image at the same reduction ratio is stored in the storage unit,
For the at least two models, processing by the reduction unit is shared,
An image processing apparatus. The reduction rate determination unit
The plurality of model images are grouped so that the variation in the provisional reduction ratio within the group is a predetermined threshold value or less,
Decide the reduction ratio for each group,
The image processing apparatus according to claim 1 . The reduction ratio determining unit determines an average value of provisional reduction ratios of model images in a group as a reduction ratio of the group.
The image processing apparatus according to claim 2 . The reduction rate determination unit
A plurality of predetermined reduction rates are stored,
Of the plurality of predetermined reduction rates, a reduction rate corresponding to the temporary reduction rate of the model image is determined as the reduction rate of the model image.
The image processing apparatus according to claim 1 . The reduction rate determination unit determines a reduction rate close to the provisional reduction rate of the model image among the plurality of predetermined reduction rates as a reduction rate of the model image;
The image processing apparatus according to claim 4 . A camera that shoots objects,
Image sensor, wherein the said model identification or recognized from the input image from the camera, and outputs the result, with the image processing apparatus according to any one of claims 1-5. An image processing apparatus that generates a model template for a plurality of models,
A model image input unit for receiving model images for a plurality of models;
For each of the plurality of model images, a provisional reduction ratio determination unit that determines a provisional reduction ratio;
A reduction rate determination unit that determines a reduction rate of each model image based on the provisional reduction rates of the plurality of model images;
A model template generation unit that generates a model template by reducing each of the plurality of model images at a corresponding reduction ratio;
An output unit that outputs the generated model template together with a corresponding reduction ratio;
An image processing apparatus. The reduction rate determination unit
The plurality of model images are grouped so that the variation in the provisional reduction ratio within the group is a predetermined threshold value or less,
Decide the reduction ratio for each group,
The image processing apparatus according to claim 7 . The reduction ratio determining unit determines an average value of provisional reduction ratios of model images in a group as a reduction ratio of the group.
The image processing apparatus according to claim 8 . The reduction rate determination unit
A plurality of predetermined reduction rates are stored,
Of the plurality of predetermined reduction rates, a reduction rate corresponding to the temporary reduction rate of the model image is determined as the reduction rate of the model image.
The image processing apparatus according to claim 7 . The reduction rate determination unit determines a reduction rate close to the provisional reduction rate of the model image among the plurality of predetermined reduction rates as a reduction rate of the model image;
The image processing apparatus according to claim 10 . An image processing method for identifying or recognizing a plurality of models from an input image,
A computer having a storage unit for storing a model template obtained by reducing model images for a plurality of models,
A model image input step for accepting input of model images for a plurality of models;
For each of the plurality of model images, a provisional reduction ratio determination step for determining a provisional reduction ratio;
A reduction rate determining step for determining a reduction rate of each model image based on the provisional reduction rates of the plurality of model images;
Generating a model template by reducing each of the plurality of model images at a corresponding reduction ratio, and storing the model template in the storage unit in association with the reduction ratio; and
A reduction step of reducing the input image at a reduction ratio of each model;
A search step for searching for the position of the model in the input image based on the reduced input image and the model template;
Run
Wherein for at least some of the multiple models, and the model templates reduced at the same reduction ratio is stored in the storage unit,
For models with the same reduction ratio, the processing of the reduction step is shared.
An image processing method. An image processing method for generating model templates for a plurality of models,
Computer
A model image input step for receiving model images for a plurality of models;
For each of the plurality of model images, a provisional reduction ratio determination step for determining a provisional reduction ratio;
A reduction rate determining step for determining a reduction rate of each model image based on the provisional reduction rates of the plurality of model images;
A model template generation step of generating a model template by reducing each of the plurality of model images at a corresponding reduction ratio;
An output step for outputting the generated model template together with a corresponding reduction ratio;
An image processing method is executed. A program for causing a computer to execute each step of the image processing method according to claim 12 . A program for causing a computer to execute each step of the image processing method according to claim 13 . The computer-readable recording medium which memorize | stores the program of Claim 14 non-temporarily. The computer-readable recording medium which memorize | stores the program of Claim 15 non-temporarily.

Images