JP5080416B2 - Image processing apparatus for detecting an image of a detection object from an input image - Google Patents

Description

  The present invention relates to an image processing apparatus that detects an image of a detection target taught in advance from an input image acquired by an imaging apparatus.

  In the field of factory automation, image processing apparatuses are widely used. For example, an industrial product is imaged by an imaging device such as a camera, and the position and orientation of the industrial product are specified by the acquired image, for example, the industrial product is handled by a robot, or the color of the industrial product is determined by the acquired image. An image processing apparatus is used to inspect the shape and the like.

  When an image of a specific object is detected from an image in the field of view of the imaging apparatus using the image processing apparatus, reference information representing the object (generally referred to as a teaching model) and the imaging apparatus are used. Generally, matching with an input image is performed, and it is generally determined that the object has been successfully detected when the degree of matching exceeds a predetermined level.

  One method used for such matching is template matching. In template matching, an image including an object to be detected expressed in pixel units is stored in advance as a template image, and this template image is stored in an arbitrary area (template image and a template image) acquired by the imaging device. The object is detected based on the similarity, that is, the degree of correlation between the two.

ここで、ｆ（ｉ，ｊ）は入力画像ｆの位置（ｉ，ｊ）の画素の輝度値を表し、ｇ（ｉ，ｊ）はテンプレート画像ｇの位置（ｉ，ｊ）の画素の輝度値を表し、Ｄは、｜ｉ−ｍ｜≦Ｍ_T／２、｜ｊ−ｎ｜≦Ｎ_T／２を満たす（ｉ，ｊ）の範囲、すなわちテンプレート画像ｇの中心を入力画像ｆ中の位置（ｍ，ｎ）に重ねて配置したときに入力画像ｆ中においてテンプレート画像ｇに相当するサイズの領域である。 Specifically, for example, the image size of the input image f is M × N pixels, the image size of the template image g is M _T × N _T pixels, and the image size of the input image f is larger than the image size of the template image g ( That is, assuming that M _T <M, N _T <N) and the center of the template image g is at the position (m, n) in the input image f, a value C representing the degree of correlation is obtained by the following equation.

Here, f (i, j) represents the luminance value of the pixel at the position (i, j) of the input image f, and g (i, j) represents the luminance value of the pixel at the position (i, j) of the template image g. D represents a range of (i, j) satisfying | i−m | ≦ M _T / 2 and | j−n | ≦ N _T / 2, that is, a position of the template image g in the input image f. This is a region having a size corresponding to the template image g in the input image f when arranged so as to overlap (m, n).

  The value C obtained by the above equation is generally called a normalized cross-correlation coefficient, and is used as a measure for judging the similarity between two images. The center of the template image g is moved to all positions (m, n) in the input image f to obtain the cross-correlation coefficient at the coordinates, and the input image f is obtained by obtaining a location where the value of the cross-correlation coefficient C is large. A region similar to the template image g can be found therein. In particular, when the image of the region D in the input image f is the same image that differs only in luminance from the template image g, that is, when the luminance of the image of the region D in the input image f is a constant multiple of the luminance of the template image g, The cross-correlation coefficient C is equal to 1. On the other hand, the greater the difference between the image in the region D in the input image f and the template image, the closer the cross-correlation coefficient C is to 0.

  In such template matching, the brightness value of each pixel of the template image, that is, the teaching model is used as a feature amount representing the object to be detected.

  In addition, instead of directly using an image containing an object as a teaching model, matching is performed using numerical values representing geometric features of the object extracted by performing image processing on the image as feature quantities. The method of doing is also known. Conventionally well-known examples of image processing include noise reduction by a median filter, contrast improvement by histogram equalization, binarization, and edge extraction by a sobel filter. Further, methods used for detecting geometric features include straight line detection by Hough transform, boundary line detection by boundary tracking, and connected region detection by labeling. Examples of geometric features include the perimeter, area, circularity, curvature, moment, etc. of the connected region. For example, when detecting a square object from an input image, it is possible to use a feature quantity that represents a geometric feature such as four straight lines, their crossing at almost right angles, and the area of a region surrounded by the four straight lines. it can.

  For the matching, various methods such as a method of detecting an object by an eigenspace method and a method of detecting a pattern related to an object by a genetic algorithm can be used.

  Furthermore, in order to detect an object image from an input image with high accuracy, a technique such as Patent Document 1 or Patent Document 2 may be used. In the image processing device disclosed in Patent Document 1, when the target object to be detected is limited to a specific object such as a human face eye, nose, mouth, or the like, the relationship between the size of the left eye and the eye An appropriate feature amount is extracted from the input image based on a small amount of information input from the operator using previously known constraint conditions such as the positional relationship between the nose and the mouth. In the information processing apparatus disclosed in Patent Document 2, a plurality of search units that search for images similar to the template image are provided, and the total similarity is calculated as a sum of weighted search results of each search unit. The weighting can be changed, and the search accuracy is improved by appropriately changing the weighting.

JP 2000-31248 A JP 2004-334339 A

  When detecting an image of an object from an input image by the method as described above, if the image of the object in the input image is stably obtained or the shape of the object is stable, Stable detection results can be obtained. However, for example, when there is a variation in the brightness of the illumination light when the input image is captured, a part of the image may be darker or brighter than the other part, and the image of the object may be In some cases, it cannot be obtained stably. In some cases, the object is an industrial product and the shape thereof is not necessarily constant. For example, if the object is a cast part such as a die-cast part, the part is manufactured by pouring molten metal into the mold, so the individual difference in shape is small in the same lot manufactured using the same mold. The individual difference of the shape may increase between different lots manufactured using different molds. Thus, when detecting an image of an object from an input image obtained by imaging an object having an individual difference in shape, the detection of the object may not be stable due to the individual difference regarding the shape. possible.

  Therefore, conventionally, the operator has manually corrected the teaching model so that the detection of the object is less likely to become unstable. For example, when an object is detected using a normalized cross-correlation coefficient, so-called pixel masking is performed in which pixels that are not used for the calculation of the cross-correlation coefficient are provided among the pixels in the template image. When pixel masking is performed, the pixel on the input image corresponding to the masked pixel is not affected by the value of the normalized cross-correlation coefficient regardless of the luminance. . Further, when the object is detected using the geometric feature of the object, as a kind of masking, unnecessary ones are excluded from a plurality of feature amounts defining the teaching model. In this way, the detection of an object is stabilized by narrowing down to feature quantities effective for detecting the object.

  However, although there is a method for stabilizing the detection of an object in this way in the prior art, there is a problem that it is difficult to determine what kind of masking is appropriate. For example, in the method using the normalized cross-correlation coefficient, if the cross-correlation coefficient value or information on which part of the template image is high or low is not presented to the operator, which part of the template image is indicated. Since there is no clue as to whether or not to mask, the operator cannot adjust the teaching model (ie, masking). In addition, even if the information on the cross-correlation coefficient value or information on which part of the template image is high or low is output, the operator relies on the above information on the unstable detection result that has occurred. , Guessing which part of the template image will be stabilized by masking, the teaching model must be adjusted on a trial and error basis, which is not only difficult to adjust optimally but also takes time and effort There is a problem. This is the same in the method using the feature amount representing the geometric feature.

  Therefore, an object of the present invention is to solve the problems existing in the prior art and to automatically correct a teaching model used when detecting an object from an input image.

In order to achieve the above object, according to the present invention, based on the degree of correlation between a feature amount model of a detection target that is defined in advance using a plurality of feature amounts and an input image acquired by an imaging device, An image processing apparatus for detecting an image of the detection object from within the input image, the matching processing means for performing a correlation degree calculation process for obtaining the correlation degree, and after performing the correlation degree calculation process, For each of the plurality of feature quantities of the feature quantity model, a contribution degree to the increase in the correlation degree in the immediately preceding correlation degree calculation process is calculated, and a contribution index is obtained based on a value obtained by accumulating the calculated contribution degrees The contribution index based on a value obtained by calculating the contribution for each of the plurality of feature quantities of the feature quantity model with respect to a plurality of different input images and a contribution contribution index calculation means Seeking Thereafter, for each of the plurality of feature quantities of the feature quantity model, it is determined whether or not the calculated contribution index is less than or equal to a predetermined threshold value, and the contribution index is less than or equal to a predetermined threshold value In this case, there is provided an image processing apparatus comprising: a feature amount model correcting unit that corrects the feature amount model by excluding the feature amount from the plurality of feature amounts of the feature amount model .

  In the image processing apparatus of the present invention, how much each of a plurality of feature quantities defining a feature quantity model of a detection target object has a degree of correlation between the feature quantity model used when detecting the detection target object from the input image and the input image A necessary feature amount is selected from a plurality of feature amounts by using a contribution level indicating whether the contribution is made. Therefore, it is possible to automate the correction of the feature amount model.

  In the image processing apparatus of the present invention, the contribution index is stored by accumulating the contribution to the increase in the degree of correlation for each of the plurality of feature quantities defining the feature quantity model (that is, the teaching model) through a plurality of correlation degree calculation processes. Since it is stored in the means, it is possible to rationally and statistically evaluate the contribution to the detection of the detection object for each of the plurality of feature amounts defining the feature amount model. In addition, when correcting a feature model, features whose stored cumulative index is equal to or less than a predetermined threshold are automatically excluded from a plurality of feature values that define the feature model as unnecessary feature values. Therefore, it becomes possible to easily correct the feature amount model to a more appropriate one.

In the image processing apparatus, the plurality of feature amounts can be luminance values of pixels of an image including the detection target.
For example, a template image representing the feature amount model is represented by M _T × N _T pixels, the correlation degree is C, the luminance value at each pixel of the template image is f (x, y), and the input image When the luminance value at each pixel is represented by g (x, y), and the area corresponding to the size of the template image when the center of the template image is at the coordinates (m, n) of the input image is represented by D. The correlation is expressed by the following equation.

  In addition, the plurality of feature amounts may represent the geometric features of the detection target by numerical values.

  According to the present invention, the feature quantity model is corrected by selecting each feature quantity based on the contribution to the correlation degree of each of the feature quantities defining the feature model, that is, the teaching model. The modification of the quantity model can be automated.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
First, the overall configuration of the image processing apparatus 10 of the present invention will be described with reference to FIG. The image processing apparatus 10 includes a CPU (Central Processing Unit) 12 for overall control. A plurality of frame memories 16, a ROM (read only memory) 18, a RAM (random access memory) 20, and a nonvolatile RAM 22 are connected to the CPU 12 via a bus 14. A camera 26 is connected to the bus 14 via a camera interface 24 and a monitor 30 is connected via a monitor interface 28. Furthermore, an external device interface 32 is connected to the CPU 12 via the bus 14.

  The ROM 18 stores programs for various processes performed by the image processing apparatus 10, and the volatile RAM 22 stores settings necessary for executing the programs. The RAM 20 stores temporary saving data necessary for executing the program, and the frame memory 16 stores image data.

  The camera 26 captures an object and acquires an image in accordance with a command from the CPU 12, and outputs a signal related to the acquired image. The camera interface 24 has a function of generating a synchronization signal for controlling the exposure timing for the camera 26 in accordance with a command from the CPU 12 and a function of amplifying a signal received from the camera 26. Such a camera 26 and a camera interface 24 are generally commercially available and are not particularly limited.

  Signals relating to images captured from the camera 26 are A / D converted at the camera interface 24 and temporarily stored in the frame memory 16 as digital image data via the bus 14. In the image processing apparatus 10, the CPU 12 performs image processing using data stored in the frame memory 16, ROM 18, RAM 20, and nonvolatile RAM 22, and the result data of the image processing is stored in the frame memory 16 again. In response to the command, the CPU 12 transfers the data stored in the frame memory 16 to the monitor interface 28 and displays it on the monitor 30 so that an operator or the like can check the contents of the data.

  The external device interface 32 is connected to various external devices. For example, a robot 34 is connected to the external device interface 32, receives a trigger signal for image processing from the robot 34, and supplies data obtained by the image processing to the robot 34. The external device interface 32 is connected to a keyboard, mouse, or the like as an input device 33 for an operator.

  Specifically, as shown in FIG. 2, the CPU 12 includes a matching processing unit 35, a feature amount contribution degree calculating unit 36, a feature amount contribution degree adding unit 38, and a feature amount model correcting unit 40. .

The matching processing unit 35 is obtained by a feature amount model in which a detection target is defined in advance using a plurality of feature amounts and stored in a feature amount model storage unit 42 such as the volatile RAM 22 and an imaging device such as the camera 26. The degree of correlation between the feature model and the input image is calculated by comparing the input image with the input image, matching processing is performed based on the calculated degree of correlation, and an image of the detection target is detected from the input image.

  In addition, the feature amount contribution degree calculating unit 36 increases the correlation degree in the immediately preceding correlation degree calculating process for each of the plurality of feature amounts of the feature amount model in consideration of the result of the correlation degree calculating process by the matching processing unit 35. The degree of contribution to is calculated according to a predetermined method. An example of how to calculate the contribution will be described later. The feature amount contribution degree adding unit 38 calculates the contribution index stored in the feature amount contribution index storage unit 44 such as the volatile RAM 22 by the feature amount contribution degree calculating unit 36 for each of the plurality of feature amounts of the feature amount model. The contribution index is updated by adding the contributions, and the updated contribution index is stored in the feature amount contribution index storage unit 44. The feature amount contribution calculating means 36, the feature amount contribution adding means 38, and the feature amount contribution index storing means 44 are contribution indexes representing the degree of contribution to the increase in the degree of correlation for each of the plurality of feature amounts of the feature amount model. The feature amount contribution index calculating means for calculating the.

  On the other hand, the feature quantity model correcting means 40 determines in advance a contribution index stored in the feature quantity contribution index storage means 44 for each of a plurality of feature quantities of the feature quantity model when correction of the feature quantity model is desired. When the contribution index is determined to be smaller than a predetermined threshold value, the feature amount model is automatically excluded by excluding the feature amount from a plurality of feature amounts specifying the feature amount model. Correct it.

  Next, an overall flow of processing performed in the image processing apparatus 10 shown in FIG. 1 will be described with reference to FIGS. 3 and 4. It is assumed that a feature amount model for specifying a detection target defined by using imax feature amounts is previously taught in the image processing apparatus 10 as a teaching model. For example, in the case of detection of a detection target using a normalized cross-correlation coefficient, the feature amount model is defined by the luminance value of each pixel of the template image including the detection target, and is converted into a geometric feature. In the case of detection of a detection target based on this, it is defined by a feature amount representing a geometric feature. Details of these feature amount models will be described later.

  First, parameters used in processing in the image processing apparatus 10 are initialized. Next, as shown in FIG. 3, an input image including an object in the field of view is acquired by the camera 26 which is an imaging device, and A / D converted by the camera interface 24, and then framed as digital image data. It is temporarily stored in the memory 16 (step S100). The input image stored in the frame memory 16 is subjected to image processing as necessary, and the matching processing means 35 performs matching processing using the feature quantity model (teaching model) taught in advance and the input image ( Step S102). When the matching is not established, that is, when the detection of the detection target image fails, it is determined that the detection target image is not included in the acquired input image, and the next input image is acquired. On the other hand, when the matching is established, that is, when the detection target image is successfully detected, it is determined that the detection target image is included in the input image, and the detection target object is detected from the input image by image processing. The position, posture, etc. are specified, the position, posture, etc. of the specified detection object are sent to a robot control device or the like, and predetermined processing is performed on the detection object using the robot 34 or the like. When all of the above processing for one input image is completed, a new input image is acquired and the same processing is repeated.

  In the image processing apparatus 10 according to the present invention, when matching is established in step S102, the degree of correlation in the matching process performed immediately before by the matching processing unit 35 for each of the plurality of feature quantities defining the feature quantity model (teaching model). The degree of contribution to the increase (that is, the degree of contribution to matching) is calculated and stored. Specifically, the feature amount contribution degree calculating means 36 calculates a contribution value Ct representing the contribution degree to matching for the i-th feature quantity of the feature amount model (step S104), and is stored in the contribution index storage means 44. The contribution index P (i) is updated by adding the calculated contribution value Ct to the contribution index P (i), and the updated contribution index P (i) is stored in the feature amount contribution index storage means 44. (Step S106). The processes in step S104 and step S106 are repeated for all imax feature values (step S108). When the above procedure is performed after one matching process, the same procedure is repeated after the matching process for the next input image.

  When correction of the feature quantity model (teaching model) is desired, for example, when detection of the detection target is not stable, the feature quantity model correcting means 40 automatically corrects the feature quantity model. Specifically, when it is desired to correct the feature amount model, the feature amount model correcting means 40 contributes the feature amount at that time to the i-th feature amount of the feature amount model, as shown in FIG. The contribution index P (i) stored in the index storage means 44 is compared with the threshold value Th (step S200). When the contribution index P (i) is smaller than the threshold value Th, the i-th feature amount is calculated as a correlation degree. It is determined that the degree of contribution to matching in the process is small, and the feature quantity is excluded from imax feature quantities defining the feature quantity model (step S202), and a variable d representing the number of excluded feature quantities is set. Increase by one. On the other hand, when the contribution index P (i) is greater than or equal to the threshold Th in step S200, the i-th feature amount is left without being excluded.

  The processes in steps S200 and S202 are repeated for all imax feature values (step S204). When the above processing is performed on all imax feature quantities, d feature quantities are excluded from the imax feature quantities defining the feature quantity model. The number is (imax-d) (step S206), and the corrected feature value model is defined by the (imax-d) feature values, and the detection target is detected based on the corrected feature value model. You can continue.

  Since feature quantities excluded in such automatic feature model correction processing have little contribution to the increase in the degree of correlation, there is little impact on the rate of matching even if they are excluded. Since the elements that become unstable are reduced, the detection of the detection object is stabilized.

  Next, with reference to FIG. 5 and FIG. 6, the details of the matching process using the cross-correlation coefficient normalized as an index representing the degree of correlation and the procedure of the automatic correction process of the feature amount model in this case will be described. . In matching processing using normalized cross-correlation coefficients, that is, template matching, a template image is stored in the feature amount model storage unit 42 as a feature amount model.

In FIG. 5, an area composed of M × N pixels indicated by f represents the input image acquired by the camera 26, with the upper left as the origin (0, 0), the downward in the vertical direction as the X axis, and the horizontal direction When the right direction is defined as the Y axis, the luminance value of the pixel at the position (x, y) is represented by f (x, y). Further, an area composed of M _T × N _T pixels indicated by g represents the template image, and the pixel at the position (x, y) when the X axis and the Y axis are defined in the same manner as the input image. Is represented by g (x, y). Here, the size of the input image is larger than the template image, that is, M _T <M and N _T <N. In FIG. 5, the template image is drawn so as to overlap the input image with the center being at the position (m, n) of the input image. The region D occupied by the template image on the input image is represented as a range (i, j) that satisfies | i−m | ≦ M _T / 2 and | j−n | ≦ N _T / 2.

Whether or not there is an image of the detection object in the region D centered on the position (m, n) shown in FIG. 5 is based on the value of the cross-correlation coefficient C obtained by the following equation (1). Is judged.

  In the matching process using the normalized cross-correlation coefficient C, the template image is normalized while moving the center of the template image to all the pixels on the input image that do not protrude from the input image area. The cross-correlation coefficient C is obtained, and it is determined that an image of the detection target exists at a position where the cross-correlation coefficient C is the largest or a position where the cross-correlation coefficient C is larger than a predetermined threshold.

したがって、検出対象物の像が位置すると判断されたときの入力画像上におけるテンプレート画像の中心位置を（ｍ_d，ｎ_d）とすると、以下の式（３）によって求められる値Ｃｔは、入力画像上の位置（ｐ，ｑ）（∈Ｄ）にある画素の輝度値の相互相関係数Ｃへの貢献度を表すことになる。なお、式（３）中のＣ₀は定数を表す。

Here, in the expression (1), the part of the following expression (2) is the square root of the square sum of the luminance value of each pixel of the template image, and therefore a constant value (positive value) regardless of (m, n). ) T.

Therefore, when the center position of the template image on the input image when it is determined that the image of the detection target is located is (m _d , n _d ), the value Ct obtained by the following equation (3) is The degree of contribution of the luminance value of the pixel at the upper position (p, q) (εD) to the cross-correlation coefficient C is expressed. Incidentally, C ₀ in the formula (3) represents a constant.

This is because the sum of the values Ct obtained for all (p, q) belonging to D is a constant multiple of the value of the cross-correlation coefficient C, and the larger the value Ct obtained by Equation (3), the more Ct It can be seen from the fact that the ratio of the correlation coefficient C to the value increases. Equation (3) also represents the degree of contribution to the cross-correlation coefficient C of the feature amount represented by the luminance value of the pixel at the position (p- _md , q- _nd ) of the template image. You will understand. Therefore, by calculating the value Ct for each pixel of the template image and using the accumulated value as the contribution index P, it is possible to evaluate the degree of contribution to the cross-correlation coefficient C of each pixel of the template image. become.

Specifically, for example, as shown in FIG. 6, a memory area corresponding to M _T × N _T pixels of the template image is prepared, and the value of the memory corresponding to each pixel is set to 0 at a certain time. Each time the detection target image included in the template image is detected in the input image, that is, each time the position (m _d , n _d ) is obtained, the contribution value for each pixel of the template image Ct may be obtained and added to each memory. Instead of simply adding the contribution value Ct, the number of additions may be stored separately, and the average value of the contribution may be stored in the memory.

FIG. 6 shows the contribution index P corresponding to each pixel of the template image at a certain time point, that is, the cumulative value or the average value of the contribution degree Ct. Since the magnitude of each value is dependent on the value of C ₀ of the formula (3), the relative size between the size than itself numeric becomes important. The numerical value stored in the memory area shown in FIG. 6, that is, the contribution index P, has a statistical meaning because it is obtained by adding the contribution degree Ct regarding the detection object for a plurality of times. The larger the value, the higher the degree to which the corresponding pixel of the template image contributes to the detection of the detection object. Conversely, the smaller the value, the more the corresponding pixel of the template image detects the detection object. It means that the degree to contribute to is low. It is preferable to display the value of the contribution index P shown in FIG. 6 in a pictorial manner because the degree of contribution of each pixel can be intuitively understood. For example, color display may be performed such that the larger the numerical value is, the closer the pixel color is to red, and the smaller the numerical value is, the closer the pixel color is to blue.

When an appropriate threshold Th is set for the contribution index P as shown in FIG. 6 and the contribution index P corresponding to each pixel of the template image is smaller than the threshold Th, the pixel is detected. It is determined that the degree of contribution to the object detection is low and masked, that is, excluded from the calculation target in the calculation of the cross-correlation coefficient C according to the equation (1). Thereby, the feature amount model is automatically corrected. Specifically, if the position of the pixel to be masked is (i _m , j _m ) and the template image corresponding to the feature model after correction by masking is g ^* , the position of the template image after correction (x, The function g ^* (x, y) and the cross-correlation coefficient C representing the luminance value of the pixel in y) are expressed as follows.
When (x, y) = (i _m , j _m ), g ^* (x, y) = 0
When (x, y) ≠ (i _m , j _m ), g ^* (x, y) = g (x, y)

  Therefore, if the equation (4) is used instead of the equation (1), the subsequent detection target can be detected by the automatically corrected feature quantity model.

  Next, with reference to FIGS. 7 to 11, details of the matching process using the geometric feature and the procedure of the automatic correction process of the feature amount model in this case will be described.

  As a general method for detecting a geometric shape from an input image, a method using a generalized Hough transform is known. Generalized Hough transform is a technique of voting and majority voting where the geometric features (contour shape) of a detection target are defined in advance as feature models and where these geometric features are in the input image area. It is something to find.

  The generalized Hough transform uses a mapping that converts the coordinate space of the input image into a parameter space whose coordinate axis is the pose (position, orientation, size) of the detection target, and the parameter space is converted into an element called a cell. The object in the input image is subjected to a process of voting to the cells in the parameter space corresponding to all the feature quantities belonging to the feature quantity model for each feature quantity representing the geometric features of the object image in the input image after being decomposed When voting is completed for all combinations of image feature quantities (geometric features) and feature quantity feature quantities, a cell having the largest vote count is extracted. If the vote count is equal to or greater than a predetermined threshold, it is determined that an image of the detection target defined in the feature model is detected in the input image, and if the vote count is smaller than the predetermined threshold, It is determined that the image of the detection object has not been detected in the input image.

  As a specific example, a case will be described in which an image of an object having an outline shape as shown in FIG. 7 is detected from an input image. Here, in order to simplify the description, an object having a relatively simple shape is illustrated, but even an object having a more complicated shape may be processed in the same procedure.

  First, in order to extract the feature amount of the detection target object, preprocessing of a template image including an image of an object having a contour shape as shown in FIG. 7 is performed. For example, the template image is converted to an edge image by applying a Sobel filter. Next, pixels having an intensity equal to or higher than a predetermined threshold are extracted from the edge image (generally referred to as edge points), and the positions and shade gradient directions of the extracted imax edge points are obtained. The method for obtaining the edge point using the Sobel filter is described in “Detection of Edges and Lines” on pages 280 to 286 of “Image Handbook” (Shojido, first edition of 1987). Since it is a general technique, it will not be described in detail here.

Furthermore, an arbitrary point in the contour line shape shown in FIG. 7, for example, the centroid of the geometric shape is set as a reference point O (O _x , O _y ), and the position and gray scale of each of the imax edge points are set. From the direction information, the vector X _i (Vx _i , Vy _i ) from each edge point (x _i , y _i ) to the reference point O (O _x , O _y ) and each edge point (x _i , y _i ) A density gradient direction ψ _i is obtained, and a feature amount model of the detection target is defined as an aggregate of feature amounts (Vx _i , Vy _i , ψ _i ). It can be said that this feature model is a numerical representation of the geometric feature of the image of the detection object. Specifically, as shown in FIG. 8, an “R table” in which the coordinate values (Vx _i , Vy _i ) of the vector X _i are registered for each density gradient direction ψ _i is created, and this is used as a feature quantity. Model. _{Here, Vx i = O x -x i} , a Vy _{i =} O y -y _i. By creating the “R table”, the reference point O in the input image when it is assumed that the feature quantity of the edge point extracted from the input image corresponds to one of the feature quantities constituting the feature quantity model of the detection target object. The position of can be obtained.

Next, preprocessing of the input image is performed in the same manner to extract the feature quantity of the object image in the input image, and each feature quantity is voted on a cell in the parameter space. Taking the case where the size of the image of the detection object shown in the input image does not change as an example, the parameter space is defined in three dimensions (u, v, θ). Voting is performed for each feature amount (Vx _i , Vy _i , ψ _i ) of the feature amount model with respect to the gradient gradient direction ψ _j of the feature amounts (x _j , y _j , ψ _j ) of all edge points in the input image. ), The position (u, v) indicated by the reference point O of the feature quantity model is obtained by rotating the shade gradient direction ψ _i by θ = ψ _j -ψ _i to match the shade gradient direction, and the corresponding three-dimensional This is done by increasing the vote count of the cell at the point (u, v, θ) on the parameter space by one. Each feature quantity of the feature quantity model is (Vx _i , Vy _i , ψ _i ) (where i = 1,..., N), and feature quantities of edge points in the input image are (x _j , y _j , ψ _j ) (Where j = 1,..., M), the point (u, v, θ) on the parameter space for the feature quantity of each edge point in the input image is obtained by the following equation. Here, R (θ) means a rotation matrix.
θ = ψ _j −ψ _{i
Vx i = O x -x i,} Vy i = O y -y i
[U v] ^T = R (θ) [Vx _i Vy _i ] ^T + [x _j y _j ] ^T (5)

When the size s of the object to be detected is taken into consideration, the parameter space is a four-dimensional space of (u, v, θ, s), but the vector (Vx _i , Vy _i ) in the above equation (5). (U, v) may be obtained by an expression obtained by multiplying the coefficient s by the magnitude s, and the basic idea is the same.

When voting to the parameter space with respect to the feature values of all edge points in the input image is completed, if the cells having a high voting frequency among the cells in the parameter space are (u ₀ , v ₀ , θ ₀ ), It is estimated that an image of the detection target represented by the feature amount model with the posture θ ₀ is reflected at the position (u ₀ , v ₀ ) in the input image. A value obtained by dividing the voting frequency by the total feature quantity of the feature quantity model (hereinafter referred to as score S) is a numerical value corresponding to the normalized cross-correlation coefficient, that is, the image of the object in the input image and the feature quantity model. If the score S is greater than or equal to a predetermined threshold value, the image of the detection object represented by the feature amount model has a score S. If it is determined that the pose corresponding to the cell is detected in the input image, and the score S is smaller than a predetermined threshold value, it is determined that the image of the detection target is not detected in the input image. The score S takes a value from 0 to 1, while the threshold value can be set to an appropriate value from 0 to 1. If this threshold value is set too high, detection may not be possible due to noise or the like in the input image, while if it is set too low, erroneous detection may occur. Therefore, about 0.7 is suitable although it depends on conditions.

  Next, when it is determined that an image of the detection target is detected in the input image, it is evaluated which feature quantity of the feature quantity model contributes to the detection, and a contribution index P of each feature quantity of the feature quantity model is determined. Ask. For this purpose, first, a memory area corresponding to each feature quantity of the feature quantity model is prepared, and the value of the memory corresponding to the feature quantity is cleared to 0 at a certain time, and the image of the detection target is detected in the input image. Each time, each feature quantity of the feature quantity model is evaluated as to whether or not the feature quantity contributed to the detection. If it contributes, a predetermined constant (for example, 1) is added to the memory as a contribution value Ct. I'll do it. The contribution index P may be a memory value itself, or may be a so-called average value obtained by separately counting the number of additions and dividing the value stored in the memory by the number of additions.

Whether a certain feature amount (Vx _i , Vy _i , ψ _i ) of the feature amount model contributes to voting for a certain pose (u, v, θ) can be evaluated as follows. That is, (Vx _i , Vy _i , ψ _i ) and (u, v, θ) are substituted into the above-described equation (5), and (x _j , y _j , ψ _j ) is obtained by back calculation. It can be determined that if the feature quantity matching the obtained (x _j , y _j , ψ _j ) is an edge point in the input image, it contributes to detection.

The contribution index P of each feature quantity (Vx _i , Vy _i , ψ _i ) of the feature quantity model has a statistical meaning because it contributes the contribution values related to multiple detections of the detection target. The larger the value, the higher the degree to which the feature quantity of the corresponding feature quantity model contributes to the detection of the image of the detection object. Conversely, the smaller the value, the more the corresponding feature quantity is the detection object. This means that the degree of contribution to image detection is low. For example, when the figure as shown in FIG. 9 is used as the outline shape of the detection object, the radius of the smaller circle is circle 2 in FIG. 9 (dotted line in FIG. 10), as shown in FIG. 2), the contribution of the feature quantity corresponding to the circle 2 is smaller than the contribution of other feature quantities included in the feature model.

  Then, as illustrated in FIG. 11, when an appropriate threshold value Th is set for the contribution index P at a certain point in time, and the contribution index P corresponding to a certain feature quantity in the feature quantity model is smaller than the threshold Th, The feature amount is determined to have a low degree of contribution to the detection of the image of the detection target, is excluded from the feature amount model, and is also excluded from the calculation target in the calculation of the score S. In this way, the feature amount model is automatically corrected.

  As described above, the detection of the detection target using the normalized cross-correlation coefficient and the detection of the detection target using the geometric feature are examples of automatic correction of the feature amount model in the image processing apparatus 10 of the present invention. Although the procedure has been described, the image processing apparatus 10 according to the present invention is limited to one that detects a detection target using the normalized cross-correlation coefficient C and detects a detection target using a geometric feature. However, any type of image processing apparatus can be applied as long as it performs a type of matching process that uses a correlation degree for matching between a feature amount model and an input image. Moreover, in the said embodiment, although the calculation method of the contribution index P is illustrated, the calculation method of the contribution index P is not limited to the illustrated method, It is possible to determine suitably.

1 is a block diagram illustrating an overall configuration of an image processing apparatus of the present invention. It is a block diagram which shows the detailed structure of CPU of the image processing apparatus shown by FIG. It is a flowchart which shows the flow of the matching process by the image processing apparatus of this invention, and a contribution index calculation process. It is a flowchart which shows the procedure of the automatic correction | amendment of the feature-value model by the image processing apparatus of this invention. It is a diagram which shows the positional relationship of the input image at the time of performing template matching, and a template image. It is explanatory drawing which shows the memory area | region for calculating | requiring the contribution index | exponent of each pixel of a template image. It is a diagram which shows the example of the outline shape of the detection target used for the matching process using a geometric feature. It is the table | surface which showed R table produced by generalized Hough transform. It is a diagram which shows another example of the outline shape of the detection target used for the matching process using a geometric feature. FIG. 10 is a diagram showing an example in which a geometric shape slightly different from the detection target shown in FIG. 9 is detected. It is the table | surface which showed the contribution index | exponent of each feature-value of the outline shape of a detection target.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 35 Matching processing means 36 Feature quantity contribution degree calculation means 38 Feature quantity contribution degree addition means 40 Feature quantity model correction means 42 Feature quantity model storage means 44 Feature quantity contribution index storage means

Claims (4)

An image of the detection target is detected from the input image based on a degree of correlation between a feature amount model of the detection target defined in advance using a plurality of feature amounts and the input image acquired by the imaging device. An image processing apparatus,
Matching processing means for performing correlation degree calculation processing for obtaining the correlation degree;
After performing the correlation degree calculation process, for each of the plurality of feature quantities of the feature quantity model, calculate a contribution degree to the increase in the correlation degree in the immediately preceding correlation degree calculation process, and calculate the calculated contribution degree A feature value contribution index calculating means for calculating a contribution index based on the accumulated value;
After calculating the contribution degree for each of the plurality of feature amounts of the feature amount model for a plurality of different input images, and determining the contribution index based on a value obtained by accumulating the contribution degrees, the feature For each of the plurality of feature quantities of the quantity model, it is determined whether or not the calculated contribution index is equal to or less than a predetermined threshold, and when the contribution index is equal to or less than a predetermined threshold, An image processing apparatus comprising: a feature amount model correcting unit that corrects the feature amount model by excluding the feature amount from the plurality of feature amounts of the feature amount model . The image processing apparatus according to claim 1 , wherein the plurality of feature amounts are luminance values of pixels of an image including the detection target. The template image representing the feature amount model is represented by M _T × N _T pixels, the correlation degree is C, the luminance value in each pixel of the template image is f (x, y), and each pixel of the input image The area corresponding to the size of the template image when the luminance value at is g (x, y) and the center of the template image is at the coordinates (m, n) of the input image is expressed as D The image processing apparatus according to claim 2 , wherein the correlation degree C is represented by the following expression.

The image processing device according to claim 1 , wherein the plurality of feature amounts represent a geometric feature of the detection target by a numerical value.

Images