JP6438774B2 - Determination device, control program, and determination method - Google Patents

Description

  The present invention relates to image processing.

  As described in Patent Documents 1 and 2, various techniques have been proposed for image processing.

JP 2001-43371 A Japanese Patent No. 4175413

  When determining whether an object corresponds to a comparison object using image processing, it is desirable that the determination accuracy be high.

  Therefore, the present invention has been made in view of the above points, and a technique capable of improving the determination accuracy when determining whether or not an object corresponds to a comparison object using image processing. The purpose is to provide.

  One aspect of the determination apparatus according to the present invention is a determination apparatus that determines whether or not an object shown in a process target image corresponds to a comparison target, and the target object is reflected from a process target image showing the target object. A feature indicating characteristics of the object based on an extraction unit that extracts a target region, a region dividing unit that performs region division on the target region, and a plurality of divided regions generated by the region dividing unit A feature amount acquisition unit that acquires an amount; and a determination unit that determines whether or not the object corresponds to a comparison target based on the feature amount; and the region dividing unit includes the entire target region. A threshold value to be used in the region division is determined based on the entropy of the luminance histogram, and in the region division, the entropy of the luminance histogram in the divided region of the object region is greater than or equal to the threshold value or the threshold value. value Further repeating the process of dividing the divided regions into a plurality of divided areas when also large Ri.

  Further, in one aspect of the determination apparatus according to the present invention, the feature amount acquisition unit is configured to perform the feature based on information indicating a size of the divided region for each of the plurality of divided regions generated by the region dividing unit. Get the quantity.

  Moreover, in one aspect of the determination apparatus according to the present invention, the feature amount acquisition unit is a distance from the center of the object region to the division region for each of the plurality of division regions generated by the region division unit. The feature amount is acquired based on the information indicating.

  In the aspect of the determination apparatus according to the present invention, the feature amount acquisition unit uses a one-dimensional histogram of the information as the feature amount.

  In the aspect of the determination apparatus according to the present invention, the feature amount acquisition unit includes first information indicating a size of the divided region for each of the plurality of divided regions generated by the region dividing unit, The feature amount is acquired based on the second information indicating the distance from the center of the object area to the divided area for each of the divided areas.

  In the aspect of the determination apparatus according to the present invention, the feature amount acquisition unit uses the two-dimensional histogram of the first information and the second information as the feature amount.

  Further, in one aspect of the determination apparatus according to the present invention, the determination unit compares the feature quantity with a standard feature quantity indicating the feature to be compared, and based on a result of the comparison, the object is the It is determined whether it corresponds to the comparison target.

  Further, in one aspect of the determination apparatus according to the present invention, the processing target image has a constant orientation in a circumferential direction of the target object shown in the processing target image, and the determination unit indicates the characteristics of the comparison target. Each of a plurality of types of standard feature values is compared with the feature value, and based on the comparison result, it is determined whether or not the object corresponds to the comparison target. , Each of which is generated based on a plurality of images in which the comparison object is reflected, and the circumferential direction of the comparison object in the image is different between the plurality of images.

  An aspect of the control program according to the present invention is a control program for causing a computer device to determine whether or not an object shown in a processing target image corresponds to a comparison target. a) a step of extracting a target object region in which the target object is shown from a processing target image in which the target object is shown; (b) a step of dividing the target object region; and (c) the step (b). A step of obtaining a feature amount indicating the feature of the target object based on the plurality of divided regions generated in step (d), and (d) whether the target object corresponds to a comparison target based on the feature amount. And determining a threshold value used in the region division based on the entropy of the luminance histogram in the entire object region in the step (b), and dividing the object region. In the area If the entropy of the degree histogram is greater than the threshold value or more or the threshold value is for repetitively perform a process further divides the divided regions into a plurality of divided regions.

  Moreover, one aspect of the determination method according to the present invention is a determination method in a determination device that determines whether or not an object shown in a processing target image corresponds to a comparison target, and (a) a process in which the target object is shown A step of extracting a target region in which the target object is reflected from a target image; (b) a step of performing region division on the target region; and (c) a plurality of divisions generated in step (b). Obtaining a feature amount indicating a feature of the object based on a region; and (d) determining whether the object corresponds to a comparison target based on the feature amount, In the step (b), a threshold value used in the region division is determined based on the entropy of the luminance histogram in the entire object region, and the entropy of the luminance histogram in the divided region of the object region is Above the threshold or Is greater than the serial threshold, the process further divides the divided regions into a plurality of divided regions is repeated.

  According to one aspect of the present invention, the determination accuracy when determining whether or not an object shown in a processing target image corresponds to a comparison target is improved.

It is a figure which shows the structure of an image processing system. It is a figure which shows an example of a captured image typically. It is a figure which shows the structure of a determination apparatus. It is a flowchart which shows operation | movement of a determination apparatus. It is a figure which shows an example of a candy area | region typically. It is a figure which shows an example of a background image typically. It is a figure which shows an example of a background difference image typically. It is a figure which shows an example of an external shape template typically. It is a figure for demonstrating template matching. It is a figure for demonstrating the extraction process of a candy area | region. It is a flowchart which shows operation | movement of an area | region division part. It is a figure which shows an example of the brightness | luminance histogram in a confectionery area | region. It is a figure which shows a mode that a confectionery area | region is divided | segmented. It is a figure which shows a mode that a division area is further divided | segmented. It is a figure which shows an example of the several uniform division area produced | generated by an area division part. It is a figure which shows an example of the two-dimensional histogram of magnitude | size information and distance information. It is a figure which shows an example of the one-dimensional histogram of magnitude | size information. It is a figure which shows the some confectionery area | region from which the direction of the circumferential direction of a candy differs mutually. It is a figure which shows a mode that the center of the candy area | region and the center of the candy reflected in the said candy area | region have shifted | deviated. It is a figure which shows the confectionery area | region from which standard feature-value is acquired. It is a figure which shows the relationship between a 1st determination evaluation value and a 2nd determination evaluation value.

<Configuration of Image Processing System 1>
FIG. 1 is a diagram illustrating a configuration of an image processing system 1 according to an embodiment. The image processing system 1 according to the present embodiment is a system that determines whether an object corresponds to a comparison object by using image processing. For example, the image processing system 1 is introduced into a production line in which a circular candy such as a cookie or chocolate is manufactured in a factory. The image processing system 1 determines whether the confectionery manufactured in the production line in operation corresponds to a confectionery manufactured correctly. In other words, the image processing system 1 determines whether or not the candy manufactured on the production line in actual operation is manufactured correctly, that is, the quality of the candy. Therefore, in this Embodiment, a target object is a candy. Moreover, the comparison object compared with a target object is the confectionery (good quality confectionery) manufactured correctly. The object may be other than sweets. Further, the outer shape of the object may be other than a circle.

  As shown in FIG. 1, the image processing system 1 includes an imaging device 2 and a determination device 3. The imaging device 2 images the confectionery manufactured on the manufacturing line, generates a captured image 10 in which the confectionery is reflected, and outputs the captured image 10 to the determination device 3. In the production line, the confectionery is conveyed by the conveyor, and the imaging device 2 images the confectionery being conveyed. A plurality of manufactured confectionery moves in a line on the transport conveyor in an irregular direction. In the present embodiment, the captured image 10 generated by the imaging device 2 is a color image, but may be a grayscale image.

  FIG. 2 is a diagram schematically illustrating an example of a captured image 10 obtained by the imaging apparatus 2. The captured image 10 shown in FIG. 2 shows a properly manufactured candy 100, that is, a non-defective candy 100. Further, the picked-up image 10 shows the conveyor 101 that is the background. On one main surface 100a of the non-defective confectionery 100, for example, a pattern of alphabet “A” is shown. The pattern of the non-defective candy 100 may be other than this. Moreover, the pattern may be attached | subjected to both the main surfaces of the good confectionery 100.

  In the present embodiment, the confectionery 100 moves on the conveyor 101 with its one main surface 100a facing upward. On the conveyor 101, a plurality of sweets 100 move in a line. And the imaging device 2 images the said candy 100 from the one main surface 100a side of the candy 100. FIG. Therefore, the captured image 10 shows the one main surface 100a of the candy 100.

  The determination device 3 determines whether or not the candy 100 reflected in the captured image 10 input from the imaging device 2 corresponds to a correctly manufactured product, that is, whether or not the candy 100 is correctly manufactured. The determination result is output. The determination result is input to a control unit that manages the operation of the production line provided in the factory. The control unit performs various operations based on the input determination result. Hereinafter, a process in which the determination device 3 determines whether or not the candy 100 shown in the captured image 10 is correctly manufactured is referred to as a “candy determination process”.

  In the present embodiment, the determination device 3 is a kind of computer device, and includes a CPU (Central Processing Unit) 300 and a storage unit 310. The storage unit 310 includes a non-transitory recording medium that can be read by the CPU 300, such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 310 stores a control program 311 for controlling the determination device 3 (computer device). Various function blocks are formed in the determination apparatus 3 by the CPU 300 executing the control program 311 in the storage unit 310.

  Note that some or all of the various functions of the determination device 3 may be realized by a dedicated hardware circuit including a logic circuit or the like that does not require a program (software) to realize the function. The storage unit 310 may include a computer-readable non-transitory recording medium other than the ROM and RAM. The storage unit 310 may include, for example, a small hard disk drive and an SSD (Solid State Drive).

  FIG. 3 is a diagram illustrating a plurality of functional blocks included in the determination device 3. As illustrated in FIG. 3, the determination device 3 includes a conversion unit 30, an extraction unit 31, a region division unit 32, a feature amount acquisition unit 33, and a determination unit 34 as functional blocks.

  The conversion unit 30 converts the captured image 10 input from the imaging device 2 from a color image to a grayscale image, and outputs the converted captured image 10 as the captured image 11. The extraction unit 31 extracts a candy region (object region) 23 in which the candy 100 is reflected from the captured image 11.

  The area dividing unit 32 performs area division on the candy area 23 extracted by the extracting unit 31. Here, the area division is a process of dividing the confectionery area 23 into a plurality of divided areas each having a uniform luminance. In other words, the area division is a process of dividing the candy area 23 into a plurality of divided areas each having a uniform feature.

  The feature amount acquisition unit 33 acquires a feature amount indicating the feature of the candy 100 shown in the captured image 11 based on the plurality of divided regions generated by the region dividing unit 32. The determination unit 34 determines whether the candy 100 shown in the captured image 11 is correctly manufactured based on the feature amount acquired by the feature amount acquisition unit 33, and outputs the determination result 21. In other words, the determination unit 34 determines whether or not the candy 100 shown in the captured image 10 obtained by the imaging device 2 is a good product based on the feature amount acquired by the feature amount acquisition unit 33, and the determination result 21 is output. In the present embodiment, the determination unit 34 compares the feature amount obtained from the captured image 11 with the standard feature amount (reference feature amount) 22 indicating the feature of the non-defective candy 100, and based on the comparison result. Then, it is determined whether or not the candy 100 shown in the captured image 11 is correctly manufactured. The standard feature 22 is stored in advance in the storage unit 310.

<About the flow of candy determination processing>
Next, a series of operations of the determination device 3 when the determination device 3 performs the candy determination process while the production line is in actual operation will be described. FIG. 4 is a flowchart showing the candy determination process.

  As shown in FIG. 4, in step s1, when the captured image 10 is input from the imaging device 2, the determination device 3 converts the captured image 10 from a color image to a grayscale image in the conversion unit 30. Therefore, the captured image 11 obtained is set as a processing target. Hereinafter, the captured image 11 to be processed while the production line is in operation may be referred to as “processing target image 11”.

  Next, in step s <b> 2, the extraction unit 31 extracts a candy region 23 in which the candy 100 is reflected from the processing target image 11. Next, in step s3, the region dividing unit 32 performs region division on the candy region 23 extracted by the extracting unit 31. Next, in step s4, the feature amount acquisition unit 33 acquires a feature amount indicating the feature of the candy 100 shown in the captured image 11 based on the plurality of divided regions generated by the region dividing unit 32. Hereinafter, the feature amount indicating the feature of the candy 100 may be referred to as a “confectionery feature amount”.

  Next, in step s5, the determination unit 34 determines whether the candy 100 shown in the captured image 11 is correctly manufactured based on the candy feature amount acquired by the feature amount acquisition unit 33. And the determination part 34 outputs the determination result 21 in step s5 to the control part provided in the factory in step s6. When the candy 100 reflected in the captured image 10 is not manufactured correctly, in other words, when the candy 100 reflected in the captured image 10 does not correspond to a non-defective product, for example, the control unit outputs a warning sound from a speaker. Or display warning information on the display, etc. to issue a warning to the outside.

  Thereafter, when a new captured image 10 is input, the determination device 3 executes steps s2 to s6 using the captured image 11 obtained from the captured image 10 as a new processing target. Thereafter, the determination device 3 performs the same operation every time the captured image 10 is input.

<Detailed description of each component>
Hereinafter, operations of the extraction unit 31, the region division unit 32, the feature amount acquisition unit 33, and the determination unit 34 will be described in more detail.

<About the extraction unit>
FIG. 5 is a diagram illustrating an example of the candy region 23 extracted from the captured image 11 by the extraction unit 31. There are various methods for extracting the candy region 23 from the captured image 11.

  For example, there exists the 1st extraction method using the external shape of the candy 100 being circular. In the first extraction method, first, edge detection is performed on the captured image 11 to generate an edge image. As an edge image generation method, for example, Sobel method, Laplacian method, Canny method and the like are used. Next, a circular area is extracted from the generated edge image. For example, Hough transform is used as a method for extracting a circular region. And the circular area | region in the captured image 11 which exists in the same position as the position of the said circular area | region in an edge image is made into the candy area | region 23. FIG.

  As another method, there is a second extraction method for extracting the candy region 23 from the captured image 11 using the background subtraction method and labeling. In this second extraction method, first, a background difference image indicating a difference between the captured image 11 and a background image (an image showing only the background of the captured image 11) is generated, and the generated background difference image is binarized. The Then, labeling such as 4-connection is performed on the binary background difference image. And the partial area | region in the captured image 11 which exists in the same position as the position of the connection area | region (independent area | region) obtained as a result of labeling in a binary background difference image is made into the candy area | region 23. FIG.

  In the present embodiment, the extraction unit 31 extracts the candy region 23 from the captured image 11 by a method different from the above two methods. The operation of the extraction unit 31 according to this embodiment will be described below. In addition, the extraction part 31 may extract the candy area | region 23 from the captured image 11 using either one of said two methods.

  First, the extraction unit 31 generates a background difference image indicating a difference between the captured image 11 and the background image 60 (an image showing only the background of the captured image 11), and binarizes the generated background difference image. FIG. 6 is a diagram schematically showing the background image 60, and FIG. 7 is a diagram schematically showing a binary background difference image 61. The background image 60 is stored in advance in the storage unit 310 of the determination device 3.

  Next, the extraction unit 31 performs template matching on the binary background difference image 61 using a binary outer shape template 62 indicating the outer shape of the candy 100. That is, the extraction unit 31 specifies where in the background difference image 61 a region similar to the outer shape template 62 exists. In other words, the extraction unit 31 specifies where in the background difference image 61 there is an area that matches the outline of the candy 100 indicated by the outline template 62. FIG. 8 is a diagram schematically showing the outer shape template 62. The outline template 62 is stored in advance in the storage unit 310 of the determination device 3. The outer shape of the outer shape template 62 is, for example, a square.

  In template matching, the extraction unit 31 moves the outline template 62 little by little in the raster scan direction on the background difference image 61 as shown in FIG. In other words, the extraction unit 31 raster scans the outline template 62 on the background difference image 61. At this time, the extraction unit 31 generates an AND image of the outline template 62 and a partial region of the background difference image 61 that overlaps the outline template 62 at each position of the outline template 62. Thereby, a plurality of binary AND images are generated. Then, the extraction unit 31 uses the background difference image 61 of the outer shape template 62 used in generating the AND image having the largest number of pixels (high luminance pixels) having the pixel value “1” among the plurality of generated AND images. Identify the top position. This position is a position where an area similar to the outline template 62 exists in the background difference image 61. And the extraction part 31 extracts the partial area | region 11a in the captured image 11 which exists in the same position as the specified position as FIG. In other words, the extraction unit 31 sets, as the candy region 23, the partial region 11a in the captured image 11 that overlaps the external shape template 62 when the external shape template 62 is arranged in the captured image 11 at the same position as the specified position. Extract. At this time, in this embodiment, in the partial area 11a, the confectionery area 23 in which the pixel value of each pixel outside the circle indicated by the outer shape template 62 on the partial area 11a is set to zero is used. The confectionery region 23 extracted by the extraction unit 31 is a grayscale image. In the present embodiment, the outer shape of the confectionery region 23 is a square, but it may be a circle.

<Region division unit>
In the present embodiment, the region dividing unit 32 performs region division on the candy region 23 using the entropy of the luminance histogram. When the entropy of the luminance histogram in the divided area of the candy area 23 is equal to or greater than the threshold value, the area dividing unit 32 repeatedly performs a process of further dividing the divided area into a plurality of divided areas.

  FIG. 11 is a flowchart showing the operation of the area dividing unit 32. As shown in FIG. 11, in step s31, the region dividing unit 32 determines a threshold value used in region division. Specifically, the area dividing unit 32 obtains a luminance histogram in the entire candy area 23. Then, the region dividing unit 32 obtains the entropy of the obtained luminance histogram. Then, the region dividing unit 32 determines a threshold value based on the obtained entropy.

  FIG. 12 is a diagram illustrating an example of the luminance histogram 150 in the entire candy region 23. The luminance histogram 150 shows the luminance distribution of a plurality of pixels constituting the confectionery region 23. The horizontal axis in FIG. 12 indicates luminance. The vertical axis in FIG. 12 indicates the number of pixels having the luminance shown on the horizontal axis, that is, the frequency in the confectionery region 23. In the present embodiment, the luminance of the pixel is expressed by 8 bits, for example, and takes a value from “0” to “255” in decimal notation.

  The area dividing unit 32 obtains the entropy EA of the luminance histogram 150 using the following formula (1).

  Here, PAi in Equation (1) indicates the appearance probability of the luminance “i” in the confectionery region 23. The appearance probability PAi is expressed by the following equation (2) using the number KAi of pixels having luminance “i” in the candy area 23 and the total number KAtotal of pixels in the candy area 23.

  For example, assuming that KAtotal = 1000 and the number of pixels having a luminance of “255” KA255 = 10 in the confectionery region 23, PA255 = 10/1000.

  Entropy EA indicates the degree of variation in the distribution of luminance histogram 150. When the entropy EA is large, it can be said that the distribution of the luminance histogram 150 varies. That is, it can be said that the variation in luminance in the confectionery region 23 is large. On the other hand, when the entropy EA is small, it can be said that the distribution of the luminance histogram 150 is biased. That is, it can be said that the variation in luminance in the confectionery region 23 is small.

  When obtaining the entropy EA, the area dividing unit 32 obtains a threshold value T used in the area division using the following equation (3).

  Here, α in Expression (3) is an adjustment parameter for adjusting the division mode of the confectionery region 23, and 0 <α <1. As α becomes smaller, the threshold value T becomes smaller, so that the confectionery area 23 is easily divided in the area division. The value of α is determined according to the type of the object for which the pass / fail determination is performed in the determination device 3. The value of α is stored in the storage unit 310 in advance. In this Embodiment, the value of (alpha) is determined by experiment etc. so that the determination apparatus 3 can perform the quality determination of the candy 100 appropriately.

  After determining the threshold value T in step s31, the area dividing unit 32 initially divides the candy area 23 in step s32. Here, the confectionery area 23 is divided into four as shown in FIG. 13, for example. In this Embodiment, as FIG. 13 shows, the confectionery area | region 23 is equally divided | segmented into four division area | regions of a square in matrix form. The area dividing unit 32 does not use the entropy of the luminance histogram when initially dividing the candy area 23. Hereinafter, the divided area obtained by the initial division of the confectionery area 23 may be referred to as a “divided area 230”.

  Next, in step s33, the area dividing unit 32 determines a divided area to be processed. In step s33 immediately after step s32, each divided region obtained in step s32 is a processing target.

  Next, in step s34, the area dividing unit 32 obtains the entropy of the luminance histogram for each divided area to be processed determined in step s33. Specifically, the area dividing unit 32 first obtains a luminance histogram of the divided area to be processed. The luminance histogram of the divided area indicates the distribution of the luminance of the plurality of pixels constituting the divided area. Then, the region dividing unit 32 obtains the entropy EB of the luminance histogram of the division region to be processed using the following equation (4).

  Here, PBi in Expression (4) indicates the appearance probability of the luminance “i” in the divided area. The appearance probability PBi is expressed by the following equation (5) using the number KBi of pixels having luminance “i” in the divided area and the total number KBtotal of pixels in the divided area.

  When the entropy EB for a divided area is large, it can be said that the luminance variation in the divided area is large. On the other hand, when the entropy EB for the divided area is small, it can be said that the variation in luminance in the divided area is small.

  When step s34 is executed, in step s35, the region dividing unit 32 determines whether there is a processing target divided region having an entropy EB equal to or greater than the threshold T in the processing target divided region determined in step s33. Determine whether. If the area dividing unit 32 determines in step s35 that there is a divided area to be processed whose entropy EB is equal to or greater than the threshold value T, in step s36, the area dividing unit 32 further determines a divided area to be processed whose entropy EB is equal to or greater than the threshold value T. To divide. As described above, when the entropy EB for a divided region is large, it can be said that the luminance variation in the divided region is large. Therefore, the region dividing unit 32 further selects a processing target divided region having a large luminance variation. It can be said that it is divided. In other words, the area dividing unit 32 further divides a divided area to be processed whose features are not uniform. In step s36, as in step s32, the division area to be processed whose entropy EB is greater than or equal to the threshold value T is further equally divided into four square square division areas. FIG. 14 is a diagram showing how the upper left divided region 230 among the four divided regions 230 obtained by the initial division in step s32 is further divided into four divided regions 231.

  When step s36 is executed, the area dividing unit 32 executes step s33 again to newly determine the division area to be processed. In step s33 immediately after step s36, a new divided area obtained in step s36, which is larger than a predetermined size, is processed. Thereafter, the area dividing unit 32 similarly executes step s34 and subsequent steps.

  In step s35, if the area dividing unit 32 determines that there is no divided area to be processed having an entropy EB equal to or greater than the threshold value T, the area dividing unit 32 ends the area division for the candy area 23.

  As described above, in the present embodiment, when the entropy EB of the luminance histogram in the divided area is equal to or greater than the threshold value, the area dividing unit 32 repeatedly performs a process of further dividing the divided area into a plurality of divided areas. Therefore, the confectionery region 23 is divided into a plurality of divided regions 235 each having a small entropy EB. That is, the confectionery area 23 is divided into a plurality of divided areas 235 having uniform brightness.

  FIG. 15 is a diagram showing an outline of a state in which region division is performed on the candy region 23 and the candy region 23 is divided into a plurality of divided regions 235 having uniform brightness in each. In the confectionery area 23, since the luminance varies in the area including the outline of the pattern of the candy 100, as shown in FIG. 15, in the confectionery area 23, the vicinity of the outline of the pattern of the candy 100 is finely divided. The Further, in the confectionery area 23, since the luminance varies in an area including the outline of the candy 100, the vicinity of the outline of the candy 100 is finely divided in the candy area 23 as shown in FIG.

  As described above, when the entropy EB of the luminance histogram in the divided area is equal to or larger than the threshold value, the process of further dividing the divided area into a plurality of divided areas is repeatedly performed. The vicinity of the pattern outline of the confectionery 100 is finely divided. Therefore, it can be said that the plurality of divided regions 235 generated by performing region division on the candy region 23 show the characteristics of the candy 100. Hereinafter, the divided area 235 may be referred to as a “uniform divided area 235”.

  In the above example, when the entropy EB of the luminance histogram in the divided area is equal to or greater than the threshold value, the divided area is further divided into a plurality of divided areas. However, the entropy EB of the luminance histogram in the divided area is If it is larger than the threshold value, the divided area may be further divided into a plurality of divided areas. In the above example, the divided area is divided into four. However, the number of divided areas may be other than this.

<About the feature acquisition unit>
In the present embodiment, the feature amount acquisition unit 33, for each of the plurality of uniform division regions 235 generated by the region division unit 32, size information indicating the size of the uniform division region 235, and sweets Distance information indicating the distance from the center of the region 23 to the uniform divided region 235 is acquired. And the feature-value acquisition part 33 acquires a sweets feature-value based on the magnitude | size information and distance information about each uniform division area 235. In the present embodiment, the feature amount acquisition unit 33 generates a two-dimensional histogram of size information and distance information for the uniform divided region 235, and uses the two-dimensional histogram as a sweet feature amount.

  As the size information of the uniform divided region 235, for example, the horizontal or vertical length of the uniform divided region 235 is employed. In the present embodiment, since the uniform divided region 235 is a square, the uniform divided region 235 has the same length in the horizontal direction and the vertical direction. Moreover, as distance information about the uniform divided region 235, for example, a distance from the center of the confectionery region 23 to the center of the uniform divided region 235 is employed. As the size information of the uniform divided region 235, the length of the diagonal line of the uniform divided region 235 may be employed.

  For each uniform divided region 235, the feature amount acquisition unit 33 stores the size information and distance information of the uniform divided region 235 in the storage unit 310 as a set of information pairs. The feature amount acquisition unit 33 then calculates the frequency distribution of combinations of size information and distance information based on a plurality of information pairs respectively corresponding to the plurality of uniform divided regions 235 generated by the region dividing unit 32. Is generated. Hereinafter, the two-dimensional histogram may be referred to as a “two-dimensional evaluation value histogram”.

  FIG. 16 is a diagram illustrating an example of the two-dimensional evaluation value histogram 160. The first axis along the X direction in FIG. 16 indicates size information, and the second axis along the Y direction in FIG. 16 indicates distance information. And the third axis along the Z direction in FIG. 16 is the size information having the value shown on the first axis and the distance information having the value shown on the second axis in a plurality of information pairs. The frequency of combination is shown. In other words, the third axis in FIG. 16 indicates how many combinations of the size information having the value indicated by the first axis and the distance information having the value indicated by the second axis in a plurality of information pairs. Indicates whether it exists.

  For example, when there are 8 combinations of size information a1 and distance information b1 in a plurality of information pairs, bins corresponding to a1 shown on the first axis and b1 shown on the second axis The frequency is “8”. In addition, in a plurality of information pairs, when there are three combinations of size information a5 and distance information b3, bins corresponding to a5 shown on the first axis and b3 shown on the second axis The frequency is “3”.

  Thus, in this Embodiment, the two-dimensional histogram of the size information and distance information of a uniform division area in the candy area | region 23 is made into a candy feature-value.

<About judgment part>
The determination unit 34 compares the confectionery feature amount acquired by the feature amount acquisition unit 33 with the standard feature amount indicating the feature of the good confectionery, and based on the comparison result, the confectionery reflected in the captured image 11 Determine if 100 is correctly manufactured. As the standard feature amount, a two-dimensional evaluation value histogram generated by the determination device 3 in the same manner as described above from the captured image 11 in which the non-defective candy 100 is reflected is used. Hereinafter, the two-dimensional evaluation value histogram generated from the captured image 11 in which the non-defective candy 100 is reflected is referred to as a “standard two-dimensional evaluation value histogram”.

  When the production line is not in actual operation, a non-defective confectionery 100 is placed on the transport conveyor 101 in order to obtain a standard two-dimensional evaluation value histogram (standard feature value). In the image processing system 1, a captured image 11 in which a non-defective confectionery 100 placed on the conveyor 101 is reflected is generated. This captured image 11 is referred to as “standard captured image 11”. In the image processing system 1, the extraction unit 31 generates a candy region 23 in which the non-defective candy 100 is reflected from the standard captured image 11, and the region dividing unit 32 performs region division on the candy region 23. Then, the feature amount acquisition unit 33 generates a standard two-dimensional evaluation value histogram (standard feature amount) based on the plurality of uniform divided regions 235 generated by the region dividing unit 32. A standard two-dimensional evaluation value histogram generated when the production line is not actually operating is stored in the storage unit 310.

  When the production line is in actual operation, the determination unit 34 compares the two-dimensional evaluation value histogram generated by the feature amount acquisition unit 33 with the standard two-dimensional evaluation value histogram in the storage unit 310, and based on the comparison result. Then, it is determined whether or not the candy 100 shown in the captured image 11 is correctly manufactured. In the present embodiment, the determination unit 34 obtains the similarity between the two-dimensional evaluation value histogram generated by the feature amount acquisition unit 33 and the standard two-dimensional evaluation value histogram in the storage unit 310, thereby obtaining both. Compare As the value indicating the similarity, for example, a Bhattacharyya distance is used. A large Bhattacharyya distance means a low similarity, and a small Bhattacharyya distance means a high similarity. Another value may be used as a value indicating the degree of similarity.

  Here, when the similarity between the two-dimensional evaluation value histogram generated from the processing target image 11 and the standard two-dimensional evaluation value histogram is high, the confectionery 100 shown in the processing target image 11 may be a non-defective product. High nature. Therefore, the value indicating the similarity between the two-dimensional evaluation value histogram generated from the processing target image 11 and the standard two-dimensional evaluation value histogram, such as the Bhattacharyya distance, is determined by the confectionery 100 reflected in the processing target image 11. It can be said that this is an evaluation value indicating the certainty of being a good product. In other words, it can be said that the value indicating the similarity is an evaluation value indicating the possibility that the candy 100 shown in the processing target image 11 is a non-defective product. Hereinafter, the evaluation value is referred to as “determination evaluation value”.

  In the present embodiment, when the similarity between the two-dimensional evaluation value histogram generated by the feature amount acquisition unit 33 and the standard two-dimensional evaluation value histogram is high, the determination unit 34 displays the processing target image 11. If it is determined that the reflected candy 100 is correctly manufactured and the similarity is low, it is determined that the candy 100 reflected in the processing target image 11 is not correctly manufactured. Specifically, the determination unit 34 performs processing when the Bhattacharyya distance between the two-dimensional evaluation value histogram generated by the feature amount acquisition unit 33 and the standard two-dimensional evaluation value histogram is equal to or less than a threshold value. It is determined that the candy 100 reflected in the target image 11 is manufactured correctly, and if it is larger than the threshold value, it is determined that the candy 100 reflected in the processing target image 11 is not manufactured correctly. In other words, when the determination evaluation value generated based on the two-dimensional evaluation value histogram obtained from the processing target image 11 and the standard two-dimensional evaluation value histogram is equal to or less than the threshold value, the determination unit 34 It is determined that the candy 100 reflected in the captured image 11 is a non-defective product, and if it is greater than the threshold value, the candy 100 reflected in the captured image 11 is determined to be a defective product. When the pattern of the candy 100 reflected in the captured image 11 is broken or the candy 100 is missing, the determination unit 34 determines that the candy 100 is a defective product. Then, the determination unit 34 outputs the determination result 21.

  As described above, in the determination device 3 according to the present embodiment, the threshold value T used when dividing the confectionery region 23 is determined based on the confectionery region 23. Therefore, the confectionery area 23 in which the candy 100 is reflected can be appropriately divided into a plurality of uniform divided areas 235 according to the candy 100. Therefore, compared with the case where the threshold value T is determined in advance, it is possible to acquire a candy feature amount that appropriately indicates the feature of the candy 100 reflected in the candy region 23. As a result, the determination accuracy when determining whether or not the candy 100 is correctly manufactured is improved.

  Moreover, in this Embodiment, when a confectionery feature-value is acquired based on the some uniform division area 235, the magnitude | size information and distance information about the uniform division area 235 are used. Therefore, compared with the case where the pixel value of each pixel in each uniform divided region 235 is used and the confectionery feature amount is acquired, the confectionery feature amount can be easily acquired. Therefore, it is possible to easily determine whether or not the candy 100 shown in the captured image 11 is correctly manufactured.

  Moreover, in this Embodiment, since the captured image 11 is a captured image obtained by imaging the confectionery 100 conveyed in an irregular direction, it is the circumferential direction of the confectionery 100 reflected in the captured image 11. The orientation is not always the same. That is, the circumferential direction of the candy 100 reflected in the captured image 11 is not constant, and the attitude of the candy 100 reflected in one captured image 11 is the orientation of the candy 100 reflected in another captured image 11. There is a possibility that the posture may be rotated around the rotation axis along the thickness direction passing through the centers of the two main surfaces. Therefore, the direction of the circumferential direction of the candy 100 reflected in the candy area 23 is not constant. On the other hand, the size information and the distance information of the uniform divided area 235 obtained from the candy area 23 are information that does not change even if the uniform divided area 235 rotates around the center of the candy area 23. Therefore, even if the direction of the circumferential direction of the candy 100 reflected in the candy area 23 varies, if the candy 100 is a non-defective product, the size information and distance of the uniform divided area 235 obtained from the candy area 23 Information is difficult to vary. Therefore, it can be said that each of the size information and the distance information of the uniform divided region 235 is information that is hardly affected by the circumferential direction of the candy 100 shown in the captured image 11. In the present embodiment, since the confectionery feature amount is acquired based on information that is not easily influenced by the circumferential direction of the candy 100 reflected in the captured image 11, the circumferential direction of the candy 100 reflected in the captured image 11. Even when the orientation of the product is not constant, it is possible to prevent the good confectionery 100 reflected in the captured image 11 from being erroneously determined to be a defective product.

<Various modifications>
Hereinafter, various modifications of the present embodiment will be described.

<Variation of confectionery features>
In the above example, the feature amount acquisition unit 33 acquires the confectionery feature amount based on both the size information and the distance information of the uniform divided region 235, but the size information of the uniform divided region 235 and The confectionery feature amount may be acquired based on either one of the distance information. For example, the feature amount acquisition unit 33 may acquire the confectionery feature amount based on the size information of the plurality of uniform division regions 235 generated by the region division unit 32. In this case, for example, the feature amount acquisition unit 33 generates a one-dimensional histogram indicating the distribution of size information of the plurality of uniform divided regions 235 generated by the region dividing unit 32, and uses the one-dimensional histogram. The confectionery feature amount. FIG. 17 is a diagram illustrating an example of a one-dimensional histogram of the size information of the uniform divided region 235. The horizontal axis in FIG. 17 indicates size information. The vertical axis in FIG. 17 indicates the number of uniform divided areas 235 having the size information indicated on the horizontal axis, that is, the frequency, in the plurality of uniform divided areas 235 generated by the area dividing unit 32.

  When the one-dimensional histogram of the size information of the uniform divided region 235 is a candy feature amount, the determination device 3 takes a captured image in which the non-defective candy 100 is reflected when the production line is not actually operating. The one-dimensional histogram (standard one-dimensional histogram) of the size information of the uniform divided region 235 generated in the same manner from 11 is used as the standard feature amount. Then, similarly to the case where the two-dimensional evaluation value histogram is used, the determination unit 34 compares the one-dimensional histogram generated by the feature amount acquisition unit 33 with the standard one-dimensional histogram, and based on the comparison result. Then, it is determined whether or not the candy 100 shown in the processing target image 11 is correct.

  In addition, when the feature amount acquisition unit 33 acquires the confectionery feature amount based on the distance information of the plurality of uniform division regions 235 generated by the region division unit 32, the feature amount acquisition unit 33 sets the plurality of uniform division regions 235. A one-dimensional histogram showing the distribution of distance information is generated, and the one-dimensional histogram is used as a confectionery feature amount.

  As described above, even if the confectionery feature amount is acquired based on either the size information or the distance information of each uniform divided region 235, the pixels of each pixel in each uniform divided region 235 are obtained. Compared with the case where the confectionery feature amount is acquired based on the value, the confectionery feature amount can be easily acquired. Therefore, it is possible to easily determine whether or not the candy 100 shown in the captured image 11 is correctly manufactured.

  In addition, since each of the size information and the distance information of the uniform divided region 235 is information that is not easily influenced by the circumferential direction of the candy 100 shown in the captured image 11, the captured image is also used in the present modification. 11 can be prevented from being erroneously determined to be a non-defective product.

<Decision determination of confectionery using multiple types of standard features>
As described above, the size information and the distance information of the uniform divided area 235 are information that is not easily influenced by the circumferential direction of the candy 100 reflected in the candy area 23, but is reflected in the candy area 23. It varies somewhat under the influence of the direction of the circumferential direction of the confectionery 100. This point will be described in detail below.

  For example, consider four candy regions 23A to 23D in which a non-defective candy 100 is shown as shown in FIG. In the following description, it is assumed that the sweet feature amount obtained from the sweet region 23A is stored in the storage unit 310 as the standard feature amount. Hereinafter, the plurality of uniform divided regions 235 obtained from the candy region 23A may be referred to as “a plurality of standard uniform divided regions 235”.

  The circumferential direction of the candy 100B reflected in the candy area 23B is the circumferential direction of the candy 100A when the candy 100A reflected in the candy area 23A is rotated 90 ° counterclockwise around its center. Same as direction. The direction of the circumferential direction of the candy 100C reflected in the candy area 23C is the circumferential direction of the candy 100A when the candy 100A reflected in the candy area 23A is rotated counterclockwise by 180 ° around the center. Same as direction. The direction of the circumferential direction of the candy 100D reflected in the candy area 23D is the circumferential direction of the candy 100A when the candy 100A reflected in the candy area 23A is rotated 270 ° counterclockwise around its center. Same as direction.

  In the above example, when the divided area of the confectionery area 23 is further divided, it is equally divided into four square divided areas in a matrix. Therefore, the plurality of uniform divided regions 235 obtained from the candy region 23B counterclockwise the plurality of standard uniform divided regions 235 obtained from the candy region 23A around the center of the candy region 23A. It matches with the rotated one. Similarly, the plurality of uniform divided areas 235 obtained from the candy area 23C coincides with the plurality of standard uniform divided areas 235 rotated by 180 ° counterclockwise around the center of the candy area 23A. To do. The plurality of uniform divided regions 235 obtained from the candy region 23D coincides with the plurality of standard uniform divided regions 235 rotated about 270 ° counterclockwise around the center of the candy region 23A. . Therefore, the size information and distance information of the plurality of standard uniform divided regions 235, the size information and distance information of the plurality of uniform divided regions 235 obtained from the candy region 23B, and the plurality of information obtained from the candy region 23C. The size information and distance information of the uniform divided region 235 and the size information and distance information of the plurality of uniform divided regions 235 obtained from the confectionery region 23D coincide with each other. Therefore, the confectionery feature amount obtained from the confectionery region 23B, the confectionery feature amount obtained from the confectionery region 23C, and the confectionery feature amount obtained from the confectionery region 23D are obtained from the confectionery region 23A. It matches the obtained confectionery feature value, that is, the standard feature value.

  As described above, the circumferential direction of the non-defective candy 100 reflected in the certain confectionery area 23 indicates that the non-defective confectionery 100A reflected in the confectionery area 23A from which the standard feature amount is acquired is around the center (90 × N) ° (N is an integer greater than or equal to 0) When the counterclockwise direction coincides with the circumferential direction of the non-defective candy 100A, a plurality of ones obtained from the certain candy area 23 The divided area 235 corresponds to a plurality of standard uniform divided areas 235 rotated around the center of the candy area 23A by (90 × N) ° counterclockwise. Therefore, the confectionery feature amount obtained from the certain confectionery region 23 matches the confectionery feature amount obtained from the confectionery region 23A. That is, the confectionery feature amount obtained from the certain confectionery region 23 matches the standard feature amount in the storage unit 310. Therefore, it is possible to appropriately determine that the candy 100 reflected in the certain candy area 23 is correctly manufactured.

  On the other hand, the circumferential direction of a good confectionery 100 reflected in a certain confectionery area 23 rotates the good confectionery 100A reflected in the confectionery area 23A around its center (90 × N) ° counterclockwise. In the case where the orientation of the good confectionery 100A does not coincide with the circumferential direction, the plurality of uniform divided regions 235 obtained from the certain confectionery region 23 includes the plurality of standard uniform divided regions 235. In any rotation angle around the center of the confectionery region 23A, the plurality of standard uniform division regions 235 may not coincide. In other words, the circumferential direction of a good confectionery 100 reflected in a certain confectionery area 23 is such that the good confectionery 100A reflected in the confectionery area 23A is γ ° (γ ≠ 90 × N, that is, γ Is a value other than a multiple of “90”) When the counterclockwise direction coincides with the circumferential direction of the non-defective candy 100A, a plurality of uniforms obtained from the confectionery region 23 are obtained. The divided area 235 may not coincide with the plurality of standard uniform divided areas 235 no matter what rotation angle the plurality of standard uniform divided areas 235 are rotated around the center of the candy area 23A. . Therefore, the size information and distance information of the plurality of uniform division regions 235 obtained from the certain confectionery region 23 may not match the size information and distance information of the plurality of standard uniform division regions 235. Therefore, the confectionery feature amount obtained from the certain confectionery region 23 may not match the confectionery feature amount obtained from the confectionery region 23A. That is, the confectionery feature amount obtained from the certain confectionery region 23 may not match the standard feature amount in the storage unit 310. As a result, it may not be properly determined that the candy 100 reflected in the confectionery area 23 is correctly manufactured. In particular, the circumferential direction of a good confectionery 100 reflected in a certain confectionery area 23 causes the good confectionery 100A reflected in the confectionery area 23A to rotate about (90 × N + 45) ° counterclockwise around its center. When the quality of the non-defective confectionery 100A coincides with the direction of the circumferential direction, the confectionery feature amount obtained from the certain confectionery region 23 may be significantly different from the standard feature amount. Therefore, in this case, there is a high possibility that the confectionery 100 reflected in the certain confectionery area 23 is not properly determined as being correctly manufactured.

  Note that the circumferential direction of a good confectionery 100 reflected in a certain confectionery area 23 rotates the good confectionery 100A reflected in the confectionery area 23A around its center (90 × N) ° counterclockwise. When the confectionery feature amount obtained from the certain confectionery region 23A does not match the circumferential direction of the good confectionery 100A, the confectionery feature amount obtained from the confectionery region 23A The degree of deviation depends on how finely the candy area 23 is divided in the area division for the candy area 23. When the confectionery area 23 is finely divided in the area division, the confectionery characteristic amount obtained from the certain confectionery area 23 is greatly deviated from the confectionery characteristic amount obtained from the confectionery area 23A. .

  As described above, the circumferential direction of the non-defective candy 100 reflected in the certain confectionery area 23 indicates that the non-defective confectionery 100A reflected in the confectionery area 23A from which the standard feature amount is acquired is around the center (90 × N) ° When the counterclockwise rotation of the non-defective candy 100A does not coincide with the circumferential direction of the non-defective candy 100A, the candy feature value obtained from the certain candy region 23 may be different from the standard feature value. There is sex. For this reason, the confectionery 100 reflected in the certain confectionery area 23 may not be properly determined as a non-defective product. Therefore, in the candy determination process, not only the one type of standard feature amount acquired from the confectionery area 23 in which the good confectionery 100 is reflected, but also the good confectionery 100 reflected in the confectionery area 23 A confectionery region 23 in which a good confectionery 100 having a circumferential direction that does not coincide with the circumferential direction of the good confectionery 100 when rotated counterclockwise by (90 × N) ° around the center It is also desirable to use standard feature quantities obtained from

  Moreover, the description so far using FIG. 18 is a premise that the center of the confectionery region 23 and the center of the non-defective confectionery 100 reflected in the confectionery region 23 coincide with each other. However, depending on the extraction accuracy of the confectionery region 23 in the extraction unit 31, the center of the confectionery region 23 may not match the center of the good confectionery 100 reflected in the confectionery region 23. FIG. 19 is a diagram illustrating an example of a state in which the center 23c of the candy area 23 and the center 100c of the non-defective candy 100 reflected in the candy area 23 do not coincide with each other.

  When the center 23c of the confectionery area 23 extracted by the extraction unit 31 and the center 100c of the good confectionery 100 reflected in the confectionery area 23 do not coincide with each other, the good confectionery reflected in the certain confectionery area 23 is displayed. If the direction of the circumferential direction of 100 is different from the direction of the circumferential direction of the good confectionery 100A reflected in the confectionery region 23A from which the standard feature amount is acquired, the circumference of the good confectionery 100 reflected in the certain confectionery region 23 Whatever direction the direction is, the plurality of uniform divided regions 235 obtained from the certain confectionery region 23 is the same as the plurality of standard uniform divided regions 235 around the center of the confectionery region 23A. Even if it is rotated at such a rotation angle, it may not coincide with the plurality of standard uniform divided regions 235. Therefore, in this case, the circumferential direction of the good confectionery 100 reflected in the certain confectionery area 23 is such that the good confectionery 100A reflected in the confectionery area 23A is (90 × N) ° around its center. Even when the counterclockwise rotation matches the circumferential direction of the non-defective candy 100A, the plurality of uniform divided regions 235 obtained from the certain confectionery region 23 include a plurality of standard ones. There is a possibility that the divided area 235 does not coincide with the one obtained by rotating (90 × N) ° counterclockwise around the center of the candy area 23A. Therefore, the size information and distance information of the plurality of uniform division regions 235 obtained from the certain confectionery region 23 may not match the size information and distance information of the plurality of standard uniform division regions 235. Therefore, the confectionery feature amount obtained from the certain confectionery region 23 may not match the confectionery feature amount obtained from the confectionery region 23A, that is, the standard feature amount.

  Thus, when the center 23c of the confectionery area 23 extracted by the extraction unit 31 and the center 100c of the non-defective confectionery 100 reflected in the confectionery area 23 do not coincide with each other, the confectionery area 23 is reflected. If the circumferential direction of the non-defective candy 100 is different from the circumferential direction of the non-defective confectionery 100A reflected in the confectionery area 23A from which the standard feature amount has been acquired, Whatever the direction of the confectionery 100 in the circumferential direction, the confectionery feature value obtained from the certain confectionery region 23 may be different from the standard feature value. For this reason, the confectionery 100 reflected in the certain confectionery area 23 may not be properly determined as a non-defective product. Therefore, in the candy determination process, not only one type of standard feature amount acquired from the confectionery area 23 in which the good confectionery 100 is reflected, but also the circumference of the good confectionery 100 in the confectionery area 23. It is desirable that the standard feature amount acquired from the confectionery region 23 in which the good confectionery 100 having a circumferential direction direction different from the direction direction is reflected is also used.

  In view of the above points, in this modification, in the case of the candy determination process, a single type of standard feature amount is not used, but the circumferential directions of the non-defective candy 100 reflected in them are plural. A plurality of types of standard feature amounts respectively generated based on the confectionery region 23 are used. The determination unit 34 compares the confectionery feature amount obtained from the processing target image 11 with each of a plurality of types of standard feature amounts in the storage unit 310, and based on the comparison result, the image displayed in the processing target image 11. It is determined whether or not the confectionery 100 is correctly manufactured.

  In this modification, four types of standard feature values are used. The four types of standard feature quantities are generated based on the four confectionery regions 23R, 23α, 23β, and 23γ, respectively, in which the circumferential direction of the good confectionery 100 reflected in them is different. FIG. 20 is a diagram illustrating an example of the candy regions 23R, 23α, 23β, and 23γ. The direction of the circumferential direction of the non-defective candy 100 reflected in the confectionery region 23α is the same as the confectionery region 23R when the non-defective confectionery 100 reflected in the confectionery region 23R rotates around the center by 45 ° counterclockwise. This coincides with the circumferential direction of the non-defective confectionery 100 that is reflected. The direction of the circumferential direction of the good confectionery 100 reflected in the confectionery region 23β is the same as that of the confectionery region 23R when the good confectionery 100 reflected in the confectionery region 23R rotates 90 ° counterclockwise around the center. This coincides with the circumferential direction of the non-defective confectionery 100 that is reflected. The circumferential direction of the non-defective candy 100 reflected in the confectionery area 23γ is the confectionery area when the non-defective confectionery 100 reflected in the confectionery area 23R rotates around the center by 135 ° counterclockwise. This coincides with the circumferential direction of the non-defective confectionery 100 reflected in 23R.

  In the present modification, the candy regions 23α, 23β, and 23γ are generated based on the candy region 23R. Hereinafter, the candy area 23R is referred to as a “reference candy area 23R”. The candy region 23α is an image obtained by rotating the image 240R of the non-defective candy 100 reflected in the reference candy region 23R by 45 ° counterclockwise around the center 23Rc of the reference candy region 23R. The candy region 23β is an image obtained by rotating the image 240R of the non-defective candy 100 reflected in the reference candy region 23R by 90 ° counterclockwise around the center 23Rc of the reference candy region 23R. The candy region 23γ is an image obtained by rotating the image 240R of the non-defective candy 100 reflected in the reference candy region 23R by 135 ° counterclockwise around the center 23Rc of the reference candy region 23R. .

  When a plurality of types of standard feature values are generated in the determination device 3, the good confectionery 100 is placed on the transport conveyor 101 when the production line is not actually operating. In the image processing system 1, a standard captured image 11 in which a non-defective confectionery 100 placed on the conveyor 101 is reflected is generated. In the image processing system 1, the extraction unit 31 extracts the reference candy region 23R in which the non-defective candy 100 is reflected from the standard captured image 11, and the region division unit 32 performs region division on the reference candy region 23R. . Then, the feature quantity acquisition unit 33 generates a standard two-dimensional evaluation value histogram based on the plurality of uniform divided regions 235 generated by the region dividing unit 32. Accordingly, one type of standard feature value, that is, one type of standard two-dimensional evaluation value histogram is generated based on the reference sweet region 23R.

  Moreover, in the determination apparatus 3, the extraction part 31 produces | generates candy area | region 23 (alpha), 23 (beta), 23 (gamma) based on the reference | standard candy area | region 23R. In the reference candy region 23R, the extraction unit 31 rotates the image 240R of the non-defective candy 100 by 45 ° counterclockwise around the center 23Rc of the reference candy region 23R. Let it be area 23α. In addition, the extraction unit 31 obtains an image obtained by rotating the image 240R of the non-defective candy 100 by 90 ° counterclockwise around the center 23Rc of the reference candy region 23R in the reference candy region 23R. It is assumed that the confectionery region 23β. Then, the extraction unit 31 obtains an image obtained by rotating the image 240R of the non-defective candy 100 around the center 23Rc of the reference candy region 23R by 135 ° counterclockwise in the reference candy region 23R. It is assumed that the confectionery region 23γ.

  When the confectionery regions 23α, 23β, and 23γ are generated in the extraction unit 31, the region division unit 32 performs region division on the candy regions 23α, 23β, and 23γ. The feature amount acquisition unit 33 generates a standard two-dimensional evaluation value histogram based on the plurality of uniform division regions 235 generated by the region division of the candy region 23α in the region division unit 32. The feature amount acquisition unit 33 generates a standard two-dimensional evaluation value histogram based on the plurality of uniform division regions 235 generated by the region division unit 32 with respect to the confectionery region 23β. Then, the feature amount acquisition unit 33 generates a standard two-dimensional evaluation value histogram based on the plurality of uniform division regions 235 generated by the region division on the candy region 23γ in the region division unit 32. As a result, three types of standard feature values, that is, three types of standard two-dimensional evaluation value histograms, are generated based on the candy regions 23α, 23β, and 23γ.

  A plurality of types of standard feature values (standard two-dimensional evaluation value histograms) generated as described above are stored in the storage unit 310.

  When the production line is in actual operation, the determination unit 34 compares the two-dimensional evaluation value histogram generated from the processing target image 11 with each of a plurality of types of standard two-dimensional evaluation value histograms in the storage unit 310. Specifically, for each of a plurality of types of standard two-dimensional evaluation value histograms in the storage unit 310, the determination unit 34 generates the standard two-dimensional evaluation value histogram and the two-dimensional evaluation value histogram generated from the processing target image 11. Find the Bhattacharyya distance between. Then, the determination unit 34 sets a minimum value among the obtained Bhattacharyya distances as a determination evaluation value. Then, the determination unit 34 determines that the confectionery 100 shown in the processing target image 11 is a non-defective product when the determination evaluation value is equal to or less than the threshold value. It is determined that the candy 100 shown in the target image 11 is a defective product.

  As described above, in the candy determination processing according to the present modification, a plurality of types of standard feature amounts respectively generated based on a plurality of images in which the circumferential directions of the non-defective candy 100 reflected in them are different from each other are used. Therefore, it is possible to appropriately determine whether or not the candy 100 is correctly manufactured regardless of the circumferential direction of the candy 100 reflected in the processing target image 11. Therefore, the determination accuracy is improved.

  In the above example, four types of standard feature values are used, but the number of types of standard feature values is not limited to this. Moreover, the direction of the circumferential direction of the non-defective confectionery 100 reflected in each of the plurality of images (the plurality of confectionery areas 23) in which a plurality of types of standard feature values are generated is not limited to the above-described direction.

<Authenticity determination of confectionery using multiple types of determination methods>
In the candy determination process, the above-described determination method and other determination methods may be used in combination to determine whether or not the candy 100 shown in the captured image 11 is correctly manufactured. As another determination method for determining whether or not the candy 100 shown in the captured image 11 is correctly manufactured, for example, a template matching method using an image of a good candy as a template image is conceivable. In this template matching method, it is determined whether or not the candy 100 shown in the captured image 11 is correctly manufactured by specifying whether or not there is an image similar to the template image in the captured image 11. Below, operation | movement of the determination apparatus 3 when the above-mentioned determination method and a template matching method are used together in a candy determination process is demonstrated. A method other than the template matching method may be used.

  Also in the template matching method, as in the above-described determination method, a determination evaluation value (also referred to as “output value”) indicating the likelihood that the candy 100 shown in the captured image 11 is a non-defective product is generated. In the template matching method, a region most similar to the template image is specified in the captured image 11 in which the candy 100 is reflected, and a value indicating the degree of similarity between the region and the template image is used as a determination evaluation value. Hereinafter, the determination evaluation value obtained by executing the above steps s2 to s5 is referred to as “first determination evaluation value EV1”, and the determination evaluation value obtained by the template matching method is referred to as “second determination evaluation value EV2”. Here, the smaller the value of the second determination evaluation value EV2, the higher the possibility that the candy 100 shown in the processing target image 11 is a non-defective product.

  The determination device 3 executes steps s2 to s5 described above to obtain the first determination evaluation value EV1 for the processing target image 11, and uses the template matching method to determine the second determination evaluation value EV2 for the processing target image 11. Ask. And in the determination apparatus 3, the determination part 34 calculates | requires the comprehensive evaluation value EV0 using the following formula | equation (6).

  In formula (6), z1 and z2 are constants. As shown in Expression (6), the overall evaluation value EV0 is a small value when both the first determination evaluation value EV1 and the second determination evaluation value EV2 are small.

  When the overall evaluation value EV0 is equal to or less than the threshold th0, the determination unit 34 determines that the candy 100 shown in the processing target image 11 is a non-defective product. On the other hand, when the comprehensive evaluation value EV0 is larger than the threshold value th0, the determination unit 34 determines that the candy 100 shown in the processing target image 11 is a defective product. The threshold th0 and the constants z1 and z2 are stored in the storage unit 310.

  Here, the captured image 11 in which the non-defective candy 100 is reflected is referred to as a “positive captured image 11”, and the captured image 11 in which the defective confectionery 100 (the confectionery 100 that has not been manufactured correctly) is reflected “negative captured image 11”. Call it. The threshold th0 and the constants z1 and z2 are the first determination evaluation value EV1 and the second determination evaluation value EV2 for the plurality of positive captured images 11, and the first determination evaluation value EV1 and the first determination evaluation value EV1 for the plurality of negative captured images 11. 2 can be determined based on the evaluation value EV2. Hereinafter, a method for determining the threshold th0 and the constants z1 and z2 will be described.

  FIG. 21 is a diagram illustrating an example of the relationship between the first determination evaluation value EV1 and the second determination evaluation value EV2 obtained for each of the plurality of positive captured images 11 and the plurality of negative captured images 11. The x-axis shown in FIG. 21 indicates the second determination evaluation value EV2. The y-axis shown in FIG. 21 indicates the first determination evaluation value EV1 of the captured image 11 for which the second determination evaluation value EV2 shown on the x-axis is obtained. 21 indicates the first determination evaluation value EV1 and the second determination evaluation value EV2 for the positive captured image 11, and the white circle in FIG. 21 indicates the first determination evaluation value EV1 for the negative captured image 11. And the 2nd judgment evaluation value EV2 is shown. Hereinafter, the pair of the first determination evaluation value EV1 and the second determination evaluation value EV2 obtained for the captured image 11 may be referred to as an “evaluation value pair”. FIG. 21 shows an evaluation value scatter diagram 500 showing the distribution of the evaluation value pairs of the plurality of captured images 11.

  In this example, when the first determination evaluation value EV1 is small, there is a high possibility that the candy 100 shown in the processing target image 11 is a non-defective product. And when 2nd determination evaluation value EV2 is small, possibility that the candy 100 reflected in the process target image 11 is a non-defective product is high. Therefore, in the evaluation value scatter diagram 500 shown in FIG. 21, the black triangle indicating the evaluation value pair of the positive captured image 11 is in a region where both the first determination evaluation value EV1 and the second determination evaluation value EV2 are small, that is, in the lower left corner. concentrate.

  As in the above example, when the quality of the candy 100 is determined only by determining whether or not the first determination evaluation value EV1 is equal to or less than the threshold th1, the first determination evaluation value EV1 is small. For the negative captured image 11 in which the second determination evaluation value EV2 is large, it is erroneously determined that the candy 100 shown in the negative captured image 11 is a non-defective product. In the evaluation value scatter diagram 500 shown in FIG. 21, the negative captured image 11 in which an evaluation value pair indicated by white circles above and below the straight line 510 along the x-axis indicating the threshold th1 is obtained. Is mistakenly determined to be a non-defective product. In the evaluation value scatter diagram 500, in the straight line 510 along the x-axis direction, the distribution of the evaluation value pair (black triangle) of the positive captured image 11 and the distribution of the evaluation value pair (white circle) of the negative captured image 11 are appropriately set. Difficult to separate.

  Moreover, when the quality determination of the confectionery 100 is performed only by determining whether the second determination evaluation value EV2 is equal to or less than the threshold value th2, the first determination evaluation value although the second determination evaluation value EV2 is small. About the negative picked-up image 11 in which EV1 becomes large, it will be erroneously determined that the candy 100 reflected in it is a non-defective product. In the evaluation value scatter diagram 500 shown in FIG. 21, the negative captured image 11 in which the evaluation value pairs indicated by the white circles on the straight line 520 along the y axis and on the left side of the straight line 520 along the y axis indicating the threshold value th2 are obtained. It is erroneously determined that the reflected candy 100 is a non-defective product. In the evaluation value scatter diagram 500, it is difficult to properly separate the distribution of evaluation value pairs of the positive captured image 11 and the distribution of evaluation value pairs of the negative captured image 11 along the straight line 520 along the y-axis direction.

  In this modification, when the production line is not actually in operation, a plurality of non-defective candy 100 and a plurality of defective candy 100 are placed on the conveyor 101, and the determination device 3 uses a plurality of positives. An evaluation value pair for each of the captured image 11 and the plurality of negative captured images 11 is obtained. Then, in the evaluation value scatter diagram 500 showing a plurality of evaluation value pairs obtained by the determination device 3, the distribution of evaluation value pairs of the positive captured image 11 and the distribution of evaluation value pairs of the negative captured image 11 are separated. An expression representing the separation straight line 530 (see FIG. 21) is obtained. The separation straight line 530 can be expressed by the following formula (7). In the formula (7), A1, A2, and B are constants.

  In the evaluation value scatter diagram 500, the region above and below the separation line 530 is a region where both the first determination evaluation value EV1 and the second determination evaluation value EV2 are small. Therefore, when the evaluation value pair obtained for the processing target image 11 is shown in the evaluation value scatter diagram 500, when the position of the evaluation value pair is on the separation line 530 or on the left side thereof, the processing target It can be considered that the candy 100 shown in the image 11 is a non-defective product.

  In the evaluation value scatter diagram 500, the region above and below the separation line 530 is expressed by the following equation (8).

  Equation (8) can be transformed into the following equation (9).

  When Expression (9) is satisfied when the first determination evaluation value EV1 and the second determination evaluation value EV2 obtained for the processing target image 11 are respectively substituted into x and y of Expression (9), The confectionery 100 shown in the processing target image 11 can be considered as a non-defective product.

  The storage unit 310 of the determination device 3 stores constants A1 and A2 in the equation (7) as constants z1 and z2 in the equation (6), respectively. Then, “A2 × B” is stored in the storage unit 310 as the threshold value th0.

  The determination unit 34 uses the first determination evaluation value EV1 and the second determination evaluation value EV2 for the processing target image 11 and the constants z1 and z2 in the storage unit 310 to determine the overall evaluation value EV0 for the processing target image 11. Ask for. Then, if the obtained comprehensive evaluation value EV0 is equal to or less than the threshold value th0 in the storage unit 310, the determination unit 34 determines that the candy 100 shown in the processing target image 11 is a non-defective product, and the threshold value th0. If it is larger than this, it is determined that the candy 100 reflected in the processing target image 11 is a defective product. Thereby, the quality determination of the candy 100 can be performed more correctly.

  Note that the expression (7) may be obtained by hand calculation by a person looking at the evaluation value scatter diagram 500, or the first determination evaluation value EV1 and the second determination evaluation value obtained by the computer device using the determination device 3. You may obtain | require using machine learning functions, such as a support vector machine, based on EV2. In the latter case, the determination device 3 in which the machine learning function is implemented may obtain Equation (7).

  Thus, in this modification, it is determined whether or not the candy 100 shown in the processing target image 11 is correctly manufactured based on a plurality of determination evaluation values obtained by different determination methods. Can be improved.

<Other variations>
In the above example, the image processing system 1 has been introduced into a production line for producing confectionery, but may be introduced into other devices or various places.

  For example, the image processing system 1 may be introduced into a vending machine into which money with a pattern on the surface is inserted. In this case, the money put into the vending machine is imaged by the imaging device 2. In the vending machine, for example, money moves while rotating on a rail. The imaging device 2 images the rotating money. The extraction unit 31 extracts a money area in which money is shown from the captured image 11. The area dividing unit 32 divides the money area extracted by the extracting unit 31. The feature amount acquisition unit 33 acquires a feature amount indicating the feature of money shown in the captured image 11 based on the plurality of divided regions generated by the region dividing unit 32. The determination unit 34 determines whether the money shown in the captured image 11 corresponds to a correct one based on the feature amount obtained by the feature amount acquisition unit 33. Since the pattern on the surface of the money can be handled in the same manner as the pattern on the surface of the candy, the image processing system 1 determines whether the money shown in the captured image 11 corresponds to the correct one in the same manner as described above. can do. Thereby, the authenticity of the money thrown into the vending machine can be determined.

  The image processing system 1 may be introduced into a production line for assembling products in a factory. More specifically, the image processing system 1 may be introduced into a production line for mounting a plurality of components on a substrate, for example. In this case, the imaging device 2 captures an image of a substrate on which a plurality of components are mounted. The extraction unit 31 extracts a board area on which a board on which a plurality of components are mounted is shown from the captured image 11. The area dividing unit 32 performs area division on the substrate area extracted by the extracting unit 31. The feature amount acquisition unit 33 acquires a feature amount indicating the feature of the board shown in the captured image 11 based on the plurality of divided regions generated by the region dividing unit 32. Based on the feature amount acquired by the feature amount acquisition unit 33, the determination unit 34 determines whether or not the board on which the plurality of components mounted in the captured image 11 corresponds to the board on which the plurality of components are correctly mounted. That is, it is determined whether or not a plurality of components are correctly mounted on the board. Since a plurality of parts on the substrate can be handled in the same manner as the pattern on the candy surface, in the same manner as described above, the image processing system 1 determines whether or not the plurality of components are correctly mounted on the substrate. Can be determined. As a result, it is possible to detect component mounting mistakes, component mounting leakage, and the like.

  Further, the image processing system 1 may be introduced into a rotating sushi restaurant. In a conveyor belt sushi restaurant, the price of a product such as sushi may be set according to the pattern of the plate on which the product is placed. And in a conveyor belt sushi restaurant, the sum total of the price according to the pattern of each dish which the customer took in hand may be calculated as food and drink charges. The image processing system 1 introduced into such a rotary sushi restaurant specifies the pattern of the dish taken by the customer by image processing.

  Specifically, when a customer picks up a plate from a rotating lane on which a plurality of plates with sushi or the like are mounted, the plate is imaged by the imaging device 2. The extraction unit 31 extracts a dish area in which the dish appears from the captured image 11. The area dividing unit 32 performs area division on the dish area extracted by the extracting unit 31.

  Here, a pattern is given to the peripheral part of the plate, and a product such as sushi is placed on the central part (the part without the pattern) of the plate. In this case, merchandise is shown in the central part of the dish area extracted by the extracting unit 31, and the central part does not indicate the characteristics of the dish. Therefore, the region dividing unit 32 performs region division on the portion of the dish region extracted by the extracting unit 31 excluding the central portion.

  The feature amount acquisition unit 33 acquires a feature amount indicating the feature of the dish shown in the captured image 11 based on the plurality of divided regions generated by the region dividing unit 32. The determination unit 34 determines whether the dish shown in the captured image 11 corresponds to a dish having a predetermined pattern, based on the feature amount obtained by the feature amount acquisition unit 33. That is, the determination unit 34 determines whether the pattern of the dish shown in the captured image 11 matches a predetermined pattern based on the feature amount. Since the pattern on the surface of the dish can be handled in the same manner as the pattern on the surface of the candy, in the same manner as described above, in the image processing system 1, the dish reflected in the captured image 11 corresponds to a dish having a predetermined pattern. It can be determined whether or not. The determination unit 34 determines, for each of the plurality of types of patterns attached to the plurality of types of dishes used in the sushi restaurant, whether or not the pattern matches the pattern of the dishes shown in the captured image 11. . Thereby, the pattern of the dish reflected in the captured image 11 is specified. Therefore, the price corresponding to the pattern of the dish shown in the captured image 11 can be automatically specified. Therefore, it is possible to automatically calculate the total price according to the pattern of each dish taken by the customer, that is, the food and beverage charges.

  Further, the image processing system 1 may be installed in a distribution center. In a distribution center, a sticker with a mailing address or the like may be affixed to a package such as cardboard. The image processing system 1 determines whether or not the seal attached to the package corresponds to the correct seal, that is, whether or not the correct seal is applied to the package. In this case, the sticker attached to the luggage is imaged by the imaging device 2. The extraction unit 31 extracts a seal area where a seal appears from the captured image 11. The area dividing unit 32 performs area division on the seal area extracted by the extracting unit 31. The feature amount acquisition unit 33 acquires a feature amount indicating the feature of the board shown in the captured image 11 based on the plurality of divided regions generated by the region dividing unit 32. The determination unit 34 determines whether or not the sticker shown in the captured image 11 is a correct sticker based on the feature amount obtained by the feature amount acquisition unit 33. Since the characters on the seal surface can be handled in the same manner as the pattern on the candy surface, the image processing system 1 can determine whether the correct seal is affixed to the package in the same manner as described above. it can. Thereby, it is possible to detect a mistake in sticking the seal.

  As described above, the image processing system 1 has been described in detail. However, the above description is illustrative in all aspects, and the present invention is not limited thereto. The various modifications described above can be applied in combination as long as they do not contradict each other. And it is understood that the countless modification which is not illustrated can be assumed without deviating from the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Image processing system 3 Judgment apparatus 11 Captured image 31 Extraction part 32 Area division part 33 Feature-value acquisition part 34 Judgment part 100 Sweets 311 Control program

Claims (10)

A determination device that determines whether or not an object shown in a processing target image corresponds to a comparison target,
An extraction unit for extracting a target object region in which the target object is reflected from a processing target image in which the target object is reflected;
A region dividing unit for performing region division on the object region;
Based on a plurality of divided regions generated by the region dividing unit, a feature amount acquiring unit that acquires a feature amount indicating the feature of the object;
A determination unit that determines whether or not the object corresponds to a comparison object based on the feature amount;
The area dividing unit includes:
Based on the entropy of the luminance histogram over the entire object area, determine a threshold value used in the area division;
In the area division, when the entropy of the luminance histogram in the divided area of the object area is greater than or equal to the threshold value or larger than the threshold value, a process of further dividing the divided area into a plurality of divided areas is repeatedly performed. , Determination device. The determination device according to claim 1,
The determination device, wherein the feature amount acquisition unit acquires the feature amount based on information indicating the size of each of the plurality of divided regions generated by the region dividing unit. The determination device according to claim 1,
The feature amount acquisition unit acquires the feature amount based on information indicating a distance from the center of the object region to the divided region for each of the plurality of divided regions generated by the region dividing unit. Judgment device. The determination device according to any one of claims 2 and 3,
The determination unit, wherein the feature amount acquisition unit uses a one-dimensional histogram of the information as the feature amount. The determination device according to claim 1,
The feature amount acquisition unit includes first information indicating a size of the divided region for each of the plurality of divided regions generated by the region dividing unit, and the object region for each of the plurality of divided regions. A determination apparatus that acquires the feature amount based on second information indicating a distance from the center of the region to the divided region. The determination device according to claim 5,
The determination unit, wherein the feature amount acquisition unit uses a two-dimensional histogram of the first information and the second information as the feature amount. A determination apparatus according to any one of claims 1 to 6,
The determination unit compares a standard feature amount indicating a feature of the comparison target with the feature amount, and determines whether the target object corresponds to the comparison target based on the comparison result apparatus. The determination device according to claim 7, wherein
The circumferential direction of the object shown in the processing target image is not constant,
The determination unit compares each of a plurality of types of standard feature amounts indicating features of the comparison target with the feature amounts, and based on the comparison result, whether or not the target matches the comparison target. Determine
The plurality of types of standard feature amounts are respectively generated based on a plurality of images in which the comparison target is reflected,
The determination apparatus in which the direction of the circumferential direction of the comparison target reflected in the images is different between the plurality of images. A control program for causing a computer device to determine whether an object shown in a processing target image corresponds to a comparison target,
In the computer device,
(A) extracting a target object region in which the target object is shown from a processing target image in which the target object is shown;
(B) performing a region division on the object region;
(C) obtaining a feature amount indicating the feature of the object based on the plurality of divided regions generated in the step (b);
(D) executing a step of determining whether or not the object corresponds to a comparison object based on the feature amount;
In the step (b),
Based on the entropy of the luminance histogram over the entire object area, the threshold used in the area division is determined,
Control for repeatedly performing the process of further dividing the divided area into a plurality of divided areas when the entropy of the luminance histogram in the divided area of the object area is greater than or equal to the threshold value or larger than the threshold value program. A determination method in a determination device for determining whether or not an object shown in a processing target image corresponds to a comparison target,
(A) extracting a target object region in which the target object is shown from a processing target image in which the target object is shown;
(B) performing a region division on the object region;
(C) obtaining a feature amount indicating the feature of the object based on the plurality of divided regions generated in the step (b);
(D) determining whether the object corresponds to a comparison object based on the feature amount,
In the step (b),
Based on the entropy of the luminance histogram over the entire object area, a threshold value used in the area division is determined,
A determination method in which, when the entropy of the luminance histogram in the divided area of the object area is equal to or larger than the threshold value or larger than the threshold value, the process of further dividing the divided area into a plurality of divided areas is repeatedly performed .

Images