JP2019184380A - Imaging device and inspection system using imaging device - Google Patents

Abstract

To provide an inspection system for using an imaging device and determining whether or not an inspection object has a defect.SOLUTION: Provided is an inspection system comprising a conveyance path for conveying an inspection object and defect determination means for determining whether or not the conveyed inspection object has a defect, in which the defect determination means includes: a detection sensor for detecting the inspection object in the conveyance path; first imaging means for imaging the inspection object on the basis of detection by the detection sensor; second imaging means for imaging the inspection object from a second imaging direction different from a first imaging direction; third image generation means for associating the first and the second images and generating an omnidirectional image of the inspection object as a third image; and an artificial intelligence program for determining from the third image whether or not the inspection object has a defect on the basis of a learned model having been taught by machine learning using learning data generated by defect criterion data for teaching an inspection object model image of the inspection object where a defect exists that a defect exists in the inspection object.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an imaging device and an inspection system using the imaging device, and in particular, is provided on a conveyance path of an inspection system that determines whether or not an inspection object conveyed on a conveyance path is defective. The present invention relates to an imaging apparatus that performs the inspection and an inspection system that uses the imaging apparatus.

  In general, a technique for capturing an inspection object and determining whether or not the inspection object has a defect based on the captured image is widely known. For example, Patent Document 1 proposes an inspection system that detects deformation of a box and determines whether or not the box is defective.

  The inspection system of Patent Document 1 includes a light source unit that irradiates a plurality of slit lights on a specific inspection target surface of a box, a camera unit that images slit light emitted by the light source unit, and an image captured by the camera unit. An image processing unit is provided that determines whether the box is defective by processing.

  This image processing unit detects the position of the end point of the slit light applied to the edge of the inspection target surface of the box, calculates the number of end points where this end point position is out of the preset allowable range, and calculated If the number of end points is equal to or greater than a predetermined number, it is determined that the box is defective.

JP 2010-197167 A

  However, since the inspection system of Patent Document 1 determines the defect of the box based on the image obtained by imaging the slit light irradiated to the specific inspection target surface of the box, there is a defect in a portion other than the inspection target surface. It is assumed that the failure cannot be determined.

  Moreover, in order to detect whether or not the end point position of the slit light irradiated to the edge of the inspection target surface of the box is within the allowable range, it is necessary to set the allowable range in advance. It is necessary to set an allowable range individually for each type of box to be performed.

  Therefore, when simultaneously determining whether or not there are defects in a plurality of boxes having different shapes, the permissible ranges of the end point positions irradiated with the slit light are set in advance for the plurality of types of boxes, respectively. Therefore, it is assumed that the workability of the inspection is lowered.

  The present invention has been made in view of the above circumstances, and an imaging apparatus capable of appropriately capturing whether or not an inspection target has a defect can be captured, and the imaging apparatus. It is an object of the present invention to provide an inspection system that can efficiently determine whether or not an inspection object has a defect.

  An imaging apparatus for solving the above-described problem is provided in an imaging system that images a test object provided in a transport path of an inspection system that determines whether or not an inspection object transported on the transport path is defective. A detection sensor that detects an inspection object conveyed on the road, and a first imaging unit that images the inspection object from the first imaging direction based on detection of the inspection object by the detection sensor and generates a first image; A second imaging unit that images the inspection object from a second imaging direction different from the first imaging direction in which the first imaging unit images the inspection object, and generates a second image; and a second imaging unit that is generated by the first imaging unit. And a third image generating means for associating one image with the second image generated by the second imaging means to generate an image of the omnidirectional image of the inspection object as a third image.

  In addition, the first imaging unit of the imaging apparatus starts imaging the inspection object from the first imaging direction at the start of detection when the inspection object is detected by the detection sensor, and continues the imaging, and detects the detection object by the detection sensor. When the detection is finished, the imaging of the inspection object is finished and the first image is generated.

  According to this imaging apparatus, an omnidirectional image of the inspection object is generated as the third image based on the first image and the second image generated by imaging with the first imaging unit and the second imaging unit. Therefore, it is possible to provide an image that can appropriately determine whether or not the inspection object is defective.

  The first imaging means of this imaging apparatus is arranged below the transport path and images the inspection object from below the transport path. Therefore, it is possible to image a portion of the inspection object that is difficult to image.

  Further, the second imaging means of the imaging device includes a plurality of imaging means for imaging the inspection object from a plurality of different imaging directions, and a plurality of imaging directions by the plurality of imaging means are set as the second imaging direction. It is characterized by. Therefore, the inspection object can be imaged in multiple ways.

  An inspection system for solving the above-mentioned problems is a defect in an inspection system including a conveyance path for conveying an inspection object, and a defect determination unit that is arranged in the conveyance path and determines whether or not the inspection object is defective. The determination unit is configured to detect the inspection object conveyed on the conveyance path, and to generate a first image by imaging the inspection object from the first imaging direction based on detection of the inspection object by the detection sensor. A first imaging unit, a second imaging unit that images the inspection object from a second imaging direction different from the first imaging direction in which the first imaging unit images the inspection object, and generates a second image; A third image generating means for associating the first image generated in step 2 with the second image generated by the second imaging means to generate an image in all directions of the inspection object as a third image, and an inspection in which a defect exists Inspection target with object imaged Transported on the transport path based on a learned model that has been machine-learned using learning data generated from the model image and the failure criterion data that is learned when there is a defect in the inspection object captured in the inspection object model image And an artificial intelligence program that autonomously determines from the third image whether there is a defect in the inspection object to be checked.

  In addition, the first imaging means of the inspection system starts imaging the inspection object from the first imaging direction at the start of detection when the inspection object is detected by the detection sensor, continues the imaging, and detects the detection object by the detection sensor. When the detection is finished, the imaging of the inspection object is finished and the first image is generated.

  According to this inspection system, an omnidirectional image of the inspection object is generated as the third image based on the first image and the second image generated by imaging with the first imaging unit and the second imaging unit. Therefore, it is possible to provide an image that can appropriately determine whether or not the inspection object is defective.

  Based on this third image, the artificial intelligence program autonomously determines whether or not the inspection object is defective, so that workability when determining whether or not the inspection object is defective is determined. Can be improved.

  This inspection system is characterized by comprising inspection object removal means for removing the inspection object determined to be defective by the defect determination means from the conveyance path. Thereby, the inspection object with a defect and the inspection object without a defect are sorted efficiently.

  The first imaging means of this inspection system is characterized in that it is arranged below the conveyance path and images the inspection object from below the conveyance path, and further, the second imaging means is from a plurality of different imaging directions. A plurality of imaging means for imaging the inspection object are provided, and a plurality of imaging directions by the plurality of imaging means are set as the second imaging direction.

  Therefore, the inspection object has a defect based on an image in which it is possible to appropriately determine whether or not the inspection object has a defect, by imaging a portion of the inspection object that is difficult to image and taking a multifaceted image. It is possible to efficiently determine whether or not there is.

  According to the present invention, it is possible to provide an image that can appropriately determine whether or not an inspection object has a defect. Furthermore, the workability at the time of determining whether or not the inspection object has a defect can be improved.

It is a figure explaining the outline of the inspection system concerning an embodiment of the invention. Similarly, it is a block diagram explaining the outline of the defect determination means of the inspection system which concerns on this Embodiment. Similarly, it is a figure explaining the outline of the imaging device of the inspection system concerning this embodiment. Similarly, it is a figure explaining the outline of the 1st imaging means of the imaging device of the inspection system concerning this embodiment. Similarly, it is a figure explaining the outline in the case of imaging a test subject with the 1st imaging means of the imaging device of the inspection system concerning this embodiment. Similarly, it is a block diagram explaining the outline of the process of the artificial intelligence program of the test | inspection system which concerns on this Embodiment. It is a figure explaining the outline of the inspection system concerning other embodiments of the present invention.

  Next, an embodiment of the present invention will be described with reference to FIGS.

  In the present embodiment, a case will be described as an example in which the inspection object for determining whether there is a defect in the inspection system is a rectangular parallelepiped cosmetic packaging box.

  FIG. 1 is a diagram for explaining the outline of the inspection system according to the present embodiment. As shown in the figure, the inspection system 1 includes a conveyance path 10 that conveys the packaging box 100, and a failure determination unit that determines whether or not the packaging box 100 that is disposed in the conveyance path 10 and is conveyed on the conveyance path 10 has a defect. 20, an actuator 30 that is an inspection object removal unit disposed downstream of the defect determination unit 20, and a display 40 that is disposed along the conveyance path 10 between the defect determination unit 20 and the actuator 30.

  The conveyance path 10 includes an input unit 11 into which the packaging box 100 is input, an unloading unit 12 from which the packaging box 100 is unloaded, and a plurality of belt conveyors 13 disposed between the input unit 11 and the unloading unit 12.

  In the present embodiment, the plurality of belt conveyors 13 are arranged in series between the input unit 11 and the carry-out unit 12 via a gap, and the packaging box 100 supplied from the input unit 11 is transferred by the plurality of belt conveyors 13. It is conveyed to the carry-out unit 12, and the packaging box 100 is carried out at the carry-out unit 12.

  In this embodiment, the defect determination means 20 is straddled at a portion where one belt conveyor 13 and the other belt conveyor 13 are connected in the conveyance path 10.

  FIG. 2 is a block diagram for explaining the outline of the defect determination means 20. As shown in the figure, in this embodiment, the defect determination means 20 is packaged based on an imaging device 21 that captures an image of the packaging box 100 that is transported in the transport path 10 and a third image that is generated by the imaging device 21 (described later). An artificial intelligence program 26 that autonomously determines whether or not the box 100 is defective is provided.

  The imaging device 21 senses the packaging box 100 from the first imaging direction when the detection box 22 is detected by the detection sensor 22 and the detection sensor 22 that detects the packaging box 100 that is transported on the transport path 10, and the first image is captured. The first camera 23 that is the first imaging means to be generated, and the second imaging means that is the second imaging means that images the packaging box 100 from a second imaging direction different from the first imaging direction of the first camera 23 and generates a second image. The camera 24 and a third image generation program 25 that is a third image generation unit that generates a third image in association with the first image and the second image are provided.

  FIG. 3 is a diagram for explaining the outline of the imaging device 21. As shown in the figure, in the present embodiment, the detection sensor 22 is disposed above the belt conveyor 13 in the gap A between one belt conveyor 13 and the other belt conveyor 13, and irradiates the laser toward the belt conveyor 13. It is a general-purpose position sensor that detects the presence or absence of the packaging box 100 at the position where it is projected onto the belt conveyor 13.

  FIG. 4 is a diagram for explaining the outline of the first camera 23. As shown in the figure, in the present embodiment, the first camera 23 is disposed below the belt conveyor 13 in a gap A between one belt conveyor 13 in which the detection sensor 22 is disposed and the other belt conveyor 13, and is a rectangular parallelepiped. The surface on which the wrapping packaging box 100 is placed on the belt conveyor 13 is imaged from the first imaging direction below the belt conveyor 13.

  In the present embodiment, the first camera 23 is implemented as a line scan camera.

  FIG. 5 is a diagram illustrating an outline when the packaging box 100 is imaged by the first camera 23. As shown in the drawing, the first camera is detected at the start of detection S when the packaging box 100 moves along the conveyance direction of the packaging box 100 along the conveyance path 10 indicated by the arrow F and the detection sensor 22 detects the packaging box 100. 23 starts imaging of the packaging box 100.

  Thereafter, the imaging by the first camera 23 is continued until the detection end time E when the packaging box 100 further moves along the conveyance direction F and the detection of the packaging box 100 is completed. At the end of detection E when the detection of 100 is finished, the imaging of the packaging box 100 is finished.

  Thereby, the surface mounted on the belt conveyor 13 in the packaging box 100 is imaged over the entire surface by the first camera 23, and a first image is generated.

  As shown in FIGS. 2 and 3, the second camera 24 images the packaging box 100 from the first imaging direction of the first camera 23, that is, from a second imaging direction different from the lower side of the belt conveyor 13.

  In the present embodiment, the second camera 24 includes a front camera 24a that images a front surface of the packaging box 100 in the transport direction F, an upper camera 24b that images an upper surface of the packaging box 100 in the transport direction F, and a packaging A right camera 24c that images the right side surface of the transport direction F of the box 100, a left camera 24d that images the left side surface of the transport direction F of the packaging box 100, and a rear surface of the transport direction F of the packaging box 100. It is comprised by the rear camera 24e which images.

  A plurality of different imaging directions in which the cameras 24a to 24e constituting the second camera 24 image the packaging box 100 are set as the second imaging directions in the present embodiment.

  In the present embodiment, each of the cameras 24a to 24e detects each surface of the packaging box 100 (the front surface in the transport direction F, the transport direction F in the transport direction F) when the detection sensor 22 detects the packaging box 100. The upper surface, the right surface in the transport direction F, the left surface in the transport direction F, and the rear surface in the transport direction F) are simultaneously imaged to generate a second image.

  In the present embodiment, the second image is captured by the cameras 24a to 24e, the front surface in the transport direction F, the upper surface in the transport direction F, the right surface in the transport direction F, and the transport direction F. It is composed of five images on the left side and the rear side in the transport direction F.

  In the present embodiment, the third image generation program 25 is a first image captured and generated by the first camera 23 and five images generated by being captured by the cameras 24a to 24e of the second camera 24. The image of all directions of the packaging box 100 is produced | generated as a 3rd image in correlation with the 2nd image comprised by these.

  In the present embodiment, the third image is associated with the first image and the second image composed of five images, and all the surfaces of the rectangular parallelepiped packaging box 100 imaged by the imaging device 21. It is composed of (six planes) images.

  FIG. 6 is a block diagram for explaining the outline of the process of the artificial intelligence program 26 according to the present embodiment. As shown in the figure, the artificial intelligence program 26 in this embodiment has a defect in the inspection object model image D1 in which the packaging box in which the defect exists is imaged and in the packaging box imaged in the inspection object model image D1. Learning data D3 is generated based on the defect determination reference data D2 to be learned if it exists.

  In the present embodiment, the inspection object model image D1 is a plurality of images obtained by imaging in advance a packaging box in which various defects are present, such as insufficient bonding of bonding parts, insufficient pressure bonding, presence of dirt, unclear printing, misalignment, and the like. Consists of images.

  In the present embodiment, the defect determination reference data D2 indicates that various defects exist in the packaging box captured in a plurality of images constituting the inspection object model image D1 based on the inspection object model image D1. Data to be learned.

  The defect determination reference data D2 is, for example, classified as “a” when there is a pasting defect on a packaging box imaged in a plurality of images constituting the inspection object model image D1, and there is a crimping defect. Is classified as “b”, and when there is dirt, it is classified as “c”. The packaging box captured in a plurality of images constituting the inspection object model image D1 is set as a defective mode. Classify accordingly.

  In this way, learning data D3 is generated by learning the packaging box corresponding to the defect mode based on the defect determination reference data D2 based on the inspection object model image D1.

  Further, the artificial intelligence program 26 generates a learned model D4 by performing machine learning with the generated learning data D3.

  As a method for performing machine learning, various algorithms such as a neural network, a random forest, and an SVM (support vector machine) are appropriately used.

  As described above, the artificial intelligence program 26 autonomously determines from the third image D5 whether or not the packaging box 100 transported in the transport path 10 is defective based on the learned model D4.

  When the artificial intelligence program 26 determines that the packaging box 100 is not defective, for example, a signal “1” is generated. When the artificial intelligence program 26 determines that the packaging box 100 is defective, for example, a signal “2” is generated. To do.

  As shown in FIGS. 1 and 2, the actuator 30 is connected to the artificial intelligence program 26 of the failure determination unit 20 and is disposed adjacent to the belt conveyor 13 in the transport path 10 downstream of the failure determination unit 20.

  In the present embodiment, the actuator 30 includes a protruding pin 30 a that protrudes into the conveyance path 10.

  When the artificial intelligence program 26 of the defect determination means 20 determines that there is a defect in the one packaging box 100 conveyed on the conveyance path 10 and generates a signal “2”, the signal is input to the actuator 30. Then, the actuator 30 causes the protruding pin 30a to protrude toward the packaging box 100 determined to be defective by the defect determination means 20, and removes the packaging box 100 from the transport path 10.

  As shown in FIGS. 1 and 2, the display 40 is connected to the artificial intelligence program 26 of the defect determination unit 20 and is disposed along the conveyance path 10 between the defect determination unit 20 and the actuator 30.

  In the present embodiment, the display 40 displays a third image generated by imaging the packaging box 100 with the imaging device 21 of the defect determination means 20 so that the worker M can monitor it. The

  Next, the operation of the inspection system 1 according to the present embodiment will be described.

  First, a packaging box 100 that is an object to be inspected is input from the input unit 11 of the conveyance path 10. The input packaging box 100 is transported along the transport direction F by the belt conveyor 13.

  When the packaging box 100 is conveyed to the defect determination means 20, the detection sensor 22 of the imaging device 21 detects the packaging box 100. At the start of detection S when the detection sensor 22 detects the packaging box 100, the first camera 23 starts imaging the packaging box 100.

  The imaging by the first camera 23 continues until the detection box 22 moves along the conveyance direction F and the detection sensor 22 ends the detection of the packaging box until the detection end time E. The imaging of the packaging box 100 is finished, and the first image is generated.

  On the other hand, each camera 24a-24e which comprises the 2nd camera 24 is each surface (the front side of the conveyance direction F, the upper surface of the conveyance direction F, the right side of the conveyance direction F) of the packaging box 100 at the time of detection start S. Image of the surface, the left surface in the transport direction F, and the rear surface in the transport direction F). Thereby, the second image is generated.

  The first image and the second image are related by the third image generation program 25, and an image in all directions of the packaging box 100 is generated as the third image. The generated third image is displayed on the display 40.

  Based on the third image generated in this way, the artificial intelligence program 26 uses the learned model D4 generated based on the inspection object model image D1 and the defect determination reference data D2 in the packaging box 100. It is determined whether there is a defect.

  When the artificial intelligence program 26 determines that the packaging box 100 is defective, the artificial intelligence program 26 generates a signal “2”, for example, and the signal is input to the actuator 30.

  In accordance with the input of the signal “2”, the actuator 30 causes the protruding pin 30 a to protrude toward the packaging box 100 determined to be defective by the defect determination means 20, and removes the packaging box 100 from the transport path 10.

  On the other hand, when the artificial intelligence program 26 erroneously determines that the packaging box 100 is defective or erroneously determines that the packaging box 100 is not defective, the worker M monitors the third image displayed on the display 40. When an erroneous determination is confirmed while the worker is doing, the worker M removes the erroneously determined packaging box 100 from the transport path 10 and uses the artificial intelligence program 26 for re-learning.

  As described above, in the inspection system 1 according to the present embodiment, the omnidirectional image of the packaging box 100 is based on the first image and the second image captured by the first camera 23 and the second camera 24 of the imaging device 21. Since it is generated as the third image, it is possible to provide an image that can be appropriately inspected as to whether or not the packaging box 100 is defective.

  In particular, in the present embodiment, the first camera 23 is disposed below the belt conveyor 13 in the gap A between one belt conveyor 13 and the other belt conveyor 13, and the packaging box 100 is placed on the belt conveyor 13. Since the surface is imaged from below the belt conveyor 13, it is possible to image a portion of the packaging box 100 that is difficult to image.

  In the present embodiment, the second camera 24 is configured by a plurality of cameras 24 a to 24 e that capture images of each surface of the packaging box 100, so that the packaging box 100 can be imaged in a multifaceted manner.

  The artificial intelligence program 26 stored in the defect determination means 20 of the inspection system 1 according to the present embodiment is stored in the packaging box 100 based on the third image generated by such an imaging device 21. Since it is determined autonomously whether or not there is a defect, it is possible to improve workability when determining whether or not the packaging box 100 has a defect.

  Since the packaging box 100 determined to have a defect is removed from the transport path 10 by the actuator 30, the packaging box 100 having a defect and the packaging box 100 having no defect are efficiently sorted.

  In addition, this invention is not limited to the said embodiment, A various change is possible in the range which does not deviate from the meaning of invention. Although the case where the inspection object is the packaging box 100 has been described in the above embodiment, the present invention is not limited to the packaging box 100, and various objects can be inspected.

  For example, as shown in FIG. 7, a defect such as a scratch, a dirt, or a dent on an automobile sunroof 101 that is transported in a factory using a rail 110 suspended from the factory ceiling in a manufacturing factory or an assembly factory as a transport path. It can be used to determine whether or not there is.

  In this case, as shown in the figure, the sunroof 101 is suspended from the viewpoint of appropriately determining whether the sunroof 101 is defective by imaging all directions of the sunroof 101 without deteriorating workability. A failure determination means 120 through which the sunroof 101 can pass is prepared so that the determination can be performed in the state.

  In the above-described embodiment, the case has been described in which each surface of the packaging box 100 having six surfaces is imaged by the six cameras 24a to 24e constituting the first camera 23 and the second camera 24, respectively. Depending on the shape of the packaging box 100 and the shape of the inspection object other than the packaging box 100, a plurality of cameras may be used to image the inspection object from the same direction.

DESCRIPTION OF SYMBOLS 1 Inspection system 10 Conveyance path 13 Belt conveyor 20, 120 Defect determination means 21 Imaging device 22 Detection sensor 23 1st camera (1st imaging means)
24 Second camera (second imaging means)
25 Third image generation program (third image generation means)
26 Artificial Intelligence Program 30 Actuator (Inspection Object Removal Means)
40 Display 100 Packaging box (object to be inspected)
101 Sunroof (object to be inspected)
D1 Inspection object model image D2 Defect determination reference data D3 Learning data D4 Learned model E Detection end F Conveyance direction S Detection start

Claims (9)

In the imaging apparatus that is provided in the conveyance path of the inspection system that determines whether or not the inspection object conveyed on the conveyance path has a defect, and images the inspection object,
A detection sensor for detecting the inspection object conveyed in the conveyance path;
First imaging means for imaging the inspection object from a first imaging direction based on detection of the inspection object by the detection sensor and generating a first image;
Second imaging means for imaging the inspection object from a second imaging direction different from the first imaging direction for imaging the inspection object by the first imaging means to generate a second image;
A third image that generates an omnidirectional image of the inspection object as a third image by associating the first image generated by the first imaging unit with the second image generated by the second imaging unit. Generating means;
An imaging apparatus comprising: The first imaging means includes
At the start of detection when the detection object is detected by the detection sensor, imaging of the inspection object is started from the first imaging direction to continue imaging, and detection ends when detection of the detection object is completed by the detection sensor The imaging apparatus according to claim 1, wherein the imaging of the inspection object is sometimes completed to generate the first image. The first imaging means includes
The imaging apparatus according to claim 1, wherein the imaging apparatus is disposed below the transport path and images the inspection object from below the transport path. The second imaging means includes
A plurality of imaging means for imaging the inspection object from a plurality of different imaging directions;
The imaging device according to claim 1, wherein a plurality of the imaging directions by the plurality of imaging units are set as the second imaging directions. In an inspection system comprising a conveyance path for conveying an inspection object, and a defect determination means for determining whether or not the inspection object conveyed on the conveyance path has a defect,
The defect determination means includes
A detection sensor for detecting the inspection object conveyed in the conveyance path;
First imaging means for imaging the inspection object from a first imaging direction based on detection of the inspection object by the detection sensor and generating a first image;
Second imaging means for imaging the inspection object from a second imaging direction different from the first imaging direction for imaging the inspection object by the first imaging means to generate a second image;
A third image that generates an omnidirectional image of the inspection object as a third image by associating the first image generated by the first imaging unit with the second image generated by the second imaging unit. Generating means;
A machine based on learning data generated by an inspection object model image in which an inspection object having a defect is imaged and defect determination reference data to be learned when a defect exists in the inspection object imaged in the inspection object model image An artificial intelligence program that autonomously determines from the third image whether there is a defect in the inspection object conveyed on the conveyance path based on a learned model that has been learned;
An inspection system comprising:   The inspection system according to claim 5, further comprising inspection object removal means for removing the inspection object determined to be defective by the defect determination means from the conveyance path. The first imaging means includes
At the start of detection when the detection object is detected by the detection sensor, imaging of the inspection object is started from the first imaging direction to continue imaging, and detection ends when detection of the detection object is completed by the detection sensor The inspection system according to claim 5 or 6, characterized in that the imaging of the inspection object is sometimes completed to generate the first image. The first imaging means includes
The inspection system according to any one of claims 5 to 7, wherein the inspection system is arranged below the conveyance path and images the inspection object from below the conveyance path. The second imaging means includes
A plurality of imaging means for imaging the inspection object from a plurality of different imaging directions;
The inspection system according to any one of claims 5 to 8, wherein a plurality of the imaging directions by the plurality of imaging means are set as the second imaging directions.

Images