JP2019028004A - Three-dimensional measuring device and method for detecting abnormality in three-dimensional measuring device - Google Patents

Abstract

To provide a technique of appropriately detecting substances attaching to a light transmitting part of an imaging part or a projection part in a three-dimensional measuring device.SOLUTION: The three-dimensional measuring device includes: a projection unit 1 for projecting a pattern light or a slit light to a measurement target; an imaging unit 2 for acquiring an image taken of the measurement target to which light is projected; and a distance acquiring unit 5 for acquiring information on the distance to the measurement target based on the taken image. The three-dimensional measuring device also includes: a region division unit 7 for dividing the taken image into at least one region according to the information on the distance to the measurement target; and a contrast comparison unit 8 for acquiring the contrast of the region and comparing the acceptable range of the contrast determined from the distance information and the contrast of the region.SELECTED DRAWING: Figure 1

Description

  The present invention detects a change (abnormality or defect) from a desired state of a part through which light passes in a three-dimensional measuring device such as a component recognition device that recognizes the position and orientation of a three-dimensional object (measurement target). It is related with the method to do.

  One technique for acquiring three-dimensional position information of a measurement target is a technique for measuring a distance image by a pattern projection method. In the pattern projection method, two-dimensional pattern light is projected onto a measurement space by projection means. Then, the pattern is detected from the image of the measurement space captured by the imaging means, the distance to the measurement target is measured by the triangulation method using the detected pattern, and the three-dimensional position information of the measurement target is acquired based on the measured distance. To do.

  The component recognition apparatus is used in various environments. For example, in an environment such as a factory where the concentration of oil mist is high, fine oil particles are floating in the air. A light transmissive plate-like member such as glass is generally provided at the boundary of the projection means and the photographing means with the outside of the apparatus. When oil fine particles adhere to this glass, projection by the projection means is caused thereby. The amount of light is reduced or the image taken by the photographing means is blurred.

  Patent Document 1 discloses a technique for detecting an abnormality or change occurring in the welding sensor itself using an image of slit light imaged by the welding sensor. A difference image between the captured image and the reference image is generated, and the abnormality of the window panel is determined using the luminance distribution of the difference image.

  Patent Document 2 discloses a technique for detecting fogging and dirt on a cover glass of a distance measuring device and defective pixels in a photosensor array. The degree of coincidence between the images of the two light receivers is used as an index for contrast reduction to detect the clouding or dirt of the optical system.

JP 2014-153155 A Japanese Patent Laid-Open No. 11-2519

  As described above, in an environment where the concentration of oil mist is high, the amount of projected light is reduced or the captured image is blurred. Therefore, problems such as misrecognizing parts occur when the dirt becomes severe. In the component recognition apparatus, since there is usually a process in which the robot takes out the recognized component, the state of the component to be imaged changes each time it is recognized. Therefore, the method for discriminating an abnormality based on the difference image disclosed in Patent Document 1 cannot be directly applied to such a three-dimensional measurement device or recognition device.

  Also, since the parts are usually piled up and placed in the pallet, there is a large difference in height between the parts above the pallet and the parts at the bottom, and all the parts are not at the just focus position. Therefore, even if the component exists within the depth of field, the contrast of the captured image has a value having a certain distribution. Therefore, in the case of performing stain detection using a lowered contrast value, erroneous detection is likely to occur unless the contrast tolerance is widened. For such problems, the technique disclosed in Patent Document 2 does not teach a solution.

  The present invention has been made in view of the above-described problems, and divides a captured image based on distance information with respect to a measurement target to acquire a contrast of the divided area, and based on this, the light transmission in the imaging unit and the projection unit Provided is a technology that makes it possible to appropriately detect abnormalities such as deposits on the part.

  A three-dimensional measurement apparatus according to an aspect of the present invention shoots a projection unit that projects pattern light or slit light onto a measurement target, and captures the measurement target on which the pattern light or slit light is projected. An imaging unit that acquires a captured image, a distance acquisition unit that acquires information on the distance to the measurement target based on the captured image, and the captured image in one or more regions according to the information on the distance to the measurement target An area dividing unit that divides the image; and a contrast comparing unit that obtains the contrast of the region and compares the contrast allowable range determined according to the distance information with the contrast of the region.

  According to one aspect of the present invention, it is possible to appropriately detect the presence of an abnormality such as a deposit on a light transmission portion in a projection unit or a photographing unit.

The figure which shows the structure of an example of the three-dimensional measuring apparatus thru | or component recognition apparatus which concern on this invention. The figure explaining the outline of a screen division | segmentation and an area | region division. The flowchart which shows the whole process of a components recognition apparatus. The flowchart which shows the deposit | attachment detection method. The figure explaining the allowable range of contrast. The flowchart which shows the other adhering matter detection method. The figure which shows the control system with which a measuring device is equipped with a holding device and is used.

  In one aspect of the present invention, information on a distance to a measurement target is acquired based on a captured image of the measurement target on which pattern light or slit light is projected, and a three-dimensional shape of the measurement target is measured based on the distance information. To do. Here, in order to detect abnormalities such as deposits on the light transmitting part of the projection unit used in the projection process and / or the imaging unit used in the imaging process, the following is performed. That is, the captured image is divided into one or more areas according to the distance information with respect to the measurement object, the contrast of the divided areas is acquired, and the contrast tolerance range determined according to the contrast of the divided areas and the distance information is set. Compare. Then, the presence or absence of the abnormality is detected based on the comparison result. Since the captured image is divided into one or more regions according to the distance information with respect to the measurement target and the contrast is acquired in the divided regions, the contrast variation of each part is not excessively large in the divided regions according to the distance. Can be used to improve the reliability of the contrast of the divided areas.

  Therefore, by comparing the contrast allowable range determined in accordance with the distance information and the contrast of the divided areas, a result of improved reliability of abnormality detection can be obtained. Therefore, based on this, it is possible to appropriately cope with an abnormality such as a deposit on the light transmitting portion in the projection unit or the photographing unit.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings. However, the scope of the present invention is not limited to these specific examples.

[First Embodiment]
FIG. 1 is a diagram illustrating a configuration of a component recognition apparatus that is a three-dimensional measurement apparatus according to the present embodiment. The component recognition apparatus includes a projector 1 that is a projection unit and a camera 2 that is a photographing unit. In addition, the measurement control unit 3, the screen division unit 4, the distance image generation unit 5 including a distance acquisition unit that acquires distance information, the component recognition unit 6, the region division unit 7, the contrast comparison unit 8, the contrast table 9, and the display unit 10. Is provided.

  The reflected light of the pattern light projected by the projector 1 is photographed by the camera 2, and the photographed image is sent to the screen dividing unit 4 and the distance image generating unit 5. The distance image generation unit 5 generates distance information of each part to be measured using a principle generally known as a triangulation method. The component recognition unit 6 recognizes the component shape (posture, position, etc. of the measurement target) using the distance information of the measurement target, and sends the recognition result to a robot controller or a sequence controller (for example, the control unit 420 in FIG. 7 described later). Tell.

  The screen dividing unit 4 divides the captured image of the camera 2 into a predetermined finite number of areas. The area dividing unit 7 further divides the section into a finite number of areas for each predetermined distance range using the distance information obtained by the distance image generating unit 5. Here, after the captured image is divided into a finite number (a plurality) of areas by the screen dividing unit, each area is further divided by the area dividing unit, but this is not restrictive. For example, when the size of the captured image is not so large (the amount of data is relatively small), or when the processing by the contrast comparison unit is very fast, the processing by the screen dividing unit may be omitted.

The contrast comparison unit 8 calculates the contrast value of the area, and if the contrast value is outside the predetermined allowable range, for example, a message indicating that the projector 1 or / and the camera 2 has detected an attachment or the like is displayed on the display unit 10. To display. The contrast value of the area is a difference in luminance between the darkest part and the brightest part in the area, and its calculation formula is generally well known. When the maximum luminance value in the region is Lmax and the minimum value is Lmin, for example, the Michelson contrast can be calculated by the following equation (1).
(Lmax-Lmin) / (Lmax + Lmin) Equation 1
Instead of Michelson contrast, the luminance ratio Lmax / Lmin can also be used.

  The contrast table 9 is a table that stores values of acceptable contrast ranges according to distance ranges. For example, since the color, material, and light reflectance differ depending on the measurement object, the allowable range value is determined in advance for each measurement object. In general, an optical lens is used for a camera or a projector, but the focus is not achieved in a range exceeding the depth of field of the lens. In an out-of-focus range, the edge of the object to be imaged becomes unclear and the contrast decreases. Even in the in-focus range, the contrast changes depending on the distance to the object to be photographed, so the recognition performance of the component recognition device is determined depending on how far blur is allowed. For this reason, in order to appropriately detect the attached matter, the range where the allowable range of contrast reaches a relatively large value depending on the distance to the object to be photographed for a relatively focused area. It is good to set to. On the other hand, for an area that is not in focus, it is preferable to set the allowable contrast range to a relatively small value in accordance with the distance from the subject.

  The outline of screen division and area division will be described with reference to FIG. FIG. 2A is a diagram showing a state in which the measurement target space is divided. X and Y are coordinate axes of a two-dimensional image taken, and Z is a coordinate axis of a distance obtained by triangulation. A pixel of the generated distance image belongs to one of the divided areas.

  In order to simplify the description, FIG. 2B is a diagram in which a measurement target is a cylindrical object, and a state in which one cylinder is accommodated in one area in the captured image is viewed from the camera. Fig. 2 (C) shows this area as seen from the side. Here, the unit length of the division into the areas is set to a predetermined value, but the value may be set to a length that can identify a decrease in contrast due to a minimum-size stain to be detected. FIG. 2 (C) shows a case where the distance to the measurement object is divided into three areas: an area Z0 from 0 to h1, an area Z1 from h1 to h2, and an area Z2 above h2, and is easy to understand. Z0 is white, Z1 is hatched, and Z2 is black. Similarly, in FIG. 2 (B), the regions are shown separately. As shown in FIG. 2B, the two-dimensional division area is further divided into areas according to the distance, and the contrast value is calculated for each area. As described above, in the region divided by the distance, the reliability of the calculated contrast value is high.

  For example, when the contrast of all the areas in the same area is out of the allowable range, it can be determined that the contrast is lowered due to a defect such as dirt of a light transmissive member such as a lens or a light transmissive member such as glass. . Alternatively, when there are a large number (for example, 2/3 or more) of regions in which the contrast is outside the allowable range in the same area, it may be determined that the contrast is lowered. The above determination is not performed for only one area and the result of determination of the presence / absence of abnormality is not generated, but the abnormality is detected only when there are more than a predetermined ratio (for example, 2/3 or more) of areas where it can be determined that the contrast is reduced. You may make it determine with there. In the case of an adhering substance, the adhering substance may be removed with a human hand or an installed wiper based on the above determination. In the latter case, the wiper can be moved and automated in response to the determination of the attached matter detection.

  A schematic flow of the entire processing of the component recognition apparatus of the present embodiment will be described with reference to the flowchart of FIG. For all measurement objects piled up in the pallet (repeating steps S101 to S106), the projector 1 projects pattern light onto the measurement object space, and the reflected light is imaged with the camera 2 (step S102). Here, the pattern light uses a bright and dark stripe pattern. Next, distance information is generated from the captured image (step S103). A method for generating distance information is generally known as a spatial encoding method, for example. For example, the adhering matter is detected using the image photographed in step S102 (step S104). Next, the three-dimensional recognition of the object is performed from the distance information (step S105), and the process ends when the parts are recognized for all the objects.

  Further, the operation of the detection process for adhered matter and the like will be described in detail with reference to the flowchart of FIG. As described above with reference to FIG. 2, the captured image is divided (step S201), and all the divided areas (steps S202 to S211 are repeated) according to the distance from the component recognition device to the measurement target. Then, the area is divided (step S203). For each divided region (repetition of steps S204 to S209), the contrast is calculated for each region using, for example, Equation 1 (step S205). A predetermined allowable contrast range is extracted from the contrast table 9 (step S206), and a comparison is made as to whether the calculated contrast value is within the allowable range (step S207). An area outside the allowable range is stored (step S208), and a warning is displayed when the contrast of all or most of the areas in the area is out of the allowable range (step S210). Here, it is possible to determine whether or not the contrast is within an allowable range for a plurality of regions in one area, and to determine whether or not to issue a warning based on the result. However, as described above, the contrast comparison unit 8 performs the comparison for all the areas, and it is determined that the contrast of the area of the predetermined ratio or more is outside the allowable range in the area of the predetermined ratio or more. Only in such a case, an abnormality detection warning may be issued.

  The relationship between the contrast and the allowable range will be described with reference to FIG. The horizontal axis of the graph is the distance to the measurement object, and the vertical axis is the contrast. (A) shows the case where there is no deposit on the lens, etc., (B) shows the case where there is a small deposit on the lens, etc., (C) shows the case where there is a large deposit on the lens, etc. It is shown superimposed on one graph. Here, the contrast of each region is expressed by a curve, but actually, one contrast value is calculated for each region. Thus, the contrast of each region is represented as a horizontal line parallel to a certain level of the horizontal axis.

  When there is no deposit on the lens, the value of (A) is used regardless of the distance in order to prevent false detection of dirt, etc. Must be set within a wide allowable range of c0 to c2. On the other hand, if it is limited to the area from the distance h1 to h2 (that is, if other areas are not taken into consideration), in order to prevent erroneous detection of dirt, for example, a narrow allowable range of c1 to c2 may be set.

  When the deposit is small, the contrast is not so low as in (B), and dirt etc. cannot be detected in the allowable range from c0 to c2, but it is set to the allowable range from c1 to c2 considering the distance. Even small dirt can be detected. That is, in the allowable range from c0 to c2, (A) and (B) are within the allowable range in all areas of Z0, Z1, and Z2, and it is determined that “no deposit”. However, if the allowable range of c1 to c2 is set, the regions Z0 and Z2 are not considered, and the contrast value is determined to be outside the allowable range in the region of Z1 with respect to (B), and small deposits can be detected. Here, the focus is in the region Z1 from h1 to h2, and it is considered that the contrast decreases in the regions Z0 and Z2 on both sides due to defocusing. Therefore, since it is not clear whether these areas Z0 and Z2 depend on dirt or out of focus due to the decrease in contrast, in order to avoid erroneous detection, it may be excluded from consideration of determination as described above. . Alternatively, in the areas Z0 and Z2, a lower allowable range may be selected from the contrast table 9 that stores the value of the allowable contrast range according to the distance range, and the determination may be performed. (C) shows the case where there is a large amount of adhered matter on the lens, etc., regardless of what tolerance is set for each area, the contrast value is judged to be outside the tolerance, and there is dirt etc. To be judged.

  In the present embodiment, the pattern light used when capturing an image necessary for contrast calculation is a stripe pattern that is used in the spatial coding method. However, a different pattern such as a checkered pattern, such as a checkered pattern, that makes the contrast more prominent. It may be. Moreover, in order to measure a three-dimensional shape by a light cutting method and to capture an image necessary for contrast calculation, it is possible to project slit light on a measurement target. In the above description, for the sake of convenience, screen division and area division have been described using a cylindrical measurement target, but the shape of the measurement target is arbitrary.

  Furthermore, depending on the measurement target, there may be a partially different reflectance distribution due to a difference in surface treatment or a composite material. In that case, the surface reflecting the pattern light is obtained using the posture of the measurement object recognized by the component recognition unit 6. It is possible to cope with this by selecting an allowable range of contrast considering the surface reflectance of the surface from the allowable range stored in the contrast table and switching to it.

  As described above, the allowable range value in the contrast table can be determined in consideration of the type of measurement target, that is, the color, material, reflectance, etc. of the measurement target. You can also adjust the value.

  FIG. 6 shows the operation of the adhering matter detection process during learning. The same functions as those in FIG. 4 are given the same numbers. The comparison result between the contrast value and the allowable range is displayed on the display unit 10 (step S301), and the apparatus adjustment person determines whether the result is acceptable and adjusts the allowable range value (step S302). The adjustment width of the value may be input by a person in charge of device adjustment, or may be generated using a generally known learning algorithm. For example, a system (for example, including a monitoring camera) that can automatically determine whether the result is acceptable is provided, and the value of the allowable range is adjusted based on the determination. If it is determined whether or not it is possible to determine that dirt or the like has been detected, and if it is wrong (no dirt or the like actually exists), the allowable range value is reset to a lower value. On the other hand, when it is determined whether or not it is possible to determine that dirt is not detected, if it is wrong (in fact, dirt or the like is present), the value of the allowable range is reset to a higher value.

  According to the present embodiment, normally, the presence of such an abnormality or defect can be appropriately detected without separately providing a camera or the like that directly monitors an abnormality such as a deposit on a projection unit or a photographing unit. Of course, a monitoring camera or the like for determining whether the determination result is acceptable can be provided separately.

[Second Embodiment]
The above-described three-dimensional measuring apparatus of the present invention can be used in a state of being linked with a robot arm. In this three-dimensional measuring apparatus, dirt and the like are removed based on the determination as described above. In the present embodiment, as an example, a control system in which the measuring device 400 is provided to the robot arm 410 (gripping device) as shown in FIG. 7 will be described. The measuring device 400 projects and images pattern light on a measurement target (work) 460 placed on the support base 450 to acquire an image. Then, the measurement apparatus 400 obtains the position and orientation of the measurement target 460 from the acquired image data. Based on the position and orientation of the measurement target 460 from the measurement apparatus 400, the control unit 420 determines which measurement target is to be gripped with which gripping posture. Here, the measuring device is provided outside the gripping device, and a command may be output to the control unit 420 from this.

  Thereafter, the control unit 420 controls the robot arm by sending a drive command to the robot arm 410 based on the determined position and orientation information. The robot arm 410 holds the measurement target 460 with a robot hand (gripping unit) at the tip, and moves the robot arm 410 such as translation and rotation. Furthermore, by assembling the measurement object 460 to other parts by the robot arm 410, an article composed of a plurality of parts, such as an electronic circuit board or a machine, can be manufactured. Moreover, an article can be manufactured by processing the moved measurement object 460. For example, the welding process can be performed on an appropriate welded part of the moved measurement object 460 using another robot arm having a welding tool in the gripping part. This robot arm also includes the measuring device of the present invention, and controls the position and orientation of the welding tool of the gripping part based on the information from now on. In an environment where such a welding tool is present, fine particles and the like are likely to float in the air, and thus the measuring device to which the abnormality detection technology of the present invention is applied can exhibit more effects.

  The control unit 420 includes an arithmetic device such as a CPU and a storage device such as a memory. Further, the measurement data measured by the measurement apparatus 400 and the obtained image may be displayed on a display unit 430 such as a display (which may also serve as the display unit 10 shown in FIG. 1).

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

1: Projector (projection unit)
2: Camera (shooting unit)
4: Screen division unit 5: Distance image generation unit (distance acquisition unit)
6: Component recognition unit 7: Area division unit 8: Contrast comparison unit

Claims (16)

A projection unit that projects pattern light or slit light onto a measurement target;
An imaging unit that captures the measurement target on which the pattern light or slit light is projected, and acquires a captured image of the measurement target;
A distance acquisition unit that acquires information on the distance to the measurement object based on the captured image;
An area dividing unit that divides the captured image into one or more areas in accordance with information on the distance to the measurement target;
A three-dimensional measurement apparatus comprising: a contrast comparison unit that obtains the contrast of the region and compares the contrast allowable range determined according to the distance information with the contrast of the region.   The three-dimensional measurement apparatus according to claim 1, wherein an abnormality of the light transmission unit of the projection unit and / or the imaging unit is detected based on a result of comparison by the contrast comparison unit.   The three-dimensional measurement apparatus according to claim 2, wherein the adhering matter attached to the light transmission part of the projection part and / or the photographing part is detected based on a result of comparison by the contrast comparison part.   The contrast comparison unit performs the comparison for all the regions, and issues a warning for the abnormality detection when it is determined that the contrast of the region of a predetermined ratio or more is outside the allowable range. The three-dimensional measuring apparatus according to claim 2 or 3.   5. The three-dimensional measurement apparatus according to claim 1, wherein the contrast comparison unit switches an allowable range of the contrast depending on a type of the measurement target.   5. The three-dimensional image according to claim 1, wherein the contrast comparison unit switches an allowable range of the contrast according to a posture of the measurement object and a surface reflectance of each part of the measurement object. 6. Measuring device. A projection step of projecting pattern light or slit light onto the measurement object;
Photographing the measurement object on which the pattern light or slit light is projected, and obtaining a photographed image of the measurement object;
A distance acquisition step of acquiring information on the distance to the measurement object based on the captured image;
An area dividing step of dividing the captured image into one or more areas according to distance information with respect to the measurement object. The contrast of the area is acquired, and an allowable range of contrast determined according to the distance information and the area A contrast comparison step for comparing the contrast;
Based on the result of the comparison in the contrast comparison step, a detection step of detecting an abnormality in the light transmission portion of the projection unit used in the projection step and / or the light transmission portion of the imaging unit used in the imaging step;
A method for detecting an abnormality in a three-dimensional measuring apparatus.   The abnormality detection method according to claim 7, wherein, in the detection step, an adhering matter attached to a projection unit used in the projection step and / or a photographing unit used in the photographing step is detected.   In the contrast comparison step, the comparison is performed for all the regions, and in the detection step, the abnormality detection warning is issued when it is determined that the contrast of the region of a predetermined ratio or more is outside the allowable range. The abnormality detection method according to claim 7 or 8, wherein the abnormality is detected.   The abnormality detection method according to any one of claims 7 to 9, wherein, in the contrast comparison step, an allowable range of the contrast is switched depending on a type of the measurement target.   10. The abnormality detection according to claim 7, wherein the contrast comparison step switches an allowable range of the contrast depending on a posture of the measurement object and a surface reflectance of each part of the measurement object. Method.   A program that causes a computer to function as means for executing each step of the abnormality detection method according to any one of claims 7 to 11. The three-dimensional measurement apparatus according to any one of claims 1 to 6, which obtains information on a position and orientation of an object to be gripped by a gripping unit;
A robot that grips and moves the object based on the information obtained by the three-dimensional measuring device;
The system characterized by having. The three-dimensional measuring device according to any one of claims 1 to 6, which obtains information on a position and a posture of an object to be welded by a welding tool;
A robot that moves the welding tool to the welded part of the object based on the information obtained by the three-dimensional measuring device;
The system characterized by having. The information on the position and orientation of the object to be gripped by the gripping unit is obtained, and the object is gripped and moved based on the information obtained by the three-dimensional measurement apparatus according to any one of claims 1 to 6. Moving process;
A processing step of processing the object moved in the moving step;
Have
An article manufacturing method comprising manufacturing an article from the object subjected to the treatment.   In the processing step, information on the position and orientation of an object to be welded by a welding tool is obtained by a robot based on the information obtained by the three-dimensional measurement apparatus according to any one of claims 1 to 6. The article manufacturing method according to claim 15, wherein a welding process is performed on the object by moving a welding tool to a welded portion of the object.

Images