JP6266244B2 - 3D image processing apparatus, 3D image processing method, 3D image processing program, computer-readable recording medium, and recorded apparatus - Google Patents

Description

  The present invention relates to a three-dimensional image processing apparatus, a three-dimensional image processing method, a three-dimensional image processing program, a computer-readable recording medium, and a recorded device.

  In many production sites such as factories, an image processing apparatus that automates and speeds up the inspection that relies on human visual inspection has been introduced. The image processing apparatus images a workpiece flowing on a production line such as a belt conveyor with a camera, and performs measurement processing such as edge detection and area calculation of a predetermined region using the obtained image data. Then, based on the processing result of the measurement process, inspection such as workpiece chipping detection or alignment mark position detection is performed, and a determination signal for determining the presence or absence of workpiece chipping or positional deviation is output. As described above, the image processing apparatus may be used as one of FA sensors.

  An image that is an object of measurement processing by an image processing apparatus used as an FA sensor is mainly a luminance image that does not include height information. For this reason, regarding the above-mentioned chipping detection of a workpiece, it is good at stably detecting the two-dimensional shape of the chipped portion, but the three-dimensional shape which is difficult to appear as a luminance image such as a dent of a scratch is stably displayed. It is difficult to detect. For example, it is conceivable to devise the type and direction of illumination that illuminates the workpiece during inspection, detect shadows due to dents on the scratch, and indirectly detect the three-dimensional shape, but in the luminance image Clear shadows are not always detected. In order to prevent misjudgment, for example, when the judgment threshold is biased to the safe side, it is judged that the non-defective product is judged as a defective product in large quantities and the yield deteriorates. There is a risk of inviting.

  Therefore, not only the luminance image with the pixel value as the gray value according to the amount of light received by the camera, but also the pixel value with the gray value according to the distance from the camera to the workpiece, the height is expressed two-dimensionally. An appearance inspection using a distance image (see, for example, Patent Document 1) can be considered.

  An example of a three-dimensional image processing apparatus is shown in the schematic diagram of FIG. The three-dimensional image processing apparatus includes a head unit including an imaging unit such as a light receiving element, and a controller that is connected to the head unit and that receives image data captured by the head unit and generates a distance image from the acquired image data. It consists of parts.

  Here, the principle of triangulation will be described with reference to FIG. In the head unit, an angle α between the optical axis of incident light emitted from the light projecting unit 110 and the optical axis of reflected light incident on the light receiving unit 120 (the optical axis of the light receiving unit 120) is set in advance. . Here, when the work is not placed on the work, the incident light emitted from the light projecting unit 110 is reflected by the point O on the work placing surface and is incident on the light receiving unit 120. On the other hand, when the work is placed on the work, the incident light emitted from the light projecting unit 110 is reflected by the point A on the surface of the work and is incident on the light receiving unit 120 as reflected light. Then, the distance d in the X direction between the point O and the point A is measured, and the height h of the point A on the surface of the workpiece is calculated based on the distance d.

  The three-dimensional shape of the workpiece is measured by calculating the heights of all points on the surface of the workpiece by applying the measurement principle of the triangulation described above. In the pattern projection method, incident light is emitted from the light projecting unit 110 according to a predetermined structured pattern in order to irradiate all points on the surface of the work, and the reflected light reflected from the work surface is received and received. The three-dimensional shape of the workpiece is efficiently measured based on the plurality of pattern images.

  As such a pattern projection method, a phase shift method, a spatial encoding method, a multi-slit method, and the like are known. By the three-dimensional measurement process using the pattern projection method, the projection pattern is changed, and a plurality of imaging operations are repeated at the head unit and sent to the controller unit. The controller unit can perform calculation based on the pattern projection image sent from the head unit, and obtain a distance image having workpiece height information.

  On the other hand, in an existing image processing apparatus, a luminance image mainly using luminance as a pixel value is used. For example, there is a system that picks up a luminance image with a monochrome camera and inspects a workpiece conveyed on a line by image processing. In such a situation, if it is going to introduce a new three-dimensional measuring device and a processing device for processing three-dimensional data (point cloud data) output from the new three-dimensional measuring device, a considerable cost is required.

  Therefore, if the height information is expressed as a distance image expressed as the shading of the image, the image data itself is equivalent to the existing luminance image. Therefore, even in the equipment of the image processing apparatus using the existing luminance image, the distance You can handle images.

  In this case, in the conventional luminance image, an image having a relatively low number of gradations has been used. For example, an image in which information per pixel is expressed in 8 gradations is often used. On the other hand, image data having height information requires a large number of gradations (for example, 16 gradations) in order to express the height information. For this reason, it is necessary to convert a multi-tone distance image into a relatively low-tone distance image so that the distance image can be processed by an existing image processing apparatus that handles low-tone images. . (Hereinafter, an image obtained by converting a multi-tone distance image into a low-tone distance image is referred to as a “low-tone distance image”, and a conventional luminance image having luminance information is referred to as a “luminance image”.)

  In this way, in addition to the conventionally used luminance image, an image such as conventional shape extraction is used at the time of inspection for determining normality / abnormality of a workpiece by using a gradation-converted low gradation distance image. In addition to processing, processing such as height measurement using height information can be used in a three-dimensional image processing apparatus.

  On the other hand, the existing inspection apparatus can handle only the luminance image. Here, if the distance image is converted into a low gradation luminance image in which the height is replaced with a luminance value, it can be handled in the same manner as an existing luminance image. That is, the inspection processing tool and image processing technology provided in the existing inspection apparatus can be applied as they are.

  On the other hand, an inspection processing tool that can be applied only to a distance image, for example, a tool that measures the height in an image, naturally cannot be used for a luminance image, and only a distance image is a target.

  However, in many inspection processing tools, there is a possibility that the user may be confused when an inspection processing tool that can be applied to the distance image and an inspection processing tool that cannot be applied to the distance image are mixed. For example, if a luminance image is selected when performing height measurement, the height measurement cannot be performed, which may cause confusion for the user.

JP 2012-21909 A

  The present invention has been made to solve such conventional problems. A main object of the present invention is a three-dimensional image processing apparatus, a three-dimensional image processing method, a three-dimensional image processing program, and a computer, which are improved in operability so that the combination of an inspection processing tool and a target image is not confused. To provide a readable recording medium and a recorded device.

Means for Solving the Problems and Effects of the Invention

In order to achieve the above object, according to the three-dimensional image processing apparatus according to the first aspect of the present invention, a distance image including height information of an inspection object, a luminance image obtained by imaging the inspection object, Is a three-dimensional image processing apparatus capable of performing image processing based on the distance image or the luminance image, and for projecting incident light as structured illumination of a predetermined projection pattern And imaging for capturing a plurality of pattern projection images by acquiring the reflected light projected by the projecting unit and reflected by the inspection object from an oblique direction with respect to the optical axis of the projecting unit Means, a distance image generating means capable of generating a distance image based on a plurality of pattern projection images imaged by the imaging means, and an inspection for executing a predetermined inspection process on the distance image or the luminance image Execution means and the inspection execution means One or more selected from a plurality of inspection processes including a height inspection process using the height information of the distance image and an image inspection process performed by replacing the height information of the distance image with grayscale information. Inspection processing selection means for performing inspection processing setting means for setting inspection processing conditions including selection of a setting image to be subjected to the inspection processing for each inspection processing selected by the inspection processing selection means And when the inspection process selected by the inspection process selection unit is an inspection process that can be performed on either a distance image or a luminance image, the inspection process setting unit sets the image as a setting image. While the distance image or the luminance image can be selected, the inspection processing setting unit can execute the inspection processing that can be performed on the distance image and cannot be performed on the luminance image. The distance image can be selected as the regular image, but the luminance image cannot be selected. The image inspection process is selected by the inspection process selection unit, and the distance image is selected as the setting image. In this case, it is possible to display the same settings relating to the image inspection processing as when the luminance image is selected. With the above configuration, setting errors can be avoided by making it possible to select only executable images for the selected inspection process.

  Further, according to the three-dimensional image processing apparatus according to the second aspect, the inspection processing setting means, when selecting the setting image to be the inspection processing target, the inspection processing that is a target for setting the inspection processing condition is: In the case of inspection processing that can be performed on either a distance image or a luminance image, the distance image or luminance image is displayed as an option, while it can be performed on the distance image and is performed on the luminance image. If the inspection process is not possible, the luminance image can be hidden while the distance image is displayed as an option. With the above configuration, when setting an inspection process that cannot be performed on a luminance image, it is possible to avoid a situation in which a user mistakenly sets a condition that cannot be set as a state where the luminance image is not displayed as an option of the registered image.

  Furthermore, the 3D image processing apparatus according to the third aspect further includes a distance image storage unit for recording a distance image and a luminance image storage unit for recording a luminance image. A unique image variable is assigned to the image and the luminance image, and when registering the distance image and the luminance image as the registered image, the image variable of each image is also registered, and the corresponding registered image can be called with the image variable. Can be configured. With the above configuration, for example, registration image selection in the inspection processing setting unit can be performed using an image variable as an index, and the user can easily select a desired distance image or luminance image.

  Furthermore, according to the three-dimensional image processing apparatus according to the fourth aspect, the height of the inspection object is measured for the inspection process that can be performed on the distance image and cannot be performed on the luminance image. It can be a height measurement process. With the above configuration, in the height measurement process, only a distance image can be selected as a registered image, and an erroneous setting can be avoided.

  Furthermore, according to the three-dimensional image processing apparatus according to the fifth aspect, both the distance image and the luminance image can be stored in the distance image storage unit and the luminance image storage unit as setting images for examination settings, respectively. it can.

  Furthermore, according to the three-dimensional image processing apparatus according to the sixth aspect, the luminance image can be an image obtained by imaging an inspection object using the light projecting means as a full illumination pattern.

  Furthermore, according to the three-dimensional image processing apparatus according to the seventh aspect, it is possible to capture a luminance image with the imaging unit using an illumination unit different from the light projecting unit.

  Furthermore, according to the three-dimensional image processing apparatus according to the eighth aspect, the luminance image can be generated by combining a plurality of pattern projection images from which the distance image is generated.

  Furthermore, according to the three-dimensional image processing device according to the ninth aspect, the light projecting means projects structured illumination for obtaining a distance image using at least the phase shift method and the spatial coding method. Can do.

  Furthermore, according to the three-dimensional image processing apparatus according to the tenth aspect, the distance image generated by the distance image generation means is further converted into the distance image having a gradation number lower than the gradation number of the distance image. Gradation conversion means for converting the gradation information into a low gradation distance image obtained by replacing the height information of the image with the gray value of the image, and gradation conversion for performing gradation conversion of the distance image by the gradation conversion means A gradation conversion condition setting means for setting parameters and a display means for displaying a low gradation distance image subjected to gradation conversion by the gradation conversion means can be provided.

  Furthermore, according to the three-dimensional image processing device according to the eleventh aspect, the head unit includes a head unit and a controller unit, the head unit includes the light projecting unit and the imaging unit, and the controller unit includes: The gradation conversion condition setting means can be provided.

  Furthermore, according to the three-dimensional image processing device according to the twelfth aspect, the gradation conversion parameter can include a plane offset amount serving as a reference for gradation conversion and a gradation width to which gradation conversion is performed. .

Furthermore, according to the three-dimensional image processing apparatus according to the thirteenth aspect, the inspection processing selection means individually selects one or more inspection processing for the distance image, the low gradation distance image, or the luminance image. The inspection execution means can be configured to execute a predetermined inspection process selected by the inspection process selection means on the distance image, the low gradation distance image, or the luminance image.
Furthermore, according to the three-dimensional image processing apparatus of the fourteenth aspect, a monochrome CCD camera or a color CCD camera that captures a luminance image can be connected instead of the imaging means, and the monochrome CCD camera or the color CCD can be connected. An image inspection process can be executed by the inspection execution unit on the luminance image captured by the camera.

Furthermore, according to the three-dimensional image processing method according to the fifteenth aspect, the distance image including the height information of the inspection object and the luminance image obtained by imaging the inspection object are acquired, and the distance image or A three-dimensional image processing method for performing image processing based on a luminance image, the step of projecting incident light as structured illumination of a predetermined projection pattern from a direction oblique to an optical axis of an imaging unit by a projecting unit; A step of acquiring reflected light projected by the light projecting means and reflected by the inspection object and capturing a plurality of pattern projection images by the image capturing means; and a plurality of pattern projection images captured by the image capturing means. Based on the step of generating a distance image by the distance image generating means, and a predetermined inspection process executed on the distance image or the luminance image, a height inspection process using height information of the distance image, the distance High image has And selecting one or more inspection process selecting means from the plurality of inspection processing including the image inspection process performed by replacing the information in the shading information for each inspection processing said selected settings for image to be the test process A step of performing a selection, and a step of executing a predetermined inspection process on the selected distance image or luminance image by an inspection execution unit. When selecting the setting image, the inspection process includes: In the case of inspection processing that can be performed on either an image or a luminance image, the distance image or the luminance image can be selected as the setting image, and can be performed on the distance image. If it is impossible inspection process performed on, while allowing the selection of the distance image as setting image, it has impossible to select the luminance image, the image inspection by the inspection processing selection means Management is selected, and when the distance image is selected as the setting image, it is Rukoto display the settings for the same image inspection process and when the luminance image is selected. Accordingly, setting errors can be avoided by making it possible to select only executable images for the selected inspection process.

Furthermore, according to the three-dimensional image processing program according to the sixteenth aspect, the distance image including the height information of the inspection object and the luminance image obtained by imaging the inspection object are acquired, and the distance image or A three-dimensional image processing program capable of performing image processing based on a luminance image, and projecting incident light from a direction oblique to the optical axis of an imaging means as structured illumination of a predetermined projection pattern A distance image generation function capable of generating a distance image based on a plurality of pattern projection images acquired by acquiring reflected light reflected by the inspection object, and a predetermined distance image or luminance image as inspection process executed by the test execution function and the inspection execution function for executing inspection processing, the height inspection process using height information the distance image has a height information the distance image has shading An inspection processing selection function for selecting one or more of a plurality of inspection processing including the image inspection process performed by replacing the distribution, for each inspection processing selected by the inspection processing selection function, for setting as a target of the inspection process An inspection processing setting function for setting inspection processing conditions including image selection is realized, and the inspection processing selected by the inspection processing selection function can be executed for either a distance image or a luminance image. In the case of inspection processing, the distance image or the luminance image can be selected as the setting image by the inspection processing setting function, while it can be executed on the distance image and cannot be executed on the luminance image. If a test process, while allowing the selection of the distance image as setting image in the inspection process setting function, constitute impossible to select the luminance image, image in the inspection processing selection function Inspection processing is selected, and when the distance image is selected as the setting image, it is possible to display the settings related to similar image inspection process and when the luminance image is selected. With the above configuration, setting errors can be avoided by making it possible to select only executable images for the selected inspection process.

Furthermore, a computer-readable recording medium or a stored device according to the seventeenth aspect stores the three-dimensional image processing program. CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, Blu-ray (registered) Trademark), HD DVD (AOD), and other magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and other media that can store programs. The program includes a program distributed in a download manner through a network line such as the Internet, in addition to a program stored and distributed in the recording medium. Furthermore, the stored devices include general-purpose or dedicated devices in which the program is implemented in a state where it can be executed in the form of software, firmware, or the like. Furthermore, each process and function included in the program may be executed by computer-executable program software, or each part of the process or hardware may be executed by hardware such as a predetermined gate array (FPGA, ASIC), or program software. And a partial hardware module that realizes a part of hardware elements may be mixed.

1 is a diagram illustrating a system configuration example of a three-dimensional image processing system including an image processing apparatus according to an embodiment of the present invention. It is a figure which shows the system configuration example of the three-dimensional image processing system which concerns on the modification of this invention. It is a schematic diagram which shows the hardware constitutions of the three-dimensional image processing apparatus which concerns on Embodiment 2 of this invention. FIG. 4A is a schematic diagram showing a head part of a 3D image processing apparatus according to Embodiment 3 of the present invention, and FIG. 4B is a schematic diagram showing a head part of the 3D image processing apparatus according to Embodiment 4. It is a block diagram which shows the three-dimensional image processing apparatus which concerns on Embodiment 3 of this invention. It is a block diagram which shows the controller part of FIG. 5 is a flowchart illustrating a processing operation of the image processing apparatus according to the present embodiment. It is a flowchart which shows the procedure of the static conversion at the time of a setting. It is an image figure which shows the initial screen which added "imaging" with the three-dimensional image processing program. It is an image figure which shows the state which selected image registration by the imaging setting menu. It is an image figure which shows the example of a screen of an image registration screen. It is an image figure which shows the example of a screen during registration of a distance image. It is an image figure which shows the example of a screen during registration of a luminance image. It is an image figure which shows an imaging setting screen. It is an image figure which shows an imaging effective setting screen. It is an image figure which shows a three-dimensional measurement setting screen. It is an image figure which shows the kind of selectable pre-processing. It is an image figure which shows the choice which can be set in a measurement impossible reference | standard setting column. It is an image figure which shows the example which selected "None" in the measurement impossible standard setting column. It is an image figure which shows the example which selected "low" in the measurement impossible reference | standard setting column. It is an image figure which shows the example which selected "medium" in the measurement impossible reference | standard setting column. It is an image figure which shows the example which selected "high" in the measurement impossible reference | standard setting column. It is an image figure which shows the choice which can be set in the equal interval process setting column. It is an image figure which shows the example which set the equal interval process to ON, and set the "display image" selection column to "height image". It is an image figure which shows the example which set the equal space | interval process to ON and set the "display image" selection column to "grayscale image". It is an image figure which shows the example which turned off the equal space | interval process and displayed the distance image. It is an image figure which shows the example which turned off the equal interval processing and displayed the luminance image. It is an image figure which shows the choice which can be set in a space code setting column. It is an image figure which shows the state which selected the "height image" in the "display image" selection column, and displayed the distance image in the 2nd image display area. It is an image figure which shows the state which selected the "grayscale image" in the "display image" selection column, and displayed the luminance image on the 2nd image display area. It is an image figure which shows the example which selected OFF in the space code setting column. The “display image” selection field indicates a state in which “grayscale image” is displayed. It is an image figure which shows the choice which can be set in a projector selection setting column. It is an image figure which shows the example which selected "1" in the projector selection setting column. It is an image figure which shows the example which selected "2" in the projector selection setting column. It is an image figure which shows the example which selected "1 + 2" in the projector selection setting column. It is an image figure which shows the choice which can be set in the "display image" selection column. It is an image figure which shows the example which selected the "stripe light projection-projector 1" in the "display image" selection column, and displayed the pattern projection image of the 1st projector on the 2nd image display area. It is an image figure which shows the example which selected "stripe light projection-projector 2" in the "display image" selection column, and displayed the pattern projection image of the 2nd projector on the 2nd image display area. It is an image figure which shows the three-dimensional measurement setting screen which set the shutter speed setting column to "1/15". It is an image figure which shows the three-dimensional measurement setting screen which set the shutter speed setting column to "1/30". It is an image figure which shows the three-dimensional measurement setting screen which set the light / dark range setting column to "normal (0)". It is an image figure which shows the three-dimensional measurement setting screen which set the light / dark range setting column to "high (1)". Image showing how to add a “height measurement” processing unit from the state of FIG. It is an image figure which shows the initial screen to which the "height measurement" processing unit was added through FIG. It is an image figure which shows a height measurement setting screen. It is an image figure which shows an inspection object area | region setting screen. It is an image figure which shows the state which displayed the drop-down menu of the "measurement area" setting column of FIG. It is an image figure which shows the state which selected the "rotation rectangle" in the "measurement area" setting column. It is an image figure which shows a measurement area edit screen. It is an image figure which shows the state which selected "circle" in the "measurement area" setting column of the measurement area edit screen. It is an image figure which shows the initial screen which added the 2nd "height measurement" processing unit. It is an image figure which shows the state which set "rotation rectangle" as a measurement area | region. It is an image figure which shows the state which sets "rotation rectangle" as a measurement area | region. It is an image figure which shows the state in which "rotation rectangle" was set as a measurement area | region. It is an image figure which shows the state which is going to add a "numerical calculation" processing unit. It is an image figure which shows the initial screen to which the "numerical calculation" processing unit was added. It is an image figure which shows the state which displayed the numerical calculation edit screen. FIG. 59 is an image diagram showing a state in which an arithmetic expression is input to the numerical operation editing screen of FIG. 58. It is an image figure which shows the initial screen in which the "numerical calculation" processing unit was set. It is an image figure which shows the initial screen which added the "area" processing unit. It is an image figure which shows an area setting screen. It is an image figure which shows the area | region setting screen which sets the detail of a rotation rectangle. It is an image figure which shows the area | region setting screen in which the rotation rectangle was set. It is an image figure which shows a height extraction selection screen. It is an image figure which shows GUI of a one-point designation | designated screen. It is an image figure which shows GUI of a one-point designation | designated screen. It is an image figure which shows GUI of a one-point designation | designated screen. 69A is an image diagram showing a profile of an input image, and FIG. 69B is an image diagram showing a profile of a low gradation distance image obtained by gradation conversion of the input image of FIG. 69A. FIG. 69 is an image diagram showing a state where gain is increased from the state of FIG. 68; It is an image figure which shows the state which reduced the gain from the state of FIG. FIG. 72 is an image diagram showing a state in which a detailed setting is selected in FIG. 71. It is an image figure which shows GUI of the emphasis method detailed setting screen for one point designation | designated. It is an image figure which shows the state which displayed the drop-down list | wrist of the "extraction height" setting column from the state of FIG. 75A is an image diagram showing a profile of an input image, FIG. 75B is an image diagram showing the input image of FIG. 75A on the basis of a reference plane, and FIG. 75C is an image diagram showing a profile of a low gradation distance image obtained by gradation conversion of FIG. It is. 76A is a luminance image, FIG. 76B is a high gradation distance image, FIG. 76C is a low gradation distance image obtained by gradation conversion of FIG. 76B, FIG. 76D is a low gradation distance image with a higher gain than FIG. 76C, and FIG. FIG. 76D is an image diagram showing a low gradation distance image in which noise removal is improved as compared with FIG. 76D, and FIG. 76F is an image diagram showing a low gradation distance image in which “extraction height” is set to a higher side in FIG. 76E. 77A is an image diagram showing a profile of an input image, and FIG. 77B is an image diagram showing a profile of a low gradation distance image obtained by gradation conversion of the input image of FIG. 77A with the “extraction height” set to a high side. It is an image figure which shows the state which displayed the gradation conversion image. FIG. 79A is a perspective view showing an example of a work in which the method of setting a reference plane by specifying one point is effective, and FIG. 79B is an image diagram of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 79A. It is an image figure which shows GUI of a height extraction selection screen. It is an image figure which shows GUI of a three-point designation | designated screen. It is an image figure which shows the state which designates the 1st point with a height extraction means from the state of FIG. It is an image figure which shows the state which designates the 2nd point further from the state of FIG. It is an image figure which shows the state which designates the 3rd point further from the state of FIG. It is an image figure which shows GUI of the detailed setting screen for three-point designation | designated. 86A is a perspective view showing an example of a work in which the method of setting a reference plane by specifying three points is effective, FIG. 86B is an image diagram of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 86A, and FIG. 86B is an image diagram of a binarized image, and FIG. 86D is an image diagram of a binarized image obtained by specifying one point when the work of FIG. 86A has an inclination. 87A is a perspective view showing an example of another work in which the method of setting a reference plane by specifying three points is effective. FIG. 87B is an image diagram of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 87A. 87C is an image diagram of an image obtained by binarizing FIG. 87B, and FIG. 87D is an image diagram of a binarized image obtained by specifying one point when the work of FIG. 87A has an inclination. It is an image figure which shows GUI of a height dynamic extraction setting screen. It is an image figure which shows the drop-down box of the "calculation method" selection column. It is an image figure which shows an average height reference | standard setting screen. It is an image figure which shows a mask area | region setting screen. It is an image figure which shows a plane reference | standard detailed setting screen. FIG. 93A is a perspective view showing an example of a work in which the method of setting the reference plane based on the average height is effective, and FIG. 93B is an image diagram of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 93A. It is an image figure which shows a plane reference | standard detailed setting screen. FIG. 95 is an image diagram showing details of an invalid pixel designation field on the screen of FIG. 94. It is an image figure which shows a free-form surface reference | standard setting screen. FIG. 96 is a conceptual diagram showing a state where the numerical value in the “extraction size” designation field is increased in FIG. FIG. 98A is a perspective view showing an example of a work in which the method of setting a reference surface based on a free-form surface is effective, and FIG. 98B is an image diagram of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 98A. FIG. 91 is an image diagram showing a state where an extraction area setting dialog is displayed from the state of FIG. 90. FIG. 99 is an image diagram showing a state where “rectangle” is selected in the extraction area selection field of FIG. 99. It is an image figure which shows the state which displayed the extraction area edit dialog from the state of FIG. FIG. 99 is an image diagram showing a state where “circular” is selected in the mask area selection field of FIG. 99. FIG. 110 is an image diagram showing a state in which a mask area editing dialog is displayed from the state of FIG. 102. It is an image figure which shows the state which set height extraction by the "area" processing unit. It is an image figure which shows a filter process setting screen. It is an image figure which shows a binarization level setting screen. It is an image figure which shows the state which set the filter process. It is an image figure which shows the determination condition setting screen. It is an image figure which shows the state in which the determination conditions were set. It is an image figure which shows the initial screen which added the "blob" processing unit. It is an image figure which shows a mode that a filter process is set in a "blob" processing unit. It is an image figure which shows a mode that a detection condition is set in a "blob" processing unit. It is an image figure which shows a mode that a determination condition is set in a "blob" processing unit. It is an image figure which shows the state which is going to add a "color inspection" processing unit to an initial screen. It is an image figure which shows a mode that the area | region of a circle is set to a "color inspection" processing unit. It is an image figure which shows the state which performed the area | region setting of the circle to the "color inspection" processing unit. It is an image figure which shows the state which set the density | concentration average to the "color test | inspection" processing unit. It is an image figure which shows the initial screen which set the "color inspection" processing unit. 12 is a flowchart showing a flow of processing during operation of the head unit of the 3D image processing apparatus according to the third embodiment. 10 is a flowchart showing a flow of processing during operation of the head unit of the 3D image processing apparatus according to the fourth embodiment. 4 is a flowchart illustrating a gradation conversion method according to the first embodiment. 10 is a flowchart illustrating a processing flow during operation in the controller unit of the 3D image processing apparatus according to the third embodiment. It is an image figure which shows the initial screen of a three-dimensional image processing program. It is an image figure which shows the state which set the search object area | region on the brightness | luminance image. It is an image figure which shows the state which set the some test object area | region on the distance image. It is an image figure which shows the state which expanded the distance image of FIG. It is an image figure which shows the state which performs height measurement with a three-dimensional image processing program. FIG. 6 is a data flow diagram for generating a distance image by combining a phase shift method and a spatial encoding method. It is a data flow figure for producing | generating a distance image only by a phase shift method, without using a spatial encoding method. It is a data flow figure which shows the procedure which turns off XY equal pitching and obtains a Z image. It is a figure which shows the example which outputs point cloud data. It is a flowchart which shows the procedure of the static conversion at the time of operation. It is a flowchart which shows the procedure of the dynamic conversion at the time of operation. It is a perspective view which shows the workpiece | work of a test object. It is a schematic diagram which shows the method of preparing a some gradation conversion parameter set previously. FIG. 136 is a flowchart showing a procedure of the method of FIG. 135. FIG. It is a perspective view which shows the example of the workpiece | work with almost no fluctuation | variation of height. FIG. 138A is a perspective view showing an example of a workpiece whose height variation is to be detected, and FIG. 138B is an image diagram of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 138A. It is a perspective view which shows the example of the workpiece | work in which designation | designated of the reference plane by dynamic conversion (average height reference | standard) is effective. It is a perspective view which shows the example of the workpiece | work in which designation | designated of the reference plane by dynamic conversion (plane reference | standard) is effective. It is a perspective view which shows the example of the workpiece | work in which designation | designated of the reference surface by dynamic conversion (free-form surface reference | standard) is effective. It is a flowchart which shows the procedure which repeats adjustment of a gradation conversion parameter until a suitable image is obtained. It is a flowchart which shows the procedure which abbreviate | omitted determination of whether an image is appropriate in FIG. It is a flowchart which shows the specific procedure of gradation conversion parameter adjustment. FIG. 145A is an external view of the workpiece, FIG. 145B is an image of a distance image obtained from the workpiece of FIG. 145A, and FIG. 145C is an image showing a state in which an inspection target region is set for the height inspection processing on the distance image of FIG. 145D is an image showing a state in which an inspection target region is set for image inspection processing on the distance image of FIG. 145B, and FIG. 145E is a low gradation distance image obtained by performing gradation conversion on the distance image of FIG. 145D. FIG. It is a flowchart which shows the procedure at the time of a setting. FIG. 145B is a conceptual diagram showing a screen for setting an “area” processing unit for the workpiece of FIG. 145A. It is an image figure which shows the screen which sets the conditions of height extraction in FIG. It is a flowchart which shows the procedure at the time of a driving | operation. It is a flowchart which shows the procedure in the case of performing gradation conversion by the test | inspection process of FIG. It is a flowchart which shows the procedure in the case of not performing gradation conversion by the test | inspection process of FIG. It is a flowchart which shows the procedure which sets an inspection process condition. It is an image figure which shows an image setting screen. It is an image figure which shows the state which called the image variable selection screen which can select a brightness | luminance image or a distance image. It is an image figure which shows the state which called the image variable selection screen which can select only a distance image. It is a flowchart which shows the procedure of selecting an inspection process and setting an inspection process condition after selecting an image. It is a schematic diagram which shows the state which acquired the brightness | luminance image and the distance image. FIG. 157 is a schematic diagram showing an inspection processing tool that can be set when a luminance image is selected in FIG. 157; FIG. 157 is a schematic diagram showing inspection processing tools that can be set when a distance image is selected in FIG. 157; It is a schematic diagram which shows a mode that a distance image is imaged by a triangulation system.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiment described below is a three-dimensional image processing device, a three-dimensional image processing method, a three-dimensional image processing program, a computer-readable recording medium, and a recorded device for embodying the technical idea of the present invention. The present invention does not specify the following three-dimensional image processing apparatus, three-dimensional image processing method, three-dimensional image processing program, computer-readable recording medium, and recorded device. Further, the present specification by no means specifies the members shown in the claims to the members of the embodiments. In particular, the dimensions, materials, shapes, relative arrangements, and the like of the component parts described in the embodiments are not intended to limit the scope of the present invention unless otherwise specified, and are merely explanations. It is just an example. Note that the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for clarity of explanation. Furthermore, in the following description, the same name and symbol indicate the same or the same members, and detailed description thereof will be omitted as appropriate. Furthermore, each element constituting the present invention may be configured such that a plurality of elements are constituted by the same member and the plurality of elements are shared by one member, and conversely, the function of one member is constituted by a plurality of members. It can also be realized by sharing.

In this specification, the term “distance image (height image)” is used to mean an image including height information. For example, a three-dimensional image obtained by pasting an optical luminance image as texture information on a distance image. A composite image is also used in the sense of being included in a distance image. Further, in the present specification, the display form of the distance image is not limited to the one displayed in a two-dimensional form, but includes the one displayed in a three-dimensional form.
(Embodiment 1)

  FIG. 1 shows the configuration of the 3D image processing apparatus according to Embodiment 1 of the present invention. The three-dimensional image processing apparatus 100 includes a head unit 1 and a controller unit 2. The head unit 1 includes a light projecting unit 20 that illuminates an inspection target (work) W, an imaging unit 10 that captures an image of the workpiece W, and a head-side communication unit 36 that is connected to the controller unit 2.

  On the other hand, the controller unit 2 performs measurement processing such as edge detection and area calculation based on the captured image. The controller unit 2 can be detachably connected to a display unit 4 such as a liquid crystal panel, an input unit 3 such as a console for a user to perform various operations on the display unit 4, a PLC (Programmable Logic Controller), and the like.

  In the above three-dimensional image processing apparatus 100, the light projecting means 20 of the head unit 1 projects measurement light onto the work W, and the reflected light reflected by the measurement light incident on the work W is patterned by the image capturing means 10. Captured as a projected image. A distance image is generated based on the pattern projection image, and the distance image is converted into a low gradation distance image in which the height information of each pixel is replaced with luminance. The controller unit 2 executes measurement processing such as edge detection and area calculation based on the converted low gradation distance image.

In addition, the workpiece | work W which is a test subject is an article | item conveyed sequentially on a line, for example, and is moving or resting. Further, the moving workpiece includes a rotating workpiece as well as a parallel movement by a conveyor or the like.
(Light projection means 20)

  The light projecting means 20 is used as illumination for illuminating the workpiece W in order to generate a distance image. Therefore, the light projecting means 20 is, for example, a light projector that projects a line-shaped laser beam onto the work, or a sinusoidal striped pattern on the work, according to the light cutting method or pattern projection method for obtaining the distance image. Can be used as a pattern projector or the like. In addition to the light projecting means, a general illumination device for performing bright field illumination or dark field illumination may be separately provided. Alternatively, the light projecting means 20 can have a function as a general lighting device.

  The controller unit 2 executes image processing using the distance image data acquired from the head unit 1, and outputs a determination signal as a signal indicating a determination result of the quality of the workpiece to an externally connected control device such as the PLC 70. To do.

  The imaging unit 10 performs imaging of a workpiece based on a control signal input from the PLC 70, for example, an imaging trigger signal that defines a timing for capturing image data from the imaging unit 10.

  The display means 4 is a display device for displaying image data obtained by imaging a workpiece and a result of measurement processing using the image data. In general, the user can confirm the operation state of the controller unit 2 by visually recognizing the display unit 4. The input means 3 is an input device for moving the focus position on the display means 4 and selecting menu items. In addition, when using a touch panel for the display means 4, a display means and an input means can be combined.

The controller unit 2 can also be connected to a personal computer PC for generating a control program for the controller unit 2. The personal computer PC can be installed with a three-dimensional image processing program for performing settings related to three-dimensional image processing, and various settings for processing performed by the controller unit 2 can be performed. Alternatively, a processing order program that defines the processing order of image processing can be generated by software operating on the personal computer PC. In the controller unit 2, each image processing is sequentially executed in the processing order. The personal computer PC and the controller unit 2 are connected via a communication network, and the processing order program generated on the personal computer PC includes, for example, layout information that defines the display mode of the display unit 4 and the controller unit. 2 is transferred. Conversely, a processing order program, layout information, and the like can be taken from the controller unit 2 and edited on the personal computer PC. The processing order program may be generated not only by the personal computer PC but also by the controller unit 2.
(Modification)

In the above example, dedicated hardware is constructed as the controller unit 2, but the present invention is not limited to this configuration. For example, as in the three-dimensional image processing apparatus 100 ′ according to the modification shown in FIG. 2, a special inspection program or a three-dimensional image processing program installed in a general-purpose personal computer or workstation functions as the controller unit 2 ′. It can also be connected to the head unit 1 for use. This three-dimensional image processing apparatus performs necessary settings such as image processing by a three-dimensional image processing program, and then performs image processing on a low gradation distance image according to a pattern projection image picked up by the head unit 1 and is necessary. Perform a proper inspection.
(Head side communication means 36)

Accordingly, an interface that can be connected to either the dedicated controller unit 2 or a personal computer that functions as the controller unit 2 can be provided on the head unit 1 side as the head-side communication means 36. For example, the head unit 1 is provided with a controller connection interface 36A for connection with the controller unit 2 as shown in FIG. 1 as the head side communication means 36, or for connection with a personal computer as shown in FIG. The PC connection interface 36B is provided. Further, by making such an interface interchangeable as a unit, other configurations of the head unit can be shared to some extent, and the common head unit can be connected to either the controller unit or the personal computer. Alternatively, a dedicated controller unit 2 and one head-side communication unit including an interface that can be connected to any of the personal computers may be provided. Such an interface can use an existing communication standard such as Ethernet (trade name), USB, RS-232C, or the like. In addition, a dedicated communication method may be used instead of a standardized or general-purpose communication method.
(PC connection mode)

  Further, the three-dimensional image processing program can be provided with a PC connection mode for setting when a personal computer is used as the controller unit 2 ′ connected to the head unit 1. In other words, settings related to three-dimensional image processing can be appropriately performed in either case by changing settable items and setting contents depending on whether the controller unit is dedicated hardware or a personal computer. It becomes possible. In addition, it is possible to check the operation and function of the connected head unit by installing a viewer program having a purpose of confirming the operation of the head unit 1 and a simple measurement function in a personal computer functioning as the controller unit 2 ′. You may do it.

  The “distance image” obtained by using the imaging unit 10 and the light projecting unit 20 shown in FIG. 1 is a gray value of each pixel according to the distance from the imaging unit 10 that images the workpiece W to the workpiece W. An image that changes. In other words, it can be said that the gray value is determined based on the distance from the imaging means 10 to the workpiece W, can be said to be a multi-valued image having a gray value corresponding to the distance to the workpiece W, or It can also be said to be a multi-valued image having a gray value corresponding to the height. Furthermore, it can be said that it is a multi-value image obtained by converting the distance from the imaging means 10 into a gray value for each pixel of the luminance image.

  There are roughly two methods for generating distance images. One is a passive method (passive measurement method) that generates a distance image using an image captured under illumination conditions to obtain a normal image. The other is an active method (active measurement method) in which a distance image is generated by actively irradiating light for measurement in the height direction. A typical passive method is a stereo measurement method. This is because a distance image can be generated simply by preparing two imaging means 10 and arranging these two cameras in a predetermined positional relationship. Therefore, a general image processing system for generating a luminance image is used. Distance images can be generated, and system construction costs can be reduced. However, in the stereo measurement method, it is necessary to determine which point in the image obtained by one camera corresponds to the point in the image obtained by the other camera. There is a problem that it takes time. Further, since the measurement position is only the corresponding point and not all the pixels, this point is also not suitable for speeding up the appearance inspection.

  On the other hand, typical methods of the active method are a light cutting method and a pattern projection method. The light cutting method is the above-described stereo measurement method, in which one camera is replaced with an optical projector, a line-shaped laser beam is projected onto the workpiece, and the line light image is distorted according to the shape of the object surface. Restore the 3D shape of the workpiece from the condition. Since the light cutting method does not require determination of corresponding points, stable measurement is possible. However, since only one line can be measured in one measurement, the object or camera must be scanned to obtain the measurement values of all pixels. On the other hand, the pattern projection method captures multiple images by shifting the shape and phase of the predetermined pattern projected on the workpiece, and restores the three-dimensional shape of the workpiece by analyzing the captured images. To do. There are several types of pattern projection methods, taking multiple images (at least 3 images) by shifting the phase of the sinusoidal fringe pattern, obtaining the phase of the sine wave for each pixel from the multiple images, Using the obtained phase to restore the 3D shape using the phase shift method to obtain 3D coordinates on the workpiece surface and the kind of spatial frequency undulation that occurs when two regular patterns are combined The moire topography method, the pattern projected onto the work itself is different for each shooting, for example, the stripe width becomes half-screen, 1/4, 1/8, etc., with a black and white duty ratio of 50%. A pattern is projected in sequence, a pattern projection image is taken with each pattern, and a spatial coding method to obtain the absolute phase of the workpiece height, and multiple thin line pattern illumination (multi slit) is projected onto the workpiece. Moving the pattern with a narrow pitch than the slit cycle, multi-slit method of performing a plurality of times imaging is typical.

  In the three-dimensional image processing apparatus 100 according to the present embodiment, a distance image is generated by combining the phase shift method and the spatial coding method described above. Thereby, a distance image can be generated without relatively moving the work or the head. The present invention is not limited to generating a distance image by a phase shift method and a spatial encoding method, and a distance image may be generated by another method. Further, any method other than the above-described method, such as an optical radar method (time of flight), a focusing method, a confocal method, a white light interferometry method, and the like that can be considered may be adopted. Absent.

The arrangement layout of the imaging unit 10 and the light projecting unit 20 shown in FIG. 1 projects light from an oblique direction with respect to the workpiece W, and receives the reflected light from the workpiece W in a substantially vertical direction. Are arranged so as to hold the image pickup means 10 in a vertical posture. In this way, by projecting the light projecting direction and the image capturing direction without making them coincide with each other, it is possible to capture a pattern projection image that captures the shadow caused by the unevenness of the surface shape of the workpiece W.
(Embodiment 2)

However, the present invention is not limited to this arrangement example. For example, as in the 3D image processing apparatus 200 according to Embodiment 2 shown in FIG. The arrangement may be such that the light projecting means 20 side is held in a vertical posture. Similarly, the head unit 1B having such an arrangement can similarly capture the pattern projection image capturing the shadow of the workpiece W by inclining the light projection direction and the imaging direction.
(Embodiment 3)

Furthermore, a plurality of one or both of the light projecting means and the imaging means can be arranged. For example, like the three-dimensional image processing apparatus 300 shown in FIG. 4A as the third embodiment, the imaging unit 10 is held in a vertical posture with respect to the workpiece W, while the two light projecting units 20 are arranged around the imaging unit 10. It can also be configured as a head portion 1C that is disposed on both sides and projects light from the left and right. In this way, by capturing each pattern projection image with a different direction of light projection, a pattern projection image cannot be partially captured, for example, a shadow pattern is hidden by the work W itself when light is projected from one direction. Therefore, the situation where height measurement is inaccurate or impossible can be reduced. In particular, if the light projecting means 20 is arranged so as to project light from a direction opposite to the work (for example, left and right or front and rear), the possibility that the image cannot be captured due to being blocked by the work itself can be greatly reduced.
(Embodiment 4)

  In the above example, a configuration in which one imaging unit is provided and two light projecting units are described, but conversely, a configuration in which two imaging units are provided and one light projecting unit is also possible. Such an example is shown in FIG. 4B as a three-dimensional image processing apparatus 400 according to Embodiment 4. In the head unit 1D shown in this example, the light projecting unit 20 is held in a vertical posture with respect to the workpiece W, and the imaging units 10 are arranged in an inclined posture with respect to the workpiece W on the left and right sides in the drawing. Even with this configuration, since the workpiece W can be imaged from different inclination angles, it is possible to suppress a situation in which it is difficult to partially capture the pattern projection image as in the third embodiment. Also, with this method, since two pattern projection images can be taken simultaneously with a single light projection, there is an advantage that the processing time can be shortened.

  On the other hand, even if the same work is imaged from two different angles with the two imaging means, the part being picked up and the field of view are different, so the work of matching the positions of each pixel is necessary, and errors may also occur. is there. On the other hand, according to the above-described third embodiment, since the image pickup unit is used in common, an image with the same field of view can be picked up even if measurement light is projected from any of the light projection units. Therefore, an advantage of simplifying the processing can be obtained by eliminating the need for a simple integration operation and avoiding the occurrence of errors associated with the integration operation.

In the above example, the example in which the imaging unit 10 and the light projecting unit 20 are integrally configured in each head unit has been described. However, the present invention is not limited to this configuration. For example, the imaging unit 10 and the light projecting unit 20 can be a head unit configured by separate members. It is also possible to provide three or more imaging means and light projecting means.
(Block Diagram)

Next, FIG. 5 shows a block diagram showing a configuration of a 3D image processing apparatus 300 according to Embodiment 3 of the present invention. As illustrated in FIG. 5, the three-dimensional image processing apparatus 300 includes a head unit 1 and a controller unit 2.
(Head 1)

The head unit 1 includes a light projecting unit 20, an imaging unit 10, a head side control unit 30, a head side calculation unit 31, a storage unit 38, a head side communication unit 36, and the like. The light projecting unit 20 includes a measurement light source 21, a pattern generation unit 22, and a plurality of lenses 23, 24, and 25. Although not shown, the imaging means 10 includes a camera and a plurality of lenses.
(Light projection means 20)

  The light projecting unit 20 is a member for projecting incident light as structured illumination with a predetermined projection pattern from an oblique direction with respect to the optical axis of the imaging unit. The light projecting means 20 can be a projector and includes a lens that is an optical member, a pattern generation unit 22, and the like. The light projecting means 20 is disposed obliquely above the position of the stationary or moving workpiece. The head unit 1 can also include a plurality of light projecting means 20. In the example of FIG. 5, the head unit 1 includes two light projecting means 20. Here, the first projector 20A (right side in FIG. 5) that can irradiate the workpiece with measurement illumination light from the first direction, and the workpiece from the second direction different from the first direction. Second projectors 20B (left side in FIG. 5) capable of irradiating illumination light are respectively arranged. The first projector 20A and the second projector 20B are arranged symmetrically across the optical axis of the imaging means 10. Measurement light is alternately projected onto the workpiece from the first projector 20A and the second projector 20B, and the pattern of each reflected light is imaged by the imaging means 10.

  As the measurement light source 21 of each of the first projector 20A and the second projector 20B, for example, a halogen lamp that emits white light, a white LED (light emitting diode) that emits white light, or the like can be used. The measurement light emitted from the measurement light source 21 is appropriately condensed by the lens and then enters the pattern generation unit 22.

  Furthermore, in addition to the light projecting means for emitting the measurement light for obtaining the pattern projection image for generating the distance image, an observation illumination light source for capturing a normal optical image (luminance image) can be provided. . As the illumination light source for observation, a semiconductor laser (LD), a halogen light, HID, or the like can be used in addition to the LED. In particular, when an element capable of imaging in color is used as the imaging element, a white light source can be used as the observation illumination light source.

The measurement light emitted from the measurement light source 21 is appropriately condensed by the lens 113 and then enters the pattern generation unit 112. The pattern generation part 22 can implement | achieve illumination of arbitrary patterns. For example, an appropriate pattern that is easy to see or measure can be expressed by inverting the pattern according to the color of the work or the background, such as black in white and white in black. For example, a DMD (digital micromirror device) can be used as the pattern generation unit 22. The DMD can express an arbitrary pattern by turning on and off a minute mirror for each pixel. Thereby, the pattern which reversed black and white can be irradiated easily. By using DMD for the pattern generation unit 22, an arbitrary pattern can be easily generated, and mechanical pattern mask preparation and replacement work can be eliminated, so that there is an advantage that the apparatus can be downsized and quickly measured. . Further, the pattern generation unit 112 using DMD can be used in the same manner as normal illumination by irradiating all illumination patterns in which all pixels are turned on, and can also be used for capturing a luminance image. The pattern generation unit 22 may be an LCD (Liquid Crystal Display), LCOS (Liquid Crystal on Silicon), or a mask. The measurement light incident on the pattern generation unit 22 is converted into a preset pattern and a preset intensity (brightness) and emitted. The measurement light emitted from the pattern generation unit 22 is converted into light having a diameter larger than the visual field that can be observed and measured by the imaging unit 10 by a plurality of lenses, and then irradiated onto the workpiece.
(Imaging means 10)

  The imaging means 10 includes a camera for acquiring reflected light that is projected by the light projecting means 20 and reflected by the work WK and that captures a plurality of pattern projection images. A CCD, CMOS, or the like can be used for such a camera. In this example, a monochrome CCD camera capable of obtaining a high resolution is used. Needless to say, it is also possible to use a camera capable of imaging in color. In addition to the pattern projection image, the imaging unit can also capture a normal luminance image.

  The head-side control unit 30 is a member for controlling the first projector 20A and the second projector 20B that are the imaging unit 10 and the light projecting unit 20. For example, the head unit side control unit 30 creates a projection pattern for the projection unit 20 to project the measurement light onto the workpiece and obtain a pattern projection image. As a result, the imaging means 10 causes the light projection means 20 to project a phase shift projection pattern to image a phase shift image, and the light projection means 20 projects a spatial coding projection pattern to produce a spatial code image. Let's take an image. As described above, the head-side control unit 30 functions as a light projection control unit for controlling the light projection unit so that the imaging unit 10 captures the phase shift image and the spatial code image.

  The head side calculation unit 31 includes a filter processing unit 34 and a distance image generation unit 32. The distance image generation unit 32 generates a distance image based on the plurality of pattern projection images captured by the imaging unit 10.

  The head-side storage unit 38 is a member for holding various settings, images, and the like, and a storage element such as a semiconductor memory or a hard disk can be used. For example, a luminance image storage unit 38b for holding the pattern projection image captured by the imaging unit 10 and a distance image storage unit 38a for holding the distance image generated by the distance image generation unit 32 are included.

The head side communication means 36 is a member for communicating with the controller unit 2. Here, the controller side communication means 42 of the controller unit 2 is connected to perform data communication. For example, the distance image generated by the distance image generation unit 32 is sent to the controller unit 2.
(Distance image generating means 32)

  The distance image generation unit 32 is a unit that generates a distance image in which the gray value of each pixel changes according to the distance from the imaging unit 10 that captures the workpiece WK to the workpiece WK. For example, when a distance image is generated by the phase shift method, the head-side control unit 30 controls the light projecting unit 20 so as to project a sinusoidal fringe pattern on the workpiece with a phase shift, and the sine is accordingly generated. The head-side control unit 30 controls the imaging unit 10 so as to capture a plurality of images in which the phase of the wavy pattern is shifted. The head-side control unit 30 obtains a sine wave phase for each pixel from a plurality of images, and generates a distance image using the obtained phase.

  When a distance image is generated by using the space encoding method, the space irradiated with light is divided into a number of small spaces having a substantially fan-shaped cross section, and a series of space code numbers are assigned to the small spaces. For this reason, even if the height of the workpiece is high, in other words, even if the height difference is large, the height can be calculated from the space code number as long as it is within the space irradiated with light. Therefore, it is possible to measure the shape of a workpiece having a high height throughout.

  By generating a distance image on the head unit side and sending it to the controller side in this way, the amount of data to be sent from the head unit to the controller unit side can be reduced, and processing delays that can occur due to a large amount of data transfer can be reduced. Can be avoided.

In the present embodiment, the distance image generation process is performed on the head unit 1 side. However, for example, the distance image generation process can be performed on the controller unit 2 side. In addition, the gradation conversion from the distance image to the low gradation distance image can be performed not only by the controller unit but also by the head unit side. In this case, the head side calculation unit 31 realizes the function of the gradation conversion means.
(Controller part 2)

The controller unit 2 includes a controller-side communication unit 42, a controller-side control unit, a controller-side arithmetic unit, a controller-side storage unit, an inspection execution unit 50, and a controller-side setting unit 41. The controller side communication means 42 is connected to the head side communication means 36 of the head unit 1 to perform data communication. The controller-side control unit is a member for controlling each member. The controller-side arithmetic unit realizes the function of the image processing unit 60. The image processing unit 60 implements functions such as an image search unit 64 and a gradation conversion unit 46.
(Tone conversion means)

  The gradation converting means 46 performs gradation conversion from a high gradation distance image to a low gradation low gradation distance image based on the distance image (the detailed procedure will be described later). Accordingly, the distance image having the height information generated by the head unit is expressed as a two-dimensional grayscale image that can be handled by existing equipment, thereby contributing to measurement processing and inspection processing. In addition, there is an advantage that the load can be distributed by sharing the distance image generation process and the gradation conversion process between the head unit and the controller unit. In addition to the generation of the distance image on the head unit side, a low gradation distance image may be generated. Such processing can be performed by the head side calculation unit. As a result, the load on the controller unit side can be further reduced and efficient operation becomes possible.

  Further, the gradation converting means does not perform gradation conversion on all the distance images, but preferably selects only necessary portions and performs gradation conversion. Specifically, gradation conversion is performed only on the portion corresponding to the inspection target area set in advance by the inspection target area setting means (details will be described later). By doing so, it is possible to reduce the load required for gradation conversion by limiting the process of converting a multi-tone distance image to a low-tone distance image only to the inspection target region. This also contributes to shortening the processing time. That is, by shortening the processing time, it can be suitably used in applications where the processing time is limited, such as inspection for FA applications, and real-time processing is realized.

  The controller-side storage means is a member for holding various settings and images, and a semiconductor storage element, a hard disk, or the like can be used.

  The controller-side setting means 41 is a member for making various settings for the controller unit, accepts an operation from the user via the input unit 3 such as a console connected to the controller unit, and sends necessary conditions to the controller side. Instruct. For example, the functions of the gradation conversion condition setting unit 43, the reference plane setting unit 44, the spatial coding switching unit 45, the interval equalization processing setting unit 47, the light projection switching unit 48, the shutter speed setting unit 49, and the like are realized.

  The reference plane setting unit 44 serves as a gradation conversion parameter that constitutes a gradation conversion condition when performing gradation conversion for converting the distance image received by the controller-side communication unit 42 into a two-dimensional low gradation distance image. A reference plane for performing the gradation conversion is set based on the distance image. The gradation converting unit 46 uses the reference plane set by the reference plane setting unit 44 as a reference, and converts the distance information into the gray value of the image with the number of gradations lower than the number of gradations of the distance image. Tone conversion is performed on the replaced low tone distance image.

The inspection execution unit 50 executes a predetermined inspection process on the low gradation distance image subjected to gradation conversion by the gradation conversion unit 46.
(Hardware configuration)

  Next, a hardware configuration example of the controller unit 2 is shown in a block diagram of FIG. The controller unit 2 shown in this figure has a main control unit 51 that performs numerical calculation and information processing based on various programs and controls each part of the hardware. The main control unit 51 stores, for example, a CPU as a central processing unit, a work memory such as a RAM that functions as a work area when the main control unit 51 executes various programs, a startup program, an initialization program, and the like. And a program memory such as a ROM, a flash ROM, or an EEPROM.

  The controller unit 2 is connected to the head unit 1 including the imaging unit 10 and the light projecting unit 20, and controls the light projecting unit 20 to project a sinusoidal fringe pattern on the workpiece with a phase shift. For the controller side connection unit 52 for taking in image data obtained by the imaging by the imaging unit 10, the operation input unit 53 to which the operation signal from the input unit 3 is input, and the display unit 4 such as a liquid crystal panel A display control unit 54 including a display DSP for displaying an image, a communication unit 55 communicably connected to an external PLC 70, a personal computer PC, etc., a RAM 56 for holding temporary data, and a setting The controller-side storage means 57 for storing the contents and the data set by the 3D image processing program installed in the personal computer PC are held. Image processing unit 60 composed of auxiliary storage means 58 for processing, an arithmetic DSP for executing measurement processing such as edge detection and area calculation, and a predetermined inspection based on the processing result in the image processing unit 60, etc. An output unit 59 and the like for outputting the results are provided. These pieces of hardware are communicably connected via an electrical communication path (wiring) such as a bus.

  The program memory in the main control unit 51 has a control for controlling each of the controller side connection unit 52, the operation input unit 53, the display control unit 54, the communication unit 55, and the image processing unit 60 by a CPU command or the like. The program is stored. The processing order program described above, that is, the processing order program generated in the personal computer PC and transferred from the personal computer PC is stored in the program memory.

  The communication unit 55 functions as an interface (I / F) that receives an imaging trigger signal from the PLC 70 when a trigger is input from a sensor (photoelectric sensor or the like) connected to the external PLC 70. It also functions as an interface (I / F) that receives a 3D image processing program transferred from the personal computer PC, layout information that defines the display mode of the display means 4, and the like.

  When receiving the imaging trigger signal from the PLC 70 via the communication unit 55, the CPU of the main control unit 51 sends an imaging command (command) to the controller side connection unit 52. Also, a command for instructing image processing to be executed is transmitted to the image processing unit 60 based on the processing order program. In addition, as a device that generates an imaging trigger signal, a trigger input sensor such as a photoelectric sensor may be directly connected to the communication unit 55 instead of the PLC 70.

  The operation input unit 53 functions as an interface (I / F) that receives an operation signal from the input unit 3 based on a user operation. The display unit 4 displays the user's operation contents using the input unit 3. For example, when a console is used as the input means 3, various components such as a cross key, a determination button, or a cancel button for moving the cursor displayed on the display means 4 up, down, left and right can be arranged. By operating these components, the user creates a flowchart for defining the processing order of the image processing on the display means 4, edits the parameter value of each image processing, and sets the reference area. The reference registration image can be edited.

  The controller side connection unit 52 captures image data. Specifically, for example, when an imaging command of the imaging unit 10 is received from the CPU, an image data capturing signal is transmitted to the imaging unit 10. And after image pick-up by image pick-up means 10, image data obtained by image pick-up is taken in. The captured image data is temporarily buffered (cached) and substituted for an image variable prepared in advance. The “image variable” is a variable that is a reference destination for measurement processing and image display by being assigned as an input image of a corresponding image processing unit, unlike a normal variable that handles numerical values.

  The image processing unit 60 performs a measurement process on the image data. Specifically, first, the controller side connection unit 52 reads image data from the frame buffer while referring to the image variables described above, and performs internal transfer to the memory in the image processing unit 60. Then, the image processing unit 60 reads the image data stored in the memory and executes measurement processing. The image processing unit 60 includes a gradation conversion unit 46, an abnormal point highlight unit 62, an image search unit 64, and the like.

  The display control unit 54 transmits a control signal for displaying a predetermined image (video) to the display unit 4 based on a display command (display command) sent from the CPU. For example, a control signal is transmitted to the display unit 4 in order to display image data before or after the measurement process. The display control unit 54 also transmits a control signal for causing the display unit 4 to display the user's operation contents using the input unit 3.

The head unit 1 and the controller unit 2 configured by the hardware as described above are configured such that the respective means and functions in FIG. 5 can be realized in software by various programs. In this example, a mode is adopted in which a three-dimensional image processing program is installed in the computer of FIG. 1 and settings necessary for three-dimensional image processing are performed.
(Gradation conversion)

  The above three-dimensional image processing apparatus acquires a distance image of a workpiece, performs image processing on the distance image, and inspects the result. The three-dimensional image processing apparatus according to the present embodiment uses information such as an area and an edge using existing hardware in addition to the height inspection process in which the height information that is the pixel value of the distance image is used as it is. Two types of inspections can be performed, that is, an image inspection process that performs an operation using the. Here, in order to maintain the accuracy of the height inspection process, it is necessary to generate a multi-tone distance image. On the other hand, with existing hardware, image inspection processing cannot be performed on such multi-tone distance images. Therefore, in order to perform image inspection processing using existing hardware, gradation conversion is performed on a multi-gradation distance image to generate a low gradation distance image.

  However, if the height information of the multi-tone distance image is converted into the low-tone distance image as it is, there is a problem that the accuracy of the height information is impaired. There are many common images used for FA applications, etc., which are monochrome and express the gray value of each pixel with 8 gradations. On the other hand, as the distance image, a high gradation image such as a 16 gradation image is used. For this reason, when gradation conversion is performed on a multi-tone distance image to a low-tone distance image, a considerable amount of height information is lost, affecting the accuracy of inspection. However, in order to increase the number of gradations of the image handled in the existing image processing in order to increase the accuracy, the introduction cost increases, the processing load increases, and the hurdle for use increases.

Therefore, in such gradation conversion, it is necessary to set conditions for gradation conversion so that necessary height information is maintained. Hereinafter, the method and procedure will be described in detail.
(Height inspection or image inspection)

  First, the processing operation for performing the height inspection process using the three-dimensional image processing apparatus will be described based on the flowchart of FIG. This three-dimensional image processing apparatus includes a height inspection processing tool for performing a height inspection on a distance image, and various image inspection processes for performing an image inspection on an existing luminance image as tools for performing calculation processing. With tools. Here, the height inspection process will be described.

  First, a distance image is generated (step S71). Specifically, the distance image generating unit 32 generates a distance image using the imaging unit 10 and the light projecting unit 20. Next, a desired calculation process is selected (step S72). Here, a tool necessary for the calculation process is selected.

  When the image inspection processing tool is selected, the process proceeds to step S73, where the gradation conversion process is performed on the high gradation distance image obtained in step S71 to convert it into a low gradation distance image. Accordingly, an inspection processing tool provided in an existing image processing apparatus can handle a low gradation distance image. The gradation conversion process is not performed on the entire range of the high gradation distance image, but is preferably performed only in the inspection target area set for the image inspection process.

On the other hand, when the height inspection tool is selected, the height information of the multi-tone distance image is used as it is, and thus the process proceeds to step S74 without performing the tone conversion.

Further, the inspection execution means 50 performs various calculation processes (step S74), and then determines whether or not the workpiece is a non-defective product based on the calculation result (step S75). When the inspection execution unit 50 determines that the workpiece is a non-defective product (step S75: YES), the determination signal output unit 160 outputs an OK signal to the PLC 70 as a determination signal (step S76). When it is determined that the workpiece is not a non-defective product, that is, a defective product (step S75: NO), an NG signal is output to the PLC 70 as a determination signal (step S77).
(Setting mode)

Next, an example of a procedure in a setting mode for performing various settings for the three-dimensional image processing apparatus prior to execution of such height inspection and image inspection processing will be described with reference to the flowchart of FIG. First, in step S81, a setting image (setting image) is selected. Here, it is also possible to input an image to be inspected in advance and call an image stored as a registered image, or obtain a new input image and set it. Here, an input image obtained by capturing a workpiece is registered as a registered image, as an alternative showing an input image that is sequentially input during operation. Alternatively, a registered image registered in advance may be called up. Next, in step S82, a gradation conversion method is selected. Here, the user is prompted to select either static conversion or dynamic conversion. Next, in step S83, the tone conversion parameter is adjusted. Here, when static conversion is selected in step S82, the tone conversion parameters are adjusted for the image acquired in step S81. A method for adjusting the gradation conversion parameter will be described later. In addition, the procedure demonstrated above is an example and can also be made into a different order. For example, the image may be acquired after selecting the gradation conversion method.
(Details of setting procedure)

Next, the details of the setting procedure will be described. In the three-dimensional image processing apparatus, necessary settings are made in advance in the setting mode prior to the operation mode. Various setting means for performing such settings can be provided on the controller unit 2 side, for example. For example, in the example of FIG. 1, a console that is one form of the input unit 3 connected to the controller unit 2 can be used. Instead of this, or in addition to this, the function of such setting means can be realized in a three-dimensional image processing program installed in a personal computer connected to the controller unit 2 as described above. Hereinafter, the details of the procedure for performing each setting using the 3D image processing program installed in the personal computer shown in FIG. 1 will be described below. The user interface (GUI) screen of the 3D image processing program shown in FIGS. Based on In these GUI examples, a distance image is displayed as a “height image”, and a luminance image is displayed as a “grayscale image”.
(Distance image and brightness image registration process)

First, a distance image and a luminance image are registered. Here, the “imaging” processing unit 263 is set from the initial screen 260 of the 3D image processing program shown in FIG. Specifically, the button of the button 263 of the “imaging” processing unit is pressed. This switches to the imaging setting menu 269 of FIG.
(3D image processing program)

  In the GUI screen example of FIG. 10, a first image display area 111 for displaying an image is provided on the right side of the screen, and a setting item button area 112 in which a plurality of buttons representing a plurality of setting items are arranged is provided on the left side. The setting item button area 112 includes an “image registration” button 113, an “imaging setting” button 284, a “camera setting” button, a “trigger setting” button, a “flash setting” button, an “illumination volume” button, and an “illumination expansion unit”. ”Button,“ Save ”button, and the like. The user can set a necessary setting item by selecting a desired setting item button from the setting item button area 112.

  When the “image registration” button 113 provided in the setting item button area 112 is pressed in the imaging setting menu 269 in FIG. 10, the screen is switched to the image registration screen 270 in FIG. 11. Various settings such as a registration target, camera selection, and registration destination can be performed from this screen. Here, the image is adjusted to a desired image using various buttons or the like provided in the operation area, and then registered, that is, the image data is stored. Here, a distance image is displayed in the second image display area 121, and an image variable assigned to this image is displayed in the operation area 122. In the example of FIG. 11, “Camera 1” is selected in the “Camera selection” column 271 for selecting an imaging means, and the image variable “& Cam1Img” is “Camera selection” as a distance image captured by this “Camera 1”. 'Is displayed below the column 271.

When the “Register” button 272 provided in the lower part of the operation area is pressed after the setting is completed, the registration of the distance image currently displayed in the second image display area 121 is started as shown in FIG. Is displayed. In addition, as shown in FIG. 13, the registration of the luminance image is also performed. In this example, the distance image is first stored in the distance image storage unit 38a, and then the luminance image is stored in the luminance image storage unit 38b. Also, the image variable “& Cam1Img” of the distance image and the image variable “& Cam1GrayImg” of the luminance image are recorded respectively. These image variables can be used as indices when calling up registered images because unique variables are assigned to the respective images. However, this example is merely an example, and the registration order of each image may be reversed or registered simultaneously. Thus, by saving the distance image and the luminance image simultaneously as registered images, the user can save labor for registering each image. However, it is also possible to adopt a configuration in which the distance image and the luminance image are individually registered as registered images.
(Phase shift method)

Here, the phase shift method will be described as one method for measuring the displacement and three-dimensional shape of the workpiece in a non-contact manner. The phase shift method is also called a grating pattern projection method, a fringe scanning method, or the like. In this method, a light beam having a lattice pattern in which the illuminance distribution is changed in a sine wave shape is projected onto a workpiece. Moreover, projection is performed with three or more lattice patterns having different sine wave phases, and each brightness value at the height measurement point is imaged for each pattern from an angle different from the projection direction of the light beam. Calculate the phase value of. Depending on the height of the measurement point, the phase of the grating pattern is projected onto the measurement point, and a light beam having a phase different from the phase observed by the light beam reflected at the reference position is observed. Therefore, the phase of the light ray at the measurement point is calculated, and the height of the measurement point (and hence the object) is measured by substituting it into the geometric relational expression of the optical device using the principle of triangulation to obtain the three-dimensional shape. Is the method. According to the phase shift method, the workpiece height can be measured with a high resolution by reducing the grating pattern period, but the measurable height range is a low height that is within 2π in terms of the phase shift amount. Can only be measured (those with a small height difference).
(Spatial coding method)

  Therefore, the space coding method is also used. According to the space encoding method, a space irradiated with light is divided into a number of small spaces having a substantially fan-shaped cross section, and a series of space code numbers are assigned to the small spaces. For this reason, even if the height of the workpiece is high, that is, when the height difference is large, the height can be calculated from the space code number if it is within the space irradiated with light. Therefore, it is possible to measure the shape of a workpiece having a high height throughout. Thus, according to the spatial coding method, the allowable height range (dynamic range) is wide, and the shape of a workpiece having a high height can be measured as a whole.

  Next, a setting example in which a distance image is captured by the imaging unit will be described with reference to FIGS. Prior to the image registration described above, when the “imaging setting” button 284 is pressed from the imaging setting menu 269 in FIG. 10, the imaging setting screen 280 in FIG. 14 is displayed. On the imaging setting screen 280. Basic settings related to imaging can be performed. For example, when an “imaging effective setting” button is pressed, an imaging effective setting screen shown in FIG. 15 is displayed, and an imaging means, that is, a camera connected to the 3D image processing apparatus can be selected. For example, instead of or in addition to a monochrome CCD camera or color CCD camera that captures a normal luminance image as an imaging means, a camera capable of acquiring height information is connected to perform three-dimensional image processing of distance images. It can be taken into the device. In addition, when a plurality of imaging means are connected to the 3D image processing apparatus, one or more of the imaging means to be used can be selected.

Further, when the “detailed setting” button 282 provided in the operation area is pressed from the imaging setting screen 280 of FIG. 14, a three-dimensional measurement setting screen 290 shown in FIG. 16 is displayed. In FIG. 16, for the convenience of explanation, different works are displayed. The three-dimensional measurement setting screen 290 of FIG. 16 includes a “display with continuous update” field 292 corresponding to the real-time update means, a shutter speed setting field 294 corresponding to the shutter speed setting means 49, a light / dark range setting field 296, a preprocessing setting field 310, a measurement. An impossible standard setting field 312, an equal interval processing setting field 314, a space code setting field 316, a projector selection setting field 318, a “display image” selection field 322, and the like are provided.
(Real-time update means)

  Here, when the setting is changed in the operation area, a real-time updating unit is provided for updating the image being displayed on the second image display area 121 to the changed setting. The real-time updating means can be switched ON / OFF. In the screen example of FIG. 16, the real-time update function can be operated by turning on the “display with continuous update” column 292 provided in the upper part of the operation area as one form of the real-time update means.

In the example of FIG. 16, items that can be set in the operation area include shutter speed, density range, preprocessing, non-measurable reference, uniform interval processing, space code, projector selection, display image, and the like. Hereinafter, the description will be made sequentially.
(Shutter speed setting means 49)

As an example of the shutter speed setting unit 49 for adjusting the shutter speed during imaging by the imaging unit, a shutter speed setting field 294 is provided in the example of FIG. From the shutter speed setting field 294, the user can specify the shutter speed. Here, a preset shutter speed, for example, 1/15, 1/30, 1/60, 1/120, 1/240, 1/500, 1/1000,... Select. The number of seconds corresponding to the selected numerical value is displayed in the numerical value display column 295 on the right side. It is also possible to directly specify an arbitrary shutter speed numerically. For example, when “numerical value input” is selected as a drop-down box option, the gray-out of the numerical value display column 295 is canceled, and a numerical value can be directly input. In this way, the exposure time of the camera (imaging device) that is the imaging means is adjusted based on the numerical value specified in the shutter speed setting field 294. When adjusting the shutter speed, it is easier to perform the confirmation work if the grayscale image of the luminance image is displayed in the second image display area 121 than the distance image. Furthermore, by the real-time update function, an image whose shutter speed has been changed in the shutter speed setting field 294 is immediately reflected in the second image display area 121, so that the user can visually determine whether the current setting is appropriate. It can be confirmed and adjustment work can be easily performed.
(Tint range setting field 296)

In the density range setting field 296, the dynamic range of the luminance image that is the density image is adjusted. Here, the dynamic range is increased or decreased by selecting one of “low (−1)”, “normal (0)”, and “high (1)” from the drop-down box.
(Pre-processing setting field 310)

In the pre-processing setting field 310, common filter processing to be performed before the distance image is generated by the head unit is defined. Examples of common filter processing include an averaging filter, a median filter, and a Gaussian filter. Here, as the filter processing for the pattern projection image, one of “none”, “median”, “Gaussian”, and “average” is selected from the drop-down box in the example of FIG. The filter process can be performed on the distance image obtained on the head unit 1 side as well as the pre-processing performed before the distance image is generated.
(Measurement impossible standard setting field 312)

  In the measurement impossible reference setting field 312, a level for cutting the noise component is set. That is, the height measurement is not performed by the amount set in the non-measurable reference setting field 312. In the measurement of three-dimensional height information using a pattern projection image, accurate height information cannot be measured without a certain amount of light. On the other hand, when multiple reflection occurs, it is necessary to reduce the amount of light because it is too bright. Thus, the noise component cut amount is selected according to the captured pattern projection image. Specifically, a threshold value for determining invalid data due to noise is determined for data for calculating height information of each pixel.

  Here, as shown in FIG. 18, one of “high”, “medium”, “low”, and “none” is selected from the drop-down box. If “None” is selected, noise components are not cut and height measurement is performed for all pixels. For example, in the example shown in FIG. 19, “None” is selected in the non-measurable reference setting field 312, and height data is calculated at all points including noise data. Although it is difficult to discriminate from this screen, an illegal height is measured by noise data at a corner portion of the workpiece.

  On the other hand, in the example shown in FIG. 20, “low” is selected in the non-measurable reference setting field 312 and the points where the height information based on the noise data is incorrect are reduced. Further, in the example shown in FIG. 21, “medium” is selected in the non-measurable reference setting field 312 and the points where the height information is incorrect are reduced.

On the other hand, it can also be confirmed from FIG. 21 that the black dots indicating that measurement is impossible increase especially in the lower left area of the workpiece, and the positions where height measurement cannot be performed due to noise are increased. Furthermore, the example shown in FIG. 22 shows a state in which “high” is selected in the non-measurable reference setting field 312, and as a result of the removal of the noise component being too effective, data that is originally desired to be lost is lost. I can confirm that. As described above, when the measurement impossible reference indicating the noise removal threshold is too low, the height is calculated based on the noise. On the other hand, if it is too high, the part that is originally desired to be left is regarded as invalid. For this reason, the user uses the real-time update function to adjust the measurement impossible standard setting, confirm the adjusted image in the second image display area, and refer to the setting result directly in the image. It can be adjusted to an appropriate value.
(Equal interval processing setting field 314)

  In the uniform interval processing setting field 314, an error due to the angle of view is corrected. The equal interval processing setting field 314 functions as the interval equalization processing setting means 47. In the uniform interval processing setting field 314, ON and OFF can be selected as shown in FIG. By turning on the equal interval processing, distance images arranged at equal pitches in the xy direction are acquired. Here, an equal pitch image whose positions in the XY directions are equally spaced regardless of the height (Z coordinate) is displayed in the second image display area 121. For example, in an application for inspecting dimensions in the XY plane, it is necessary to turn on the uniform interval processing. In addition, the portion where data is lost after correction is treated as invalid. 24 and 25 show a state in which the equal interval processing is turned on. In the example of FIG. 24, the “display image” selection field 322 is displayed in the second image display area 121 as a “height image”, that is, a distance image. FIG. Each displayed example is shown.

On the other hand, when the uniform interval processing is turned off, an image as it is seen (Z image) is obtained as shown in FIG. 26, and distortion occurs in the XY directions as it goes to the edge of the screen. However, since the equal interval processing is not performed, the time required to display the image can be shortened. FIG. 27 shows an example in which “grayscale image” is selected in the display image column and a luminance image is displayed in the second image display area 121.
(Space code setting field 316)

  In the space code setting field 316, whether or not the space coding method is used is selected. That is, the space code setting field 316 functions as the space coding switching means 45. In this three-dimensional image processing apparatus, the phase shift method is indispensable for the generation of the distance image, and in addition to the phase shift method, whether or not the spatial encoding method is applied can be selected in the spatial code setting field 316. In the space code setting field 316, ON and OFF can be selected as shown in FIG. When the space code setting field 316 is set to ON, height measurement is performed by a combination of the space coding method and the phase shift method. This example is shown in FIGS. In these drawings, FIG. 29 shows a state in which a distance image is selected as an image to be displayed in the second image display area 121. Specifically, “height image” is selected in the “display image” selection field 322. On the other hand, FIG. 30 shows a state in which a luminance image is displayed in the second image display area 121, and “grayscale image” is selected in the “display image” selection field 322. By using the spatial encoding method in addition to the phase shift method, an appropriate distance image can be acquired. Specifically, phase jump correction (phase unwrapping) by the phase shift method can be performed by the spatial encoding method, so that measurement can be performed with high resolution while ensuring a wide dynamic range. However, the imaging time is about twice as long as when the imaging time is OFF.

  On the other hand, as shown in FIGS. 31 and 32, when OFF is selected in the space code setting field 316, the height is measured only by the phase shift method. In this case, since the height measurement dynamic range is narrowed, in the case of a workpiece having a large difference in height, if the phase is shifted by one cycle or more, the height cannot be measured correctly. On the contrary, in the case of a work whose change in height is small, since the striped image is not captured or synthesized by the spatial coding method, the processing can be speeded up by that amount, and there is an advantage that the imaging time can be reduced to about half. For example, when measuring workpieces with small differences in height direction, it is not necessary to increase the dynamic range, so the processing time can be shortened while maintaining high-precision height measurement performance using only the phase shift method. Can do. In this case, since the height measurement dynamic range is narrowed, in the case of a workpiece having a large difference in height, if the phase is shifted by one cycle or more, the height cannot be measured correctly. On the other hand, in the case of a work with little change in height, the advantage that the imaging time can be halved by turning the space code OFF can be obtained.

  In the example of FIG. 31, the “display image” selection field 322 shows a state in which the “height image”, that is, the distance image is displayed, while in the example of FIG. 32, the “display image” selection field 322 displays the “grayscale image”. "Is displayed.

In this example, the phase shift method is essential, but ON / OFF of the phase shift method may be selectable.
(Projector selection setting field 318)

  The projector selection setting field 318 functions as a light projection switching unit 48 that switches ON / OFF of the first projector and the second projector. Here, in the projector selection setting field 318, the light projection means (projector) to be used is selected from the first projector and the second projector which are two light projection means. In the example of the projector selection setting field 318, as shown in FIG. 33, “1” (first projector), “2” (second projector), “1 + 2” (first projector and second projector) are selected from the drop-down box. Select one of the projectors.

  When “1” or “2” is selected in the projector selection setting field 318, that is, in the case of one-side projection that is projection from either the first projector or the second projector, the portion that is shaded by projection No height measurement is performed. The example in FIG. 34 shows an example in which “1” is selected in the projector selection setting field 318, and the example in which “2” is selected in the example in FIG. In each screen, the shadow data is displayed in black, and it can be confirmed from each screen that there is an area where the height cannot be measured on the workpiece. Further, as is clear from these figures, it can be seen that the area where measurement is impossible depends on the light projecting means. In other words, even if one of the light projecting means becomes an unmeasurable area, the other light projecting means can project light, and thus there are many areas where height measurement is possible. Therefore, by combining these, the non-measurable area can be reduced. In particular, the first projector and the second projector are arranged so that the workpiece is sandwiched from both sides, so that the first projection from the first projector and the second projection from the second projector are opposed to each other. Since the work is irradiated, even if it is an area that is shaded by any one of the projections, the risk of being shaded by the projection from the other in the opposite direction can be reduced.

Specifically, as shown in FIG. 36, when “1 + 2” is selected in the projector selection setting field 318, the projector is switched to both projections in which light is projected from both the first projector and the second projector. In this state, even a portion that is shaded by light projection from one of the projectors can be interpolated if it can be projected by the other projector. However, in this case, an imaging time that is about twice as long as that of single projection is required. The user selects which projection is to be used according to the unevenness of the workpiece to be inspected and the allowable imaging time.
("Display image" selection field 322)

In the “display image” selection field 322, an image displayed in the second image display area 121 is selected. For example, the validity of each setting can be visually confirmed from the actually displayed image by selecting the display target according to the purpose of the inspection. In particular, by turning on the real-time updating means described above, it is possible to sequentially update changes in settings and compare them before and after the changes, so that the settings can be adjusted based on the image so that the intended image is adapted to the application. . In addition, even a beginner who is not familiar with the meaning of each setting parameter can obtain the advantage of setting while viewing the image. In this example, as shown in FIG. 37, from the “display image” selection field 322, “height image”, “grayscale image”, “whiteout / blackout image”, “stripe projection-projector 1”, “striping”. One of “projection-projector 2” is selected. The “height image” is a distance image, and displays an image colored in a contour line for each height. A “grayscale image” is a luminance image. In this example, an image obtained by combining a plurality of pattern projection images captured based on the phase shift method is used as a luminance image. However, it is also possible to illuminate the work and take an optical image with the imaging means and use it as a luminance image.
(Abnormal point highlighting means 62)

  Further, the three-dimensional image processing apparatus includes abnormal point highlighting means 62 as shown in FIG. For example, the “whiteout / blackout image” that can be selected in the “display image” selection field 322 described above partially includes pixels that are saturated and overexposed, blackout pixels that are insufficient in light quantity, and the like with respect to the luminance image. It is a colored image. In this way, by highlighting the parts that are not accurate in the image and are considered to have low measurement accuracy, the part with low measurement accuracy is visually displayed to the user. To make it easier to confirm whether an image corresponding to the desired inspection application is obtained. In this example, the whiteout pixels are colored yellow and the blackout pixels are colored blue. As a result, the user can visually confirm how the whitened areas are distributed in the image with the color as a clue. In the lower part of the second image display area 121, the number of overexposed pixels and underexposed pixels is counted and displayed. With reference to this, the user adjusts each setting item so that the number of pixels approaches zero.

  In addition, the color and aspect to be colored are not limited to this, and various known aspects such as displaying with other colors or blinking can be used as appropriate. In addition, by changing the color to be colored by the overexposed pixels and the underexposed pixels, it is possible to notify the user of the reason why the measurement reliability is low, and it is easy to take countermeasures. However, the same color or highlight may be applied to the whiteout pixel and the blackout pixel.

  “Flying light projection—projector 1” is a pattern projection image expressed by shading obtained by pattern projection only by the first projector. “Flying light projection—projector 2” is a pattern projection image obtained only by the second projector. FIG. 38 shows an example in which “striped light projection—projector 1” is selected in the “display image” selection field 322, and the pattern projection image of the first projector is displayed in the second image display area 121. FIG. An example in which the pattern projection image of the second projector is displayed in the second image display area 121 by selecting the “stripe projection-projector 2” is shown. By checking the fringe image from this screen, it is possible to identify the cause when the workpiece height cannot be measured, for example, because the workpiece material is translucent, the projected light is embedded, etc. It can be confirmed from the original pattern projection image before the distance image is generated that the work is difficult.

  The user confirms whether the shutter speed and the light / dark range are appropriate while referring to the image displayed in the second image display area 121, and adjusts it to an appropriate value. Specifically, the adjustment is performed while confirming that the overexposed and underexposed pixels are reduced in the state where the overexposed and underexposed images are displayed in the second image display area 121. For example, by adjusting the shutter speed in the shutter speed setting field 294, blackout pixels, that is, portions that are too dark due to insufficient light quantity are eliminated. Further, by adjusting the density range, there are no overexposed pixels, that is, parts that are too bright. In the example of FIG. 16, there are many blacked out pixels in the overexposed / blackout image because it is too dark. Therefore, the shutter speed is adjusted.

  For example, when the shutter speed setting field 294 is set to “1/120” and switched to “1/15” as shown in FIG. 40, the luminance image becomes slightly brighter and the number of blackout pixels becomes 0. It turns out that it became. However, the number of overexposed pixels has increased. Thus, as shown in FIG. 41, when the shutter speed is switched to “1/30”, it can be seen that the number of blackout pixels remains 0 and the number of overexposed pixels decreases.

  Furthermore, when the density range setting field 296 is set to “low (−1)” and switched to “normal (0)” by one step as shown in FIG. 42, the overexposed pixels are reduced. I understand. Further, as shown in FIG. 43, when switching to “high (1)”, it is understood that the overexposed pixel has become zero. As a result, both the number of overexposed pixels and the number of underexposed pixels become 0, and the adjustment operation of the shutter speed and the light / dark range is completed.

After setting the desired imaging conditions as described above, the distance image and the luminance image are registered as shown in FIGS. 12 to 13 described above, and the image registration operation is completed.
(Height measurement setting procedure)

  Next, a procedure for setting an area for height measurement for an input image of a workpiece to be inspected that is sequentially input during operation will be described with reference to FIGS. 44 to 55. Here, a typical workpiece representing a plurality of workpieces to be input is registered in advance as a registered image by the above-described procedure, and an area for height measurement is designated for the registered image.

Specifically, the “height measurement” processing unit 266 is added from the initial screen 260 of FIG. 9 as shown in FIGS. 44 to 45. In the example of FIG. 44, “Add” is selected from the first submenu 370 displayed by right-clicking or the like at the bottom of the “imaging” processing unit 263 in the flow display area 261, and “Measurement” in the second submenu 372 is selected. The “height measurement” processing unit 266 for performing “height measurement” is added from the inspection processes listed in the “measurement” menu 373 displayed by selecting “”. As a result, a “height measurement” processing unit 266 is newly added below the “imaging” processing unit 263 in the flow display area 261 as shown in FIG. In this way, the “measurement” menu 373 functions as an inspection process selection unit for selecting an inspection process to be executed by the inspection execution unit.
(Inspection area setting screen 120)

  Next, a procedure for setting an area as a setting to be performed by the “height measurement” processing unit 266 will be described with reference to FIGS. 46 to 49. First, when the edit screen of the “height measurement” processing unit is called from the screen of FIG. 45, the screen shifts to the height measurement setting screen 460 shown in FIG. Also in the GUI screen example of FIG. 46, as in FIG. 10, a first image display area 111 for displaying an image is provided on the right side of the screen, and a setting item button area 112 in which a plurality of setting item buttons are arranged is provided on the left side. The setting item button area 112 includes an “image registration” button 113, an “image setting” button 114, an “area setting” button 115, a “preprocessing” button 117, a “detection condition” button 118, a “detailed setting” button 119, A “judgment condition” button, a “display setting” button, a “save” button, and the like are provided. From this screen, when the user presses an “area setting” button 115 corresponding to the inspection area setting means, the screen changes to an inspection area setting screen 120 shown in FIG. On the inspection target area setting screen 120, an area to be inspected can be designated. In the example of FIG. 47, the second image display area 121 is provided on the left side of the screen, and the operation area 122 for performing various operations is arranged on the right side of the screen. In the upper part of the operation area 122, a “display image” selection column 124 for selecting an image displayed in the second image display area 121 is provided. In the example of FIG. 47, the registered image is selected in the “display image” selection field 124. Further below that, a “measurement area” setting field 126 is provided as an inspection object area setting means for designating an area to be inspected.

  In the “measurement area” setting field 126, a predetermined area can be selected. Here, when the “measurement area” setting field 126 is selected, a drop-down box is displayed as shown in FIG. 48, and a desired measurement area shape can be selected. In this example, “None”, “Rectangle”, “Rotating Rectangle”, “Circle”, “Oval”, “Circle”, “Arc”, “Polygon”, “Composite area” or the like is displayed. If “None” is selected, the entire image displayed in the second image display area 121 is used as the inspection target area.

  Furthermore, detailed dimensions and the like can be set according to the shape selected in the “measurement area” setting field 126. In the example of FIG. 48, an eraser is used for the workpiece, and the “measurement area” setting field 126 shows an example in which “rotation rectangle” is selected as shown in FIG. When the “edit” button 128 is pressed in this state, a measurement area edit screen 130 shown in FIG. 50 is displayed. In the example of FIG. 50, in the second image display area 121, the rotation rectangle is displayed so as to overlap the work. Here, a rectangular measurement region is drawn on the eraser case, and is displayed overlaid on the distance image. In addition, the basic vector of the rotation rectangle is displayed as an arrow in the frame shape of the rotation rectangle, and the width and height of the rotation rectangle, the XY coordinates of the center, the inclination angle of the basic vector, etc. Is displayed. The user can directly adjust the shape, position, etc. of the rotation rectangle by directly inputting a numerical value from the measurement area editing screen 130 or operating the handle displayed on the rotation rectangle with a mouse or the like.

Items that can be set on the measurement area editing screen 130 change according to the shape selected in the “measurement area” setting field 126. For example, when “circumference” is selected, parameters relating to the circumference, such as designation of the outer diameter and inner diameter of the circumference, can be set as shown in FIG. Furthermore, it is possible to designate a mask area from the screen of FIG. As the mask region, a circular shape, a donut shape, a rectangular shape, other polygonal shapes, a free curve, or the like can be designated. In this way, according to the shape of the workpiece to be inspected, it is possible to appropriately set the inspection target area and eliminate areas unnecessary for inspection such as perforated portions and backgrounds, thereby improving processing efficiency. .
(Second measurement display area)

  When the measurement area is set in this way, the set measurement area is displayed over the workpiece as shown in FIG. Subsequently, the same operation is repeated when another measurement region is designated. That is, as shown in FIG. 52, another second “height measurement” processing unit 266B is added below the “height measurement” processing unit 266 in the flow display area 261. As shown in FIGS. 53 and 54, a rotation rectangle is set as a new measurement area on an area of the eraser which is a work, which is not covered with a case. As a result, as shown in FIG. 55, the newly set second measurement area is displayed over the workpiece.

In the above example, one “height measurement” processing unit performs one height measurement process. That is, in order to perform a plurality of height measurement processes, it is necessary to add a plurality of “height measurement” processing units. However, it is needless to say that a plurality of height measurement processes can be performed in one “height measurement” processing unit.
(Measurement process)

When the setting of the measurement area is completed in this way, a process for actually performing measurement is added next. Here, as shown in FIG. 56, a “numerical calculation” processing unit for performing “calculation” is added below the second “height measurement” processing unit 266B in the flow display area 261. As the contents of the calculation executed by the “numerical calculation” processing unit, numerical calculation, image calculation, calibration, image connection, and the like can be selected. Here, as shown in FIG. 57, a “numerical calculation” processing unit in which numerical calculation is selected is added.
("Numerical operation" processing unit)

  In the “numerical operation” processing unit, a specific arithmetic expression can be input. For example, as shown in FIG. 58, a numerical calculation editing screen in which a mathematical expression can be directly input is displayed, and the user defines the mathematical expression. Here, a calculator-like input pad is prepared, and editing buttons such as copy, cut, and paste are also prepared to make it easy to create an arithmetic expression. The user inputs a desired arithmetic expression from this screen. An example of the input arithmetic expression is shown in FIG.

When the contents of the numerical calculation process are defined in this way, an arithmetic expression is displayed on the third image display area 262 on the initial screen 260 as shown in FIG.
("Area" processing unit)

Furthermore, in the example of FIG. 61, an “area” processing unit is added below the “numerical calculation” processing unit. In the “area” processing unit, conditions for actually performing pass / fail determinations and the like are defined. Specifically, in accordance with a registered image or an input image, a gradation conversion parameter (details will be described later) is used to appropriately change the gradation conversion condition for converting the distance image into a low gradation distance image. A region for acquiring reference information, a condition for extracting height from this region, a condition for performing filter processing when generating a distance image, and the like are set. That is, in the “area” processing unit, region setting, height extraction, preprocessing, determination, and the like are set. First, the region setting procedure is the same as that of the registered image described above. That is, when the “area setting” button 115 arranged in the setting item button area 112 is pressed from the area setting screen 620 as shown in FIG. 62, the area setting screen shown in FIG. 63 is displayed, and the target area is designated. Again, the rotation rectangle is selected, and more detailed coordinates and the like are designated as necessary. In this manner, the area in the “area” processing unit is determined, and the rotation rectangle is displayed on the work in the second image display area as shown in FIG.
(Height extraction setting screen)

  Next, set the height extraction. The setting of height extraction is to set a gradation conversion parameter when performing gradation conversion. That is, when the “height extraction” button 116 is pressed from the setting item button area 112 in FIG. 62, the screen moves to a height extraction selection screen 140 shown in FIG. 65, and a display image, an extraction method, and the like can be selected. In the height extraction selection screen 140, as in FIG. 47 and the like, a second image display area 121 is provided on the left side of the screen, and an operation area 122 for performing various operations is arranged on the right side of the screen. In the upper part of the operation area 122, a “display image” selection column 124 for selecting an image displayed in the second image display area 121 is provided. In the example of FIG. 65, the registered image is selected in the “display image” selection field 124. Further below, an extraction method selection means 142 for selecting an extraction method of the height extraction function is provided.

Here, the “height extraction” button 116 functions as a gradation conversion condition setting unit 43 that sets gradation conversion parameters for performing gradation conversion of the distance image by the gradation conversion unit. In particular, the gradation conversion condition setting unit 43 is displayed when a process that does not require image height information is selected by the inspection process selection unit. On the other hand, when a process that requires image height information is selected by the inspection process selection unit, the gradation conversion condition setting unit is not displayed. Specifically, when the “height measurement” processing unit 266 is selected as the inspection processing tool, the “height extraction” button is not displayed in the flow display area 261. For other inspection processing tools such as the “area” processing unit, the “blob” processing unit 267, the “color inspection” processing unit 267B, the “Shapetrax2” processing unit 264, the “position correction” processing unit 265, etc. An “Extract” button 116 is displayed, and gradation conversion conditions can be set. In this way, when gradation conversion is necessary, the gradation conversion condition setting means 43 is displayed to prompt the user to make necessary settings. By not displaying the condition setting unit itself, the user is prevented from being confused by unnecessary settings, and an easy-to-use environment is realized.
(Extraction method selection means 142)

The extraction method selection unit 142 designates a gradation conversion method. For example, the user selects either static conversion or dynamic conversion. In the example of FIG. 65, as a choice, either “single point designation” or “three point designation (plane)” corresponding to static conversion or “real time extraction” corresponding to dynamic conversion is selected from a drop-down box. Let
(One point designation screen 150)

  When “single point designation” is selected from the extraction method selection means 142 on the screen of FIG. 65, the screen shifts to the single point designation screen 150 of FIG. 66 to 96 show an example in which a 50-yen coin is used as a work for explanation. 66, the height of the part designated on the second image display area 121 is set as the reference height (reference height). In the example of FIG. 66, when the “extract” button 144 provided in the operation area 122 on the right side of the screen is selected, the screen shown in FIG. 67 is displayed, and an arbitrary position on the second image display area 121 on the left side of the screen can be designated. It becomes like this. Here, the position which becomes the center of the height in the work displayed in the second image display area 121 is designated by using the height extracting means. In the example of FIG. 67, the height extracting means is configured by an “extract” button 144 displaying a dropper-shaped icon SI. When the “extract” button 144 is pressed, a dot is displayed on the second image display area 121. A pointer 146 is displayed. The position designated by the pointer 146 is registered as the intermediate height of the distance range.

  A range for obtaining the height around the point designated by the pointer 146 can be designated in the “extraction area” designation field 145. In the “extraction area” designation field 145, one side of the area for which the average height is calculated is designated by the number of pixels. In the example of FIG. 67, “16” is designated in the “extraction area” designation field 145, and the average height in the 16 pixel × 16 pixel area centered on the point designated by the pointer 146 is extracted. , The height extracted by the pointer 146. The size of the area designated by the pointer 146 on the second image display area 121 can be changed in conjunction with the numerical value designated by the “extraction area” designation field 145.

In the “Z height” display field 152, the height information of the designated part is displayed as a numerical value (in the example of FIG. 68, 1.253 is displayed in the “Z height” display field 152). For example, when the distance range is expressed by 2 ⁸ = 256 gradations (0 to 255), the height (density value / mm; details will be described later) is 128 with the height specified by the height extraction means as the center value. Is set to be With this configuration, the user can specify the height to be inspected directly on the screen, and the tone is converted into a low-tone range image in the range centered on the specified height. The situation where information is damaged can be avoided.
(Simple display function)

As described above, when the distance range and span are determined as the gradation conversion parameters necessary for gradation conversion, it is possible to perform gradation conversion from a high gradation distance image to a low gradation distance image. In addition, on the second image display area 121, as shown in FIG. 68, a low gradation distance image subjected to gradation conversion under the gradation conversion conditions currently set in the operation area 122 is simply displayed. . Further, when the gradation conversion condition is changed in the operation area 122, the simple display of the low gradation distance image after gradation conversion on the second image display area 121 according to this is also performed according to the changed gradation conversion condition. Updated. As a result, the user can quickly and visually confirm the change after adjustment of the gradation conversion condition, and can easily perform adjustment work by trial and error. In this way, the image displayed in the second image display area 121 displays a mode for displaying a distance image before gradation conversion, a mode for displaying a low gradation distance image after gradation conversion, and a normal luminance image. It can be changed by switching the mode.
(Gain adjustment means)

  Further, the user can perform gain adjustment, which is one of the gradation conversion parameters, using the gain adjusting means. In the example of FIG. 68, an emphasis method setting field 154 is provided in the middle of the operation area 122, and a gain adjustment field 156 is arranged here as a gain adjusting means. The gain adjustment field 156 displays the current gain as a numerical value. Here, the gain [gradation / mm] is a parameter corresponding to a span when gradation conversion is performed. For example, when converting a gradation image of 16 gradations to 8 gradations, it is set how many gradations in 8 gradations are converted per mm. If the gain value is increased, gradation conversion with clear contrast is obtained. For example, when the gain value is set to 100 [gradation / mm], gradation conversion is set to 0.010 mm per gradation. If the height information of the distance image before conversion has a resolution of 0.00025 mm per gradation, the difference between the obtained reference plane and the input height data is N gradations before conversion. After conversion, it can be calculated as N [gradation] × 0.00025 [mm / gradation] × 100 [gradation / mm] = N × 0.025 gradation.

  Here, the reference plane is a plane obtained by a method such as single point designation, average height standard, three-point designation, plane standard, or free-form surface standard, which will be described later, and is a standard plane for gradation conversion. For example, as shown in FIG. 69A, when the cross-sectional profile of a distance image (input image) before conversion of 16 gradations has a shape shown by a solid line, the reference plane is shown by a wavy line. The profile of the low gradation distance image obtained by converting the gradation of such an input image from 16 gradations to 8 gradations is as shown in FIG. 69B, and the gain (conversion coefficient) is directly applied to the difference from the reference plane. It will be in such a state.

Further, the height per gradation (the reciprocal of the gain value) is automatically calculated according to the gain value described above, and can be displayed together. In the example of FIG. 68, the gain value is 250 [gradation / mm], and the height per gradation is 0.0040 mm. The user can adjust the gain value by changing the gain value. For example, when the gain value is increased, as shown in the screen of FIG. 68 to the screen of FIG. 70, the height difference can be emphasized by intensifying the density difference, but the inspectable height range is narrowed. Conversely, when the gain value is lowered, as shown in FIG. 71, inspection can be performed up to a wide height range, but fine changes are lost. As described above, when the gain is adjusted by the gain adjusting means, the gradation conversion image obtained under the gradation conversion condition can be confirmed in the second image display area 121. Accordingly, the user can adjust the gain value to an appropriate value according to the inspection purpose or the like while checking the gradation conversion image updated in real time.
(Extraction height setting)

  Further, the setting item of the enhancement method can include the setting of the extraction height in addition to the gain value. For example, when the “detailed setting” button 158 provided at the lower right of the operation area 122 is pressed on the screen of FIG. 72, the screen is switched to the emphasis method detail setting screen 160 of FIG. In addition to 156, an “extraction height” setting field 162 is displayed. In the “extracted height” setting field 162, as height information to be extracted by the height extracting means, any one of high height information, low height information, and high and low height information included in the region is selected. Can be selected. Here, as shown in FIG. 74, one of “high side”, “low side”, and “both high and low” can be selected from the drop-down list provided in the “extraction height” setting field 162. For example, when “higher side” is selected, gradation conversion is performed so that the height of the position designated by the pointer 146 becomes the lower limit of the distance range. As a result, a low gradation distance image in which only the side higher than the designated height is extracted is generated. Similarly, when “low side” is selected, gradation conversion is performed so that the height of the position designated by the pointer 146 becomes the upper limit of the distance range. As a result, a low gradation distance image in which only the side lower than the specified height is extracted is generated. Further, in the case of “both high and low”, gradation conversion is performed so that the height of the position designated by the pointer 146 is in the middle of the above-described distance range. Note that pixels that are out of range after gradation conversion are clipped to black (pixel value 0 in the case of 8 gradations) on the lower side and white (pixel value 255) on the higher side.

Further, in the emphasis method detailed setting screen 160 of FIG. 73, a noise removal setting field 164 for removing noise and an invalid pixel designation field 166 for designating a value to be given to invalid pixels are also provided.
(Noise removal setting field 164)

  In the noise removal setting column 164, as one of the gradation conversion parameters, it is specified how many mm of the difference from the reference plane is removed as noise. For example, if the noise removal parameter is set to 0.080 mm, the difference of 0.080 mm from the reference plane is removed. Here, assuming that the height information before conversion has a resolution of 0.00025 mm per gradation, the difference of 0.080 [mm] ÷ 0.00025 [mm] = 320 [gradation] is ignored. It becomes. This state will be described with reference to FIGS. 75A to 75C. In these drawings, FIG. 75A shows a cross-sectional profile of a distance image before conversion of 16 gradations by a solid line, a reference plane thereof by a wavy line, and a range from which noise is removed by an alternate long and short dash line, as in FIG. 69A. Yes. As a result of performing noise removal on such an input image with reference to the reference plane, the profile shown in FIG. 75B is obtained. Further, the profile of the low gradation distance image obtained by converting the gradation image from 16 gradations to 8 gradations with respect to the distance image of FIG. 75B is as shown in FIG. 75C. Will be applied.

  Further, the effects of gain adjustment and noise removal will be described based on FIGS. 76A to 76F. First, assume that a luminance image as shown in FIG. 76A and a high gradation (16 gradations) distance image as shown in FIG. 76B are obtained. Here, the distance image in FIG. 76B remains at the initial settings (here, the gain is 100 [gradation / mm] and the noise removal is 0.000 [mm]), and the low gradation (8 gradations). FIG. 76C shows the low gradation distance image obtained by gradation conversion to. This low gradation distance image has a relatively low contrast. Therefore, when the gain is increased from this state, as shown in FIG. 76D, a low gradation distance image with high contrast is newly subjected to gradation conversion from FIG. 76B and displayed. However, the noise component is also large in this image. In the example of FIG. 76D, the gain is set to 1000 [gradation / mm] and the noise removal is set to 0.000 [mm]. Therefore, FIG. 76E shows a low gradation distance image in which the noise removal amount is increased from FIG. 76D. Here, the gain is set to 1000 [gradation / mm] and the noise removal is set to 0.080 [mm]. As a result, the noise component is reduced. On the other hand, it can be confirmed that noise having a height lower than that of the reference plane exists in the upper right of the upper left “E”. Therefore, when “higher side” is set in the “extraction height” setting field 182 shown in FIG. 74 and the like, the reference plane is converted so as to have the lowest value (pixel value 0). As a result of being ignored and extracting only the side higher than the reference plane, a low gradation distance image as shown in FIG. 76F is obtained. For example, when the “extraction height” is set to “high side” for the distance image (16 gradations) of the profile as shown in FIG. 77A, only the high side from the reference plane is extracted. A low gradation distance image (8 gradations) as shown in FIG. 77B is obtained. In this way, it is possible to obtain a low gradation distance image having a higher contrast and less noise components than those in FIG. 76C and the like. In this example, as final gradation conversion parameters, the gain is set to 1000 [gradation / mm], the noise removal is set to 0.080 [mm], and the “extraction height” is set to the “higher side”. The gradation conversion is performed from the distance image of 76B to the low gradation distance image of FIG. 76F.

  In this way, when the conditions necessary for executing the single point specification are set, the input image is scaled from the high gradation distance image to the low gradation distance image according to the designated gradation conversion condition, that is, the reference height. The tone is converted and displayed in the first image display area 111 as shown in FIG.

Here, an example of a work in which the method of designating the reference plane by specifying one point will be described based on FIGS. 79A to 79B. FIG. 79A shows a workpiece WK7 that does not have a planar inclination on the measurement surface of the workpiece or does not affect the inspection process even if there is a slight inclination. Here, an inspection process for reading whether or not the character string is appropriate is performed on the workpiece WK7 in which numbers and character strings are three-dimensionally formed on the surface of the casting. In such an application, by pressing an “extract” button 144 shown in FIG. 67 or the like, a dropper-shaped icon SI is displayed on the second image display area 121. Then, with the pointer 146, as shown in FIG. 79A, a point on the plane (background surface) on which the character string is not formed on the upper surface of the work WK7 is designated. Thereby, gradation conversion is performed using the height of the extraction area (16 pixels in the example of FIG. 67) designated by the pointer 146 as a reference plane, and the image is converted into a low gradation distance image shown in FIG. 79B. In this low gradation distance image, the character string portion protruding from the plane of the work WK7 is clearly extracted, so that accurate OCR can be easily performed. As described above, the one-point designation can be effectively used in a case where there is no influence on the inspection process even if the workpiece is slightly inclined. One-point designation has the advantage of being able to process at high speed with low load.
(Specify three points)

  The method for setting the gradation conversion condition by specifying one point has been described above. Next, a method for setting gradation conversion conditions by specifying three points will be described based on the GUI screens shown in FIGS. The three-point designation is a method for gradation-converting a distance image into a low gradation distance image using a plane obtained from the three points designated by the user as a reference plane. The reference plane is also set to a height in the middle of the height range (distance range) to be converted to a low gradation distance image, for example, among the height information of the distance image, similarly to the reference height designated by one point described above. Alternatively, the upper limit (the highest position where gradation conversion is performed) and the lower limit (the lowest position where gradation conversion is performed) of the distance range can be set.

62. When the “height extraction” button 116 is pressed on the GUI screen of the 3D image processing program in FIG. 62 and the screen is shifted to the height extraction selection screen 140 shown in FIG. , “Specify three points (plane)” is selected. Thereby, a three-point designation screen 170 for performing height extraction setting shown in FIG. 81 is displayed.
(Three point designation screen 170)

  On the three-point designation screen 170, three reference points on the second image display area 121 are designated and set as a reference plane for gradation conversion. For this reason, a height extraction means is provided on the three-point designation screen 170 of FIG. More specifically, by selecting an “extract” button 144 provided in the operation area 122 on the right side of the screen, the screen shown in FIG. 82 is obtained, and an arbitrary position is set on the second image display area 121 on the left side of the screen. Point can be specified. Here, a point-like pointer 146 is displayed as the height extracting means as in FIG. 67, and the user sequentially designates a desired position with a pointing device such as a mouse, a trackball, or a touch panel. First, when the first point is designated on the second image display area 121, the rectangular shape changes to a cross shape at the designated position as shown in FIG. 83 to indicate the designated position, and the next second point is Similarly, the pointer 146 can be used for designation. At this time, the color distance image displayed in FIG. 82 is subjected to gradation conversion with reference to the horizontal plane including the designated height of the first point, and the low gradation distance image after gradation conversion is converted to a grayscale image. Is displayed in the second image display area. Further, when the second point is designated, the position of the second point changes from a rectangular shape to a cross shape as shown in FIG. 84, and the third point can be designated. At this time, gradation conversion is performed again with reference to the inclined plane including the two specified heights, and the low gradation distance image is updated. When the third point is designated, a reference plane is set by a plane including these designated three points. Further, when each point is designated by the height extraction means, the height of the point currently designated on the height extraction screen display area may be displayed in the “Z height” display column 152.

  Furthermore, an inclination angle can also be displayed as the reference plane information. In the example of FIG. 84, the height extraction display column 172 provided in the operation area 122 displays the X direction inclination, the Y direction inclination, and the third Z direction height of the reference plane.

  Further, as in the case of single point designation, it is possible to designate an emphasis method as necessary. For example, the gain adjustment means is used to perform gain adjustment, or the three-point designation “detailed setting” button 174 is pressed to call a three-point designation detailed setting screen 180 as shown in FIG. In addition to the gain adjustment field 156, an “extraction height” setting field 182 is displayed. As height information to be extracted by the height extraction means, “high side”, “low side”, “both high and low” are selected from the drop-down list. You can select either of these.

  In this way, it is possible to perform tone conversion of a distance image using an arbitrary plane defined by the designated three points as a reference plane. As a result, not only the gradation conversion based on the horizontal plane as described above, but also the gradation conversion using the inclined plane as the reference plane is possible. For example, in the inspection of scratches and foreign objects on the inclined surface of the workpiece surface, the distance range becomes narrow if the inclined surface is left as it is, but the inclined surface can be canceled efficiently by setting the reference surface along the inclined surface. Scratches and foreign objects can be detected. In this way, flexible gradation conversion using height information according to the workpiece and inspection purpose can be realized.

  Here, an example of a work in which the method of designating the reference plane by three-point designation will be described based on FIGS. 86A to 86D. FIG. 86A shows a workpiece WK8 that has an influence on the result of the inspection process when a plane inclination occurs on the measurement surface of the workpiece or a minute plane inclination exists. Here, an inspection process for detecting a ball grid array (BGA) formed on the substrate is performed. In such an application, by pressing an “extract” button 144 shown in FIG. 84 or the like, a dropper-shaped icon SI is displayed on the second image display area 121. Then, with the pointer 146, as shown in FIG. 86A, three points on the upper surface of the work WK8 where no BGA is formed are designated. As a result, a plane including the designated three points is extracted as a reference plane, gradation conversion is performed, and the low gradation distance image shown in FIG. 86B is converted. In this low gradation distance image, the BGA protruding from here is clearly extracted with the plane of the work WK8 as the background. For example, this can be binarized as shown in FIG. 86C to confirm the shape of the BGA. With this method, there is an advantage that it can be accurately detected even if the plane of the workpiece WK8 is inclined. If the workpiece WK8 shown in FIG. 86A has an inclination, for example, if one point is specified, a binarized image becomes as shown in FIG. 86D and cannot be detected correctly. On the other hand, in the three-point designation, an accurate detection result can be obtained by correcting the inclination as described above. As described above, the three-point designation is effective in the case where the inclination of the plane affects the inspection processing result.

  Furthermore, when there is a gentle depression on the upper surface of the planar workpiece WK9 as shown in FIG. 87A, an inspection process for detecting this depression will be considered. Here, as shown to FIG. 87A, measurement area | region ROI is set to the area | region containing a hollow. Thereby, gradation conversion is performed using the plane obtained from the height data of the entire measurement region ROI including the depression as a reference plane, and a low gradation distance image as shown in FIG. 87B is obtained. In this example, the reference plane is estimated by the least square method. Further, the obtained low gradation distance image is binarized to obtain a binarized image shown in FIG. 87C. In this way, the inclination can be corrected and only the recessed portion can be stably extracted. If a binarized image is obtained by specifying one point in a state where there is an inclination, the result as shown in FIG. 87D is obtained, and it can be seen that it is difficult to detect the depression due to the inclined surface. Thus, the three-point designation is effective for estimating the reference plane with high accuracy. In terms of processing time, although processing time is longer than single point designation, processing can be performed at a relatively high speed.

In the above, static conversion in which gradation conversion conditions are specified in advance at the setting stage and gradation conversion is performed under the specified conditions during operation has been described. In other words, in the static conversion, the gradation conversion parameter is a constant value regardless of the input image. Next, a specific example of the dynamic conversion that adjusts the gradation conversion condition according to the input image to be inspected will be described. First, in dynamic conversion, as a specific method for correcting the reference of height information to be left when gradation conversion from a distance image to a low gradation distance image,
(B1) An average height reference for gradation conversion using an average height (average distance) in an average extraction area designated for the input image as an average reference height;
(B2) a plane reference for generating an estimated plane in the designated area of the input image and performing gradation conversion using this as a reference plane;
(B3) A free-form surface reference that generates a free-form surface from which high-frequency components are removed from an input image and performs gradation conversion using this as a reference surface is included. Hereinafter, each method will be sequentially described.
(B1: Average height reference)

  The average height reference is a method of calculating the average height in a specified average extraction area for each input image and performing gradation conversion using this as the average reference height. The average extraction area for defining the average reference height is set in advance prior to operation (step S83 in FIG. 8 described above). Hereinafter, an example of a procedure for designating the average extraction region in step S83 in FIG. 8 will be described based on the GUIs in FIGS. 62, 65, and 88 to 92. First, the “height extraction” button 116 is selected on the GUI screen shown in FIG. 62, the process proceeds to the height extraction selection screen 140 shown in FIG. 65, and the “real-time extraction” corresponding to the dynamic conversion is selected by the extraction method selection means 142. Then, the height dynamic extraction setting screen 190 of FIG. 88 is displayed. In this example, an eraser is used again for the workpiece. Next, in the “calculation method” selection column 192 provided below the extraction method selection means 142, a dynamic conversion criterion is designated. Here, as shown in FIG. 89, one of “average height standard”, “plane standard”, and “free curved surface standard” is selected from the drop-down box. Here, “average height reference” is selected. Thereby, the screen shifts to the average height reference setting screen 210 of FIG. 90 and 91 show an example in which a 50-yen coin is used for the work for convenience of explanation.

  When setting the dynamic conversion, it is necessary to set an average reference height or the like for an image different from the distance image input during actual operation. For this reason, an image corresponding to the work at the time of operation is captured in advance and saved as a registered image. When setting dynamic conversion, the registered image is read and used as a work image at the time of operation. Various settings are made in the form. For this reason, the corresponding “registered image” is designated in the “display image” selection field 124 on the screen of FIG.

On the average height reference setting screen 210, a separately set inspection target area is used as it is, or an arbitrary average extraction area is designated as necessary. The average extraction area can be specified by any method such as a rectangular shape, four corners, a circle by specifying the center and radius, or a free curve. Further, only one point of the workpiece can be designated, or conversely, the entire workpiece or the entire image displayed in the second image display area 121 can be set as the average extraction area. Alternatively, the inspection target area separately designated as described above can be used as the average extraction area. In these cases, the designation operation of the average extraction area by the height extraction means may be omitted.
(Mask area)

  In addition, a mask area from which the average height is not extracted can be designated for the average extraction area. For example, when the “extraction area” button 194 provided in the operation area 122 is pressed from the screen of FIG. 90, the screen shifts to the mask area setting screen 220 shown in FIG. On the mask area setting screen 220, one or more mask areas that are unnecessary for extracting the average height can be designated. The mask area can also be designated by designating an arbitrary area such as a rectangular shape or a circular shape from the second image display area 121 as described above.

  Furthermore, gain adjustment or the like can be performed as necessary. For example, when the “detailed setting” button 196 provided in the lower right of the operation area 122 is pressed from the screen of FIG. 90, the screen shifts to the average height reference detailed setting screen 230 shown in FIG. In addition, detailed setting items such as extraction height designation and noise removal are displayed.

When the average extraction area is defined in this way, the setting screen is terminated. In the gradation conversion, gradation conversion is performed using the average value (average reference height) of the height information included in the average extraction area as the reference height. For example, the gradation conversion is performed so that the average reference height becomes the center value of the distance range (in the case of 2 ⁸ = 256 gradations, the center value of the distance range of 0 to 255 is 128). Further, it is not always necessary to use all the height information of all the points included in the average extraction region, and the processing can be simplified such as thinning out or averaging appropriately.

  During operation, dynamic conversion is performed according to the procedure shown in FIG. For example, a distance image is generated by imaging a workpiece conveyed on the line (step S 13301), the average height of the average extraction area set above is calculated (step S 13302), and gradation conversion is performed based on this. This is executed to generate a low gradation distance image (step S13303), and the obtained low gradation distance image is inspected (step S13304). With this method, even if there are variations in the height direction of the workpiece, the reference plane for gradation conversion can be reset for each workpiece every time, so accurate inspection is achieved regardless of variations in the workpiece height direction. it can.

Here, an example of a work in which the method of designating the reference surface based on the average height is effective will be described with reference to FIGS. 93A to 93B. FIG. 93A is a workpiece WK7 that does not affect the inspection process even if there is no planar inclination or some inclination on the workpiece measurement surface, as in FIG. 79A described above. Numbers and character strings are present on the surface of the casting. An inspection process is performed on the three-dimensionally formed workpiece WK7 to read whether the character string is appropriate by OCR. In such an application, a measurement region ROI for determining an average height reference in a rectangular shape is set on the second image display region 121 on the extraction region setting screen shown in FIG. 92 and the like. Here, as shown in FIG. 93A, the plane surrounding the character string on the upper surface of the work WK7 is designated as the measurement region ROI. Thereby, gradation conversion is performed using the height of the measurement region ROI as a reference plane, and the image is converted into a low gradation distance image shown in FIG. 93B. Even in this low gradation distance image, as in FIG. 79B, since the character string portion protruding from the plane of the work WK7 is clearly extracted, accurate OCR can be easily performed. Further, in FIG. 79A described above, only one point on the work WK7 is designated, so that the selected point may be affected by noise, whereas in FIG. 93A, such a noise is designated because it is designated by a plane. The advantage that can reduce the influence of is obtained.
(Plane basis)

  In the above, the example which performs dynamic conversion on the basis of average height was demonstrated. Next, as another dynamic conversion, a plane reference that estimates a plane included in a reference plane estimation area designated in advance for an input image and performs gradation conversion using the estimated plane as a reference plane will be described. In this method, for example, when the surface of the workpiece is inclined, gradation conversion can be performed by canceling the inclination component. Therefore, there is an advantage that it can be used similarly to the above-described three-point designation of static conversion. Hereinafter, a specific method for setting the plane reference will be described. Also in the plane reference, the reference plane estimation region for determining the reference plane is set in advance prior to the operation (step S83 in FIG. 8 described above) as in the above-described average height reference. Hereinafter, an example of a procedure for designating the reference plane estimation region in step S83 in FIG. 8 will be described based on the GUIs in FIGS. 62, 88, and 92 to 95. First, the “height extraction” button 116 is selected on the GUI screen shown in FIG. 62, and the process proceeds to the height extraction selection screen 140 shown in FIG. 80. The extraction method selection means 142 selects “real time extraction” corresponding to the dynamic conversion. Then, the height dynamic extraction setting screen 190 of FIG. 88 is displayed. Next, in the “calculation method” selection field 192, when “plane reference” is selected as the dynamic conversion reference as shown in FIG.

  In the plane reference setting screen, similarly to the height extraction unit in the height extraction selection screen 140 of FIG. 80 described above, the inspection target area set separately is used as it is, or an arbitrary reference plane estimation area is designated as necessary. . The reference plane estimation area can be specified by any method such as a rectangular shape, four corners, a circle by specifying the center and radius, a free curve, or the like. Further, only one point of the workpiece can be designated, or conversely, the entire workpiece or the entire image displayed in the second image display area 121 can be set as the reference plane estimation area.

  In addition, as in the above-described FIG. 90 and the like, it is possible to designate a mask area where the estimation plane is not estimated with respect to the reference plane estimation area. Further, as described above, gain adjustment, extraction height designation, noise removal, and the like can be performed as necessary. For example, when the “detailed setting” button 222 provided in the operation area 122 is pressed on the screen of FIG. 92, the plane reference detailed setting screen 240 of FIG. 94 is displayed, and in the enhancement method setting field 154, in addition to the gain adjustment field 156, An “extraction height” setting field 162 for designating the extraction height, a noise removal setting field 164 for noise removal, and an invalid pixel designating field 166 for designating invalid pixels are displayed, and these detailed settings can be made. Become. In addition, in the invalid pixel designation field 166, as shown in FIG. 95, invalid pixels whose distance cannot be obtained can be filled with a designated pixel value, a background pixel value, or an arbitrary value designated by the user. it can.

  When the reference plane estimation area is defined in this way, the setting screen is terminated. At the time of gradation conversion, a planar estimated surface is calculated from the height information included in this reference surface estimated region. For fitting the height information distributed in the reference plane estimation region, a known method such as a least square method can be used as appropriate. Note that it is not always necessary to use the height information of all points included in the reference plane estimation region for the calculation of the estimation plane, and it is possible to simplify processing such as thinning out or averaging as appropriate as described above. It is.

  When the estimated surface is determined in this way, gradation conversion is performed using this estimated surface as a reference. For example, gradation conversion is performed so that the estimated surface becomes the center value of the distance range. Moreover, the information of the calculated estimated surface can also be displayed. For example, in the example shown in FIG. 92, in the estimated surface display column provided in the operation area 122, the X direction inclination, the Y direction inclination, and the Z direction height of the estimated surface are displayed. In this example, the estimation plane is a single plane, but an estimation plane combining a plurality of planes may be used. In this example, the estimation surface is a plane, but the estimation surface can be calculated as a simple curved surface such as a spherical surface.

Moreover, as an example of a work in which the method of designating the reference plane based on the plane reference is effective, the above-described FIG. 86A and FIG. 87A can be cited. By setting the plane reference for such a workpiece, even if the upper surface of the workpiece is inclined, the inclination is corrected and the reference plane is detected. Pattern detection is realized.
(B3: Free curved surface standard)

  Finally, a free-form surface reference that generates a free-form surface from which a high-frequency component is removed from a predetermined region (free-form surface target region) of the input image and performs gradation conversion using the free-form surface as a reference surface will be described. For example, when approximation is difficult with a simple plane such as a workpiece having a curved surface, it is difficult to accurately extract height information of an area to be inspected. Therefore, by generating a simplified image by removing high-frequency components from the input image and using the surface shape (free-form surface) of this image as a reference surface, the rough shape and gentle changes are ignored, and abrupt changes are made. It is possible to inspect only the part where the change has occurred, that is, the fine shape.

  Hereinafter, a specific method for setting the free-form surface reference will be described based on the GUI shown in FIGS. Also in the free-form surface reference, the necessary conditions for determining the reference surface are set in advance prior to the operation (step S83 in FIG. 8 described above), as in the above-described average height reference. First, the “height extraction” button 116 is selected on the GUI screen shown in FIG. 62, and the process proceeds to the height extraction selection screen 140 shown in FIG. 80. The extraction method selection means 142 selects “real time extraction” corresponding to the dynamic conversion. Then, the height dynamic extraction setting screen 190 of FIG. 88 is displayed. Next, in the “calculation method” selection column 192, “free-form surface criterion” is selected as the dynamic conversion criterion as shown in FIG. This shifts to the free curved surface reference setting screen 250 of FIG.

Although it is possible to specify an arbitrary area as the free curved surface target area on the free curved surface reference setting screen 250, preferably, the entire image displayed in the second image display area 121 or an inspection target specified separately. The area is used as a free curved surface target area as it is. A free-form surface is generated by removing harmonic components from the area designated as the free-form surface target area. Then, a gradation-converted image that has been subjected to gradation conversion using the free-form surface as a reference plane is superimposed and displayed on the free-form surface target area displayed on the second image display area 121. Also, in the tone conversion, gain adjustment, extraction height designation, noise removal, and the like can be performed as necessary, as in FIG. 90 described above.
(Extraction size adjustment means)

  Furthermore, an extraction size adjustment function for adjusting the fineness (extraction size) of the extraction surface extracted based on the free-form surface is also provided. Specifically, in FIG. 96, an “extraction size” designation field 252 is provided as an extraction size adjustment means in the “detailed extraction surface setting” field of the operation field. When the numerical value in the “extraction size” designation field 252 is increased or decreased, the curvature of the free-form surface changes accordingly, and the size of the defect that can be extracted changes. Here, a free-form surface image is generated and displayed in the second image display area 121 so that unevenness having a size smaller than the set size is extracted. When the extraction size is increased, a smooth free-form surface is generated, and a defect having a size corresponding to the set extraction size can be extracted. On the other hand, if the extraction size is reduced, a free-form surface along the surface shape of the workpiece is generated, and only small defects corresponding to the set extraction size are extracted. For example, when the extraction size is increased, the free-form surface becomes smooth with respect to the surface shape of the workpiece as shown in FIG. 97, and the unevenness extracted by the smoothed reference surface becomes clear. On the contrary, if the numerical value is reduced, the free-form surface approaches the detailed shape along the unevenness of the surface shape of the workpiece as shown in FIG. 90. As a result, the unevenness extracted with such a complicated reference surface is unclear. Become. Further, when the numerical value in the “extraction size” designation field 252 is increased or decreased, the gradation conversion image in the free curved surface target area displayed in the second image display area 121 is generated internally as the reference plane accordingly. The state of the free-form surface changes, and the size of the object extracted and displayed as the difference from the reference surface changes in real time. The user can optimally adjust the numerical value in the “extraction size” designation field 252 while referring to the second image display area 121.

  When the free-form surface target area is defined in this way, the setting screen is terminated. At the time of gradation conversion, a free curved surface is calculated from the height information of the image included in the free curved surface target area. For fitting height information distributed in a free-form surface target area, a method of generating a free-form surface image by performing image reduction processing, filter processing, and image enlargement processing according to a set extraction size can be used. Alternatively, as described above, a known method such as a least square method can be used as appropriate. Note that it is not always necessary to use the height information of all points included in the free-form surface target area for the calculation of the estimation surface, and as described above, the processing can be simplified such as thinning out or averaging appropriately. When the free-form surface is determined in this way, gradation conversion is performed using the free-form surface as a reference. For example, gradation conversion is performed so that the free-form surface becomes the center value of the distance range.

Here, an example of a work in which the method of designating the reference surface as a free-form surface is effective will be described with reference to FIGS. 98A to 98B. FIG. 98A shows an inspection process for detecting defects such as protrusions and depressions included in the curved surface of the workpiece. Here, a region including a defect is designated from the curved surface as the measurement region ROI. Then, a free curved surface is obtained from height information included in the designated measurement region ROI, and gradation conversion is performed using the obtained free curved surface as a reference plane. The obtained low gradation distance image is shown in FIG. 98B. As shown in this figure, on the basis of a free-form surface, a portion higher than this surface is a projection (displayed as a white dot in FIG. 98B), and a lower portion is a depression (displayed as a black dot in FIG. 98B), respectively. It can be detected. As described above, the specification of the reference surface by the free-form surface can be effectively used for the workpiece having the curved surface shape. The processing load for detecting a free-form surface is higher than the one-point designation or the three-point designation described above.
(Extraction area setting dialog 148)

Further, when performing height extraction, the target area (extraction area) for calculating the reference plane from the input image is the same area as the inspection target area (measurement area) for performing the inspection process, and is separate from the measurement area. Can also be set. As an example, when an “extraction area” button 147 is pressed in FIG. 90, an extraction area setting dialog 148 capable of setting the extraction area is displayed as shown in FIG. The extraction area setting dialog 148 is provided with an extraction area selection field 149 from which the user can select “same as measurement area”, “rectangle”, “circular”, “rotation rectangle”, and the like. When “same as measurement region” is selected, the extraction region becomes the same as the measurement region as described above. On the other hand, by selecting other than “same as measurement area”, an area different from the measurement area can be set as the extraction area. For example, as shown in FIG. 100, when “rectangle” is selected in the extraction area selection field 149, a rectangular frame is displayed in the second image display area 121, and the user can specify a desired area by dragging the mouse or the like. When the “edit” button 324 provided on the right side of the extraction area selection field 149 in FIG. 100 is pressed, an extraction area editing dialog 326 is displayed as shown in FIG. 101, and the rectangular extraction area is numerically expressed in xy coordinates. Can be specified. When the extraction area is adjusted in the extraction area editing dialog 326, the change is reflected in the second image display area 121.
(Mask area setting field 330)

  Further, from the extraction area setting dialog 148, a mask area for designating an area not to be an extraction area can be set. In the example of FIG. 99, a mask area setting field 330 is provided below the extraction area selection field 149. In the mask area setting column 330, a plurality of mask areas can be set. Here, a maximum of four mask areas from 0 to 3 can be designated, and each mask area can be set independently. For example, as shown in FIG. 102, when “circular” is selected as the mask area 0, the circular mask area 0 is displayed on the second image display area 121. When the “edit” button 332 is further pressed from this state, a mask area edit dialog 334 for defining details of the circular mask area 0 is displayed as shown in FIG. The user can define the circular mask region 0 by the center xy coordinate and the radius. The mask area is displayed in the second image display area 121 with a frame line having a color different from that of the extraction area, so that the user can easily visually distinguish the extraction area and the mask area. In the example of FIG. 103, the extraction area is displayed in green and the mask area is displayed in yellow. However, the present invention is not limited to this example, and it is needless to say that it can be distinguished by different colors, line thickness, line type (solid line, broken line, etc.), highlight (flashing or emphasis), etc. .

In this way, the extraction area can be set independently of the measurement area. The setting contents of the extraction area can be displayed as character information. For example, in the example of FIG. 90, the text “Same as Measurement Area” is displayed below the “Extraction Area” button 147, indicating that the extraction area for calculating the plane reference is the same as the measurement area. In addition, “mask area: invalid” is displayed at the lower part thereof, indicating that no mask area is set in the extraction area. As a result, the user can check the outline of the “extraction area” as text information.
(Pre-processing setting)

As described above, when the setting of height extraction is completed in the “area” processing unit, in the third image display area in the rectangle of the area set by the “area” processing unit as shown in FIG. A low gradation distance image obtained by gradation conversion based on the extracted height is superimposed and displayed. Next, pre-processing is set. The preprocessing is common filter processing performed before generating a distance image as described above, and various filters can be selected here. More specifically, when a “pre-process” button provided in the setting item button area 112 is selected from the screen of FIG. 104, a filter processing setting screen 340 of FIG. 105 is displayed, and an applied filter can be selected. Examples of filters that can be selected include an averaging filter, a median filter, and a Gaussian filter. Further, in addition to such a filter, for example, a binarization level can be set. For example, on the binarization level setting screen 350 in FIG. 106, an upper limit value, a lower limit value, and the number of times for binarization can be set. Further, a histogram indicating the binarized pixel distribution may be displayed, or a function of updating the histogram in conjunction with the input image may be provided. When the filter processing setting is completed in this way, as shown in FIG. 107, in the third image display region, the low gradation distance image binarized through the filter processing within the set region is displayed in an overlapping manner. .
(Judgment setting)

Furthermore, the “area” processing unit converts the input image into a low gradation distance image according to the set region, height extraction, preprocessing, and other conditions, and then converts the low gradation distance image to a high gradation distance image. The conditions for performing determinations such as height inspection and image inspection are also specified. For example, when the “determination condition” button provided in the setting item button area 112 is pressed from the screen of FIG. 107, the determination condition setting screen 360 of FIG. 108 is displayed, and the determination condition is set. In this example, the pixels of the binarized low gradation distance image are counted, and when the numerical value is within a predetermined range, OK is set and when it is not set, NG is set. In the example of FIG. 108, since the determination condition is 0 to 30 and the current value is 166, it is determined as NG, and “determination result: NG” and red characters are displayed in the third image display area as shown in FIG. Is displayed. As described above, when the determination result is NG, a user can easily recognize the character at the time of operation by adopting a conspicuous aspect such as a red character.
("Blob" processing unit 267)

Further, the determination based on the height inspection using the height information has been described above. However, the present invention is not limited to this, and a determination process based on the image inspection on the conventional luminance image can be added. Such an image inspection is called a blob. For example, as shown in FIG. 110, a “blob” processing unit 267 for performing a blob (image inspection) is added below the “area” processing unit in the flow display area 261. Also in the “blob” processing unit 267, in the same manner as described above, the target region is set and the preprocessing is set (FIG. 111), the detection condition is set (FIG. 112), and the determination condition is set and the determination is made. The result can be output (FIG. 113).
("Color inspection" processing unit 267B)

  Furthermore, when a color optical image can be input, such as when a color CCD camera is connected as an imaging means, it is possible to combine color inspection. For example, as shown in FIG. 114, a “color inspection” processing unit 267B that performs color inspection as a measurement process is added below the “blob” processing unit 267 in the flow display area 261. Also in the “color inspection” processing unit 267B, in the same manner as described above, the target region is set (FIGS. 115 and 116), detailed settings such as density averaging are performed (FIG. 117), and determination conditions are set. The determination result can be output (FIG. 118).

As described above, after various settings are made in the setting mode, the workpiece is actually imaged in the operation mode to obtain an input image, and determination processing is performed based on the results of the height inspection and the image inspection. In the above example, the configuration is such that the determination result can be output even in the setting mode, so that the user can easily recognize the determination result in the setting stage. However, it goes without saying that the determination result can be output only in the operation mode.
(Operation mode)

Next, processing during operation in the head unit and controller unit in the 3D image processing apparatus shown in FIGS. 4A and 4B will be described based on the flowcharts in FIGS. 119 and 120. First, the flow of processing during operation of the head unit of the 3D image processing apparatus according to Embodiment 3 shown in FIGS. 4A and 5 will be described based on the flowchart of FIG.
(Processing flow according to Embodiment 3)

  First, when a trigger is input from the outside (step S11901), one projection pattern is projected from the first projector 20A onto the work (step S11902), and an image is picked up by the imaging means (step S11903). Next, it is determined whether imaging has been completed for all the light projection patterns (step S11904). If not yet, the light projection patterns are switched (step S11905), and the process returns to step S11902 to repeat the process. Here, a total of 16 pattern projection images are captured, including eight pattern projection images with a projection pattern using the phase shift method and eight pattern projection images with a projection pattern using the spatial encoding method. .

  On the other hand, if all the light projection patterns have been imaged in step S11904, the process branches to step S11906 and step S11907. First, in step S11906, a three-dimensional measurement calculation is executed to generate a distance image A.

  On the other hand, in step S11907, an average image A ′ obtained by averaging the plurality of pattern projection images (pattern projection image group) captured by the phase shift method is calculated.

Steps S11902 to S11906 described above are three-dimensional measurement by pattern projection from the first projector 20A. Next, three-dimensional measurement is performed by pattern projection from the second projector 20B. Here, following step S11906, in step S11908, similarly to steps S11902 to S11905, the projection pattern is projected from the second projector 20B onto the workpiece (step S11908), and is imaged by the imaging means (step S11909). It is determined whether or not imaging has been completed for all the light projection patterns (step S11910). If not, the projection pattern is switched (step S11911) and the process returns to step S11902 to repeat the process. On the other hand, if all the projection patterns have been imaged, the process branches to step S11912 and step S11912. In step S11912, a three-dimensional measurement calculation is performed to generate a distance image B. On the other hand, in step S11913, an average image B ′ of the pattern projection image group captured by the phase shift method is calculated. When the three-dimensional distance images A and B are generated in this way, in step S11914, the three-dimensional distance images A and B are combined to generate a distance image. In step S11915, the average images A ′ and B ′ are used to generate a luminance image (average two-dimensional grayscale image) obtained by combining these images. In this manner, the distance image having the workpiece height information is acquired in the three-dimensional image processing apparatus of FIG. If a luminance image is not necessary, steps S11907, S11913, and S11915 can be omitted.
(Processing flow of Embodiment 4)

The processing flow of the head unit of the 3D image processing apparatus according to Embodiment 3 illustrated in FIG. 4A has been described above. Next, the processing flow of the head unit of the 3D image processing apparatus according to Embodiment 4 shown in FIG. 4B will be described based on the flowchart of FIG. explain. First, when a trigger is input from the outside (step S12001), one light projection pattern is projected onto the work from the light projecting unit 20 (step S12002), and the first imaging unit 10A captures an image (step S12003). At the same time, the second image pickup means 10B picks up an image (step S12004). Then, it is determined whether imaging has been completed for all the light projection patterns (step S12005). If not, the projection pattern is switched (step S12006) and the process returns to step S12002 to repeat the process. On the other hand, if all the projection patterns have been imaged, the process branches to steps S12007 to S12010. In step S12007, a three-dimensional measurement calculation is executed to generate a distance image A. In step S12008, a three-dimensional measurement calculation is executed to generate a distance image B. On the other hand, in step S12009, an average image A ′ of the pattern projection image group imaged by the phase shift method is calculated, and in step S12010, an average image B ′ of the pattern projection image group imaged by the phase shift method is calculated. In step S12011, the three-dimensional distance images A and B obtained in steps S12007 and S12008 are combined to generate a distance image. Also, a luminance image is generated by combining the average images A ′ and B ′ obtained in steps S12009 and S10. According to this method, two images can be captured simultaneously, so that the imaging time can be shortened.
(Inspection area setting means)

  At the time of actual operation inspection, it is necessary to designate in advance an area (inspection target area) to be inspected by the inspection execution means 50 for the workpiece. Such setting of the inspection target area is performed by the inspection target area setting means in the setting stage. The inspection target area setting means can be provided on the controller unit 2 side as described above, or can be realized by a three-dimensional image processing program. Specifically, as described above, when the “area setting” button 115 corresponding to the inspection area setting means of the 3D image processing program shown in FIG. 62 is pressed, a transition is made to the inspection area setting screen 120 shown in FIG. On the inspection area setting screen 120, an area to be inspected can be designated.

  When the inspection target area is designated in this way, the three-dimensional image processing apparatus performs image processing on the inspection target area, and further executes the inspection. That is, as shown in the flowchart of FIG. 121, an inspection target area is assigned to the distance image (step S12101), and a gradation conversion parameter is set based on the distance image (step S12102). Tone conversion is performed (step S12103), image processing is further performed on the gradation-converted image, and a predetermined inspection is performed (step S12104).

Note that the gradation conversion process is performed on the inspection target area set by the inspection target area setting means described above. That is, in this example, the inspection target area setting means is common to the gradation conversion target area specifying means for specifying the gradation conversion target area. However, the area used for determining the parameters of the gradation conversion process may be set independently of the inspection target area. For example, the gradation conversion parameter creation region is designated using the inspection target region setting means or the gradation conversion parameter creation region designating means prepared separately.
(Operation flow of the controller during operation)

Next, a procedure for processing the distance image obtained on the head unit side on the controller unit side during operation will be described based on the flowchart in FIG. 122 and the GUI in FIGS. 123 to 127. 123 to 126 show the GUI of the three-dimensional image processing program. The GUI of the three-dimensional image processing program shown in FIG. 123 shows an initial screen 260, and a flow display area 261 is provided on the left side of the screen, and a third image display area 262 is provided on the right side. In the flow display area 261, a flow diagram in which the contents of each process performed by the 3D image processing apparatus are connected in a processing unit shape is displayed. Here, an “imaging” processing unit 263, a “Shapetrax2” processing unit 264, a “position correction” processing unit 265, and a “height measurement” processing unit 266 are displayed in the flow display area 261 as processing units. Further, at the stage of setting before operation, detailed setting can be performed by selecting each processing unit. The third image display area 262 displays a luminance image, a distance image, an inspection result, or the like according to the processing content. In the example of FIG. 123, a luminance image obtained by imaging a workpiece (IC in this example) is displayed, and a search target area SA described later is displayed in a green frame shape.
(Imaging step)

  First, in step S 12201 in FIG. 122, a distance image and a luminance image are acquired from the head unit 1. Here, a pattern projection image is captured on the head unit side, and a distance image and a luminance image are generated. In the example of the GUI in FIG. 123, the “imaging” processing unit 263 displayed in the flow display area 261 corresponds.

For the image data, the distance image is first transmitted from the head unit to the controller unit, and then the luminance image is also transmitted to the controller unit. Conversely, the distance image may be transferred after the luminance image is transferred first, or these transfer may be performed simultaneously.
(Search step)

In step S 12202, the controller unit performs a pattern search on the input image input during operation. That is, a part to be inspected is specified so as to follow the movement of the work included in the captured luminance image. In the example of FIG. 123, the “Shapetrax2” processing unit 264 displayed in the flow display area 261 corresponds to the pattern search. Here, the image search means 64 in FIG. 5 performs a pattern search on the luminance image (input image) obtained by imaging the workpiece to perform positioning. Specifically, the position where the preset inspection target region is included in the input luminance image is specified. The search target area SA for performing the pattern search is set in advance. For example, in the example of FIG. 124, the search target area SA is designated as a rectangle on the luminance image displayed in the third image display area.
(Position correction step)

In step S12203, the position of the inspection target area is corrected using the pattern search result. In the example of FIG. 123, the “position correction” processing unit 265 displayed in the flow display area 261 corresponds. The inspection target area is set in advance on the inspection target area setting screen. In the example of FIG. 125, a plurality of inspection target areas are set on the distance image displayed in the third image display area. Specifically, as shown in FIG. 126, a rectangular area is set for each pin of the IC as a work. The position correction is performed by, for example, a method for calculating a positional deviation amount by a normalized correlation search, a method based on the result of pattern search, or the like. In this way, the position correction corrects the position of the inspection target area in the inspection process executed in the next stage.
(Inspection processing step)

  Finally, in step S12204, an inspection is executed using the distance image at the corrected position. In the example of FIG. 123, the “height measurement” processing unit 266 displayed in the flow display area 261 corresponds. In the example of FIG. 127, height measurement is performed. That is, the height is measured and inspected in each of the inspection target areas at the corrected positions. For example, it is determined whether the plane height is within a predetermined reference value, and the determination result is output.

In the above procedure, the example in which the position is corrected using the luminance image and the inspection is performed based on the distance image has been described. However, the present invention is not limited to this, and an image for position correction and an image for inspection processing can be arbitrarily set. For example, contrary to the above, after performing position correction using the distance image, the inspection process can also be executed using the luminance image. As an example, in an example where it is difficult to perform an accurate pattern search with a luminance image, such as when a white workpiece is placed on a white background, a pattern search based on height information using a distance image is effective. Become. Further, the inspection process can be determined not only based on the height information, but also based on the result of image processing using a luminance image such as reading a character string printed on a work by OCR.
(Height information output format)

The measured three-dimensional height information is obtained as three-dimensional point cloud data having X, Y, and Z values, respectively. In addition to the three-dimensional point cloud data, it is also possible to convert, for example, how to output the actually obtained values into, for example, a Z image or an XY pitch Z image.
(1: Z image)

A Z image is height image data of only the Z coordinate. For example, when the unevenness of the position of the workpiece imaged by the imaging means is important and the X and Y coordinates need not be accurate, the X and Y coordinates are unnecessary, so a Z image that is only data of the Z coordinates is used. Output is enough. In this case, the amount of data to be transmitted is reduced, and the transmission time can be shortened. In addition, since data can be handled as an image as in a normal two-dimensional imaging means, image processing can be performed using an existing image processing apparatus for two-dimensional images.
(2: XY-equal pitch Z image)

  The XY equal pitch Z image is height image data in which the XY coordinates are equal pitches regardless of the height. Specifically, an XY equi-pitch Z image is obtained by interpolating the Z-coordinate at a position where the XY coordinates are equal pitches from surrounding point cloud data.

In general, when the lens of the imaging means is not an objective telecentric lens, the position (X, Y coordinates) imaged by the imaging means differs depending on the height position (Z coordinate) of the imaged work. For this reason, the actual XY coordinate position differs depending on the height even if the object appears at the same position on the image sensor. For example, when it is desired to inspect a three-dimensional difference such as a volume, the value becomes large if there is a workpiece close to the camera, and the value becomes small if it is far away. Therefore, by obtaining Z data having an XY uniform pitch from the point cloud data, a Z image having an XY position that is not affected by the height can be obtained.
(3: XYZ (point cloud data))

Alternatively, the point cloud data can be output as it is as three-dimensional information. For example, it is used when the measured three-dimensional data is to be handled as it is. In this case, the data amount is three times that of the case of only the Z coordinate, but since it is raw data, it can be used for purposes such as obtaining a three-dimensional difference from the three-dimensional CAD data.
(Generation of equal pitch images)

Next, FIG. 128 shows a data flow diagram for creating an equal pitch image in which the angle of view is corrected in addition to the distance image and the luminance image. As shown in this figure, a spatial code image is first generated from a spatial encoding pattern projecting work image group imaged by the head unit according to a spatial encoding method. On the other hand, a phase image is generated from the phase shift pattern projecting work image group according to the phase calculation. Then, phase expansion calculation is performed from these spatial code image and phase image to generate an extended phase image. Further, it is possible to apply a common filter process such as applying a filter to these spatial encoding pattern projecting work image groups and phase shift pattern projecting work image groups. Common filter processing includes application of a two-dimensional filter such as a median filter, a Gaussian filter, or an averaging filter. On the other hand, a luminance image (average gray image) is generated by averaging the phase shift pattern projecting work image group.
(Equal interval processing)

  As described above, after the three-dimensional data and the luminance image are generated by the head unit, a distance image such as a Z image or an equal pitch Z image is further created. First, the distance image is transferred from the head unit 1 to the controller unit 2 to perform gradation conversion for converting phase information into height information. Here, after obtaining the X image, Y image, and Z image from the phase information, the XY is equalized to obtain an XY equal pitch Z image and an XY equal pitch Z average image on the XY plane. Such an equal interval process is performed by the interval equalization process setting means 47.

  In the example of FIG. 128, the example in which the phase shift method and the spatial encoding method are combined in the generation of the distance image has been described. However, the distance image can be generated only by the phase shift method without using the spatial encoding method. . Spatial encoding processing can be switched ON / OFF by the spatial encoding switching means 45 shown in FIG. Such an example is shown in the data flow diagram of FIG. As shown in this figure, the number of images can be reduced by not using the spatial coding method, and a distance image can be generated at high speed.

  On the other hand, FIG. 130 shows a data flow diagram when obtaining the Z image by turning off the pitching function such as XY. In this example, there is no need to decompose the image into an X image or a Y image at the time of gradation conversion, so that the processing can be simplified accordingly.

Further, FIG. 131 shows an example of outputting point cloud data that outputs XYZ coordinate information as it is. In this example, since the X image, Y image, and Z image after phase-to-height conversion can be output as they are, the advantage that they can be output with a light load can be obtained.
(Tone conversion method)

Next, a procedure in which the gradation converting means 46 of the three-dimensional image processing apparatus automatically performs gradation conversion from a high gradation distance image to a low gradation low gradation distance image based on the distance image will be described. . Here, in an inspection apparatus installed on a line on which a plurality of workpieces are conveyed, a range image (input image) that is sequentially input, such as an application that performs tone conversion to a low tone distance image in real time, is used. A procedure for performing gradation conversion on the input image will be described. The gradation conversion processing in this case is roughly divided into (A) a method for determining gradation conversion parameters in advance (static conversion), and (B) a method for determining gradation conversion parameters according to the input image. (Dynamic method) can be mentioned. Hereinafter, these will be described.
(A: Static conversion)

First, static conversion in which gradation conversion parameters are determined in advance will be described. Here, the gradation conversion parameters for performing gradation conversion are adjusted for the input image and the registered image registered in advance at the time of setting. During operation, the distance image is subjected to gradation conversion using the gradation conversion parameter set at the time of setting, and the low gradation distance image after gradation conversion is inspected. The procedure at the time of setting is as described based on the flowchart of FIG.
(Procedure for operation)

When the tone conversion parameter is adjusted, tone conversion is performed on an input image input during operation with the tone conversion parameter. Here, the procedure of static conversion during operation will be described based on the flowchart of FIG. First, in step S 13201, a distance image is acquired. Here, the controller unit 2 captures the distance image generated by the distance image generation means 32. In step S 13202, the input distance image is subjected to gradation conversion processing. Here, gradation conversion processing is executed according to the gradation conversion parameter adjusted at the time of setting, and a low gradation distance image in which the number of gradations of the distance image, that is, the dynamic range is reduced, is generated. Finally, inspection processing is executed by the inspection execution means 50 in step S13203. According to this method, by setting gradation conversion parameters in advance, it is not necessary to calculate gradation conversion parameters during operation, and the processing can be lightly loaded.
(B: Dynamic conversion)

  Next, dynamic conversion for calculating gradation conversion parameters at the time of gradation conversion based on an input image will be described. First, the setting procedure is performed according to the flowchart of FIG. 8 as described above. Specifically, an input image or a registered image is acquired in step S81, and then a gradation conversion method is selected in step S82. Here, it is assumed that the user has selected dynamic conversion. In step S83, the tone conversion parameter is adjusted. Here, the conditions under which the gradation conversion parameter is calculated or adjusted for the input image input during operation are set based on the image acquired in step S81.

  When the gradation conversion parameter calculation conditions are set in this manner, the gradation conversion parameters corresponding to the input image are individually calculated according to the set gradation conversion parameter calculation conditions during operation. Next, a procedure at the time of operation for converting a distance image into a low gradation distance image by dynamic conversion will be described based on a flowchart of FIG. First, in step S13301, a distance image is acquired. Here again, as in step S 13201 described above, the controller unit 2 captures the distance image generated by the distance image generation means 32. In step S 13302, tone conversion parameters are determined based on the distance image that is the input image. As a method for adjusting the gradation conversion parameter, the method described above can be used. In step S13303, gradation conversion is executed. Finally, inspection processing is executed in step S13304. According to this method, since the gradation conversion parameter can be changed according to the input image, it is possible to perform an accurate inspection by flexibly performing gradation conversion even for different workpieces. For example, the inspection of the workpiece surface with variations in height can be performed without reducing accuracy.

Here, as an example of dynamic conversion, a method of optimally creating a low gradation distance image after gradation conversion by adjusting gradation conversion parameters will be described with reference to FIG. Here, there are two methods: a method of dynamically setting one gradation conversion parameter set and a method of preparing a plurality of gradation conversion parameter sets in advance and dynamically selecting the gradation conversion parameter set. .
(Method for setting one gradation conversion parameter set for 1A)

  First, a method for dynamically setting one gradation conversion parameter set will be described. In this method, the value of the gradation conversion parameter used for gradation conversion is adjusted based on image information of a predetermined area of a plurality of distance images that are input images, and the adjusted gradation conversion parameter is adjusted. The gradation conversion means 46 executes the distance image gradation conversion process. Here, an example in which gradation conversion is performed on a 16-gradation distance image (image before gradation conversion) to an 8-gradation low gradation distance image (image after gradation conversion) will be described. Further, as shown in the perspective view of FIG. 134, the workpiece to be inspected is such that the entire measurement surface for each workpiece is shifted up and down, and the range is 5 mm. The range in the height direction including the thickness and distortion of the entire measurement surface is 0.5 mm. Consider an example in which each workpiece having different surface heights is inspected by image processing to determine whether or not the surface has a flaw. By using the distance image and converting the gradation of the distance image into a two-dimensional grayscale image (low gradation distance image), it is possible to inspect with an existing image processing apparatus for a two-dimensional image.

  First, an area to be inspected on the workpiece is designated in advance as an inspection target area by the inspection target area setting means. In addition to specifying a part of the distance image that is the input image, the inspection target region may be the entire distance image. In this case, the work of specifying the inspection target area may be omitted.

  First, the average distance of the inspection target area is obtained. Next, a difference (distance range) between the maximum distance and the minimum distance of the inspection target area is obtained. Further, a numerical value obtained by multiplying the distance range by 1.2 is set as the distance range of the converted image. For example, in the work distribution of FIG. 134, assuming that the distance range is 0.5 mm, 0.6 mm obtained by multiplying the distance range by 1.2 is set as the distance range of the image after gradation conversion. Therefore, the range of ± 0.3 mm around the average distance is the measurement range.

  Furthermore, the span for the distance image that is the input image is obtained so that the range of 0.6 mm (± 0.3 mm centered on the average distance) that is the distance range of the image after gradation conversion is 256 gradations. The span can be a predetermined constant. For the distance image, an average distance of −0.3 mm or less may be 0, while an average distance of +0.3 mm or more may be 255.

In this way, a range necessary for inspection is determined from a plurality of distance images that are input, and the height information of this range is appropriately maintained in the low gradation distance image after gradation conversion. Tone conversion parameters can be set. As a result, it is possible to appropriately inspect the presence / absence and position of each workpiece distributed in the height direction using an existing image processing apparatus.
(1B Method of preparing a plurality of tone conversion parameter sets in advance)

  In the above-described embodiment, a method has been described in which the tone conversion parameter set is appropriately obtained by using the acquired height information of the distance image so as not to lose the height information necessary for detection of scratches. On the other hand, a method for preparing a plurality of tone conversion parameter sets in advance will also be described below. In this method, a gradation-converted image obtained by gradation conversion using a plurality of preset gradation conversion parameters is created for a plurality of distance images that are input images, and a predetermined inspection of the distance image is performed. Based on the image information of the target area, an image after gradation conversion to be used for inspection is selected.

Such a specific example will be described with reference to the schematic diagram of FIG. 135 and the flowchart of FIG. In this example as well, as in FIG. 134 described above, a case is considered where the entire measurement surface for each workpiece is shifted up and down within a range of 5 mm. For such a workpiece, nine gradation-converted images are created by converting the gradation by changing the conversion center every 0.5 mm. Then, the obtained gradation-converted images are preferably displayed side by side on the display means, and the user is allowed to select a converted image having a conversion center closest to the average value of the distance images as the input image. Then, a gradation conversion parameter set applied to the image selected by the user is set. As shown in FIG. 136, a specific procedure is as follows. First, a distance image of a workpiece is generated (step S13601), and gradation conversion processing is performed a plurality of times while changing the gradation conversion parameter as described above (step S13602). . Then, the user selects a desired image from the simple low gradation distance image that is the obtained gradation conversion image (step S13603), adjusts the gradation conversion parameter as necessary, and obtains the low An inspection is performed on the gradation distance image (step S13604).
(Details of static conversion and dynamic conversion)

Next, details of these static conversion and dynamic conversion will be described. First, static conversion will be described. As a specific method for static conversion, in order to correct the reference of height information to be left when converting a distance image to a low gradation distance image,
(A1) One point specification to be corrected at the specified height (distance);
(A2) Three-point designation for correction on a plane can be used.
(A1: One point designation)

The one-point designation is a method for converting the gradation of a distance image into a low gradation distance image based on the height (distance) of a point or region designated by the user. The reference height is, for example, the intermediate height of the height range (distance range) to be converted into the low gradation distance image in the height information of the distance image. Alternatively, the upper limit (the highest position where gradation conversion is performed) and the lower limit (the lowest position where gradation conversion is performed) of the distance range can be set. The specific procedure for specifying one point is as described above with reference to FIGS.
(A2: Three point designation)

  The three-point designation is a method for gradation-converting a distance image into a low gradation distance image using a plane obtained from the three points designated by the user as a reference plane. The reference plane is also set to a height in the middle of the height range (distance range) to be converted to a low gradation distance image, for example, among the height information of the distance image, similarly to the reference height designated by one point described above. Alternatively, the upper limit (the highest position where gradation conversion is performed) and the lower limit (the lowest position where gradation conversion is performed) of the distance range can be set. A specific example of the three-point designation is as described based on the GUI screens of FIGS.

  Here, an example in which designation of the reference plane by static conversion is effective will be described based on FIG. In this example, the workpiece WK7 having the same height is conveyed on the same surface by the belt conveyor BC. In this case, since there is almost no variation in the height of the workpiece WK7, dynamic conversion for changing the reference plane according to the workpiece is unnecessary, and can be handled by static conversion. In particular, static conversion is faster than dynamic conversion, and can enjoy the advantages of static conversion.

FIG. 138A is an example of a workpiece for which the height fluctuation itself is to be suppressed. In this example, an inspection process for detecting a case where the bottle cap, which is the work WK10 conveyed by the belt conveyor BC, is “floating” as an abnormality is targeted. In this case, by adopting static conversion, it is possible to detect a variation in height as in the low gradation distance image shown in FIG. 138B. On the other hand, when dynamic conversion is adopted, the fluctuation is corrected, so that the abnormality cannot be detected. Therefore, static conversion can be preferably used in such applications (B: specific example of dynamic conversion).

In the above, static conversion in which gradation conversion conditions are specified in advance at the setting stage and gradation conversion is performed under the specified conditions during operation has been described. Next, a specific example of the dynamic conversion that adjusts the gradation conversion condition according to the input image to be inspected will be described. First, in dynamic conversion, as a specific method for correcting the reference of height information to be left when gradation conversion from a distance image to a low gradation distance image,
(B1) An average height reference for gradation conversion using an average height (average distance) in an average extraction area designated for the input image as an average reference height;
(B2) a plane reference for generating an estimated plane in the designated area of the input image and performing gradation conversion using this as a reference plane;
(B3) A free-form surface reference that generates a free-form surface from which high-frequency components are removed from an input image and performs gradation conversion using this as a reference surface is included.

  The average extraction area for defining the average reference height is set in advance prior to operation (step S83 in FIG. 8 described above). An example of the procedure for designating the average extraction region in step S83 in FIG. 8 is as described based on the GUI in FIGS. 88 to 92. On the other hand, during operation, dynamic conversion is performed according to the procedure described with reference to FIG. For example, a distance image is generated by imaging a workpiece conveyed on the line (step S 13301), the average height of the average extraction area set above is calculated (step S 13302), and gradation conversion is performed based on this. This is executed to generate a low gradation distance image (step S13303), and the obtained low gradation distance image is inspected (step S13304). With this method, even if there are variations in the height direction of the workpiece, the reference plane for gradation conversion can be reset for each workpiece every time, so accurate inspection is achieved regardless of variations in the workpiece height direction. it can.

  Next, also in the plane reference, the reference plane estimation region for determining the reference plane is set in advance prior to the operation (step S83 in FIG. 8 described above) in the same manner as the above-described average height reference. An example of the procedure for designating the reference plane estimation region in step S83 in FIG. 8 is as described based on the GUI in FIGS. 88 and 92 to 95. During operation, dynamic conversion is performed according to the procedure described with reference to FIG. For example, the workpiece conveyed on the line is imaged to generate a distance image (step S13301), the reference plane estimation area set as described above is extracted, the estimation plane is calculated (step S13302), and the obtained estimation is obtained. Gradation conversion is performed with reference to the plane to generate a low gradation distance image (step S13303), and the obtained low gradation distance image is inspected (step S13304). With this method, even if the workpiece surface has an inclination or the like, the inspection can be canceled and an accurate inspection can be realized regardless of the inclination of the workpiece.

  Finally, the specific method for setting the free-form surface reference is as described based on the GUIs in FIGS. 126 to 32. Also in the free-form surface reference, the necessary conditions for determining the reference surface are set in advance prior to the operation (step S83 in FIG. 8 described above), as in the above-described average height reference. In operation, dynamic conversion is performed according to the procedure described in FIG. For example, the workpiece conveyed on the line is imaged to generate a distance image (step S 13301), a free curved surface is calculated for the free curved surface target area set above (step S 13302), and the obtained free curved surface Is used as a reference to generate gradation conversion to generate a low gradation distance image (step S13303), and the obtained low gradation distance image is inspected (step S13304). With this method, there is an advantage that an operation that is difficult to perform an accurate inspection by a conventional method such as a surface inspection of a curved workpiece can be performed with high accuracy.

  Here, an example in which the specification of the reference plane by dynamic conversion is effective will be described based on FIG. In this example, workpieces WK11 having different heights are conveyed on the same surface by the belt conveyor BC. In this case, since the height of the workpiece WK11 differs individually, the average height reference or the like is used, the reference plane is changed according to the height of each individual, and the gradation conversion is performed to change the workpiece WK11. Even if there is, there is an optimum gradation conversion.

  FIG. 140 shows an example of the work WK12 having a minute inclined surface and a dent on a plane. In this example, since there is a minute inclined surface on the surface of the workpiece WK12 due to the dent DE, the inclination may affect the detection accuracy of the dent. Therefore, by using a plane reference or the like and dynamically obtaining a plane for each workpiece and converting the gradation as a reference plane, a highly accurate inspection such as detection of a slight depression or dent is possible.

Further, FIG. 141 shows an example of a curved workpiece WK13 having different radii. Also in this example, by using a free curved surface reference or the like, a curved surface is obtained for each workpiece, and gradation conversion is performed using the curved surface as a reference surface, thereby enabling an inspection with reduced influence of shape variation for each individual.
(Automatic adjustment of gradation conversion parameters)

  The procedure for manually setting the gradation conversion parameter based on the image image after gradation conversion has been described above. On the other hand, when the tone conversion parameter is set manually, if tone conversion is performed using the initially set tone conversion parameter, an image suitable for inspection may not be obtained due to changes in the work or environment. In such a case, the image conversion suitable for the inspection can be performed by correcting the gradation conversion parameter using the data after the gradation conversion without performing fine adjustment with reference to the image and performing the conversion again. It can also be done. In this method, after initial setting of gradation conversion, gradation conversion is first performed using an arbitrary gradation conversion parameter as an initial value, and then the gradation conversion parameter is adjusted.

  For example, the gradation conversion condition automatic setting means can function as a gradation conversion condition automatic setting means and a gradation conversion condition manual setting means. That is, the gradation conversion condition automatic setting means sets a simple gradation conversion condition when the gradation conversion means converts the distance image into a low gradation distance image. Further, the gradation conversion condition manual setting means is configured to display the simple low gradation distance image subjected to gradation conversion on the display means based on the simple gradation conversion condition set by the gradation conversion condition automatic setting means. The manual adjustment of the gradation conversion condition is accepted. As a result, instead of prompting the user to set the gradation conversion condition suddenly, the provisional simple gradation conversion condition is automatically set and the gradation conversion is performed, and then the obtained one or more simple conversions are performed. Since it is possible to set the desired gradation conversion conditions while referring to the low gradation distance image, even if the user is not familiar with the meaning of the gradation conversion parameter or is unfamiliar with the setting, it is automated to some extent. It is possible to facilitate the setting work.

  A specific procedure will be described based on the flowchart of FIG. First, in step S14201, a distance image creation process is executed. Next, in step S14202, initial gradation conversion processing is performed using the initial value of the gradation conversion parameter. Further, in step S14203, it is determined whether or not the obtained gradation conversion image is appropriate. If not, the gradation conversion parameter is adjusted again in step S14204, and then the process returns to step S14202 to repeat the process. On the other hand, if it is determined in step S14203 that an appropriate gradation-converted image has been obtained, the process advances to step S14205 to execute a predetermined inspection.

  In the above method, whether or not the gradation conversion parameter is appropriate is determined in step S14203, but this procedure may be omitted. FIG. 143 shows a procedure in this case. Each procedure is almost the same as the example of FIG. 142 described above. In step S14301, a distance image creation process is executed, and then in step S14302, an initial gradation conversion process is executed. Then, after adjusting the gradation conversion parameter in step S14303, a predetermined inspection is performed in step S14304.

  As an initial gradation conversion method, for example, a method of applying an input image to a conversion function f (x, y, z), a method of applying shift and span to an input image, and compressing the result to n gradations, or an arbitrary method Using the plane as the reference plane, the difference between the input image and the reference plane is taken, shifted and spanned, and the result is compressed to n gradations, or the difference between the input image and the reference image is taken and shifted and spanned. And a method of compressing the result to n gradations.

Next, a specific example of automatically adjusting the tone conversion parameters after tone conversion will be described. As the automatic adjustment method of the gradation conversion parameter, (C1) a method using the median of the histogram of the distance image data after conversion, (C2) a method using the maximum value and minimum value of the histogram, and (C3) C1 and C2 Combinations are possible.
(C1: Method using the median value of the histogram)

First, a method using the median value of the histogram will be described based on the flowchart of FIG. First, in step S14401, a histogram of the converted distance image data is calculated. Next, in step S14402, the median value of the histogram is obtained. In step S14403, the tone conversion parameter is changed so that the median value becomes a predetermined value. The tone conversion process is executed again using the changed tone conversion parameter. Note that an average value of 2n + 1 including n before and after the center may be used as the median. The maximum value and the minimum value may be the median values after removing n from the larger one and m from the smaller one, respectively.
(C2: Method using maximum and minimum values of histogram)

First, a histogram of distance image data after conversion is calculated. Next, the maximum value and the minimum value of the histogram are obtained, and the gradation conversion parameter is changed so that the maximum value-minimum value width becomes a predetermined value. Then, the gradation conversion process is executed again with the changed gradation conversion parameter. Here, the maximum value and the minimum value may be average values of n from the larger one and m from the smaller, respectively. Alternatively, the maximum value and the minimum value may be the maximum value and the minimum value after removing n from the larger one and m from the smaller one, respectively.
(C3: Combination of C1 and C2)

  You may combine said C1 and C2. That is, after the histogram is calculated, the gradation conversion parameter is changed based on the median value and the width between the maximum value and the minimum value, and the gradation conversion is performed again.

  In the above method, a low-pass filter can be applied to the histogram in advance. Alternatively, a histogram may be obtained after applying a low-pass filter to the converted distance image in advance.

By such gradation conversion, a distance image having a high gradation number including height information can be converted into a low gradation distance image having a low gradation. Since this low gradation distance image can be processed as a two-dimensional image, the image processing apparatus corresponding to the existing two-dimensional image can also handle the low gradation distance image. For example, height information of each pixel included in a distance image obtained by imaging a workpiece for inspecting the presence or absence of a flaw is expressed as a binary value in hexadecimal notation as a gray value. Here, when the difference between the distance image and the reference distance image is calculated, the defect information of the shallow and small scratch appearing on the surface of the work is collected in the lower eight gradations. For this reason, the information amount of the difference image can be greatly compressed while preventing a decrease in detection accuracy by reducing the upper eight gradations, for example, by the gradation conversion means. In this way, the gradation conversion means reduces the upper half of the gradation when the gradation value of each pixel of the difference image is expressed as a gradation, thereby preventing a reduction in detection accuracy and reducing the difference in the detection accuracy. The amount of information can be greatly compressed.
(Partial execution of gradation conversion processing)

Further, the gradation conversion process can be performed only on a part of the distance image, not on all of the distance images. Specifically, the gradation conversion means executes the gradation conversion process only for the designated inspection target area in the distance image. Thereby, the gradation conversion processing is reduced, and the processing load is reduced and the processing speed is increased. As an example, a height inspection process (first inspection process) and an image inspection process (second inspection process) are combined using a workpiece WK14 having a shape in which cylinders with different diameters are overlapped in a triple manner as shown in FIG. 145A. Consider a case where a plurality of inspection processes are performed. Here, as the height inspection process, the height of each cylindrical surface of the workpiece WK14 is measured, and as the image inspection process, chipping or cracking is detected in the cylindrical portion from the top to the second stage of the workpiece WK14. Perform an appearance inspection. The inspection process is selected by an inspection process selection unit. Further, specific setting for the inspection process selected by the inspection process selection means is performed by the inspection process setting means.
(Inspection processing selection means)

The inspection process selection means is a means for selecting a plurality of inspection processes executed by the inspection execution means for the distance image. Here, as shown in FIG. 44, FIG. 56, etc., an inspection process is selected from the initial screen 260 when a processing unit is added. Specifically, a desired inspection process is selected from a list of inspection processes displayed by selecting a “measurement” process from the “add” submenu of the processing unit. In this example, “area”, “pattern search”, “Shapetrax 2”, “edge position”, “edge width”, “edge pitch”, “edge angle”, “pair edge”, “scratch”, “blob”, “ “Tint Blob”, “Trend Edge Position”, “Trend Edge Width”, “Trend Edge Defect”, “Tint Inspection”, “Color Inspection”, “OCR”, “2D Code Reader”, “1D Code Reader”, “High The desired inspection process is selected from “Measurement”.
(Inspection processing setting means)

On the other hand, the inspection process setting means sets details of each inspection process selected by the inspection process selection means. Here, as shown in FIGS. 46, 63, 78, etc., each setting item is individually set from each button arranged in the setting item button area 112. In this example, specific setting items are arranged as setting item buttons. For example, an “image registration” button 113, an “image setting” button 114, an “area setting” button 115, a “height extraction” button 116, “ A “preprocessing” button 117, a “detection condition” button 118, a “detailed setting” button 119, a “judgment condition” button, a “display setting” button, a “save” button, and the like are included. Thus, the setting item button area 112 functions as an inspection processing setting unit.
(Height inspection process)

FIG. 145B shows an example of the obtained distance image for the workpiece as shown in FIG. 145A. This distance image is expressed in 16 gradations before gradation conversion. When height measurement (height inspection processing) is performed on such a distance image, high accuracy is obtained by using height information with high accuracy of 16 gradations without performing gradation conversion. Can be realized. Specifically, as shown in FIG. 145C, in order to measure the height of each surface of the workpiece, the inspection target regions are set on three surfaces of the workpiece, and the height of each inspection target region is set to the 16th floor. Measure as it is.
(Image inspection processing)

  On the other hand, when measurement without using height information (image inspection processing) is performed in the inspection processing, high gradation information is unnecessary, and the load is less when processing is performed with a distance image having a lower gradation. For this reason, tone conversion is performed on a high-tone distance image to obtain a low-tone distance image, and processing is performed. Here, it is not necessary to convert the entire distance image into a low gradation distance image, and it is sufficient to perform gradation conversion only for an object on which image inspection processing is performed. That is, the area to be subjected to gradation conversion is an inspection target area set for image inspection processing that does not use height information, among one or more set inspection target areas. In the example shown in FIG. 145D, as in FIG. 145B, an inspection target area for image inspection processing is set for the high gradation distance image before gradation conversion. In this example, in order to detect chipping and the like on the second stage from the top of the workpiece, the inspection target area is set so as to surround the second stage columnar shape. In other words, since the third column is not subject to appearance inspection, this portion is set to be excluded from the inspection target region. Then, gradation conversion is performed on the image processing inspection target area. The low gradation distance image obtained as a result is shown in FIG. 145E. The low gradation distance image after gradation conversion shown in this figure is expressed in eight gradations, and the height information is somewhat lost compared to the high gradation distance image, but the presence or absence of missing parts is detected. Since sufficient accuracy is maintained for visual inspection applications, there is no problem in image inspection processing. On the other hand, the area that requires gradation conversion is greatly reduced as compared to FIG. 145B and the like, which contributes to simplification and speeding up of processing.

In this way, the presence or absence of gradation conversion is selected according to the inspection process. For example, among the inspection processes displayed in the flow display area 261 in FIG. 110, the “height measurement” processing unit and the second “height measurement” processing unit have the 16th floor that the distance image has without gradation conversion. High-precision height measurement is realized using the key height information. On the other hand, in the other inspection processes, for example, the “numerical calculation” processing unit, the “area” processing unit, and the “blob” processing unit 267, the gradation-converted 8-tone low gradation image is used. Although not shown, when the inspection process is performed on the luminance image (for example, the color inspection on the color luminance image), the height information is not necessary in the first place, and therefore, gradation conversion is naturally not necessary.
(Display of gradation conversion condition setting means 43)

  In addition, for image inspection processing that requires such tone conversion processing, tone conversion conditions for performing appropriate tone conversion are set from the tone conversion condition setting means 43. At this time, the gradation conversion condition setting means 43 can be set only when the gradation conversion process is necessary, and conversely, in the inspection process that does not require the gradation conversion process, for example, in the height inspection process, the gradation conversion condition By making the setting impossible, the user can smoothly set only necessary items without being confused by unnecessary setting work. Therefore, in the present embodiment, when setting an inspection process that does not require image height information other than the height inspection process, a gradation conversion parameter for performing gradation conversion of a distance image is set. While the tone conversion condition setting unit 43 is displayed, the tone conversion condition setting unit 43 is not displayed when the height inspection process is set.

  A specific setting procedure will be described based on the flowchart of FIG. First, in step S14601, an inspection process is selected. Here, an inspection process to be executed by the inspection execution means is selected in the “measurement” menu which is the inspection process selection means. Next, in step S14602, it is determined whether the inspection process selected in step S14601 requires gradation conversion. If the inspection process requires gradation conversion, the process proceeds to step S14603, and gradation conversion is enabled. In addition, the gradation conversion setting means is displayed in the setting item.

  For example, when image inspection processing is performed on the work in FIG. 145A by the “area” processing unit, as shown in FIG. 147, the gradation conversion condition setting means 43 is set in the setting item button area 112 as inspection processing setting means. The “height extraction” button 116 is displayed. When the “height extraction” button 116 is pressed, a height extraction setting screen is displayed as shown in FIG. 148. From this screen, the user can set conditions necessary for gradation conversion processing.

  In the example of FIG. 148, a low gradation distance image subjected to gradation conversion in accordance with the gradation conversion condition set in the right operation area is displayed on the image display area. As a result, the user can visually confirm whether or not a desired inspection result can be obtained under the current gradation conversion condition, and an advantage that the adjustment operation of the gradation conversion condition can be easily performed is obtained. In particular, in the example of FIG. 148, the inspection target area is set to be circular and is arranged so as to surround the second cylindrical shape of the workpiece. Also, by setting the detection method to dynamic conversion (real-time extraction) and setting the plane reference as the calculation method, the top surface at the center of the workpiece and the outer peripheral surface of the workpiece are detected. Detects scratches and dents on the surface of the. Further, the parameter display of the reference plane detected according to the gradation conversion condition can be switched and displayed by console operation.

  Further, in the example of FIG. 148, gradation conversion is performed only within the set region of the inspection target area, instead of displaying a low gradation distance image obtained by gradation conversion of the entire work displayed on the image display area. A low gradation distance image is displayed. In other words, the original distance image is displayed as it is for images other than the inspection target region. Accordingly, it is possible to visually indicate to the user that the gradation conversion is not performed on the entire input image, but is performed only within the inspection target region that is a part of the input image. When the gradation conversion condition setting is thus completed, the process advances to step S14604.

  On the other hand, in the case of inspection processing that does not require gradation conversion, the process jumps to step S14604 without passing through step S14603. In this case, the gradation conversion condition setting means is not displayed on the setting screen for inspection processing. For example, when the height inspection process is performed by the “height measurement” processing unit, as shown in FIG. 46 and the like, the setting item button area 112 which is the inspection process setting unit is the gradation conversion condition setting unit 43. "Height extraction" button is not displayed. As a result, the user can recognize that there is no need to set conditions regarding gradation conversion that are not necessary for the height inspection process. Alternatively, it is possible to avoid confusion caused by unnecessary setting by making it impossible to set the gradation conversion condition for this inspection process.

  In step S14604, inspection processing is set. Finally, in step S14605, it is determined whether or not all the inspection processes have been set. If not, the process returns to step S14601 to repeat the selection of the inspection process, and if completed, the process is completed.

  On the other hand, the procedure at the time of driving | operation is demonstrated based on the flowchart of FIG. First, in step S14901, a distance image is input as an input image. Next, in step S14902, initialization is performed. Here, 1 is set to n. In step S14903, the nth inspection process is executed. In step S14904, it is determined whether or not n <N (N is the number of set inspection processes). If YES, n is incremented by 1 in step S14905, and the process returns to step S14903 to execute the next inspection process. Repeat the process to execute. On the other hand, if n <N, it is determined that all the inspection processes have been completed, and the process is completed. In this way, all the inspection processes are sequentially executed.

Here, the execution of the inspection process in step S14903 will be described in detail. First, in the case where gradation conversion is required in the inspection process, as shown in the flowchart of FIG. On the other hand, tone conversion is performed. Then, in step S15002, inspection processing is executed on the low gradation distance image after gradation conversion. On the other hand, in the case of inspection processing that does not perform tone conversion, as shown in the flowchart of FIG. 151, the inspection processing in the inspection target region is executed without changing the tone, with the high-tone distance image being maintained ( Step S15101).
(Image selection hiding function)

  Further, at the time of setting, it is possible to limit the selection of an image to be subjected to setting of inspection processing according to the type of inspection processing. That is, when the inspection process selected by the inspection process selection unit can be executed on either the distance image or the luminance image, the distance image or the luminance image can be called as a registered image. On the other hand, some inspection processes can be performed on distance images, but not on luminance images. For example, the height measurement process is executed for a distance image having highly accurate height information. It cannot be performed on a normal luminance image having no height information. Therefore, an inspection process that can be performed only on an image having height information, that is, a distance image, such as a height measurement process, allows only a distance image to be selected at the time of calling a registered image at the time of setting. The brightness image cannot be selected. As a result, it is possible to prohibit or eliminate a setting operation that cannot be originally performed, such as erroneously performing settings relating to the height measurement process on the luminance image, to eliminate setting waste, and to improve user operability.

  Hereinafter, the procedure for setting the inspection processing condition will be described with reference to the flowchart of FIG. First, in step S15201, an inspection process is selected. In step S 15202, it is determined whether the inspection process selected in step S 15201 is an inspection process that can specify either a distance image or a luminance image, or an inspection process that can specify only a distance image. Here, in the case of the inspection processing in which either the distance image or the luminance image can be specified, the process proceeds to step S15203, and either the distance image or the luminance image is selected.

  For example, as shown in FIG. 61, consider a case where an “area” processing unit is selected as the inspection processing. In this case, either a distance image or a luminance image can be specified. Therefore, as an inspection processing setting item in the “area” processing unit, for example, when the “image setting” button 114 in FIG. 62 is pressed to display the image setting screen 380 shown in FIG. You can choose. Here, an image to be displayed in the second image display area 121 can be selected in the “display image” selection field 124 provided in the image selection field 382. In the image setting field 384, an input image and a registered image can be selected. From this image setting field 384, an input image can be designated by an image variable. Here, when the “input image” selection field 386 is selected, an image variable selection screen 390 of FIG. 154 is displayed, and a list of selectable images is displayed. On the image variable selection screen 390, any of images captured by a plurality of imaging means (cameras) connected to the 3D image processing apparatus can be selected. Here, different image variables are assigned to each imaging unit, and the imaging unit and the image variable are associated with each other. For example, the image variable “& Cam1Img” is used for the distance image captured by the camera 1, the image variable “& Cam2Img” is used for the distance image of the camera 2, the image variable “& Cam3Img” is used for the distance image of the camera 3, and the distance image of the camera 4 is used. The image variable “& Cam4Img” is assigned to each. Furthermore, an image variable “& Cam1GrayImg” is assigned to the luminance image captured by the camera 1. The user selects a desired image from the image variable selection screen 390. As described above, in the “area” processing unit, either a distance image or a luminance image can be designated, and therefore, not only the distance image but also the luminance image is displayed as an option candidate.

  Furthermore, if necessary, the image variables displayed in a list on the image variable selection screen 390 can be sorted and displayed so that the user can easily select a desired image.

  On the other hand, if the inspection process is a process that can specify only a distance image, the process advances to step S15204 to select a distance image. That is, the luminance image cannot be selected. For example, as shown in FIG. 45, when the “height measurement” processing unit 266 is selected as the inspection process, only the distance image can be selected. Therefore, as shown in FIG. 46, when an “image setting” button 114 is selected as an inspection processing setting item in the “height measurement” processing unit, an image setting screen 380 is displayed in the same manner, and an image can be selected. . If the “input image” selection field 386 is selected in the same manner, the image variable selection screen 390 shown in FIG. 155 is displayed, and a list of selectable images is displayed. In this image variable selection screen 390, unlike FIG. 154, the luminance image is not displayed as an option, and only the distance image is displayed as an option. With this configuration, when the height is measured, the user can avoid the situation of erroneously selecting a luminance image that does not have height information, and an operation environment with less confusion is provided.

When an image is selected in this way, the process proceeds to step S15205, and inspection processing conditions are set for the selected image. As described above, for each inspection process, it is determined whether an inspection process that can specify either a distance image or a luminance image or an inspection process that can specify only a distance image is used. By defining the types of images that can be selected, setting mistakes are avoided, which contributes to user convenience.
(Inspection processing hidden function)

  The above is an example in which the user is allowed to select the inspection process first, and the selectable image is limited when selecting the image to be subjected to the setting of the inspection process according to the type of the selected inspection process. Explained. Conversely, after selecting an image first, the types of inspection processing that can be performed on this image can be limited. That is, when a distance image or a luminance image is first selected by the image selection means, and then one or more inspection processes executed by the inspection execution means are selected by the inspection processing selection means, the selected image is a distance image or luminance. Only the inspection process that can be executed for each image can be selected depending on the image. With this configuration, it is possible to avoid a situation in which an inspection process that cannot be set by mistake is selected for the selected image or an inspection process condition related to this inspection process is set.

  Hereinafter, the procedure for setting the inspection processing conditions by this method will be described based on the flowchart of FIG. First, in step S15601, an image is selected. Here, a distance image or a luminance image is selected. In step S15602, it is determined whether the image selected in step S15601 is a distance image or a luminance image. In the case of a distance image, the process advances to step S15603 to allow the user to select an inspection process that can be performed on the distance image by the inspection process selection unit. On the other hand, in the case of the luminance image, the process proceeds to step S15604, and the inspection process executable on the luminance image is selected by the user using the inspection process selection unit. When the inspection process is selected in this way, the process proceeds to step S15605, and the inspection process condition for the selected inspection process is set from the inspection process condition setting means.

  For example, as shown in FIG. 157, consider a case where a luminance image and a distance image are acquired for a work. As described above, when an image is registered, a luminance image and a distance image are registered simultaneously. The user selects one of these images, selects the inspection process for the next selected image, and sets the inspection process condition. For example, when a luminance image is selected, as shown in FIG. 158, a tool corresponding to the inspection process performed on the luminance image is added. Here, as in FIG. 44, FIG. 56, and the like described above, when an addition is selected from the submenu displayed by selecting an image, a list of inspection processes that can be performed on the luminance image is displayed. For example, as the “measurement” process, an “area” processing unit, a “scratch” processing unit, and a “blob” processing unit 267 are displayed as options. At this stage, inspection processing that cannot be performed on the luminance image, for example, a “height measurement” processing unit is not displayed. As a result, it is possible to avoid a situation in which the user erroneously selects an inspection process that cannot be set in the luminance image. It is also possible to make the selection impossible by displaying all measurement processes in a list and graying them out. When the inspection processing is selected, the processing unit is determined, and then the inspection processing condition necessary for this inspection processing is set from the inspection processing condition setting means.

  When a distance image is selected, a tool corresponding to an inspection process that can be performed on the distance image is added as shown in FIG. In this case as well, when an add is selected from the submenu displayed by selecting a distance image, a list of inspection processes that can be performed on the luminance image is displayed. For example, in addition to the “area” processing unit, the “scratch” processing unit, and the “blob” processing unit 267, the “height measurement” processing unit is displayed as an option as the “measurement” processing. When the user selects a desired inspection process, the processing unit is determined, and the inspection process condition setting means necessary for the inspection process is set from the inspection process condition setting means.

  As described above, by associating the inspection processing tool with the image, the combination of the image that cannot be selected and the inspection processing is physically excluded, and a setting error by the user can be easily avoided.

The three-dimensional image processing apparatus, the three-dimensional image processing method, the three-dimensional image processing program, the computer-readable recording medium, and the recorded apparatus according to the present invention can be used in an inspection apparatus using the principle of triangulation.

100, 100 ', 200, 300, 400 ... 3D image processing apparatus 1, 1B, 1C, 1D ... head unit 2, 2' ... controller unit 3 ... input means 4 ... display means 10, 10A, 10B ... imaging means 20 ... Projection means; 20A ... First projector; 20B ... Second projector 21 ... Measurement light source 22 ... Pattern generation unit 23, 24, 25 ... Lens 30 ... Head side control unit 31 ... Head side calculation unit 32 ... Distance image generation means 34 ... Filter processing section 36 ... Head side communication means; 36A ... Controller connection interface; 36B ... PC connection interface 38 ... Storage means; 38a ... Distance image storage section; 38b ... Luminance image storage section 41 ... Controller side setting means 42 ... Controller side communication means 43 ... Gradation conversion condition setting means 44 ... Reference plane setting means 45 ... Spatial coding switching means 4 ... gradation conversion means 47 ... interval equalization processing setting means 48 ... projection switching means 49 ... shutter speed setting means 50 ... inspection execution means 51 ... main control section 52 ... controller side connection section 53 ... operation input section 54 ... display control Unit 55 ... Communication unit 56 ... RAM
57 ... Controller-side storage means 58 ... auxiliary storage means 59 ... output section 60 ... image processing section 62 ... abnormal point highlight means 64 ... image search means 66 ... real-time update means 70 ... PLC
110 ... 3D image processing program GUI screen 111 ... First image display area 112 ... Setting item button area 113 ... "Image registration" button 114 ... "Image setting" button 115 ... "Area setting" button 116 ... "Height extraction" Button 117 ... "Pre-processing" button 118 ... "Detection condition" button 119 ... "Detailed setting" button 120 ... Inspection object area setting screen 121 ... Second image display area 122 ... Operation area 124 ... "Display image" selection field 126 ... "Measurement area" setting field 128 ... "Edit" button 130 ... Measurement area edit screen 140 ... Height extraction selection screen 142 ... Extraction method selection means 144 ... "Extraction" button 145 ... "Extraction area" designation field 146 ... Point pointer 147 ... "Extraction area" button 148 ... Extraction area setting dialog 149 ... Extraction area selection field 150 ... Single point designation screen 152 ... "Z height "Display field 154 ... Enhancement method setting field 156 ... Gain adjustment field 158 ..." Detailed setting "button 160 ... Enhancement method detailed setting screen 162 ..." Extraction height "setting field 164 ... Noise removal setting field 166 ... Invalid pixel designation field 170 ... three-point designation screen 172 ... height extraction display field 174 ... three-point designation "detailed setting" button 180 ... three-point designation detailed setting screen 182 ... "extraction height" setting field 190 ... height dynamic extraction setting screen 192 ... "Calculation method" selection field 194 ... "Mask area" button 196 ... "Detailed setting" button 210 ... Average height reference setting screen 220 ... Mask area setting screen 222 ... "Detail setting" button 230 ... Average height reference detailed setting screen 240 ... plane reference detailed setting screen 250 ... free-form surface reference setting screen 252 ... "extraction size" designation field 260 ... initial screen 261 ... flow display area 262 ... third image Display area 263 ... "imaging" processing unit 264 ... "Shapetrax2" processing unit 265 ... "position correction" processing unit 266 ... "height measurement" processing unit; 266B ... second "height measurement" processing unit 267 ... "blob" Processing unit: 267B ... "Color inspection" processing unit 268 ... "Edit" button 269 ... Image setting menu 270 ... Image registration screen 271 ... "Camera selection" column 272 ... "Register" button 280 ... Image setting screen 282 ... "Detailed settings "Button 284 ..." imaging setting "button 290 ... 3D measurement setting screen 292 ..." display with continuous update "field 294 ... shutter speed setting field 295 ... numerical value display field 296 ... shade range setting field 310 ... pre-processing setting field 312 ... non-measurable reference setting field 314 ... uniform interval processing setting field 316 ... spatial code setting field 318 ... Projector selection setting field 322 ... "Display image" selection field 324 ... "Edit" button 326 ... Extraction area editing dialog 330 ... Mask area setting field 332 ... "Edit" button 334 ... Mask area editing dialog 340 ... Filter processing setting screen 350 ... Binarization level setting screen 360 ... Judgment condition setting screen 370 ... First submenu 372 ... Second submenu 373 ... "Measurement" menu 380 ... Image setting screen 382 ... Image selection field 384 ... Image setting field 386 ... ""Inputimage" selection field 390 ... Image variable selection screen 460 ... Height measurement setting screen 620 ... Area setting screen WK, WK7, WK8, WK9, WK10, WK11, WK12, WK13, WK14 ... Work PC ... Personal computer SA ... Search target Area SI ... Dropper icon ROI ... Measurement area BC ... Bell Conveyor DE ... dents

Claims (17)

A three-dimensional image processing apparatus capable of acquiring a distance image including height information of an inspection object and a luminance image obtained by imaging the inspection object and performing image processing based on the distance image or the luminance image Because
A light projecting means for projecting incident light as structured illumination of a predetermined projection pattern;
Imaging means for acquiring reflected light that is projected by the light projecting means and reflected by the inspection object from an oblique direction with respect to the optical axis of the light projecting means, and images a plurality of pattern projection images;
A distance image generating means capable of generating a distance image based on a plurality of pattern projection images imaged by the imaging means;
Inspection execution means for executing predetermined inspection processing on the distance image or the luminance image;
As inspection processing executed by the inspection execution means,
A height inspection process using height information of the distance image;
An inspection process selection means for selecting one or more of a plurality of inspection processes including an image inspection process performed by replacing the height information of the distance image with grayscale information;
For each inspection process selected by the inspection process selection unit, an inspection process setting unit for setting an inspection process condition including selection of a setting image to be subjected to the inspection process;
With
The inspection process selected by the inspection process selection means is
When the inspection process can be performed on either the distance image or the luminance image, the inspection image setting unit can select the distance image or the luminance image as a setting image.
In the case of an inspection process that can be performed on a distance image and cannot be performed on a luminance image, the inspection image setting unit can select the distance image as a setting image and When the image inspection process is selected by the inspection process selection unit and the distance image is selected as the setting image, the image inspection process is the same as when the luminance image is selected. 3. A three-dimensional image processing apparatus configured to display settings related to A three-dimensional image processing apparatus according to claim 1,
The inspection processing setting means, when selecting a setting image to be subjected to inspection processing, the inspection processing to be subject to setting inspection processing conditions is
When the inspection process can be performed on either the distance image or the luminance image, while displaying the distance image or the luminance image as an option,
A three-dimensional display characterized in that, in the case of inspection processing that can be performed on a distance image and cannot be performed on a luminance image, the luminance image is not displayed while the distance image is displayed as an option. Image processing device. The three-dimensional image processing apparatus according to claim 2, further comprising:
A distance image storage unit for recording a distance image;
A luminance image storage unit for recording the luminance image;
A unique image variable is assigned to the distance image and the luminance image, and when registering the distance image and the luminance image as a registered image, the image variable of each image is also registered,
A three-dimensional image processing apparatus configured to be able to call a corresponding registered image with an image variable. A three-dimensional image processing apparatus according to claim 2 or 3,
3. A three-dimensional image processing apparatus characterized in that the inspection process that can be performed on a distance image and cannot be performed on a luminance image is a height measurement process that measures the height of an inspection object. A three-dimensional image processing apparatus according to claim 3,
A three-dimensional image processing apparatus characterized in that both a distance image and a luminance image can be stored in the distance image storage unit and the luminance image storage unit as setting images for examination setting. A three-dimensional image processing apparatus according to any one of claims 1 to 5,
3. The three-dimensional image processing apparatus according to claim 1, wherein the luminance image is an image obtained by capturing an inspection object using the light projecting unit as a full illumination pattern. A three-dimensional image processing apparatus according to any one of claims 1 to 6,
A three-dimensional image processing apparatus, wherein a luminance image can be picked up by the image pickup means using an illumination means different from the light projecting means. A three-dimensional image processing apparatus according to any one of claims 1 to 7,
A three-dimensional image processing apparatus, wherein the luminance image is generated by combining a plurality of pattern projection images from which a distance image is generated. A three-dimensional image processing apparatus according to any one of claims 1 to 8,
3. A three-dimensional image processing apparatus, wherein the light projecting means projects structured illumination for obtaining a distance image using at least a phase shift method and a spatial coding method. The three-dimensional image processing apparatus according to any one of claims 1 to 9, further comprising:
The distance image generated by the distance image generating means is a low gradation distance image having a gradation number lower than the number of gradations of the distance image and replacing the height information of the distance image with the gray value of the image. Gradation conversion means for gradation conversion;
Gradation conversion condition setting means for setting gradation conversion parameters for performing gradation conversion of a distance image by the gradation conversion means;
A three-dimensional image processing apparatus comprising: display means for displaying a low gradation distance image subjected to gradation conversion by the gradation conversion means. The three-dimensional image processing apparatus according to claim 10,
It has a head part and a controller part,
The head unit includes the light projecting unit and the imaging unit.
The three-dimensional image processing apparatus, wherein the controller unit includes the gradation conversion condition setting unit. A three-dimensional image processing apparatus according to claim 10 or 11,
The three-dimensional image processing apparatus, wherein the gradation conversion parameter includes a plane offset amount serving as a reference for gradation conversion and a gradation width to be subjected to gradation conversion. A three-dimensional image processing apparatus according to any one of claims 10 to 12,
The inspection process selection means can individually select one or more inspection processes for a distance image, a low gradation distance image, or a luminance image,
The inspection execution means is configured to execute a predetermined inspection process selected by the inspection process selection means for a distance image, a low gradation distance image, or a luminance image. Image processing device. A three-dimensional image processing apparatus according to any one of claims 1 to 13,
Instead of the imaging means, a monochrome CCD camera or a color CCD camera that captures a luminance image can be connected,
3. A three-dimensional image processing apparatus, wherein the inspection execution means can execute an image inspection process on a luminance image captured by the monochrome CCD camera or the color CCD camera. A three-dimensional image processing method for obtaining a distance image including height information of an inspection object and a luminance image obtained by imaging the inspection object and performing image processing based on the distance image or the luminance image,
Projecting incident light from the oblique direction with respect to the optical axis of the imaging means by the light projecting means as structured illumination of a predetermined projection pattern;
Obtaining reflected light projected by the light projecting means and reflected by the inspection object, and imaging a plurality of pattern projection images by the imaging means;
A step of generating a distance image by a distance image generation unit based on a plurality of pattern projection images imaged by the imaging unit;
As a predetermined inspection process executed on the distance image or the luminance image,
A height inspection process using height information of the distance image;
A step of selecting one or more inspection processing selection means from a plurality of inspection processing including image inspection processing performed by replacing the height information of the distance image with grayscale information;
For each selected inspection process, a step of selecting a setting image to be an object of the inspection process;
A step of executing a predetermined inspection process on the selected distance image or luminance image by an inspection execution unit;
Including
When selecting the setting image, the inspection process includes:
When the inspection process can be performed on either the distance image or the luminance image, the distance image or the luminance image can be selected as the setting image.
If the inspection process can be performed on the distance image and cannot be performed on the luminance image, the distance image can be selected as the setting image, but the luminance image cannot be selected. ,
When image inspection processing is selected by the inspection processing selection means and a distance image is selected as a setting image, the same settings relating to image inspection processing as when a luminance image is selected are displayed. A three-dimensional image processing method. A three-dimensional image processing program capable of acquiring a distance image including height information of an inspection object and a luminance image obtained by imaging the inspection object and performing image processing based on the distance image or the luminance image And to the computer,
Based on a plurality of pattern projection images obtained by projecting incident light as a structured illumination of a predetermined projection pattern from an oblique direction with respect to the optical axis of the imaging means, obtaining reflected light reflected by the inspection object A distance image generation function capable of generating a distance image,
As an inspection execution function for executing a predetermined inspection process on a distance image or a luminance image and an inspection process executed by the inspection execution function,
A height inspection process using height information of the distance image;
An inspection process selection function for selecting one or more of a plurality of inspection processes including an image inspection process performed by replacing the height information of the distance image with grayscale information;
For each inspection process selected by the inspection process selection function, an inspection process setting function for setting an inspection process condition including selection of a setting image to be subjected to the inspection process;
Realized,
The inspection process selected by the inspection process selection function is
When the inspection process can be performed on either the distance image or the brightness image, the distance image or the brightness image can be selected as the setting image by the inspection process setting function.
In the case of an inspection process that can be performed on a distance image and cannot be performed on a luminance image, the distance image can be selected as a setting image using the inspection processing setting function. Make the selection impossible,
When an image inspection process is selected by the inspection process selection function and a distance image is selected as a setting image, the same settings related to the image inspection process as when a luminance image is selected are displayed. A three-dimensional image processing program.   A computer-readable recording medium or a recorded device on which the three-dimensional image processing program according to claim 16 is recorded.