KR101757240B1

KR101757240B1 - Method for generating reference pattern of 3D shape measuring device

Info

Publication number: KR101757240B1
Application number: KR1020150123925A
Authority: KR
Inventors: 박찬화; 신나라; 최경진
Original assignee: 주식회사 미르기술
Priority date: 2015-09-02
Filing date: 2015-09-02
Publication date: 2017-07-13
Also published as: KR20170027892A

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a reference pattern generating method in a three-dimensional shape measuring apparatus, and more particularly, to a method of generating a reference pattern in a three-dimensional shape measuring apparatus that irradiates an inspection object with pattern illumination using a plurality of projectors, A method of generating a reference pattern performed by a shape measuring apparatus, the method comprising: acquiring a pattern image photographed through a camera when the inspection object is irradiated with pattern illumination having an equal interval pattern by the projector; Calculating a pattern intensity profile for calculating a period of the pattern using the pattern image; Extracting at least one zero crossing point in the pattern intensity profile and calculating point spacing between the zero crossing points to calculate point spacing data; And a step of calculating a variation amount of the interval with respect to the pattern using the point interval data and applying the variation amount of the interval to the pattern to generate a reference pattern having unequal intervals. Therefore, the present invention can acquire a uniform pattern image in the view angle photographing range of the camera with respect to the area irradiated with the pattern illumination at a boiling interval irrespective of the mounting angle or direction of the projector, Accordingly, since the phenomenon of widening or narrowing the interval between the patterns of the pattern illumination is minimized, the error of the height value can be minimized or minimized in the measurement of the three-dimensional shape, thereby increasing the measurement accuracy of the three-dimensional shape.

Description

[0001] The present invention relates to a method of generating a reference pattern in a three-dimensional shape measuring apparatus,

The present invention relates to a reference pattern generating method in a three-dimensional shape measuring apparatus, and more particularly, to a method of generating a reference pattern in a three-dimensional shape measuring apparatus, And a method of generating a reference pattern in a three-dimensional shape measuring apparatus capable of obtaining a pattern image of an interval.

With the rapid development of information and communication, there is a demand for technology to make electronic devices such as mobile phones, notebooks, smart phones, and tablet PCs smaller, lighter, and more functional. In order to realize the demands of these technologies, it is necessary to build an overall infrastructure for high-density integration that can produce not only miniaturization and high integration of electronic parts mounted on electronic devices but also good products such as corresponding mounting equipment, operation technology, inspection technology, The mounting element technology became necessary.

Among them, the vision inspection analyzes an image taken by using a camera on the appearance of a chip moving at a high speed, and judges whether there is a defect in the appearance of the product. In general, the vision test acquires and analyzes images within a short time of 10 to 20 msec. If the vision inspection time is delayed, the production time is delayed and the productivity is lowered. Therefore, there is a need for a vision inspection technology for accurately judging whether or not the appearance of the product is defective within a short period of time.

Such a vision inspection apparatus generally includes a lighting device for illuminating a light toward a subject to be inspected, a camera for photographing the illuminated inspection object, and a screen photographed by the camera, And a discriminating means for discriminating a discrimination result.

In the case of a lighting device and a camera, usually, a light source is disposed on the opposite side of the object to be inspected, and light is irradiated toward the object to be inspected and the illuminated object is photographed. The distinguishing means outputs a screen shot by the camera, And it is determined whether the object to be inspected is manufactured in the correct state.

In order to accurately inspect the object to be inspected, it is important to accurately measure the three-dimensional shape of the object to be inspected, particularly the height.

As the three-dimensional measurement method of the object to be inspected, there are various methods such as an interferometer method, a laser method, and a moire method. Moire method is widely used because it can accurately measure the height of an object over a wide area.

The 3D shape measurement technique using moiré uses a three-dimensional shape measurement method using a child moiré and a three-dimensional shape measurement method using a projection moiré. Among them, the three dimensional shape measurement method using the projection moire is to measure the moire shape by the phase shift method in order to precisely measure the surface shape of the object.

In other words, the 3D shape measurement technique using the phase shift projection type moiré measures the shape of the object while shifting the lattice phase by quarter cycle, so the image is captured four times in one cycle. In this case, when the gratings of different periods are used, four times of image capturing is performed in a large period lattice and four times of images are taken in a small period lattice. Therefore, a total of 8 images .

As the prior art data, the optical inspection method of Korean Patent Laid-Open No. 10-2007-0115067 is a method in which the image with lowered the regular reflection degree of the regular reflection surface and the image with the lattice plane near the periphery are obtained, And the photographed image are compared with each other to judge the presence or absence of a defect in the inspection object, thereby enabling accurate image acquisition.

In addition, Korean Patent No. 10-0663323 discloses a three-dimensional shape measuring apparatus and a three-dimensional shape measuring method using the apparatus, which obtains distorted grid data from a grid pattern reflected from a subject and obtained through a camera, To obtain a moire pattern, thereby analyzing the moire pattern to analyze the three-dimensional shape.

By measuring the intensity profile of the reference pattern of the grid pattern used in the prior art reference image or the reference grid data, it is possible to analyze the visual information by separating the intensity profile into sinusoids having spatial frequencies.

The spatial frequency of the grid pattern is the number of cycles of the grid pattern per field, and one cycle is the bright line segment and the dark line segment once. Thus, the thin bar segments are associated with a higher spatial frequency, and the spatial frequency depends on the distance between the grid and the projector.

1 is a view for explaining a strength profile for a pattern image in a conventional three-dimensional shape measuring technique using a moire.

As shown in FIG. 1, a conventional moiré-based three-dimensional shape measuring technique can calculate an intensity profile for measuring an image intensity in units of pixels from a reference pattern image, The reference pattern is used.

Conventionally, in a three-dimensional shape measurement technique using a moiré, a non-uniform spacing pattern is generated on the surface of an object to be inspected when projecting an equidistant reference pattern on a projector mounted at a specific angle. That is, since the pattern period or pitch is narrow on the surface of the object to be inspected near the projector and the pattern pitch on the surface of the object to be inspected is far from the projector, the pattern pitch varies depending on the distance from the projector A pattern of different unequal intervals is generated.

Therefore, in the conventional three-dimensional shape measurement technique using moiré, there is a problem that an error occurs in the height value according to the distance between the projector and the object to be inspected when measuring the three-dimensional shape of the object to be inspected. In order to calculate the inter-pattern spacing with respect to the sine wave type reference pattern, the distance between the zero crossing points, that is, the number of pixels must be measured. However, there is a problem that it is difficult to find an accurate zero crossing point due to the noise existing in the pattern image.

Korean Patent Laid-Open No. 10-2007-0115067 "Optical inspection method" Korean Patent No. 10-0663323 entitled "Three-dimensional shape measuring apparatus and method for measuring three-dimensional shape using the same"

The present invention is characterized in that a reference pattern of pattern illumination illuminated on an object to be inspected for three-dimensional shape measurement is generated at unequal intervals, so that the viewing angle of the camera And a pattern image photographed in a predetermined range has a uniform interval pattern.

Among the embodiments, a reference pattern generating method in a three-dimensional shape measuring apparatus is a method of generating a reference pattern by irradiating a pattern illumination on an object to be inspected using a plurality of projectors and then measuring the three-dimensional shape of the object to be inspected A method of generating a reference pattern performed by a shape measuring apparatus, the method comprising: acquiring a pattern image photographed through a camera when the inspection object is irradiated with pattern illumination having an equal interval pattern by the projector; Calculating a pattern intensity profile for calculating a period of the pattern using the pattern image; Extracting at least one zero crossing point in the pattern intensity profile and calculating point spacing between the zero crossing points to calculate point spacing data; And a step of calculating a variation amount of the interval with respect to the pattern using the point interval data and applying the variation amount of the interval to the pattern to generate a reference pattern having unequal intervals.

Wherein the pattern image includes at least one first pattern image having a first phase value and at least one second pattern image having a second phase value, the first pattern image having a horizontal pattern, The image is characterized by having a vertical pattern.

Wherein the step of calculating the pattern intensity profile for calculating the period of the pattern using the pattern image comprises the steps of: applying a noise intensity-removed pattern intensity profile (I ') on the basis of the first pattern image, And a step of computing

Wherein the step of calculating the pattern intensity profile for calculating the period of the pattern using the pattern image includes the step of calculating a pattern intensity profile in which a noise is removed using a projected intensity for a predetermined area of the first pattern image, And a step of computing

Wherein the step of calculating the spacing change amount with respect to the pattern using the point spacing data and applying the spacing variation amount to the pattern inversely to generate the reference pattern having the unequal spacing comprises: A linear approximation expression or a quadratic approximation expression is constructed, and an inverse function of the linear approximation expression or the quadratic approximation expression is obtained to generate a reference pattern having unequal intervals.

The reference pattern generating method in the three-dimensional shape measuring apparatus according to the present invention is a method of generating a reference pattern of pattern illumination at a non-uniform interval, so that a viewing angle of a camera It is possible to acquire the pattern image of the equal interval in the range.

Therefore, according to the present invention, since the phenomenon of widening or narrowing the distance between patterns of the pattern illumination is minimized according to the distance between the object and the object to be inspected, there is little or no error in the height value during the measurement of the three- The measurement accuracy can be improved.

1 is a view for explaining a strength profile of a pattern image of a three-dimensional shape measuring apparatus using a general moire.
2 is a front view for explaining a configuration of a three-dimensional shape measuring apparatus according to an embodiment of the present invention.
3 is a plan view of Fig.
4 is a flowchart illustrating a method of generating a reference pattern in a three-dimensional shape measuring apparatus according to an embodiment of the present invention.
FIG. 5 is a view for explaining a pattern image projected in the pattern image of FIG. 4; FIG.
FIG. 6 is a diagram illustrating a modified pattern intensity profile with noise removed for the pattern image of FIG. 5;
7 is a diagram illustrating a pattern intensity profile using a projection technique for the pattern image of FIG.
FIG. 8 is a view for explaining point interval data extracted from the pattern intensity profile of FIG. 4; FIG.
Fig. 9 is a diagram for explaining a state in which an approximate expression for point interval data in Fig. 8 is constructed.

The description of the present invention is merely an example for structural or functional explanation, and the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the present invention should be understood to include equivalents capable of realizing technical ideas. Also, the purpose or effect of the present invention should not be construed as limiting the scope of the present invention, since it does not mean that a specific embodiment should include all or only such effect.

Meanwhile, the meaning of the terms described in the present invention should be understood as follows.

The terms "first "," second ", and the like are intended to distinguish one element from another, and the scope of the right should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected to the other element, but there may be other elements in between. On the other hand, when an element is referred to as being "directly connected" to another element, it should be understood that there are no other elements in between. On the other hand, other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

It should be understood that the singular " include "or" have "are to be construed as including a stated feature, number, step, operation, component, It is to be understood that the combination is intended to specify that it does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

In each step, the identification code (e.g., a, b, c, etc.) is used for convenience of explanation, the identification code does not describe the order of each step, Unless otherwise stated, it may occur differently from the stated order. That is, each step may occur in the same order as described, may be performed substantially concurrently, or may be performed in reverse order.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used predefined terms should be interpreted to be consistent with the meanings in the context of the related art and can not be interpreted as having ideal or overly formal meaning unless explicitly defined in the present invention.

FIG. 2 is a front view for explaining a configuration of a three-dimensional shape measuring apparatus according to an embodiment of the present invention, and FIG. 3 is a plan view of FIG.

2 and 3, the three-dimensional shape measuring apparatus 100 includes a stage 110, a plurality of lights (not shown), an LED driving unit 120, a vertical camera 130, a plurality of projectors 140 A projector driving unit 145, a grabber 150, an image processing unit 160, and an interface unit 170.

The stage 110 supports the object to be inspected, and moves the object to be inspected to the inspection position while moving under the control of the image processor 160.

The illumination is arranged on the object to be inspected in such a manner that a plurality of LEDs for R, G, B color illumination or white illumination are set in a predetermined manner. Such illumination may include various numbers and types of illumination, such as two, three, or six.

The LED driving unit 120 may control the LEDs of the plurality of projectors 140 or the LEDs of the plurality of projectors 140 for a predetermined period of time according to the lighting control signal, Adjust the brightness of each LED according to it.

The vertical camera 130 is installed on the stage 110 to capture an image of the object to be inspected and output a video signal. The vertical camera 130 is a camera for acquiring a two-dimensional image in a state in which the color illumination is on, and performs a two-dimensional inspection using the captured color image. Here, a color image is used to shape a three-dimensional image through texturing of three-dimensional data.

The vertical camera 130 acquires a pattern image in a state in which an illumination pattern of a vertical pattern or a horizontal pattern is irradiated by the projector 140, and calculates a three-dimensional shape of the object to be inspected Perform dimension checking.

The projector 140 can load the illumination pattern data for an illumination pattern corresponding to the purpose of the digital vision inspection and project a specific illumination pattern to the object to be inspected, Can be set. The projector 140 may be a 3D optical system for irradiating a vertical pattern or a horizontal pattern onto the surface of the object to be inspected, and the period of the pattern may be determined according to the pattern data provided by the image processor 160.

A plurality of such projectors 140 are disposed around the vertical camera 130 so as to face each other. 3, the first projector 141 and the third projector 133 are provided so as to oppose each other on the right and left sides of the vertical camera 130, and the second projector 142 and the fourth projector 133, And the vertical camera 130 are installed so as to face each other on the upper and lower sides of the vertical camera 130. The first to fourth projectors 141, 142, 143, and 1144 may be disposed on the same concentric circle around the vertical camera 130.

The projector driving unit 145 operates the projector 140 in accordance with the illumination control signal when the trigger signal of the image processing unit 160 is transmitted and outputs the image acquisition signal And transmits it to the grabber 150. In addition, the projector driving unit 145 loads and stores pattern data for a pattern that meets the purpose of the vision inspection in a digital manner.

The grabber 150 instructs the photographing of the vertical camera 130 according to the image acquisition signal and converts the image signal of the pattern image transmitted through the connection line of the vertical camera 130 into a digital signal, To the processing unit 160.

The image processor 160 processes the digital image signal transmitted from the grabber 150 and outputs the digital image signal. The image processing unit 160 controls the operation of the stage 110, the illumination, the LED driving unit 120, the vertical camera 130, the projector 140, and the grabber 150 as a whole.

The image processing unit 160 can output the inspection image or the inspection result data of the inspection object to the display means (not shown) by using the image signals picked up by the vertical camera 130, It is possible to check the shape of the object to be inspected or to judge whether the object is defective through the image or the inspection result data.

Here, the image processing unit 160 may be a communication device capable of connecting to the network such as a desktop computer, a laptop computer, a smart phone, a tablet PC, and the like. The image processing unit 160 may include image processing and vision inspection There is no limit to the type of terminal that can be used.

The interface unit 170 receives the trigger signal generated by the image processor 160 and transmits the trigger signal to the projector driver 145.

4 is a flowchart for explaining a reference pattern generating method in a three-dimensional shape measuring apparatus according to an embodiment of the present invention, FIG. 5 is a view for explaining a pattern image projected from the pattern image of FIG. 4, Fig. 7 is a view for explaining a pattern intensity profile using a projection technique for the pattern image of Fig. 5, and Fig. 7 is a view for explaining a modified pattern intensity profile in which noise is removed with respect to the pattern image of Fig. 8 is a view for explaining the point interval data extracted from the pattern intensity profile of Fig. 4, and Fig. 9 is a diagram for explaining a state in which an approximate equation for the point interval data of Fig. 8 is constructed.

Referring to FIG. 4, a reference pattern generation method in a three-dimensional shape measurement apparatus is performed by a three-dimensional shape measurement apparatus 100, and includes an application form in which reference pattern generation functions are executed under the control of the image processing unit 160 So that the detailed functions can be updated as needed.

The projector 140 irradiates the inspection object with the pattern illumination having the uniform interval pattern under the control of the image processing unit 160. The vertical camera 130 transmits the pattern image of the inspection object to the image processing unit 160 (S1 and S2) At this time, the pattern image includes at least one first pattern image having a first phase value ([pi]) and at least one second pattern image having a second phase value ([pi] / 2) . The first pattern image is two images having a horizontal pattern and the second pattern image is four images (I_1, I_2, I_3, I_4) having a vertical pattern.

The image processing unit 160 calculates a pattern intensity profile that can clearly calculate the period of the pattern using two first pattern images or four second pattern images. Let the period of the pattern be a pitch, and extract the intensity in units of pixels.

6, the image processing unit 160 calculates a pattern intensity profile I 'from which noises E1 and E3 have been removed by applying the following Equation 1 using a pattern image having a horizontal pattern, I 'has characteristics matching I _n .

In the formula (1), when n is 1, m is 3 or when n is 2, m is 4.

7, in order to extract a noise-canceled pattern intensity profile while maintaining the original image characteristic of the pattern image, the image processor 160 calculates the predicted intensity (Projected Intensity) can be used.

The image processing unit 160 extracts at least one zero crossing point in the pattern intensity profile as shown in FIG. 8, and calculates point interval data between zero crossing points (S4) (Number of pixels) and a zero crossing point number.

The image processing unit 160 calculates the amount of variation of the pattern with respect to the pattern using the point interval data, and generates the reference pattern having the unequal spacing by applying the variation amount of the distance to the pattern inversely. (S5 and S6)

That is, as shown in FIG. 9, the image processing unit 160 obtains an interval variation by constructing a linear approximation equation or a quadratic approximation equation with respect to the point interval data, obtains an inverse function of a linear approximation equation or a quadratic approximation equation, As shown in FIG.

Since the plurality of projectors 140 are mounted at a specific angle so as to be arranged on the same concentric circle centering on the vertical camera 130, the closer to the projector 140 the pattern illumination having the reference pattern of equal interval is irradiated There is a characteristic in which the pattern pitch becomes narrower and the pattern pitch becomes wider.

Using this characteristic, the reference pattern generating method in the three-dimensional shape measuring apparatus generates the reference pattern at a non-uniform interval and applies it to the pattern illumination of the projector 140, thereby obtaining uniform intervals in the field of view (FOV) Can be obtained.

That is, the reference pattern is applied to the reference pattern so that the pattern pitch is widened by using the feature that the pattern pitch becomes narrower as the closer to the projector, and the pattern pitch is widened as the pattern pitch becomes wider as the distance from the projector becomes larger. Apply to the reference pattern to narrow. Therefore, when the reference pattern illumination having the unequal spacing is irradiated on the object to be inspected, the pattern image photographed by the vertical camera 130 becomes a pattern image of equal intervals.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

110: stage 120: LED driver
130: vertical camera 140: plural projectors
145: Projector driving part 150: Grabber
160: image processor 170:

Claims

A method of generating a reference pattern performed by a three-dimensional shape measuring apparatus for measuring a three-dimensional shape of an object to be inspected by irradiating a pattern illumination to an object to be inspected using a plurality of projectors and using a reflected pattern image,
Acquiring a pattern image photographed through a camera when the inspection object is irradiated with a pattern illumination having an equally-spaced pattern by the projector;
Calculating a pattern intensity profile for calculating a period of the pattern using the pattern image;
Extracting at least one zero crossing point in the pattern intensity profile and calculating point spacing between the zero crossing points to calculate point spacing data; And
Calculating a variation amount of the gap with respect to the pattern using the point interval data and applying the gap variation amount to the pattern to generate a reference pattern having an unequal gap, A reference pattern generating step of generating a reference pattern;

The method according to claim 1,
Wherein the pattern image includes at least one first pattern image having a first phase value and at least one second pattern image having a second phase value,
Wherein the first pattern image has a horizontal pattern and the second pattern image has a vertical pattern.

3. The method of claim 2,
Wherein the step of calculating the pattern intensity profile for calculating the period of the pattern using the pattern image comprises:
Further comprising the step of calculating a pattern intensity profile (I ') from which noise has been removed by applying the following equation (1) based on the first pattern image: Way.
Equation 1

Here, when n is 1, m is 3 or when n is 2, m is 4

The method according to claim 2 or 3,
Wherein the step of calculating the pattern intensity profile for calculating the period of the pattern using the pattern image comprises:
Further comprising the step of calculating a pattern intensity profile in which noises are removed using the projected intensity for a predetermined area of the first pattern image. Way.

The method according to claim 1,
Wherein the step of calculating the spacing change amount with respect to the pattern using the point spacing data and applying the spacing variation amount to the pattern inversely generates the reference pattern having the unequal spacing
Wherein the interval variation is obtained by constructing a linear approximation expression or a quadratic approximation expression with respect to the point interval data, and an inverse function of the linear approximation expression or the quadratic approximation expression is found to generate a reference pattern having an unequal interval A method of generating a reference pattern in a dimensional shape measuring apparatus.