CN115272256A - Sub-pixel level sensing optical fiber path Gaussian extraction method and system - Google Patents

Abstract

The embodiment of the specification provides a sub-pixel level sensing optical fiber path Gaussian extraction method and system, wherein the method is used for detecting the path of a sensing optical fiber laid on the surface of a silicon wafer and comprises the following steps: determining a visual lighting scheme according to the material characteristics, and acquiring an image based on the visual lighting scheme; denoising the image through bilateral filtering, and extracting edge information of the distributed sensing optical fiber by adopting a sub-pixel edge detection technology based on a Canny algorithm; and closing the edge pairs based on the edge information, and extracting skeleton information by using a Gaussian line detection method to obtain the path of the distributed sensing optical fiber.

Description

Method and system for Gaussian extraction of sub-pixel-level sensing fiber path

technical field

This document relates to the field of computer technology, in particular to a method and system for Gaussian extraction of sub-pixel-level sensing optical fiber paths.

Background technique

Distributed sensing optical fiber has the characteristics of controllable ultra-high spatial resolution, small mass and diameter, corrosion resistance, electrical insulation, and high sensitivity. In addition, based on its soft and tough texture, distributed sensing optical fiber has good adaptability to the shape of the structure surface, so it is often used to monitor strain, force deformation, temperature change and other purposes. In the wafer inspection system based on distributed sensing optical fiber, it is necessary to make the measurement point data correspond to the position of the wafer surface and provide a data basis for the investigation of the causes of warpage and deformation. path for precise data reconstruction.

Distributed sensing optical fiber is usually used in tunnels, slopes, mines and other large and long working distance occasions, its spatial resolution is usually 0.5m or 1m, and it is not necessary to accurately locate each measuring point, but this is obviously not applicable In the field of precision measurement in the development of modern semiconductor technology, and to apply distributed sensing optical fiber to precision measurement, a method is needed to accurately obtain the spatial position of the optical fiber measurement point. In recent years, machine vision technology has become more and more mature. As a typical non-contact detection technology, due to its advantages of high precision and high intelligence, it is often used in various occasions that require precise monitoring, such as: defect detection, image restoration , Medical images and other fields have achieved good results. In the prior art, one method uses the Forstner feature extraction operator to extract part of the optical fiber feature points on the image, and stitches adjacent images to obtain a panoramic image of the component, and uses the maximum inter-class variance method to extract the required areas respectively image. Finally, the texture noise is removed by fast Fourier transform, and the line extraction operator is used to obtain the complete path of the fiber. An improved SIFT algorithm is also provided in the prior art, and related parameter setting methods and empirical formulas are given. Through the detection of actual surface defects, the robustness and anti-interference ability of the SIFT algorithm are verified by comparison, and The correctness and feasibility of the relevant parameter setting method. It shows that the SIFT algorithm has obvious superiority in the detection rate of sunken and spot defects, especially in the case of noisy image interference, and has a greater advantage in the false detection rate of cracks; in addition, in the prior art, based on The packaging surface defect detection method of image processing technology solves the problem of low accuracy. Use the CCD camera to collect the surface image of the packaging box, use the binarization processing technology to distinguish the background area and the target image, use the shade correction difference to process the background and the foreground, use the image redrawing algorithm to detect the edge of the image, eliminate the noise to obtain obvious image edge features, and Use the threshold to define various defects in the image to complete defect feature extraction, design a defect classifier based on the extracted defect features, and use this to divide and identify defect content to complete packaging surface defect detection; in the prior art, the seed growth method can be used for solar cell electrodes Perform edge extraction; the OTSU method can also be used to detect image surface defects, but this method is mostly used in situations where the global gray level is uniform and the interference information is relatively small.

To sum up, in the prior art, the line extraction operator in machine vision is used to extract the optical fiber path, but the detection object is the optical fiber path in the composite material interlayer, and the main problem is to identify the multi-image path and then splice it. It does not involve the detection of optical fiber paths on highly reflective mirrors, and it does not give information on the recognition accuracy of the algorithm; in the prior art, the SIFT algorithm is used to detect product surface defects, although this algorithm is not effective on noisy and complex images It can achieve high-precision category recognition, but it is not focused on high-precision measurement, so it cannot be competent for the reconstruction of optical fiber path data; in the prior art, defect classifiers are used to detect product packaging defects, and its edge detection accuracy for images The requirements are not high and cannot meet the requirements of optical fiber position data reconstruction; in the prior art, the seed growth method is used to extract the edge of the solar cell electrode. The above three methods are all for detecting specific types of defects and are not suitable for continuity fiber path extraction. In the prior art, the OTSU method is used to detect image surface defects, but this method is not suitable for occasions where the global gray scale is not uniform and there are many interference information.

Contents of the invention

The purpose of the present invention is to provide a method and system for Gaussian extraction of a sub-pixel-level sensing fiber path, aiming at solving the above-mentioned problems in the prior art.

The invention provides a method for Gaussian extraction of sub-pixel-level sensing optical fiber paths, which is used for path detection of small-diameter sensing optical fibers laid on the surface of silicon wafers. The method includes:

determining a visual lighting scheme according to material properties, and acquiring an image based on the visual lighting scheme;

The image is denoised by bilateral filtering, and the edge information of the distributed sensing optical fiber is extracted by sub-pixel edge detection technology based on the Canny algorithm;

Edge pairs are closed based on the edge information, and the skeleton information is extracted by using the Gaussian line detection method to obtain the path of the distributed sensing optical fiber.

The present invention provides a Gaussian extraction system for sub-pixel-level sensing optical fiber path, which is used for the above method, and the system includes:

An industrial camera, connected to a computer, is used to obtain images of silicon wafers for inspection and transmit them to the computer;

The computer is used to determine the visual lighting scheme according to the material characteristics, acquire the image, denoise the image through bilateral filtering, and extract the edge information of the distributed sensing optical fiber by using the sub-pixel edge detection technology based on the Canny algorithm, Closing edge pairs based on the edge information, and using a Gaussian line detection method to extract skeleton information to obtain the path of the distributed sensing optical fiber;

A workbench for fixing the industrial camera above the silicon wafer to be detected;

The square shadowless light source is used for visually illuminating the silicon wafer to be inspected arranged in the center based on a determined visual illumination scheme.

By adopting the embodiment of the present invention, it is possible to accurately detect distributed sensing optical fiber paths and extract optical fiber measuring points, and has high robustness and high precision, and can meet requirements such as accurate extraction of optical fiber measuring point coordinates and high anti-interference performance.

Description of drawings

In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or prior art. Obviously, in the following description The accompanying drawings are only some embodiments described in this specification, and those skilled in the art can also obtain other drawings according to these drawings without any creative work.

FIG. 1 is a flow chart of a Gaussian extraction method for a sub-pixel-level sensing fiber path according to an embodiment of the present invention;

Fig. 2 is a schematic side view of the system hardware of an embodiment of the present invention;

3 is a schematic top view of system hardware according to an embodiment of the present invention;

Fig. 4 is a preferred processing flowchart of the Gaussian extraction method of the sub-pixel-level sensing optical fiber path according to the embodiment of the present invention.

Detailed ways

In order to solve the above-mentioned problems in the prior art, the embodiment of the present invention provides a method and device for extracting the measuring point coordinate sequence in the length direction of the sensing fiber based on Gaussian line detection, so as to realize the strain sensing of the distributed sensing fiber on the surface of the wafer. Detect data reconstruction. According to the optical characteristics and surface height difference between the silicon wafer and the optical fiber, a square shadowless light source is used for illumination in a dark field environment to improve the discrimination of the sensing optical fiber on the wafer surface, and the preprocessed image is obtained after reducing image noise through bilateral filtering , and then adopt the sub-pixel edge detection technology based on the Canny algorithm and perform characteristic screening to eliminate the interference edge to obtain the target area of the distributed sensing fiber. Finally, the Gaussian line detection method is used to extract the fiber path. Segment the path and extract the coordinates of the optical fiber measurement points.

In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will describe the technical solutions in one or more embodiments of this specification in conjunction with the drawings in one or more embodiments of this specification The technical solution is clearly and completely described, and obviously, the described embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on one or more embodiments in this specification, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the scope of protection of this document.

method embodiment

According to an embodiment of the present invention, a sub-pixel-level sensing fiber path Gaussian extraction method is provided, which is characterized in that it is used to detect the path of a small-diameter sensing fiber laid on the surface of a silicon wafer. Figure 1 is a diagram of the present invention The flow chart of the Gaussian extraction method of the sub-pixel-level sensing fiber path in the embodiment, as shown in FIG. 1 , the Gaussian extraction method of the sub-pixel-level sensing fiber path according to the embodiment of the present invention specifically includes:

Step 101, determining a visual lighting scheme according to material properties, and acquiring an image based on the visual lighting scheme;

Step 102, denoising the image by bilateral filtering, and extracting edge information of the distributed sensing optical fiber by using sub-pixel edge detection technology based on Canny algorithm;

Step 103, closing edge pairs based on the edge information, and extracting skeleton information using a Gaussian line detection method to obtain a path of the distributed sensing optical fiber.

Step 101 specifically includes: using a square shadowless light source to illuminate from around the wafer in a dark field environment, and using a camera with sufficient resolution to obtain an image without reflection and with a high degree of discrimination between the optical fiber and the surface of the silicon wafer.

Step 102 specifically includes:

First, the acquired image is opened with the original size without stretching, the colored image is converted into a grayscale image, the area to be processed is set in the grayscale image, and the rectangular ROI area to be processed is drawn, which is recorded as region1, draw multiple regions for other useless fiber segments, and then merge the drawn regions, the merged region is recorded as region2, and obtain the image processing region. On the grayscale image, the fiber to be processed is obtained from region1-region2 The area identified by the path, and the preprocessed image is obtained;

According to Equation 1 and Equation 2, bilateral filtering performs noise on the image by bilateral filtering:

Among them, the parameters σ _d and σ _r are smoothing parameters, I(i,j) and I(k,l) are the gray levels of pixels (i,j) and (k,l) respectively, after calculating the weight Normalize them, then ID is the _grayscale of the pixel (i, j) after denoising;

Use the Gaussian filter to convolve with the image to smooth the image. The generation equation of the Gaussian filter kernel with a size of (2+1)×(2+1) is shown in Equation 3:

According to formula 4, use the sobel filter to obtain the gradient image in the x and y directions, and then obtain the gradient strength G and gradient direction θ:

Among them, G _x and G _y are the convolutions of sobel operators S _x and S _y on the 3×3 window A in the image, respectively;

Based on formula 5, the edge points are linked into edges by using the algorithm of computing non-maximum suppression and hysteresis threshold operation, and if the amplitude A of the point I _{(x, y)} of the image before detection is greater than ω _max , it is immediately accepted as an edge point , and at the same time set the gray value of the output image point O to 255, and the points whose amplitude is smaller than _ωmin are rejected, and other points are also accepted as edges if they are connected with accepted edge points;

Acquire the sub-pixel edge coordinates by quadratic polynomial fitting shown in formula 6 to finally obtain O _{(i, j)} ;

Among them, x and _y are the horizontal and vertical coordinates of the current integer coordinate edge point, GL and _GR are the left and right gradient values of the edge point, G is the gradient value of the edge point, and w is the distance from the adjacent pixel to the edge point;

Regions are selected based on area characteristics, and for each region input, the area characteristics are calculated, and if the computed characteristics of each region are within the limit (6000,2e+006), the region will be output.

Close the XLD outline, and then fill it into a region, denoted as region3.

The region region3 is opened and processed, and the edge of the fiber is extracted by subtracting the corroded region3 from the preprocessed image.

Step 103 specifically includes:

The sub-pixel-level edge is composed of edge pairs and then closed to form a region, and the image interference is reduced through threshold selection and image opening operation, that is, corrosion processing, and finally a Gaussian line detection method is used to extract the skeleton path; wherein, the Gaussian line detection method specifically includes:

Determine the Taylor quadratic polynomial parameters of each pixel in the image in the x and y directions through the partial derivative of the convolution of the image and a Gaussian mask, and calculate the line direction of each pixel;

In the second-order partial derivative Y perpendicular to the direction of the line, mark the pixel showing the local maximum value as the skeleton point, perform the hysteresis threshold operation, accept the line point whose second-order derivative is greater than Y _max , and reject the second-order derivative less than Y _min , if all other line points are adjacent to the accepted points, they will be marked as skeleton points, and finally the found line points will be connected as skeleton lines;

Based on Equation 7, the parameters Y _max and Y _min are calculated from the respective gray value contrasts P _max and P _min in the line to be extracted with the selected σ value:

Among them, the parameter σ determines the amount of smoothing to be performed by the Gaussian mask, which is proportional to the smoothness of the image, but inversely proportional to the positioning accuracy of the line. The extraction result of the skeleton line splits its path into a set of points, which is used for data reconstruction. The structure provides coordinate points;

Extract the skeleton of the Gaussian detection and extract the line width of the XLD contour line at the same time, use the LineMode to select the parabolic mode, merge the sub-pixel skeleton lines into a continuous skeleton line, and finally perform smoothing.

After executing step 103, the following processing is also carried out:

Obtain the sub-pixel skeleton coordinates extracted by Gaussian detection through the get_contour_xld operator, and then filter out the coordinates of the interval d on the skeleton according to the interval d of optical fiber data collection determined by the spatial resolution of the actual system, so as to realize the connection between the optical fiber detection data and the silicon wafer One-to-one correspondence of actual locations.

The technical solutions of the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

The detection object in the embodiment of the present invention is the sensing optical fiber laid on the surface of the silicon wafer.

As shown in Figures 2-3, the system architecture of the method of the embodiment of the present invention consists of a computer (not shown), an industrial camera 1, a 12mm focal length lens 2, a workbench 3, a shadowless light source 4, and a detection object sensing fiber 5 and Silicon wafer 6 composition. Among them, the industrial camera 1 is installed on the workbench, and can move in three directions of x, y, and z to adjust the photographing area and distance. The detection object is the sensing optical fiber 5 laid on the silicon wafer 6, which is located under the lens. A plurality of lamp beads are distributed around the shadowless light source 4 to illuminate from around the silicon wafer 6 . At the same time, the whole system is in a dark field environment to avoid problems such as wafer reflection caused by surrounding light sources. The reason for this is that the sensing fiber 5 is made of highly transparent silica material with an extremely small diameter, and the surface of the silicon wafer is smooth and the specular reflection is serious, so the side lighting can make full use of the laying of the sensing fiber 5 The height difference on the silicon wafer 6 makes the side light only reflect into the lens through the sensing fiber 5, and due to the smooth surface of the silicon wafer 6, almost no light is reflected into the area where the sensing fiber 5 is not laid. lens.

As shown in Figure 4, the method of the embodiment of the present invention specifically includes the following processing:

Image acquisition: The computer acquires the original image with high discrimination between the sensing fiber 5 and the silicon wafer 6 from the industrial lens through the Gigabit network cable.

Image preprocessing:

1. First, open image 1 with its original size without stretching, so as to facilitate subsequent image processing.

2. Convert color image 1 to grayscale image to get image 2

3. Set the area to be processed in image 2. First draw the rectangular ROI area that needs to be processed, which is recorded as region1. In addition, since only some segments of the fiber in region1 need path identification for data reconstruction, it is necessary to draw multiple regions for other useless fiber segments, and then merge the drawn regions, and the merged region is denoted as region2.

4. Acquire the image processing area. On image 2, the area that needs to be identified for the optical fiber path can be obtained by: region1-region2, and image 3 is obtained.

Filter noise reduction:

In the vision processing system, dust and other particles are easy to exist on the surface of the actual object, and noise may be generated in the process of image acquisition and transmission, which affects the quality of the image and interferes with the extraction of target information, especially in edge detection, due to random The gray value distribution characteristics of the noise are usually recognized as edges by the detection algorithm. Therefore, it is extremely necessary to preprocess the acquired original image. The display quality of the image can be enhanced by noise reduction and image contrast adjustment, so as to facilitate For subsequent processing steps such as feature extraction, commonly used traditional noise reduction methods include mean filtering, median filtering, and Gaussian filtering. The noise reduction method used in this method is bilateral filtering. As a typical nonlinear filtering, it can not only eliminate random noise, but also cause a low edge blurring effect. By smoothing the pixels in the homogeneous area, the larger contrast The edge pixels of are retained to achieve the effect, and the definition is shown in formula 1 and formula 2.

In the formula, the parameters σ _d and σ _r are smoothing parameters, I(i,j) and I(k,l) are the gray levels of pixels (i,j) and (k,l) respectively, after calculating the weight Normalize them, then ID is the _grayscale of pixel (i, j) after denoising.

Edge extraction:

Since the data reading of the sensing fiber corresponds to the length information of the fiber, in order to ensure the consistency of data reconstruction, the path extraction in the length direction of the distributed sensing fiber on the surface of the silicon wafer is performed. Due to the accuracy of the detection results The requirements are extremely high, so the edge detection technology based on the Canny algorithm is used to detect the edge of the optical fiber, and then the area is formed by closing the edge pair, and the Gaussian line detection method is used to extract the skeleton information to obtain the sensing fiber path.

Sub-pixel edge detection based on Canny algorithm: As one of the technologies with high practical application value in the field of machine vision, edge detection has very important research significance. The accuracy achieved by the traditional edge detection algorithm is at the pixel level. With the progress of the times and the development of the semiconductor industry, the accuracy required by industrial detection has also been rapidly improved. The traditional pixel-level edge detection cannot meet the detection of the sensing fiber 5 Accuracy is required, so this method chooses the sub-pixel detection technology based on Canny algorithm that divides the pixels again to improve the image resolution.

1. Use the edges_sub_pix operator to perform sub-pixel contour extraction on image 3. The filter is Canny.

The specific principle is as follows: first, the Gaussian filter is used to convolve the image to smooth the image, and the generation equation of the Gaussian filter kernel with a size of (2+1)×(2+1) is shown in Equation 3.

Then use the sobel filter to obtain the gradient image in the x and y directions, and then obtain the gradient strength G and gradient direction θ, as shown in formula 4, where G _x and G _y are the sobel operators S _x and S _y respectively in the image Convolution over a 3×3 window A.

Edge points are linked into edges using an algorithm that computes non-maximum suppression and operates similarly to hysteresis thresholding. If the amplitude A of the point I _{(x, y)} of the image before detection is greater than ω _max , it will be accepted as an edge point immediately, and the gray value of the output image point O is set to 255, while the point whose amplitude is smaller than ω _min is are rejected, and other points are also accepted as edges if they are connected to accepted edge points. The expression is as formula 5.

Finally, the sub-pixel edge coordinates are obtained by fitting the quadratic polynomial formula 6, x and _y are the horizontal and vertical coordinates of the current integer coordinate edge point, GL and _GR are the left and right gradient values of the edge point, G is the gradient value of the edge point, and w is the phase The distance from the adjacent pixel to the edge point is finally O _(i,j) .

2. Select an area according to the area feature, and calculate the area feature for each input area. If the computed properties of each region are within the limit (6000,2e+006), the region will be output.

3. First close the XLD outline, and then fill it into a region, which is recorded as region3.

4. Perform an opening operation on the region region3, and extract the edge of the optical fiber by subtracting the etched region3 from the image 3.

Gaussian detection extracts the skeleton:

The above sub-pixel-level edges are combined to form edge pairs and then closed to form an area, and then the image interference is further reduced through threshold selection and image opening operation (that is, erosion processing), and finally the Gaussian line detection method is used to extract the skeleton path.

The detection method is to determine the Taylor quadratic polynomial parameters of each pixel point in the image in the x and y directions through the partial derivative of the convolution of the image and a Gaussian mask, so as to calculate the line direction of each pixel point. Pixels exhibiting local maxima in the second-order partial derivative Y perpendicular to the line direction are marked as skeleton points. Similar to the hysteresis threshold operation in Equation 5, accept the line points whose second derivative is greater than Y _max , reject the points whose second derivative is less than Y _min , and all other line points are also marked if they are adjacent to the accepted point is the skeleton point, and finally connect the found line points as the skeleton line. The parameters Y _max and Y _min can be calculated from the respective gray value contrasts P _max and P _min in the lines to be extracted and the selected value of σ, as shown in Equation 7, where the parameter σ determines the Gaussian mask to perform The amount of smoothing, which is proportional to the smoothness of the image, but inversely proportional to the positioning accuracy of the lines, may cause inaccurate positioning of the extracted lines. The extraction result of the skeleton line can be split into point sets to provide coordinate points for data reconstruction.

Specifically, the Gaussian detection extracts the skeleton and the line width of the XLD contour line. Since the edge of the picture is sharper here, the LineMode selects the parabolic mode. Finally, the sub-pixel-level skeleton lines are merged into a continuous skeleton line, and finally smoothed.

Output coordinates:

Since the data reading of the sensing optical fiber 5 is corresponding to the length information of the optical fiber, the measuring point position (actual measurement position) determined according to the actual system spatial resolution of the sensing optical fiber 5 can be found corresponding to the extracted skeleton line. coordinates, and then output. So far, the one-to-one correspondence between the measurement data of the sensing optical fiber 5 and the specific position of the silicon wafer 6 can be realized.

Specifically, the sub-pixel-level skeleton coordinates extracted by Gaussian detection can be obtained through the get_contour_xld operator, and then the coordinates of the interval d on the skeleton are screened out according to the interval d of the measuring points determined by the system spatial resolution of the actual optical fiber, and finally input .

In summary, the embodiment of the present invention realizes sub-pixel-level high-precision path detection and optical fiber measuring point coordinate extraction of the sensing optical fiber laid on the silicon wafer. The line width of the extracted sensing fiber is analyzed, and the line width of the single side is evenly distributed, and the system stability is high. According to the calculation of the actual diameter of the sensing fiber, the accuracy of the detection method can reach 10 microns, which also shows the high stability and high precision of the detection method of the Gaussian line skeleton path. The technical solution of the embodiment of the present invention has high robustness. It has been verified by experiments that although the actually laid sensing optical fiber 5 is poorly bonded and there is similar edge information interference around the sensing optical fiber, this method can still accurately identify the actual sensing optical fiber 5. The skeleton path of the sensing fiber 5 is free from defects and accidental disturbances.

System embodiment

According to an embodiment of the present invention, a sub-pixel-level sensing fiber path Gaussian extraction system is provided, which is characterized in that it is used for the method described in the above-mentioned method embodiment, and the sub-pixel-level sensing fiber path according to the embodiment of the present invention The Gaussian extraction system specifically includes:

The industrial camera is connected with the computer, and is used to obtain the image of the silicon wafer for detection and transmit it to the computer; the industrial camera is a five-megapixel camera with a resolution of 3856×2764, and the focal length of the lens is 12mm.

The computer is used to determine the visual lighting scheme according to the material characteristics, acquire the image, denoise the image through bilateral filtering, and extract the edge information of the distributed sensing optical fiber by using the sub-pixel edge detection technology based on the Canny algorithm, Closing edge pairs based on the edge information, and using a Gaussian line detection method to extract skeleton information to obtain the path of the distributed sensing optical fiber;

A workbench for fixing the industrial camera above the silicon wafer to be detected;

The square shadowless light source is used for visually illuminating the silicon wafer to be inspected arranged in the center based on a determined visual illumination scheme. The visual lighting scheme specifically includes: using a square shadowless light source to illuminate from the sides of the wafer in a dark field environment, and using a camera with sufficient resolution to obtain an image without reflection and with a high degree of discrimination between the optical fiber and the surface of the silicon wafer.

The computer is specifically used for:

First, the acquired image is opened with the original size without stretching, the colored image is converted into a grayscale image, the area to be processed is set in the grayscale image, and the rectangular ROI area to be processed is drawn, which is recorded as region1, draw multiple regions for other useless fiber segments, and then merge the drawn regions, the merged region is recorded as region2, and obtain the image processing region. On the grayscale image, the fiber to be processed is obtained from region1-region2 The area identified by the path, and the preprocessed image is obtained;

According to Equation 1 and Equation 2, bilateral filtering performs noise on the image by bilateral filtering:

Among them, the parameters σ _d and σ _r are smoothing parameters, I(i,j) and I(k,l) are the gray levels of pixels (i,j) and (k,l) respectively, after calculating the weight Normalize them, then ID is the _grayscale of the pixel (i, j) after denoising;

Use the Gaussian filter to convolve with the image to smooth the image. The generation equation of the Gaussian filter kernel with a size of (2+1)×(2+1) is shown in Equation 3:

According to formula 4, use the sobel filter to obtain the gradient image in the x and y directions, and then obtain the gradient strength G and gradient direction θ:

Among them, G _x and G _y are the convolutions of sobel operators S _x and S _y on the 3×3 window A in the image, respectively;

Based on formula 5, the edge points are linked into edges by using the algorithm of computing non-maximum suppression and hysteresis threshold operation, and if the amplitude A of the point I _{(x, y)} of the image before detection is greater than ω _max , it is immediately accepted as an edge point , and at the same time set the gray value of the output image point O to 255, and the points whose amplitude is smaller than _ωmin are rejected, and other points are also accepted as edges if they are connected with accepted edge points;

Acquire the sub-pixel edge coordinates by quadratic polynomial fitting shown in formula 6 to finally obtain O _{(i, j)} ;

Among them, x and _y are the horizontal and vertical coordinates of the current integer coordinate edge point, GL and _GR are the left and right gradient values of the edge point, G is the gradient value of the edge point, and w is the distance from the adjacent pixel to the edge point;

Regions are selected based on area characteristics, and for each region input, the area characteristics are calculated, and if the computed characteristics of each region are within the limit (6000,2e+006), the region will be output.

Close the XLD outline, and then fill it into a region, denoted as region3.

Perform an open operation on the region region3, and extract the edge of the fiber by subtracting the corroded region3 from the preprocessed image;

The sub-pixel-level edge is composed of edge pairs and then closed to form a region, and the image interference is reduced through threshold selection and image opening operation, that is, corrosion processing, and finally a Gaussian line detection method is used to extract the skeleton path; wherein, the Gaussian line detection method specifically includes:

Determine the Taylor quadratic polynomial parameters of each pixel in the image in the x and y directions through the partial derivative of the convolution of the image and a Gaussian mask, and calculate the line direction of each pixel;

In the second-order partial derivative Y perpendicular to the direction of the line, mark the pixel showing the local maximum value as the skeleton point, perform the hysteresis threshold operation, accept the line point whose second-order derivative is greater than Y _max , and reject the second-order derivative less than Y _min , if all other line points are adjacent to the accepted points, they will be marked as skeleton points, and finally the found line points will be connected as skeleton lines;

Based on Equation 7, the parameters Y _max and Y _min are calculated from the respective gray value contrasts P _max and P _min in the line to be extracted with the selected σ value:

Among them, the parameter σ determines the amount of smoothing to be performed by the Gaussian mask, which is proportional to the smoothness of the image, but inversely proportional to the positioning accuracy of the line. The extraction result of the skeleton line splits its path into a set of points, which is used for data reconstruction. The structure provides coordinate points;

Extract the skeleton of the Gaussian detection and extract the line width of the XLD contour line at the same time, use the LineMode to select the parabolic mode, merge the sub-pixel skeleton lines into a continuous skeleton line, and finally perform smoothing.

The computer is further used to:

Obtain the sub-pixel skeleton coordinates extracted by Gaussian detection through the get_contour_xld operator, and then filter out the coordinates of the interval d on the skeleton according to the interval d of the actual optical fiber data collection, so as to realize the one-to-one correspondence between the optical fiber detection data and the actual position of the silicon wafer .

In other words, the hardware part of the system is mainly composed of computers, industrial cameras, workbenches, and square shadowless light sources. The industrial camera is a 5-megapixel camera with a resolution of 3856×2764 and a focal length of the lens of 12mm. It is connected to the computer through a Gigabit network cable. The silicon wafer to be inspected is placed in the center of the square shadowless light source, and the industrial camera is installed above the silicon wafer through the workbench to take pictures. In order to improve the discrimination of distributed sensing optical fiber paths on the surface of silicon wafers, it is necessary to design a suitable visual lighting scheme according to the material characteristics of silicon wafers and optical fibers, so as to perform image preprocessing for grayscale and noise reduction after image acquisition.

The optical fiber is usually made of high-transparency silica material, and its volume is very small, so when it is laid on the surface of the silicon wafer, there will be a phenomenon of low visual distinction, and the surface of the silicon wafer is extremely smooth, with obvious Specular reflection phenomenon, so it is easily disturbed by ambient light, which also increases the requirements for light source selection. To prevent the camera from acquiring images with sufficient information, a suitable lighting scheme needs to be designed.

The lighting solution of the embodiment of the present invention is to use a square shadowless light source for side lighting from around the wafer in a dark field environment. The problem of ambient light interference is solved by arranging the dark field environment, and the height difference formed by laying the sensing fiber on the surface of the wafer is cleverly used, and the problem of low discrimination of the fiber on the surface of the wafer is solved by side lighting, while avoiding The specular reflection effect produced by the wafer when the light source is illuminated from the front is eliminated, and the original image to be detected is finally obtained.

As shown in Figures 2-3, the system architecture of the method of the embodiment of the present invention consists of a computer (not shown), an industrial camera 1, a 12mm focal length lens 2, a workbench 3, a shadowless light source 4, and a detection object sensing fiber 5 and Silicon wafer 6 composition. Among them, the industrial camera 1 is installed on the workbench, and can move in three directions of x, y, and z to adjust the photographing area and distance. The detection object is the sensing optical fiber 5 laid on the silicon wafer 6, which is located under the lens. A plurality of lamp beads are distributed around the shadowless light source 4 to illuminate from around the silicon wafer 6 . At the same time, the whole system is in a dark field environment to avoid problems such as wafer reflection caused by surrounding light sources. The reason for this is that the sensing fiber 5 is made of highly transparent silica material with an extremely small diameter, and the surface of the silicon wafer is smooth and the specular reflection is serious, so the side lighting can make full use of the laying of the sensing fiber 5 The height difference on the silicon wafer 6 makes the side light only reflect into the lens through the sensing fiber 5, and due to the smooth surface of the silicon wafer 6, almost no light is reflected into the area where the sensing fiber 5 is not laid. lens.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (10)

1. A sub-pixel level sensing optical fiber path Gaussian extraction method is used for carrying out path detection on a sensing optical fiber laid on the surface of a silicon wafer, and comprises the following steps:

determining a visual lighting scheme according to the material characteristics, and acquiring an image based on the visual lighting scheme;

denoising the image through bilateral filtering, and extracting edge information of the distributed sensing optical fiber by adopting a sub-pixel edge detection technology based on a Canny algorithm;

and closing the edge pairs based on the edge information, and extracting skeleton information by using a Gaussian line detection method to obtain the path of the distributed sensing optical fiber.

2. The method according to claim 1, wherein a visual illumination scheme is determined from the material properties, the acquiring of the image based on the visual illumination scheme comprising in particular:

and in a dark field environment, a square shadowless light source is used for carrying out side illumination from the periphery of the wafer, and an image which is free of reflection and has large discrimination between the optical fiber and the surface of the silicon wafer is obtained by a camera with enough resolution.

3. The method according to claim 1, wherein the noise is applied to the image through bilateral filtering, and the extracting of the edge information of the distributed sensing optical fiber by using a sub-pixel edge detection technology based on a Canny algorithm specifically comprises:

firstly, opening the acquired image in an original size without stretching, converting the colored image into a gray image, setting a region to be processed in the gray image, drawing a rectangular ROI (region of interest) region to be processed, marking the rectangular ROI region as region, drawing a plurality of regions of other useless optical fiber line segments, merging the plurality of drawn regions, marking the merged region as region2, acquiring an image processing region, and acquiring the region to be subjected to optical fiber path identification on the gray image by using the regions 1-2 to obtain a preprocessed image;

according to formula 1 and formula 2, bilateral filtering performs noise on the image by bilateral filtering:

wherein the parameter σ _d And σ _r For smoothing the parameters, I (I, j) and I (k, l) are the gray levels of the pixels (I, j) and (k, l), respectively, and after the weights are calculated, they are normalized, so that I _D The gray level of the pixel point (i, j) after noise reduction;

convolution of the image with a Gaussian filter to smooth the image, the generation equation of the Gaussian filter kernel of size (2k + 1) × (2k + 1) is as in equation 3:

according to equation 4, the gradient image in the x and y directions is obtained by using a sobel filter, and then the gradient strength G and the gradient direction θ are obtained:

wherein, G _x And G _y Respectively sobel operator S _x And S _y Convolution of a 3 × 3 window a in the image;

based on formula 5, the edge points are linked into edges by using an algorithm for calculating non-maximum suppression and hysteresis threshold operation, and a point I of the previous image is detected _(x，y) If the amplitude A of (a) is larger than omega _max Is immediately accepted as an edge point and the gray value of the output image point O is set to 255 with an amplitude smaller than ω _min The other points are rejected, and if connected with the accepted edge points, the other points are also accepted as edges;

obtaining sub-pixel edge coordinates by fitting a quadratic polynomial shown in formula 6 to finally obtain O _(i，j) ；

Wherein x and y are the horizontal and vertical coordinates of the edge point of the current integer coordinate, G _L And G _R The gradient value of the edge point is left and right, G is the gradient value of the edge point, and w is the distance from the adjacent pixel to the edge point;

regions are selected according to the area characteristics, and for each region input, the area characteristics are calculated, and if the calculation characteristics of each region are within the limit (6000,2e + 006), the region is output.

The XLD contour is closed and then filled in as a region, denoted region3.

The region3 is subjected to an opening operation process, and the fiber edge is extracted by subtracting the region3 after the etching from the preprocessed image.

4. The method of claim 1, wherein closing edge pairs based on the edge information and extracting skeleton information using a gaussian line detection method to obtain paths of the distributed sensing fiber specifically comprises:

forming an edge pair rear closing forming area through sub-pixel level edges, reducing image interference through threshold selection and image opening operation, namely corrosion treatment, and finally extracting a skeleton path by using a Gaussian line detection method; the Gaussian line detection method specifically comprises the following steps:

determining Taylor quadratic polynomial parameters of each pixel point in the image in the x and y directions through a partial derivative of convolution of the image and a Gaussian mask, and calculating the line direction of each pixel point;

marking pixels showing local maximum values as skeleton points in a second-order partial derivative Y vertical to the line direction, performing hysteresis threshold operation, and receiving the condition that the second-order derivative is larger than Y _max Rejecting second derivative less than Y _min If all other line points are adjacent to the accepted points, the other line points are marked as skeleton points, and finally the found line points are connected into skeleton lines;

based on equation 7, the respective gray value contrasts P from the lines to be extracted _max And P _min Calculating the parameter Y from the selected sigma value _max And Y _min ：

The parameter sigma determines the smoothing quantity to be executed by the Gaussian mask, is in direct proportion to the smoothness of the image, but is in inverse proportion to the positioning accuracy of the lines, and the path is divided into point sets by the extraction result of the skeleton line, so that coordinate points are provided for data reconstruction;

and extracting the skeleton in Gaussian detection, simultaneously extracting the line width of an XLD contour line, selecting a parabolic mode by adopting LineMode, combining sub-pixel level skeleton lines into a continuous skeleton line, and finally performing smoothing treatment.

5. The method of claim 1, further comprising:

and obtaining sub-pixel level skeleton coordinates extracted through Gaussian detection through a get _ contourr _ xld operator, and screening out the coordinates of the interval d on the skeleton according to the interval d of optical fiber data acquisition determined by the spatial resolution of an actual optical fiber demodulator, so as to realize the one-to-one correspondence of the optical fiber detection data and the actual position of the silicon wafer.

6. A sub-pixel level sensing fiber path gaussian extraction system for use in the method of any of the preceding claims 1 to 5, the system comprising:

the industrial camera is connected with the computer and used for acquiring an image of the detected silicon wafer and transmitting the image to the computer;

the computer is used for determining a visual lighting scheme according to material characteristics, obtaining the image, reducing noise of the image through bilateral filtering, extracting edge information of the distributed sensing optical fiber by adopting a sub-pixel edge detection technology based on a Canny algorithm, closing an edge pair based on the edge information, and extracting skeleton information by using a Gaussian line detection method to obtain a path of the distributed sensing optical fiber;

the workbench is used for fixing the industrial camera above a silicon wafer to be detected;

and the square shadowless light source is used for visually illuminating the silicon wafer to be detected arranged in the center of the square shadowless light source based on the determined visual illumination scheme.

7. The system of claim 6, wherein the industrial camera is a five million pixel camera with a resolution of 3856 x 2764 and a lens focal length of 12mm.

8. The system according to claim 6, characterized in that the visual lighting scheme is in particular: and in a dark field environment, a square shadowless light source is used for carrying out side illumination from the periphery of the wafer, and an image which is free of reflection and has large discrimination between the optical fiber and the surface of the silicon wafer is obtained by a camera with enough resolution.

9. The system of claim 6, wherein the computer is specifically configured to:

firstly, opening the acquired image in an original size without stretching, converting the colored image into a gray image, setting a region to be processed in the gray image, drawing a rectangular ROI (region of interest) region to be processed, marking the rectangular ROI region as region1, drawing a plurality of regions of other useless optical fiber line segments, merging the drawn plurality of regions, marking the merged region as region2, acquiring an image processing region, and acquiring the region to be subjected to optical fiber path identification on the gray image by using the region1-region2 to obtain a preprocessed image;

according to formula 1 and formula 2, bilateral filtering performs noise on the image by bilateral filtering:

wherein the parameter σ _d And σ _r For smoothing the parameters, I (I, j) and I (k, l) are the gray levels of the pixels (I, j) and (k, l), respectively, and after the weights are calculated, they are normalized, so that I _D The gray level of the pixel point (i, j) after noise reduction;

convolving with the image using a Gaussian filter to smooth the image, the generation equation of the Gaussian filter kernel of size (2k + 1) × (2k + 1) is as in equation 3:

according to equation 4, the gradient image in the x and y directions is obtained by using a sobel filter, and then the gradient strength G and the gradient direction θ are obtained:

wherein, G _x And G _y Respectively sobel operator S _x And S _y Convolution of a 3 × 3 window a in the image;

based on equation 5, the edge points are linked into edges by an algorithm that calculates non-maximum suppression and hysteresis threshold operations, and point I of the pre-image is detected _(x，y) If the amplitude A of (a) is larger than omega _max Is immediately accepted as an edge point and the gray value of the output image point O is set to 255 with an amplitude smaller than ω _min The other points are rejected, and if connected with the accepted edge points, the other points are also accepted as edges;

obtaining sub-pixel edge coordinates by fitting a quadratic polynomial shown in formula 6 to finally obtain O _(i，j) ；

Wherein x and y are the horizontal and vertical coordinates of the edge point of the current integer coordinate, G _L And G _R The gradient value of the edge point is left and right, G is the gradient value of the edge point, and w is the distance from the adjacent pixel to the edge point;

regions are selected according to the area characteristics, and for each region that is input, the area characteristics are calculated and if the calculated characteristics of each region are within the limit (6000, 2e + 006), that region will be output.

The XLD contour is closed and then filled in as a region, noted region3.

Performing opening operation processing on the region3, and extracting the edge of the optical fiber by subtracting the region3 subjected to corrosion from the preprocessed image;

forming an edge pair rear closing forming area through sub-pixel level edges, reducing image interference through threshold selection and image opening operation, namely corrosion treatment, and finally extracting a skeleton path by using a Gaussian line detection method; the Gaussian line detection method specifically comprises the following steps:

determining Taylor quadratic polynomial parameters of each pixel point in the image in the x direction and the y direction through a partial derivative of convolution of the image and a Gaussian mask, and calculating the line direction of each pixel point;

marking pixels showing local maximum values as skeleton points in a second-order partial derivative Y vertical to the line direction, performing hysteresis threshold operation, and receiving the condition that the second-order derivative is larger than Y _max Rejecting second derivative less than Y _min If all other line points are adjacent to the accepted points, the other line points are marked as skeleton points, and finally the found line points are connected into skeleton lines;

based on equation 7, the respective gray value contrasts P from the lines to be extracted _max And P _min Calculating the parameter Y from the selected sigma value _max And Y _min ：

The parameter sigma determines the smooth quantity to be executed by the Gaussian mask, the smooth quantity is in direct proportion to the smoothness of the image and in inverse proportion to the positioning accuracy of the lines, the path of the extracted result of the skeleton line is divided into a point set by the extracted result of the skeleton line, and coordinate points are provided for data reconstruction;

and extracting the skeleton in Gaussian detection, simultaneously extracting the line width of an XLD contour line, selecting a parabolic mode by adopting LineMode, combining sub-pixel level skeleton lines into a continuous skeleton line, and finally performing smoothing treatment.

10. The system of claim 6, wherein the computer is further configured to:

and obtaining sub-pixel level skeleton coordinates extracted through Gaussian detection through a get _ contourr _ xld operator, and screening out the coordinates of the interval d on the skeleton according to the interval d of optical fiber data acquisition determined by the spatial resolution of the actual demodulator, so as to realize the one-to-one correspondence of the optical fiber detection data and the actual position of the silicon wafer.

Images