CN116823791A

CN116823791A - PIN defect detection method, device, equipment and computer readable storage medium

Info

Publication number: CN116823791A
Application number: CN202310831958.4A
Authority: CN
Inventors: 蔡光湖; 曹鹏飞; 刘枢; 吕江波; 沈小勇
Original assignee: Shenzhen Smartmore Technology Co Ltd
Current assignee: Shenzhen Smartmore Technology Co Ltd
Priority date: 2023-07-06
Filing date: 2023-07-06
Publication date: 2023-09-29

Abstract

The embodiment of the application discloses a PIN defect detection method, a PIN defect detection device, PIN defect detection equipment and a computer readable storage medium, wherein the PIN defect detection method comprises the following steps: obtaining a depth map of a product to be detected through a 3D line laser camera, wherein the product to be detected comprises at least one PIN needle and at least two characteristic marks; the at least two feature markers are used for establishing a reference coordinate system; converting floating point type image data of the depth map into integer type image data; fitting a reference plane according to integer image data corresponding to the reference plane area of the PIN needle and the reference plane area of the characteristic mark respectively, and establishing a reference coordinate system on the reference plane according to the integer image data and the area of the characteristic mark; respectively obtaining the height and the position degree of each PIN needle according to the areas of the PIN needles in a reference coordinate system, wherein the position degree comprises an X-axis position degree and a Y-axis position degree; and obtaining a detection result of whether each PIN needle is normal or not according to the height and the position degree of each PIN needle. The method has small calculated amount and can improve the accuracy and efficiency of PIN needle defect detection.

Description

PIN defect detection method, device, equipment and computer readable storage medium

Technical Field

The application relates to the technical field of defect detection, in particular to a PIN needle defect detection method, a PIN needle defect detection device, PIN needle defect detection equipment and a computer readable storage medium.

Background

The PIN is a very common component for connecting with external equipment in electronic equipment, is a metal substance for completing conductive transmission of electric signals, has the advantages of reliable communication connection and convenient disassembly and assembly, and is widely applied to industries such as printed circuit boards (PCB, printed Circuit Board), 3C (computer, communication and consumer electronics) and automobile electronics.

The equipment of pegging graft PIN needle can be equipped with corresponding hole, and the PIN needle inserts the hole and realizes connecting, along with the rapid development of application trade, the requirement for the quality of PIN needle is higher and higher.

The deviation of the height and the position of the PIN needle can cause the problems of abnormal plugging, abnormal communication and the like, so that the information transmission is unstable. Therefore, the defect detection of the PIN needle is particularly important.

In the prior art, two groups of 2D cameras are utilized to obtain the images of the PIN needle, the structure is complex, the images obtained by the two groups of 2D cameras are required to be integrated, and the efficiency is low. Moreover, the influence on the detection result is large when the relative shooting positions of the two groups of 2D cameras and the product fluctuate, so that the PIN defect detection is inaccurate.

Disclosure of Invention

In view of this, embodiments of the present application provide a method, an apparatus, a device, and a computer readable storage medium for detecting PIN defects, which can improve accuracy and efficiency of PIN defect detection.

In a first aspect, an embodiment of the present application provides a PIN defect detection method, including:

obtaining a depth map of a product to be detected through a 3D line laser camera, wherein the product to be detected comprises at least one PIN needle and at least two characteristic marks; the at least two feature markers are used for establishing a reference coordinate system;

converting floating point type image data of the depth map into integer type image data;

fitting a reference plane according to integer image data corresponding to the reference plane area of the PIN needle and the reference plane area of the characteristic mark respectively, and establishing a reference coordinate system on the reference plane according to the integer image data and the area of the characteristic mark;

respectively obtaining the height and the position degree of each PIN needle according to the areas of the PIN needles in a reference coordinate system, wherein the position degree comprises an X-axis position degree and a Y-axis position degree; and obtaining a detection result of whether each PIN needle is normal or not according to the height and the position degree of each PIN needle.

In a second aspect, an embodiment of the present application further provides a PIN defect detection device, including:

The obtaining module is used for obtaining a depth image of a product to be detected through the 3D line laser camera, wherein the product to be detected comprises at least one PIN needle and at least two characteristic marks; the at least two feature markers are used for establishing a reference coordinate system;

the preprocessing module is used for converting floating point image data of the depth map into integer image data;

the fitting module is used for fitting the reference surface according to the integer image data respectively corresponding to the reference surface area of the PIN needle and the reference surface area of the characteristic mark;

the construction module is used for establishing a reference coordinate system on the reference plane according to the integer image data and the region of the characteristic mark;

the calculation module is used for respectively obtaining the height and the position degree of each PIN needle according to the areas of the PIN needles in the reference coordinate system, wherein the position degree comprises an X-axis position degree and a Y-axis position degree;

and the detection module is used for obtaining a detection result of whether each PIN needle is normal or not according to the height and the position degree of each PIN needle.

In a third aspect, embodiments of the present application also provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the processor implementing the steps of the above method when executing the computer program.

In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the above method.

In a fifth aspect, embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements the steps of the above method.

From this, the embodiment of the application has the following beneficial effects:

according to the PIN needle defect detection method provided by the embodiment of the application, three-dimensional stereo data of a product to be detected can be obtained by using only one image by using the 3D line laser camera, the height of the PIN needle can be obtained by directly reflecting the height of the PIN needle by using the depth value of the depth map, the distance between the highest point of the PIN needle and a reference plane, namely the distance between the point and the plane, the height of the PIN needle can be obtained, and the position degree can be obtained by using the coordinate of the PIN needle and the distance between the X axis and the Y axis in the reference coordinate system. In order to utilize the 2D maturation algorithm, the floating point type data is firstly converted into integer type data, and the calculation is simple and accurate. Compared with the traditional 2D vision, the method provided by the application only processes one image, has small calculation amount, can detect the position degree and the height simultaneously by only using one camera, and can improve the accuracy and the efficiency of PIN needle defect detection.

Drawings

Fig. 1 is an application scenario diagram of a PIN defect detection method provided by an embodiment of the present application;

Fig. 2 is a schematic diagram of a tool for PIN image acquisition according to an embodiment of the present application;

FIG. 3 is a schematic top view of a product to be tested according to an embodiment of the present application;

FIG. 4 is a flowchart of a method for detecting a PIN defect according to an embodiment of the present application;

FIG. 5 is a depth map of a PIN needle for a product to be tested according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a reference surface fit provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a position correction according to an embodiment of the present application;

fig. 8 is a schematic diagram of PIN calibration according to an embodiment of the present application;

fig. 9 is a schematic diagram of a PIN defect detecting device according to an embodiment of the present application;

FIG. 10 is a diagram illustrating an internal architecture of a computer device according to an embodiment of the present application;

FIG. 11 is a diagram illustrating an internal architecture of another computer device provided by an embodiment of the present application;

fig. 12 is an internal structural diagram of a computer-readable storage medium according to an embodiment of the present application.

Detailed Description

In order to facilitate understanding of the technical solution of the embodiments of the present application, technical terms related to the embodiments of the present application will be described first.

3D line laser camera: a three-dimensional camera for capturing laser line information projected on the surface of an object by a laser generator through a single or multiple image sensors and reconstructing contour information of the surface of the object based on a triangulation principle, wherein each laser point contains abundant measurement information such as coordinates, brightness and the like.

Depth map: and sampling the surface of the target in the X and Y directions, and forming an image by using the Z-direction information through gray scale or color.

XY-axis resolution XY-advanced resolution: the real physical scale (unit: mm) which can be resolved by the three-dimensional camera in the XY coordinate axis direction.

Z-axis resolution/depth resolution Z-axis resolution: the real physical scale (unit: mm) that the three-dimensional camera can distinguish in the Z coordinate axis direction.

Dynamic repeatability: the corresponding measuring items of a piece of material are tested 32 times in a picking and placing mode, data jumping of the corresponding measuring items is observed, and the smaller the data jumping is, the better the dynamic repeatability is.

ROI: region of Interest, the region of interest, in machine vision, the region to be processed is outlined from the processed image in the form of a square, circle, ellipse, irregular polygon, etc., which is called the region of interest.

Gray threshold: all luminance values in the image are classified into two categories, within a threshold range and not within the threshold range according to a specified luminance value (i.e., threshold).

Morphological corrosion treatment: mathematical morphology (Mathematical morphology) is an image analysis discipline based on lattice and topology, and is the fundamental theory of mathematical morphological image processing. The basic operation includes: binary corrosion and expansion, binary opening and closing operation, skeleton extraction, extreme corrosion, hit-miss conversion, morphological gradient, top-hat conversion, particle analysis, drainage basin conversion, gray value corrosion and expansion, gray value opening and closing operation, gray value morphological gradient and the like. Corrosion operation is generally used for deburring purposes.

Median filtering: the median filtering is a nonlinear signal processing technology capable of effectively suppressing noise based on a sequencing statistical theory, and the basic principle of median filtering is to replace the value of a point in a digital image or a digital sequence with the median of the point values in a neighborhood of the point, so that surrounding pixel values are close to a true value, and isolated noise points are eliminated.

The PIN needle defect detection method provided by the embodiment of the application can be applied to an application environment shown in figure 1. Wherein the terminal 102 communicates with the server 104 via a communication network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, etc. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.

In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of embodiments of the application will be rendered by reference to the appended drawings and appended drawings.

First, a detection tool of the PIN defect detection method provided by the embodiment of the application is introduced.

Referring to fig. 2, the figure is a schematic diagram of a tool for acquiring a PIN image according to an embodiment of the present application.

The embodiment of the application is not particularly limited to the type of the product to be tested, and any product which can comprise the PIN needle can be used, so that the product to be tested is taken as the transformer in the embodiment of the application, namely whether the PIN needle of the transformer has defects is detected. The defect detection of the PIN needles includes the height of each PIN needle, as well as the location, i.e. the coordinates of the corresponding x and y.

Fig. 2 is only an exemplary data collection tool, including two directions of X and Y, a gripper grabs a product 10 to be measured, the gripper is disposed in the X axis direction, the 3D line laser camera 20 is fixedly mounted, the gripper can move along the Y axis direction, the product 10 to be measured scans once within the field of view of the 3D line laser camera 20, and the 3D line laser camera 20 transmits depth map data of the product 10 to be measured to a memory of an industrial personal computer (IPC, industrial Personal Computer) through a network cable. The IPC can analyze the depth map data, set a height threshold and a position threshold according to the specification of the PIN needle, judge whether the PIN needle is qualified or not, and finally obtain a detection result; the detection result can be displayed on an interface or transmitted to a background monitoring system.

For ease of understanding, the PIN of the transformer will be described by way of example with reference to the accompanying drawings. Referring to fig. 3, a schematic diagram of a PIN of a product to be tested according to an embodiment of the present application is shown.

Fig. 3 is a top view of only the PIN needle. Taking the transformer including 4 PINs to be tested as an example and respectively represented by A, B, C and D, the embodiment of the application does not specifically limit the number of PINs, the product to be tested includes at least one PIN to be tested, and in addition, the position relation among the PINs is not specifically limited, and the detection of the application is a defect of each PIN. The shape of the cross section of the PIN needle is not particularly limited.

In addition, in order to establish the reference plane during detection, the application also does not specifically limit the type of the feature mark, and the feature mark can be set according to the needs of a user, for example, the feature point, the feature column or other types with the markedness can be used. In fig. 3, two feature columns on the product to be measured 10 are taken as feature marks, namely, a first feature column 11 and a second feature column 22, respectively, and it should be understood that the first feature column 11 and the second feature column 22 have a certain height, and the heights of the two feature columns are the same for convenience in calculation. The embodiment of the present application does not particularly limit the relationship between the heights of the first and second feature posts 11 and 22 and the PIN needles.

The embodiment of the application does not particularly limit the position relation between the two positioning columns and the PIN needle to be detected, and in order to obtain more accurate data, for example, one possible mode is that the first positioning column is positioned in a space surrounded by all the PIN needles, and the second positioning column is positioned outside the space surrounded by all the PIN needles; the PIN needle is the PIN needle of the transformer.

It should be understood that the image shot by the 3D line laser camera includes much content, the present application is only for detecting PIN needles, and therefore, a part of the area is selected from the shot image, and this area may be referred to as a core area for convenience of description, and the core area includes 4 PIN needle areas to be measured and 2 location post areas for constructing a position reference, and at least two areas for constructing a reference plane.

The implementation process of the PIN needle defect detection method provided by the embodiment of the application is specifically described below with reference to the accompanying drawings.

Referring to fig. 4, a flowchart of a PIN defect detection method according to an embodiment of the present application is shown. The PIN needle defect detection method provided by the embodiment of the application comprises the following steps:

s401: obtaining a depth map of a product to be detected through a 3D line laser camera, wherein the product to be detected comprises at least one PIN needle and at least two characteristic marks; at least two signature are used to establish a reference coordinate system.

It should be appreciated that the present application obtains a 3D perspective view of the product to be measured, not a two-dimensional image obtained by a conventional 2D camera. Only one image is needed to be obtained by using the 3D line laser camera, and the height and the position degree of the PIN needle can be obtained according to the depth map data of the image. A specific photographing manner may be referred to in fig. 2, and will not be described herein.

Since the height and the position degree of the PIN needle are calculated, it is necessary to obtain the coordinates of the height and the coordinates of the position degree, which is the coordinates of X and Y of the PIN needle. The height corresponds to the Z coordinate. The specific coordinate values need to have a reference coordinate system. And the 3D stereo image also needs to establish a reference plane, otherwise, when the data do not take the same plane as the reference, the plane inclination may occur, so that the measurement result is deviated. Therefore, the application needs to establish the reference surface according to at least two characteristic marks on the product to be tested.

S402: floating point type image data of the depth map is converted into integer type image data.

The gray values of the depth map data are conventionally converted into floating point image values (i.e., into actual physical height values) for subsequent processing, but such processing cannot well utilize the rich algorithm library in 2D vision, such as shape module matching, and the like. The 3D algorithm library is limited. The application converts floating point type image data into integer type image data. Specifically, the gray value in the depth map is divided by the Z-axis resolution and rounded to be the gray value of the current position, for example, the Z-axis resolution is 0.005mm, if the gray value of a certain position (x 1, y 1) is 1.005mm, floating point image data of the depth map is converted into integer image data 201, namely, the depth map with integer gray value, so that the depth map can be processed later by using a rich 2D vision algorithm library. The above resolution of the Z-axis is merely illustrative and other resolutions may be provided.

As the height and the position of the PIN needle are finally required to output actual physical values, the unit conversion is required to be carried out in the subsequent processing, and the resolution values of the X axis, the Y axis and the Z axis and the integer image data can be combined into one data type during the pretreatment, so that the subsequent use is convenient.

Because the X-axis resolution of the 3D line laser camera is a fixed parameter determined by the camera, the Y-axis resolution varies according to the scanning speed, and therefore, the X-axis resolution and the Y-axis resolution are inconsistent, and the scanning speed is usually increased at the expense of the Y-axis resolution, which results in poor accuracy of the Y-axis resolution compared with the X-axis resolution, and a compression phenomenon or a stretching phenomenon occurs on the final imaging effect. For example, a circular object, the imaging effect after scanning is elliptical, which is disadvantageous for subsequent calculation. In order to solve the above problems, the present application performs scaling processing on an image, and performs scaling processing on original depth map data according to a multiple relationship between an X-axis resolution and a Y-axis resolution, for example, the original integer image data is an image with 1024 columns by 500 rows, the X-axis resolution is 0.015mm, the Y-axis resolution is 0.03mm, that is, the two are in a two-time relationship, and the size of the image with 1024 columns by 500 rows is enlarged to 1024 columns by 1000 rows by using a nearest neighbor interpolation algorithm, at this time, the XY resolution is 0.015mm, so as to ensure that the final imaging effect is consistent with the shape of a real object.

S403: and fitting a reference plane according to the integer image data corresponding to the reference plane region of the PIN needle and the reference plane region of the characteristic mark respectively, and establishing a reference coordinate system on the reference plane according to the integer image data and the region of the characteristic mark.

Because the pose of the product to be measured is changeable, the numerical values of different positions of the reference surface are not completely consistent, the difference is larger, such as the area with the upper left corner and the lower right corner as the reference surfaces in fig. 5, but the difference of gray values is obvious, which indicates that the product has obvious inclination, in order to ensure the accuracy of PIN needle detection, the area around the PIN needle and the characteristic mark needs to be selected as much as possible when the reference surface is fitted, and the directions of the PIN needle up, down, left, right and the like can be specifically selected. For example, as shown in fig. 6, five areas 1-5 can be selected to fit the reference plane, so that the accuracy of the fitted reference plane can be ensured. The above only takes five areas as an example to fit the reference plane, the number of the areas can be selected according to actual needs, for example, a larger number of areas are selected, and the fitting result is more accurate.

It should be understood that the reference plane may be custom or may be a physical plane on the product to be tested.

S404: respectively obtaining the height and the position degree of each PIN needle according to the areas of the PIN needles in a reference coordinate system, wherein the position degree comprises an X-axis position degree and a Y-axis position degree; and obtaining a detection result of whether each PIN needle is normal or not according to the height and the position degree of each PIN needle.

Before the height and the position degree are calculated, preprocessing can be performed to filter out some noise points. Since the PIN needle should be the highest of the PIN needle areas, i.e. z the largest area. Specifically, morphological corrosion can be used for reducing the size of the PIN needle region, then median filtering is used for removing noise points with abnormal values, then the maximum height value maxZ in the PIN needle region is calculated, then the lower limit is maxZ-offset (offset is a preset parameter), the point set of the PIN needle region is subjected to secondary threshold segmentation to extract a new region, and the extracted new region can truly reflect the PIN needle region.

Specifically, obtaining the height and the position of the PIN needle all needs to obtain a final actual physical value, and then comparing the actual physical value with a corresponding threshold value to judge whether a defect exists. Therefore, obtaining the altitude and position degree from the reference coordinate system requires conversion into actual physical values. The calculated height is multiplied by the Z-axis resolution to obtain a physical height value. And respectively calculating the point-line distance between the X axis and the Y axis of the reference coordinate system by using the coordinates (X, Y) of the PIN needle as the X-axis position of the PIN needle in the reference coordinate system, the point-line distance between the X axis and the reference coordinate system as the Y-axis position of the PIN needle in the reference coordinate system, and finally multiplying the resolutions of the X axis and the Y axis to obtain the physical value of the X-axis position and the physical value of the Y-axis position.

The position fluctuation of the product to be measured in the depth map is large because the grabbing gesture of the product to be measured may change each time, and the position of the region to be measured, such as the positions of a PIN needle, a reference surface, a positioning column and the like, may be affected. In order to quickly correct the position deviation, the method provided by the embodiment of the application can also obtain the position deviation before detection, and correct the position of the product to be detected by using the position deviation every time the product to be detected is detected. The following detailed description refers to the accompanying drawings.

Referring to fig. 7, a schematic diagram of position correction according to an embodiment of the present application is shown.

The method provided by the embodiment of the application, before fitting the reference surface according to the integer image data respectively corresponding to the reference surface area of the PIN needle and the reference surface area of the characteristic mark, further comprises the following steps:

correcting the positions of the PIN needle area, the characteristic mark area, the PIN needle reference surface area and the characteristic mark reference surface area according to the pre-obtained position deviation to obtain a corrected PIN needle area, a corrected characteristic mark area, a corrected PIN needle reference surface area and a corrected characteristic mark reference surface area; the positional deviation includes an X-axis positional difference, a Y-axis positional difference, and an angle difference.

Further comprising obtaining a positional deviation in advance by:

creating a shape template according to the depth map;

setting a search area including an area of the PIN needle, an area of the characteristic mark, a reference surface area of the PIN needle and a reference surface area of the characteristic mark;

searching to obtain a new position in the searching area by using a shape template;

a positional deviation is obtained from the new position and the reference position.

First, in the depth map, a triangle M is created using a shape template algorithm with reference template positions as in FIG. 7, and PIN needle areas as in circle 33 are set as in FIG. 7. A search region including a region of the PIN needle, a region of the feature marker, a reference plane region of the PIN needle, and a reference plane region of the feature marker, that is, an ROI region is set. The relative position between the ROI and the reference position is fixed.

In the second step, as shown in fig. 7, the inclined triangle N and the circle 44 are set, for example, in the dashed box of fig. 7, a new position is obtained after searching is successful by using the shape template, the affine transformation matrix can be calculated by using the reference position and the new position, that is, the position deviation is calculated, the X-axis position difference Δx, the Y-axis position difference Δy and the angle difference Δθ can be calculated, and the corrected PIN area, that is, the circle 44, can be obtained by using the above position deviations.

Through the position correction of the application, the requirement on shooting tools can be reduced, thereby reducing the cost.

The following is a specific procedure in connection with fitting the reference plane.

Fitting a reference plane according to integer image data respectively corresponding to the reference plane region of the PIN needle and the reference plane region of the characteristic mark, wherein the fitting comprises the following steps:

removing data with gray values larger than a first threshold value from integer image data corresponding to the reference surface area of the PIN needle and the reference surface area of the characteristic mark respectively to obtain preprocessed data;

obtaining data of an update area by using a morphological corrosion algorithm according to the preprocessed data;

filtering noise points by using a median filtering algorithm according to the data of the updated region to obtain final fitting data;

And fitting the reference surface by using a fitting plane algorithm according to the fitting data.

The above description of fig. 6 describes that five regions are selected to fit a reference plane, the data contained in these regions usually further contains noise points, the difference between the height value of a partial region and the surrounding value is obvious, the position should be similar in value to be normal in practice, and the direct fitting plane can lead to large fitting error, so that pretreatment can be performed first, invalid points are removed by using the lower limit of a set threshold value, the region meeting the set threshold value is extracted, then the region range is narrowed by using a morphological corrosion algorithm, the region with noise points easy to appear in the surrounding is removed, then the noise points with larger value runout are removed by using a median filtering algorithm from the point set in the region, and finally the reference plane is fitted by using a fitting plane algorithm.

The general formula equation of the plane in the three-dimensional coordinate system is ax+by+cz+d=0, the unknown coefficient in the equation needs to be determined, and the formula is deformed to obtain: z= -a/C x-B/C y-D/C = αx + βy + γ, substituting all sets of points into the equation to obtain a system of equations, and then obtaining a least squares solution of α, β, γ to obtain parameters of the fitting reference plane. And then subtracting the reference surface image from the original image to obtain a new image, wherein the gray value of the new image is the height value based on the reference surface.

The establishment of the reference coordinate system is described below.

In the embodiment of the application, two positioning columns are taken as an example for introduction, namely at least two characteristic marks comprise a first positioning column and a second positioning column;

establishing a reference coordinate system on a reference plane according to the integer image data and the region of the characteristic mark, wherein the reference coordinate system comprises the following steps:

and determining an X-axis on the reference plane according to the integer image data by taking the first positioning column as an origin of a coordinate system and taking a straight line determined by the first positioning column and the second positioning column, wherein the Y-axis passes through the origin and is perpendicular to the X-axis. For example, a straight line defined by the first positioning column and the second positioning column may be a straight line rotated by 37.6 ° counterclockwise around the first positioning column, and the above is merely illustrative, and the X axis may or may not be wound by other degrees, and may be specifically determined according to an actual product.

Performing secondary threshold segmentation on a first positioning area comprising a first positioning column and a second positioning area comprising a second positioning column respectively to obtain a first positioning column area and a second positioning column area;

taking the average value and the center of gravity of the Z value of the first positioning column area as the Z value and the position XY value of the first positioning column, and taking the average value and the center of gravity of the Z value of the second positioning column area as the Z value and the position XY value of the second positioning column;

And establishing a reference coordinate system according to the Z value and the position XY value of the first positioning column and the Z value and the position XY value of the second positioning column.

Specifically, the key after establishing the coordinate axis is to accurately obtain the coordinates of the first positioning column and the second positioning column. Firstly, filtering invalid numerical values in a set positioning column ROI by using a set lower limit threshold value, extracting a region meeting the threshold value, then reducing the size of the region by using morphological corrosion treatment, removing surrounding parts which are easy to generate noise points, removing noise points of abnormal jumping values by using median filtering, calculating the maximum value maxZ1 in the region, then carrying out secondary threshold segmentation on the region point set by using the lower limit maxZ-offset1 (the offset1 is a set parameter), extracting a new region, and using the strategy because the positioning column is actually the highest region of the region.

The manner in which the height and position of the PIN is obtained is described below.

The method for respectively obtaining the height and the position degree of each PIN needle according to the areas of the PIN needles in the reference coordinate system comprises the following steps:

when the area of the PIN needle is larger than or equal to a preset area threshold value, determining that the area of the PIN needle is qualified, and taking the average value of the Z values and the gravity center of the area of the PIN needle as the height and the position degree of the PIN needle respectively;

when the area of the PIN needle is smaller than a preset area threshold, taking the area of the remaining area after subtracting the area of the PIN needle from the area of the PIN needle after position correction as an area difference value until the area difference value is larger than the preset area threshold, stopping iterative calculation, taking the remaining area corresponding to the largest area difference value in all iteration times as a PIN needle calibration area, and taking the Z value average value and the gravity center of the PIN needle calibration area as the height and the position degree of the PIN needle respectively.

Firstly, using a lower limit threshold value to filter and set invalid values of the ROI of a PIN needle, extracting a region meeting the lower limit threshold value, then using morphological corrosion to reduce the size of the region, then using median filtering to remove noise points of abnormal values, then calculating the maximum value maxZ2 in the region, then using the lower limit of maxZ-offset2 (offset 2 is a set parameter), carrying out secondary threshold segmentation on the point set of the region to extract a new region, and using the strategy because the PIN needle is actually the highest region of the region. As shown in fig. 8, the two PIN needles a and B, the cross position represents the calculated position degree, which is more accurate, but the cross extracts abnormality, and the cross of B deviates from the true PIN position.

In order to solve the problems, an area card control is additionally arranged, a preset area threshold is set, and when the area of the selected PIN area is larger than or equal to the preset area threshold, the selected PIN area is used as the PIN area; when the iteration number is not enough to meet the preset area threshold (the iteration number can be set according to the calculation accuracy requirement), the remaining area corresponding to the largest area difference value in all iteration numbers is used as the PIN needle area, the Z value average value of the area is finally calculated as the Z value of the PIN needle, and the gravity center (x, y) of the area is used as the position degree of the PIN needle.

And then multiplying the Z value by the Z-axis resolution to obtain the actual physical height value. And the PIN needle position degree (X, Y) is respectively calculated to obtain an X-axis position degree by respectively calculating the distance from the Y-axis point line to the X-axis point line and the Y-axis position degree by respectively multiplying the X-axis resolution and the Y-axis resolution.

The PIN needle defect detection method provided by the embodiment of the application can measure the height and the position degree of the PIN needle, and achieve higher measurement accuracy, for example, the measurement accuracy of the height is 0.005mm, and the dynamic repeatability is less than 0.015mm; namely, when PIN defect detection is repeatedly carried out on the product to be detected, the deviation between the PIN defect detection and the last time is smaller than 0.05mm, and the dynamic repeatability is good.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a PIN needle defect detection device. The implementation of the solution provided by the device is similar to that described in the above method, so the specific limitation in the embodiments of the PIN defect detection device or devices provided below may be referred to the limitation of the PIN defect detection method hereinabove, and will not be repeated herein.

Fig. 9 is a schematic diagram of a PIN defect detection device according to an embodiment of the present application.

The PIN needle defect detection device provided by the embodiment of the application comprises:

the obtaining module 901 is configured to obtain a depth map of a product to be tested by using a 3D line laser camera, where the product to be tested includes at least one PIN and at least two feature marks; the at least two feature markers are used for establishing a reference coordinate system;

a preprocessing module 902, configured to convert floating point image data of a depth map into integer image data;

a fitting module 903, configured to fit a reference plane according to integer image data corresponding to the reference plane area of the PIN and the reference plane area of the feature mark;

a construction module 904, configured to establish a reference coordinate system on a reference plane according to the integer image data and the region of the feature tag;

a calculation module 905, configured to obtain, in a reference coordinate system, a height and a position degree of each PIN according to a region of the PIN, where the position degree includes an X-axis position degree and a Y-axis position degree;

and the detection module 906 is used for obtaining a detection result of whether each PIN needle is normal or not according to the height and the position degree of each PIN needle.

In some embodiments, the fitting module 903 is specifically configured to, in terms of fitting a reference plane from the integer image data corresponding to the reference plane region of the PIN and the reference plane region of the feature tag, respectively:

In some embodiments, the building block 904 is specifically configured to, in terms of building a reference coordinate system on a reference plane from the integer image data and the regions of the signature:

determining an X-axis on a reference plane according to the integer image data by taking a first positioning column as an origin of a coordinate system and taking a straight line determined by the first positioning column and a second positioning column, wherein a Y-axis passes through the origin and is vertical to the X-axis;

In some embodiments, the calculation module 905 is specifically configured to, in terms of obtaining the height and the position of each PIN from the region of the PIN in the reference coordinate system, respectively:

In some embodiments, the PIN defect detection device further includes a correction module, configured to perform position correction on the PIN area, the feature mark area, the PIN reference plane area, and the feature mark reference plane area according to the pre-obtained position deviation, to obtain a post-position-correction PIN area, a post-position-correction feature mark area, a post-position-correction PIN reference plane area, and a post-position-correction feature mark reference plane area; the positional deviation includes an X-axis positional difference, a Y-axis positional difference, and an angle difference.

In some embodiments, the PIN needle defect detection apparatus further comprises a position deviation module for creating a shape template from the depth map; setting a search area including an area of the PIN needle, an area of the characteristic mark, a reference surface area of the PIN needle and a reference surface area of the characteristic mark; searching to obtain a new position in the searching area by using a shape template; a positional deviation is obtained from the new position and the reference position.

In some embodiments, the first location post is located within the space enclosed by all PIN needles and the second location post is located outside the space enclosed by all PIN needles; the PIN needle is the PIN needle of the transformer.

The above-described individual modules in the PIN fault detection device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In some embodiments, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 10. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing the original image of the product to be tested. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program when executed by a processor implements the steps of the PIN needle defect detection method described above.

In some embodiments, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 11. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input device. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program when executed by a processor implements the steps of the PIN needle defect detection method described above. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer equipment can be a touch layer covered on a display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structures shown in fig. 10 or 11 are merely block diagrams of portions of structures associated with aspects of the application and are not intended to limit the computer device to which aspects of the application may be applied, and that a particular computer device may include more or fewer components than those shown, or may combine certain components, or may have a different arrangement of components.

In some embodiments, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the method embodiments described above when the computer program is executed.

In some embodiments, an internal structural diagram of a computer-readable storage medium is provided as shown in fig. 12, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method embodiments described above.

In some embodiments, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.

Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program, which may be stored on a non-transitory computer readable storage medium and which, when executed, may comprise the steps of the above-described embodiments of the methods. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A PIN needle defect detection method, comprising:

converting floating point image data of the depth map into integer image data;

fitting a reference plane according to the integer image data respectively corresponding to the reference plane area of the PIN needle and the reference plane area of the characteristic mark, and establishing a reference coordinate system on the reference plane according to the integer image data and the area of the characteristic mark;

Respectively obtaining the height and the position degree of each PIN needle according to the region of the PIN needle in the reference coordinate system, wherein the position degree comprises an X-axis position degree and a Y-axis position degree; and obtaining a detection result of whether each PIN needle is normal or not according to the height and the position degree of each PIN needle.

2. The method of claim 1, wherein prior to fitting a reference plane from the integer image data to which the reference plane region of the PIN needle and the reference plane region of the feature signature respectively correspond, the method further comprises:

performing position correction on the PIN needle area, the characteristic mark area, the PIN needle reference surface area and the characteristic mark reference surface area according to the pre-obtained position deviation to obtain a position-corrected PIN needle area, a position-corrected characteristic mark area, a position-corrected PIN needle reference surface area and a position-corrected characteristic mark reference surface area; the positional deviation includes an X-axis positional difference, a Y-axis positional difference, and an angle difference.

3. The method according to claim 2, further comprising obtaining the positional deviation in advance by:

Creating a shape template according to the depth map;

searching in the searching area by using the shape template to obtain a new position;

and obtaining the position deviation according to the new position and the reference position.

4. The method according to claim 1, wherein said fitting a reference plane from said integer image data to which the reference plane area of said PIN needle and the reference plane area of said feature tag respectively correspond, comprises:

removing data with gray values larger than a first threshold value from the integer image data corresponding to the reference surface area of the PIN needle and the reference surface area of the characteristic mark respectively to obtain preprocessed data;

filtering noise points by using a median filtering algorithm according to the data of the updating area to obtain final fitting data;

and fitting a reference plane by using a fitting plane algorithm according to the fitting data.

5. The method of claim 1, wherein the at least two signature comprises a first location post and a second location post;

The establishing a reference coordinate system on the reference plane according to the integer image data and the region of the characteristic mark comprises the following steps:

determining an X-axis on the reference plane according to the integer image data by taking the first positioning column as an origin of a coordinate system and a straight line determined by the first positioning column and the second positioning column, wherein a Y-axis passes through the origin and is perpendicular to the X-axis;

performing secondary threshold segmentation on a first positioning area comprising the first positioning column and a second positioning area comprising the second positioning column to obtain a first positioning column area and a second positioning column area;

taking the average value and the gravity center of the Z value of the first positioning column area as the Z value and the position XY value of the first positioning column, and taking the average value and the gravity center of the Z value of the second positioning column area as the Z value and the position XY value of the second positioning column;

6. The method according to claim 2, wherein the obtaining the height and the position of each PIN in the reference coordinate system from the region of the PIN, respectively, comprises:

When the area of the PIN needle area is larger than or equal to a preset area threshold value, determining that the PIN needle area is qualified, and taking the Z value average value and the gravity center of the PIN needle area as the height and the position of the PIN needle respectively;

when the area of the PIN needle area is smaller than the preset area threshold, subtracting the area of the remaining area after the PIN needle area is subtracted from the PIN needle area after the position correction to obtain an area difference value which is larger than the preset area threshold, stopping iterative calculation, taking the remaining area corresponding to the largest area difference value in all iteration times as a PIN needle calibration area, and taking the Z value average value and the gravity center of the PIN needle calibration area as the height and the position of the PIN needle respectively.

7. The method of claim 5, wherein the first location post is located within a space enclosed by all PIN needles and the second location post is located outside the space enclosed by all PIN needles; the PIN needle is the PIN needle of the transformer.

8. A PIN needle defect detection device, comprising:

the device comprises an obtaining module, a processing module and a processing module, wherein the obtaining module is used for obtaining a depth map of a product to be detected through a 3D line laser camera, and the product to be detected comprises at least one PIN needle and at least two characteristic marks; the at least two feature markers are used for establishing a reference coordinate system;

The preprocessing module is used for converting floating point type image data of the depth map into integer type image data;

the fitting module is used for fitting a reference plane according to the integer image data respectively corresponding to the reference plane area of the PIN needle and the reference plane area of the characteristic mark;

the calculation module is used for respectively obtaining the height and the position degree of each PIN needle according to the region of the PIN needle in the reference coordinate system, wherein the position degree comprises an X-axis position degree and a Y-axis position degree;

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.