CN117557531A - Sub-pixel precision robust PIN needle positive position detection method, device and storage medium - Google Patents
Sub-pixel precision robust PIN needle positive position detection method, device and storage medium Download PDFInfo
- Publication number
- CN117557531A CN117557531A CN202311556553.0A CN202311556553A CN117557531A CN 117557531 A CN117557531 A CN 117557531A CN 202311556553 A CN202311556553 A CN 202311556553A CN 117557531 A CN117557531 A CN 117557531A
- Authority
- CN
- China
- Prior art keywords
- needle
- sub
- pixel
- pin
- pin needle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 51
- 238000003860 storage Methods 0.000 title claims abstract description 21
- 230000011218 segmentation Effects 0.000 claims abstract description 52
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000000605 extraction Methods 0.000 claims description 20
- 230000004927 fusion Effects 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 abstract description 4
- 230000003287 optical effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000001788 irregular Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000005282 brightening Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device and a storage medium for detecting the correct position of a robust PIN needle with sub-pixel precision, wherein the method comprises the steps of obtaining an image of the PIN needle of a product to be detected, and displaying contour information of a needle body/a needle point in the image; positioning the PIN area to obtain an interested area of each PIN; sub-pixel segmentation is carried out on each PIN needle in the region of interest of each PIN needle, and a needle body/needle tip segmentation result with sub-pixel level precision is obtained; according to the result, acquiring the central coordinate of the needle body/needle point; if the number of the centers of the needle bodies/needle points is not equal to the number of the centers of the templates, judging the product as a disqualified product; if the number is the same, a coordinate system O is established according to the acquired needle body/needle point center coordinates; and carrying out point set registration on the coordinate system O and the template center coordinate system, and judging whether the positive position meets the detection requirement according to the point set registration result. Compared with the existing positive position detection method, the sub-pixel segmentation can ensure high segmentation precision and reduce the hardware cost of a high-precision imaging scheme.
Description
Technical Field
The invention relates to the technical field of industrial vision detection, in particular to a method and a device for detecting the correct position of a robust PIN needle with sub-pixel precision and a storage medium.
Background
The detection of the positive position of the PIN needle is to detect the gap between the position of the PIN needle and a standard template, is an important and common link in the production process of the connector, and is the final link for controlling the quality of the product. Most of the existing positive detection is manually detected by a grid plate, so that the problems of high cost, low measurement means precision, low speed and the like exist, and the realization of the vision-based positive detection is needed to improve the detection precision and efficiency and improve the quality control capability of enterprises on output products.
The existing rightness detection algorithm is mostly based on traditional binarization, edge detection, template matching and the like, and is not enough in robustness and poor in effect on the conditions of non-luminous needle points, irregular needle points, interference of needle side surfaces and the like. In the actual production process, the repeated positioning accuracy of the algorithm can not meet the requirement, and the traditional subpixel edge extraction algorithm has poor anti-interference capability and complex extraction capability. Thus, studying robust sub-pixel PIN needle localization algorithms is a problem that currently needs to be addressed.
If a coordinate system is established through the central coordinate, primary matching is possibly not robust, the existing point set matching algorithm mostly adopts least square estimation to calculate an optimal transformation matrix, the transformation matrix contains rotation parameters and scaling factors, the scene is not adapted in the positive position detection, and a quick point set registration algorithm only with translation needs to be researched to meet the requirement of the positive position detection.
Disclosure of Invention
The invention provides a method, a device and a storage medium for detecting the correct position of a robust PIN needle with sub-pixel precision, which are used for solving the technical problems in the prior art.
The technical scheme adopted by the invention is as follows: in a first aspect, the present invention provides a method for detecting the correct position of a robust PIN needle with sub-pixel precision, comprising:
acquiring an image of a PIN needle of a product to be detected, wherein contour information of a needle body/needle point can be presented in the image;
positioning the PIN area to obtain an interested area of each PIN;
sub-pixel segmentation is carried out on each PIN needle in the region of interest of each PIN needle, and a needle body/needle tip segmentation result with sub-pixel level precision is obtained;
acquiring the center coordinates of the needle body/needle tip according to the PIN needle righting degree segmentation result;
if the number of the centers of the needle bodies/needle points is not equal to the number of the centers of the templates, the missing needle and the askew needle are judged to exist, and then the product is judged to be a disqualified product; if the number of the needle body/needle point centers is equal to the number of the template centers, establishing a coordinate system O according to the obtained needle body/needle point center coordinates;
and carrying out point set registration on the coordinate system O and the template center coordinate system, and judging whether the positive position meets the detection requirement according to the point set registration result.
Further, the splitting network used for sub-pixel splitting of the PIN needle is as follows: a sub-pixel feature extraction module, a feature fusion module and a downsampling learning module are added in the segmentation network.
Further, the feature extraction in the sub-pixel feature extraction module uses a backbone network, and pixel-level features are extracted from the region of interest of each PIN through the backbone network; and up-sampling the pixel-level features by adopting a super-division network to obtain sub-pixel features.
Further, the sub-pixel features obtained by the sub-pixel feature extraction module and the features of the segmentation network coding stage are subjected to multi-scale feature splicing and fusion to obtain fused features.
Further, the fused features are subjected to downsampling through a downsampling learnable module, so that segmented images as large as input are obtained; inputting the fused characteristics into a segmentation network decoding stage to obtain an input size n 2 Multiple divided image with division precision of 1/n 2 A pixel, n is an up-sampling multiple, n>1 and is an integer.
Further, the method further comprises the following steps: will input the size n 2 The multiple divided images are subjected to downsampling processing through a downsampling learning module, and the divided images as large as the input are obtained.
Further, the method for performing point set registration on the coordinate system O and the template center coordinate system, and judging whether the positive position meets the detection requirement according to the point set registration result includes:
the origin of the needle/needle tip center coincides with the origin of the template center in the coordinate system O, and the offset (dx) between the rest of the needles and the template center is calculated 0 ，dy 0 ) Distance d 0 ；
Determine the offset (dx) of all pins from the center of the template 0 ，dy 0 ) Distance d 0 Whether the positive position detection standard is met, if so, judging that the product is a qualified product; if not, calculating the average value of the absolute values of all needle/needle tip center and template center offset except the original point
Taking the center of the template as the center, and searching the range in the x direction asThe search range in the y-direction is +.>The rectangular region of (2) is subjected to point set registration, the searching step length is s, and the total searching times areWherein k is 1 、k 2 Is a positive number;
the PIN needle center coordinates are translated according to the step length and the direction in each cycle, and the offset (dx i ，dy i ) Distance d i If the standard meets the positive position detection standard, judging the product to be a qualified product, otherwise, continuing to circulate;
if the cycle ends and neither meets the positive detection criteria, then the cycle (dx i ，dy i ) The smallest one is displayed as the optimal matching result.
In a second aspect, the present invention provides a robust PIN needle alignment detection device with sub-pixel precision, comprising:
the PIN needle image acquisition module is used for acquiring an image of a PIN needle of a product to be detected, and contour information of a needle body/needle point can be presented in the image;
the PIN needle region positioning module is used for positioning the PIN needle regions to obtain the interested region of each PIN needle;
the PIN needle sub-pixel segmentation module is used for sub-pixel segmentation of the PIN needles in the region of interest of each PIN needle to obtain a needle body/needle point segmentation result with sub-pixel level precision;
the PIN needle center coordinate positioning module is used for acquiring the needle body/needle point center coordinate according to the PIN needle righting degree segmentation result;
the PIN needle number judging module is used for judging whether the number of the obtained needle body/needle point centers is equal to the number of the template centers, if not, the PIN needle number judging module judges that the missing needle and the skewed needle exist, and further judges that the product is a disqualified product; if the number of the needle body/needle point centers is equal to the number of the template centers, establishing a coordinate system O according to the acquired needle body/needle point center coordinates through a PIN needle center coordinate system conversion module;
and the point set registration module is used for carrying out point set registration on the coordinate system O and the template center coordinate system, and judging whether the positive position degree meets the detection requirement according to the point set registration result.
In a third aspect, the present invention provides an electronic device comprising:
one or more processors;
a storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the sub-pixel accurate robust PIN needle alignment detection method as described in the first aspect.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements a robust PIN needle alignment detection method of sub-pixel accuracy as described in the first aspect.
The beneficial effects of the invention are as follows: the invention obtains 1/n through a sub-pixel network segmentation strategy and adding a sub-pixel feature extraction module, a feature fusion module and a downsampling learning module in the existing segmentation network 2 (n>1, an integer) of pixel precision PIN needle segmentation results, hardware cost of a high-precision imaging scheme is reduced, segmentation precision of PIN needles is improved in a learning mode, repeated positioning precision of an algorithm can be further improved, and a deep learning algorithm can adapt to situations of non-lighting of a needle point, irregular needle point, interference of a needle side surface and the like; by means of a global iterative point set registration strategy, the offset during initial matching is used as a point set searching range, point set registration of the PIN needle center coordinate and the template center coordinate can be iteratively performed, compared with an existing positive position detection method, sub-pixel segmentation can ensure high segmentation precision, hardware cost of a high-precision imaging scheme is reduced, the adopted point set registration method is free of scaling and rotation, high in efficiency, controllable and high in robustness, and positive position detection requirements are met.
Drawings
FIG. 1 is a schematic flow chart of a method for detecting the correct position of a robust PIN needle with sub-pixel precision;
FIG. 2 is a schematic illustration of a PIN needle subpixel segmentation disclosed herein;
FIG. 3 is a schematic illustration of the registration of a PIN set of points disclosed herein;
FIG. 4 is a block diagram of a sub-pixel accurate robust PIN needle alignment detection device;
fig. 5 is a schematic structural diagram of an electronic device disclosed in the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Example 1:
referring to fig. 1, the embodiment discloses a method for detecting the correct position of a robust PIN needle with sub-pixel precision, which comprises the following steps:
s1: and acquiring an image of the PIN needle of the product to be detected, wherein contour information of the needle body/needle point can be presented in the image.
Specifically, the PIN image acquisition in step S1 includes, but is not limited to, a 2D camera, a 3D camera, and the like. In this embodiment, the implementation is specifically as follows: the color industrial camera, the telecentric lens and the annular light are adopted, the resolution and the lens of the camera are selected according to the size and the precision of a workpiece, for example, the size of the workpiece is about 20mm, a Basler 2000 ten thousand color camera is selected, the visual field is 25.7mm x17.1mm, and the single pixel precision is 0.0047mm. When sampling is carried out, shooting is carried out in a overlooking mode, wherein the needle point is the point brightening part of the PIN needle; the needle body refers to the entire tip of the PIN needle, not the shank portion.
S2: and positioning the PIN needle areas to obtain the interested area of each PIN needle.
Specifically, the method used in step S2 includes, but is not limited to, prior information, template matching, target detection, and the like.
In this embodiment, the implementation is specifically as follows: and (3) segmenting the ROI (region of interest) of the workpiece region by adopting a Unet segmentation model to obtain the outer contour and the minimum circumscribed rotation rectangle of the workpiece, acquiring the rotation angle of the workpiece according to the minimum circumscribed rotation rectangle, and correcting the workpiece. And converting the center coordinates of the template into an image coordinate system, calculating offset coordinates of a first needle at the upper left corner and the upper left corner of the workpiece, adding the offset coordinates to the coordinates of all the needles in the image coordinate system to obtain prior coordinates of the center of the needles, and taking the prior coordinates as the center, and intercepting a region of interest which can contain the needles, such as 128x128.
S3: and sub-pixel segmentation is carried out on the PIN needles in the region of interest of each PIN needle, so that a needle body/needle tip segmentation result with sub-pixel level precision is obtained.
Specifically, referring to fig. 2, the segmentation network used for sub-pixel segmentation of the PIN needle in step S3 is: the method comprises the steps of adding a sub-pixel feature extraction module, a feature fusion module and a downsampling learnable module into the existing segmentation network. Finally obtaining 2 outputs which are respectively divided images with the same size as the input and the input size n 2 Multiple (n is magnification, n)>1, integer) with a segmentation accuracy of 1/n 2 Individual pixels (sub-pixels).
Specifically, the feature extraction in the sub-pixel feature extraction module in the PIN sub-pixel segmentation may use a common backhaul network (backbone network), such as ResNet, to extract the pixel-level feature from the region of interest of each PIN, and upsample the pixel-level feature by using a depth-to-space module in the super-division network to obtain a sub-pixel feature of n×m×n H (where N is a multiplier, W and H are the width and height of the input image, N is a batch (batch), M is the number of feature channels, N is an upsampling multiple, N >1, and is an integer).
Specifically, a feature fusion module in PIN needle sub-pixel segmentation carries out multi-scale concat feature fusion (namely feature splicing fusion) on sub-pixel features of the sub-pixel feature extraction module and features of a segmentation network coding stage to obtain fused features; the resolution of the fusion may be selected from a plurality of, e.g., the same size resolution feature as the input, 1/2 input size resolution feature, etc.
Specifically, the fused characteristics are input into a segmentation network decoding stage to obtain an input size n 2 Multiple divided image with division precision of 1/n 2 A pixel, n is an up-sampling multiple, n>1 and is an integer. Will input the size n 2 The multiple segmented image and the fused feature are subjected to downsampling by a space-to-depth module in a space-to-deep learning module in PIN needle sub-pixel segmentation, and the output of NxMnW nH and the feature are mapped back to WxH size, so that the segmented image as large as the input is obtained.
The decoded input size n 2 The multiple divided images are up-sampled images, the divided images processed by the down-sampling learning module are down-sampled images, the two images are images with sub-pixel level precision, and in actual use, one of the two images is selected according to actual needs. Further, the decoded segmented image can be input to a downsampling learning module for processing, so that a downsampled image is obtained, which is better than a segmented image just decoded, and is particularly less in jaggies and better in segmentation effect.
As shown in fig. 2, in this embodiment, the implementation is as follows: the existing segmentation network adopts a classical encoding and decoding network Unet, the pixel-level feature extraction in a sub-pixel feature extraction module can use a common ResNet18, the extracted features are 32X H X W, the sub-pixel feature extraction adopts a depth-to-space module in a super-segmentation network to amplify the features by 2 times, the features are rearranged from channel dimensions to width and height dimensions, and finally the obtained feature dimension is 8X 2H X2W. And performing concat feature fusion on the features in the sub-pixel feature extraction module and the features of the Unet coding network, wherein the fused features are selected from a plurality of scales (H.times.W.size, 1/2 H.times.W.size, 1/4 H.times.W.size and 1/8 H.times.W.size), and inputting the fused features into the Unet decoding network. The downsampling learning module downsamples the feature map with the size of 2H x 2W and the segmented image with the size of 2H x 2W to the size of H x 2W by using the space-to-depth module, the number of the output segmented images obtained finally is 2, the output segmented images are respectively the size of H x 2W and the size of 2H x 2W, and the segmentation precision is 1/4 pixel.
S4: and acquiring the center coordinates of the needle body/needle tip according to the PIN needle righting degree segmentation result.
Specifically, the PIN center coordinate positioning in step S4 includes, but is not limited to, centroid, minimum circumscribed rectangular center, minimum circumscribed circular center, etc. calculated by the contour second moment. The needle/tip center should be interpreted as the center of the needle or the tip center, which is the center of the PIN tip lighting area, and the center of the needle refers to the center of the entire PIN tip.
In this embodiment, the implementation is specifically as follows: and binarizing the output segmented image which is downsampled and has the same size as the input segmented image according to a threshold value of 0.5, obtaining a minimum circumscribed rectangle of the outer contour, and taking the center of the minimum circumscribed rectangle as a needle center coordinate.
S5: if the number of the centers of the needle bodies/needle points is not equal to the number of the centers of the templates, the missing needle and the askew needle are judged to exist, and then the product is judged to be a disqualified product; if the number of the needle body/needle point centers is equal to the number of the template centers, a coordinate system O is established according to the obtained needle body/needle point center coordinates.
Specifically, if the number of the obtained needle centers is not equal to the number of the template centers, the conditions of missing needles, skewed needles and the like are determined, and the product can be judged to be a disqualified product without point set registration in S6.
Specifically, in the step S5, a coordinate system O is established, two PINs are selected according to the detection requirement to establish an x-axis of the coordinate system, the y-axis is perpendicular to the x-axis, and the coordinates of the center of the PIN are converted into the established coordinate system according to the geometric relationship.
In this embodiment, the implementation is specifically as follows: selecting a first needle at the lower left corner and a last needle at the lower right corner to be connected to obtain an angle theta under a current coordinate system, wherein the rotation center is (centrX, centrY), the center coordinates of an original needle are (x, y), and the coordinates in a new coordinate system are (dstX, dstY):
srcX＝x-centerX，srcY＝y-centerY
dstX＝srcX*cos(θ)+srcY*sin(θ)+centerX
dstY＝-srcX*sin(θ)+srcY*cos(θ)+centerY
s6: and carrying out point set registration on the coordinate system O and the template center coordinate system, and judging whether the positive position meets the detection requirement according to the point set registration result.
Specifically, referring to fig. 3, the step S6 includes: s61: the origin of the needle/needle tip center coincides with the origin of the template center in the coordinate system O, and the offset (dx) between the rest of the needles and the template center is calculated 0 ，dy 0 ) Distance d 0 ；
S62: determine the offset (dx) of all pins from the center of the template 0 ，dy 0 ) Distance d 0 Whether the positive position detection standard is met, if so, judging that the product is a qualified product; if not, S63 is performed;
s63: calculating the mean value of the absolute values of all needle/tip center deviations from the template center except the origin
S64: considerIs in the form of a moduleThe plate center is the center, the search range in the x direction is +.> The search range in the y-direction is +.>The rectangular area of (2) is subjected to point set registration, the searching step length is s, and the total searching times is +.>Wherein k is 1 、k 2 Positive number, s is step length;
s65: the PIN needle center coordinates are translated according to the step length and the direction in each cycle, and the offset (dx i ，dy i ) Distance d i If the standard meets the positive position detection standard, judging the product to be a qualified product, otherwise, continuing to circulate;
s66: if the cycle ends and neither meets the positive detection criteria, then the cycle (dx i ，dy i ) The smallest one is displayed as the optimal matching result.
In this embodiment, the implementation is specifically as follows: 100 pins are arranged, the 1 st pin at the left lower corner is taken as an origin point and coincides with the origin point of the center of the template, the offset between the rest 99 pins and the center of the template is calculated, if the offset of 1 pin is 0.3mm and is larger than the standard 0.2mm, and the process goes to S63. The average offset of 99 needles from the center of the template was calculated and converted to pixel (8, 10), step size was set to 1, and the total number of searches was 2x8x2x 9/1=288 according to the procedure described above. And translating the center coordinates of the template every time, calculating the offset between all the pins and the center of the template, wherein the offset between all the pins and the center of the template is smaller than 0.2mm in the 101 th cycle, and returning OK by the positive position detection.
The embodiment provides a robust PIN needle rightness detection method with sub-pixel precision, which has the beneficial effects that: by "sub-pixel network splitting" strategy, by being in an existing splitting networkAdding a sub-pixel feature extraction module, a feature fusion module and a downsampling learning module to obtain 1/n 2 (n>1, an integer) of pixel precision PIN needle segmentation results, hardware cost of a high-precision imaging scheme is reduced, segmentation precision of PIN needles is improved in a learning mode, repeated positioning precision of an algorithm can be further improved, and a deep learning algorithm can adapt to situations of non-lighting of a needle point, irregular needle point, interference of a needle side surface and the like; by means of the global iterative point set registration strategy, the offset during initial matching is used as a point set searching range, point set registration of the PIN needle center coordinates and the template center coordinates can be iteratively performed, and compared with an existing point set registration mode, the point set registration method is free of scaling and rotation, high in efficiency, controllable and high in robustness, and the positive position detection requirement is met.
Example 2:
referring to fig. 4, the present embodiment discloses a robust PIN needle righting degree detection device with sub-pixel precision, which includes:
the PIN image acquisition module 410 is configured to acquire an image of a PIN of a product to be detected, where contour information of a needle body/needle tip can be presented in the image;
the PIN area positioning module 420 is configured to position the PIN areas to obtain an area of interest of each PIN;
the PIN sub-pixel segmentation module 430 is configured to sub-pixel segment the PIN in the region of interest of each PIN, so as to obtain a sub-pixel-level accurate needle body/needle tip segmentation result;
the PIN needle center coordinate positioning module 440 is configured to obtain a needle body/needle point center coordinate according to the PIN needle rightness segmentation result;
the PIN number judging module 450 is used for judging whether the number of the centers of the acquired needle bodies/needle points is equal to the number of the centers of the templates, if not, the PIN number judging module judges that the missed needle and the skewed needle exist, and further judges that the product is a disqualified product; if the number of the needle body/needle point centers is equal to the number of the template centers, a coordinate system O is established according to the acquired needle body/needle point center coordinates through a PIN needle center coordinate system conversion module 460;
the point set registration module 470 is configured to perform point set registration on the coordinate system O and the template center coordinate system, and determine whether the positive position meets the detection requirement according to the point set registration result.
The device can execute the robust PIN needle righting degree detection method with sub-pixel precision provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the robust PIN needle righting degree detection method with sub-pixel precision.
Example 3:
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. Fig. 5 shows a block diagram of an exemplary electronic device 50 suitable for use in implementing the embodiments of the present invention. The electronic device 50 shown in fig. 5 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 5, the electronic device 50 is embodied in the form of a general purpose computing device. Components of electronic device 50 may include, but are not limited to: one or more processors or processing units 501, a system memory 502, and a bus 503 that connects the various system components (including the system memory 502 and processing units 501).
Bus 503 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Electronic device 50 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by electronic device 50 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 502 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 504 and/or cache memory 505. Electronic device 50 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 506 may be used to read from or write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, commonly referred to as a "hard disk drive"). Although not shown in fig. 5, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 503 through one or more data medium interfaces. Memory 502 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of embodiments of the invention.
A program/utility 508 having a set (at least one) of program modules 507 may be stored, for example, in memory 502, such program modules 507 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 507 typically perform the functions and/or methods of the described embodiments of the invention.
The electronic device 50 may also communicate with one or more external devices 509 (e.g., keyboard, pointing device, display 510, etc.), one or more devices that enable a user to interact with the electronic device 50, and/or any device (e.g., network card, modem, etc.) that enables the electronic device 50 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 511. Also, the electronic device 50 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through a network adapter 512. As shown, the network adapter 512 communicates with other modules of the electronic device 50 over the bus 503. It should be appreciated that although not shown in fig. 5, other hardware and/or software modules may be used in connection with electronic device 50, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processing unit 501 executes programs stored in the system memory 502 to perform various functional applications and data processing, for example, to implement a robust PIN alignment position detection method with sub-pixel precision according to an embodiment of the present invention.
Example 4:
the present embodiment provides a storage medium containing computer executable instructions for performing a robust PIN needle positive location detection method of sub-pixel accuracy when executed by a computer processor.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for embodiments of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be appreciated by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for detecting the correct position of a robust PIN needle with sub-pixel precision is characterized by comprising the following steps:
acquiring an image of a PIN needle of a product to be detected, wherein contour information of a needle body/needle point can be presented in the image;
positioning the PIN area to obtain an interested area of each PIN;
sub-pixel segmentation is carried out on each PIN needle in the region of interest of each PIN needle, and a needle body/needle tip segmentation result with sub-pixel level precision is obtained;
acquiring the center coordinates of the needle body/needle tip according to the PIN needle righting degree segmentation result;
if the number of the centers of the needle bodies/needle points is not equal to the number of the centers of the templates, the missing needle and the askew needle are judged to exist, and then the product is judged to be a disqualified product; if the number of the needle body/needle point centers is equal to the number of the template centers, establishing a coordinate system O according to the obtained needle body/needle point center coordinates;
and carrying out point set registration on the coordinate system O and the template center coordinate system, and judging whether the positive position meets the detection requirement according to the point set registration result.
2. The method for detecting the correct position of a robust PIN needle with sub-pixel precision according to claim 1, wherein the splitting network used for sub-pixel splitting of the PIN needle is: a sub-pixel feature extraction module, a feature fusion module and a downsampling learning module are added in the segmentation network.
3. The method for detecting the correct position of the robust PIN needle with sub-pixel precision according to claim 2, wherein the feature extraction in the sub-pixel feature extraction module uses a backbone network, and pixel-level features are extracted from the region of interest of each PIN needle through the backbone network; and up-sampling the pixel-level features by adopting a super-division network to obtain sub-pixel features.
4. The method for detecting the correct position of the robust PIN needle with sub-pixel precision according to claim 3, wherein the sub-pixel features obtained by the sub-pixel feature extraction module and the features of the segmentation network coding stage are subjected to multi-scale feature stitching and fusion to obtain fused features.
5. The method for detecting the correct position of a robust PIN needle with sub-pixel precision as recited in claim 4, wherein the method is characterized in thatThe method is characterized in that the fused features are subjected to downsampling through a downsampling learnable module to obtain segmented images as large as input; inputting the fused characteristics into a segmentation network decoding stage to obtain an input size n 2 Multiple divided image with division precision of 1/n 2 A pixel, n is an up-sampling multiple, n>1 and is an integer.
6. The method for detecting the correct position of a robust PIN needle with sub-pixel precision according to claim 5, further comprising: will input the size n 2 The multiple divided images are subjected to downsampling processing through a downsampling learning module, and the divided images as large as the input are obtained.
7. The method for detecting the correct position of a robust PIN needle with sub-pixel precision according to claim 5, wherein the method for performing point set registration on the coordinate system O and the central coordinate system of the template and judging whether the correct position meets the detection requirement according to the point set registration result comprises the following steps:
the origin of the needle/needle tip center coincides with the origin of the template center in the coordinate system O, and the offset (dx) between the rest of the needles and the template center is calculated 0 ，dy 0 ) Distance d 0 ；
Determine the offset (dx) of all pins from the center of the template 0 ，dy 0 ) Distance d 0 Whether the positive position detection standard is met, if so, judging that the product is a qualified product; if not, calculating the average value of the absolute values of all needle/needle tip center and template center offset except the original point
Taking the center of the template as the center, and searching the range in the x direction asThe search range in the y direction isThe rectangular region of (2) is subjected to point set registration, the searching step length is s, and the total searching times areWherein k is 1 、k 2 Is a positive number;
the PIN needle center coordinates are translated according to the step length and the direction in each cycle, and the offset (dx i ，dy i ) Distance d i If the standard meets the positive position detection standard, judging the product to be a qualified product, otherwise, continuing to circulate;
if the cycle ends and neither meets the positive detection criteria, then the cycle (dx i ，dy i ) The smallest one is displayed as the optimal matching result.
8. A robust PIN needle alignment detection device of sub-pixel accuracy, comprising:
the PIN needle image acquisition module is used for acquiring an image of a PIN needle of a product to be detected, and contour information of a needle body/needle point can be presented in the image;
the PIN needle region positioning module is used for positioning the PIN needle regions to obtain the interested region of each PIN needle;
the PIN needle sub-pixel segmentation module is used for sub-pixel segmentation of the PIN needles in the region of interest of each PIN needle to obtain a needle body/needle point segmentation result with sub-pixel level precision;
the PIN needle center coordinate positioning module is used for acquiring the needle body/needle point center coordinate according to the PIN needle righting degree segmentation result;
the PIN needle number judging module is used for judging whether the number of the obtained needle body/needle point centers is equal to the number of the template centers, if not, the PIN needle number judging module judges that the missing needle and the skewed needle exist, and further judges that the product is a disqualified product; if the number of the needle body/needle point centers is equal to the number of the template centers, establishing a coordinate system O according to the acquired needle body/needle point center coordinates through a PIN needle center coordinate system conversion module;
and the point set registration module is used for carrying out point set registration on the coordinate system O and the template center coordinate system, and judging whether the positive position degree meets the detection requirement according to the point set registration result.
9. An electronic device, the electronic device comprising:
one or more processors;
a storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the sub-pixel accurate robust PIN needle position detection method of any of claims 1-7.
10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a sub-pixel accurate robust PIN needle position detection method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311556553.0A CN117557531A (en) | 2023-11-21 | 2023-11-21 | Sub-pixel precision robust PIN needle positive position detection method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311556553.0A CN117557531A (en) | 2023-11-21 | 2023-11-21 | Sub-pixel precision robust PIN needle positive position detection method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117557531A true CN117557531A (en) | 2024-02-13 |
Family
ID=89814370
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311556553.0A Pending CN117557531A (en) | 2023-11-21 | 2023-11-21 | Sub-pixel precision robust PIN needle positive position detection method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117557531A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119146851A (en) * | 2024-11-12 | 2024-12-17 | 上海禾赛科技有限公司 | Detection method and detection system |
-
2023
- 2023-11-21 CN CN202311556553.0A patent/CN117557531A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119146851A (en) * | 2024-11-12 | 2024-12-17 | 上海禾赛科技有限公司 | Detection method and detection system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7564962B2 (en) | IMAGE PROCESSING METHOD, IMAGE PROCESSING APPARATUS AND NON-TRANSITORY STORAGE MEDIUM | |
| EP3309703B1 (en) | Method and system for decoding qr code based on weighted average grey method | |
| CN108122256B (en) | A method of it approaches under state and rotates object pose measurement | |
| JP5699788B2 (en) | Screen area detection method and system | |
| US20150262346A1 (en) | Image processing apparatus, image processing method, and image processing program | |
| CN111986159B (en) | Electrode defect detection method and device for solar cell and storage medium | |
| JP5468332B2 (en) | Image feature point extraction method | |
| CN103902953B (en) | A kind of screen detecting system and method | |
| WO2019128504A1 (en) | Method and apparatus for image processing in billiards game, and terminal device | |
| CN113705564B (en) | Pointer type instrument identification reading method | |
| CN117557531A (en) | Sub-pixel precision robust PIN needle positive position detection method, device and storage medium | |
| CN113793370B (en) | Three-dimensional point cloud registration method and device, electronic equipment and readable medium | |
| CN116993612A (en) | A method for correcting nonlinear distortion of fisheye lens | |
| CN104655041B (en) | A kind of industrial part contour line multi-feature extraction method of additional constraint condition | |
| Seo et al. | Real-time visual tracking of less textured three-dimensional objects on mobile platforms | |
| CN113920525A (en) | Text correction method, device, equipment and storage medium | |
| CN118520893A (en) | Method, device and storage medium for identifying bar code label applied to AOI | |
| CN114998347B (en) | Semiconductor panel corner positioning method and device | |
| US20240242318A1 (en) | Face deformation compensating method for face depth image, imaging device, and storage medium | |
| CN111696154B (en) | Coordinate positioning method, device, equipment and storage medium | |
| CN114742705A (en) | Image splicing method based on halcon | |
| CN114549613A (en) | Structural displacement measuring method and device based on deep super-resolution network | |
| CN113850100A (en) | Method and device for correcting two-dimensional code | |
| CN113012132A (en) | Image similarity determining method and device, computing equipment and storage medium | |
| CN116225236B (en) | Intelligent home scene interaction method based on acousto-optic control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |