KR102649038B1 - System and method for finding lines in an image with a vision system - Google Patents

Abstract

The present invention relates to a method and system for finding multiline features in an image. Two related steps are used to identify line features. The first process computes the x and y components of the gradient field at each image location, projects the gradient field over multiple sub-regions, and generates a number of edge points with positions and gradients. Detect extreme values (a plurality of gradient extrema). The second process selects two edge points, fits a model line to them, and if the edge point slope is consistent with the model, then the entire set of inner layer points whose positions and slopes match the model (consistent) Compute a full set of inlier points). The candidate line with the maximum inner layer count is maintained and a set leaving the outlier points is derived. The process iteratively applies the line fitting operation and subsequent set of outer layers to find multiple line results. This process can be accomplished on a thorough RANSAC basis.

Description

System and method for finding lines in an image with a vision system {SYSTEM AND METHOD FOR FINDING LINES IN AN IMAGE WITH A VISION SYSTEM}

Related applications

This application is a co-pending U.S. provisional patent application entitled “SYSTEM AND METHOD FOR FINDING LINES IN AN IMAGE WITH A VISION SYSTEM,” filed November 2, 2015, which is hereby incorporated by reference in its entirety. Claims the benefit of serial number 62/249,918.

technology field

The present invention relates to machine vision systems, and more specifically to vision system tools for discovering line features in acquired images.

Machine vision systems (also referred to herein simply as “vision systems”) are used for many tasks in manufacturing, logistics, and industry. These operations may include inspection of surfaces and parts, alignment of objects during assembly, reading of patterns and ID codes, and any other operation where visual data is acquired and interpreted for use in further processing. Vision systems typically use one or more cameras to acquire images of objects of interest or scenes containing objects of interest. This object/object may be stationary or relatively movable. Movement may also be controlled by information derived from a vision system, such as when manipulating parts by a robot.

A common task in vision systems is discovering and characterizing line features in images. Several tools are used to identify and analyze these line features. Typically, this tool relies on sharp contrast differences in parts of the image. This contrast difference can be analyzed, for example using a caliper tool, to determine whether individual points in the image with contrast differences can be assembled into line-like features. If it can be assembled, the line is identified in the image. In particular, the tool for finding edge points and the tool for attempting to fit lines to points are independent of each other. This increases processing overhead and reduces reliability. If an image contains multiple lines, these tools may be limited in their ability to accurately identify these lines. Furthermore, traditional line-finding tools designed to find a single line in an image can be problematic to use when the image contains multiple closely spaced lines with similar orientation and polarity.

The present invention overcomes the shortcomings of the prior art by providing a system and method for discovering line features in an image that can efficiently and accurately identify and characterize multiple lines. First, the process computes the x- and y-components of the gradient field at each location in the image, projects the gradient field over a plurality of image subregions, and projects By detecting a plurality of gradient extrema within the range, a plurality of edge points with associated positions and gradients are calculated. Next, the process iteratively selects two edge points, fits model lines to these edge points, and, if the gradients of these edge points are consistent with the model line, then the positions consistent with the model line. Compute the entire set of inlier points with gradients. The candidate line with the maximum normal count is retained as the line result, and the remaining set of outlier points is derived. The process iteratively applies a line fitting operation to this and subsequent outlier sets to discover multiple line results. The line-fitting process can be performed throughout, or can be based on random sample consensus (RANSAC) techniques.

In an example embodiment, a system for discovering line features in an acquired image is provided. The vision system processor receives image data of the scene including line features. An edge point extractor generates intensity gradient images from the image data and discovers edge points based on the intensity gradient images. A line-finder fits the edge points to one or more lines based on the intensity gradient at the edge points. Illustratively, the line finder operates a RANSAC-based process to fit normal edge points to new lines, including iteratively defining lines from outlier edge points with respect to previously defined lines. The edge point extractor performs a gradient field projection of the line-feature-containing regions of the intensity gradient images. Illustratively, the gradient field projection is oriented along a direction set in response to the expected orientation of one or more of the line features, and the gradient field projection may define granularity based on a Gaussian kernel. You can. Exemplarily, the image data may include data from a plurality of images acquired from a plurality of cameras and converted to a common coordinate space. The image data may also be smoothed using a smoothing (weighting) kernel, which may include a 1D Gaussian kernel or other weighting function. The edge points may be selected based on a threshold defined by absolute contrast and normalized contrast based on the average intensity of the image data. Illustratively, the line finder swaps edge points representing parallel lines or portions of intersecting lines to correct erroneous orientation, and/or determines the line finder based on the gradient values of the edge points. Constructed and arranged to identify lines with polarity variation, including mixed polarities of features. Additionally, exemplarily, the edge point extractor is arranged to find a plurality of gradient magnitude maxima in each of the gradient projection sub-regions. Each of these gradient magnitude maximums can be identified as some of the plurality of edge points and can be described by a position vector and a gradient vector. Additionally, the line finder determines consistency between at least one edge point among the extracted plurality of edge points and at least one candidate line among the discovered plurality of lines by calculating a metric. can be arranged to decide. This metric may be based on the distance of the at least one edge point from the candidate line, and the angular difference between the gradient direction of the edge point and the normal direction of the candidate line.

The following description of the present invention refers to the accompanying drawings.
1 illustrates an example vision system arrangement for acquiring an image of an object, including a vision system processor and multiple edge features including an edge-discovery tool/module in accordance with an example embodiment;
Figure 2 shows an overview of a system and method for extracting edge points and finding lines from an acquired image, according to an example embodiment;
Figure 3 is a flow chart of a procedure for extracting edge points according to the system and method of Figure 2;
Figure 4 shows a gradient field projection on a region of the image containing edge features that are part of the edge point extraction procedure of Figure 3;
Figure 5 illustrates applying a Gaussian kernel to an image to smooth it for use in the edge point extraction procedure of Figure 3;
Figure 6 shows a gradient field projection including applying a Gaussian kernel to smooth the projection, for use in the edge point extraction procedure of Figure 3;
Figure 7 graphically outlines the procedure for extracting edge points of Figure 3, including applying Gaussian kernels and calculating absolute and normalized contrast thresholds for the edge points;
Figure 8 is a graph showing the region of qualified contrasts for edge points, with sufficient absolute contrast threshold and normalized contrast threshold;
Figure 9 is a flow diagram of a procedure for finding a line based on edge points found in Figure 3 using an example RANSAC process according to an example embodiment;
Figures 10 and 11 show error and correct alignment of edge points for closely spaced parallel line features, respectively;
12 and 13 illustrate correctly and erroneously aligned edge points, respectively, for intersecting line features that can be analyzed according to the line-finder of the example system and method;
14-17 illustrate light-to-dark polarity, dark-to-light polarity that can be analyzed according to the line-finder of the example system and method. light polarity), a diagram each showing an example of a line representing either a light-to-dark polarity or a dark-to-light polarity, or a mixed polarity thereof; and
Figure 18 illustrates the change in coverage score for lines found considering a user-defined mask.

An example vision system arrangement 100 that may be used in accordance with an example embodiment is shown in FIG. 1 . System 100 includes at least one vision system camera 110 and may include one or more additional optional cameras 112 (shown in dashed lines). Exemplary camera(s) 110, 112 include an image sensor (or imager) S and associated electronics that acquire image frames and transmit the image frames to a vision system processor 130. ), and this image frame may be instantiated in an independent processor and/or computing device 140. Cameras 110 (and 112) include appropriate lenses/optics 116 to focus on a scene containing the object 150 being inspected. Cameras 110 (and 112) may include internal and/or external illuminators (not shown) that operate according to the image acquisition process. Computational device 140 may be any acceptable processor-based system capable of storing and manipulating image data in accordance with example embodiments. For example, computing device 140 may include a PC (shown), a server, a laptop, a tablet, a smartphone, or other similar device. Computational device 140 may include suitable peripherals, such as a bus-based image capture card interconnected to the camera. In alternative embodiments, the vision processor may be partially or fully contained within the camera body itself and may be networked with other PCs, servers, and/or camera-based processors to share and process image data. Computing device 140 may optionally display a suitable display 142 capable of supporting vision system tools provided to vision system processor 130 and a suitable graphical user interface (GUI) capable of operating in accordance with processor 132. It can be included. It is understood that the display may be omitted in various embodiments and/or provided only for setup and service functions. The vision system tool may include any acceptable software and/or software acceptable for use in inspecting an object, such as commercially available from Cognex Corporation, Natick, MA. It may be part of a hardware package. The computing device may include associated user interface (UI) components, including, for example, a keyboard 144 and mouse 146, and a touchscreen in display 142.

Camera(s) 110 (and 112) image some or all of an object 150 located within the scene. Each camera defines an optical axis (OA), and a field of view is established around this optical axis based on optics 116, focal length, etc. Object 150 includes a plurality of edges 152, 154, and 156 respectively arranged in various directions. For example, object edges may include the edge of a cover glass mounted within a smartphone body. By way of example, the camera(s) may image the entire object or a specific location (eg, a corner where glass meets the body). A (common) coordinate space can be established with respect to the object, one of the cameras, or another reference point (e.g. a moving stage supporting the object 150). As shown, the coordinate space is represented by axes 158. These axes define exemplary orthogonal x, y, and z axes, and a rotation (θ _z ) about the z axis in the xy plane.

According to an example embodiment, vision system process 130 interacts with one or more applications/processes (running on computing device 140) that collectively include a set of vision system tools/processes 132. It works. These tools may include a number of conventional specialized applications used to analyze image data - for example, using various calibration tools and affine transform tools to transform acquired image data. It can be converted to a predetermined (e.g. common) coordinate system. Tools may be included to convert image grayscale intensity data to a binary image based on a predetermined threshold. Additionally, tools may be provided to analyze gradients in intensity (contrast) between adjacent image pixels (and subpixels).

Vision system process (processor) 130 includes a line-discovery process, tool, or module 134 that finds multiple lines in an acquired image, according to an example embodiment. Referring to FIG. 2, FIG. 2 graphically depicts an overview of a line-discovery procedure 200 according to an example embodiment. This procedure 200 consists of two main parts. Input image 210 is provided to the processor. As shown, this image includes a pair of intersecting edges 212 and 214. These intersecting edges may represent corner areas of object 150 described above. Edge point extractor 220 processes input image 210 to obtain a set of candidate edge points 230 including edge points 232 and 234 along edges 212 and 214, respectively. Edge points 232, 234 and their associated data (e.g. intensity gradient information described below) are provided to a recursive line finder 240, which determines the selected edge points. Performs a series of iterative processes. The goal of the iterative process is to attempt to match other found edge points to candidate line features. As a result of performing the line-discovery process 240, lines 252 and 254 are found, as shown. These results may be provided to other downstream processes 260 that use the information - such as alignment processes, robotic manipulation, inspection, ID reading, part/surface inspection, etc.

Referring to FIG. 3, FIG. 3 explains a procedure for extracting edge points according to one embodiment. One or more images containing objects or surfaces having edge features to be found in the scene are acquired (step 310). These image(s) may be extracted by a single camera or multiple cameras. In either case, the image pixels may (optionally) be transformed at step 320 into a new coordinate space and/or a common coordinate space with appropriate calibration parameters. This step may further include smoothing the image as described below. In certain embodiments, when multiple cameras are imaging discontinuous regions of a scene - for example, focusing on a corner region of a larger object - the common coordinate space is an empty region between the cameras' fields of view. can be considered. As described below, lines extending between these fields of view (e.g., an object edge connecting two discovered corner regions) may be extrapolated by the systems and methods of example embodiments. The edge points required to find the lines are extracted from the image(s) in the appropriate coordinate space by an edge point extractor using a gradient field projection at step 330. First, the gradient values are computed for each pixel, creating two images for the x and y gradient components. This image(s) is further processed by projecting the gradient field over many caliper-like regions. Unlike conventional caliper tools that project intensity values, by projecting the draft field according to this embodiment, the draft orientation can be preserved, which facilitates the subsequent line-finding process as described below. do.

Referring also to the diagram of FIG. 4 at step 340, the portion of the image (caliper-like region) 400 containing the candidate edge feature is divided into a gradient field (represented by a plurality of projections 410, 420, 430). The projection is taken and searched over the (approximately) expected orientation of the edges in the search direction (arrow SD), which is repeated over region 400 in the orthogonal projection direction (arrow PD). The edges in each projection (e.g., projection 420) appear as local maxima in the gradient field 440 associated with this projection. In general, a series of edge points in the projection associated with an edge represent an intensity gradient (vectors 552, 554) perpendicular to the direction of extension of this edge. As explained below, the user can define the projection direction based on the expected line orientation. Alternatively, this may be provided by default or by another mechanism - for example by analyzing features within the image.

Two particle size parameters are involved in the gradient projection step described above. Before calculating the gradient field, the user can choose to smooth the image using an isotropic Gaussian kernel. The first granularity determines the size of this Gaussian smoothing kernel. As shown in diagram 500 of FIG. 5, the image 210 is smoothed by applying a Gaussian kernel of an appropriate size (e.g., large 512, medium 514, small 516). Thus, the first granularity parameter determines the size of the isotropic Gaussian smoothing kernel before calculating the gradient field.

After calculating the gradient field, a Gaussian-weighted projection is performed by this process rather than the uniform weighting performed in conventional caliper tools. Accordingly, the second granularity parameter is the size of the one-dimensional (1D) Gaussian kernel used during gradient field projection, as shown in Figure 6, where region 600 has received Gaussian-smoothed kernels 610, 620, and 630. Decide. During typical operation, the user verifies (using the GUI) all extracted edges overlaid on the image, and then determines how many extracted edges along the line are to be found, while avoiding an excessive number of edges due to background noise in the image. Adjust the granularity and contrast thresholds until they look satisfactory. In other words, this step allows the signal-to-noise ratio to be optimized for the image characteristics. This adjustment may be performed automatically by the system using default values in some embodiments. It is understood that using a Gaussian weighting function is one approach among several approaches to weighting projections, including (for example) uniform weighting.

The overall flow of extracting and projecting the gradient field is graphically depicted in diagram 700 of FIG. 7. Two granularity parameters, the isotropic Gaussian kernel 710 and the 1D Gaussian kernel 720, are shown in each half of the overall diagram 700. As shown, each acquired image 210 is subjected to smoothing and decimation 730. The resulting image 740 is subjected to a gradient field calculation 750 as described above, resulting in two gradient images 752 and 754. These gradient images are also represented by g _x and g _y , which represent two orthogonal axes in a common coordinate space. In addition to the two gradient images, the processed intensity information is also used to calculate normalized contrast in one embodiment - an embodiment described below, so that the intensity image 756 is also typically (1D Gaussian kernel 720 ) is understood to be subjected to a smoothing, decimation and projection process (760) using a Gaussian-weighted projection (770) based on ). This results in a projection profile of gradient image 772 (g _x ), image 774 (g _y ), and intensity image 776 .

Referring also to step 350 in procedure 300 (FIG. 3), qualified edge points are extracted by combining the 1D projection profiles of the x and y gradient images. This is accomplished using calculating raw contrast (780) and calculating normalized contrast (790) based on the intensity image. More specifically, any local peak with raw projected gradient magnitude and normalized projected gradient magnitude exceeding the respective threshold is considered a candidate edge point for finding the subsequent line according to the following example formula:

(g _x ² + g _y ² ) ^1/2 > T _ABS

(g _x ² + g _y ² ) ^1/2 /I > T _NORM

_{where g ​ ​} -is the normalized contrast threshold for the normalized projected gradient magnitude.

In particular, when the absolute contrast and normalized contrast of a point both exceed the respective thresholds, only this point is considered as a candidate edge point. This is shown in the upper right quadrant 810 in an example graph 800 of normalized contrast threshold (T _NORM ) versus absolute contrast threshold (T _ABS ). The use of dual thresholding (an absolute threshold and a normalized threshold) is generally different from the traditional approach, which typically uses an absolute contrast threshold. The benefit of a dual contrast threshold is evident, illustratively, when the image contains both dark and light intensity regions containing edges of interest. To detect edges in dark areas of the image, it is desirable to set a low contrast threshold. However, these low contrast settings may detect false edges in bright parts of the image. Conversely, to avoid detecting false edges in bright areas of the image, it is desirable to set a high contrast threshold. However, at high contrast settings the system may not be able to properly detect edges in dark areas of the image. By using a second normalized contrast threshold in addition to the traditional absolute contrast threshold, the system can properly detect edges in both dark and bright regions and avoid detecting false edges in bright regions of the image. You can. Thus, through the use of a dual contrast threshold, the speed and robustness of the subsequent line discovery step in the overall process is maximized by being able to detect relevant edges while avoiding spurious edges. It performs the function of

Referring further to procedure step 350 (FIG. 3), once all edge points have been extracted, these edge points are represented and stored in a data structure convenient for subsequent line-finders to operate in. For example, the following set (tuple):

It is expressed as p = (x,y,gx,gy,gm,go,I,gm/I,m,n),

Here, (x,y) is the position of the edge point, (g _x ,g _y ) is the value of each x-gradient and y-gradient projection, and ( _{g m} , _{g o} ) is are the computed gradient magnitude and orientation, I is the intensity at the edge point location, g _m /I is the intensity-normalized contrast obtained by dividing the gradient magnitude (g _m ) by the intensity (I), and m is is the image index, and n is the projection area index. As with standard caliper tools, the positions of edge points can be interpolated to improve accuracy.

The process of extracting edge points is generally understood to operate to perform a gradient field projection in a single direction that substantially matches the expected line angle. The tool is thus most sensitive to edges of this angle, and its sensitivity drops off progressively for edges of other angles, where the rate of drop depends on the grain size setting, which indirectly determines the gradient field projection length. As a result, this process is limited to finding lines with angles “near” the expected line angle, according to the angle range specified by the user. Although this process has been adapted to find non-orthogonal lines, in some embodiments this process finds arbitrary angular lines across 360 degrees by performing projections in multiple directions, including orthogonal directions (omni-directional lines). It is considered that it can be generalized to discover ).

Referring now to step 360 in procedure 300 (FIG. 3), edge point candidates that exceed the threshold are provided to the line-finder in accordance with an exemplary embodiment. For example, line-finders operate recursively and use (for example) RANSAC (RANdom SAmple Concensus)-based techniques. See also line-discovery procedure 900 in Figure 9. At step 910, the user, via a GUI (e.g.), selects the expected line in the image along with the expected angle, angle tolerance, distance tolerance, and (by way of example) the minimum coverage score (generally defined below). Specifies the maximum number of . This parameter is used by the line-finder to trigger the next process. This line is found for each subregion of the image by recursively running the RANSAC line finder, with edge outliers from one step becoming the input points for the next step. Accordingly, at step 920, procedure 900 selects a pair of edge points that are part of a group of edge points identified as extreme values in an edge-discovery process. Procedure 900 attempts to fit a model line to selected edge points based on matching gradient values (within a selected tolerance range) that match the model line. At step 924, one or more line candidate(s) from step 922 are returned. Each line-discovery step returns a candidate line, its normal values, and its outlier values. The returned line(s) are computed on normal edge points with positions and gradients matching the line candidate (step 926). At step 928, the candidate line with the highest normal count is identified. The line-finding steps described above (steps 920-928) terminate when the maximum number of RANSAC iterations allowed is reached (decision step 930). The maximum number of iterations within each line-detection step is calculated automatically using the assurance level specified by the user and an internally computed worst-case ratio of outliers. Each line discovery step returns the line with the maximum number of captured edge points among all iterations, according to user-specified custom tolerances, geometric constraints, and polarity. Each edge point can only be assigned to the normal value list of a single line, and each line is allowed to contain at most one edge point from each projection region. The gradient orientation of an edge point along with its location is used to determine whether the edge point should be included in the normal value list of candidate lines. In particular, the edge points must have a gradient orientation that matches the angle of the candidate line.

If the decision step 930 determines that more iterations are allowed, the outliers from the best normal value candidates are returned to the RANSAC process (step 920) for use in finding line candidates (step 940).

During each RANSAC iteration, two edge points belonging to different projective regions are randomly selected, and there may be a line fitting these two points. The resulting candidate line receives further consideration only if the angle of the candidate line matches the draft angles of the two edges at the pair of points and the angle of the line matches the uncertainty range specified by the user. Receive. Typically, the draft directions of edge points are nominally orthogonal, but are allowed to differ by a user-configured angular tolerance. If the candidate line passes these initial tests, the number of inlier edge points is evaluated; if it does not pass, a new RANSAC iteration is initiated. An edge point is considered a normal value for a candidate line only if its draft direction and location match the line based on user-specified draft angle and distance tolerances.

Once the RANSAC iteration reaches the maximum (decision step 930), the normal values of the best found line candidates are refined using (e.g.) least squares regression or another acceptable approximation technique. Obtain a line fit and re-evaluate the set of normal edge points by repeating this step up to N times (e.g. 3 or more times) until the number of normal values increases or decreases further (step 960 ). This is the line indicated as found in step 970.

Decision step 980 determines whether more lines are to be found ((e.g., based on searching additional sub-regions or other criteria) and, if so, the process returns to step 920 to create a new line. Operates on edge points of the set (step 982). If the points have been run through or the maximum iteration count has been reached, procedure 900 returns at step 990 a set of lines (i.e., multiple) found in the image.

The multi-line finder is adapted to make final adjustments to existing results in cases where two line results intersect each other in the inspection area. Generally, as shown in FIGS. 10 and 11, for closely spaced parallel lines 1010 and 1020, erroneous line results (i.e. FIG. 10) can often be obtained due to the statistical nature of the RANSAC procedure. . However, when these errors occur, swapping groups of normal values (arrow 1120 in group 1110 in Figure 11) often results in an accurate line with increased coverage score and reduced fit residual. can be found. Exchanging points may be most effective when the image contains closely spaced parallel lines, as shown. Conversely, when the image contains lines 1210 and 1220 that actually intersect each other as shown in FIGS. 12 and 13, the coverage score is 1230), the original results obtained before the exchange are maintained to successfully detect the intersecting lines by the process.

The RANSAC procedure is understood to mean that the line-finder is just one of several techniques that can fit points into lines. In an alternative embodiment, candidate points may be selected according to a set displacement between these points, or the image may be processed using (for example) an exhaustive search technique. Accordingly, as used herein, reference to RANSAC technology should be broadly interpreted to include several similar point-fitting technologies.

Additional features of the systems and methods may be provided. These features include supporting mixed-polarization, automatically calculating projection zone widths, supporting multi-view line discovery, and eliminating optical distortion by removing pre-warpage from the input image. It includes doing. These functions are explained further below.

With further reference to the example of Figures 14-16, the line-finding system and method of an example embodiment may include a standard light-to-dark polarity, dark-to-light polarity, and Either polarity setting (respectively) is generally supported. Furthermore, the system and method can further support a mixed-polarity setup (FIG. 17) where the light-to-dark polarity and dark-to-light polarity characteristics are both visible on the same line. Line-discovery results for all four settings are shown in the following figures. In an exemplary embodiment, the system and method may include a mixed-polarity setup that can find a single line containing edge points of opposite polarity. This is different from the conventional setup where all the edge points of a single line are of one polarity - but have only one polarity - "either" polarity. A mixed-polarity setup may be advantageous when used to analyze (for example) the light and dark checkerboard of a calibration plate, among other applications.

Users can choose improved shift invariance for line-finding. In these cases, the edge point extractor uses substantially overlapping projective regions to improve result stability. If the regions do not overlap, the pixels considered can potentially shift from the projection region when the image is shifted, resulting in poor shift invariance in the line-finding result. Overlapping projective areas ensure that the pixels considered are continuously covered by the projective areas. When overlapping projective regions are used, incremental operations can be performed to maintain speed with as low a level of optimization as possible.

The user can provide a mask that omits certain portions of the image and/or the imaged surface obtained from analysis of line features. This may be desirable if the surface contains known line features that are not of interest (e.g., barcodes that are analyzed by other mechanisms, text, and any other structure unrelated to the task for which the lines are found). Accordingly, the edge point extractor can support image masking in which areas of the image that “don’t care” can be masked out and areas that “don’t care” can be masked in. When such masking occurs, the coverage scores of the found lines are exemplarily reweighted according to the number of edge points within the mask.

Referring to the example image region 1800 of FIG. 18, FIG. 18 shows coverage scores when image masks are present, and the effect of image masking on these coverage scores. The edge point extractor supports image masking, where “regions of no interest” in the image can be masked out. As shown, found lines 1810 are characterized by edge points of interest (based on “interest” mask regions 1820). These edge points of interest are composed of an edge point of interest normal value 1830 on line 1810 and an edge point of interest outlier value 1840 on line 1810. Edge points 1850 of no interest in the line 1810 exist between regions of interest 1820 of the mask as shown in this example, and the coverage score is calculated even when these edge points of no interest are present on the line at normal values. not included in Potential locations 1860 of edge points along line 1810 are also determined as shown. These potential locations are located between known points at predictable intervals based on the spacing of the points found. Illustratively, the coverage scores of found lines are reweighted according to the number of edge points within the mask. So the coverage score changes as follows:

Coverage score = number of edge normals of interest for the line/(number of edge normals of interest for the line + edge outliers of interest for the line + number of potential locations of interest for edge points).

After executing the line-discovery process according to the systems and methods herein, the discovered lines may be classified in several ways based on classification criteria specified by the user (e.g., via a GUI). The user can select from intrinsic classification measures such as normal coverage score, intensity or contrast. Users can also choose from extrinsic classification measures such as signed distance or relative angle. When using an external classification measure, the user can specify a reference line segment from which to calculate the external measure of the found lines.

As generally described above, the system and method may include a Multi-Field-of-View (MFOV) overload that can pass image vectors from different fields of view to the process. The images should all be in a common client coordinate space based on calibration. As mentioned above, this functionality can be extremely advantageous in application scenarios where multiple cameras are used to capture sub-areas of a single part. Because edge points retain gradient information, projected line features between gaps in the field of view can still be analyzed (when the gradients in both FOVs match a given line orientation and alignment in each FOV).

In particular, these systems and methods do not require that distortion (in the image) be removed (i.e., that the image be undistorted) in order to remove non-linear distortion, assuming that the distortion is not significant. If the image is not distorted, the system and method can still detect candidate edge points and map the point's location and gradient vector through a non-linear transformation.

It is clear that the line-finder provided in accordance with this system and method and several alternative embodiments/improvements is an effective and powerful tool for determining multiple line features under various conditions. In general, when these systems and methods are used to discover line features, there is no specific limitation on the maximum number of lines found in an image. Only memory and computation time place practical limits on the number of lines that can be discovered.

The above is a detailed description of exemplary embodiments of the present invention. Various modifications and additions may be made without departing from the spirit and scope of the present invention. Features of each of the various embodiments described above may be combined, where appropriate, with features of other described embodiments to provide multiple combinations of features in associated new embodiments. Furthermore, although the foregoing describes a number of distinct embodiments of the apparatus and method of the invention, those described herein are merely illustrative of the application of the principles of the invention. For example, as used herein, the terms “process” and/or “processor” should be broadly interpreted to include various electronic hardware and/or software-based functions and components (and alternatively functional “modules”). " or "element"). Furthermore, the depicted process or processor may be combined with other processes and/or processors or may be divided into several sub-processes or processors. These sub-processes and/or sub-processors may be combined in a number of ways according to embodiments disclosed herein. Additionally, any function, process and/or processor herein is explicitly stated to be implemented using electronic hardware, software consisting of a non-transitory computer-readable medium of program instructions, or a combination of hardware and software. is considered. Additionally, as used herein, “vertical,” “horizontal,” “up,” “down,” “bottom,” “top,” “side,” “front,” “posterior,” “left,” Various orientation and placement terms such as "right" are used only as relative conventions and are not used as absolute directions/positions in a fixed coordinate space, such as the direction of gravity. Additionally, when the terms "substantially" or "approximately" are used for a given measurement, value, or characteristic, these terms refer to the amount that is within the normal operating range to achieve the desired result, but this does not take into account the inherent inaccuracies of the system and the allowable Includes some variation due to within-tolerance errors (e.g., 1-5 percent). Accordingly, this description should be construed as illustrative only and is not intended to limit the scope of the invention.

Claims (20)

A system for discovering line features in an image, comprising:
A graphical user interface (GUI) configured to receive user inputs; and
a processor in communication with the graphical user interface;
The processor:
receive image data of a scene including line features;
receive user input from the graphical user interface including edge point inlier gradient angle tolerance;
analyze the image data to obtain edge points corresponding to edge features within the image;
generating a plurality of lines consistent with the edge point normal value gradient angle tolerance and the edge points obtained from the image data; and
A system configured to display the plurality of lines in the image via the graphical user interface. According to claim 1,
The system of claim 1, wherein the user input further includes at least one of: maximum number of expected lines, distance tolerance, expected angle, and minimum coverage score. According to claim 1,
wherein the user input further comprises a mask configured to omit a portion of the image data prior to generating the plurality of lines. According to claim 1,
The system of claim 1, wherein generating the plurality of lines includes performing a line fit approximation. According to claim 4,
The system of claim 1, wherein the line fit approximation is least squares regression. According to claim 1,
The system of claim 1, wherein the user input further includes polarity selection. According to claim 1,
The system of claim 1, wherein the processor is further configured to smooth the image data via a smoothing kernel. According to claim 7,
The system of claim 1, wherein the smoothing kernel includes a Gaussian kernel. According to claim 1,
wherein the user input further includes an expected angle, and the processor is configured to obtain the edge points by executing field projections in a single direction that substantially matches the expected angle. . According to claim 1,
The system of claim 1, wherein the edge points are selected based on a threshold defined by absolute contrast and contrast normalized based on the average intensity of the image data. According to claim 1,
The system further comprising a camera configured to capture the image and provide the image data to the processor. According to claim 1,
The system further comprising a display configured to provide the graphical user interface. According to claim 1,
Obtaining the edge points is:
(a) calculating a gradient vector field from the image data;
(b) projecting the gradient vector field over a plurality of gradient projection sub-regions; and
(c) finding a plurality of edge points in each of the gradient projection sub-regions based on projected gradient data;
system, including. According to claim 1,
Obtaining the edge points is:
(a) applying the image data to a gradient field calculation process to generate two gradient component images and an intensity image;
(b) applying a weighted projection to the two gradient component images and the intensity image to generate one-dimensional (1D) projection profiles of the two gradient component images and the projection image; and
(c) combining the 1D projection profiles of the two gradient component images to extract edge points;
system, including. A method for discovering line features in an image, comprising:
Receiving image data of a scene including line features;
Receiving user input including edge point normal value gradient angle tolerance from a graphical user interface;
analyzing the image data to obtain edge points corresponding to edge features within the image;
generating a plurality of lines matching the edge point normal value gradient angle tolerance and the edge points obtained from the image data; and
A method comprising displaying the plurality of lines in the image via the graphical user interface. According to claim 15,
Before analyzing the image data, the method further includes smoothing the image data through a smoothing kernel. According to claim 15,
Receiving a mask through the user input; and
The method further includes omitting a portion of the image data based on the mask prior to generating the plurality of lines. According to claim 15,
The method of claim 1, wherein generating the plurality of lines includes performing a line fit approximation. According to claim 15,
Obtaining the edge points is:
(a) calculating a gradient vector field from the image data;
(b) projecting the gradient vector field over a plurality of gradient projection sub-regions; and
(c) finding a plurality of edge points in each of the gradient projection sub-regions based on projected gradient data;
Method, including. A method for discovering line features in an image, comprising:
Receiving image data of a scene including line features;
Receiving user input corresponding to the mask and edge point normal value gradient angle tolerance;
omitting a portion of the image data corresponding to the mask;
analyzing remaining image data to obtain edge points corresponding to edge features within the image;
generating a plurality of lines matching the edge point normal value gradient angle tolerance and the edge points obtained from the image data; and
A method comprising displaying the plurality of lines in the image.