CN115100226B - A Contour Extraction Method Based on Monocular Digital Image - Google Patents

Abstract

The invention discloses a contour extraction method based on a monocular digital image. In the scale space of the image, the Gaussian difference pyramid is utilized to combine the luminance and chrominance information of the digital image, the chrominance information gradient image and the luminance information gradient image of the image are fused together, the calculated edge direction of each point forms an edge tangent line flow, the Gaussian difference algorithm based on the flow is adopted to detect and extract the contour edge line of the digital image, the problem of poor continuity caused by isotropy of the traditional Gaussian difference kernel is overcome, and a digital image contour map which keeps significant edge characteristics and removes noise and interference information is generated. The beneficial effects of the invention are as follows: and organically combining the brightness information and the color information, fully considering the edge tangential flow information of the image, reducing the interference of noise, and extracting the edge contour of the image which is clear and accurate.

Description

A Contour Extraction Method Based on Monocular Digital Image

Technical Field

The invention relates to the technical field related to image processing, and in particular to a contour extraction method based on a monocular digital image.

Background Art

Image contours refer to the lines and boundaries of areas of interest in digital images, including object boundaries and area boundaries defined by sudden changes in brightness, color, or texture. Human vision usually distinguishes target objects by their edges and contours. In digital images, edge contours are important features for distinguishing different areas. Edge detection is a basic task in digital image processing and is crucial for analyzing image content. Therefore, it is widely used in many computer vision, target recognition, and automatic detection.

Generally speaking, there are two main types of edge detection technology. The first region-based technology classifies pixels with high similarity into a region by identifying and analyzing the color, brightness and texture attributes in digital images, and extracts edge information of the image by detecting the boundaries between regions. The second line-based extraction technology uses high contrast in brightness, color or texture to find lines or boundaries. There are three types of region-based contour extraction technologies: region growing, region segmentation and region merging. Line-based technologies include active contour technology, edge detection-based technology and edge grouping-based technology.

In digital image processing, there are several edge detection methods available, which can be divided into first-order and second-order difference detection. In the first-order difference detection, the input digital image is convolved with a suitable mask to generate a gradient image. The edges in the image are detected by thresholding, and the edge information is converted to white and the non-edge information is converted to black to obtain the edge information. Common first-order differential operators include Roberts operator (cross differential algorithm), Prewitt operator, Sobel operator, and Canny operator. In the second-order difference detection, the digital image is first smoothed by an adaptive filter. Since the second-order derivative is very sensitive to noise, the filtering function is very important. Second-order differential operators include Laplace operator, Zuniga-Haralick localization operator, LOG operator (Laplacian of Gaussian operator) and DOG operator (Gaussian difference operator).

In the process of digital image contour extraction, there are problems such as discontinuity of lines and region boundaries, complex shapes, textures and a lot of noise.

Summary of the invention

The present invention aims to overcome the above-mentioned deficiencies in the prior art and provides a contour extraction method based on a monocular digital image, which reduces noise interference and improves edge contour clarity.

In order to achieve the above object, the present invention adopts the following technical solutions:

A contour extraction method based on a monocular digital image specifically comprises the following steps:

(1) Image scale space: Under the premise of keeping the image size unchanged, the structural information of the digital image at each scale is obtained through smoothing and filter iteration. On this basis, the image at each scale is studied and processed using a preset experimental window to achieve the effect of image preprocessing.

(2) Gaussian difference operator edge detection: The window for observing the image is set to a fixed value. The pixel size of the image observed under this window changes, which generates an image pyramid. Based on the image pyramid, the image is operated by Gaussian smoothing and downsampling. The obtained set of processed images are arranged in a set to obtain a Gaussian pyramid. The prediction residual is calculated between each group of upper and lower images of the Gaussian pyramid to form a Gaussian difference pyramid and obtain the Gaussian difference operator DOG.

(3) Flow-based Gaussian difference edge detection: The image is preprocessed by Gaussian smoothing and image grayscale to obtain a gradient image, and then the extracted contour lines are processed using a bilateral filter operation to construct an edge tangent flow and harmonize the image. The contour edge lines of the digital image are then extracted using the Gaussian difference algorithm based on the edge tangent flow.

In the scale space of the image, the Gaussian difference pyramid is used to combine the brightness and color information of the digital image, and the image's color information gradient image and brightness information gradient image are fused together. The edge direction of each point is calculated to form an edge tangent flow. The flow-based Gaussian difference algorithm is used to detect and extract the contour edge line of the digital image, overcoming the poor coherence problem caused by the isotropy of the traditional Gaussian difference kernel, and generating a digital image contour map that retains significant edge features and removes noise and interference information. The method is compared with several commonly used contour extraction methods, and it can be seen that the contour extraction results obtained by this method are more coherent and clear. That is, this method proposes a simple edge detection method based on computational geometry technology, which organically combines brightness information and color information, fully considers the edge tangent flow information of the image, reduces noise interference, and extracts a clearer and more accurate edge contour of the image.

Preferably, in step (1), taking image I(x, y) as an example, the scale space T _t is generated by an image smoothing operator with a scale parameter t. The scale parameter t reflects the degree of smoothing of the digital image. {T _t } _t∈R is called the image of the two-dimensional image I(x, y) under the scale parameter t. R refers to a set of real numbers. The larger the parameter t is, the simpler the content of the image is.

Preferably, in step (1), the scale space operator T _t needs to satisfy visual invariance, including the following:

Grayscale invariance: T _t (I+h) = T _t (I) + h, where h is an arbitrary constant;

Contrast invariance: T _t (f(I)) = f(T _t (I)), where f is an arbitrary non-decreasing real function;

Translation invariance: T _t (τ _a (I)) = τ _a (T _t (I)), where τ _a (I) = I (x + a), a is an arbitrary constant;

Scalability invariance: satisfies t'(t,δ)＞0, and can make H _δ T _t ＝T _t H _δ hold, and H _δ I＝I(δx), where δ is an arbitrary positive real number and t is a scale parameter;

Euclidean invariance: For any orthogonal matrix R, there exists T _t (R·I) = R·T _t (I), where (R·I)(x) = I(R·x);

Affine invariance: For any affine transformation A and any scale parameter t, t'(δ,A)＞0 and AT _t ＝T _t A holds;

Here, T _t (I) refers to the scale space operator function, and I refers to the input image I(x,y).

Preferably, in step (2), the image pyramid is used for image compression and computer vision, and multiple resolution images from the same original image are arranged in a pyramid shape from large to small, which are obtained by stepwise down sampling until a certain termination condition is reached. In the image pyramid, the higher the level of the pyramid, the smaller the size and resolution of the image. The pixel size of the base level J is 2 ^J ×2 ^J or N×N, where J=log ₂ N, the size of the 0th level vertex is 1×1, that is, a pixel point, and the pixel size of the jth level is 2 ^j ×2 ^j , where 0≤j≤J. Considering the image distortion problem, the image pyramid will be shortened to P+1 level, where 1≤P≤J, and j=JP, ..., J-2, J-1, J. The level is limited to P to reduce the resolution of the original digital image approximately, where the total number of pixels in the P+1 level pyramid (P＞0) is:

Preferably, in step (2), each image resolution size is a group, and there are several layers in the Gaussian pyramid. Each layer is constructed by Gaussian smoothing and downsampling on the basis of the previous layer of image. The specific construction process is as follows:

(21) The first layer of the first group of the image pyramid is the original digital image, which is convolved with a Gaussian kernel, with σ as the standard deviation, and the new image is placed in the second layer of the first group, where the Gaussian convolution formula is:

Where D(x,y) is the distance between the midpoint (x,y) in the frequency domain and the midpoint of the frequency rectangle;

(22) Use the new standard deviation σ' to perform Gaussian kernel convolution on the image of the second layer of the first group, and place the resulting image in the third layer of the first group of the image pyramid;

(23) Repeat step (22) to finally obtain the first group of N layers of images. In each group of results of the Gaussian pyramid, each layer of images is obtained by convolving the same image with different standard deviations.

(24) Down-sampling the original image, placing the resulting image in the first layer of the second group of the image pyramid, and performing the same three steps on it to obtain an N-layer image;

(25) Repeat step (24) to obtain M groups of N-layer images and finally construct a Gaussian pyramid.

Preferably, in step (2), the Gaussian difference operator DOG is obtained as follows:

In the scale space constructed by the Gaussian pyramid, σ is the coordinate of the scale space, M is the number of groups in the Gaussian pyramid structure, and N is the number of layers of each group in the Gaussian pyramid;

Among them, σ ₀ is the reference scale, m is the group number in the Gaussian pyramid, and n is the number of the layer within the group; in the Gaussian pyramid, there are M groups of N layers of images, and the images in the Gaussian pyramid use the symbol (m, n) for positioning and one-to-one correspondence;

In the Gaussian pyramid, the Gaussian filter of the image is expressed as:

Substituting different parameter values of σ, we get another Gaussian filtered image:

The Gaussian filter function is:

Then, the difference between the two Gaussian filtered images is calculated to obtain:

Where G _σ1 -G _σ2 is the Gaussian difference operator DOG, and we get:

Preferably, in step (3), the gradient image is obtained as follows:

At the (x,y) pixel of image f, the intensity and direction of the surrounding pixel set are determined. The tool used is the gradient. To represent it, and define it in vector form:

The gradient vector indicates that the image f changes fastest and at the largest rate along the gradient direction at the pixel point (x, y). This is a significant geometric feature of the gradient vector. The length of is represented by M(x,y):

M(x,y) represents the value of the rate of change of the gradient vector direction. Among them, _gx , _gy and M(x,y) are the same as the original image in pixel size. They are obtained by taking the derivative of x and y at all pixel positions in the image f. M(x,y) is called the gradient image.

The angle of the gradient vector relative to the x-axis is:

The edge direction of a pixel is perpendicular to the gradient vector of that point, so the gradient is used to determine the edge strength and direction of a point;

Only brightness information can be obtained in grayscale images, so the image is converted from RGB space to YUV color space for processing, and the brightness and chromaticity information of the image is obtained in YUV space. The conversion matrix is:

Among them, Y is the pixel brightness value in the image, and the three letters R, G, and B represent the three base colors of red, green, and blue respectively. Therefore, the brightness information of the image is:

Y＝0.299R+0.587G+0.144B

In the process of taking the first-order or second-order derivative of the image pixels, the influence of noise on the derivative result cannot be ignored. The Sobel operator, as a discrete differential operator, is effective in suppressing noise. Therefore, the Sobel operator is used to calculate the gradient of the image brightness information:

The gradient length of the brightness of the image is:

Because solving the square and square root takes a lot of time in the calculation process, in order to improve the calculation efficiency, the approximate value without square root is taken:

|ΔY|＝|ΔY _x |+|ΔY _y |

When calculating the gradient value of chromaticity information, the internationally accepted color measurement standard CIE-L ^* a ^* b ^* is applied. This standard can be applied to the calculation of the light source color and object color of the image. To convert the RGB image to the CIE-L ^* a ^* b ^* color space, first convert RGB to the XYZ color space:

Where XYZ is the tristimulus value of the object, and the conversion formula of CIE-L ^* a ^* b ^* space is:

X ₀ Y ₀ Z ₀ represents the tristimulus values of the CIE standard illuminant, L ^* is the psychological lightness, a ^* and b ^* are the psychological chromaticity;

Therefore, the gradient containing the image chromaticity information is expressed as:

Then the gradient length of the image chromaticity information is:

in, is the distance in CIE-L ^* a ^* b ^* color space:

For the convenience of calculation, the approximate values of the two are:

Similarly, take the approximate value of the gradient length of the image chromaticity information:

The obtained brightness gradient approximation and the obtained chromaticity gradient approximation are unified into a roughly the same numerical range and normalized:

ΔY'＝(ΔY-min)(max-min)

ΔC'＝(ΔC-min)(max-min)

From this, the brightness gradient and chromaticity gradient are obtained, and both satisfy a linear relationship. The fusion gradient is calculated as:

Preferably, in step (3), the specific method of constructing the edge tangent flow is as follows:

In order to retain enough detail information in the contour extraction of the image, the edge tangent flow ETF is used for smoothing;

The ETF construction filter is defined as:

Where Ω _μ represents the neighborhood of x, whose radius is μ, k is the normalization factor of the vector, t ^cur (y) represents the normalized tangent vector of y, and Φ(x,y) represents the direction of the vector t ^cur (y);

ω _s (x, y) in the detailed description is a spatial weight function, which is essentially a rectangular filter with a radius of μ, and its expression is:

ω _m (x, y) is the amplitude weight function, and its expression is:

Where e(x) represents the normalized gradient value at point x, and η represents the rate of descent;

ω _d (x, y) is the direction weight function, and its expression is:

ω _d (x,y)＝|t ^cur (x)·t ^cur (y)|

The amplitude weight function ω _m (x, y) and the direction weight function ω _d (x, y) play an important role in maintaining image features. The value of ω _d (x, y) decreases as the two normalized tangent vectors become perpendicular and increases as the two vectors become parallel. At this time, the sign function Φ(x, y)∈{1, -1} is used to represent the direction of t ^cur (y):

The initial fused gradient M ₀ (x) is obtained, and then the edge tangent flow t ⁰ (x) is obtained according to the vertical relationship of the gradient field. By iterating the formula ^ti (x)→ti ⁺¹ (x) 2-3 times, a good edge tangent flow ETF is obtained.

Preferably, in step (3), a one-dimensional Gaussian blur is calculated along the edge tangent orthogonal to the edge tangent by DOG, and edge alignment smoothing is performed by line integral convolution along the edge tangent flow. The two-channel flow guidance method is called the Gaussian difference algorithm based on the edge tangent flow FDOG. The FDOG filter is defined as:

Where l _s (t) represents the point on the line l _S at parameter t, l _S is perpendicular to the curve C _x and is used to represent the width of the curve compared to C _x at point x; C _x represents the flow curve at point x, s represents the arc length parameter, and its value range is [-S, S]; therefore, I(l _s (t)) represents the value represented by the input image I at point l _s (t). For the function f, the DOG edge model is used:

Where G _σ (x) is a single variable Gaussian function with a variance of 1; σ _c and σ _s respectively control the center edge interval and the interval size with the peripheral edge of the filtering result, and the values of both are σ _s = 1.6σ _c , which can make the function f closer to the Gaussian Laplace operator; ρ controls the level and sensitivity of the noise, and its value is between [0.79, 1.0];

The cumulative response formula of the FDOG filter along _Cx is:

After obtaining the Gaussian difference filtering result based on the edge tangent flow, FDOG obtains the binary image of the edge by thresholding:

Among them, τ controls the size of the center-surround difference and takes values in [0, 1].

The beneficial effects of the present invention are: organically combining brightness information and color information, fully considering the edge tangent flow information of the image, reducing the interference of noise, and extracting a clearer and more accurate edge contour of the image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of multi-scale expression of an image;

Fig. 2 is a schematic diagram of an image pyramid;

Figure 3 is a demonstration diagram of an image pyramid;

FIG4 is a system model diagram of the approximation and residual pyramids;

Fig. 5 is a schematic diagram of the Gaussian pyramid structure;

FIG6 is a schematic diagram of a Gaussian difference pyramid structure;

Fig. 7 is a flow chart of the contour extraction method;

FIG8 is a schematic diagram showing the principle of gradient confirmation of edge direction;

FIG9 is a diagram showing the effect of edge tangent flow construction;

FIG10 is a diagram of the FDOG filter structure;

FIG11 is a schematic diagram of multi-scale expression of an image.

FIG12 is a schematic diagram of multi-scale expression of an image.

DETAILED DESCRIPTION

The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

A contour extraction method based on a monocular digital image specifically comprises the following steps:

(1) Image scale space: Under the premise of keeping the image size unchanged, the structural information of the digital image at each scale is obtained through smoothing and filter iteration. On this basis, the image at each scale is studied and processed using a preset experimental window to achieve the effect of image preprocessing.

In computer vision, digital images show different degrees of smoothness in different application contexts. The change in grayscale value corresponds to the spatial phenomenon of the object and can be identified in areas of different sizes, ranging from a few pixels to large areas. Identifying the change in grayscale value is essential for decoding the inherent information in the image. The scale space of an image is an abstract framework formed in the process of combining the same image for observation and study by obtaining multiple images with different blur levels after various smoothing processes and filter iterations.

Under different scale parameters, the scale structure in the digital image is different. As the scale parameter increases, the subtle scale structure can be minimized, thereby achieving the purpose of separating the digital image structure. Therefore, by processing the digital image at a sufficiently coarse scale, the subtle scale structure that is irrelevant or not within the scope of the study can be removed as much as possible. Such an operation will greatly reduce the difficulty of processing digital images, and experiments have shown that image processing guided by such an idea is meaningful.

Taking image I(x,y) as an example, the scale space _Tt is generated by an image smoothing operator with scale parameter t. The scale parameter t reflects the degree of smoothing of the digital image. { _Tt } _t∈R is called the image of the two-dimensional image I(x,y) under scale parameter t. R refers to the set of real numbers. The larger the parameter t is, the simpler the image content is. The schematic diagram of the multi-scale expression of an image is shown in Figure 1.

The scale space operator T _t needs to satisfy visual invariance, including the following:

Satisfy grayscale invariance and contrast invariance: When the brightness and angle of light received by a three-dimensional object in a real scene change, the grayscale and contrast of the image perceived by the human eye will also change accordingly. Therefore, the scale space operator T _t needs to keep the grayscale and contrast unchanged during analysis.

Grayscale invariance: T _t (I+h) = T _t (I) + h, where h is an arbitrary constant;

Contrast invariance: T _t (f(I)) = f(T _t (I)), where f is an arbitrary non-decreasing real function.

Satisfy translation invariance, scaling invariance, Euclidean invariance and affine invariance: When the relative direction and angle between the real three-dimensional object and the observer change, the position, angle, size and shape of the image perceived by the human eye will also change accordingly. Therefore, the scale space operator T _t needs to be independent of the relative position, relative angle, size and affine transformation when performing analysis.

Translation invariance: T _t (τ _a (I)) = τ _a (T _t (I)), where τ _a (I) = I (x + a), a is an arbitrary constant;

Scalability invariance: satisfies t'(t,δ)＞0, and can make H _δ T _t ＝T _t H _δ hold, and H _δ I＝I(δx), where δ is an arbitrary positive real number and t is a scale parameter;

Euclidean invariance: For any orthogonal matrix R, there exists T _t (R·I) = R·T _t (I), where (R·I)(x) = I(R·x);

Affine invariance: For any affine transformation A and any scale parameter t, t'(δ,A)＞0 and AT _t ＝T _t A holds;

Here, T _t (I) refers to the scale space operator function, and I refers to the input image I(x,y).

(2) Gaussian difference operator edge detection: The window for observing the image is set to a fixed value. The pixel size of the image observed under this window changes, which generates an image pyramid. Based on the image pyramid, the image is operated by Gaussian smoothing and downsampling. The obtained set of processed images are arranged in a set to obtain a Gaussian pyramid. The prediction residual is calculated between each group of upper and lower images of the Gaussian pyramid to form a Gaussian difference pyramid and obtain the Gaussian difference operator DOG.

In scale space, the number of pixels of digital images observed at different scales is different. From another perspective, the window for observing the image is set to a fixed value, and the pixel size of the image observed under this window changes, which produces an image pyramid. The purpose of constructing an image pyramid is to obtain the expression of the image at different scales, and this scale is usually the resolution of the image. Digital images are obtained through continuous smoothing and sampling to obtain digital images of various pixel sizes.

The Gaussian difference operator DOG (Different of Gaussian) used in this method is established based on the image pyramid. Its edge extraction algorithm has a small convolution kernel, a fast algorithm speed, and can well retain the detail information of the original image. Therefore, the Gaussian difference operator has a significant effect in image edge detection.

In the multi-scale representation of images, one of the structures that views image information from the perspective of resolution is the image pyramid, which is efficient, clear and easy to understand. Image pyramid is used for image compression and computer vision. Multiple resolution images from the same original image are arranged in a pyramid shape from large to small. It is obtained by downsampling in stages until a certain termination condition is reached. As shown in Figure 2, the bottom of the pyramid is a multi-pixel representation of the original image, which is mostly the original image itself, while the top has only a small number of pixels or even only one pixel.

In the image pyramid, the higher the level of the pyramid, the smaller the image size and resolution. The pixel size of the base level J is 2 ^J ×2 ^J or N×N, where J=log ₂ N. The size of the vertex at level 0 is 1×1, that is, a pixel point. The pixel size of the jth level is 2 ^j ×2 ^j , where 0≤j≤J. Considering the image distortion problem, the image pyramid will be shortened to level P+1, where 1≤P≤J, and j=JP, ..., J-2, J-1, J. Limiting the level to P is to reduce the resolution of the original digital image approximately, where the total number of pixels in the P+1 level pyramid (P＞0) is:

Taking an original digital image of size 1024×1024 as the experimental object, the relationship between the image size and the number of levels at each level of the image pyramid is shown in the following table:

In the image pyramid, the higher the level of the digital image used in the experiment, the less information it contains, and of course the smaller its pixel size is. Figure 3 is a demonstration diagram of the image pyramid.

A system model of an approximate and prediction residual pyramid is constructed based on the image pyramid. Taking the J-th level image as input, the image required by the J-1-th level approximate pyramid is output through the approximate filter and the downsampler, and the image required by the J-th level supplementary prediction residual pyramid is output through the upsampler and the interpolation filter, as shown in Figure 4.

Both the approximate pyramid and the prediction residual pyramid are implemented in an iterative manner. In the first iteration, the original digital image is placed in the Jth level of the pyramid, and then the following three steps are performed P times, where j = J-P+1,…,J-2,J-1,J:

(1) Calculate an approximate image with reduced resolution of the input image at level j through an approximate filter and a downsampler, and place it at level j-1 of the approximate pyramid;

(2) the approximate image generated in step (1) is passed through an upsampler and an interpolation filter to generate an estimate of the j-th level image, where the dimension of the predicted image is the same as that of the j-th level image;

(3) Calculate the difference between the j-th level input image in step 1 and the predicted image in step 2 to obtain the j-th level prediction residual, and place it at the j-th level of the prediction residual pyramid.

The interpolation filter and approximate filter in the system model for constructing the approximate and prediction residual pyramid can adopt a variety of filtering techniques. Interpolation filters include nearest neighbor interpolation, bilinear interpolation, spline and wavelet techniques. In approximate filtering: using neighborhood averaging can obtain an average pyramid; without filtering, a sub-sampling pyramid can be obtained; using low-pass Gaussian filtering can obtain a Gaussian pyramid, which is also the pyramid structure used in this method.

The image is operated by Gaussian smoothing and subsampling, and a group of processed images are arranged in a set to obtain a Gaussian pyramid. Each image resolution size is a group, and there are several layers in the Gaussian pyramid. Each layer is constructed on the basis of the previous layer of image by means of Gaussian smoothing and downsampling. The specific construction process is as follows:

(21) The first layer of the first group of the image pyramid is the original digital image, which is convolved with a Gaussian kernel, with σ as the standard deviation, and the new image is placed in the second layer of the first group, where the Gaussian convolution formula is:

Where D(x,y) is the distance between the midpoint (x,y) in the frequency domain and the midpoint of the frequency rectangle;

(22) Use the new standard deviation σ' to perform Gaussian kernel convolution on the image of the second layer of the first group, and place the resulting image in the third layer of the first group of the image pyramid;

(23) Repeat step (22) to finally obtain the first group of N layers of images. In each group of results of the Gaussian pyramid, each layer of images is obtained by convolving the same image with different standard deviations.

(24) Down-sampling the original image, placing the resulting image in the first layer of the second group of the image pyramid, and performing the same three steps on it to obtain an N-layer image;

(25) Repeat step (24) to obtain M groups of N-layer images, and finally construct a Gaussian pyramid, whose structure is shown in FIG5 .

The Gaussian difference operator DOG is a further development of the Gaussian pyramid. The main pyramid structure is to calculate the prediction residual of each group of upper and lower images of the Gaussian pyramid to form a new Gaussian difference pyramid, and its structure is shown in Figure 6. The Gaussian difference operator DOG is obtained as follows:

In the scale space constructed by the Gaussian pyramid, σ is the coordinate of the scale space, M is the number of groups in the Gaussian pyramid structure, and N is the number of layers of each group in the Gaussian pyramid;

Among them, σ ₀ is the reference scale, m is the group number in the Gaussian pyramid, and n is the number of the layer within the group; in the Gaussian pyramid, there are M groups of N layers of images, and the images in the Gaussian pyramid use the symbol (m, n) for positioning and one-to-one correspondence;

In the Gaussian pyramid, the Gaussian filter of the image is expressed as:

Substituting different parameter values of σ, we get another Gaussian filtered image:

The Gaussian filter function is:

Then, the difference between the two Gaussian filter images in equation (5) and equation (6) is calculated to obtain:

Where G _σ1 -G _σ2 is the Gaussian difference operator DOG, and we get:

(3) Flow-based Gaussian difference edge detection: The image is preprocessed by Gaussian smoothing and image grayscale to obtain a gradient image, and then the extracted contour lines are processed using a bilateral filter operation to construct an edge tangent flow and harmonize the image. The contour edge lines of the digital image are then extracted using the Gaussian difference algorithm based on the edge tangent flow.

In the grayscale value of a digital image, the edge pixels and background pixels of the image contour have a large difference in most areas of the contour, but it does not rule out the existence of some areas with similar grayscale values. Therefore, in order to make the digital image contour extracted by edge detection have better edge information characteristics, this method uses a flow-based Gaussian difference algorithm to detect and extract the contour edge line of the digital image, pre-processes the image with Gaussian smoothing and image graying, and then uses filtering operations to process the extracted contour line. The contour extraction algorithm used in this method is shown in Figure 7.

In order to achieve the purpose of edge detection, the first-order derivative or second-order derivative of the image pixels is often used to find the intensity and direction of the edge pixel set. In this way, the part of the image pixel grayscale value that changes rapidly is identified and image segmentation is performed on it. However, the traditional method of using grayscale changes only considers the brightness information in the grayscale image, and the results of edge detection may have a certain error with the results perceived by the human eye. Therefore, on the basis of brightness information, this method adds the chromaticity information of the image to improve the accuracy of edge detection. The acquisition of gradient image is as follows:

At the (x,y) pixel of image f, the intensity and direction of the surrounding pixel set are determined. The tool used is the gradient. To represent it, and define it in vector form:

The gradient vector indicates that the image f changes fastest and at the largest rate along the gradient direction at the pixel point (x, y). This is a significant geometric feature of the gradient vector. The length of is represented by M(x,y):

M(x,y) represents the value of the rate of change of the gradient vector direction. Among them, _gx , _gy and M(x,y) are the same as the original image in pixel size. They are obtained by taking the derivative of x and y at all pixel positions in the image f. M(x,y) is called the gradient image.

The angle of the gradient vector relative to the x-axis is:

The edge direction of a pixel is perpendicular to the gradient vector of the point, so the gradient is used to determine the edge strength and direction of a point, as shown in FIG8 , which shows the principle diagram of the gradient determining the edge direction. In the figure, one square represents one pixel.

Only brightness information can be obtained in grayscale images, so the image is converted from RGB space to YUV color space for processing, and the brightness and chromaticity information of the image is obtained in YUV space. The conversion matrix is:

Among them, Y is the pixel brightness value in the image, and the three letters R, G, and B represent the three base colors of red, green, and blue respectively. Therefore, the brightness information of the image is:

Y＝0.299R+0.587G+0.144B (14)

In the process of taking the first-order or second-order derivative of the image pixels, the influence of noise on the derivative result cannot be ignored. The Sobel operator, as a discrete differential operator, is effective in suppressing noise. Therefore, the Sobel operator is used to calculate the gradient of the image brightness information:

According to formula (11), the gradient length of the brightness of the image is:

Because solving the square and square root takes a lot of time in the calculation process, in order to improve the calculation efficiency, the approximate value without square root is taken:

|ΔY|＝|ΔY _x |+|ΔY _y | (17)

When calculating the gradient value of chromaticity information, the internationally accepted color measurement standard CIE-L ^* a ^* b ^* is applied. This standard can be applied to the calculation of the light source color and object color of the image. To convert the RGB image to the CIE-L ^* a ^* b ^* color space, first convert RGB to the XYZ color space:

Where XYZ is the tristimulus value of the object, and the conversion formula of CIE-L ^* a ^* b ^* space is:

X ₀ Y ₀ Z ₀ represents the tristimulus values of the CIE standard illuminant, L ^* is the psychological lightness, a ^* and b ^* are the psychological chromaticity;

Therefore, the gradient containing the image chromaticity information is expressed as:

Then the gradient length of the image chromaticity information is:

in, is the distance in CIE-L ^* a ^* b ^* color space:

For the convenience of calculation, the approximate values of the two are:

Similarly, take the approximate value of the gradient length of the image chromaticity information in equation (21):

The brightness gradient approximation obtained by equation (17) and the chromaticity gradient approximation obtained by equation (24) are unified into a roughly the same numerical range and normalized:

ΔY'=(ΔY-min)(max-min) (25)

ΔC'=(ΔC-min)(max-min) (26)

From this, the brightness gradient and chromaticity gradient are obtained, and both satisfy a linear relationship. The fusion gradient is calculated as:

Images with random noise or textures may produce many disconnected and small edges. Some studies have adopted a better solution, which is to adjust the filter according to the approximate edge direction. The idea behind these methods is to first respond to brightness changes that occur at the edges, and then use edge-aligned blur to smooth these responses. The edge direction calculated at each point forms an edge tangent flow (ETF). The one-dimensional Gaussian blur along the edge tangent orthogonal to the edge tangent is calculated by DOG, and edge-aligned smoothing is performed by line integral convolution along the edge tangent flow. The two-channel flow guidance method is called the flow-based difference of Gaussians (FDOG) algorithm. The specific method of constructing the edge tangent flow is as follows:

In order to retain enough detail information in the image contour extraction, the edge tangent flow ETF is used for smoothing;

The ETF construction filter is defined as:

Where Ω _μ represents the neighborhood of x, whose radius is μ, k is the normalization factor of the vector, t ^cur (y) represents the normalized tangent vector of y, and Φ(x,y) represents the direction of the vector t ^cur (y);

Detailed description: ω _s (x, y) in equation (28) is a spatial weight function, which is essentially a rectangular filter with a radius of μ, and its expression is:

ω _m (x, y) is the amplitude weight function, and its expression is:

Where e(x) represents the normalized gradient value at point x, and η represents the rate of descent;

ω _d (x, y) is the direction weight function, and its expression is:

ω _d (x,y)＝|t ^cur (x)·t ^cur (y)| (31)

The amplitude weight function ω _m (x, y) and the direction weight function ω _d (x, y) play an important role in maintaining image features. The value of ω _d (x, y) decreases as the two normalized tangent vectors become perpendicular and increases as the two vectors become parallel. At this time, the sign function Φ(x, y)∈{1, -1} is used to represent the direction of t ^cur (y):

The initial fusion gradient M ₀ (x) is obtained by formula (27), and then the edge tangent flow t ⁰ (x) is obtained according to the vertical relationship of the gradient field. By iterating the formula ^ti (x)→ ^ti+1 (x) 2-3 times, a good edge tangent flow ETF is obtained. Figure 9 shows the effect of edge tangent flow construction with an ETF kernel size of 5.

In contour extraction, the traditional Gaussian difference filter has the problems of poor coherence and unclear directionality, which is caused by the isotropic structure of its kernel. This method adopts a flow-based Gaussian difference algorithm to overcome some of the previous problems. Figure 10 illustrates the Gaussian difference method based on edge tangent flow. Figure (a) is intercepted from Figure 9. In Figure (b), _Cx represents the flow curve at point x, s represents the arc length parameter, and its value range is [-S,S]. _lS represents being perpendicular to the curve _Cx and compared with _Cx at point x, which is used to represent the width of the curve.

The FDOG filter is defined as:

Where l _s (t) represents the point on the line l _S at parameter t, l _S is perpendicular to the curve C _x and is used to represent the width of the curve compared to C _x at point x; C _x represents the flow curve at point x, s represents the arc length parameter, and its value range is [-S, S]; therefore, I(l _s (t)) represents the value represented by the input image I at point l _s (t). For the function f, the DOG edge model is used:

Where G _σ (x) is a single variable Gaussian function with a variance of 1; σ _c and σ _s respectively control the center edge interval and the interval size with the peripheral edge of the filtering result, and the values of both are σ _s = 1.6σ _c , which can make the function f closer to the Gaussian Laplace operator; ρ controls the level and sensitivity of the noise, and its value is between [0.79, 1.0];

The cumulative response formula of the FDOG filter along _Cx is:

After obtaining the Gaussian difference filtering result based on the edge tangent flow, FDOG obtains the binary image of the edge by thresholding:

Among them, τ controls the size of the center-surround difference and takes a value in [0, 1], and most of the time it is 0.5.

As shown in Figure 11, this method selects two monocular digital images to test the contour extraction algorithm, where (a) is a simple calligraphy image with obvious edge features and a clear background, and (b) is a complex cartoon image with many contours and background color interference.

The edge detection algorithms using several different operators and filters are used to extract the contours of these two images with large differences, and the results of the contour edge detection are compared. Figure 12 is a comparison of the results.

In each contour extraction program, the two hysteresis thresholds used by the Canny algorithm are 150 and 100, and the Gaussian radius of the operator is 3; the Gaussian radius used in the Sobel algorithm is 3, and the differential orders in the x and y directions are both 1; the differential orders in the x and y directions in the Scharr filter are both 1, and the scaling factor when calculating the derivative is 1; the kernel size in the ETF algorithm is 5, and the threshold is 0.8; the parameters selected in this method are σ _c =0.3, σ _m =3, τ =0.8, and ρ =0.998.

By comparing Figure 11, we can see the differences between several algorithms. Among them, the contour noise and interference extracted by the Canny algorithm are relatively small, and the contour is thin and has a sense of breakage, and it is easy to identify image noise as an edge; the Sobel algorithm and the Scharr filter have the same problem, the edge is not clear enough in the simple calligraphy contour without background, and the Scharr filter is particularly obvious. The anti-interference ability of the background in the cartoon image contour with a complex background is poor, and a lot of noise will be generated; the image contour information extracted by the ETF algorithm will be broken when there is interference, and the contour is somewhat distorted; from the results, this algorithm can better retain the image contour when extracting the text contour and graphic image contour, and has strong anti-interference ability, and the generated picture is clearer.

In summary, in the scale space of the image, the Gaussian difference pyramid is used to combine the brightness and color information of the digital image, and the image's color information gradient image and brightness information gradient image are fused together. The edge direction of each point is calculated to form an edge tangent flow, and the flow-based Gaussian difference algorithm is used to detect and extract the contour edge line of the digital image, overcoming the poor coherence problem caused by the isotropy of the traditional Gaussian difference kernel. A digital image contour map is generated that retains significant edge features and removes noise and interference information, and is compared with several commonly used contour extraction methods. It can be seen that the contour extraction results obtained by this method are more coherent and clear.

Claims (8)

1. A contour extraction method based on a monocular digital image, characterized in that it specifically comprises the following steps: (1) Image scale space: Under the premise of keeping the image size unchanged, the structural information of the digital image at each scale is obtained through smoothing and filter iteration. On this basis, the image at each scale is studied and processed using a preset experimental window to achieve the effect of image preprocessing. (2) Gaussian difference operator edge detection: The window for observing the image is set to a fixed value. The pixel size of the image observed under this window changes, which generates an image pyramid. Based on the image pyramid, the image is operated by Gaussian smoothing and downsampling. The obtained set of processed images are arranged in a set to obtain a Gaussian pyramid. The prediction residual is calculated between each group of upper and lower images of the Gaussian pyramid to form a Gaussian difference pyramid and obtain the Gaussian difference operator DOG. (3) Flow-based Gaussian difference edge detection: The image is preprocessed by Gaussian smoothing and image graying to obtain a gradient image, and then the extracted contour lines are processed using a bilateral filter operation to construct an edge tangent flow and harmonize the image. After that, the Gaussian difference algorithm based on the edge tangent flow is used to extract the contour edge lines of the digital image. In step (3), the gradient image is obtained as follows: At the (x,y) pixel of image f, the intensity and direction of the surrounding pixel set are determined. The tool used is the gradient. To represent it, and define it in vector form: The gradient vector indicates that the image f changes fastest and at the largest rate along the gradient direction at the pixel point (x, y). This is a significant geometric feature of the gradient vector. The length of is represented by M(x,y): M(x,y) represents the value of the rate of change of the gradient vector direction. Among them, _gx , _gy and M(x,y) are the same as the original image in pixel size. They are obtained by taking the derivative of x and y at all pixel positions in the image f. M(x,y) is called the gradient image. The angle of the gradient vector relative to the x-axis is: The edge direction of a pixel is perpendicular to the gradient vector of that point, so the gradient is used to determine the edge strength and direction of a point; Only brightness information can be obtained in grayscale images, so the image is converted from RGB space to YUV color space for processing, and the brightness and chromaticity information of the image is obtained in YUV space. The conversion matrix is: Among them, Y is the pixel brightness value in the image, and the three letters R, G, and B represent the three base colors of red, green, and blue respectively. Therefore, the brightness information of the image is: Y＝0.299R+0.587G+0.144B In the process of taking the first-order or second-order derivative of the image pixels, the influence of noise on the derivative result cannot be ignored. The Sobel operator, as a discrete differential operator, is effective in suppressing noise. Therefore, the Sobel operator is used to calculate the gradient of the image brightness information: The gradient length of the brightness of the image is: Because solving the square and square root takes a lot of time in the calculation process, in order to improve the calculation efficiency, the approximate value without square root is taken: |ΔY|＝|ΔY _x |+|Δ _Y | When calculating the gradient value of chromaticity information, the internationally accepted color measurement standard CIE-L ^* a ^* b ^* is applied. This standard can be applied to the calculation of the light source color and object color of the image. To convert the RGB image to the CIE-L ^* a ^* b ^* color space, first convert RGB to the XYZ color space: Where XYZ is the tristimulus value of the object, and the conversion formula of CIE-L ^* a ^* b ^* space is: Where Y/Y ₀ ＞0.01 X ₀ Y ₀ Z ₀ represents the tristimulus values of the CIE standard illuminant, L ^* is the psychological brightness, a ^* and b ^* are the psychological chromaticity; therefore, the gradient containing the image chromaticity information is expressed as: Then the gradient length of the image chromaticity information is: in, is the distance in CIE-L ^* a ^* b ^* color space: For the convenience of calculation, the approximate values of the two are: Similarly, take the approximate value of the gradient length of the image chromaticity information: The obtained brightness gradient approximation and the obtained chromaticity gradient approximation are unified into the same numerical range and normalized: ΔY'＝(ΔY-min)(max-min) ΔC'＝(ΔC-min)(max-min) From this, the brightness gradient and chromaticity gradient are obtained, and both satisfy a linear relationship. The fusion gradient is calculated as: 2. According to claim 1, a contour extraction method based on a monocular digital image is characterized in that, in step (1), taking image I(x, y) as an example, the scale space T _t is generated by an image smoothing operator with a scale parameter t, and the scale parameter t reflects the degree of smoothing of the digital image. {T _t } _t∈R is called the image of the two-dimensional image I(x, y) under the scale parameter t, and R refers to a set of real numbers. The larger the parameter t, the simpler the content of the image. 3. The contour extraction method based on a monocular digital image according to claim 2 is characterized in that, in step (1), the scale space operator T _t needs to satisfy visual invariance, including the following: Grayscale invariance: T _t (I+h) = T _t (I) + h, where h is an arbitrary constant; Contrast invariance: T _t (f(I)) = f(T _t (I)), where f is an arbitrary non-decreasing real function; Translation invariance: T _t (τ _a (I)) = τ _a (T _t (I)), where τ _a (I) = I (x + a), a is an arbitrary constant; Scalability invariance: satisfies t'(t,δ)＞0, and can make H _δ T _t ＝T _t H _δ hold, and H _δ I＝I(δx), where δ is an arbitrary positive real number and t is a scale parameter; Euclidean invariance: For any orthogonal matrix R, there exists T _t (R·I) = R·T _t (I), where (R·I)(x) = I(R·x); Affine invariance: For any affine transformation A and any scale parameter t, t'(δ,A)＞0 and AT _t ＝T _t A holds; Here, T _t (I) refers to the scale space operator function, and I refers to the input image I(x,y). 4. A contour extraction method based on a monocular digital image according to claim 1, characterized in that, in step (2), an image pyramid is used for image compression and computer vision, and multiple resolution images from the same original image are arranged in a pyramid shape from large to small, which are obtained by stepwise down sampling until a certain termination condition is reached. In the image pyramid, the higher the level of the pyramid, the smaller the size and resolution of the image, the pixel size of the base level J is 2 ^J ×2 ^J or N×N, where J=log ₂ N, the size of the 0th level vertex is 1×1, that is, a pixel point, and the pixel size of the jth level is 2 ^j ×2 ^j , where 0≤j≤J; considering the image distortion problem, the image pyramid will be shortened to P+1 level, where 1≤P≤J, and j=JP,...,J-2,J-1,J; limiting the level to P to reduce the resolution of the original digital image approximately, where the total number of pixels in the P+1 level pyramid (P＞0) is: 5. The method for extracting contours based on a monocular digital image according to claim 4, wherein in step (2), each image resolution size is a group, and there are several layers in the Gaussian pyramid, and each layer is constructed by Gaussian smoothing and downsampling on the basis of the image of the previous layer, and the specific construction process is as follows: (21) The first layer of the first group of the image pyramid is the original digital image, which is convolved with a Gaussian kernel, with σ as the standard deviation, and the new image is placed in the second layer of the first group, where the Gaussian convolution formula is: D(x,y)＝[(xx ₀ ) ² +(yy ₀ ) ² ] ^1/2 Where D(x,y) is the distance between the midpoint (x,y) in the frequency domain and the midpoint of the frequency rectangle; (22) Use the new standard deviation σ' to perform Gaussian kernel convolution on the image of the second layer of the first group, and place the resulting image in the third layer of the first group of the image pyramid; (23) Repeat step (22) to finally obtain the first group of N layers of images. In each group of results of the Gaussian pyramid, each layer of images is obtained by convolving the same image with different standard deviations. (24) Down-sampling the original image, placing the resulting image in the first layer of the second group of the image pyramid, and performing the same three steps on it to obtain an N-layer image; (25) Repeat step (24) to obtain M groups of N-layer images and finally construct a Gaussian pyramid. 6. The method for extracting contours based on a monocular digital image according to claim 5, characterized in that: In step (2), the Gaussian difference operator DOG is obtained as follows: In the scale space constructed by the Gaussian pyramid, σ is the coordinate of the scale space, M is the number of groups in the Gaussian pyramid structure, and N is the number of layers of each group in the Gaussian pyramid; Among them, σ ₀ is the reference scale, m is the group number in the Gaussian pyramid, and n is the number of the layer within the group; in the Gaussian pyramid, there are M groups of N layers of images, and the images in the Gaussian pyramid use the symbol (m, n) for positioning and one-to-one correspondence; In the Gaussian pyramid, the Gaussian filter of the image is expressed as: Substituting different parameter values of σ, we get another Gaussian filtered image: The Gaussian filter function is: Then, the difference between the two Gaussian filtered images is calculated to obtain: Where G _σ1 -G _σ2 is the Gaussian difference operator DOG, and we get: 7. The method for contour extraction based on a monocular digital image according to claim 1, wherein in step (3), the specific method for constructing the edge tangent flow is as follows: In order to retain enough detail information in the contour extraction of the image, the edge tangent flow ETF is used for smoothing; The ETF construction filter is defined as: Where Ω _μ represents the neighborhood of x, whose radius is μ, k is the normalization factor of the vector, t ^cur (y) represents the normalized tangent vector of y, and Φ(x,y) represents the direction of the vector t ^cur (y); ω _s (x, y) in the detailed description is a spatial weight function, which is essentially a rectangular filter with a radius of μ, and its expression is: ω _m (x, y) is the amplitude weight function, and its expression is: Where e(x) represents the normalized gradient value at point x, and η represents the rate of descent; ω _d (x, y) is the direction weight function, and its expression is: ω _d (x,y)＝|t ^cur (x)·t ^cur (y)| The amplitude weight function ω _m (x, y) and the direction weight function ω _d (x, y) play an important role in maintaining image features. The value of ω _d (x, y) decreases as the two normalized tangent vectors become perpendicular and increases as the two vectors become parallel. At this time, the sign function Φ(x, y)∈{1, -1} is used to represent the direction of t ^cur (y): The initial fused gradient M ₀ (x) is obtained, and then the edge tangent flow t ⁰ (x) is obtained according to the vertical relationship of the gradient field. By iterating the formula ^ti (x)→ti ⁺¹ (x) 2-3 times, a good edge tangent flow ETF is obtained. 8. A contour extraction method based on a monocular digital image according to claim 7, characterized in that, in step (3), a one-dimensional Gaussian blur along a line orthogonal to the edge tangent is calculated by DOG, and edge alignment and smoothing are performed by line integral convolution along the edge tangent flow. The two-channel flow guidance method is called a Gaussian difference algorithm based on edge tangent flow FDOG, The FDOG filter is defined as: Where l _s (t) represents the point on the line l _S at parameter t, l _S is perpendicular to the curve C _x and is used to represent the width of the curve compared to C _x at point x; C _x represents the flow curve at point x, s represents the arc length parameter, and its value range is [-S, S]; therefore, I(l _s (t)) represents the value represented by the input image I at point l _s (t). For the function f, the DOG edge model is used: Where G _σ (x) is a single variable Gaussian function with a variance of 1; σ _c and σ _s respectively control the center edge interval and the interval size with the peripheral edge of the filtering result, and the values of both are σ _s = 1.6σ _c , which can make the function f closer to the Gaussian Laplace operator; ρ controls the level and sensitivity of the noise, and its value is between [0.79, 1.0]; The cumulative response formula of the FDOG filter along _Cx is: After obtaining the Gaussian difference filtering result based on the edge tangent flow, FDOG obtains the binary image of the edge by thresholding: Among them, τ controls the size of the center-surround difference and takes values in [0, 1].