CN111311657B - Infrared image homologous registration method based on improved corner principal direction distribution - Google Patents
Infrared image homologous registration method based on improved corner principal direction distribution Download PDFInfo
- Publication number
- CN111311657B CN111311657B CN202010169423.1A CN202010169423A CN111311657B CN 111311657 B CN111311657 B CN 111311657B CN 202010169423 A CN202010169423 A CN 202010169423A CN 111311657 B CN111311657 B CN 111311657B
- Authority
- CN
- China
- Prior art keywords
- corner
- image
- matching
- gradient
- infrared image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000002146 bilateral effect Effects 0.000 claims abstract description 11
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 239000013598 vector Substances 0.000 claims description 47
- 239000011159 matrix material Substances 0.000 claims description 16
- 238000009825 accumulation Methods 0.000 claims description 5
- 230000014509 gene expression Effects 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 2
- 238000006073 displacement reaction Methods 0.000 description 10
- 238000000605 extraction Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000005316 response function Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20164—Salient point detection; Corner detection
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image processing, and discloses an infrared image homologous registering method based on improved corner principal direction distribution, which comprises the steps of respectively calculating a plurality of corners in two infrared images to be registered, improving a corner principal direction distribution method, distributing principal directions for each corner, establishing a matching relation between descriptors and the principal directions so as to realize rotation invariance of the images, further obtaining local intensity characteristic unchanged descriptors corresponding to each corner, namely PIIFD descriptors, and finally registering one infrared image to be registered and a reference infrared image by adopting nearest neighbor priority BBF combined with a bilateral matching algorithm on the basis of extracting a plurality of PIIFD descriptors. The whole algorithm is simple and efficient in calculation and high in accuracy, and the problems of high registering difficulty and low registering precision caused by low resolution of infrared images and large loss of texture information are effectively solved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an infrared image homologous registration method based on improved corner main direction distribution.
Background
The automatic registration of images is indispensable in the fields of binocular stereoscopic vision technology and image fusion, for example, registration and fusion of a plurality of scanned images of cranium or eyeball are carried out in the medical industry to obtain the clearest diagnostic image, and registration and stitching of remote sensing images shot by an unmanned aerial vehicle are carried out in geological measurement. The David Lowe teaching from Canada proposes a scale invariant feature transform (Scale Invariant Feature Transformation, SIFT) registration algorithm in 1999, supports image registration with obvious scale transform, exploits a new direction of the image registration direction, and greatly improves the application scene of the image registration. Significant efforts have also been made by many students along with David's idea, such as the accelerated robust feature (Speeded Up Robust Features, SURF) algorithm proposed by Herbert Bay, which is also widely cited by those skilled in the art.
Because the parameters of the infrared cameras are inconsistent and the cost is inconsistent, the two infrared images cannot be completely matched in the practical application process. The invention registers the two infrared images to obtain a measurement result with smaller error. The infrared image has unclear texture and relatively large noise, so that the registering difficulty of the infrared image and the infrared image is large, and the precision is low. The registration algorithm based on gray level or regional features has low fault tolerance to the problems, so that the feature-based matching algorithm is reasonably selected. In addition, the method and the device for registering the homologous images finish the registration work of the homologous images by combining the corner extraction, the improved corner main direction distribution method and the bilateral matching method in consideration of the specificity of the image scene of the electrical equipment.
Disclosure of Invention
The invention provides an infrared image homologous registering method based on improved corner principal direction distribution, which solves the problem of low registering precision of the existing infrared homologous images.
The invention can be realized by the following technical scheme:
an infrared image homologous registering method based on improved corner principal direction distribution comprises the following steps:
step one, respectively detecting a plurality of corner points in two infrared images to be registered through a corner point detection algorithm.
I is marked as an image pixel value matrix, G u And G v Representing gradients on the u and v axes, the lateral and longitudinal gray scale gradient matrix expressions of the image are
Let window sliding displacement vector be (x, y), gaussian displacement length be wx×wy, gaussian displacement window be w= (wx, wy), pixel difference value in two windows after displacement is recorded as
Wherein the method comprises the steps of
A. B, C the convolution of the gradient on the u and v axes with the Gaussian displacement window, M is the matrix of the convolution, tr is the trace of the matrix M, and Det is the determinant value of the matrix M.
Let the corner response function r=det-k Tr 2 K is a constant and is generally 0.04 to 0.06.
Judging whether the pixel point is a corner point according to the response value R of the pixel point, wherein the criterion is that
And obtaining candidate corner points in the image according to the criteria, and finally selecting the corner point with the largest local R value as a final result by utilizing a maximum value suppression mode.
And step two, distributing main directions for each angular point by improving an angular point main direction distribution method, and establishing a matching relationship between descriptors and the main directions so as to realize rotation invariance of the image.
Since the gradients of the infrared image for the same pixel point may be the same or opposite, the angle of the gradient vector must be limited to 0,180 °. Establishing a new gradient vector for image I, modifying equation (1) to
The modified vector expression processes the initial gradient with a sign function so that corner gradients opposite to the infrared image gradient are converted into the same direction.
And vector accumulation is carried out on gradients in the neighborhood pixel window of the corner point, but the accumulation can lead opposite gradients to cancel each other, and the final gradient vector of the corner point is 0 vector. To solve this problem, a gradient square vector is introduced, denoted as
Wherein G is s,u And G s,v Representing the square of the improved gradient on the u and v axes, representing further recording the average squared gradient as
Wherein therein is
And->
Representing an improved mean square gradient, ω, on the u and v axes σ Refers to a gaussian convolution kernel with full width at half maximum σ. Because the window of the Gaussian convolution kernel is smaller and more sensitive to noise, the value of sigma only needs to take five pixels.
Finally, the main direction of each corner point is determined according to the formula (7).
Thus determining for each corner point its main direction as phi (u, v).
In the traditional registration algorithm, a neighborhood gradient histogram method is adopted to determine the principal direction of the corner point, the dimension depending on the histogram is compared in the calculation process, and the results are discrete values. The main direction determined by the formula (7) is a continuous value, the main directions of the same angular points in the image are ensured to be the same, and compared with the traditional SIFT, the algorithm has less calculation amount, can calculate the gradient once, and does not need to use histogram statistics.
Step three, using a main direction as a reference direction and using corner points as a central neighborhood gradient vector, so as to obtain local intensity feature invariant descriptors (PIIFD descriptors) corresponding to each corner point;
in order to ensure the rotation invariance of the descriptors, in the process of calculating the characteristic corner neighborhood gradient direction, the main direction of the corner is taken as the reference direction, namely the 0-degree direction. Taking a 4×4 neighborhood of corner points, dividing the corner points into 16 sub-areas, and calculating 8 gradient directions in each sub-area. A gradient direction is represented by a gradient square column, and 8 gradient directions are taken as an example, as shown in fig. 2, 8 gradient vectors are obtained after the gradient statistics of the white region, and all the subareas form a 128-dimensional vector.
However, there are a large number of gradient reversals in the infrared image of the power equipment, and modification of the neighborhood gradient direction is required to ensure local intensity invariance. Firstly, carrying out sectional weighting treatment on the gradient amplitude, giving a weight of 1 to the amplitude of the first 20%, giving a weight of 0.75 to 20-40%, giving a weight of 0.5 to 40-60%, and the like, wherein the part weight of the 20% with the smallest amplitude is 0. In addition, to limit the gradient direction to [0,180 ], the gradient magnitude is separated by a 64-dimensional vector of pi.
After the two-step processing method, the gradient vector of the ith row and the jth column subareas is marked as H ij The gradient vector of the whole neighborhood is recorded as
It should be noted that even if the gradient direction is limited to within 0,180 ° when the principal directions of the corner are calculated, the two principal directions calculated by equation (7) are still reversed for the case where the image to be registered and the reference image are rotated 180 ° with respect to each other, and in order to solve this problem, an auxiliary matrix q=rot (H, 180 °) is introduced, and finally the PIIFD descriptor is obtained as
H i ＝[H i1 H i2 H i3 H i4 ]
Q i ＝[Q i1 Q i2 Q i3 Q i4 ] (10)
Where c is a scale factor for adjusting the magnitude of the PIIFD descriptor.
It is apparent that the Des descriptor maintains the dimension size of the gradient vector, i.e. 4 x 4. To facilitate the subsequent bilateral matching work, it is converted into one-dimensional row vectors in the order of row vectors, and DES' =des/|des|.
By introducing PIIFD descriptors, the situation that the gradients of two infrared images are opposite can be solved, and next step can be performed with matching operation on the descriptor vector of each angular point.
Step four, on the basis of extracting a plurality of PIIFD descriptors, carrying out registration on an infrared image to be registered and a reference infrared image by adopting a nearest neighbor preferential BBF (binary coded field) combined with a bilateral matching algorithm to obtain a plurality of registration point sets;
assume image I to be registered 1 The corresponding descriptor set is F 1 Reference image I 2 The corresponding descriptor set is F 2 For a set of F 1 The i-th element f in (a) 1i Definition f 1i To set F 2 Is the distance of (2)
For f 1i The BBF algorithm is referred to as search set D (f 1i ,F 2 ) F corresponding to the maximum value of (a) 2i As its matching point. In order to ensure the applicability of the algorithm, f is used herein 2i-max And f 2i-smax Respectively represent the set D (f 1i ,F 2 ) Maximum and sub-maximum of (a); only two satisfy the relation f 2i-smax ＜f 2i-max Only when t is less than the maximum value f 2i Corresponding f 2i As its matching point, otherwiseThe matching is regarded as failure, and values between 0.8 and 0.9 are suitably taken according to the empirical t value. The two descriptors (or the corner points corresponding to the descriptors) with successful matching are recorded as a unilateral matching set M (I 1 ,I 2 ). Obviously, one-sided matching occurs in many-to-one cases, i.e. F 1 Multiple elements in (1) may be matched to F 2 Is a single element of the group. To solve the problem, I is 2 For the images to be registered, I 2 For the reference image, a matching set M (I 2 ,I 1 ). Reserved M (I) 1 ,I 2 ) And M (I) 2 ,I 1 ) As the point of success of the initial match.
The bidirectional matching process for completing the feature point descriptor matching is double-sided matching. The uniqueness of the matching points is guaranteed through bilateral matching, and the influence of many to one is removed.
The beneficial technical effects of the invention are as follows:
due to the difference of the sensors, the resolution ratio of the infrared image is low, the texture information is more missing, most of the corner points extracted by the Harris corner point extraction algorithm are distributed on the outline of the infrared image, registration errors caused by the difference of the images are reduced, the distribution of the main directions of the corner points is improved, the dimension of a histogram is not relied on, and the calculated amount is less compared with that of the traditional method. Finally, an auxiliary matrix is introduced to improve PIIFD descriptors, so that the situation that the gradients of two infrared images are opposite is solved, the registration accuracy is improved, and a good matching effect is ensured.
Drawings
FIG. 1 is a schematic general flow diagram of the present invention;
FIG. 2 is a graph of gradient histogram statistics;
fig. 3 is a schematic representation of the results after registration of the present invention.
Detailed Description
The following detailed description of the invention refers to the accompanying drawings and preferred embodiments.
Because the parameters of the infrared cameras are inconsistent and the cost is inconsistent, the two infrared images cannot be completely matched in the practical application process. The invention registers the two infrared images to obtain a measurement result with smaller error. The infrared image has unclear texture and relatively large noise, so that the registering difficulty of the infrared image and the infrared image is large, and the precision is low. The registration algorithm based on gray level or regional features has low fault tolerance to the problems, so that the feature-based matching algorithm is reasonably selected. In addition, the method and the device for registering the homologous images finish the registration work of the homologous images by combining the corner extraction, the improved corner main direction distribution method and the bilateral matching method in consideration of the specificity of the image scene of the electrical equipment.
As shown in fig. 1, the invention provides an infrared image homologous registering method based on improved corner principal direction distribution, which comprises the steps of respectively calculating a plurality of corners in two infrared images to be registered, improving a corner principal direction distribution method, distributing principal directions for each corner, establishing a matching relationship between descriptors and the principal directions so as to realize rotation invariance of the images, further obtaining local intensity characteristic unchanged descriptors corresponding to each corner, namely PIIFD descriptors, and finally registering one infrared image to be registered and a reference infrared image by adopting nearest neighbor priority BBF combined with a bilateral matching algorithm on the basis of extracting a plurality of PIIFD descriptors. The specific embodiment comprises the following steps:
step one, image preprocessing, and determining whether filtering operation is needed according to the image quality. Firstly, the gray values of an image are redistributed, the gray image is subjected to linear transformation, and then the gray values are stretched to be between 0 and 255, and the specific formula is as follows:
and step two, respectively calculating a plurality of angular points in the two infrared images to be registered, and taking the diagonal line of the angular point as the main direction of the angular point.
Corner points are points of maximum curvature on the edge curve of an image or the brightness of the image changes drastically. As the detail textures of the infrared images are seriously lost, most remarkable angular points are distributed on the outline of the object, and the method and the device detect a plurality of angular points in two infrared images to be registered respectively through an angular point detection algorithm.
I is marked as an image pixel value matrix, G u And G v Representing gradients on the u and v axes, the lateral and longitudinal gray scale gradient matrix expressions of the image are
Let window sliding displacement vector be (x, y), gaussian displacement length be wx×wy, gaussian displacement window be w= (wx, wy), pixel difference value in two windows after displacement is recorded as
Wherein the method comprises the steps of
A. B, C the convolution of the gradient on the u and v axes with the Gaussian displacement window, M is the matrix of the convolution, tr is the trace of the matrix M, and Det is the determinant value of the matrix M.
Let the corner response function r=det-k Tr 2 K is a constant and is generally 0.04 to 0.06.
Judging whether the pixel point is a corner point according to the response value R of the pixel point, wherein the criterion is that
And obtaining candidate corner points in the image according to the criteria, and finally selecting the corner point with the largest local R value as a final result by utilizing a maximum value suppression mode.
And thirdly, distributing main directions for each angular point by improving an angular point main direction distribution method, and establishing a matching relationship between descriptors and the main directions so as to realize rotation invariance of the image.
Since the gradients of the infrared image for the same pixel point may be the same or opposite, the angle of the gradient vector must be limited to 0,180 °. Establishing a new gradient vector for image I, modifying equation (1) to
The modified vector expression processes the initial gradient with a sign function so that corner gradients opposite to the infrared image gradient are converted into the same direction.
And vector accumulation is carried out on gradients in the neighborhood pixel window of the corner point, but the accumulation can lead opposite gradients to cancel each other, and the final gradient vector of the corner point is 0 vector. To solve this problem, a gradient square vector is introduced, denoted as
Wherein G is s,u And G s,v Representing the square of the improved gradient on the u and v axes, representing further recording the average squared gradient as
Wherein therein is
And->
Representing an improved mean square gradient, ω, on the u and v axes σ Refers to a gaussian convolution kernel with full width at half maximum σ. Because the window of the Gaussian convolution kernel is smaller and more sensitive to noise, the value of sigma only needs to take five pixels.
Finally, the main direction of each corner point is determined according to the formula (7).
Thus determining for each corner point its main direction as phi (u, v).
In the traditional registration algorithm, a neighborhood gradient histogram method is adopted to determine the principal direction of the corner point, the dimension depending on the histogram is compared in the calculation process, and the results are discrete values. The main direction determined by the formula (7) is a continuous value, the main directions of the same angular points in the image are ensured to be the same, and compared with the traditional SIFT, the algorithm has less calculation amount, can calculate the gradient once, and does not need to use histogram statistics.
Step four, taking the main direction as a reference direction, taking the corner points as the neighborhood gradient vectors of the center, and further obtaining local intensity feature invariant descriptors corresponding to each corner point, namely PIIFD descriptors;
in order to ensure the rotation invariance of the descriptors, in the process of calculating the characteristic corner neighborhood gradient direction, the main direction of the corner is taken as the reference direction, namely the 0-degree direction. Taking a 4×4 neighborhood of corner points, dividing the corner points into 16 sub-areas, and calculating 8 gradient directions in each sub-area. A gradient direction is represented by a gradient square column, and 8 gradient directions are taken as an example, as shown in fig. 2, 8 gradient vectors are obtained after the gradient statistics of the white region, and all the subareas form a 128-dimensional vector.
However, there are a large number of gradient reversals in the infrared image of the power equipment, and modification of the neighborhood gradient direction is required to ensure local intensity invariance. Firstly, carrying out sectional weighting treatment on the gradient amplitude, giving a weight of 1 to the amplitude of the first 20%, giving a weight of 0.75 to 20-40%, giving a weight of 0.5 to 40-60%, and the like, wherein the part weight of the 20% with the smallest amplitude is 0. In addition, to limit the gradient direction to [0,180 ], the gradient magnitude is separated by a 64-dimensional vector of pi.
After the two-step processing method, the gradient vector of the ith row and the jth column subareas is marked as H ij The gradient vector of the whole neighborhood is recorded as
It should be noted that even if the gradient direction is limited to within 0,180 ° when the principal directions of the corner are calculated, the two principal directions calculated by equation (7) are still reversed for the case where the image to be registered and the reference image are rotated 180 ° with respect to each other, and in order to solve this problem, an auxiliary matrix q=rot (H, 180 °) is introduced, and finally the PIIFD descriptor is obtained as
H i ＝[H i1 H i2 H i3 H i4 ]
Q i ＝[Q i1 Q i2 Q i3 Q i4 ] (10)
Where c is a scale factor for adjusting the magnitude of the PIIFD descriptor.
It is apparent that the Des descriptor maintains the dimension size of the gradient vector, i.e. 4 x 4. To facilitate the subsequent bilateral matching work, it is converted into one-dimensional row vectors in the order of row vectors, and DES' =des/|des|.
By introducing PIIFD descriptors, the situation that the gradients of two infrared images are opposite can be solved, and next step can be performed with matching operation on the descriptor vector of each angular point.
Fifthly, on the basis of extracting a plurality of PIIFD descriptors, carrying out registration on an infrared image to be registered and a reference infrared image by adopting a nearest neighbor preferential BBF (binary coded field) combined with a bilateral matching algorithm to obtain a plurality of registration point sets;
assume image I to be registered 1 The corresponding descriptor set is F 1 Reference image I 2 The corresponding descriptor set is F 2 For a set of F 1 The i-th element f in (a) 1i Definition f 1i To set F 2 Is the distance of (2)
For f 1i The BBF algorithm is referred to as search set D (f 1i ,F 2 ) F corresponding to the maximum value of (a) 2i As its matching point. In order to ensure the applicability of the algorithm, f is used herein 2i-max And f 2i-smax Respectively represent the set D (f 1i ,F 2 ) Maximum and sub-maximum of (a); only two satisfy the relation f 2i-smax ＜f 2i-max Only when t is less than the maximum value f 2i Corresponding f 2i If the matching point is not considered as matching failure, the value between 0.8 and 0.9 is properly taken according to the empirical value of t. The two descriptors (or the corner points corresponding to the descriptors) with successful matching are recorded as a unilateral matching set M (I 1 ,I 2 ). Obviously, one-sided matching occurs in many-to-one cases, i.e. F 1 Multiple elements in (1) may be matched to F 2 Is a single element of the group. To solve the problem, I is 2 For the images to be registered, I 2 For the reference image, a matching set M (I 2 ,I 1 ). Reserved M (I) 1 ,I 2 ) And M (I) 2 ,I 1 ) As the point of success of the initial match.
In order to verify the feasibility of the method, registration experiments are carried out on the self-built infrared image database by using different algorithms, and the experimental contents comprise:
experiment platform:
MATLAB 2017b，
evaluation index: root mean square error Root Mean Squard Error (RMSE)
Wherein (x) i ,y i ) Registration point coordinates, (x 'obtained by image registration' i ,y’ i ) And (3) the theoretical registration point coordinates after the registration points pass through the theoretical perspective transformation matrix. The index can objectively reflect that the matching precision of the registration algorithm is high, the smaller the value is,the higher the registration accuracy.
The present invention uses three different conventional algorithms, SIFT, SURF, harris-SIFT, to compare with the present invention, and to compare an image in an infrared database of a self-built electrical device, as shown in fig. 3. The indexes are shown in the following table.
The comparison of the results obtained by the method and the results of the traditional three methods shows that the root mean square error of the infrared image registration method is minimum. Most notably, the method of the invention obtains more than twice the number of correct registration point pairs in the registration process than other methods, and the distribution dispersion degree of the registration points on the image is also most comprehensive. Experimental results show that the method provided by the invention has higher practicability and more accurate results than the traditional method.
TABLE 1 results obtained with the present invention and results of the conventional three types of methods
| Method | RMSE | Correct registration point duty cycle |
| SIFT | 27.959 | 0.833 |
| SURF | 25.091 | 0.500 |
| Harris-SIFT | 8.154 | 0.643 |
| The method of the invention | 7.437 | 0.922 |
While particular embodiments of the present invention have been described above, it will be appreciated by those skilled in the art that these are merely illustrative, and that many changes and modifications may be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims.
Claims (4)
1. An infrared image homologous registering method based on improved corner principal direction distribution is characterized by comprising the following steps:
step one, respectively detecting a plurality of corner points in two infrared images to be registered through a corner point detection algorithm;
step two, distributing main directions for each angular point by improving an angular point main direction distribution method, and establishing a matching relationship between descriptors and the main directions so as to realize rotation invariance of the image;
step three, using a main direction as a reference direction and using corner points as a central neighborhood gradient vector, so as to obtain local intensity feature invariant descriptors (PIIFD descriptors) corresponding to each corner point;
step four, on the basis of extracting a plurality of PIIFD descriptors, carrying out registration on an infrared image to be registered and a reference infrared image by adopting a nearest neighbor preferential BBF (binary coded field) combined with a bilateral matching algorithm to obtain a plurality of registration point sets;
the second step is characterized in that:
if vector accumulation is directly performed on gradients in the neighborhood pixel window of the corner point, opposite gradients are mutually counteracted, and the final gradient vector of the corner point is 0 vector, and a gradient square vector is introduced, wherein the gradient square vector is as follows:
further recording the average square gradient as
Wherein w is σ A Gaussian convolution kernel with the full width of half maximum sigma is indicated, and the window of the Gaussian convolution kernel is smaller and is more sensitive to noise, so that the value of sigma is five pixels,
finally, the main direction of each corner point is determined, and the main direction of each corner point (u, v) is determined to be phi (u, v).
2. The infrared image homology registration method based on the improved principal direction distribution of corner points as claimed in claim 1, wherein:
aiming at the characteristics of low resolution and large texture information loss of an infrared image, the transverse and longitudinal gray gradient matrixes of the image are improved, and a new gradient vector is established for an image pixel matrix I as follows:
the improved vector expression processes the gradient in the y direction with a sign function so that the corner gradients opposite to the gradient of the reference infrared image to be registered are converted into the same direction.
3. The infrared image homology registration method based on the improved principal direction distribution of corner points as claimed in claim 1, wherein:
aiming at the condition that an image to be registered and a reference image are mutually rotated by 180 degrees, the PIIFD descriptor calculation method is improved, an auxiliary matrix Q=rot (H, 180 degrees) is introduced, and the PIIFD descriptor is finally obtained as
H i ＝[H i1 H i2 H i3 H i4 ]
Q i ＝[Q i1 Q i2 Q i3 Q i4 ] (10)
Wherein c is a scale factor for adjusting the magnitude of the PIIFD descriptor;
it is apparent that the Des descriptor maintains the dimension size of the gradient vector, i.e. 4 x 4; to facilitate the subsequent bilateral matching work, it is converted into one-dimensional row vectors in the order of row vectors, and DES' =des/|des|.
4. The infrared image homology registration method based on the improved principal direction distribution of corner points as claimed in claim 1, wherein:
on the basis of extracting a plurality of PIIFD descriptors, an infrared image to be registered and a reference infrared image are registered by adopting nearest neighbor priority and double-side matching algorithm, so that accuracy of a matching result is enhanced, and an image I to be registered is registered 1 The corresponding descriptor set is F 1 Reference image I 2 The corresponding descriptor set is F 2 For f 1i ∈F 1 F is then 1i To set F 2 Is the distance of (2)
For f 1i The BBF algorithm is referred to as search set D (f 1i ，F 2 ) F corresponding to the maximum value of (a) 2i As its matching point, use f 2i’ And f 2i” Respectively represent the set D (f 1i ，F 2 ) Maximum and sub-maximum of (a); only two satisfy the relation f 2i″ ＜f 2i′ Only when < i, the maximum value f is selected 2i′ Corresponding f 2i As the matching point, if not, the matching is considered as failure, and the value between 0.8 and 0.9 is properly taken according to the empirical t value;
the two descriptors successfully matched are recorded as a unilateral matching set M (I 1 ，I 2 ) Obviously, one-sided matching occurs in many-to-one cases, i.e. F 1 Multiple elements in (1) may be matched to F 2 In the same element, as I 2 For the images to be registered, I 1 For the reference image, a matching set M (I 2 ，I 1 ) Reserving M (I) 1 ，I 2 ) And M (I) 2 ，I 1 ) The same elements in the process are used as points of initial matching success, so that the uniqueness of the matching points is ensured, and the influence of many to one is removed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010169423.1A CN111311657B (en) | 2020-03-12 | 2020-03-12 | Infrared image homologous registration method based on improved corner principal direction distribution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010169423.1A CN111311657B (en) | 2020-03-12 | 2020-03-12 | Infrared image homologous registration method based on improved corner principal direction distribution |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111311657A CN111311657A (en) | 2020-06-19 |
| CN111311657B true CN111311657B (en) | 2023-04-25 |
Family
ID=71145514
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010169423.1A Active CN111311657B (en) | 2020-03-12 | 2020-03-12 | Infrared image homologous registration method based on improved corner principal direction distribution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111311657B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112070813B (en) * | 2020-08-21 | 2024-08-06 | 国网山东省电力公司青岛供电公司 | Feature matching method based on link feature consistency |
| CN113095385B (en) * | 2021-03-31 | 2023-04-18 | 安徽工业大学 | Multimode image matching method based on global and local feature description |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012058902A1 (en) * | 2010-11-02 | 2012-05-10 | 中兴通讯股份有限公司 | Method and apparatus for combining panoramic image |
| WO2017049994A1 (en) * | 2015-09-25 | 2017-03-30 | 深圳大学 | Hyperspectral image corner detection method and system |
| WO2018076137A1 (en) * | 2016-10-24 | 2018-05-03 | 深圳大学 | Method and device for obtaining hyper-spectral image feature descriptor |
| CN109285110A (en) * | 2018-09-13 | 2019-01-29 | 武汉大学 | The infrared visible light image registration method and system with transformation are matched based on robust |
| CN110223330A (en) * | 2019-06-12 | 2019-09-10 | 国网河北省电力有限公司沧州供电分公司 | A kind of method for registering and system of visible light and infrared image |
-
2020
- 2020-03-12 CN CN202010169423.1A patent/CN111311657B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012058902A1 (en) * | 2010-11-02 | 2012-05-10 | 中兴通讯股份有限公司 | Method and apparatus for combining panoramic image |
| WO2017049994A1 (en) * | 2015-09-25 | 2017-03-30 | 深圳大学 | Hyperspectral image corner detection method and system |
| WO2018076137A1 (en) * | 2016-10-24 | 2018-05-03 | 深圳大学 | Method and device for obtaining hyper-spectral image feature descriptor |
| CN109285110A (en) * | 2018-09-13 | 2019-01-29 | 武汉大学 | The infrared visible light image registration method and system with transformation are matched based on robust |
| CN110223330A (en) * | 2019-06-12 | 2019-09-10 | 国网河北省电力有限公司沧州供电分公司 | A kind of method for registering and system of visible light and infrared image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111311657A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ashraf et al. | Learning patch correspondences for improved viewpoint invariant face recognition | |
| CN105427298B (en) | Remote sensing image registration method based on anisotropic gradient metric space | |
| CN111223133B (en) | Registration method of heterogeneous images | |
| CN110223330B (en) | Registration method and system for visible light and infrared images | |
| CN111340701B (en) | Circuit board image splicing method for screening matching points based on clustering method | |
| CN105809626A (en) | Self-adaption light compensation video image splicing method | |
| CN110490913B (en) | Image matching method based on feature description operator of corner and single line segment grouping | |
| CN112085772B (en) | Remote sensing image registration method and device | |
| CN107316031A (en) | The image characteristic extracting method recognized again for pedestrian | |
| CN107066969A (en) | A kind of face identification method | |
| Fan et al. | New point matching algorithm using sparse representation of image patch feature for SAR image registration | |
| CN105787943B (en) | SAR image registration method based on multi-scale image block feature and rarefaction representation | |
| Wang et al. | A segmentation based robust deep learning framework for multimodal retinal image registration | |
| CN105631872B (en) | Remote sensing image registration method based on multi-characteristic points | |
| CN107330928B (en) | Based on the Image Feature Matching method for improving Shape context | |
| Fan et al. | SAR image registration using multiscale image patch features with sparse representation | |
| CN109472770B (en) | Method for quickly matching image characteristic points in printed circuit board detection | |
| CN103136525A (en) | High-precision positioning method for special-shaped extended target by utilizing generalized Hough transformation | |
| CN111311657B (en) | Infrared image homologous registration method based on improved corner principal direction distribution | |
| CN112614167A (en) | Rock slice image alignment method combining single-polarization and orthogonal-polarization images | |
| CN108805915A (en) | A kind of close-range image provincial characteristics matching process of anti-visual angle change | |
| CN107862319A (en) | A kind of heterologous high score optical image matching error elimination method based on neighborhood ballot | |
| CN113095385A (en) | Multimode image matching method based on global and local feature description | |
| CN116977316A (en) | Full-field detection and quantitative evaluation method for damage defects of complex-shape component | |
| CN113792788B (en) | Infrared and visible light image matching method based on multi-feature similarity fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |