CN103473756B - Autoregistration merges the method for Aerial Images and close shot image - Google Patents

Abstract

The present invention proposes a kind of method that autoregistration merges Aerial Images and close shot image, comprise: to the Aerial Images collected with arrange that identification plate close shot image is smoothing, sharpening, abatement Aerial Images and the noise that identification plate close shot image is set, strengthen Aerial Images and the edge that identification plate close shot image is set, then carry out the Aerial Images after coordinate conversion process acquisition image enhaucament and identification plate close shot image vertical view is set; Indicating board size determined mathematics conversion proportion relation according to Aerial Images with arranging in identification plate close shot image vertical view, being transformed in the coordinate system of Aerial Images by arranging identification plate close shot image, complete uniform coordinate conversion; In Aerial Images, the described correspondence position arranging identification plate close shot image in search location, determines the registration transformation relation between two width images according to cross-correlation similarity measure; Image pixel rgb value weighted method is adopted to realize Aerial Images and the fusion of identification plate close shot image overlapping region is set.

Description

Method for automatic registration and fusion of aerial images and close-range images

technical field

The invention relates to the field of traffic accident scene information processing, in particular to a method for automatically registering and fusing aerial images and close-range images.

Background technique

Aerial images play an important role in traffic accident scene surveys. They can efficiently restore various elements of the accident scene. However, in order to obtain global information, aerial images are often shot at a high location, so the ability to distinguish small traces on the ground is insufficient. The close-range image provides detailed information of each element with a higher resolution in traffic accident scene survey, but it cannot reflect its position in the whole scene. In order to better analyze accidents, it is the future development trend to use registration and fusion technology to synthesize aerial images and close-range image information. Combining various traffic accident scene survey images, making full use of the characteristics of different survey images, and combining them on one image At the same time, it expresses various information from the accident scene, so that the location of key elements and details of key elements in the accident scene can be reflected through the images after registration and fusion, so as to provide more intuitive information on the scene of traffic accidents.

At present, it is possible to manually search and locate feature objects in aerial images and close-range images, and then use feature objects to realize artificial registration and fusion of aerial images and close-range images. However, when there are many close-range images, the manual method is time-consuming and laborious, and the accuracy is not high. Therefore, a marker and an implementation method for automatic registration and fusion of aerial images and close-range images are very necessary.

Contents of the invention

The present invention aims to at least solve the technical problems existing in the prior art, and particularly innovatively proposes a method for automatically registering and fusing aerial images and close-range images.

In order to achieve the above object of the present invention, the present invention provides a method for automatic registration and fusion of aerial images and close-range images, the key of which is to include the following steps:

Step 1: Smooth and sharpen the collected aerial image and the close-range image of the signboard, reduce the noise of the aerial image and the close-up image of the signboard, enhance the edge of the aerial image and the close-up image of the signboard, and then perform coordinate conversion processing Obtain the image-enhanced aerial image and set the top view of the close-range image of the sign board;

Step 2, according to the mathematical conversion ratio relationship determined by the aerial image and the sign plate size in the top view of the close-range image of the set sign plate, the close-range image of the set sign plate is converted into the coordinate system of the aerial image, and the unified coordinate transformation is completed;

Step 3, searching and locating the corresponding position of the close-range image of the set sign board in the aerial image, and determining the registration transformation relationship between the two images according to the cross-correlation similarity measure;

Step 4, using the image pixel RGB value weighting method to realize the fusion of the overlapping area of the aerial image and the close-range image of the set sign board.

The beneficial effect of the above technical solution is that the image after fusion and registration has richer and more comprehensive scene information, can better reflect the overall and key local scene conditions of the accident scene, and is convenient for the analysis and utilization of traffic accident scene information.

The method for automatic registration and fusion of aerial images and close-range images, preferably, said step 3 includes:

Step 3-1: According to the fact that the color of the signboard in the close-range image of the set signboard has rotation invariance and scale invariance, the structural color of the signboard is used as the index feature of the close-range image of the set signboard to search for it in the aerial image Positioning, to obtain the position area of the close-range image of the set sign board in the aerial image;

Step 3-2, using the Harris corner detection algorithm to use the center of the circle where the fan-shaped structure of the signboard is located as the registration control point for setting the close-range image of the signboard and the aerial image;

Step 3-3, according to the aerial image and the close-range image of the sign board are all RGB color images, choose to use the image RGB value cross-correlation registration to obtain the best registration position.

In the method for automatic registration and fusion of aerial images and close-range images, preferably, the registration control point algorithm is:

E(u,v)＝∑ _x,y w(x,y)[I(x+u,y+v)-I(x,y)] ²

Among them, w(x,y) is the window function, [I(x+u,y+v)-I(x,y)] ² is the gradient value of the image, and w(x,y) is a rectangular window or a Gaussian window , for each displacement (u, v), the bilinear approximation in the above formula is expressed as,

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&cong;</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ]</span>\mathrm{<span\; class="notranslate">\; m</span>}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$

M is a 2*2 matrix, the specific formula is,

$\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; A</span>}& \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}& \mathrm{<span\; class="notranslate">\; B</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; e</span>}}^{\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; \&CircleTimes;</span>\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; Ix</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; IxI</span>}\\ \mathrm{<span\; class="notranslate">\; IxI</span>}& {\mathrm{<span\; class="notranslate">\; Iy</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]<span\; class="notranslate">\; ,</span>$

Ix and Iy are the gradient values in the x and y directions of the image respectively; E is approximated as a local cross-correlation function, and M describes the shape at this point. Let λ ₁ and λ ₂ be the two eigenvalues of the matrix M, then λ ₁ , λ ₂ can represent the curvature of the local autocorrelation function. Due to isotropy, M maintains rotation invariance, where A, B, and C represent the element values of matrix M.

In the method for automatic registration and fusion of aerial images and close-range images, preferably, the cross-correlation registration algorithm is:

Select a temporary window W(l,m) on the aerial image, and find the most similar window corresponding to it on the close-range image S(i,j) of the sign board , which is the rotation transformation window, measured by cross-correlation similarity, where l and m are positive integers,

$\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; j</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}$

where W(i,j) is a window image of size M×M, is a temporary window in the aerial image, and the window image W(i,j) is automatically searched in a scanning manner in the close-range image S of the search setting sign board to find the maximum point of the correlation function value R _N (i,j) in the process position (i ^* , j ^* ), and think that this is the best registration position.

The method for automatic registration and fusion of aerial images and close-range images, preferably, said step 4 includes:

Step 4-1, the RGB value R ₁ (i,j), G ₁ (i,j), B ₁ (i,j) of the aerial image and the RGB value R ₂ (i,j) of the close-range image of the sign board are set , G ₂ (i,j), and B ₂ (i,j) are each multiplied by a weight coefficient, and fused to form a new image F(i,j),

F(i,j)=aI ₁ (i,j)+(1-a)I ₂ (i,j),

Among them, a is the weight factor, and 0≤a≤1, I ₁ is the aerial image, and I ₂ is the close-up image of the sign board.

In the method for automatically registering and fusing aerial images and close-range images, preferably, the sign board is a right-angle fan-shaped structure.

In the method for automatic registration and fusion of aerial images and close-range images, preferably, the colors of the sign boards are different.

In summary, owing to adopting above-mentioned technical scheme, the beneficial effect of the present invention is:

The image after fusion and registration has richer and more comprehensive scene information, which can better reflect the overall and key local scene conditions of the accident scene, and facilitate the analysis and utilization of traffic accident scene information;

Using registration and fusion technology to synthesize aerial images and close-range image information of sign boards, combine multiple traffic accident scene survey images, make full use of the characteristics of different survey images, and express multiple aspects of information from the accident scene on one image at the same time;

The location of key elements and details of key elements in the accident scene can be reflected through the images after registration and fusion, so as to provide traffic accident scene information more intuitively.

Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and comprehensible from the description of the embodiments in conjunction with the following drawings, wherein:

Fig. 1 is a flow chart of the method for automatically registering and fusing aerial images and close-range images of the present invention;

Fig. 2 is a schematic diagram of the registration and fusion of the aerial image and the close-range image of the sign board in the method for automatic registration and fusion of the aerial image and the close-range image of the present invention;

Fig. 3 is a flow chart of automatic registration of aerial images and close-range images of sign boards in the method for automatic registration and fusion of aerial images and close-range images of the present invention;

Fig. 4 is a sign board used for automatic registration and fusion of aerial images and close-range images by the method of automatic registration and fusion of aerial images and close-range images of the present invention.

detailed description

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

In describing the present invention, it should be understood that the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", The orientation or positional relationship indicated by "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than Nothing indicating or implying that a referenced device or element must have a particular orientation, be constructed, and operate in a particular orientation should therefore not be construed as limiting the invention.

In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be mechanical connection or electrical connection, or two The internal communication of each element may be directly connected or indirectly connected through an intermediary. Those skilled in the art can understand the specific meanings of the above terms according to specific situations.

The technical scheme of the present invention redesigns the pattern of the sign board placed at the accident site, proposes to use the color of the sign board as the index feature for setting the close-range image of the sign board, and uses the image corner detection algorithm to determine the corresponding position of the close-range image of the set sign board in the aerial image The registration transformation relationship between the close-range image of the sign board and the aerial image is determined by the cross-correlation similarity measure. Finally, the fusion of the close-range image of the sign board and the aerial image is realized by using the image pixel RGB value weighting method.

As shown in Figure 1, the present invention provides a method for automatic registration and fusion of aerial images and close-range images, the key of which is to include the following steps:

Step 1: Smooth and sharpen the collected aerial image and the close-range image of the signboard, reduce the noise of the aerial image and the close-up image of the signboard, enhance the edge of the aerial image and the close-up image of the signboard, and then perform coordinate conversion processing Obtain the image-enhanced aerial image and set the top view of the close-range image of the sign board;

Step 2, according to the mathematical conversion ratio relationship determined by the aerial image and the sign plate size in the top view of the close-range image of the set sign plate, the close-range image of the set sign plate is converted into the coordinate system of the aerial image, and the unified coordinate transformation is completed;

Step 3, searching and locating the corresponding position of the close-range image of the set sign board in the aerial image, and determining the registration transformation relationship between the two images according to the cross-correlation similarity measure;

Step 4, using the image pixel RGB value weighting method to realize the fusion of the overlapping area of the aerial image and the close-range image of the set sign board.

The beneficial effect of the above technical solution is that the image after fusion and registration has richer and more comprehensive scene information, can better reflect the overall and key local scene conditions of the accident scene, and is convenient for the analysis and utilization of traffic accident scene information.

The method for automatic registration and fusion of aerial images and close-range images, preferably, said step 3 includes:

Step 3-1: According to the fact that the color of the signboard in the close-range image of the set signboard has rotation invariance and scale invariance, the structural color of the signboard is used as the index feature of the close-range image of the set signboard to search for it in the aerial image Positioning, to obtain the position area of the close-range image of the set sign board in the aerial image;

Step 3-2, using the Harris corner detection algorithm to use the center of the circle where the fan-shaped structure of the signboard is located as the registration control point for setting the close-range image of the signboard and the aerial image;

Step 3-3, according to the aerial image and the close-range image of the sign board are all RGB color images, choose to use the image RGB value cross-correlation registration to obtain the best registration position.

More details and feature descriptions of the present invention will be further shown below:

1. Image preprocessing: smooth and sharpen the collected aerial image and the close-range image of the signboard, reduce the noise of the aerial image and the close-up image of the signboard, enhance the edge of the aerial image and the close-up image of the signboard, and then perform Coordinate transformation processing acquires the enhanced aerial image and sets the top view of the close-range image of the sign board. Utilize the homogeneous coordinate transformation to transform the aerial image and the close-range image of the sign board to the top-view image, and the homogeneous coordinate transformation equation adopted is:

$\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\frac{\left(\begin{array}{cc}\mathrm{<span\; class="notranslate">\; a</span>}& \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}& \mathrm{<span\; class="notranslate">\; d</span>}\end{array}\right)\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right)<span\; class="notranslate">\; +</span>\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\end{array}\right)}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\left(\begin{array}{cc}\mathrm{<span\; class="notranslate">\; u</span>}& \mathrm{<span\; class="notranslate">\; v</span>}\end{array}\right)\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right)}<span\; class="notranslate">\; ,</span>$

Among them, a, b, c, d, e, f, u, v are coordinate conversion parameters, x, y are coordinate vectors, and x′, y′ are coordinate vectors to be solved;

2. Unified coordinates: According to the mathematical conversion ratio relationship determined by the aerial image and the signboard size in the overhead view of the close-up image of the set signboard, the close-up image of the set signboard is converted into the coordinate system of the aerial image to complete the unified coordinate transformation.

3. Image registration: Search and locate the corresponding position of the sign board in the aerial image of the close-range image of the sign board, and determine the registration transformation relationship between the two images according to the cross-correlation similarity measure.

3.1 According to the rotation invariance and scale invariance of the color feature of the sign board, the fan-shaped structure color of the sign board is used as the index feature of the close-range image of the sign board to search and locate it in the aerial image, and the close-up image of the sign board is obtained in the aerial image the location area of

3.2 According to the fact that the registration control points should be clear and stable, and they are prominent feature points that are easily identifiable on the aerial image and the close-range image of the sign board, and the target is small, by using the Harris corner detection algorithm, the circle where the fan-shaped structure of the sign board is located The center of the circle is used as the registration control point for setting the close-range image of the sign board and the aerial image. The mathematical formula of its algorithm is:

E(u,v)＝∑ _x,y w(x,y)[I(x+u,y+v)-I(x,y)] ² formula (1)

Among them, w(x,y) is the window function, and [I(x+u,y+v)-I(x,y)] ² is the gradient value of the image. w(x, y) is a rectangular window or a Gaussian window. For each small displacement (u, v), u and v represent the displacement respectively. Equation (1) can be expressed as a bilinear approximation

$\mathrm{E.}(u,v)\&cong;[u,v]m\left[\begin{array}{c}u\\ v\end{array}\right]$ Formula (2)

M is a 2*2 matrix:

$\left[\begin{array}{cc}A& C\\ C& B\end{array}\right]=e^{\frac{x^2+{\mathrm{the\; y}}^2}{2{\mathrm{\&sigma;}}^2}}\&CircleTimes;\left[\begin{array}{cc}{\mathrm{Ix}}^2& \mathrm{IxI}\\ \mathrm{IxI}& {\mathrm{Iy}}^2\end{array}\right]$ Formula (3)

Ix and Iy are the gradient values in the x and y directions of the image respectively; E can be approximated as a local cross-correlation function, and M describes the shape at this point. Let λ ₁ and λ ₂ be the two eigenvalues of the matrix M, then λ ₁ and λ ₂ can represent the curvature of the local autocorrelation function. Due to isotropy, M maintains rotation invariance, where A, B, and C represent the element values of matrix M.

3.3 According to the fact that the aerial image and the close-range image of the set sign board are all RGB color images, choose to use the image RGB value cross-correlation registration. Select a temporary window W(l,m) on the aerial image, and find the most similar window corresponding to it on the close-range image S(i,j) of the sign board (rotation transformation window), using the cross-correlation similarity measure as:

$R(i,j)=\frac{\underset{i=1}{\overset{m}{\mathrm{\&Sigma;}}}\underset{m=1}{\overset{m}{\mathrm{\&Sigma;}}}W(l,m)S_m^{i,j}(l,m)}{\underset{i=1}{\overset{m}{\mathrm{\&Sigma;}}}\underset{m=1}{\overset{m}{\mathrm{\&Sigma;}}}{\left[S^{i,j}(l,m)\right]}^2}(i^{*},j^{*}),1\&le;i,j\&le;L-m+1$ Formula (4)

where W(i,j) is a window image of size M×M, is a temporary window in the aerial image. The window image W(i,j) is automatically searched in a scanning manner in the close-range image S of the search setting sign board, and the position (i ^* ,j ^* ) of the maximum point in the correlation function value R _N (i,j) is found in the process , and think that this is the best registration position, where l and m are positive integers, and the subscript N is a positive integer.

4. Image fusion: The image pixel RGB value weighting method is used to realize the fusion of the overlapping area of the aerial image and the close-range image of the sign board. The method is to combine the RGB values R ₁ (i,j), G ₁ (i,j), B ₁ (i,j) of the aerial image I ₁ and the RGB values R ₂ (i,j) of the close-range image I ₂ of the sign board ), G ₂ (i,j), and B ₂ (i,j) are each multiplied by a weight coefficient, and fused to form a new image F(i,j).

F(i,j)＝aI ₁ (i,j)+(1-a)I ₂ (i,j)Formula (5)

Where: a is a weight factor, and 0≤a≤1, and the size of a can be adjusted as required.

As shown in Figure 2, to use the specific embodiment of the above-mentioned method, the close-range image 1 of the signboard is set as the fusion process using the blue signboard, and the close-up image 2 of the signboard is set as the fusion process using the red signboard. In the specific implementation process , is not limited to the fusion method shown in FIG. 2 .

As shown in Figure 3, the method for registration is,

S1, firstly preprocess the close-range image I ₂ of the sign board; determine the initial transformation value of the registration control point R;

S2, rotate and transform the initial transformation value R, and the transformed image is I ^* =R(I ₂ );

S3, analyzing the image window determined by the registration control point of the close-range image of the signboard and the aerial image _I1 , and obtaining a similarity measure evaluation;

S4. Determine whether the transformation value R is optimal. If it is optimal, the execution ends. If it is not optimal, continue to update the transformation value R until the optimal value R is reached.

As shown in Figure 4, 1 is a blue right-angle fan structure, 2 is a green right-angle fan-shaped structure, 3 is a red right-angle fan-shaped structure, and 4 is a yellow right-angle fan-shaped structure, but Figure 4 is just an example, and the actual operation is not limited to the above The type of color, the right-angle fan-shaped structure is the preferred fan-shaped structure. Figure 4 is a set of sign boards that fuse 4 close-range images of set sign boards into aerial images. The colors of the signboards are different, and not only one set of four signboards, but also N sets of signboards can be set according to the scene conditions of the accident scene.

The beneficial effects of the present invention are:

The image after fusion and registration has richer and more comprehensive scene information, which can better reflect the overall and key local scene conditions of the accident scene, and facilitate the analysis and utilization of traffic accident scene information;

Using registration and fusion technology to synthesize aerial images and close-range image information of sign boards, combine multiple traffic accident scene survey images, make full use of the characteristics of different survey images, and express multiple aspects of information from the accident scene on one image at the same time;

The location of key elements and details of key elements in the accident scene can be reflected through the images after registration and fusion, so as to provide traffic accident scene information more intuitively.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the invention is defined by the claims and their equivalents.

Claims (7)

1. A method for automatic registration and fusion of aerial images and close-range images, characterized in that, comprising the steps: Step 1: Smooth and sharpen the collected aerial image and the close-range image of the signboard, reduce the noise of the aerial image and the close-up image of the signboard, enhance the edge of the aerial image and the close-up image of the signboard, and then perform coordinate conversion processing Obtain the image-enhanced aerial image and set the top view of the close-range image of the sign board; Step 2, according to the mathematical conversion ratio relationship determined by the aerial image and the sign plate size in the top view of the close-range image of the set sign plate, the close-range image of the set sign plate is converted into the coordinate system of the aerial image, and the unified coordinate transformation is completed; Step 3, searching and locating the corresponding position of the close-range image of the set sign board in the aerial image, and determining the registration transformation relationship between the two images according to the cross-correlation similarity measure; Step 4, using the image pixel RGB value weighting method to realize the fusion of the overlapping area of the aerial image and the close-range image of the set sign board. 2. The method for automatic registration and fusion of aerial images and close-range images according to claim 1, wherein said step 3 comprises: Step 3-1: According to the fact that the color of the signboard in the close-range image of the set signboard has rotation invariance and scale invariance, the structural color of the signboard is used as the index feature of the close-range image of the set signboard to search and locate the close-range image, and obtain The location area of the close-range image in the aerial image; Step 3-2, using the Harris corner detection algorithm to use the center of the circle where the fan-shaped structure of the signboard is located as the registration control point for setting the close-range image of the signboard and the aerial image; Step 3-3, according to the aerial image and the close-range image of the sign board are both RGB color images, choose to use the image pixel RGB value cross-correlation registration to obtain the best registration position. 3. The method for automatic registration and fusion of aerial images and close-range images according to claim 2, wherein the registration control point algorithm is: E(u,v)＝Σ _x,y w(x,y)[I(x+u,y+v)-I(x,y)] ² Among them, w(x,y) is the window function, [I(x+u,y+v)-I(x,y)] ² is the gradient value of the image, and w(x,y) is a rectangular window or a Gaussian window , for each displacement (u, v), the bilinear approximation in the above formula is expressed as, $\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>\stackrel{<span\; class="notranslate">\; ~</span>}{<span\; class="notranslate">\; =</span>}<span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; \&rsqb;</span>\mathrm{<span\; class="notranslate">\; m</span>}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$ M is a 2*2 matrix, the specific formula is, $\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; A</span>}& \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}& \mathrm{<span\; class="notranslate">\; B</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; e</span>}}^{\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; \&CircleTimes;</span>\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; Ix</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; I</span>}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; I</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; I</span>}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; I</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}& {\mathrm{<span\; class="notranslate">\; Iy</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]$ Ix and Iy are the gradient values in the x and y directions of the image respectively; E is approximated as a local cross-correlation function, and M describes the shape at this point. Let λ ₁ and λ ₂ be the two eigenvalues of the matrix M, then λ ₁ , λ ₂ can represent the curvature of the local autocorrelation function. Due to isotropy, M maintains rotation invariance, where A, B, and C represent the element values of matrix M. 4. The method for automatic registration and fusion of aerial images and close-range images according to claim 2, wherein the cross-correlation registration algorithm is: Select a temporary window W(l,m) on the aerial image, and find the most similar window corresponding to it on the close-range image S(i,j) of the sign board It is a rotation transformation window, measured by cross-correlation similarity, where l and m are positive integers, $\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; \&lsqb;</span>{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&rsqb;</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}$ The window image W(i,j) is automatically searched in a scanning manner in the search for the foreground image S, to find the position (i ^* ,j ^* ) of the maximum point of the correlation function value R(i,j) in the process, and consider this is the best registration position. 5. The method for automatic registration and fusion of aerial images and close-range images according to claim 1, wherein said step 4 comprises: Step 4-1, the RGB value R ₁ (i,j), G ₁ (i,j), B ₁ (i,j) of the aerial image and the RGB value R ₂ (i,j) of the close-range image of the sign board are set , G ₂ (i,j), and B ₂ (i,j) are each multiplied by a weight coefficient, and fused to form a new image F(i,j), F(i,j)=aI ₁ (i,j)+(1-a)I ₂ (i,j) Among them, a is the weight factor, and 0≤a≤1, I ₁ is the aerial image, and I ₂ is the close-up image of the sign board. 6 . The method for automatic registration and fusion of aerial images and close-range images according to claim 2 , wherein the sign board is a right-angle fan-shaped structure. 7. The method for automatic registration and fusion of aerial images and close-range images according to claim 2, characterized in that the colors of the sign boards are different.