CN115031636B - Atmospheric turbulence error weakening method in visual displacement measurement of multi-angle point target - Google Patents

Abstract

The present invention provides a method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets, comprising: first, making a multi-corner point target used in visual displacement measurement, and fixing the multi-corner point target on a monitoring target; second, collecting images of the multi-corner point target; third, calculating the displacement of each corner point on a subsequent image relative to a first frame. 4. Average the displacement values of all corner points of the multi-corner target in the same frame to obtain the pixel displacement of each frame image after correction 5. Displace the target’s pixels Convert to physical displacement The present invention is based on the fact that the visual displacement measurement error caused by atmospheric turbulence has randomness in spatial distribution. By installing multi-corner point targets on the monitoring target, the coordinates of different corner points in the target space are obtained by using target detection and positioning algorithms, and the physical displacement of the target is finally obtained by averaging the displacement values of different corner points in space, thereby achieving the effect of weakening the displacement measurement error caused by atmospheric turbulence.

Description

A method to reduce atmospheric turbulence errors in visual displacement measurement of multi-corner targets

Technical Field

The invention relates to the technical field of visual measurement, and in particular to a method for weakening atmospheric turbulence errors in visual displacement measurement of multi-corner point targets.

Background technique

With the aging of engineering structures, environmental changes and the occurrence of accidents, engineering structures will suffer varying degrees of damage, which will not only cause material and property losses, but also endanger human life. In order to timely detect damage and harmful deformation of engineering structures and ensure their long-term safe use, it is very important to evaluate and maintain the structures on time. In the process of evaluating the structure, displacement is an important indicator for structural state assessment and performance evaluation. In recent years, vision-based displacement measurement systems have been successfully applied to the field of structural health monitoring. The system uses non-contact sensors to measure the target and obtain the displacement information of the structure. It can achieve long-distance, non-contact, high-precision, low-cost, multi-point high-frequency real-time monitoring, which has significant advantages over traditional monitoring methods.

However, visual sensors are easily affected by atmospheric turbulence when measuring structural displacement. Changes in the external environment cause the air temperature gradient to change continuously, the air density to change accordingly, and the atmospheric refractive index to become uneven, thus forming atmospheric turbulence. When light propagates in the atmosphere, the light beam from the same light source is affected by atmospheric turbulence and reaches the focal plane of the camera imaging device through a random path, and the resulting target image will show geometric distortion and blur. Therefore, the target image collected by the structural visual displacement measurement in the atmospheric turbulent medium will be distorted to varying degrees in time and space; among them: in the time domain, there is a certain difference in the grayscale between the collected image sequences; in the spatial domain, the collected target image is affected by atmospheric turbulence and shows degradation phenomena such as blur, jitter, offset, and random noise. Therefore, it is difficult to extract the displacement of the target structure with high precision from the image affected by atmospheric turbulence, resulting in displacement measurement errors; at the same time, the displacement measurement errors caused by random changes in atmospheric turbulence are mixed in the real displacement of the structural target. If the displacement measurement errors caused by atmospheric turbulence cannot be weakened, it will not be able to meet the needs of high-precision structural displacement measurement engineering projects.

At present, scholars have carried out relevant research on the visual displacement measurement errors caused by atmospheric turbulence. The main methods can be divided into three categories:

The first category is based on adaptive optics. The adaptive optics system originated from the field of ground-based astronomy to eliminate the problem of telescopes being affected by atmospheric turbulence. It uses a wavefront detector to feed back the distortion of the wavefront to a deformable mirror, thereby correcting the wavefront distortion and obtaining a target image that eliminates or weakens the influence of atmospheric turbulence, thereby obtaining information in the image.

The second category is based on image post-processing methods. This type of method restores images affected by atmospheric turbulence, thereby eliminating the influence of atmospheric turbulence on visual displacement measurement. This type of method can be roughly divided into two types: one is to select and fuse images with less distortion in the acquired image sequence to achieve image restoration; the other is to eliminate random local deformations in the image sequence through methods such as image registration, and use the deconvolution algorithm to restore the image;

The third type is a method based on a stable background. It uses a visual displacement measurement system to collect fixed objects in the image background (such as buildings, mountains, bridge piers, etc.) as reference objects, thereby eliminating the errors caused by the influence of atmospheric turbulence on the visual displacement measurement. The operation is convenient and simple.

However, the above three methods for processing visual displacement measurement errors caused by atmospheric turbulence still have the following shortcomings:

The first type of method based on adaptive optics requires that the light wave is approximately perpendicular to the ground during its propagation through the atmosphere, so as to achieve the purpose of eliminating atmospheric turbulence. However, the atmospheric turbulence near the ground is very complex. The use of such methods requires wavefront detectors, deformable mirrors and some precision optical instruments, which are relatively expensive. At the same time, such methods cannot completely compensate for atmospheric turbulence, and the high-frequency information of the target will be suppressed and attenuated. Image post-processing technology is needed to obtain a clearer target image.

The second type of image post-processing method can effectively remove image geometric distortion and reduce image blur, but it also loses a lot of texture detail information of the image, eliminates the real displacement information of the target structure, and reduces the accuracy of visual displacement measurement;

The third type of method based on a stable background is difficult to implement in practical applications. Previous studies have shown that the visual displacement measurement error caused by atmospheric turbulence increases with the distance from the camera to the target. In practical engineering applications, it is difficult to ensure that a stationary building is found as a fixed reference background in the same plane as the monitored target.

Summary of the invention

The present invention provides a method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets, comprising the following steps:

Step 1: Make a multi-corner point target used in visual displacement measurement, and fix the multi-corner point target on the monitoring target; the multi-corner point target is set to include more than fifteen corner points and the imaging area on the imaging plane of the industrial camera is greater than 50%, and the multiple corner points on the target are set to be square corner points of the same size, regularly arranged on the same plane, and the distance between two adjacent corner points is equal and the distance between two adjacent corner points is greater than 10 pixels;

Step 2: Install the visual displacement monitoring system to collect multi-corner target images;

Step 3: Use the target detection and positioning algorithm to detect the coordinates of each corner point on each frame of the multi-corner target image, and use the first frame as the reference frame to calculate the displacement of each corner point on the subsequent image relative to the first frame.

Step 4: Average the displacement values of all corner points of the multi-corner target in the same frame to obtain the pixel displacement of each frame image after correction. According to the pixel size _Limage of the target on the image plane and the real physical size _Lphysical , the scale factor SF is calculated to shift the pixel size of the target Convert to physical displacement

Optionally, in step 1, the design size of the multi-corner target is calculated using the following formula:

Where: d is the number of pixels corresponding to the multi-corner target in the image plane, z is the plane distance from the camera to the multi-corner target, f is the focal length of the camera lens, and d _pixel is the pixel size of the camera.

Optionally, in step 3, the displacement of each corner point on the target image relative to the corresponding corner point on the reference frame is The calculation formula is as follows:

Where: k = 1...N, i = 1...n, is the coordinate of each corner point on the target in the subsequent frame, is the coordinate of each corner point on the target in the reference frame, k is the current frame number, N is the total number of image frames, i is the number of target corner points, and n is the total number of target corner points.

Optionally, in step 4, the displacement values of all corner points of the multi-corner point target in the same frame are averaged to obtain the pixel displacement of each frame image after correction. The calculation formula is as follows:

Optionally, in step 4, the calculation formula of the scale factor SF is as follows:

Optionally, in step 4, the actual physical displacement of the target The calculation formula is as follows:

Compared with the prior art, the present invention has the following beneficial effects:

(1) Based on the fact that the visual displacement measurement error caused by atmospheric turbulence has a random characteristic in spatial distribution, the present invention installs multi-corner point targets on the monitoring target, uses target detection and positioning algorithms to obtain the coordinates of different corner points in the target space, calculates its displacement, and finally obtains the physical displacement of the target by averaging the displacement values of different corner points in space, thereby achieving the effect of weakening the displacement measurement error caused by atmospheric turbulence. The method of the present invention has a simple measuring device, is easy to operate, has low cost, and is easy to implement. It effectively reduces the impact of atmospheric turbulence on visual displacement measurement and can improve the accuracy of visual displacement measurement. At the same time, the method of the present invention does not require the use of stable reference targets and additional measuring instruments, is applicable to a variety of engineering application scenarios, and has good engineering practicality.

(2) The present invention provides a method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets. First, a multi-corner point measurement target is produced based on the characteristics of atmospheric turbulence errors in visual displacement measurement; then, target images affected by atmospheric turbulence on the path of the visual measurement sensor are continuously collected; further, the coordinates of the corner points of the multi-corner point target are obtained by using a target detection and positioning algorithm, and the displacement of each corner point relative to the corresponding point on the reference frame is obtained using the first frame as a reference frame; finally, the displacement values of the corner points in the target space are averaged, thereby reducing the visual displacement measurement error caused by atmospheric turbulence, accurately extracting the short-term and high-frequency displacement of the target structure, and obtaining an accurate visual displacement measurement result.

(3) The method steps provided by the present invention can be used to reduce the visual displacement measurement error caused by atmospheric turbulence, which can be applied to the short-term and high-frequency visual displacement measurement of engineering structures to meet the needs of high-precision visual displacement measurement.

In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

FIG1 is a schematic flow diagram of a method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets according to an embodiment of the present invention;

FIG2 is a schematic diagram of a multi-corner point target in an embodiment of the present invention;

FIG3 is a schematic diagram of the results before and after using a multi-corner point target to weaken the influence of atmospheric turbulence on visual displacement measurement in an embodiment of the present invention.

Detailed ways

In order to make the above-mentioned purposes, features and advantages of the present invention more clear and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It should be noted that the drawings of the present invention are all simplified and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the implementation of the present invention; the several mentioned in the present invention are not limited to the specific numbers in the examples in the accompanying drawings; the directions or positional relationships indicated by "front", "middle", "back", "left", "right", "up", "down", "top", "bottom", "middle", etc. mentioned in the present invention are based on the directions or positional relationships shown in the drawings of the present invention, and do not indicate or imply that the devices or components referred to must have specific directions, nor can they be understood as limitations on the present invention.

As shown in FIG1 , a method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets includes the following steps:

Step 1. The random fluctuation of atmospheric refractive index caused by atmospheric turbulence will cause random fluctuation of light when it reaches the camera. At the same time, different positions of the target are affected differently by atmospheric turbulence, and the movement of the target structure in space is an overall translation in the same direction. Therefore, the present invention adopts a target with multiple corner points of uniform size, square structure and regular arrangement on the same plane fixed at the monitoring target to reduce the influence of atmospheric turbulence on visual displacement measurement, thereby obtaining a more accurate displacement measurement result; preferably, the distance between two adjacent corner points of the multi-corner point target is set equal (specifically, in order to ensure that the corner points are clearly imaged on the imaging plane of the industrial camera, the distance between two adjacent corner points should be greater than 10 pixels), and under the premise of saving calculation efficiency, the number of corner points is as large as possible to ensure that the area of the target on the camera imaging plane is maximized as much as possible (specifically, the area of the target on the camera imaging plane should be not less than 50%).

Specifically, in order to make the target area on the camera imaging plane greater than or equal to 50% and set multiple corner points of the target, according to the design concept of multi-corner point target, the distance from the visual displacement measurement system to the fixed multi-corner point target plane and the camera parameters, the design size D of the multi-corner point target is calculated using the following formula to ensure the target area on the camera imaging plane:

Where: d is the number of pixels corresponding to the multi-corner target in the image plane, z is the plane distance from the camera to the multi-corner target, f is the focal length of the camera lens, and d _pixel is the pixel size of the camera.

Step 2: Install the visual displacement measurement system at a suitable distance from the monitoring target (the visual displacement measurement system consists of an industrial camera, a tripod, a computer, and a data cable connecting the computer and the industrial camera), adjust the camera lens, aperture and focal length in the visual displacement measurement system so that the multi-corner target is evenly imaged within the camera's field of view, turn on the camera to a stable state, and collect multi-corner target images in a short time and high frequency, and save them to the computer.

Step 3: Calculate the image coordinates of each corner point of the target: Use the target detection and positioning algorithm to detect the coordinates of the corner points on each frame of the multi-corner target image, and use the first frame as the reference frame to calculate the coordinates of each corner point on the target image relative to the corresponding corner point on the reference frame.

Where: k = 1...N, i = 1...n, is the coordinate of each corner point on the target in the subsequent frame, is the coordinate of each corner point on the target in the reference frame, k is the current frame number, N is the total number of image frames, i is the number of target corner points, and n is the total number of target corner points.

Step 4: Average the displacement values of all corner points of the multi-corner target in the same frame to obtain the pixel displacement of each frame image after correction. To achieve the effect of weakening the influence of atmospheric turbulence on visual displacement measurement; pixel displacement The calculation formula is as follows:

The scale factor SF is calculated based on the pixel size L _image (pixel) of the target on the image plane and the actual physical size L _physical (mm), thereby shifting the pixel size of the target Convert to real physical displacement Wherein: the pixel size _Limage is obtained by calculating the pixel length on the image plane according to the pixel coordinates of two adjacent corner points on the target image, and the unit is pixel; the real physical size _Lphysical is the real design size of two adjacent corner points when making a multi-corner target, and the unit is mm.

Specifically, the calculation formula of the scale factor SF is as follows:

Specific, real physical displacement The calculation formula is as follows:

Embodiment 1:

The specific process of applying the above-mentioned method of using multiple corner point targets to reduce the atmospheric turbulence error in visual displacement measurement in the laboratory to verify the accuracy of visual displacement measurement is as follows:

Step 1: Make a multi-corner target. Specifically, when the visual displacement measurement system is 5m away from the monitoring target, the lens focal length is 75mm, and the camera pixel size is 3.45μm/pixel, a multi-corner target is designed according to the design concept of the multi-corner target and the camera parameters, as shown in Figure 2. The target size is designed to be 12×27cm, and there are 8×3 corner points on the target plane. The spacing between two adjacent corner points is 105 pixels, and the imaging area of the target on the visual displacement measurement system exceeds 70%. The multi-corner target is made by the above method and fixed on a stationary wall 5m away from the visual displacement measurement system. By collecting images of the multi-corner target affected by atmospheric turbulence, the effectiveness of using multi-corner targets to reduce the displacement measurement errors caused by atmospheric turbulence is verified.

Step 2: The process of installing the visual displacement measurement system and collecting multi-corner target images is as follows:

(1) The verification experiment of the present invention was carried out in a closed laboratory. Since the manufactured multi-corner point target was fixed on a stationary wall 5 m away from the visual displacement measurement system, the displacement of the target containing the multi-corner points was monitored to be 0;

(2) Install the visual displacement measurement system on a stable ground, adjust the camera lens, aperture and focal length in the visual displacement measurement system so that the multi-corner target is evenly imaged within the camera field of view, and wait for the camera to reach a stable state;

(3) Place a heater to simulate atmospheric turbulence between the wall of the fixed target and the camera, adjust the heater to a stable temperature, continuously collect images of the wall target and save them to a computer;

(4) Multi-corner target images are collected with a camera sampling rate of 90 frames/s and a sampling time of 20 seconds, and the scale factor is approximately 0.28 mm/pixel.

Step 3: Use the corner detection algorithm to obtain the coordinates of the 24 corner points on the multi-corner target, and use the first frame of the acquired image as the reference frame to calculate the displacement of all corner points on each frame of the target image relative to the corresponding points in the reference frame.

Step 4: Further average the displacement values of all target corner points in the same frame of the multi-corner point target, and finally obtain the displacement result that weakens the influence of atmospheric turbulence on visual displacement measurement.

Specifically, the results before and after using multi-corner point targets to weaken the influence of atmospheric turbulence on visual displacement measurement are shown in Figure 3 and Table 1, respectively.

Table 1

Step 5. The above experimental results show that before using multi-corner target to weaken the influence of atmospheric turbulence on visual displacement measurement, the maximum displacement error (Y _max ) in the vertical direction reached -0.656 mm, the root mean square error (RMSE) was 0.126 mm, and the variance σ ² was 0.016 mm ² ; and after using multi-corner target to weaken the influence of atmospheric turbulence on visual displacement measurement, the maximum displacement error (Y _max ) in the vertical direction was reduced to 0.299 mm, the root mean square error (RMSE) was 0.061 mm, and the variance σ ² was 0.004 mm ² . It can be seen that after adopting the method of using multi-corner target to weaken the visual displacement measurement error in this application, the root mean square error (RMSE) correction rate reached 51.6% and the variance σ ² was reduced by 75%, which verifies the effectiveness of the method proposed in the present invention.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims (5)

1. A method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets, characterized in that it comprises the following steps: Step 1: Make a multi-corner point target used in visual displacement measurement, and fix the multi-corner point target on the monitoring target; the multi-corner point target is set to include more than fifteen corner points and the imaging area on the imaging plane of the industrial camera is greater than 50%, and the multiple corner points on the target are set to be square corner points of the same size, regularly arranged on the same plane, and the distance between two adjacent corner points is equal and the distance between two adjacent corner points is greater than 10 pixels; Step 2: Install the visual displacement monitoring system to collect multi-corner target images; Step 3: Use the target detection and positioning algorithm to detect the coordinates of each corner point on each frame of the multi-corner target image, and use the first frame as the reference frame to calculate the displacement of each corner point on the subsequent image relative to the first frame. Step 4: Average the displacement values of all corner points of the multi-corner target in the same frame to obtain the pixel displacement of each frame image after correction. According to the pixel size _Limage of the target on the image plane and the real physical size _Lphysical , the scale factor SF is calculated to shift the pixel size of the target Convert to physical displacement In the step 1, the design size of the multi-corner target is calculated using the following formula: Where: d is the number of pixels corresponding to the multi-corner target in the image plane, z is the plane distance from the camera to the multi-corner target, f is the focal length of the camera lens, and d _pixel is the pixel size of the camera. 2. The method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets according to claim 1, characterized in that in step 3, the displacement of each corner point on the target image relative to the corresponding corner point on the reference frame The calculation formula is as follows: Where: k = 1...N, i = 1...n, is the coordinate of each corner point on the target in the subsequent frame, is the coordinate of each corner point on the target in the reference frame, k is the current frame number, N is the total number of image frames, i is the number of target corner points, and n is the total number of target corner points. 3. The method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets according to claim 1 is characterized in that in the step 4, the displacement values of all corner points of the multi-corner point target in the same frame are averaged to obtain the pixel displacement of each frame image after correction. The calculation formula is as follows: 4. The method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets according to claim 3, characterized in that in the step 4, the calculation formula of the scale factor SF is as follows: 5. The method for reducing atmospheric turbulence errors in visual displacement measurement of multi-corner point targets according to claim 4, characterized in that in step 4, the real physical displacement of the target The calculation formula is as follows: