Atmospheric turbulence error weakening method in visual displacement measurement of multi-angle point target
Technical Field
The invention relates to the technical field of vision measurement, in particular to an atmospheric turbulence error weakening method in vision displacement measurement of a multi-point target.
Background
With the aging of engineering structures, environmental changes and accidents, the engineering structures can be damaged to different degrees, which not only can bring about the loss of substances and property, but also can endanger the life safety of people. In order to discover the damage and harmful deformation of the engineering structure in time, the long-term safe use of the engineering structure is ensured, and the structure is evaluated and maintained on time. In the process of evaluating the structure, the displacement is an important index for evaluating the structure state and performance. In recent years, a vision-based displacement measurement system has been successfully applied to the field of structural health monitoring, and the system utilizes a non-contact sensor to measure a target so as to obtain structural displacement information, so that long-distance, non-contact, high-precision, low-cost and multi-point high-frequency real-time monitoring can be realized, and the system has remarkable advantages compared with a traditional monitoring method.
However, visual sensors are susceptible to atmospheric turbulence when measuring structural displacements. The change of the external environment makes the air temperature gradient change continuously, the air density changes along with the change, and the refractive index of the atmosphere becomes uneven, thereby forming the atmosphere turbulence. When light propagates in the atmosphere, the light beams from the same light source are affected by the turbulence of the atmosphere, and reach the focal plane of the camera imaging device through random paths, so that geometrical distortion, blurring and the like of a formed target image occur. Therefore, the structural visual displacement measurement of the target image acquired in the atmosphere turbulence medium can be distorted to different degrees in time and space; wherein: in the time domain, a certain difference exists in gray level between the acquired image sequences; in the spatial domain, the acquired target image is influenced by atmospheric turbulence to generate degradation phenomena such as blurring, shaking, offset, random noise and the like. Therefore, it is difficult to extract the displacement of the target structure with high accuracy from the image affected by the atmospheric turbulence, resulting in a displacement measurement error; meanwhile, the displacement measurement error caused by the atmospheric turbulence is mixed in the actual displacement of the structural target, and if the displacement measurement error caused by the atmospheric turbulence cannot be weakened, the requirement of a high-precision structural displacement measurement engineering project cannot be met.
Currently, regarding to visual displacement measurement errors caused by atmospheric turbulence, related researches have been carried out by students, and main methods are divided into three categories:
The first is based on the adaptive optics method, the adaptive optics system is originally from the field of foundation astronomy to eliminate the problem that the telescope is influenced by the atmospheric turbulence, it uses the wave front detector to feed back the wave front distortion to the deformable mirror, thus correct the wave front distortion, get the goal image which eliminates or weakens the influence of the atmospheric turbulence, thus obtain the information in the image;
The second type is an image post-processing-based method that performs restoration processing on an image affected by atmospheric turbulence, thereby eliminating the influence of atmospheric turbulence on the visual displacement measurement, and such methods can be roughly classified into two types: one is to select and fuse the image with less distortion in the collected image sequence to realize the image restoration; one is to eliminate random local deformation in an image sequence by image registration and other methods, and restore the image by using a deconvolution algorithm;
The third type is a method based on stable background, which uses a visual displacement measurement system to collect fixed objects (such as buildings, mountains, bridge piers and the like) in the image background as reference objects, so that errors caused by the influence of atmospheric turbulence on visual displacement measurement are eliminated, and the operation is convenient and simple.
However, the three methods for processing visual displacement measurement errors caused by atmospheric turbulence still have the following disadvantages:
The first type of self-adaptive optics-based method requires light waves to be approximately vertical to the ground in the process of transmitting through the atmosphere, so that the aim of eliminating the atmospheric turbulence is fulfilled, but the atmospheric turbulence near the ground is quite complex, and the cost is relatively high by using the method which requires a wavefront detector, a deformable mirror and a plurality of precise optical instruments; meanwhile, the method is incomplete in compensating for atmospheric turbulence, high-frequency information of a target can be restrained and attenuated, and a clearer target image is required to be acquired by using an image post-processing technology;
The second type of image post-processing-based method can effectively remove geometric distortion of an image and reduce image blurring, also loses a large amount of texture detail information of the image, eliminates real displacement information of a target structure, and reduces the accuracy of visual displacement measurement;
The third class of stable background-based methods are difficult to realize in practical application; studies have shown that the visual displacement measurement error caused by atmospheric turbulence increases with the distance from the camera to the target, and it is difficult in practical engineering applications to ensure that stationary buildings are found in the same plane of the monitored target as a fixed reference background.
Disclosure of Invention
The invention provides an atmospheric turbulence error weakening method in visual displacement measurement of a multi-point target, which comprises the following steps:
Step one, manufacturing a multi-point target used in visual displacement measurement, and fixing the multi-point target on a monitoring target; the multi-angle point target is a target which comprises more than fifteen angle points and has an imaging area on an imaging plane of an industrial camera of more than 50%, the angle points on the target are all square angle points which are consistent in size and are regularly arranged on the same plane, the distance between two adjacent angle points is equal, and the distance between two adjacent angle points is more than 10 pixels;
Step two, installing a visual displacement monitoring system, and collecting a multi-corner target image;
detecting the coordinates of each corner point on each frame of multi-corner target image by using a target detection and positioning algorithm, and calculating the displacement of each corner point on the subsequent image relative to the first frame by taking the first frame as a reference frame
Step four, carrying out average treatment on displacement values of all angular points on the same frame of the multi-angular point target to obtain pixel displacement of each corrected frame of imageCalculating scale factors SF according to the pixel size L image and the real physical size L physical of the target on the image plane, and shifting the pixels of the targetConversion to physical displacement
Optionally, in the first step, the design size of the multi-point target is calculated by the following formula:
wherein: d is the corresponding pixel number of the multi-angle point target on the image plane, z is the plane distance from the camera to the multi-angle point target, f is the focal length of the camera lens, and d pixel is the camera pixel size.
Optionally, in the third step, the displacement of each corner point on the target image relative to the corresponding corner point on the reference frameThe calculation formula of (2) is as follows:
Wherein: k=1 … N, i=1 … N, For the coordinates of each corner on the target on the subsequent frame,For the coordinates of each corner point on the target on the reference frame, k is the current frame number, N is the total frame number of the image, i is the number of target corner points, and N is the total number of target corner points.
Optionally, in the fourth step, the displacement values of all the corners on the same frame of the multi-corner target are averaged to obtain the pixel displacement of each corrected frame of imageThe calculation formula of (2) is as follows:
optionally, in the fourth step, a calculation formula of the scale factor SF is as follows:
optionally, in the fourth step, the actual physical displacement of the target The calculation formula of (2) is as follows:
compared with the prior art, the invention has the following beneficial effects:
(1) The invention is based on the characteristic that the visual displacement measurement error caused by the atmospheric turbulence has randomness in space distribution, the coordinates of different angular points in the space of the target are obtained by installing the multi-angular point target on the monitored target and utilizing the target detection and positioning algorithm, the displacement is calculated, and the physical displacement of the target is finally obtained by averaging the displacement values of different angular points in the space, thereby achieving the effect of weakening the displacement measurement error caused by the atmospheric turbulence. The method has the advantages of simple measuring device, easy operation, lower cost and small implementation difficulty, effectively reduces the influence of atmospheric turbulence on visual displacement measurement, and can improve the accuracy of visual displacement measurement. Meanwhile, the method of the invention does not need to use a stable reference target and an additional measuring instrument, is suitable for various engineering application scenes, and has good engineering practicability.
(2) According to the atmospheric turbulence error weakening method in visual displacement measurement of the multi-point target, firstly, a multi-point measurement target is manufactured based on the characteristics of atmospheric turbulence errors in visual displacement measurement; continuously collecting target images influenced by atmospheric turbulence on a path of a vision measurement sensor; further, acquiring coordinates of target corner points of the multi-corner points by using a target detection and positioning algorithm, and acquiring displacement of each corner point relative to a corresponding point on a reference frame by taking the first frame as the reference frame; and finally, carrying out average treatment on displacement values of angular points in a target space, so as to weaken visual displacement measurement errors caused by atmospheric turbulence, accurately extract short-time high-frequency displacement of a target structure, and obtain an accurate visual displacement measurement result.
(3) The method provided by the invention weakens the visual displacement measurement error caused by atmospheric turbulence, can be applied to short-time high-frequency visual displacement measurement of engineering structures, and meets the requirement of high-precision visual displacement measurement.
In addition to the objects, features and advantages described above, the present invention has other objects, features and advantages. The present invention will be described in further detail with reference to the drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
FIG. 1 is a schematic flow chart of a method for weakening atmospheric turbulence errors in visual displacement measurement of a multi-point target in an embodiment of the invention;
FIG. 2 is a schematic illustration of a multi-point target in an embodiment of the invention;
FIG. 3 is a schematic representation of the results of an embodiment of the present invention before and after using a multi-point target to attenuate atmospheric turbulence to effect visual displacement measurements.
Detailed Description
In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It should be noted that the drawings of the present invention are in simplified form and are not precisely scaled, so as to facilitate the clear and convenient explanation of the implementation of the present invention; the invention is not limited to the specific numbers mentioned in the examples of the drawings; the directions and positional relationships indicated by the terms "rear", "left", "right", "upper", "lower", "top", "bottom", "middle", etc. in the present invention are all based on the directions and positional relationships shown in the drawings of the present invention, and do not indicate or imply that the device or component to be referred to must have a specific direction, nor should it be construed as limiting the present invention.
Referring to fig. 1, a method for weakening atmospheric turbulence errors in visual displacement measurement of a multi-point target comprises the following steps:
Step one, random fluctuation of the atmospheric refractive index caused by atmospheric turbulence can cause random fluctuation when light reaches a camera, different positions of a target are affected by the atmospheric turbulence at the same moment, and the movement of a target structure in space is integral translation in the same direction, so that targets which are consistent in size, square in structure and regularly arranged on the same plane and comprise a plurality of corner points are fixed at a monitoring target, the influence of the atmospheric turbulence on visual displacement measurement is reduced, and a more accurate displacement measurement result is obtained; here, preference is given to: the distances between two adjacent angular points of the multi-angular point target are equal (specifically, in order to ensure that the angular points are imaged clearly on the imaging plane of the industrial camera, the distance between the two adjacent angular points is larger than 10 pixels), and on the premise of saving the calculation efficiency, the number of the angular points is as large as possible so as to ensure that the area of the target on the imaging plane of the camera is maximized as much as possible (specifically, the area of the target on the imaging plane of the camera is not smaller than 50%).
Specifically, in order to enable the area of the target on the camera imaging plane to be greater than or equal to 50% and set a plurality of corner points of the target, according to the design idea of the multi-corner target, the distance from the visual displacement measurement system to the fixed multi-corner target plane and camera parameters, the design size D of the multi-corner target is calculated by adopting the following formula so as to ensure the area of the target on the camera imaging plane:
wherein: d is the corresponding pixel number of the multi-angle point target on the image plane, z is the plane distance from the camera to the multi-angle point target, f is the focal length of the camera lens, and d pixel is the camera pixel size.
Installing a visual displacement measurement system (the visual displacement measurement system consists of an industrial camera, a tripod, a computer and a data line connecting the computer and the industrial camera) at a proper position from a monitoring target, adjusting a camera lens, an aperture and a focal length in the visual displacement measurement system to enable the target of the multi-angle point to be uniformly imaged within a camera view field range, starting the camera to a stable state, and then acquiring a multi-angle point target image in a short time and high frequency mode and storing the multi-angle point target image in the computer.
Calculating the image coordinates of each corner point of the target: detecting coordinates of corner points on each frame of multi-corner target image by using a target detection and positioning algorithm, and calculating corresponding angles of each corner point on the target image relative to a reference frame by taking a first frame as the reference frame
Wherein: k=1 … N, i=1 … N,For the coordinates of each corner on the target on the subsequent frame,For the coordinates of each corner point on the target on the reference frame, k is the current frame number, N is the total frame number of the image, i is the number of target corner points, and N is the total number of target corner points.
Step four, carrying out average treatment on displacement values of all angular points on the same frame of the multi-angular point target to obtain pixel displacement of each corrected frame of imageSo as to achieve the effect of weakening the influence of atmospheric turbulence on visual displacement measurement; pixel displacementThe calculation formula of (2) is as follows:
Calculating scale factor SF according to pixel size L image (pixel) and true physical size L physical (mm) of target on image plane, thereby shifting pixel of target Conversion to true physical displacementWherein: the pixel size L image is obtained by calculating the pixel length of the target image on the image plane according to the pixel coordinates of two adjacent corner points on the target image, wherein the unit is a pixel; the real physical dimension L physical is specifically the real design dimension of two adjacent corner points in mm when the multi-corner target is manufactured.
Specifically, the calculation formula of the scale factor SF is as follows:
In particular, true physical displacement The calculation formula of (2) is as follows:
example 1:
The method for weakening the atmospheric turbulence error in the visual displacement measurement by using the multi-point target has the following specific process of carrying out the experiment for verifying the visual displacement measurement accuracy in a laboratory:
Step one, manufacturing a multi-point target, specifically, when the visual displacement measurement system is 5m away from the monitoring target, the focal length of the lens is 75mm, and the pixel size of the camera is 3.45 mu m/pixel, designing the multi-point target according to the design thought of the multi-point target and the camera parameters, and referring to fig. 2. The target size is designed to be 12 multiplied by 27cm, the total design is 8 multiplied by 3 corner points on the target plane, the distance between two adjacent corner points is 105 pixels, and the imaging area of the target on the visual displacement measurement system is more than 70%. The method is used for manufacturing the multi-corner targets and fixing the multi-corner targets on a static wall surface which is 5m away from a visual displacement measuring system. By acquiring images of the multi-angular point targets affected by atmospheric turbulence, the effectiveness of displacement measurement errors caused by atmospheric turbulence is demonstrated to be weakened by using targets with multi-angular points.
Step two, installing a visual displacement measurement system and acquiring a multi-corner target image, wherein the process is specifically as follows:
(1) The verification experiment of the invention is carried out in a closed laboratory, and the manufactured multi-angle point target is fixed on a static wall surface which is 5m away from a visual displacement measurement system, so that the displacement of the target containing the multi-angle point is monitored to be 0;
(2) Installing a visual displacement measurement system on the stable ground, adjusting a camera lens, an aperture and a focal length in the visual displacement measurement system to enable a multi-angle target to be imaged uniformly in a camera view field range, and waiting for the camera to be in a stable state;
(3) Placing a heater simulating the generation of atmospheric turbulence between the wall surface of the fixed target and the camera, regulating the heater to a stable temperature, continuously collecting wall surface target images, and storing the wall surface target images into a computer;
(4) The multi-corner target image is acquired in a mode that the sampling rate of a camera is 90 frames/second and the sampling time is 20 seconds, and the scale factor is about 0.28 mm/pixel.
And thirdly, acquiring coordinates of 24 corners on the multi-corner target by using a corner detection algorithm, and calculating displacement of all corners on each frame of image of the target relative to corresponding points of a reference frame by taking the acquired first frame of image as the reference frame.
And step four, further carrying out average treatment on the displacement values of all target corner points on the same frame of the multi-corner target, and finally obtaining a displacement result weakening the influence of atmospheric turbulence on visual displacement measurement.
Specifically, the results before and after the impact of atmospheric turbulence on visual displacement measurement using a multi-point target are shown in fig. 3 and table 1, respectively.
TABLE 1
Step five, the experimental results show that the maximum displacement error (Y max) in the vertical direction reaches-0.656 mm, the Root Mean Square Error (RMSE) is 0.126mm, and the variance sigma 2 is 0.016mm 2 before the influence of the multi-point target on visual displacement measurement is weakened; and the maximum displacement error (Y max) in the vertical direction is reduced to 0.299mm, the Root Mean Square Error (RMSE) is 0.061mm, and the variance sigma 2 is 0.004mm 2 after the influence of the multi-point target on visual displacement measurement is weakened. Therefore, the correction rate of Root Mean Square Error (RMSE) reaches 51.6% and the variance sigma 2 is reduced by 75% after the method for weakening vision displacement measurement errors by using the multi-point targets is adopted, so that the effectiveness of the method is verified.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.