CN111079550A - Bridge monitoring method and system based on binocular vision - Google Patents
Bridge monitoring method and system based on binocular vision Download PDFInfo
- Publication number
- CN111079550A CN111079550A CN201911159173.7A CN201911159173A CN111079550A CN 111079550 A CN111079550 A CN 111079550A CN 201911159173 A CN201911159173 A CN 201911159173A CN 111079550 A CN111079550 A CN 111079550A
- Authority
- CN
- China
- Prior art keywords
- camera
- image information
- coordinate system
- calibration plate
- bridge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000006073 displacement reaction Methods 0.000 claims abstract description 24
- 238000006243 chemical reaction Methods 0.000 claims abstract description 17
- 230000008859 change Effects 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 9
- 238000012163 sequencing technique Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 2
- 238000005259 measurement Methods 0.000 abstract description 10
- 230000009286 beneficial effect Effects 0.000 description 9
- 238000001514 detection method Methods 0.000 description 8
- 230000001360 synchronised effect Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 238000009966 trimming Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 239000004566 building material Substances 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to a bridge monitoring method and a system based on binocular vision, wherein the method comprises the steps of calibrating a first camera and a second camera, acquiring the relative pose relation between the two cameras, and determining the real-time three-dimensional coordinate of a central point on a calibration plate under a first camera coordinate system; calculating a coordinate conversion relation between a first camera coordinate system and a calibration plate coordinate system according to the calibration information, and converting a three-dimensional coordinate of the central point under the first camera coordinate system into a real-time three-dimensional coordinate under the calibration plate coordinate system; and determining displacement information of the bridge by combining the three-dimensional coordinates of the initial moment center point under the coordinate system of the calibration plate. According to the invention, through calibrating the relative pose relationship between the two cameras, converting the three-dimensional coordinate of the central point of the calibration plate under the coordinate system of the first camera into the coordinate system of the calibration plate, determining the displacement information of the bridge by combining the coordinate of the central point of the initial moment under the coordinate system of the calibration plate, replacing manual measurement, the precision is high, the efficiency is high, and the price is low.
Description
Technical Field
The invention relates to the technical field of bridge monitoring, in particular to a bridge monitoring method and system based on binocular vision.
Background
Most of the existing bridge monitoring systems use a manual measurement method to measure the displacement of the bridge once at a long time interval, and the method is slow in measurement, cannot monitor the displacement of the bridge in real time and cannot measure a plurality of points simultaneously.
Concrete is the most widely used building material in building frames. The concrete research and application practice of modern science shows that: in the bridge with the concrete material structure, due to expansion with heat and contraction with cold and the bridge is under the action of external force, the bridge is inevitably deviated. At present, the detection of the danger mostly stays in a manual stage, a short-distance detection instrument or manual detection is usually used, the offset of the bridge is obtained by inaccurate measurement or estimation, the method needs detection personnel to regularly detect by means of road detection or erected manual equipment, the working intensity is high, the detection cost is high, and the requirement on the safety of the personnel is high; and although the bridge part is found to be in a specified offset range during manual detection, after a period of time, the change of the offset does not pay attention, when severe environment and load aggravation suddenly change, the offset is suddenly increased under instant strong pressure, and serious consequences are generated because real-time detection and quick early warning cannot be carried out.
Most of the existing bridge monitoring systems use a manual measurement method to measure the displacement of a bridge once at a long time interval, and the method has the disadvantages of slow measurement, low efficiency, incapability of monitoring the displacement of the bridge in real time and simultaneous measurement of a plurality of points, and high consumption of manpower and material resources.
Disclosure of Invention
The invention aims to solve the technical problem of providing a bridge monitoring method and system based on binocular vision aiming at the defects of the prior art.
The technical scheme for solving the technical problems is as follows: a bridge monitoring method based on binocular vision comprises the following steps:
calibrating a first camera and a second camera which are set to be a distance away from the bridge by matching with a calibration plate arranged on the bridge, acquiring a relative pose relation between the first camera and the second camera according to calibration information, and determining a real-time three-dimensional coordinate of a central point on the calibration plate under a first camera coordinate system;
calculating a coordinate conversion relation between the first camera coordinate system and a calibration plate coordinate system according to the calibration information, and converting the real-time three-dimensional coordinate of the central point under the first camera coordinate system into the real-time three-dimensional coordinate under the calibration plate coordinate system;
and determining the displacement information of the bridge according to the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the current moment and the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the initial moment, and generating and displaying a displacement curve.
The invention has the beneficial effects that: according to the bridge monitoring method based on binocular vision, a binocular vision system consisting of the first camera and the second camera monitors the bridge, real-time three-dimensional coordinates of a central point on a calibration plate under a first camera coordinate system are determined by calibrating the relative pose relationship between the first camera and the second camera, the three-dimensional coordinates are converted into a calibration plate coordinate system, displacement information of the bridge is determined by combining coordinates of the central point under the calibration plate coordinate system at the initial moment, manual measurement is replaced, the advantages of real-time performance, high precision and low price of the binocular vision system are combined, multi-point simultaneous monitoring is conveniently achieved, and monitoring efficiency is greatly improved.
On the basis of the technical scheme, the invention can be further improved as follows:
further: the calibration of the first camera and the second camera at the set distance from the bridge specifically comprises the following steps:
synchronously shooting a calibration plate arranged on the bridge through a first camera and a second camera which are away from the bridge by a set distance to obtain first image information and second image information;
respectively identifying the first image information and the second image information, and respectively acquiring a plurality of feature points and corresponding pixel coordinates from the first image information and the second image information;
respectively sequencing and matching a plurality of feature points in the first image information and the second image information, and determining a relative pose relation between the first camera and the second camera according to pixel coordinates of a plurality of matched pairs of feature points;
wherein the calibration information comprises: the image processing device comprises first image information, second image information, a plurality of feature points and corresponding pixel coordinates in the first image information, and a plurality of feature points and corresponding pixel coordinates in the second image information.
The beneficial effects of the further scheme are as follows: the first camera and the second camera respectively acquire first image information and second image information, and extract the characteristic points for sequencing and matching, so that the relative pose relationship between the first camera and the second camera can be accurately determined according to the pixel coordinates of a plurality of matched pairs of characteristic points, the data processing amount is small, the efficiency is greatly improved, and the method can completely adapt to practical application scenes.
Further: after the first image information and the second image information are obtained, the method further comprises the following steps:
and respectively carrying out primary cutting on the first image information and the second image information by taking the central point of the calibration plate as a center, and respectively carrying out identification processing on the first image information and the second image information after the primary cutting.
The beneficial effects of the further scheme are as follows: by respectively carrying out primary cutting on the first image information and the second image information, the image size can be reduced as much as possible, corresponding feature points can be found after the calibration plate is displaced, the calculation speed is improved, and the memory consumption is reduced.
Further: the determining of the real-time three-dimensional coordinate of the central point on the calibration plate under the first camera coordinate system specifically includes:
and determining the real-time three-dimensional coordinate of the central point on the calibration plate under the first camera coordinate system according to the first image information and the second image information which are real-time after the initial cutting and the relative pose relationship.
The beneficial effects of the further scheme are as follows: the coordinates of the central point in the first image information and the second image information are respectively obtained through the first image information and the second image information which are real-time after the first cutting, and then the real-time three-dimensional coordinates of the central point under the coordinate system of the first camera can be determined by combining the first image information and the second image information, so that the accurate positioning is realized.
Further: before determining the three-dimensional coordinates of the center point on the calibration plate in the first camera coordinate system, the method further comprises the following steps:
and respectively secondarily cutting the first image information and the second image information which are in real time after the primary cutting by taking the central point of the calibration plate as a center, and determining the real-time three-dimensional coordinate of the central point on the calibration plate under a first camera coordinate system according to the combination of the first image information and the second image information which are in real time after the secondary cutting and the relative pose relationship.
The beneficial effects of the further scheme are as follows: through will be real-time after the first image information and the second image information of tailorring for the second time, both can further dwindle first image information and second image information can guarantee again to find the central point after the calibration plate takes place the displacement, improve the computational rate, reduce and consume the memory.
Further: the first camera and the second camera synchronously shoot the calibration plates arranged on the bridge respectively, and the specific implementation of acquiring the first image information and the second image information is as follows:
taking a flash light signal of the first camera as a shooting trigger signal of the second camera, and caching image information shot by the first camera and the second camera;
and selecting two pieces of image information with the same shooting time from the image information cached by the first camera and the second camera respectively as the first image information and the second image information.
The beneficial effects of the further scheme are as follows: the flash light signal of the first camera is used as the shooting trigger signal of the second camera, so that the first camera and the second camera can be kept completely synchronous during shooting, and two pieces of image information with the same shooting time are selected from the buffered image information after shooting, so that the first image information and the second image information used for determining the relative pose relationship between the first camera and the second camera can be images shot by the first camera and the second camera at the same moment, the accuracy of the result is ensured, and the deviation of the result caused by the asynchronous image information is avoided.
Further: the step of calculating the coordinate conversion relationship between the first camera coordinate system and the calibration board coordinate system specifically comprises the following steps:
determining three-dimensional coordinates of the plurality of feature points in a first camera coordinate system according to the first image information and the second image information after primary cutting and the relative pose relationship;
establishing a seven-parameter conversion model according to the plurality of characteristic points and the three-dimensional coordinates thereof in the first camera coordinate system;
and calculating the coordinate conversion relation between the first camera coordinate system and the calibration board coordinate system according to the seven-parameter conversion model.
The beneficial effects of the further scheme are as follows: the coordinates of the feature points in the first image information and the second image information are respectively obtained through the first image information and the second image information after the first cutting, and then the three-dimensional coordinates of the feature points under the first camera coordinate system can be determined by combining the first image information and the second image information, so that the accurate positioning is realized.
Further: the method further comprises the steps of:
and recalibrating the relative pose relationship between the first camera and the second camera corresponding to each interval set period, and correcting the three-dimensional coordinate of the central point under the coordinate system of the first camera at the current moment according to the recalibrated relative pose relationship.
The beneficial effects of the further scheme are as follows: by recalibrating the relative pose relationship between the first camera and the second camera and correcting the three-dimensional coordinate of the central point under the coordinate system of the first camera at the current moment according to the recalibrated relative pose relationship, errors caused by the change of the relative pose between the first camera and the second camera can be eliminated, and the accuracy of the monitoring result is improved.
Further: the step of correcting the three-dimensional coordinate of the central point in the first camera coordinate system specifically comprises the following steps:
respectively acquiring real-time first image information and second image information shot by the first camera and the second camera at each set interval period;
recalibrating the relative pose relationship between the first camera and the second camera at the current moment according to the real-time first image information and the real-time second image information, and determining pose change information of the first camera according to the first image information at the current moment and the first image information at the initial moment;
and calculating the three-dimensional coordinate of the central point under the first camera coordinate system at the current moment according to the relative pose relationship between the first camera and the second camera at the current moment, and calculating the three-dimensional coordinate of the central point under the first camera coordinate system at the initial moment by combining the pose change information of the first camera.
The beneficial effects of the further scheme are as follows: the three-dimensional coordinate of the central point in the coordinate system of the first camera at the current moment can be adjusted to the three-dimensional coordinate in the coordinate system of the first camera at the initial moment through the re-calibrated relative pose relationship and the pose change information of the first camera, so that errors caused by the relative pose change between the first camera and the second camera can be eliminated, and the accuracy of the monitoring result is improved.
The invention also provides a bridge monitoring system based on binocular vision, which comprises a first camera, a second camera, a calibration plate and an industrial personal computer, wherein the first camera and the second camera are respectively arranged at positions away from the bridge by a set distance, and the calibration plate is arranged on the bridge;
the industrial personal computer is used for calibrating a first camera and a second camera which are away from the bridge by a set distance through a calibration plate arranged on the bridge, acquiring the relative pose relation between the first camera and the second camera according to calibration information, and determining the real-time three-dimensional coordinate of a central point on the calibration plate under a first camera coordinate system;
the industrial personal computer is used for calculating a coordinate conversion relation between the first camera coordinate system and a calibration plate coordinate system according to the calibration information and converting the three-dimensional coordinate of the central point under the first camera coordinate system into a real-time three-dimensional coordinate under the calibration plate coordinate system;
and the industrial personal computer is used for determining the displacement information of the bridge according to the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the current moment and the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the initial moment, and generating and displaying a displacement curve.
According to the bridge monitoring system based on binocular vision, the first camera and the second camera form a binocular vision system to monitor the bridge, the real-time three-dimensional coordinate of the central point on the calibration plate under the coordinate system of the first camera is determined by calibrating the relative pose relation between the first camera and the second camera, the three-dimensional coordinate is converted into the coordinate system of the calibration plate, the displacement information of the bridge is determined by combining the coordinate of the central point under the coordinate system of the calibration plate at the initial moment, manual measurement is replaced, the advantages of real time, high precision and low price of the binocular vision system are combined, multi-point simultaneous monitoring is conveniently achieved, and the monitoring efficiency is greatly improved.
Drawings
FIG. 1 is a schematic flow chart of a bridge monitoring method based on binocular vision according to the present invention;
FIG. 2 is a schematic plan view of a calibration plate of the present invention;
fig. 3 is a schematic structural view of the bridge monitoring system based on binocular vision.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
As shown in fig. 1, a bridge monitoring method based on binocular vision includes the following steps:
s11: calibrating a first camera and a second camera which are set to be a distance away from the bridge by matching with a calibration plate arranged on the bridge, acquiring a relative pose relation between the first camera and the second camera according to calibration information, and determining a real-time three-dimensional coordinate of a central point on the calibration plate under a first camera coordinate system;
s12: calculating a coordinate conversion relation between the first camera coordinate system and a calibration plate coordinate system according to the calibration information, and converting the three-dimensional coordinate of the central point under the first camera coordinate system into a real-time three-dimensional coordinate under the calibration plate coordinate system;
here, in the present invention, the first camera is selected as the center camera during calibration as an example, and in practice, the second camera may be selected as the center camera, which is not limited herein, and any one of the first camera and the second camera may be selected.
And S13, determining the displacement information of the bridge according to the three-dimensional coordinates of the central point under the coordinate system of the calibration board at the current moment and the initial moment (the initial moment in the invention is the moment of carrying out first calibration on the first camera and the second camera) and the three-dimensional coordinates of the central point under the coordinate system of the calibration board, and generating and displaying a displacement curve.
According to the bridge monitoring method based on binocular vision, a binocular vision system consisting of the first camera and the second camera monitors the bridge, real-time three-dimensional coordinates of a central point on a calibration plate under a first camera coordinate system are determined by calibrating the relative pose relationship between the first camera and the second camera, the three-dimensional coordinates are converted into a calibration plate coordinate system, displacement information of the bridge is determined by combining coordinates of the central point under the calibration plate coordinate system at the initial moment, manual measurement is replaced, the advantages of real-time performance, high precision and low price of the binocular vision system are combined, multi-point simultaneous monitoring is conveniently achieved, and monitoring efficiency is greatly improved.
In one or more embodiments provided by the present invention, the calibrating the first camera and the second camera at the set distance from the bridge specifically includes the following steps:
s21: synchronously shooting a calibration plate arranged on the bridge through a first camera and a second camera which are away from the bridge by a set distance to obtain first image information and second image information;
in practice, a calibration plate printed with 25 black squares (as shown in fig. 2, the size of the middle square is relatively large, and the sizes of the other squares are the same and relatively small) is fixed on a bridge, then a ground surface approximately 4 meters away from the calibration plate is selected to fix the first camera and the second camera, the first camera and the second camera are separated by a certain distance, the shooting directions of the two cameras are approximately 90 degrees, then the shooting angle of the cameras is adjusted to enable the cameras to shoot the calibration plate, and then the aperture and the focal distance are adjusted to enable the brightness of the image of the calibration plate to be moderate and the image to be clear. Then, the shooting range of the camera is set to a range including the calibration plate and having an area about twice as large as the size of the area of the calibration plate. The first and second cameras may be secured with a viewing pier.
S22: respectively identifying the first image information and the second image information, and respectively acquiring a plurality of feature points and corresponding pixel coordinates from the first image information and the second image information;
after the first camera and the second camera acquire images, the images are acquired and decoded by using a plurality of threads, and because the images are acquired by multiple threads, the images in the image queue are not stored in sequence.
S23: respectively sequencing and matching a plurality of feature points in the first image information and the second image information, and determining a relative pose relation between the first camera and the second camera according to pixel coordinates of a plurality of matched pairs of feature points;
wherein the calibration information comprises: the image processing device comprises first image information, second image information, a plurality of feature points and corresponding pixel coordinates in the first image information, and a plurality of feature points and corresponding pixel coordinates in the second image information.
The first camera and the second camera respectively acquire first image information and second image information, and extract feature points (the central point of each square, including the central point of the central square) for sequencing and matching, so that the relative pose relationship between the first camera and the second camera can be accurately determined according to pixel coordinates of a plurality of matched pairs of feature points, the data processing amount is small, the efficiency is greatly improved, and the method can completely adapt to practical application scenes.
Taking the calibration board shown in fig. 2 as an example, in practice, the world coordinates of the center points of the predetermined 25 squares are respectively set as: (0, 0, 0), (1 × 18.8, 0, 0), (2 × 18.8, 0, 0), (3 × 18.8, 0, 0), (4 × 18.8, 0, 0), (0, 1 × 18.8, 0), (1 × 18.8, 1 × 18.8, 0), (2 × 18.8, 1 × 18.8, 0), (3 × 18.8, 1 × 18.8, 0), (4 × 18.8, 1 × 18.8, 0), (0, 2 × 18.8, 0), (1 × 18.8, 2 × 18.8, 0), (2 × 18.8, 2 × 18.8, 0), (3 × 18.8, 2 × 18.8, 0), (4 × 18.8, 2 × 18.8, 0), (0, 3 × 18.8, 0), (1 × 18.8, 3 × 18.8, 0), (2 × 18.8, 3 × 18.8, 0), (3 × 18.8 ), (3 × 18 × 18.8, 0), (8 × 18.8, 8), (8, 18.8, 8, 0), (4, 18.8, 18, 8, 0), (8, 18, 8, 0), (4, 8, 4 × 18.8, 0), (4 × 18.8, 4 × 18.8, 0); the internal parameters and the external parameters of the two cameras can be determined by combining the world coordinates of the center points of the 25 squares and the corresponding pixel coordinates, namely the relative pose relationship between the two cameras, and then the three-dimensional coordinates of the points in the first camera coordinate system can be calculated by using the pixel coordinates in the calibration image information of the two cameras corresponding to one characteristic point in the world coordinate system and combining the relative pose relationship between the two cameras.
Preferably, in one or more embodiments provided by the present invention, in the step S21, after the obtaining the first image information and the second image information, the method further includes the following steps:
and respectively carrying out primary cutting on the first image information and the second image information by taking the central point of the calibration plate as a center, and respectively carrying out identification processing on the first image information and the second image information after the primary cutting.
By respectively carrying out primary cutting on the first image information and the second image information, the image size can be reduced as much as possible, corresponding feature points can be found after the calibration plate is displaced, the calculation speed is improved, and the memory consumption is reduced. Here, since the center points of all the squares are required to be used as the feature points when the calibration is performed, when the first image information and the second image information are subjected to the initial trimming, the first image information and the second image information are subjected to the initial trimming with the center point of the calibration plate (the center point of the center square) as the center to have an area twice as large as the area of the calibration plate, so that the image size can be reduced, it is also possible to ensure that all 25 squares are located in the first image information and the second image information, and the center points of all the squares can be accurately found when the calibration plate is slightly displaced.
In one or more embodiments provided by the present invention, the determining the real-time three-dimensional coordinates of the central point on the calibration board in the first camera coordinate system specifically includes:
and determining the real-time three-dimensional coordinate of the central point on the calibration plate under the first camera coordinate system according to the first image information and the second image information which are real-time after the initial cutting and the relative pose relationship.
The coordinates of the central point in the first image information and the second image information are respectively obtained through the first image information and the second image information which are real-time after the first cutting, and then the real-time three-dimensional coordinates of the central point under the coordinate system of the first camera can be determined by combining the first image information and the second image information, so that the accurate positioning is realized.
Preferably, in one or more embodiments provided by the present invention, in step S11, before determining the three-dimensional coordinates of the center point on the calibration board in the first camera coordinate system, the method further includes the following steps:
and respectively secondarily cutting the first image information and the second image information after the primary cutting by taking the central point of the calibration plate as a center, and determining the three-dimensional coordinate of the central point on the calibration plate under a first camera coordinate system according to the combination of the first image information and the second image information after the secondary cutting and the relative pose relation.
Through after will first cutting first image information and second image information carry out the secondary and cut out, both can further dwindle first image information and second image information can guarantee again to find the central point after the calibration plate takes place the displacement, improve computational rate, reduce and consume the memory.
Here, after the calibration is completed, the center points of the squares other than the center square are not required to be operated as feature points, so when the first image information and the second image information are subjected to secondary cropping, the first image information and the second image information are subjected to secondary cropping with the center point of the calibration plate (the center point of the center square) as the center to be twice as large as the area of the center square, which can further reduce the image size, can also ensure that the center squares are both located in the first image information and the second image information, and can accurately find the center points of all the center squares (i.e., the center points of the first image information and the second image information) when the calibration plate is slightly displaced.
In the actual measurement process, the size of the calibration block in the image information is basically unchanged, so the size of the maximum block (central block) is marked during calibration, and when the maximum block is detected after calibration, the block with the size too large different from that of the calibration time is filtered, so that the error detection can be avoided.
In one or more embodiments provided by the present invention, the first camera and the second camera synchronously shoot the calibration plates arranged on the bridge, and the specific implementation of acquiring the first image information and the second image information is as follows:
s31: taking a flash light signal of the first camera as a shooting trigger signal of the second camera, and caching image information shot by the first camera and the second camera;
s32: and selecting two pieces of image information with the same shooting time from the image information cached by the first camera and the second camera respectively as the first image information and the second image information.
The flash light signal of the first camera is used as the shooting trigger signal of the second camera, so that the first camera and the second camera can be kept completely synchronous during shooting, and two pieces of image information with the same shooting time are selected from the buffered image information after shooting, so that the first image information and the second image information used for determining the relative pose relationship between the first camera and the second camera can be images shot by the first camera and the second camera at the same moment, the accuracy of the result is ensured, and the deviation of the result caused by the asynchronous image information is avoided. In practice, image information data of two seconds are cached, and calibration image information with the same shooting time of two cameras is selected as a synchronous image from the previous second of the cached data every time, an image in which the synchronous image is not found is reserved for the next second, and if the synchronous image is not found in the next second, the image is directly deleted.
In one or more embodiments provided in the present invention, the calculating a coordinate transformation relationship between the first camera coordinate system and the calibration board coordinate system specifically includes the following steps:
s41: determining three-dimensional coordinates of the plurality of feature points in a first camera coordinate system according to the first image information and the second image information after primary cutting and the relative pose relationship;
s42: establishing a seven-parameter conversion model according to the plurality of characteristic points and the three-dimensional coordinates thereof in the first camera coordinate system;
s43: and calculating the coordinate conversion relation between the first camera coordinate system and the calibration board coordinate system according to the seven-parameter conversion model.
The coordinates of the feature points in the first image information and the second image information are respectively obtained through the first image information and the second image information after the first cutting, and then the three-dimensional coordinates of the feature points under the first camera coordinate system can be determined by combining the first image information and the second image information, so that the accurate positioning is realized.
In one or more embodiments provided herein, the method further includes the steps of:
s14: and recalibrating the relative pose relationship between the first camera and the second camera corresponding to each interval set period, and correcting the three-dimensional coordinate of the central point under the coordinate system of the first camera at the current moment according to the recalibrated relative pose relationship.
By recalibrating the relative pose relationship between the first camera and the second camera and correcting the three-dimensional coordinate of the central point under the coordinate system of the first camera at the current moment according to the recalibrated relative pose relationship, errors caused by the change of the relative pose between the first camera and the second camera can be eliminated, and the accuracy of the monitoring result is improved.
Specifically, in one or more embodiments provided in the present invention, the correcting the three-dimensional coordinates of the central point in the first camera coordinate system specifically includes the following steps:
s51: respectively acquiring real-time first image information and second image information shot by the first camera and the second camera at each set interval period;
s52: recalibrating the relative pose relationship between the first camera and the second camera at the current moment according to the real-time first image information and the real-time second image information, and determining pose change information of the first camera according to the first image information at the current moment and the first image information at the initial moment;
s53: and calculating the three-dimensional coordinate of the central point under the first camera coordinate system at the current moment according to the relative pose relationship between the first camera and the second camera at the current moment, and calculating the three-dimensional coordinate of the central point under the first camera coordinate system at the initial moment by combining the pose change information of the first camera.
The three-dimensional coordinate of the central point in the coordinate system of the first camera at the current moment can be adjusted to the three-dimensional coordinate in the coordinate system of the first camera at the initial moment through the re-calibrated relative pose relationship and the pose change information of the first camera, so that errors caused by the relative pose change between the first camera and the second camera can be eliminated, and the accuracy of the monitoring result is improved.
As shown in fig. 3, the invention further provides a bridge monitoring system based on binocular vision, which comprises a first camera, a second camera, a calibration plate and an industrial personal computer, wherein the first camera and the second camera are respectively arranged at positions away from the bridge by a set distance, and the calibration plate is arranged on the bridge;
the industrial personal computer is used for calibrating a first camera and a second camera which are away from the bridge by a set distance through a calibration plate arranged on the bridge, acquiring the relative pose relation between the first camera and the second camera according to calibration information, and determining the real-time three-dimensional coordinate of a central point on the calibration plate under a first camera coordinate system;
the industrial personal computer is used for calculating a coordinate conversion relation between the first camera coordinate system and a calibration plate coordinate system according to the calibration information and converting the three-dimensional coordinate of the central point under the first camera coordinate system into a real-time three-dimensional coordinate under the calibration plate coordinate system;
and the industrial personal computer is used for determining the displacement information of the bridge according to the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the current moment and the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the initial moment, and generating and displaying a displacement curve.
According to the bridge monitoring system based on binocular vision, the first camera and the second camera form a binocular vision system to monitor the bridge, the real-time three-dimensional coordinate of the central point on the calibration plate under the coordinate system of the first camera is determined by calibrating the relative pose relation between the first camera and the second camera, the three-dimensional coordinate is converted into the coordinate system of the calibration plate, the displacement information of the bridge is determined by combining the coordinate of the central point under the coordinate system of the calibration plate at the initial moment, manual measurement is replaced, the advantages of real time, high precision and low price of the binocular vision system are combined, multi-point simultaneous monitoring is conveniently achieved, and the monitoring efficiency is greatly improved.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A bridge monitoring method based on binocular vision is characterized by comprising the following steps:
calibrating a first camera and a second camera which are set to be a distance away from the bridge by matching with a calibration plate arranged on the bridge, acquiring a relative pose relation between the first camera and the second camera according to calibration information, and determining a real-time three-dimensional coordinate of a central point on the calibration plate under a first camera coordinate system;
calculating a coordinate conversion relation between the first camera coordinate system and a calibration plate coordinate system according to the calibration information, and converting the real-time three-dimensional coordinate of the central point under the first camera coordinate system into the real-time three-dimensional coordinate under the calibration plate coordinate system;
and determining the displacement information of the bridge according to the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the current moment and the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the initial moment, and generating and displaying a displacement curve.
2. The binocular vision-based bridge monitoring method according to claim 1, wherein the step of calibrating the first camera and the second camera at a set distance from the bridge specifically comprises the steps of:
synchronously shooting a calibration plate arranged on the bridge through a first camera and a second camera which are away from the bridge by a set distance to obtain first image information and second image information;
respectively identifying the first image information and the second image information, and respectively acquiring a plurality of feature points and corresponding pixel coordinates from the first image information and the second image information;
respectively sequencing and matching a plurality of feature points in the first image information and the second image information, and determining a relative pose relation between the first camera and the second camera according to pixel coordinates of a plurality of matched pairs of feature points;
wherein the calibration information comprises: the image processing device comprises first image information, second image information, a plurality of feature points and corresponding pixel coordinates in the first image information, and a plurality of feature points and corresponding pixel coordinates in the second image information.
3. The binocular vision based bridge monitoring method according to claim 2, wherein after the first image information and the second image information are acquired, the method further comprises the following steps:
and respectively carrying out primary cutting on the first image information and the second image information by taking the central point of the calibration plate as a center, and respectively carrying out identification processing on the first image information and the second image information after the primary cutting.
4. The binocular vision based bridge monitoring method of claim 3, wherein the determining of the real-time three-dimensional coordinates of the center point on the calibration plate in the first camera coordinate system specifically comprises:
and determining the real-time three-dimensional coordinate of the central point on the calibration plate under the first camera coordinate system according to the first image information and the second image information which are real-time after the initial cutting and the relative pose relationship.
5. The binocular vision based bridge monitoring method of claim 4, further comprising, before determining the three-dimensional coordinates of the center point on the calibration plate in the first camera coordinate system, the steps of:
and respectively secondarily cutting the first image information and the second image information which are real-time after the primary cutting by taking the central point of the calibration plate as a center, and determining the real-time three-dimensional coordinate of the central point on the calibration plate under the coordinate system of the first camera according to the combination of the first image information and the second image information which are real-time after the secondary cutting and the relative pose relationship.
6. The binocular vision based bridge monitoring method according to claim 2, wherein the first camera and the second camera synchronously shoot calibration plates arranged on the bridge respectively, and the specific implementation of acquiring the first image information and the second image information is as follows:
taking a flash light signal of the first camera as a shooting trigger signal of the second camera, and caching image information shot by the first camera and the second camera;
and selecting two pieces of image information with the same shooting time from the image information cached by the first camera and the second camera respectively as the first image information and the second image information.
7. The binocular vision-based bridge monitoring method of claim 4, wherein the calculating of the coordinate transformation relationship between the first camera coordinate system and the calibration plate coordinate system specifically comprises the steps of:
determining three-dimensional coordinates of the plurality of feature points in a first camera coordinate system according to the first image information and the second image information after primary cutting and the relative pose relationship;
establishing a seven-parameter conversion model according to the plurality of characteristic points and the three-dimensional coordinates thereof in the first camera coordinate system;
and calculating the coordinate conversion relation between the first camera coordinate system and the calibration board coordinate system according to the seven-parameter conversion model.
8. The binocular vision based bridge monitoring method of claim 2, further comprising the steps of:
and recalibrating the relative pose relationship between the first camera and the second camera corresponding to each interval set period, and correcting the three-dimensional coordinate of the central point under the coordinate system of the first camera at the current moment according to the recalibrated relative pose relationship.
9. The binocular vision based bridge monitoring method of claim 8, wherein the correcting the three-dimensional coordinates of the center point in the first camera coordinate system specifically comprises the steps of:
respectively acquiring real-time first image information and second image information shot by the first camera and the second camera at each set interval period;
recalibrating the relative pose relationship between the first camera and the second camera at the current moment according to the real-time first image information and the real-time second image information, and determining pose change information of the first camera according to the first image information at the current moment and the first image information at the initial moment;
and calculating the three-dimensional coordinate of the central point under the first camera coordinate system at the current moment according to the relative pose relationship between the first camera and the second camera at the current moment, and calculating the three-dimensional coordinate of the central point under the first camera coordinate system at the initial moment by combining the pose change information of the first camera.
10. A bridge monitoring system based on binocular vision is characterized by comprising a first camera, a second camera, a calibration plate and an industrial personal computer, wherein the first camera and the second camera are respectively arranged at positions away from a bridge by a set distance, and the calibration plate is arranged on the bridge;
the industrial personal computer is used for calibrating a first camera and a second camera which are away from the bridge by a set distance through a calibration plate arranged on the bridge, acquiring the relative pose relation between the first camera and the second camera according to calibration information, and determining the real-time three-dimensional coordinate of a central point on the calibration plate under a first camera coordinate system;
the industrial personal computer is used for calculating a coordinate conversion relation between the first camera coordinate system and a calibration plate coordinate system according to the calibration information and converting the three-dimensional coordinate of the central point under the first camera coordinate system into a real-time three-dimensional coordinate under the calibration plate coordinate system;
and the industrial personal computer is used for determining the displacement information of the bridge according to the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the current moment and the three-dimensional coordinate of the central point under the coordinate system of the calibration plate at the initial moment, and generating and displaying a displacement curve.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911159173.7A CN111079550A (en) | 2019-11-22 | 2019-11-22 | Bridge monitoring method and system based on binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911159173.7A CN111079550A (en) | 2019-11-22 | 2019-11-22 | Bridge monitoring method and system based on binocular vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111079550A true CN111079550A (en) | 2020-04-28 |
Family
ID=70311391
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911159173.7A Pending CN111079550A (en) | 2019-11-22 | 2019-11-22 | Bridge monitoring method and system based on binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111079550A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111551151A (en) * | 2020-06-04 | 2020-08-18 | 江苏集萃智能光电系统研究所有限公司 | Binocular vision-based near space vehicle relative pose measurement method and device |
| CN113192063A (en) * | 2021-05-25 | 2021-07-30 | 中铁第四勘察设计院集团有限公司 | Bridge linear monitoring system and bridge linear monitoring method |
| CN113781548A (en) * | 2020-06-10 | 2021-12-10 | 华为技术有限公司 | Multi-device pose measurement method, electronic device and system |
| CN114266835A (en) * | 2021-12-27 | 2022-04-01 | 深圳供电局有限公司 | Deformation monitoring control method and system for non-measuring camera |
| US12073071B2 (en) | 2020-07-29 | 2024-08-27 | Huawei Technologies Co., Ltd. | Cross-device object drag method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106840011A (en) * | 2015-12-03 | 2017-06-13 | 北京国网富达科技发展有限责任公司 | Steel tower deformation measuring device and its method |
| CN108398091A (en) * | 2018-02-06 | 2018-08-14 | 山东理工大学 | The accurate vision measuring method of blade tip three-D displacement amount |
| CN109186902A (en) * | 2018-09-26 | 2019-01-11 | 中国计量大学 | A kind of bridge structure health detection system of view-based access control model sensing |
| CN109813509A (en) * | 2019-01-14 | 2019-05-28 | 中山大学 | The method that high-speed rail bridge vertically moves degree of disturbing measurement is realized based on unmanned plane |
-
2019
- 2019-11-22 CN CN201911159173.7A patent/CN111079550A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106840011A (en) * | 2015-12-03 | 2017-06-13 | 北京国网富达科技发展有限责任公司 | Steel tower deformation measuring device and its method |
| CN108398091A (en) * | 2018-02-06 | 2018-08-14 | 山东理工大学 | The accurate vision measuring method of blade tip three-D displacement amount |
| CN109186902A (en) * | 2018-09-26 | 2019-01-11 | 中国计量大学 | A kind of bridge structure health detection system of view-based access control model sensing |
| CN109813509A (en) * | 2019-01-14 | 2019-05-28 | 中山大学 | The method that high-speed rail bridge vertically moves degree of disturbing measurement is realized based on unmanned plane |
Non-Patent Citations (1)
| Title |
|---|
| 贾程宏 等: "三维摄影测量技术在土工离心模型试验中的应用" * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111551151A (en) * | 2020-06-04 | 2020-08-18 | 江苏集萃智能光电系统研究所有限公司 | Binocular vision-based near space vehicle relative pose measurement method and device |
| CN113781548A (en) * | 2020-06-10 | 2021-12-10 | 华为技术有限公司 | Multi-device pose measurement method, electronic device and system |
| US12073071B2 (en) | 2020-07-29 | 2024-08-27 | Huawei Technologies Co., Ltd. | Cross-device object drag method and device |
| CN113192063A (en) * | 2021-05-25 | 2021-07-30 | 中铁第四勘察设计院集团有限公司 | Bridge linear monitoring system and bridge linear monitoring method |
| CN113192063B (en) * | 2021-05-25 | 2024-02-02 | 中铁第四勘察设计院集团有限公司 | Bridge line-shaped monitoring system and bridge line-shaped monitoring method |
| CN114266835A (en) * | 2021-12-27 | 2022-04-01 | 深圳供电局有限公司 | Deformation monitoring control method and system for non-measuring camera |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111079550A (en) | Bridge monitoring method and system based on binocular vision | |
| CN111179358B (en) | Calibration method, device, equipment and storage medium | |
| US10690492B2 (en) | Structural light parameter calibration device and method based on front-coating plane mirror | |
| US11024052B2 (en) | Stereo camera and height acquisition method thereof and height acquisition system | |
| CN108510551B (en) | Method and system for calibrating camera parameters under long-distance large-field-of-view condition | |
| CN102798350B (en) | Method, device and system for measuring deflection of arm support | |
| CN104173054A (en) | Measuring method and measuring device for height of human body based on binocular vision technique | |
| WO2011125937A1 (en) | Calibration data selection device, method of selection, selection program, and three dimensional position measuring device | |
| CN105953741B (en) | System and method for measuring local geometric deformation of steel structure | |
| CN111288967A (en) | Remote high-precision displacement detection method based on machine vision | |
| CN112284260A (en) | Visual displacement monitoring method, equipment and system | |
| KR100935898B1 (en) | Automatic displacement detecting apparatus and method, and slop maintenance and management system and method using the same | |
| CN107092905B (en) | Method for positioning instrument to be identified of power inspection robot | |
| CN115588040A (en) | System and method for counting and positioning coordinates based on full-view imaging points | |
| TW200817651A (en) | Distance measurement system and method | |
| CN110749311B (en) | Positioning method, positioning device and storage medium | |
| CN210664375U (en) | Antiaircraft gun detection device | |
| KR20140024745A (en) | System and method for calibrating of object for measuring deformation structure | |
| CN110736438A (en) | antiaircraft gun detection device and detection method thereof | |
| CN114754695B (en) | Multi-view-field bridge deflection measuring device and method and storage medium | |
| KR101459522B1 (en) | Location Correction Method Using Additional Information of Mobile Instrument | |
| CN115683046A (en) | Distance measuring method, distance measuring device, sensor and computer readable storage medium | |
| CN116202423A (en) | Line laser two-dimensional positioning method based on laser triangulation ranging | |
| CN113112551A (en) | Camera parameter determination method and device, road side equipment and cloud control platform | |
| CN112883809A (en) | Target detection method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200428 |