Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The camera-based power transmission line distance measurement method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein two binocular cameras 102 on the power transmission line upload captured images to a server 104. The data storage system may store image data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The server 104 obtains the position of the construction machine vertex based on the photographed images of the two binocular cameras, and the included angle between the line connecting the optical center of each binocular camera and the construction machine vertex and the optical axis of the binocular camera; and determining the distance between the position of the vertex of the construction machine and the power transmission line according to the relative distance and the included angle between the two binocular cameras. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
In one embodiment, as shown in fig. 2, a method for measuring distance of a power transmission line based on a camera is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
Step 202, after calibrating each binocular camera respectively, respectively acquiring a picture depth map of each binocular camera based on a parallax relation;
The binocular camera comprises two monocular cameras, the two monocular cameras are respectively arranged on the left side and the right side of the binocular camera body, and when the positions of the construction machine vertexes are obtained, the positions of the construction machine vertexes can be obtained based on the images shot by the monocular cameras on the same side in the two monocular cameras.
The two binocular cameras are respectively arranged on the spacing bars, the spacing bars are used for connecting the wires together, and the two binocular cameras are arranged on the two spacing bars, so that the distance between the vertex of the construction machine under the power transmission line and the power transmission line between the two spacing bars can be detected. As shown in fig. 3, two binocular cameras are respectively installed on the power transmission lines. The method comprises the steps of acquiring shooting images of monocular cameras on the same side of two binocular cameras on a power transmission line, respectively identifying the positions of construction machinery in detection frames of the two binocular cameras based on an artificial intelligence algorithm acquisition picture identification method, wherein an image area where a construction machinery body is located is a detection frame corresponding to the binocular cameras, and acquiring a picture depth map of an area where the construction machinery is located in the detection frame corresponding to the two binocular cameras based on a parallax relation of the binocular cameras.
The parallax relation between the binocular camera and the object is shown in fig. 4. P is an object in front of the binocular camera, P L、PR is a point of the object on the imaging plane of the camera, O L、OR is the optical center of the left and right cameras, F is the focal length of the camera, D is the base line of the binocular camera, X R、XL is the distance from the imaging point of the object on the imaging plane to the leftmost end of the imaging plane, Z is the distance from the object to the binocular camera, and L is the width value of the imaging plane.
The method can be obtained according to the proportion relation:
From formula (1):
After the binocular camera is calibrated, the camera focal length F and the binocular camera base line D are of known quantity and are fixed, X R、XL can be obtained through calculation, the distance from the P point to the binocular camera can be obtained through the formula (2), and then the picture depth map can be obtained.
Specifically, as shown in fig. 5, the photographed images of two binocular cameras on the power transmission line are obtained, the construction machine is marked in the photographed images, image preprocessing is performed after marking, the preprocessed images are input into a neural network model, the weight of the neural network model is initialized, a loss function is calculated after forward training, when the precision reaches the expected value, the final weight is obtained, the position of an image area where the construction machine body is located is identified based on the final weight and a target detection algorithm, the image area where the construction machine body is located is a detection frame corresponding to the binocular camera, and the image depth map of each binocular camera is obtained based on the parallax relation of the binocular camera.
Step 204, performing binarization processing on the picture depth map corresponding to each binocular camera based on a preset binarization threshold value to obtain a binarized image, traversing each row of pixel points of the binarized image corresponding to the detection frame of each binocular camera from top to bottom in sequence, and determining the vertex of the construction machine based on the pixel points corresponding to the target row in each binarized image; the detection frame corresponding to the binocular camera is an image area where the construction machine body is located; the target line includes a pixel point whose first-appearing pixel value is a first value.
The relative positions of the two binocular cameras on the power transmission line and the construction machine are shown in fig. 3, and according to prior knowledge of the positions, the vertex of the construction arm of the construction machine approaches to the position above the detection frame in the detection frame of the binocular camera, so that all image data of the binary image do not need to be traversed during traversal, only each row of pixel points of the binary image corresponding to the detection frame of each binocular camera need to be traversed from top to bottom until all pixel points corresponding to the target row in the binary image corresponding to the detection frame are traversed.
The maximum inter-class variance method is adopted to calculate the binarization threshold, and the concrete calculation method comprises the following steps: setting the size of the picture depth map as M x N, the binarization initial threshold as T, the number of pixels with the pixel depth value smaller than T in the picture depth map as N 0, the average gray scale as mu 0, the number of pixels with the pixel depth value larger than T in the image as N 1, the average gray scale as mu 1, the proportion of N 0 to the total number of pixels as w 0,The ratio of N 1 to the total number of pixels is w 1,The total average gray level is mu, mu=w 0×μ0+w1×μ1, the inter-class variance is g, g=w 0(μ0-μ)2+w1(μ1-μ)2, the inter-class variance obtained after simplification is g=w 0×w1(μ0-μ1), and traversing is carried out in the range of [0, 255] to obtain a threshold T which enables the inter-class variance g to be maximum as a preset binarization threshold.
And carrying out binarization processing on the picture depth map corresponding to each binocular camera based on the following formula:
wherein F (i, j) is a binarized pixel value, gray (i, j) is an original depth value of the picture depth map, and Gray th is a binarized final threshold value for maximizing the inter-class variance g. The image depth map is subjected to binarization processing through the formula to obtain a binarized image, the pixel point with the binarized pixel value of 1 is defined as the pixel point of the construction machine, and the pixel point with the binarized pixel value of 0 is defined as the pixel point of the background image. The image data of the binarized image is a multi-dimensional array, the rows and columns of the multi-dimensional array represent the two-dimensional coordinates of the pixel points corresponding to the array, and the elements of the multi-dimensional array are the depth values of the pixel points corresponding to the array. Traversing each row of pixel points of the binarized image corresponding to the detection frame of each binocular camera from top to bottom in sequence to obtain the pixel points corresponding to the target row in the binarized image corresponding to each detection frame, wherein the detection frame corresponding to the binocular camera is an image area where the construction machine body is located; the target row comprises pixel points with first-appearing pixel values as first values, the first values represent the pixel values as 1, and the target row represents a row corresponding to the pixel points with the first-appearing pixel values as 1. If the target row has only one pixel point with the pixel value of 1, judging the pixel point as the construction machine vertex; if the target line has a plurality of pixel points with the pixel value of 1, determining an intermediate point based on an average value among the plurality of pixel points with the pixel value of 1 in the target line, and determining the intermediate point as the construction machine vertex. For example, if the target line has 5 pixels with 1 pixel value, the middle point is the third pixel with 1 pixel value, that is, the third pixel with 1 pixel value is the construction machine vertex.
Specifically, taking the boom of the construction machine tilting leftwards as an example, calculating a binarization threshold value by adopting a maximum inter-class variance method, carrying out binarization processing on a picture depth image corresponding to each binocular camera based on the binarization threshold value to obtain a binarization image, defining a pixel point with a pixel value of 1 after binarization as a pixel point of the construction machine, defining a pixel point with a pixel value of 0 after binarization as a pixel point of a background image, traversing each row of pixel points of the binarization image corresponding to a detection frame of each binocular camera from top to bottom in sequence, and obtaining a pixel point corresponding to a target row in each binarization image corresponding to the detection frame, wherein the target row represents a row corresponding to the pixel point with the pixel value of 1 appearing for the first time. If the target row has only one pixel point with the pixel value of 1, judging the pixel point as the construction machine vertex; if the target line has a plurality of pixel points with the pixel value of 1, determining an intermediate point based on an average value among the plurality of pixel points with the pixel value of 1 in the target line, and determining the intermediate point as the construction machine vertex. The construction machine vertices corresponding to the two binocular cameras obtained by the vertex determination method are a left vertex and a right vertex of the highest portion of the boom, and the left vertex and the right vertex may not be the same vertex.
And 206, respectively acquiring the included angles between the connecting line of the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera according to the photographed images of the two binocular cameras.
The optical center of the binocular camera is the optical center of a monocular camera in the binocular camera. As shown in fig. 3, the optical axes of the two binocular cameras are approximately parallel to the power lines. Because the power transmission line is approximately parallel to the ground, the optical axes of the two binocular cameras can be approximately considered to be coincident, and the connecting line between the two binocular cameras and the vertex of the construction machine can form a triangle.
Specifically, according to the photographed images of the monocular cameras on the same side in the two binocular cameras, the connection line between the optical center of the first binocular camera and the vertex of the construction machine, the included angle between the optical axis of the first binocular camera, the connection line between the optical center of the second binocular camera and the vertex of the construction machine, and the included angle between the optical center of the second binocular camera and the optical axis of the second binocular camera are respectively obtained, namely, the angle delta 1 and the angle delta 2 in fig. 3 are respectively obtained.
And step 208, determining the distance between the position of the vertex of the construction machine and the power transmission line according to the relative distance and the included angle between the two binocular cameras.
Wherein the relative distance between the two binocular cameras is approximately the length of the power transmission line between the two binocular cameras.
In one embodiment, the binocular camera includes a Real-time differential positioning (Real-TIME KINEMATIC, RTK) module and a binocular RGB camera, the RTK module being used to detect Real-time position coordinates of the binocular camera. The relative distance of the two binocular cameras is equal to the real-time position coordinate difference of the two binocular cameras parallel to the ground.
The construction machine vertex is connected with the optical centers of the two binocular cameras, and the relative distance between the two binocular cameras forms a triangle, and the included angle between the connection line of the optical center of each binocular camera and the construction machine vertex and the optical axis of the binocular camera is two angles of the triangle.
Specifically, as shown in fig. 3, the position coordinates of the two binocular cameras are respectively obtained, and the relative distance between the two binocular cameras, namely the connection line a in fig. 3, is obtained based on the position coordinates of the two binocular cameras; respectively acquiring the connecting line of the optical center of the first binocular camera and the vertex of the construction machine, and the included angle between the connecting line of the optical center of the second binocular camera and the vertex of the construction machine and the connecting line of the optical center of the second binocular camera and the optical axis of the second binocular camera, namely respectively acquiring the delta 1 angle and the delta 2 angle in fig. 3; and determining the distance between the position of the vertex of the construction machine and the power transmission line based on the trigonometric function relation, the relative distance between the two binocular cameras, and the included angle between the connecting line of the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera.
In the above-mentioned power transmission line ranging method based on the camera, on one hand, each row of pixel points of the binary image corresponding to the detection frame of each binocular camera is traversed from top to bottom in turn, and the construction machinery vertex is determined based on the pixel point corresponding to the target row in each binary image; the detection frame corresponding to the binocular camera is an image area where the construction machine body is located; the target row comprises pixel points with first-appearing pixel values as first values; the amount of calculation can be reduced and the obtained vertex position is accurate. On the other hand, the optical center connecting line of the construction machine vertex and the two binocular cameras and the relative distance between the two binocular cameras form a triangle, the relative distance between the two binocular cameras and the included angle between the connecting line of the optical center of each binocular camera and the construction machine vertex and the optical axis of the binocular camera are obtained, the distance between the position of the construction machine vertex and the power transmission line is determined based on the trigonometric function relation, the relative distance between the two binocular cameras and the included angle, and the distance between the construction machine vertex and the power transmission line is calculated based on the relative angle between the construction machine vertex and the binocular cameras and the accurate position coordinates of the binocular cameras.
In one embodiment, as shown in fig. 6, according to the photographed images of the two binocular cameras, the included angles between the connection line between the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera are respectively obtained, and the method includes the following steps:
step 602, obtaining a first relative angle between an optical center connecting line of the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the optical axis of the first binocular camera.
The optical center of the first binocular camera in this embodiment is the optical center of the right monocular camera in the first binocular camera, and the optical axis of the first binocular camera is the optical axis of the right monocular camera in the first binocular camera.
Step 604, obtaining a second relative angle between the vertex of the construction machine in the detection frame corresponding to the second binocular camera and the optical center connecting line of the second binocular camera, and the optical axis of the second binocular camera.
The optical center of the second binocular camera in this embodiment is the optical center of the right monocular camera in the second binocular camera, and the optical axis of the second binocular camera is the optical axis of the right monocular camera in the second binocular camera.
Step 606, determining a distance between the position of the vertex of the construction machine and the power transmission line according to the relative distance and the included angle between the two binocular cameras, including:
The distance between the position of the construction machine vertex and the power line is obtained based on the relative distance, the first relative angle, and the second relative angle. The construction machine comprises a construction machine vertex, two binocular cameras, a construction machine, a first relative angle and a second relative angle, wherein the construction machine vertex is connected with optical centers of the two binocular cameras, the relative distance between the two binocular cameras forms a triangle, the relative distance between the two binocular cameras is the base of the triangle, and the first relative angle and the second relative angle are two angles of the triangle respectively.
Specifically, a relative distance between two binocular cameras, a first relative angle and a second relative angle are obtained, and a distance between the position of the vertex of the construction machine and the power transmission line is determined based on a trigonometric function relation, the relative distance between the two binocular cameras, the first relative angle and the second relative angle.
In one embodiment, the method of acquiring the first relative angle is the same as the method of acquiring the second relative angle, and therefore, only the method of acquiring the first relative angle will be described herein.
In one embodiment, as shown in fig. 7, a first relative angle between a vertex of the construction machine in the detection frame corresponding to the first binocular camera and an optical center line of the first binocular camera, and an optical axis of the first binocular camera is obtained, including the following steps:
Step 702, obtaining a first linear distance between a vertex of the construction machine in the detection frame corresponding to the first binocular camera and a center of the detection frame corresponding to the first binocular camera.
The imaging schematic diagrams of the construction machine are shown in fig. 8 and 9, wherein K represents the vertex of the construction machine in the detection frame corresponding to the first binocular camera; o' represents the optical center of the right monocular camera in the first binocular camera; o represents the center of a detection frame corresponding to the first binocular camera; OO' denotes an optical axis, a length value of which is equal to a focal length value of the first binocular camera; OK represents a first linear distance between the in-frame construction machine vertex K corresponding to the first binocular camera and the center O of the detection frame corresponding to the first binocular camera.
Step 704, obtaining a tangent value of the first relative angle based on the first linear distance and the focal length value of the first binocular camera.
Wherein, the focal length value of the first binocular camera is equal to the length value of the optical axis OO'. The ratio of the first linear distance OK to the focal length value of the first binocular camera is the tangent of the first relative angle. The mathematical expression of the tangent of the first relative angle is:
Step 706, obtaining a first relative angle based on the arctangent function and a tangent value of the first relative angle.
Wherein the arctangent function of the tangent of the first relative angle is the first relative angle. The mathematical expression of the first relative angle is:
In this embodiment, based on the focal length value of the first binocular camera and the first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera, the first relative angle between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the optical axis of the first binocular camera is obtained, the calculation method is simple, the optical center connecting line distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the optical axis of the first binocular camera, which is obtained based on the traditional binocular distance principle, is not used, and the calculation result is more accurate.
In one embodiment, as shown in fig. 10, obtaining a first linear distance between a vertex of the construction machine in the detection frame corresponding to the first binocular camera and a center of the detection frame corresponding to the first binocular camera includes the following steps:
Step 1002, obtaining a rectangle with a diagonal line of a center line of a vertex of the construction machine in the detection frame corresponding to the first binocular camera and the detection frame corresponding to the first binocular camera.
The rectangle OQKR shown in fig. 8 and 9 represents a rectangle with a line between the vertex K of the construction machine in the detection frame corresponding to the first binocular camera and the center O of the detection frame corresponding to the first binocular camera as a diagonal line; OQ represents the length value of rectangle OQKR; OR represents the width value of rectangle OQKR.
In step 1004, a length value and a width value of the rectangle are obtained.
Step 1006, determining a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera according to the length value and the width value of the rectangle based on the Pythagorean theorem.
The first straight line distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera is the diagonal line of the rectangle OQKR, and the first straight line distance is determined based on the Pythagorean theorem and the length value and the width value of the rectangle.
Specifically, the mathematical expression of the first straight-line distance is:
in this embodiment, by adopting a construction of a rectangle with the diagonal line connecting the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera, the first straight line distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera is determined based on the pythagorean theorem and the length value and the width value of the rectangle, and the calculation method is simple.
In one embodiment, as shown in fig. 11, acquiring the length value and the width value of the rectangle includes the steps of:
In step 1102, a length proportional relation and a width proportional relation of a detection frame corresponding to the rectangle and the first binocular camera are obtained.
As shown in fig. 8, M represents a length value of a detection frame corresponding to the first binocular camera; n represents the width value of the detection frame corresponding to the first binocular camera.
Specifically, the length proportional relation of the rectangular detection frame corresponding to the first binocular camera is a ratio between the length value of the rectangular detection frame corresponding to the first binocular camera, and the mathematical expression of the length proportional relation is expressed as:
The proportional relation of the width of the detection frame corresponding to the first binocular camera is the ratio of the width value of the rectangle to the width value of the detection frame corresponding to the first binocular camera. The mathematical expression of the width proportional relation is expressed as:
in step 1104, a first angle value is obtained based on a proportional relationship between the horizontal angle of view and the length of the first binocular camera.
Wherein the horizontal angle of view of the first binocular camera is a known parameter, in this embodimentRepresenting the horizontal field angle of the first binocular camera.
Specifically, as shown in fig. 9, the product of the horizontal angle of view and the length proportional relation of the first binocular camera determines the first angle value. The mathematical expression of the first angle value is expressed as: Alpha represents a first angle value.
In step 1106, a second angle value is obtained based on a proportional relationship between the vertical field angle and the width of the first binocular camera.
The vertical field angle of the first binocular camera is a known parameter, and γ is used in this embodiment to represent the vertical field angle of the first binocular camera.
Specifically, as shown in fig. 9, the product of the vertical angle of view and the width proportional relation of the first binocular camera determines the second angle value. The mathematical expression of the second angle value is expressed as: Beta represents a second angle value.
In step 1108, a length value of the rectangle is obtained based on the tangent value of the first angle value and the focal length value of the first binocular camera.
Specifically, the product of the tangent value of the first angle value and the focal length value of the first binocular camera determines the length value of the rectangle. The mathematical expression of the length value of the rectangle is expressed as: |oq|=tanα |OO' |.
Step 1110, obtaining a width value of the rectangle based on the tangent value of the second angle value and the focal length value of the first binocular camera.
Specifically, the product of the tangent value of the second angle value and the focal length value of the first binocular camera determines the width value of the rectangle. The mathematical expression of the width value of the rectangle is expressed as: |or|=tan β |OO' |. The mathematical expression of the first straight line distance is:
in one embodiment, as shown in fig. 12, determining the distance between the position of the vertex of the construction machine and the power transmission line according to the relative distance and the included angle between the two binocular cameras includes the following steps:
step 1202, calculating a second linear distance between the vertex of the construction machine in the detection frame corresponding to one of the binocular cameras and the optical center of the one of the binocular cameras based on the relative distance and the included angle.
The distance between the vertex of the construction machine and the optical center of one of the binocular cameras is directly measured by the traditional binocular distance measuring principle, and the method is influenced by the base line and the light rays of the binocular cameras during long-distance measurement, so that a larger error exists in the measured distance between the vertex of the construction machine in the detection frame corresponding to one of the binocular cameras and the optical center of one of the binocular cameras.
Specifically, the construction machinery vertex is connected with the optical centers of the two binocular cameras, and the relative distance between the two binocular cameras forms a triangle; the relative distance between the two binocular cameras and the included angle between the connecting line of the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera are obtained, the relative distance between the two binocular cameras is the bottom edge of a triangle, the included angle between the connecting line of the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera is two angles of the triangle, and as the triangle is known to be one bottom edge and two angles, the second linear distance between the construction machine vertex and the optical center of one binocular camera in the detection frame corresponding to one binocular camera can be determined based on the trigonometric function relation, the relative distance between the two binocular cameras and the included angle between the optical center of each binocular camera and the connecting line of the vertex of the construction machine and the optical axis of the binocular camera.
And 1204, acquiring the distance between the position of the vertex of the construction machine and the power transmission line based on the sine value of the included angle and the second linear distance.
Specifically, an included angle between a connecting line of one binocular camera optical center and the construction machine vertex and an optical axis of the binocular camera and a second linear distance between the construction machine vertex and the optical center of one binocular camera in a detection frame corresponding to the one binocular camera are obtained, and the distance between the position of the construction machine vertex and the power transmission line is obtained based on the product of the sine value of the included angle and the second linear distance.
In this embodiment, based on the relative distance between the two binocular cameras and the included angle between the line between the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera, the second linear distance between the vertex of the construction machine in the detection frame corresponding to one of the binocular cameras and the optical center of one of the binocular cameras is determined, and compared with the conventional method for directly obtaining the second linear distance based on the binocular distance principle, the method is higher in accuracy and is not affected by the baseline and the light of the binocular cameras.
In one embodiment, calculating a second linear distance between the vertex of the construction machine in the detection frame corresponding to one of the binocular cameras and the optical center of one of the binocular cameras based on the relative distance and the included angle includes the following steps: acquiring a second linear distance according to the relative distance, the included angle and a preset relation; the preset relation comprises a mapping relation among a relative distance variable, an included angle variable and a linear distance variable.
The second linear distance may be a linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the optical center of the first binocular camera, or may be a linear distance between the vertex of the construction machine in the detection frame corresponding to the second binocular camera and the optical center of the second binocular camera.
Specifically, the preset relational expression may be:
wherein A represents the relative distance between the two binocular cameras; b represents the straight line distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the optical center of the first binocular camera; c represents the straight line distance between the vertex of the construction machine in the detection frame corresponding to the second binocular camera and the optical center of the second binocular camera; delta 1 represents an included angle between a connecting line of the optical center of the first binocular camera and the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the optical axis of the first binocular camera; delta 2 represents the angle between the line connecting the optical center of the second binocular camera and the vertex of the construction machine in the detection frame corresponding to the second binocular camera and the optical axis of the second binocular camera. B or C can be calculated through a preset relation, and then a second linear distance is obtained.
The distance between the position of the construction machine vertex and the power line is expressed by a mathematical expression: e=sin δ 1·B=sinδ2 ·c, where E represents the distance between the position of the construction machine vertex and the transmission line.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the present embodiment also provides a camera-based power line ranging apparatus for implementing the above-mentioned related camera-based power line ranging method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitations in the embodiments of one or more camera-based powerline ranging devices provided below may be referred to the limitations of the camera-based powerline ranging method above, and will not be repeated here.
In one embodiment, as shown in fig. 13, there is provided a camera-based power line ranging apparatus comprising: the device comprises a first acquisition module, a second acquisition module and a determination module, wherein:
The first acquisition module is used for respectively acquiring the picture depth map of each binocular camera based on the parallax relation after respectively calibrating each binocular camera; performing binarization processing on a picture depth map corresponding to each binocular camera based on a preset binarization threshold value to obtain a binarized image, traversing each row of pixel points of the binarized image corresponding to a detection frame of each binocular camera from top to bottom in sequence, and determining a construction machine vertex based on the pixel points corresponding to a target row in each binarized image; the detection frame corresponding to the binocular camera is an image area where the construction machine body is located; the target line includes a pixel point whose first-appearing pixel value is a first value.
And the second acquisition module is used for respectively acquiring the included angle between the connecting line of the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera according to the photographed images of the two binocular cameras.
And the determining module is used for determining the distance between the position of the vertex of the construction machine and the power transmission line according to the relative distance and the included angle between the two binocular cameras.
In one embodiment, the camera-based power line ranging device further comprises a first relative angle acquisition module, a second relative angle acquisition module, wherein:
The first relative angle acquisition module is used for acquiring a first relative angle between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the optical center connecting line of the first binocular camera, and the optical axis of the first binocular camera.
And the second relative angle acquisition module is used for acquiring a second relative angle between the vertex of the construction machine in the detection frame corresponding to the second binocular camera and the optical center connecting line of the second binocular camera, and the optical axis of the second binocular camera.
In one embodiment, the camera-based powerline ranging apparatus further comprises: the device comprises a first linear distance acquisition module, a first calculation module and a second calculation module, wherein:
The first linear distance acquisition module is used for acquiring a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera.
The first calculation module is used for obtaining a tangent value of the first relative angle based on the first linear distance and the focal length value of the first binocular camera.
And the second calculation module is used for acquiring the first relative angle based on the arc tangent function and the tangent value of the first relative angle.
In one embodiment, the camera-based powerline ranging apparatus further comprises: the device comprises a construction module, a third acquisition module and a third calculation module, wherein:
And the construction module is used for acquiring a rectangle taking the central connecting line of the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the detection frame corresponding to the first binocular camera as a diagonal line.
And the third acquisition module is used for acquiring the length value and the width value of the rectangle.
And the third calculation module is used for determining a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera according to the length value and the width value of the rectangle based on the Pythagorean theorem.
In one embodiment, the camera-based powerline ranging apparatus further comprises: relation module, first angle value acquisition module, second angle value acquisition module, rectangle length calculation module, rectangle width calculation module, wherein:
and the relational expression module is used for acquiring a length proportional relational expression and a width proportional relational expression of the detection frame corresponding to the rectangle and the first binocular camera.
The first angle value acquisition module is used for acquiring a first angle value based on a proportional relation between the horizontal field angle and the length of the first binocular camera.
The second angle value obtaining module is used for obtaining a second angle value based on a proportional relation between the vertical field angle and the width of the first binocular camera.
And the rectangle length calculation module is used for obtaining the length value of the rectangle based on the tangent value of the first angle value and the focal length value of the first binocular camera.
And the rectangle width calculation module is used for acquiring the width value of the rectangle based on the tangent value of the second angle value and the focal length value of the first binocular camera.
In one embodiment, the camera-based powerline ranging apparatus further comprises: the device comprises a second linear distance acquisition module and a fourth calculation module, wherein:
And the second linear distance acquisition module is used for calculating a second linear distance between the vertex of the construction machine in the detection frame corresponding to one of the binocular cameras and the optical center of the one of the binocular cameras based on the relative distance and the included angle.
And the fourth calculation module is used for acquiring the distance between the position of the vertex of the construction machine and the power transmission line based on the sine value of the included angle and the distance between the second straight line.
In one embodiment, the camera-based powerline ranging apparatus further comprises: the mapping module is used for acquiring a second linear distance according to a relation formula based on the relative distance, the included angle and the preset relation; the preset relation comprises a mapping relation among a relative distance variable, an included angle variable and a linear distance variable.
The various modules in the camera-based powerline ranging apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 14. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a camera-based powerline ranging method. The display unit of the computer equipment is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device, wherein the display screen can be a liquid crystal display screen or an electronic ink display screen, the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on a shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 14 is merely a block diagram of a portion of the structure associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements are applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
After calibrating each binocular camera respectively, respectively acquiring a picture depth map of each binocular camera based on a parallax relation;
Performing binarization processing on a picture depth map corresponding to each binocular camera based on a preset binarization threshold value to obtain a binarized image, traversing each row of pixel points of the binarized image corresponding to a detection frame of each binocular camera from top to bottom in sequence, and determining a construction machine vertex based on the pixel points corresponding to a target row in each binarized image; the detection frame corresponding to the binocular camera is an image area where the construction machine body is located; the target row comprises pixel points with first-appearing pixel values as first values;
According to the photographed images of the two binocular cameras, respectively obtaining the included angle between the connecting line of the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera;
And determining the distance between the position of the vertex of the construction machine and the power transmission line according to the relative distance and the included angle between the two binocular cameras.
In one embodiment, the processor when executing the computer program further performs the steps of:
acquiring a first relative angle between a construction machine vertex in a detection frame corresponding to the first binocular camera and an optical center connecting line of the first binocular camera, and an optical axis of the first binocular camera;
Acquiring a second relative angle between the vertex of the construction machine in the detection frame corresponding to the second binocular camera and the optical center connecting line of the second binocular camera and the optical axis of the second binocular camera;
according to the relative distance and the included angle between the two binocular cameras, determining the distance between the position of the vertex of the construction machine and the power transmission line comprises the following steps:
the distance between the position of the construction machine vertex and the power line is obtained based on the relative distance, the first relative angle, and the second relative angle.
In one embodiment, the processor when executing the computer program further performs the steps of:
acquiring a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera;
acquiring a tangent value of a first relative angle based on the first linear distance and a focal length value of the first binocular camera;
the first relative angle is obtained based on the arctangent function and a tangent value of the first relative angle.
In one embodiment, the processor when executing the computer program further performs the steps of:
acquiring a rectangle taking a connecting line between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera as a diagonal line;
Acquiring a length value and a width value of a rectangle;
Based on the Pythagorean theorem, a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera is determined according to the length value and the width value of the rectangle.
In one embodiment, the processor when executing the computer program further performs the steps of:
Acquiring a length proportional relation and a width proportional relation of a rectangular detection frame corresponding to a first binocular camera;
Acquiring a first angle value based on a proportional relation between the horizontal field angle and the length of the first binocular camera;
acquiring a second angle value based on a proportional relation between the vertical field angle and the width of the first binocular camera;
acquiring a length value of a rectangle based on a tangent value of the first angle value and a focal length value of the first binocular camera;
and acquiring a width value of the rectangle based on the tangent value of the second angle value and the focal length value of the first binocular camera.
In one embodiment, the processor when executing the computer program further performs the steps of:
Calculating a second linear distance between the vertex of the construction machine in the detection frame corresponding to one of the binocular cameras and the optical center of the one of the binocular cameras based on the relative distance and the included angle;
And acquiring the distance between the position of the vertex of the construction machine and the power transmission line based on the sine value of the included angle and the second linear distance.
In one embodiment, the processor when executing the computer program further performs the steps of:
acquiring a second linear distance according to the relative distance, the included angle and a preset relation; the preset relation comprises a mapping relation among a relative distance variable, an included angle variable and a linear distance variable.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
After calibrating each binocular camera respectively, respectively acquiring a picture depth map of each binocular camera based on a parallax relation;
Performing binarization processing on a picture depth map corresponding to each binocular camera based on a preset binarization threshold value to obtain a binarized image, traversing each row of pixel points of the binarized image corresponding to a detection frame of each binocular camera from top to bottom in sequence, and determining a construction machine vertex based on the pixel points corresponding to a target row in each binarized image; the detection frame corresponding to the binocular camera is an image area where the construction machine body is located; the target row comprises pixel points with first-appearing pixel values as first values;
According to the photographed images of the two binocular cameras, respectively obtaining the included angle between the connecting line of the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera;
And determining the distance between the position of the vertex of the construction machine and the power transmission line according to the relative distance and the included angle between the two binocular cameras.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a first relative angle between a construction machine vertex in a detection frame corresponding to the first binocular camera and an optical center connecting line of the first binocular camera, and an optical axis of the first binocular camera;
Acquiring a second relative angle between the vertex of the construction machine in the detection frame corresponding to the second binocular camera and the optical center connecting line of the second binocular camera and the optical axis of the second binocular camera;
according to the relative distance and the included angle between the two binocular cameras, determining the distance between the position of the vertex of the construction machine and the power transmission line comprises the following steps:
the distance between the position of the construction machine vertex and the power line is obtained based on the relative distance, the first relative angle, and the second relative angle.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera;
acquiring a tangent value of a first relative angle based on the first linear distance and a focal length value of the first binocular camera;
the first relative angle is obtained based on the arctangent function and a tangent value of the first relative angle.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a rectangle taking a connecting line between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera as a diagonal line;
Acquiring a length value and a width value of a rectangle;
Based on the Pythagorean theorem, a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera is determined according to the length value and the width value of the rectangle.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Acquiring a length proportional relation and a width proportional relation of a rectangular detection frame corresponding to a first binocular camera;
Acquiring a first angle value based on a proportional relation between the horizontal field angle and the length of the first binocular camera;
acquiring a second angle value based on a proportional relation between the vertical field angle and the width of the first binocular camera;
acquiring a length value of a rectangle based on a tangent value of the first angle value and a focal length value of the first binocular camera;
and acquiring a width value of the rectangle based on the tangent value of the second angle value and the focal length value of the first binocular camera.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Calculating a second linear distance between the vertex of the construction machine in the detection frame corresponding to one of the binocular cameras and the optical center of the one of the binocular cameras based on the relative distance and the included angle;
And acquiring the distance between the position of the vertex of the construction machine and the power transmission line based on the sine value of the included angle and the second linear distance.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a second linear distance according to the relative distance, the included angle and a preset relation; the preset relation comprises a mapping relation among a relative distance variable, an included angle variable and a linear distance variable.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:
After calibrating each binocular camera respectively, respectively acquiring a picture depth map of each binocular camera based on a parallax relation;
Performing binarization processing on a picture depth map corresponding to each binocular camera based on a preset binarization threshold value to obtain a binarized image, traversing each row of pixel points of the binarized image corresponding to a detection frame of each binocular camera from top to bottom in sequence, and determining a construction machine vertex based on the pixel points corresponding to a target row in each binarized image; the detection frame corresponding to the binocular camera is an image area where the construction machine body is located; the target row comprises pixel points with first-appearing pixel values as first values;
According to the photographed images of the two binocular cameras, respectively obtaining the included angle between the connecting line of the optical center of each binocular camera and the vertex of the construction machine and the optical axis of the binocular camera;
And determining the distance between the position of the vertex of the construction machine and the power transmission line according to the relative distance and the included angle between the two binocular cameras.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a first relative angle between a construction machine vertex in a detection frame corresponding to the first binocular camera and an optical center connecting line of the first binocular camera, and an optical axis of the first binocular camera;
Acquiring a second relative angle between the vertex of the construction machine in the detection frame corresponding to the second binocular camera and the optical center connecting line of the second binocular camera and the optical axis of the second binocular camera;
according to the relative distance and the included angle between the two binocular cameras, determining the distance between the position of the vertex of the construction machine and the power transmission line comprises the following steps:
the distance between the position of the construction machine vertex and the power line is obtained based on the relative distance, the first relative angle, and the second relative angle.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera;
acquiring a tangent value of a first relative angle based on the first linear distance and a focal length value of the first binocular camera;
the first relative angle is obtained based on the arctangent function and a tangent value of the first relative angle.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a rectangle taking a connecting line between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera as a diagonal line;
Acquiring a length value and a width value of a rectangle;
Based on the Pythagorean theorem, a first linear distance between the vertex of the construction machine in the detection frame corresponding to the first binocular camera and the center of the detection frame corresponding to the first binocular camera is determined according to the length value and the width value of the rectangle.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Acquiring a length proportional relation and a width proportional relation of a rectangular detection frame corresponding to a first binocular camera;
Acquiring a first angle value based on a proportional relation between the horizontal field angle and the length of the first binocular camera;
acquiring a second angle value based on a proportional relation between the vertical field angle and the width of the first binocular camera;
acquiring a length value of a rectangle based on a tangent value of the first angle value and a focal length value of the first binocular camera;
and acquiring a width value of the rectangle based on the tangent value of the second angle value and the focal length value of the first binocular camera.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Calculating a second linear distance between the vertex of the construction machine in the detection frame corresponding to one of the binocular cameras and the optical center of the one of the binocular cameras based on the relative distance and the included angle;
And acquiring the distance between the position of the vertex of the construction machine and the power transmission line based on the sine value of the included angle and the second linear distance.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a second linear distance according to the relative distance, the included angle and a preset relation; the preset relation comprises a mapping relation among a relative distance variable, an included angle variable and a linear distance variable.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magneto-resistive random access Memory (Magnetoresistive RandomAccess Memory, MRAM), ferroelectric Memory (Ferroelectric RandomAccess Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (RandomAccess Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static RandomAccess Memory, SRAM) or dynamic random access memory (Dynamic RandomAccess Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.