[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN116261046A - Tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation - Google Patents

Tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation Download PDF

Info

Publication number
CN116261046A
CN116261046A CN202310538972.5A CN202310538972A CN116261046A CN 116261046 A CN116261046 A CN 116261046A CN 202310538972 A CN202310538972 A CN 202310538972A CN 116261046 A CN116261046 A CN 116261046A
Authority
CN
China
Prior art keywords
image
tower crane
camera
cameras
panoramic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310538972.5A
Other languages
Chinese (zh)
Inventor
赵晓东
赵焕
张朝威
牛梅梅
杜赛楠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Advanced Institute of Information Technology AIIT of Peking University
Hangzhou Weiming Information Technology Co Ltd
Original Assignee
Advanced Institute of Information Technology AIIT of Peking University
Hangzhou Weiming Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Advanced Institute of Information Technology AIIT of Peking University, Hangzhou Weiming Information Technology Co Ltd filed Critical Advanced Institute of Information Technology AIIT of Peking University
Priority to CN202310538972.5A priority Critical patent/CN116261046A/en
Publication of CN116261046A publication Critical patent/CN116261046A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

The embodiment of the application provides a tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation. The method comprises the following steps: at least one camera is respectively arranged at a plurality of positions of the tower crane, so that the sum of the fields of view of all cameras covers all surrounding areas of the tower crane, and each camera is arranged on a cradle head and comprises a gyroscope sensor; when the gyro sensor of at least one camera detects over-amplitude jitter, electronic stability enhancement processing is carried out on the acquired image based on the acquired gyro sensor data, and inter-frame jitter is smoothed, so that a stable shooting image is obtained; and after all cameras are shot and electronic stability augmentation processing is completed, panoramic stitching is carried out on shot images of all cameras, and panoramic images of the tower crane are obtained. According to the panoramic image processing method and device, through an electronic stability augmentation mode and accurate position and movement compensation calculation, the technical problems that the splicing of panoramic images is delayed and the spliced panoramic images are asymmetric due to image blurring are solved.

Description

Tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation
Technical Field
The application relates to the technical field of intelligent tower cranes, in particular to a tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation.
Background
Because the tower crane has a huge structure and is accompanied with high-altitude operation, serious personal casualties are easy to occur, and once accidents occur, huge economic losses are brought to construction enterprises and individuals. Therefore, in the installation link and the lifting link of the tower crane, the monitoring cradle head needs to be installed in different areas so as to ensure the safety of the tower crane.
At present, a tower crane cradle head of a tower crane can only monitor the condition of one direction in one area, and can not provide 360-degree panoramic images without dead angles, so that potential safety hazards exist. In addition, vibration and movement in the operation process of the tower crane can also cause the shaking phenomenon of the tower crane cradle head, so that a certain blur of the monitoring image occurs.
Disclosure of Invention
In view of this, the purpose of this application is to provide a tower crane tripod head panoramic image anti-shake method and device based on electron increases steady, and this application can be targeted solve current tower crane monitoring dead angle and the image blurring problem that the tripod head shake leads to.
Based on the above purpose, the application provides a tower crane cradle head panoramic image anti-shake method based on electronic stability augmentation, which comprises the following steps:
At least one camera is respectively arranged at a plurality of positions of the tower crane, so that the sum of the fields of view of all cameras covers all surrounding areas of the tower crane, and each camera is arranged on a cradle head and comprises a gyroscope sensor;
when the gyro sensor of at least one camera detects over-amplitude jitter, electronic stability enhancement processing is carried out on the acquired image based on the acquired gyro sensor data, and inter-frame jitter is smoothed, so that a stable shooting image is obtained;
after all cameras are shot and electronic stability augmentation processing is completed, panoramic stitching is carried out by using shot images of all cameras;
drawing a map of an operation site of the tower crane, marking the positions of all cameras on the map, and displaying a map display interface, wherein the map display interface displays the positions of all cameras;
and receiving panoramic image calling request information of clicking the target camera on the map display interface, and displaying the panoramic image of the corresponding target camera.
Further, the electronic stability enhancement processing is performed on the acquired image based on the acquired gyroscope sensor data, and inter-frame jitter is smoothed to obtain a stable photographed image, including:
Judging the movement condition of the camera according to the gyroscope sensor data;
filtering and denoising image data acquired by a camera;
judging the motion effect of the image by a frame difference method, namely firstly calculating the difference between a current frame and a previous frame to obtain a forward frame difference, then calculating the difference between a next frame and the current frame to obtain a backward frame difference, and obtaining the intersection of the forward frame difference and the backward frame difference to obtain a rough motion area of a moving object;
performing optical flow calculation on image pixel points, selecting characteristic points of an image, estimating a motion vector, and taking the motion vector as a motion compensation path, wherein the optical flow calculation comprises the following steps: calculating to obtain characteristic points of each pixel point based on each pixel point and gray values of each pixel point, and performing optical flow calculation on all the characteristic points to obtain amplitude values of each characteristic point; screening out characteristic points with the amplitude within a preset amplitude range to obtain a plurality of optical flow characteristic points; describing the plurality of optical flow characteristic points by using an ORB algorithm, matching the characteristic points by using an approximate nearest neighbor FLANN algorithm after the characteristic points are described, removing mismatching point pairs generated in a matching process by using a random sampling consistency algorithm based on a statistical method, and finally carrying the obtained matching point pairs into an affine transformation model of a reference frame and a current frame image to obtain a set of overdetermined equations, and obtaining a global motion vector between image sequences by using least square fitting;
And reversely compensating the image according to the motion vector to obtain a stable shooting image, comprising: and compensating the motion vector of the video sequence by using the global motion vector parameter and an affine transformation model between the current frame image and the reference frame image in the video sequence to obtain a stable video sequence.
Further, the determining the movement condition of the camera according to the gyroscope sensor data comprises;
collecting six-axis data of a gyroscope sensor;
performing data filtering and zero point correction on the six-axis data;
performing temperature compensation on the six-axis data, and performing data fusion on the compensated data;
calculating Euler angles of three XYZ axial directions to represent the movement direction of the camera;
performing variance calculation on Euler angles of three axes to represent actual vibration amplitude of the camera;
and setting the variance threshold control as a hysteresis loop mode, when the variance is larger than the high threshold, turning on the electronic stability augmentation, and when the variance is smaller than the low threshold, turning off the electronic stability augmentation.
Further, the electronic stability enhancement processing is performed on the acquired image based on the acquired gyroscope sensor data, and inter-frame jitter is smoothed to obtain a stable photographed image, including:
1) Jitter estimation, namely estimating motion parameters between a current frame and a previous frame based on acquired gyroscope sensor data to obtain global motion parameters;
2) Dithering filtering, namely filtering global motion parameters to obtain an intentional motion vector and an unintentional random motion vector;
3) And jitter compensation, namely performing frame-by-frame compensation on the video image sequence by utilizing compensation parameters obtained by filtering.
Further, the panoramic stitching is performed on the photographed images of all cameras to obtain panoramic images of the tower crane, including:
carrying out image distortion correction and scaling treatment on the photographed images of all cameras to obtain all images to be fused;
calculating optimal stitching center lines of all images to be fused of all cameras in a fusion area, wherein the fusion area comprises a first fusion area and a second fusion area, the first fusion area is formed by an area which is positioned at the bottom of each image to be fused and is overlapped with each other, and the second fusion area is formed by an area which is formed by the edges of two adjacent images to be fused and is overlapped with each other;
calculating a weight table of the first fusion zone and the second fusion zone based on the optimal stitching center line;
and fusing and stretching all the images to be fused according to the weight tables of the first fusion area and the second fusion area.
Further, calculating the splicing position during the first splicing, directly calculating the similarity of the overlapped part in the subsequent splicing, splicing according to the position of the previous splicing if the similarity is higher than a preset threshold value, and calculating the compensation value or splicing according to the first method again if the similarity is lower than the preset threshold value.
Further, after the panoramic stitching is performed on the photographed images using all cameras to obtain the panoramic image of the tower crane, the method further includes: and (3) giving a display frame, covering each frame of image into the images spliced in advance according to the matching position, and then cutting the spliced images according to the position of the calibration object.
Further, the drawing of the map of the operation site of the tower crane, marking the positions of all cameras on the map, and presenting a map display interface, wherein the map display interface displays all the positions of the cameras, and the method comprises the following steps:
drawing a map of the tower crane operation site according to the panoramic image of the tower crane, and marking the positions of all cameras on the map;
and displaying a map display interface according to the tower crane operation site map, wherein the map display interface displays the tower crane operation site map and all camera positions.
Further, the receiving the image retrieving request information of clicking the target camera on the map display interface, displaying the image of the corresponding target camera, includes:
receiving image calling request information of a target camera clicked on the map display interface, and displaying a selection menu, wherein the selection menu can select or input time information;
and displaying the image of the target camera at the moment closest to the selected or input moment information according to the moment information selected or input by the user.
The application also provides a tower crane cradle head panoramic image anti-shake system based on electron increases steady, include:
the camera shooting module is used for respectively installing at least one camera at a plurality of positions of the tower crane, so that the sum of the fields of view of all cameras covers all surrounding areas of the tower crane, and each camera is installed on the cloud platform and comprises a gyroscope sensor;
the electronic stability augmentation module is used for carrying out electronic stability augmentation processing on the acquired image based on the acquired gyroscope sensor data when the gyroscope sensor of at least one camera detects the over-amplitude jitter, smoothing the inter-frame jitter and obtaining a stable shooting image;
The panoramic stitching module is used for performing panoramic stitching by using the shooting images of all cameras after all cameras have shot and electronic stability augmentation processing is completed;
the map drawing module is used for drawing an operation site map of the tower crane, marking the positions of all cameras on the map, and presenting a map display interface, wherein the map display interface displays the positions of all cameras;
and the panoramic display module is used for receiving panoramic image calling request information of the target camera clicked on the map display interface and displaying the panoramic image of the corresponding target camera.
Overall, the advantages of the present application and the experience brought to the user are:
1. according to the method, the panoramic image of the tower crane operation site can be retrieved by monitoring personnel far outside the construction site in a regional mode through the site panoramic image collection mode, the operation site condition at the appointed moment is restored, the monitoring dead angle can be effectively eliminated, and the construction process is accurately restored.
2. Through an electronic stability augmentation mode and accurate position and movement compensation calculation, the problem of image blurring caused by excessive shake of a tower crane cradle head in the shooting process due to vibration of the tower crane is eliminated, and the technical problems that the panoramic image is delayed in splicing and asymmetric after splicing due to image blurring are solved.
Drawings
In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments according to the disclosure and are not therefore to be considered limiting of its scope.
Fig. 1 shows a schematic diagram of the architecture principle of the present application.
Fig. 2 shows a flowchart of a tower crane pan-tilt panoramic image anti-shake method based on electronic stability augmentation according to an embodiment of the application.
Fig. 3 shows a schematic view of a panorama stitching manner according to an embodiment of the present application.
Fig. 4 shows a configuration diagram of a tower crane pan-tilt panoramic image anti-shake device based on electronic stability augmentation according to an embodiment of the application.
FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 6 shows a schematic diagram of a storage medium according to an embodiment of the present application.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 shows a schematic diagram of the system architecture principle of the present application. In the embodiment of the application, at least one camera (ABCD) is respectively installed at a plurality of positions of a large arm of the tower crane, so that the sum of fields of view of all cameras covers all surrounding areas of the tower crane, and each camera is installed on a cradle head and comprises a gyroscope sensor; when the gyro sensor of at least one camera detects over-amplitude jitter, electronic stability enhancement processing is carried out on the acquired image based on the acquired gyro sensor data, and inter-frame jitter is smoothed, so that a stable shooting image is obtained; and after all cameras are shot and electronic stability augmentation processing is completed, panoramic stitching is carried out by using the shot images of all cameras. The user can see the map of the construction site and each camera and its location within the map through a panoramic monitoring display located in the control room. The panoramic spliced images of all cameras can be seen on the screen, and the images acquired by the single cameras can be seen by clicking the single camera on the map.
Fig. 2 shows a flowchart of a tower crane pan-tilt panoramic image anti-shake method based on electronic stability augmentation according to an embodiment of the application. As shown in fig. 2, the anti-shake method for the panoramic image of the tower crane cradle head based on electronic stability augmentation comprises the following steps:
s1, respectively installing at least one camera at a plurality of positions of the tower crane, so that the sum of the fields of view of all cameras covers all surrounding areas of the tower crane, and each camera is installed on a tripod head and comprises a gyroscope sensor.
S2, when the gyro sensor of at least one camera detects excessive jitter, electronic stability enhancement processing is carried out on the acquired image based on the acquired gyro sensor data, and the jitter between frames is smoothed, so that a stable shooting image is obtained; here the jitter detection may take the form of an accelerometer plus a gyroscope.
S3, after all cameras have shot and electronic stability augmentation processing is completed, panoramic stitching is carried out on shot images of all cameras, and panoramic images of the tower crane are obtained;
s4, drawing a map of an operation site of the tower crane, marking the positions of all cameras on the map, and displaying a map display interface, wherein the map display interface displays the positions of all cameras;
S5, receiving image calling request information of clicking the target camera on the map display interface, and displaying the corresponding image of the target camera.
More specifically, in the first embodiment, step S2 includes:
judging the movement condition of the camera according to the gyroscope sensor data;
filtering and noise reduction are carried out on image data acquired by a camera, so that misjudgment of image characteristics caused by noise points is avoided;
judging the motion effect of the image by a frame difference method, namely firstly calculating the difference between a current frame and a previous frame to obtain a forward frame difference, then calculating the difference between a next frame and the current frame to obtain a backward frame difference, and obtaining the intersection of the forward frame difference and the backward frame difference to obtain a rough motion area of a moving object;
performing optical flow calculation on image pixel points, selecting characteristic points of an image, estimating a motion vector, and taking the motion vector as a motion compensation path, wherein the optical flow calculation comprises the following steps: calculating to obtain characteristic points of each pixel point based on each pixel point and gray values of each pixel point, and performing optical flow calculation on all the characteristic points to obtain amplitude values of each characteristic point; screening out characteristic points with the amplitude within a preset amplitude range to obtain a plurality of optical flow characteristic points; describing the plurality of optical flow characteristic points by using an ORB algorithm, matching the characteristic points by using an approximate nearest neighbor FLANN algorithm after the characteristic points are described, removing mismatching point pairs generated in a matching process by using a random sampling consistency algorithm based on a statistical method, and finally carrying the obtained matching point pairs into an affine transformation model of a reference frame and a current frame image to obtain a set of overdetermined equations, and obtaining a global motion vector between image sequences by using least square fitting;
And reversely compensating the image according to the motion vector to obtain a stable shooting image, comprising: and compensating the motion vector of the video sequence by using the global motion vector parameter and an affine transformation model between the current frame image and the reference frame image in the video sequence to obtain a stable video sequence.
The method for judging the movement condition of the camera according to the gyroscope sensor data comprises the following steps of;
collecting six-axis data of a gyroscope sensor; the acquisition frequency is required to be in accordance with the sampling theorem according to the dithering frequency, and the sampling frequency is set to be 500Hz.
Performing data filtering and zero point correction on the six-axis data;
because the gyroscope has the property of temperature drift, temperature compensation is needed to be carried out on data, and the compensated data are subjected to data fusion;
calculating Euler angles of three XYZ axial directions to represent the movement direction of the camera;
performing variance calculation on Euler angles of three axes to represent actual vibration amplitude of the camera;
in order to avoid frequent switching of critical points, the variance threshold control is set to be in a hysteresis loop mode, when the variance is larger than a high threshold, electronic stability augmentation is started, and when the variance is smaller than a low threshold, electronic stability augmentation is stopped. According to the embodiment, the gyroscope data is collected and analyzed to perform video image anti-shake, so that the stability of the image is greatly improved, the misjudgment of electronic anti-shake is reduced, the phenomena of abnormal video shake and background drag are avoided, and the visual effect is stable and reliable.
More specifically, in the second embodiment, the step S2 is further designed to include the following three steps: 1) Jitter estimation, namely estimating motion parameters between a current frame and a previous frame based on acquired gyroscope sensor data to obtain global motion parameters; 2) Dithering filtering, namely filtering global motion parameters to obtain an intentional motion vector and an unintentional random motion vector; 3) And jitter compensation, namely performing frame-by-frame compensation on the video image sequence by utilizing compensation parameters obtained by filtering.
In addition, according to different jitter estimation modes, the electronic stability augmentation algorithm of the method can further use a pure image electronic stability augmentation method and a gyro-based electronic stability augmentation method. The pure image electronic stability augmentation method only utilizes the characteristic information of the image to estimate the shaking amount of the camera, and has the advantage of no need of any additional equipment, but just because the method only utilizes the characteristic of the image to estimate the motion, the image imaging is required on one hand, namely the image has rich characteristic information, for example, the image fails in a single background such as sky or sea; on the other hand, because a large amount of work such as feature extraction, feature matching and the like is needed for the image at any time, the calculation is complex, the energy consumption is high, and the power requirement for the cradle head is generally high. The electronic stability augmentation method based on the gyroscope utilizes the gyroscope sensor to estimate the camera shake quantity, and is more suitable for the digital image acquisition task of the cradle head.
More specifically, in addition to the above-mentioned electronic stability enhancement mode, in the third embodiment, the electronic image stabilization method is adopted as follows: block matching, gray-scale projection and feature matching. The block matching method is the most common motion vector estimation method, and the best matching block is quickly and accurately searched through a proper search path to obtain a motion vector. The gray projection method has a high image processing speed, but has a high quality requirement on the processed image, mainly because if the processed image has a low quality, the gray projection curve changes inconspicuously, and it is difficult to accurately calculate the motion vector. The feature matching method is to select typical features in the image, such as edges, contours, corner points and the like, and perform motion estimation through feature matching. The key technique is how to extract features and match the correct features. Because the method better approximates the visual characteristics of human beings and uses the useful information of the image in a large amount, the method can provide better image stabilizing results.
According to the embodiment, through an electronic stability augmentation mode and accurate position and movement compensation calculation, the problem of image blurring caused by excessive shake of the tower crane cradle head in the shooting process due to vibration of the tower crane is solved, and the technical problems that the panoramic image is delayed in splicing due to image blurring and the panoramic image after splicing is asymmetric are solved.
More specifically, step S3 includes:
carrying out image distortion correction and scaling treatment on the photographed images of all cameras to obtain all images to be fused; as shown in fig. 3, three adjacent images to be fused are taken, and the overlapping areas at the bottoms of the three adjacent images are Img1, img2 and Img3 respectively to form a first fusion area; the adjacent two mutually overlapped areas 2 and 3, 4 and 5, 6 and 1 of the images to be fused respectively form second fusion areas, and three second fusion areas are formed in total.
Calculating optimal stitching center lines of all images to be fused of all cameras in a fusion area, wherein the fusion area comprises a first fusion area and a second fusion area, the first fusion area is formed by an area which is positioned at the bottom of each image to be fused and is overlapped with each other, and the second fusion area is formed by an area which is formed by the edges of two adjacent images to be fused and is overlapped with each other; the optimal stitched centerline is the centerline of the repeated portion between the first and second fused regions. For example, there may be an exactly repeating portion of the scene for the same portion between 2 and 3, i.e. comprising a plurality of exactly identical columns of pixels, the optimal stitching centre line selecting the middle column of the plurality of exactly identical columns of pixels.
Calculating a weight table of the first fusion zone and the second fusion zone based on the optimal stitching center line; the first fusion area and the second fusion area correspond to the two overlapping areas, and the completely repeated part of the same part of the scene corresponds to the common repeated part between the two overlapping areas, so that the non-repeated part, the common repeated part and the non-repeated part of the first fusion area together form a stitching part, the proportion of the pixel columns of the three parts is calculated respectively, and the proportion of each part in the stitching part can be obtained and used as a weight table of the first fusion area and the second fusion area.
And fusing and stretching all the images to be fused according to the weight tables of the first fusion area and the second fusion area.
In this application, since the camera position is fixed, it is necessary to calculate the splice position at the time of the first splice. However, in the subsequent splicing process, compensation is only needed according to the first result. Therefore, the similarity of the overlapped parts can be directly calculated by the subsequent splicing, the splicing is performed according to the position of the previous splicing when the similarity is higher than the threshold value, and the compensation value is calculated or the splicing is performed again according to the first method when the similarity is lower than the threshold value. The purpose of this is to reduce the amount of computation and to increase the splicing speed.
In another embodiment, after the first stitching is completed, a display frame may be further given, each subsequent frame of image is covered in the previously stitched image according to the matching position, and then the stitched image is trimmed according to the position of the calibration object, so that the speed of stitching the images is increased.
The method and the device have the advantages that the cameras at different positions are used for respectively collecting images or videos of different areas, and all the images are spliced to obtain the panoramic image of the tower crane, so that monitoring dead angles are eliminated; in addition, the problem of image blurring caused by shaking and dithering of the camera can be intelligently detected, and the blurring of the spliced image is eliminated by adopting an electronic stability augmentation method, so that the panoramic image of the tower crane is always clear, and the effective monitoring of production safety is ensured.
More specifically, step S4 includes:
drawing a map of the tower crane operation site according to the panoramic image of the tower crane, and marking the positions of all cameras on the map;
and displaying a map display interface according to the tower crane operation site map, wherein the map display interface displays the tower crane operation site map and all camera positions.
The method and the device draw corresponding maps through the panoramic image of the whole tower crane operation site, so that the map can be displayed through the panoramic image display in fig. 1. The map can flash or highlight the position of each camera, so that the user can find and click conveniently.
More specifically, step S5 includes:
receiving image calling request information of a target camera clicked on the map display interface, and displaying a selection menu, wherein the selection menu can select or input time information;
and displaying the image of the target camera at the moment closest to the selected or input moment information according to the moment information selected or input by the user.
When a user wants to view the construction operation process of the tower crane in a certain area, the user can view the map of the construction operation site according to the prompt of the panoramic image display, find the camera nearest to the area and click the camera mark on the display screen. At the moment, a dialog box and/or a menu are popped up, the user is allowed to select or input a certain moment, and the server invokes the image of the camera at the latest time point of the moment and displays the image to the user, so that the panoramic monitoring and restoring effect is achieved. Especially when construction is problematic, and specific problems need to be looked back, monitored and searched, the construction process can be accurately, rapidly and clearly restored by adopting the method, and the problems and reasons existing in construction can be found.
The field panoramic image acquisition mode enables monitoring personnel far outside a construction field to call the panoramic image of the tower crane operation field in regions and restore the operation field condition at the appointed moment, so that the monitoring dead angle can be effectively eliminated, and the construction process can be accurately restored.
The application embodiment provides a tower crane cradle head panoramic image anti-shake device based on electronic stability augmentation, which is used for executing the tower crane cradle head panoramic image anti-shake method based on electronic stability augmentation described in the above embodiment, as shown in fig. 4, and the device comprises:
the cradle head shooting module 401 is configured to respectively install at least one camera at a plurality of positions of the tower crane, so that the sum of fields of view of all cameras covers all surrounding areas of the tower crane, and each camera is installed on the cradle head and comprises a gyroscope sensor;
the electronic stability augmentation module 402 is configured to, when the gyro sensor of the at least one camera detects the over-amplitude shake, perform electronic stability augmentation processing on the acquired image based on the acquired gyro sensor data, smooth the inter-frame shake, and obtain a stable photographed image;
the panorama stitching module 403 is configured to perform panorama stitching by using the photographed images of all cameras after all cameras have been photographed and electronic stability augmentation processing is completed;
The map drawing module 404 is configured to draw a map of the operation site of the tower crane, mark positions of all cameras on the map, and present a map display interface, where all the positions of the cameras are displayed on the map display interface;
and the panorama display module 405 is configured to receive panorama image retrieving request information for clicking the target camera on the map display interface, and display a panorama image of the corresponding target camera.
The electronic stability augmentation-based tower crane pan-tilt-head panoramic image anti-shake device provided by the embodiment of the application and the electronic stability augmentation-based tower crane pan-tilt-head panoramic image anti-shake method provided by the embodiment of the application are the same in concept and have the same beneficial effects as the method adopted, operated or realized by the stored application program.
The embodiment of the application also provides electronic equipment corresponding to the electronic stability augmentation-based panoramic image anti-shake method for the tower crane cradle head provided by the previous embodiment, so as to execute the electronic stability augmentation-based panoramic image anti-shake method for the tower crane cradle head. The embodiments of the present application are not limited.
Referring to fig. 5, a schematic diagram of an electronic device according to some embodiments of the present application is shown. As shown in fig. 5, the electronic device 2 includes: a processor 200, a memory 201, a bus 202 and a communication interface 203, the processor 200, the communication interface 203 and the memory 201 being connected by the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and when the processor 200 runs the computer program, the method for preventing the panoramic image of the tower crane pan-tilt based on electronic stability augmentation provided in any embodiment of the present application is executed.
The memory 201 may include a high-speed random access memory (RAM: random Access Memory), and may further include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the system network element and at least one other network element is implemented via at least one communication interface 203 (which may be wired or wireless), the internet, a wide area network, a local network, a metropolitan area network, etc. may be used.
Bus 202 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be classified as address buses, data buses, control buses, etc. The memory 201 is configured to store a program, and after receiving an execution instruction, the processor 200 executes the program, and the electronic stability augmentation-based panoramic image anti-shake method for a tower crane pan-tilt disclosed in any embodiment of the present application may be applied to the processor 200 or implemented by the processor 200.
The processor 200 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 200 or by instructions in the form of software. The processor 200 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but may also be a Digital Signal Processor (DSP), application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201, and in combination with its hardware, performs the steps of the above method.
The electronic equipment provided by the embodiment of the application and the electronic stability augmentation-based tower crane cradle head panoramic image anti-shake method provided by the embodiment of the application have the same beneficial effects as the method adopted, operated or realized by the electronic equipment and the method for preventing shake of the tower crane cradle head panoramic image based on electronic stability augmentation are based on the same invention conception.
The present application further provides a computer readable storage medium corresponding to the electronic stability augmentation-based panoramic image anti-shake method for a tower crane cradle head provided in the foregoing embodiment, referring to fig. 6, the computer readable storage medium is shown as an optical disc 30, and a computer program (i.e. a program product) is stored thereon, where the computer program, when executed by a processor, executes the electronic stability augmentation-based panoramic image anti-shake method for a tower crane cradle head provided in any of the foregoing embodiments.
It should be noted that examples of the computer readable storage medium may also include, but are not limited to, a phase change memory (PRAM), a Static Random Access Memory (SRAM), a Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a flash memory, or other optical or magnetic storage medium, which will not be described in detail herein.
The computer readable storage medium provided by the above embodiment of the present application and the electronic stability augmentation-based panoramic image anti-shake method for a tower crane cradle head provided by the embodiment of the present application have the same beneficial effects as the method adopted, operated or implemented by the application program stored therein, because of the same inventive concept.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a construction of such a system is apparent from the description above. In addition, the present application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and the above description of specific languages is provided for disclosure of preferred embodiments of the present application.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed application requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the present application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in a virtual machine creation system according to embodiments of the present application may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present application may also be embodied as a device or system program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present application may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think of various changes or substitutions within the technical scope of the present application, and these should be covered in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. The method for preventing the panoramic image of the tower crane cradle head from shaking based on electronic stability augmentation is characterized by comprising the following steps of:
at least one camera is respectively arranged at a plurality of positions of the tower crane, so that the sum of the fields of view of all cameras covers all surrounding areas of the tower crane, and each camera is arranged on a cradle head and comprises a gyroscope sensor;
when the gyro sensor of at least one camera detects over-amplitude jitter, electronic stability enhancement processing is carried out on the acquired image based on the acquired gyro sensor data, and inter-frame jitter is smoothed, so that a stable shooting image is obtained;
after all cameras are shot and electronic stability augmentation processing is completed, panoramic stitching is carried out on shot images of all cameras, and panoramic images of the tower crane are obtained;
drawing a map of an operation site of the tower crane, marking the positions of all cameras on the map, and displaying a map display interface, wherein the map display interface displays the positions of all cameras;
and receiving image calling request information of clicking the target camera on the map display interface, and displaying the image of the corresponding target camera.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
The electronic stability enhancement processing is performed on the acquired image based on the acquired gyroscope sensor data, the inter-frame jitter is smoothed, and a stable shooting image is obtained, and the method comprises the following steps:
judging the movement condition of the camera according to the gyroscope sensor data;
filtering and denoising image data acquired by a camera;
judging the motion effect of the image by a frame difference method, namely firstly calculating the difference between a current frame and a previous frame to obtain a forward frame difference, then calculating the difference between a next frame and the current frame to obtain a backward frame difference, and obtaining the intersection of the forward frame difference and the backward frame difference to obtain a rough motion area of a moving object;
performing optical flow calculation on image pixel points, selecting characteristic points of an image, estimating a motion vector, and taking the motion vector as a motion compensation path, wherein the optical flow calculation comprises the following steps: calculating to obtain characteristic points of each pixel point based on each pixel point and gray values of each pixel point, and performing optical flow calculation on all the characteristic points to obtain amplitude values of each characteristic point; screening out characteristic points with the amplitude within a preset amplitude range to obtain a plurality of optical flow characteristic points; describing the plurality of optical flow characteristic points by using an ORB algorithm, matching the characteristic points by using an approximate nearest neighbor FLANN algorithm after the characteristic points are described, removing mismatching point pairs generated in the matching process by using a random sampling consistency algorithm based on a statistical method, and finally carrying the obtained matching point pairs into an affine transformation model of the reference frame and the current frame image to obtain a set of overdetermined equations, and obtaining a global motion vector between image sequences by using least square fitting;
Performing reverse compensation on the image according to the global motion vector to obtain a stable shooting image, wherein the method comprises the following steps: and compensating the motion vector of the video sequence by using the global motion vector and an affine transformation model between the current frame image and the reference frame image in the video sequence to obtain a stable video sequence.
3. The method of claim 2, wherein the step of determining the position of the substrate comprises,
the method for judging the movement condition of the camera according to the gyroscope sensor data comprises the following steps of;
collecting six-axis data of a gyroscope sensor;
performing data filtering and zero point correction on the six-axis data;
performing temperature compensation on the six-axis data, and performing data fusion on the compensated data;
calculating Euler angles of three XYZ axial directions to represent the movement direction of the camera;
performing variance calculation on Euler angles of three axes to represent actual vibration amplitude of the camera;
and setting the variance threshold control as a hysteresis loop mode, when the variance is larger than the high threshold, turning on the electronic stability augmentation, and when the variance is smaller than the low threshold, turning off the electronic stability augmentation.
4. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the electronic stability enhancement processing is performed on the acquired image based on the acquired gyroscope sensor data, the inter-frame jitter is smoothed, and a stable shooting image is obtained, and the method comprises the following steps:
1) Jitter estimation, namely estimating motion parameters between a current frame and a previous frame based on acquired gyroscope sensor data to obtain global motion parameters;
2) Dithering filtering, namely filtering global motion parameters to obtain an intentional motion vector and an unintentional random motion vector;
3) And jitter compensation, namely performing frame-by-frame compensation on the video image sequence by utilizing compensation parameters obtained by filtering.
5. The method of any one of claim 1 to 4,
panoramic stitching is carried out on the shot images of all cameras to obtain panoramic images of the tower crane, and the panoramic stitching method comprises the following steps:
carrying out image distortion correction and scaling treatment on the photographed images of all cameras to obtain all images to be fused;
calculating optimal stitching center lines of all images to be fused of all cameras in a fusion area, wherein the fusion area comprises a first fusion area and a second fusion area, the first fusion area is formed by an area which is positioned at the bottom of each image to be fused and is overlapped with each other, and the second fusion area is formed by an area which is formed by the edges of two adjacent images to be fused and is overlapped with each other;
calculating a weight table of the first fusion zone and the second fusion zone based on the optimal stitching center line;
and fusing and stretching all the images to be fused according to the weight tables of the first fusion area and the second fusion area.
6. The method of claim 5, wherein,
calculating the splicing position during the first splicing, directly calculating the similarity of the overlapped part in the subsequent splicing, splicing according to the position of the previous splicing if the similarity is higher than a preset threshold value, and calculating the compensation value or splicing again according to the first method if the similarity is lower than the preset threshold value.
7. The method of any one of claim 1 to 4,
and after panoramic stitching is carried out on the shot images of all cameras to obtain panoramic images of the tower crane, the method further comprises the steps of: and (3) giving a display frame, covering each frame of image into the images spliced in advance according to the matching position, and then cutting the spliced images according to the position of the calibration object.
8. The method of claim 7, wherein,
the method for drawing the operation site map of the tower crane, marking the positions of all cameras on the map, and presenting a map display interface, wherein the map display interface displays all the positions of the cameras comprises the following steps:
drawing a map of the tower crane operation site according to the panoramic image of the tower crane, and marking the positions of all cameras on the map;
And displaying a map display interface according to the tower crane operation site map, wherein the map display interface displays the tower crane operation site map and all camera positions.
9. The method of claim 8, wherein the step of determining the position of the first electrode is performed,
the receiving the image calling request information of clicking the target camera on the map display interface, displaying the image of the corresponding target camera includes:
receiving image calling request information of a target camera clicked on the map display interface, and displaying a selection menu, wherein the selection menu can select or input time information;
and displaying the image of the target camera at the moment closest to the selected or input moment information according to the moment information selected or input by the user.
10. Tower crane cradle head panoramic image anti-shake system based on electron increases steady, its characterized in that includes:
the camera shooting module is used for respectively installing at least one camera at a plurality of positions of the tower crane, so that the sum of the fields of view of all cameras covers all surrounding areas of the tower crane, and each camera is installed on the cloud platform and comprises a gyroscope sensor;
the electronic stability augmentation module is used for carrying out electronic stability augmentation processing on the acquired image based on the acquired gyroscope sensor data when the gyroscope sensor of at least one camera detects the over-amplitude jitter, smoothing the inter-frame jitter and obtaining a stable shooting image;
The panoramic stitching module is used for performing panoramic stitching by using the shooting images of all cameras after all cameras have shot and electronic stability augmentation processing is completed;
the map drawing module is used for drawing an operation site map of the tower crane, marking the positions of all cameras on the map, and presenting a map display interface, wherein the map display interface displays the positions of all cameras;
and the panoramic display module is used for receiving panoramic image calling request information of the target camera clicked on the map display interface and displaying the panoramic image of the corresponding target camera.
CN202310538972.5A 2023-05-15 2023-05-15 Tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation Pending CN116261046A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310538972.5A CN116261046A (en) 2023-05-15 2023-05-15 Tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310538972.5A CN116261046A (en) 2023-05-15 2023-05-15 Tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation

Publications (1)

Publication Number Publication Date
CN116261046A true CN116261046A (en) 2023-06-13

Family

ID=86681049

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310538972.5A Pending CN116261046A (en) 2023-05-15 2023-05-15 Tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation

Country Status (1)

Country Link
CN (1) CN116261046A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130050401A1 (en) * 2009-09-04 2013-02-28 Breitblick Gmbh Portable wide-angle video recording system
CN109618091A (en) * 2018-10-23 2019-04-12 天津天地伟业信息系统集成有限公司 Merge the video anti-fluttering method and device of gyroscope
US10477104B1 (en) * 2015-11-02 2019-11-12 Amazon Technologies, Inc. Image sensor selection in a multiple image sensor device
CN113938647A (en) * 2021-09-13 2022-01-14 杭州大杰智能传动科技有限公司 Intelligent tower crane operation panoramic monitoring and restoring method and system for intelligent construction site
CN114979489A (en) * 2022-05-30 2022-08-30 西安理工大学 Gyroscope-based heavy equipment production scene video monitoring and image stabilizing method and system
CN115100087A (en) * 2022-06-27 2022-09-23 西安理工大学 Image stabilization processing method for tower crane hook video

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130050401A1 (en) * 2009-09-04 2013-02-28 Breitblick Gmbh Portable wide-angle video recording system
US10477104B1 (en) * 2015-11-02 2019-11-12 Amazon Technologies, Inc. Image sensor selection in a multiple image sensor device
CN109618091A (en) * 2018-10-23 2019-04-12 天津天地伟业信息系统集成有限公司 Merge the video anti-fluttering method and device of gyroscope
CN113938647A (en) * 2021-09-13 2022-01-14 杭州大杰智能传动科技有限公司 Intelligent tower crane operation panoramic monitoring and restoring method and system for intelligent construction site
CN114979489A (en) * 2022-05-30 2022-08-30 西安理工大学 Gyroscope-based heavy equipment production scene video monitoring and image stabilizing method and system
CN115100087A (en) * 2022-06-27 2022-09-23 西安理工大学 Image stabilization processing method for tower crane hook video

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
黄樟钦;母召;岑陈;高寒;: "一种多路视频的360度实时拼接技术", 西安电子科技大学学报, no. 02 *

Similar Documents

Publication Publication Date Title
CN113938647B (en) Intelligent tower crane operation panoramic monitoring and restoring method and system for intelligent construction site
EP2640057B1 (en) Image processing device, image processing method and program
US20210227132A1 (en) Method for tracking target in panoramic video, and panoramic camera
US9479709B2 (en) Method and apparatus for long term image exposure with image stabilization on a mobile device
CN107808111B (en) Method and apparatus for pedestrian detection and attitude estimation
CN113556464B (en) Shooting method and device and electronic equipment
GB2593335A (en) Method and apparatus for 3-D auto tagging
US20050052533A1 (en) Object tracking method and object tracking apparatus
CN111915483B (en) Image stitching method, device, computer equipment and storage medium
JP2021533660A (en) Optical image stabilization behavior to generate super-resolution images of scenes
EP2446612A1 (en) Real time video stabilization
US10904562B2 (en) System and method for constructing optical flow fields
CN116208855B (en) Multi-tower crane cradle head panoramic image jitter coordination inhibition method and system
CN113850881A (en) Image generation method, device, equipment and readable storage medium
Chew et al. Panorama stitching using overlap area weighted image plane projection and dynamic programming for visual localization
CN116261046A (en) Tower crane cradle head panoramic image anti-shake method and system based on electronic stability augmentation
CN116363185B (en) Geographic registration method, geographic registration device, electronic equipment and readable storage medium
CN116188275B (en) Single-tower crane panoramic image stitching method and system
CN107993247B (en) Tracking and positioning method, system, medium and computing device
JP2009077022A (en) Driving support system and vehicle
JP2006311099A (en) Device and method for automatic tracking
JP2002094849A (en) Wide view image pickup device
CN114845055A (en) Method and device for determining shooting parameters of image acquisition equipment and electronic equipment
JP2000059762A (en) Camera controller, method and computer-readable storage medium thereof
KR100856474B1 (en) Stabilizing method for shaking video image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20230613