CN118553091A - Method and system for generating real-time video of multiple traffic intersections through stratosphere space monitoring - Google Patents

Abstract

The invention belongs to the technical field of intelligent traffic control, and provides a method and a system for generating real-time videos of multiple traffic intersections by stratosphere space monitoring, wherein the method comprises the following steps: 10. acquiring a high-precision image set; 20. performing spatial transformation of the image according to the rotation angle; 30. constructing a target intersection ground network coordinate set; 40. extracting features and obtaining a feature map; 50. the feature map and the target intersection ground network coordinate set are fused to obtain a high-precision image with network coordinates; 60. expanding and cutting the high-precision image with the network coordinates by taking the intersection coordinates as vertexes, and merging the high-precision image with the network coordinates into a corresponding traffic intersection image set; 70. and converting the image sets of different intersections into video streams in time sequence. The method can monitor a plurality of traffic intersections in a large range at the same time, and a plurality of intersections under the same stratosphere space monitoring system can use a uniform control mode, so that the number of equipment and the cost are reduced; the method has great potential and provides powerful support for urban traffic planning and management.

Description

Method and system for generating real-time video of multiple traffic intersections through stratospheric space monitoring

Technical Field

The present invention belongs to the technical field of intelligent traffic control, and relates to a method and system for generating real-time videos of multiple traffic intersections through stratospheric space monitoring.

Background Art

In recent years, the increase in urban traffic flow has led to increasingly serious congestion problems at urban intersections. In order to deal with this problem, it is necessary to monitor intersection conditions in real time and provide accurate traffic flow information for traffic management and dispatch. At present, camera monitoring of intersections is a common method, but there are problems such as limited monitoring range and untimely data sharing.

With the progress of modern science and technology, the unique resource advantages of the stratospheric space have become a hot topic of world concern. According to the distribution of atmospheric thermal structure with altitude, the atmosphere can be divided into five layers: troposphere, stratosphere, mesosphere, thermosphere and exosphere. Among them, the stratosphere (the airspace 20km-50km above sea level) has almost no water vapor condensation, no thunderstorms and other weather, and no up and down convection of the atmosphere. It has the characteristics of good stability, slow changes in parameters such as temperature and humidity, and is not affected by cloud cover above the clouds. During the day, sufficient and stable solar energy can be used for power generation; the stratosphere is below the ionosphere, and information transmission is not affected by the ionosphere, providing strong external conditions for drones to stay in this space for a long time and stably; stratospheric space aircraft are equipped with remote sensing imaging systems, which are favored by people for their rich information and clear image characteristics. Compared with satellite images that require fixed-point and timed shooting and have poor timeliness, and ordinary aerial images that are limited by flight altitude and have a small field of view and narrow image format, ultra-high altitude UAV remote sensing imaging systems can achieve real-time road condition monitoring of multiple traffic intersections, avoiding problems such as ground cameras being easily blocked and obstructed, and improving the accuracy and stability of image capture.

Therefore, by using drones and other equipment to install cameras and transmission equipment in the stratosphere, real-time image monitoring of traffic intersections can be achieved. By deploying stratospheric space monitoring equipment, real-time monitoring and traffic flow prediction can be achieved for a large range and multiple intersections. Although stratospheric space monitoring equipment is expensive in some aspects, its ability to monitor more intersections means that its total cost will be lower than that of cameras that can only monitor one intersection. Moreover, as technology continues to develop, the cost of stratospheric space monitoring equipment will gradually decrease and become more practical.

At the same time, multiple intersections under the same stratospheric space monitoring system can use a unified control mode, thereby reducing the number of equipment and costs. This emerging traffic intersection monitoring method has great advantages and potential, which helps to improve the ability of traffic management departments to grasp the situation at intersections and provide strong support for urban traffic planning and management.

Summary of the invention

In view of the limitations of the prior art, the present invention proposes a method and system for generating real-time videos of multiple traffic intersections through stratospheric space monitoring. The method maps several layers of images shot at a rotation angle into an image set at the same angle, calibrates the image set with the coordinate data of the target area satellite map, and obtains a set of traffic intersection images by cropping the coordinate points of the image set. Finally, the image data of different intersections are converted into video streams in chronological order.

The technical solution of the present invention is achieved in this way:

A method for generating real-time video of multiple traffic intersections by stratospheric space monitoring, comprising:

10. The camera pan/tilt rotates to capture high-precision images and obtain a high-precision image set;

The initial state of the camera device is vertically downward, that is, θ=0. The camera head rotates at an angle θ to capture high-precision images. The shooting area range is recorded in chronological order into a set Q ^t _θ with the camera device shooting angle θ.

20. Perform spatial transformation of images according to the rotation angle;

Take the image with rotation angle θ=0 as the reference image, and transform the image with rotation angle θ≠0 in _the set ^Qtθ by the affine transformation formula

Perform image space transformation, where (u, v) is the original image pixel coordinate, (x, y) is the image pixel coordinate after transformation, is the perspective transformation matrix; we get the image set Q ^t ₀ with rotation angle θ=0.

30. Construct the target intersection ground network coordinate set;

Obtain satellite map data of the target area, extract all traffic intersection information from the satellite map data, construct a network topology graph G = (V, E) with all intersections in the target area as vertices, and construct the target intersection ground network coordinate set I.

40. Feature extraction, obtaining feature maps;

41. The high-precision image taken by the camera carried by the drone is used as the input image, and the input image is preprocessed to have the same contrast, brightness, etc. as the target intersection ground network coordinate set I, so as to better match it with the network topology map;

42. The preprocessed image is subjected to feature extraction through the feature pyramid network to obtain the feature map F.

50. The feature map is fused with the target intersection ground network coordinate set to obtain a high-precision image with network coordinates;

Set the target intersection coordinates x _i , integrate the feature graph F and the node I _i of the target intersection ground grid coordinate set to obtain F _i , and use the iterative matching formula Determine the coordinate information of the next intersection x _i+1 until the current intersection is the last intersection; where π is the iterative matching strategy, π is calculated by optimizing the formula Optimize and finally obtain a high-precision image with network coordinates.

60. Expand and crop the high-precision image with network coordinates with the intersection coordinates as the vertex and incorporate it into the corresponding traffic intersection image set;

61. Using intersection coordinates as vertices, crop the entire image into several separate intersection coordinate images A ^t _i , where i = (x, y) is the number of intersections with coordinate data obtained by cropping, and t is the timestamp of the current input image; classify the intersection images according to the intersection coordinate data to obtain a set A ^t _i = {A ^t _i丨t＞0};

According to the location and size of the intersection, the image processing technology can be used to process the cropped multiple individual intersection images into a uniform size of a*b, wherein the size of a*b is the size of the format of the final video image;

62. Repeat step 61 to enrich the set A ^t _i ={A ^t _i丨t＞0}.

70. Convert the image collections of different intersections into video streams in time sequence;

71. Perform secondary image processing on the set A ^t _i to obtain expected image data;

72. The expected image data of different intersections are input into the video encoder in time sequence, and the video encoder converts the image sequence into a video stream.

Stratospheric space monitoring generates real-time video systems for multiple traffic intersections, including

Drone: Once the target intersection is determined, the drone’s position will no longer change; the drone must be suitable for working in the stratosphere. The drone needs to have height, stability, and safety to ensure that it can fly for a long time in the stratosphere and carry the corresponding equipment.

Control equipment: determine the target intersection and select the next target intersection position according to the intersection ground grid coordinates; calculate the rotation angle and rotation direction of the camera pan/tilt to capture high-precision images.

Camera equipment: Installed on the drone, select a high-definition and high-frame-rate camera, preferably a professional-grade drone camera or a high-resolution video camera; the camera equipment should be able to adapt to changes in atmospheric conditions and be able to capture real-time images of traffic intersections.

Ground server: Establish a ground-side server system to receive, store and process image data transmitted by the drone; the server should have high performance and large-capacity storage capabilities to cope with the processing needs of real-time image data.

Transmission equipment: installed on the drone, the drone needs to be equipped with transmission equipment so that the captured images can be transmitted in real time to the ground server for processing and analysis; wireless signal transmission equipment or satellite communication equipment can be used to achieve data transmission.

The working principle and beneficial effects of the present invention are:

By deploying stratospheric space monitoring equipment, real-time monitoring and traffic flow prediction can be achieved for a large area and multiple intersections; compared with traditional intersection monitoring cameras, one intersection requires 1 to 4 monitoring cameras to complete the monitoring of a complete intersection, and the number of intersections monitored by stratospheric space monitoring equipment is equivalent to the effect achieved by thousands of cameras, so the total cost of stratospheric space monitoring equipment will be lower. Moreover, with the continuous development of technology, the cost of stratospheric space monitoring equipment will gradually decrease and become more practical.

Multiple intersections under the same stratospheric monitoring can use a unified control mode, reducing the number of equipment and costs; and when congestion or other traffic problems occur at a certain intersection, the entire stratospheric monitoring system can automatically adjust to better meet the needs of traffic management. The use of this control mode can also improve the efficiency and reliability of the system while reducing dependence on personnel.

The method of generating real-time videos of multiple traffic intersections through stratospheric space monitoring helps improve the ability of traffic management departments to grasp and cover the conditions of intersections in the entire range, and provides strong support for urban traffic planning and management.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

FIG1 is an overall flow chart of a method for generating real-time video of multiple traffic intersections by stratospheric space monitoring;

FIG2 is a schematic diagram of a method for generating real-time video of multiple traffic intersections by stratospheric space monitoring and a system shooting range;

FIG3 is a schematic diagram of the method and system for generating real-time video of multiple traffic intersections by stratospheric space monitoring;

FIG4 is a local network topology diagram of the ground grid coordinates of the target intersection of the method for generating real-time video of multiple traffic intersections by stratospheric space monitoring;

5 is a schematic diagram of the matching of characteristic graphs and ground grid coordinates of the method for generating real-time videos of multiple traffic intersections by stratospheric space monitoring;

FIG6 is a schematic diagram of the composition of a system for generating real-time video of multiple traffic intersections through stratospheric space monitoring;

1. Drone; 2. Control equipment; 3. Camera equipment; 4. Transmission equipment; 5. Ground server;

DETAILED DESCRIPTION

The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

It should be noted that before any flight operation is carried out, relevant laws and regulations and aviation management regulations should be complied with, and the safety of the operation should be ensured. In addition, the specific implementation of stratospheric space monitoring technology and equipment selection need to be further evaluated and selected based on specific project needs and technical requirements.

A method for generating real-time video of multiple traffic intersections by stratospheric space monitoring, comprising:

10. The camera pan/tilt rotates to capture high-precision images and obtain a high-precision image set;

The initial state of the camera device is vertically downward, that is, θ=0. The camera head rotates at an angle θ to capture high-precision images. Image processing operations such as denoising, image enhancement, and edge detection are performed on the captured high-precision images to improve image quality and visibility of traffic elements. The captured images are recorded in chronological order into a set Q ^t _θ with the camera device's shooting angle θ.

20. Perform spatial transformation of images according to the rotation angle;

Take the image with rotation angle θ=0 as the reference image, and transform the image with rotation angle θ≠0 in _the set ^Qtθ by the affine transformation formula

Perform image space transformation, where (u, v) is the original image pixel coordinate, (x, y) is the image pixel coordinate after transformation, is the perspective transformation matrix; we get the image set Q ^t ₀ with rotation angle θ=0.

30. Construct the target intersection ground network coordinate set;

Obtain satellite map data of the target area, extract all traffic intersection information from the satellite map data, construct a network topology graph G = (V, E) with all intersections in the target area as vertices, and construct a target intersection ground network coordinate set I; as shown in Figure 4, a local network topology diagram of the satellite map data of the target area is illustrated;

Among them, the Dijkstra algorithm can be used to complete the regional network topology graph G = (V, E), the steps are as follows:

1. Initialize the distance of all vertices to infinity, but set the distance of the source vertex to 0;

2. Create an empty priority queue and put the source vertex into the queue;

3. Take the vertex closest to the source vertex from the queue and remove it from the queue;

4. For all adjacent nodes of the vertex, if the distance to the adjacent node through the edge is shorter than the previously calculated distance, update the distance of the adjacent node;

5. Repeat steps 3 and 4 until the queue is empty.

40. Feature extraction, obtaining feature maps;

41. The high-precision image taken by the camera carried by the drone is used as the input image, and the input image is preprocessed to have the same contrast, brightness, etc. as the satellite map of the target area, so as to better match it with the regional network topology map, as shown in Figure 5, which illustrates a matching schematic diagram;

42. The preprocessed image is subjected to feature extraction through the feature pyramid network to obtain a feature map F. Feature extraction generally includes extraction of sidewalks, zebra crossings, road boundaries, center lines, stop lines, lanes, and ground markings;

50. The feature map is fused with the target intersection ground network coordinate set to obtain a high-precision image with network coordinates;

Set the target intersection coordinates x _i , integrate the feature graph F and the node I _i of the target intersection ground grid coordinate set to obtain F _i , and use the iterative matching formula Determine the coordinate information of the next intersection x _i+1 until the current intersection is the last intersection; where π is the iterative matching strategy, π is calculated by optimizing the formula Optimize and finally obtain a high-precision image with network coordinates.

60. Expand and crop the high-precision image with network coordinates with the intersection coordinates as the vertex and incorporate it into the corresponding traffic intersection image set;

61. Using intersection coordinates as vertices, crop the entire image into several separate intersection coordinate images A ^t _i , where i = (x, y) is the number of intersections with coordinate data obtained by cropping, and t is the timestamp of the current input image; classify the intersection images according to the intersection coordinate data to obtain a set A ^t _i = {A ^t _i丨t＞0};

According to the location and size of the intersection, the image processing technology can be used to process the cropped multiple individual intersection images into a uniform size of a*b, wherein the size of a*b is the size of the format of the final video image;

62. Repeat step 61 to enrich the set A ^t _i ={A ^t _i丨t＞0}.

70. Convert the image collections of different intersections into video streams in time sequence;

71. Perform secondary image processing on the set A ^t _i to obtain expected image data;

72. Input the image collections of different intersections into the video encoder in chronological order, select appropriate video encoding algorithms and parameters to balance the relationship between video quality and file size; at the same time, set parameters such as frame rate, resolution and compression ratio according to requirements; generate real-time video streams through the video encoder, and display and playback the generated video streams in real-time dynamic video in the monitoring center, traffic management system or other related applications.

As shown in Figure 6, the stratospheric space monitoring system generates real-time video of multiple traffic intersections, including

Drone 1: Determine the target intersection, and the position of Drone 1 will no longer change; Drone 1 must be suitable for working in the stratosphere. Drone 1 needs to have height, stability and safety to ensure that it can fly for a long time in the stratosphere and carry corresponding equipment.

Control device 2: Determine the target intersection and select the next target intersection position according to the intersection ground grid coordinates; calculate the rotation angle and rotation direction of the camera pan/tilt to capture high-precision images.

Camera device 3: Installed on the drone 1, select a camera device 3 with high definition and high frame rate, preferably a professional-grade drone 1 camera or a high-resolution video camera; the camera device 3 should have the ability to adapt to changes in atmospheric conditions and be able to capture real-time images of traffic intersections.

Transmission device 4: installed on the drone 1, the drone 1 needs to be equipped with transmission device 4 so that the captured images can be transmitted to the ground server 5 in real time for processing and analysis; wireless signal transmission device 4 or satellite communication equipment can be used to achieve data transmission.

Ground server 5: Establish a ground-side server system for receiving, storing and processing image data transmitted by UAV 1; the server should have high performance and large-capacity storage capabilities to cope with the processing requirements of real-time image data.

It is worth noting that this embodiment only shows a simple data example of the method and system for generating real-time videos of multiple traffic intersections through stratospheric space monitoring, just for the convenience of demonstrating the principle of this method. Those skilled in the art should understand that the timing of the method steps provided in the above embodiment can be adaptively adjusted according to actual conditions, and can also be performed concurrently according to actual conditions.

It is important to note that when processing images and generating video streams, data security and privacy protection should be taken into consideration. Ensure that appropriate measures are taken to encrypt, store, and transmit image and video data to avoid any potential leakage or misuse.

All or part of the steps in the methods involved in the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a storage medium readable by a computer device to execute all or part of the steps described in the methods of the above embodiments.

Finally, it should be noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, product or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, product or apparatus, and does not exclude the presence of other identical elements in the process, method, product or apparatus that includes the elements.

Claims (10)

1. A method for generating real-time video of multiple traffic intersections by stratospheric space monitoring, characterized in that it includes: 10. The camera pan/tilt rotates to capture high-precision images and obtain a high-precision image set; 20. Perform spatial transformation of images according to the rotation angle; 30. Construct the target intersection ground network coordinate set; 40. Feature extraction, obtaining feature maps; 50. The feature map is fused with the target intersection ground network coordinate set to obtain a high-precision image with network coordinates; 60. Expand and crop the high-precision image with network coordinates with the intersection coordinates as the vertex and incorporate it into the corresponding traffic intersection image set; 70. Convert image collections from different intersections into video streams in chronological order. 2. The method for generating real-time videos of multiple traffic intersections by stratospheric space monitoring according to claim 1 is characterized in that step 10 comprises: the initial state of the camera device is vertically downward, that is, θ=0, the camera head rotates at an angle θ to shoot high-precision images, and the shooting area range is recorded in a set Q ^t _θ with the camera device shooting angle θ in chronological order. 3. The method for generating real-time video of multiple traffic intersections by stratospheric space monitoring according to claim 1, characterized in that step 20 comprises taking the image with rotation angle θ=0 as the reference image, and transforming the image with rotation angle θ≠0 in the set Q ^t _θ by the affine transformation formula Perform image space transformation, where (u, v) is the original image pixel coordinate, (x, y) is the image pixel coordinate after transformation, is the perspective transformation matrix; we get the image set Q ^t ₀ with rotation angle θ=0. 4. The method for generating real-time video of multiple traffic intersections by stratospheric space monitoring according to claim 1 is characterized in that step 30 includes obtaining satellite map data of the target area, extracting all traffic intersection information from the satellite map data, constructing a network topology graph G = (V, E) with all intersections in the target area as vertices, and constructing a ground network coordinate set I of the target intersection. 5. The method for generating real-time video of multiple traffic intersections by stratospheric space monitoring according to claim 1, characterized in that step 40 comprises 41. The high-precision image taken by the camera carried by the drone is used as the input image, and the feature is extracted through the feature pyramid network to obtain the feature map F; 42. Perform image preprocessing on the input image to make it have the same contrast, brightness, etc. as the target intersection grid coordinate set I. 6. The method for generating real-time video of multiple traffic intersections by stratospheric space monitoring according to claim 1, characterized in that step 50 comprises Set the target intersection coordinates x _i , integrate the feature graph F and the node I _i of the target intersection ground grid coordinate set to obtain F _i , and use the iterative matching formula Determine the coordinate information x _i+1 of the next intersection until the current intersection is the last intersection; Where π is the iterative matching strategy, π is optimized by formula Optimize and finally obtain a high-precision image with network coordinates. 7. The method for generating real-time video of multiple traffic intersections by stratospheric space monitoring according to claim 1, characterized in that step 60 comprises 61. Using intersection coordinates as vertices, crop the entire image into several separate intersection coordinate images A ^t _i , where i = (x, y) is the number of intersections with coordinate data obtained by cropping, and t is the timestamp of the current input image; classify the intersection images according to the intersection coordinate data to obtain a set A ^t _i = {A ^t _i丨t＞0}; 62. Repeat step 61 to enrich the set A ^t _i ={A ^t _i丨t＞0}. 8. The method for generating real-time video of multiple traffic intersections by stratospheric space monitoring according to claim 1, characterized in that step 70 comprises 71. Perform secondary image processing on the set A ^t _i to obtain expected image data; 72. The expected image data of different intersections are input into the video encoder in time sequence, and the video encoder converts the image sequence into a video stream. 9. A system for generating real-time video of multiple traffic intersections by stratospheric space monitoring, characterized by: Drone: Once the target intersection is determined, the drone’s position will no longer change; Control equipment: determine the target intersection and select the next target intersection position according to the intersection ground grid coordinates; calculate the rotation angle and rotation direction of the camera head and take high-precision images; Camera equipment: installed on the UAV, with high definition and high frame rate, the camera equipment should be able to adapt to changes in atmospheric conditions and be able to capture real-time images of traffic intersections; Ground server: Establish a ground-side server system to receive, store and process image data transmitted by the drone. The ground server should have high performance and large-capacity storage capabilities to cope with the processing requirements of real-time image data; Transmission equipment: installed on the drone, the drone needs to be equipped with transmission equipment so that the captured images can be transmitted to the ground server in real time for processing and analysis. Wireless signal transmission equipment or satellite communication equipment can be used to achieve data transmission. 10. The system for generating real-time video of multiple traffic intersections by monitoring the stratospheric space according to claim 9 is characterized in that the drone is selected to be suitable for working in the stratospheric space, and the drone needs to have height, stability and safety to ensure that it can fly for a long time in the stratospheric space and carry corresponding equipment.