CN117152210B

CN117152210B - Image dynamic tracking method and related device based on dynamic observation field angle

Info

Publication number: CN117152210B
Application number: CN202311210513.0A
Authority: CN
Inventors: 张雪兵
Original assignee: Shenzhen Euclideon Technology Co ltd
Current assignee: Shenzhen Euclideon Technology Co ltd
Priority date: 2023-09-19
Filing date: 2023-09-19
Publication date: 2024-07-02
Anticipated expiration: 2043-09-19
Also published as: CN117152210A

Abstract

The invention relates to the technical field of image processing, and discloses an image dynamic tracking method and a related device based on a dynamic observation field angle, which are used for improving the accuracy of image dynamic tracking based on the dynamic observation field angle. Comprising the following steps: performing image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data; calibrating an observation field angle of a target vehicle to obtain an initial observation field angle range; performing edge detection on the gray image data to obtain an edge binarization image and an edge point set of the target vehicle; carrying out random Hough transformation on the edge point set to obtain a parameter space corresponding to the edge point set, and carrying out image fitting on the edge binarized image to obtain a target fitting image; and (3) analyzing vehicle point cloud data of the target vehicle to obtain vehicle point cloud data to be registered, and analyzing a tracking range of the initial observation field angle range to obtain an observation field angle tracking range.

Description

Image dynamic tracking method and related device based on dynamic observation field angle

Technical Field

The invention relates to the technical field of image processing, in particular to an image dynamic tracking method based on dynamic observation of a field angle and a related device.

Background

In recent years, the field of computer vision and perception systems has made tremendous progress, making application of autopilot, intelligent transportation, robotic navigation, etc. practical. Among other things, target vehicle tracking technology plays a key role in these applications. The target vehicle tracking is to monitor the surrounding environment in real time through a perception system so as to identify, track and understand vehicles on the road and provide important data and control information for intelligent traffic and automatic driving.

Traditional image processing methods face accuracy challenges when dealing with complex scenes and fast moving vehicles. For example, a fast moving vehicle causes blurring or distortion in the image, thereby reducing the accuracy of target detection and tracking. Sensing systems in the real world typically require the simultaneous processing of a variety of sensor data, such as images and point clouds. Conventional approaches often fail to effectively fuse and analyze such multimodal data, thereby limiting the perceptibility of the system. For applications with high real-time requirements such as automatic driving and intelligent transportation, the traditional method has high calculation cost and is difficult to meet the real-time requirements. In target tracking, matching errors occur due to environmental changes, sensor noise and other factors, resulting in inaccurate estimation of the position and posture of the target vehicle.

Disclosure of Invention

The invention provides an image dynamic tracking method and a related device based on a dynamic observation field angle, which are used for improving the accuracy of image dynamic tracking based on the dynamic observation field angle.

The first aspect of the present invention provides an image dynamic tracking method based on a dynamic observation field angle, the image dynamic tracking method based on the dynamic observation field angle comprising:

acquiring vehicle image data of a target vehicle in a preset range through an image acquisition terminal installed on the preset target vehicle;

Performing image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data;

Calibrating an observation field angle of the target vehicle through the gray image data and the depth image data to obtain an initial observation field angle range;

Performing edge detection on the gray image data to obtain an edge binarization image and an edge point set of the target vehicle;

Carrying out random Hough transformation on the edge point set to obtain a parameter space corresponding to the edge point set, and carrying out image fitting on the edge binarized image based on the parameter space to obtain a target fitting image;

based on the depth image data, carrying out vehicle point cloud data analysis on the target vehicle through the target fitting image to obtain vehicle point cloud data to be registered, and meanwhile, obtaining reference point cloud data;

And carrying out tracking range analysis on the initial observation field angle range based on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain an observation field angle tracking range, and carrying out dynamic tracking on the target vehicle based on the observation field angle tracking range.

With reference to the first aspect, in a first implementation manner of the first aspect of the present invention, the performing image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data includes:

removing image noise from the vehicle image data to obtain denoising image data;

carrying out gray value calculation on the denoising image data to obtain a gray value data set;

Performing first image conversion on the denoising image data through the gray value data set to obtain gray image data corresponding to the vehicle image data;

performing depth value calculation on the denoising image data to obtain a depth value data set;

and performing second image conversion on the denoising image data through the depth value data set to obtain depth image data corresponding to the vehicle image data.

With reference to the first aspect, in a second implementation manner of the first aspect of the present invention, the calibrating the observation field angle of the target vehicle by using the gray image data and the depth image data to obtain an initial observation field angle range includes:

Carrying out data alignment processing on the gray image data and the depth image data to obtain first alignment image data corresponding to the gray image data and second alignment image data corresponding to the depth image data;

Performing first contour segmentation processing on the first aligned image data to obtain a first vehicle contour;

Performing second contour segmentation processing on the second aligned image data to obtain a second vehicle contour;

Performing contour overlapping region analysis on the first vehicle contour and the second vehicle contour to obtain corresponding contour overlapping regions;

Performing contour data analysis on the target vehicle based on the contour overlapping region to obtain vehicle contour data corresponding to the target vehicle;

carrying out vehicle boundary analysis on the vehicle contour data to obtain corresponding vehicle boundary data;

And calibrating the observation field angle of the target vehicle according to the vehicle boundary data to obtain the initial observation field angle range.

With reference to the first aspect, in a third implementation manner of the first aspect of the present invention, the performing edge detection on the gray-scale image data to obtain an edge binarized image of the target vehicle and an edge point set includes:

Performing pixel gradient calculation on the gray image data to obtain a corresponding pixel gradient data set;

Analyzing the maximum gradient region of the pixel gradient data set to obtain a maximum gradient region corresponding to the pixel gradient data set;

Performing non-maximum suppression processing on the gray image data through the pixel gradient data based on the maximum gradient region to obtain candidate image data;

performing edge tracking processing on the candidate image data to obtain a corresponding strong edge pixel set, weak edge pixel set and non-edge pixel set;

Performing position relation analysis on the strong edge pixel set, the weak edge pixel set and the non-edge pixel set to obtain corresponding pixel position relation data;

Performing edge connection processing on the strong edge pixel set, the weak edge pixel set and the non-edge pixel set based on the pixel position relation data to obtain an edge connection image;

and extracting edge points from the edge connection image to obtain the edge point set, and performing binarization processing on the edge connection image to obtain an edge binarization image of the target vehicle.

With reference to the first aspect, in a fourth implementation manner of the first aspect of the present invention, the performing a random hough transform on the edge point set to obtain a parameter space corresponding to the edge point set, and performing image fitting on the edge binarized image based on the parameter space to obtain a target fitted image, where the method includes:

carrying out edge point data numbering on the edge point set to obtain a plurality of edge point numbering data;

randomly sampling the edge point number data to obtain a plurality of sampling edge point numbers;

matching a corresponding plurality of sampling edge point data based on the plurality of sampling edge point numbers;

carrying out parameter estimation on the plurality of sampling edge point data to obtain corresponding estimated parameter data;

Performing parameter projection mapping on the estimated parameter data to obtain a parameter space corresponding to the edge point set, and performing peak analysis on the parameter space to obtain a parameter peak value corresponding to the parameter space;

And carrying out parameter point conversion and fitting on the edge binarized image by the parameter peak value to obtain the target fitting image.

With reference to the first aspect, in a fifth implementation manner of the first aspect of the present invention, the performing a random hough transform on the edge point set to obtain a parameter space corresponding to the edge point set, and performing image fitting on the edge binarized image based on the parameter space to obtain a target fitted image, where the method includes:

With reference to the first aspect, in a sixth implementation manner of the first aspect of the present invention, the performing tracking range analysis on the initial observation field angle range based on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain an observation field angle tracking range, and performing dynamic tracking on the target vehicle based on the observation field angle tracking range includes:

Calculating the vehicle pose of the point cloud data of the vehicle to be registered and the reference point cloud data to obtain initial pose data corresponding to the target vehicle;

performing point cloud segmentation on the point cloud data of the vehicle to be registered to obtain a plurality of sub-point cloud data sets;

Respectively carrying out gesture transformation matrix matching on each sub-point cloud data set to obtain a target gesture transformation matrix corresponding to each sub-point cloud data set;

Based on a target gesture transformation matrix corresponding to each sub-point cloud data set, respectively carrying out gesture coordinate transformation on each sub-point cloud data set to obtain transformed point cloud data;

tracking range analysis is carried out on the initial observation field angle range through a preset kd-tree algorithm, and an initial observation field angle tracking range is obtained;

and carrying out iterative analysis on the initial observation field angle tracking range until the initial observation field angle tracking range meets a preset requirement, obtaining a corresponding target observation field angle tracking range, and carrying out dynamic tracking on the target vehicle based on the observation field angle tracking range.

A second aspect of the present invention provides an image dynamic tracking apparatus based on a dynamic observation field angle, the image dynamic tracking apparatus based on a dynamic observation field angle including:

the acquisition module is used for acquiring vehicle image data of the target vehicle in a preset range through an image acquisition terminal installed on the preset target vehicle;

The conversion module is used for carrying out image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data;

The calibration module is used for calibrating the observation field angle of the target vehicle through the gray image data and the depth image data to obtain an initial observation field angle range;

The detection module is used for carrying out edge detection on the gray image data to obtain an edge binarization image and an edge point set of the target vehicle;

The processing module is used for carrying out random Hough transformation processing on the edge point set to obtain a parameter space corresponding to the edge point set, and carrying out image fitting on the edge binarized image based on the parameter space to obtain a target fitting image;

The analysis module is used for carrying out vehicle point cloud data analysis on the target vehicle through the target fitting image based on the depth image data to obtain vehicle point cloud data to be registered, and meanwhile, obtaining reference point cloud data;

And the tracking module is used for carrying out tracking range analysis on the initial observation field angle range based on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain an observation field angle tracking range and carrying out dynamic tracking on the target vehicle based on the observation field angle tracking range.

A third aspect of the present invention provides an image dynamic tracking apparatus based on a dynamic observation field angle, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the dynamic image tracking device based on the dynamic observation field angle to perform the dynamic image tracking method based on the dynamic observation field angle described above.

A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored therein that, when executed on a computer, cause the computer to perform the above-described dynamic image tracking method based on a dynamic observation field angle.

According to the technical scheme provided by the application, vehicle image data of a target vehicle in a preset range is acquired through an image acquisition terminal installed on the target vehicle; performing image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data; calibrating an observation field angle of a target vehicle through gray image data and depth image data to obtain an initial observation field angle range; performing edge detection on the gray image data to obtain an edge binarization image and an edge point set of the target vehicle; carrying out random Hough transformation on the edge point set to obtain a parameter space corresponding to the edge point set, and carrying out image fitting on the edge binarized image based on the parameter space to obtain a target fitting image; based on the depth image data, carrying out vehicle point cloud data analysis on a target vehicle through a target fitting image to obtain vehicle point cloud data to be registered, and meanwhile, obtaining reference point cloud data; and carrying out tracking range analysis on the initial observation field angle range based on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain an observation field angle tracking range, and carrying out dynamic tracking on the target vehicle based on the observation field angle tracking range. According to the scheme, the image acquisition terminal is used for acquiring the vehicle image data and combining the depth information, so that the target vehicle tracking with high precision can be realized. The image is fitted through the depth information and the parameter space, and the tracking accuracy is improved. The kd-Tree improved algorithm is adopted to accelerate point cloud registration and track the range analysis of the observation field angle, so that the target tracking has instantaneity and is suitable for a high-speed moving scene. Through edge detection and random Hough transformation processing of gray image data and utilization of depth information, matching errors in the tracking process of a target vehicle can be effectively reduced, and tracking accuracy is improved. The target vehicle is calibrated by the gray image data and the depth information, so that the system can be automatically adapted to different viewing angle requirements, and is more flexible and adaptive.

Drawings

FIG. 1 is a schematic diagram of an embodiment of an image dynamic tracking method based on dynamic observation field angle according to an embodiment of the present invention;

FIG. 2 is a flow chart of calibrating an observation angle of view of a target vehicle with gray scale image data and depth image data in an embodiment of the present invention;

FIG. 3 is a flow chart of edge detection of gray scale image data according to an embodiment of the present invention;

fig. 4 is a flowchart of performing a random hough transform process on an edge point set in an embodiment of the present invention;

FIG. 5 is a schematic diagram of an embodiment of an image dynamic tracking apparatus based on dynamic observation field angle according to an embodiment of the present invention;

Fig. 6 is a schematic diagram of an embodiment of an image dynamic tracking apparatus based on a dynamic observation field angle in an embodiment of the present invention.

Detailed Description

The embodiment of the invention provides an image dynamic tracking method and a related device based on a dynamic observation field angle, which are used for improving the accuracy of image dynamic tracking based on the dynamic observation field angle.

The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

For ease of understanding, a specific flow of an embodiment of the present invention is described below, referring to fig. 1, and an embodiment of a method for dynamically tracking an image based on a dynamic observation field angle in an embodiment of the present invention includes:

S101, acquiring vehicle image data of a target vehicle in a preset range through an image acquisition terminal installed on the preset target vehicle;

It can be understood that the execution subject of the present invention may be an image dynamic tracking device based on a dynamic observation field angle, and may also be a terminal or a server, which is not limited herein. The embodiment of the invention is described by taking a server as an execution main body as an example.

Specifically, the server selects an appropriate image capturing terminal to be mounted on the target vehicle. This terminal typically includes a camera, sensors and data storage devices. The server selects high resolution cameras and lidar etc. sensors to obtain high quality image and depth data. These devices need to conform to the size and shape of the target vehicle and be capable of reliable operation under a variety of environmental conditions. When a suitable image acquisition terminal is selected, it needs to be mounted on the target vehicle. This involves mounting cameras and sensors on the front, rear, side or roof of the vehicle. The location and angle of installation should be precisely planned to ensure that the terminal is able to capture the desired image range. In addition, the terminal needs to be connected to a power supply system and a communication network of the vehicle. When the image acquisition terminal is mounted on the target vehicle, it can start acquiring vehicle image data. This process is automatic and can be performed as follows: the camera and the sensor monitor the surrounding environment in real time, including roads, other vehicles and obstacles; the image acquisition terminal stores the image data captured in real time into the storage device; the data may be saved in the form of a sequence of images or a video stream for subsequent analysis and processing. For example, assume that a server is developing an autonomous car and requires acquisition of vehicle image data to train the visual perception system of the vehicle. The server selects a high-resolution camera and a laser radar as the image acquisition terminal. The camera is used to capture images of the surroundings of the vehicle, while the lidar is used to acquire depth information. The server mounts cameras in front of the car to acquire images of the road and traffic conditions ahead. Meanwhile, a laser radar is installed on the top of the vehicle by the server to acquire three-dimensional depth information of the surrounding environment. These devices are connected to the battery system and communication bus of the vehicle for real-time data transmission and power. When the image acquisition terminal is installed, the image acquisition terminal starts to acquire images and depth data in real time. These data are stored in the form of image sequences and depth images in a storage device on the vehicle. Every second, the server captures hundreds of images, each of which contains environmental information about the vehicle surroundings. These image data can then be used for training and testing of the autopilot system to assist the vehicle in identifying and understanding various conditions on the road, thereby enabling safer and more intelligent autopilot. In addition, the image acquisition system can be used in the application fields of traffic monitoring, road condition evaluation, vehicle performance test and the like.

S102, performing image conversion on vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data;

Specifically, the server performs image noise removal on the collected vehicle image data to improve the data quality. This may be achieved by various image processing techniques such as mean filtering, median filtering, gaussian filtering, etc. The goal of noise removal is to reduce random disturbances in the image, making the image clearer and more reliable. For example, suppose that the server has a vehicle equipped with an image acquisition terminal that captures an image of a road, but the image is subject to random noise, such as snow spots in the image. By applying gaussian filtering, the server smoothes the image and removes noise, resulting in de-noised image data. After obtaining the denoised image data, the server calculates the gray value of each pixel. Gray scale images are typically used to represent the brightness information of an image, without containing color information. The calculation of the gray value may be achieved by converting an RGB color image into a gray image. One common approach is to weight average the R, G and B-channel values for each pixel. For example, assume that the server has a denoising image that contains pixel values for red, green, and blue channels. The server calculates the gray value of each pixel using the following formula: gray value=0.299×r+0.587×g+0.114×b. By applying this formula, the server obtains a set of gray value data for each pixel in the image. When the server obtains the gray value data set, a first image conversion may be performed to convert the denoised image data into gray image data. This process is actually to assign the gray value of each pixel to a new image, thereby creating a gray image. Gray scale images are commonly used for image processing tasks such as edge detection, feature extraction, and the like. For example, the server generates a gray image by applying the gray value data set calculated previously to the denoised image. In a gray scale image, the luminance value of each pixel represents gray scale information of a corresponding position in the original image. The server calculates a depth value data set. This may be achieved by lidar, a stereo vision system or other depth sensor. The depth value represents the distance of each pixel in the image from the observer, and is used to generate a depth image that displays distance information of objects in the scene. For example, assume that a server uses lidar to obtain depth information. The lidar emits a laser beam, measures the distance from the sensor to the object, and maps the distance data to pixel locations in the image. In this way, the server obtains a set of depth value data representing distance information for each pixel in the image. Finally, a second image conversion is performed using the depth value data set, converting the de-noised image data into depth image data. Depth images typically represent objects of different distances in gray or pseudo-colors, which are very useful for visual perception and distance measurement. For example, the server generates a depth image by applying the previously calculated depth value data set to the denoised image. The pixel values in the depth image represent the distance from the sensor, which is very important for dynamic tracking and obstacle avoidance of the vehicle.

S103, calibrating an observation field angle of a target vehicle through gray image data and depth image data to obtain an initial observation field angle range;

Specifically, the server performs data alignment processing on the grayscale image data and the depth image data to ensure that they spatially correspond. This may be done by using camera calibration parameters to align the depth image data with the grayscale image data. For example, assume that the server has a set of grayscale images and depth images that are acquired from the vehicle. By using camera calibration parameters, the server aligns the pixels of the depth image with the pixels of the grayscale image to ensure that they are in the same spatial coordinate system. And performing contour segmentation processing on the image data subjected to the alignment processing. This step aims at extracting the contour information of the target vehicle from the image data. For example, using image processing techniques such as Canny edge detection or contour detection algorithms, the server extracts the first vehicle contour and the second vehicle contour from the aligned grayscale images. These profiles will be used for subsequent analysis. After the contours of the first and second vehicles are obtained, it is necessary to analyze their contour overlapping areas. This is to determine overlapping portions of the target vehicle in different images, thereby providing more accurate vehicle position information. For example, by comparing the first vehicle profile and the second vehicle profile, the overlap region therebetween may be determined, which regions correspond to portions of the target vehicle. These overlapping areas can be used for further analysis. Vehicle contour data analysis is required for the contour overlapping region. The object is to extract contour information of a target vehicle from a region of overlap to determine its shape and position. For example, by analyzing the contour in the overlapping region, vehicle model contour data of the target vehicle, including the contour shape and size of the vehicle, can be obtained. These data will aid in further viewing angle calibration. Finally, the boundary information of the target vehicle is determined by vehicle boundary analysis of the vehicle profile data. This step is to define the outline of the target vehicle more accurately. For example, by analyzing the vehicle profile data, the server determines the outer boundary of the target vehicle, including the front, rear, and sides of the vehicle. This information will be used for calibration of the observation field angle.

S104, carrying out edge detection on the gray image data to obtain an edge binarization image and an edge point set of the target vehicle;

Specifically, pixel gradient calculation is required for the gradation image data. The purpose is to obtain gradient information for each pixel in the image for subsequent edge detection and analysis. For example, assume that the server has a grayscale image of a road that includes a vehicle. The server uses a Sobel operator to carry out convolution operation on the image, and calculates the horizontal gradient and the vertical gradient of each pixel point. This will produce two gradient images representing gradient information in the horizontal and vertical directions, respectively. A maximum gradient region analysis of the pixel gradient data set is required to determine the most significant gradient change region in the image. These areas generally correspond to edges. For example, the server determines the maximum gradient region by applying thresholding techniques or using region growing algorithms. These areas will contain the main edge features in the image, such as the vehicle contour. To refine the edges and avoid the problem of multiple responses, non-maximum suppression processing of the maximum gradient region is required. The goal of this step is to preserve local maxima in the gradient direction. For example, within the maximum gradient region, the gradient values around each pixel are checked, and the local maximum value in the gradient direction is retained. This will generate a refined edge image in which only the locally strongest edge response is retained. Edge tracking processing is required for the non-maxima suppressed image to connect edge segments and obtain a continuous edge. For example, using an edge tracking algorithm, a starting point is selected from the non-maximal suppressed image and tracked along the edge direction until the edge breaks or a complete edge line is formed. This will generate a set of consecutive edge line segments. After the edge line segments are obtained, the pixel positional relationship needs to be analyzed to determine which pixels belong to strong edges, which belong to weak edges, and which belong to non-edges. For example, by comparing the gradient intensity and direction of each pixel, the pixels can be divided into three classes: strong edge pixels, weak edge pixels, and non-edge pixels. Strong edge pixels typically have a pronounced gradient peak, while weak edge pixels have a smaller gradient. Based on the pixel positional relationship data, an edge connection process is required to connect weak edge pixels to strong edge pixels, thereby forming a complete edge. For example, weak edge pixels are connected to strong edge pixels using an edge connection algorithm to construct a continuous edge line. This will generate an edge connection image representing the contour of the target vehicle. Finally, edge points are extracted from the edge connection image, the edge points constituting a set of contour points of the target vehicle. For example, by applying an edge point extraction algorithm on the edge connection image, edge points of the target vehicle contour can be extracted. These points represent edge features of the target vehicle.

S105, carrying out random Hough transformation on the edge point set to obtain a parameter space corresponding to the edge point set, and carrying out image fitting on the edge binarized image based on the parameter space to obtain a target fitting image;

Specifically, the edge point sets are numbered for subsequent processing. Each edge point is assigned a unique identifier or number. For example, assume that the server has a set of edge points for a vehicle contour. For each point, the server assigns it a unique number for subsequent data processing and tracking. A set of edge points is randomly selected from the numbered data for sampling. These sampling points will be used for subsequent parameter estimation. For example, a small portion of points from the edge point set of the vehicle contour is randomly selected as the sampling points. These points will represent the entire contour for parameter estimation. And establishing the corresponding relation between the sampling points and the actual data by matching the numbers of the sampling points with the actual edge point data. For example, a randomly selected sampling point is matched to the original set of edge points to determine which actual points correspond to each sampling point. For each set of matched sampling points, a parameter estimation is performed to obtain parameter data describing the target profile. For example, using the position information of the sampling points, parameter estimation, such as fitting an ellipse, straight line, etc., model may be performed to describe the contour of the target vehicle. The parameter data is mapped to a parameter space, forming a peak in the parameter space. For example, the parameter data is projected into a parameter space, where each parameter combination corresponds to a peak. These peaks represent the profile parameters present in the parameter space. Peak analysis is performed in the parameter space to determine which combinations of parameters have the highest peak, which parameters will correspond to the best fit. For example, finding peaks in the parameter space is typically accomplished by finding the peak with the highest value. These peaks represent the most contour parameters. And finally, carrying out parameter point conversion and fitting on the edge binarized image by using the peak value parameters so as to obtain a fitted image of the target. For example, the highest peak is transformed into the actual contour point according to its parameters, and then curve or line segment fitting is performed to obtain a fitted image of the target, i.e. an approximate representation of the vehicle contour.

S106, based on the depth image data, carrying out vehicle point cloud data analysis on a target vehicle through a target fitting image to obtain vehicle point cloud data to be registered, and meanwhile, obtaining reference point cloud data;

Specifically, semantic feature fusion is required for depth image data. The purpose is to combine the depth information with the semantic information to obtain a depth semantic feature image. For example, assume that the server has a depth image that includes the target vehicle. By combining the depth image with the semantic segmentation result, a semantic label such as a vehicle or a background can be assigned to each pixel. This will generate a depth semantic feature image, where each pixel has depth information and a semantic label. The depth semantic feature image is used for constructing a point cloud space, and image pixels are mapped to three-dimensional point cloud coordinates. For example, pixels in an image may be converted to three-dimensional points by mapping depth information and semantic tags to a point cloud space. This will generate point cloud data representing the shape and location of the target vehicle in three-dimensional space. After the point cloud space data are obtained, point cloud data analysis is needed to extract relevant information of the target vehicle, and vehicle point cloud data to be registered are further obtained. For example, in the point cloud data, various analysis tasks such as vehicle shape analysis, position estimation, motion detection, and the like may be performed. This will help determine the characteristics of the target vehicle and provide a basis for subsequent tracking. Meanwhile, reference point cloud data, which is previously recorded point cloud data of a vehicle or a scene, needs to be acquired for comparison and matching with the point cloud data of the target vehicle. For example, the reference point cloud data is typically pre-acquired or from map data. The method is used for comparing with the point cloud of the vehicle to be registered so as to identify the position and the gesture of the target vehicle.

And S107, carrying out tracking range analysis on the initial observation field angle range based on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain an observation field angle tracking range, and carrying out dynamic tracking on the target vehicle based on the observation field angle tracking range.

It should be noted that, vehicle pose calculation needs to be performed on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain initial pose data of the target vehicle. For example, assuming that the server has point cloud data to be registered and reference point cloud data of one vehicle, by comparing the features of the two, the initial pose of the target vehicle, that is, its position and orientation in three-dimensional space, can be calculated. The point cloud segmentation needs to be carried out on the point cloud data of the vehicle to be registered, and the point cloud data are divided into a plurality of sub-point cloud data sets. These sub-point clouds correspond to different portions or features of the vehicle. For example, the point cloud data of the vehicle is divided into sub-point cloud data sets of a roof, a door, a window, and the like, so that subsequent pose matching is performed more accurately. And respectively carrying out gesture transformation matrix matching on each sub-point cloud data set to determine the relative gesture transformation of each sub-point cloud data set and the reference point cloud. For example, by comparing the features of each sub-point cloud dataset to the reference point cloud, a pose transformation matrix for each sub-point cloud dataset relative to the reference point cloud may be calculated, which will be used for subsequent pose coordinate transformations. And carrying out gesture coordinate transformation on each sub-point cloud data set based on a target gesture transformation matrix corresponding to each sub-point cloud data set to obtain transformed point cloud data. For example, each sub-point cloud dataset is transformed from its local coordinate system to a global coordinate system using a pose transformation matrix to ensure that they have a consistent coordinate system with the reference point cloud. And carrying out tracking range analysis on the initial observation field angle range through a preset kd-tree algorithm so as to determine the approximate position of the target vehicle. For example, the point cloud data is spatially segmented using a kd-Tree algorithm to obtain an initial position estimation range of the target vehicle. This range will include the target location. And finally, carrying out iterative analysis on the tracking range of the initial observation field angle until the preset requirement is met. This will help further narrow the observation field angle tracking range to achieve more accurate dynamic tracking. For example, through multiple iterations, the observation field angle tracking range is continuously adjusted to ensure that the position and attitude estimation of the target vehicle is more accurate. Different optimization algorithms may be used in the iterative process to improve accuracy.

In the embodiment of the application, the image data of the target vehicle in a preset range is acquired through an image acquisition terminal installed on the target vehicle; performing image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data; calibrating an observation field angle of a target vehicle through gray image data and depth image data to obtain an initial observation field angle range; performing edge detection on the gray image data to obtain an edge binarization image and an edge point set of the target vehicle; carrying out random Hough transformation on the edge point set to obtain a parameter space corresponding to the edge point set, and carrying out image fitting on the edge binarized image based on the parameter space to obtain a target fitting image; based on the depth image data, carrying out vehicle point cloud data analysis on a target vehicle through a target fitting image to obtain vehicle point cloud data to be registered, and meanwhile, obtaining reference point cloud data; and carrying out tracking range analysis on the initial observation field angle range based on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain an observation field angle tracking range, and carrying out dynamic tracking on the target vehicle based on the observation field angle tracking range. According to the scheme, the image acquisition terminal is used for acquiring the vehicle image data and combining the depth information, so that the target vehicle tracking with high precision can be realized. The image is fitted through the depth information and the parameter space, and the tracking accuracy is improved. The kd-Tree improved algorithm is adopted to accelerate point cloud registration and track the range analysis of the observation field angle, so that the target tracking has instantaneity and is suitable for a high-speed moving scene. Through edge detection and random Hough transformation processing of gray image data and utilization of depth information, matching errors in the tracking process of a target vehicle can be effectively reduced, and tracking accuracy is improved. The target vehicle is calibrated by the gray image data and the depth information, so that the system can be automatically adapted to different viewing angle requirements, and is more flexible and adaptive.

In a specific embodiment, the process of executing step S102 may specifically include the following steps:

(1) Removing image noise from the vehicle image data to obtain denoised image data;

(2) Carrying out gray value calculation on the denoising image data to obtain a gray value data set;

(3) Performing first image conversion on the denoising image data through the gray value data set to obtain gray image data corresponding to the vehicle image data;

(4) Performing depth value calculation on the denoising image data to obtain a depth value data set;

(5) And performing second image conversion on the denoising image data through the depth value data set to obtain depth image data corresponding to the vehicle image data.

Specifically, the image generally contains various types of noise, such as random noise, pretzel noise, and the like. The goal of noise removal is to reduce or eliminate this noise to obtain a clearer image. Common image noise removal methods include smoothing filters (e.g., mean filtering, gaussian filtering, median filtering), wavelet noise reduction, and the like. For example, suppose the server has an image of a vehicle that contains some random noise points and salt and pepper noise (black and white spots). The server applies a median filter that considers the neighborhood pixels around each pixel and replaces the value of the current pixel with the median of the neighborhood pixel values. This will help to remove noise, resulting in a denoised image. After obtaining the denoised image, the gray value of each pixel is calculated. The gray value is a single value representing the brightness or gray level of a pixel. A grayscale image is a conversion of a color image into a single channel image, where each pixel represents its brightness. The gray value is typically calculated using the following formula: gray value = 0.299 x red channel + 0.587 x green channel + 0.114 x blue channel this formula considers the weights of the different color channels to better approximate the perception of an image by the human eye. The calculated gray value data set will constitute gray image data of the vehicle image. For example, for each pixel of the denoised image, the server uses the above gray value calculation formula to calculate the corresponding gray value from the values of its red, green and blue channels. This will produce a single-channel gray scale image, where each pixel represents brightness. If depth information is available, a depth value may be calculated. The depth value represents the distance of each pixel in the image to the camera or sensor for creating a depth image. Depth images typically store depth information in floating point format. The manner in which the depth values are calculated depends on the type of depth sensor and the data format. For example, the lidar may measure range directly, while other sensors provide depth images. For example, assume that a server uses lidar to obtain depth information. By emitting a laser beam to various points in the image and measuring the return time, the depth value of each pixel can be calculated. This will generate a depth value data set for constructing the depth image.

In a specific embodiment, as shown in fig. 2, the process of performing step S103 may specifically include the following steps:

S201, carrying out data alignment processing on gray image data and depth image data to obtain first alignment image data corresponding to the gray image data and second alignment image data corresponding to the depth image data;

S202, performing first contour segmentation processing on first aligned image data to obtain a first vehicle contour;

S203, performing second contour segmentation processing on the second aligned image data to obtain a second vehicle contour;

s204, analyzing the contour overlapping areas of the first vehicle contour and the second vehicle contour to obtain corresponding contour overlapping areas;

S205, analyzing contour data of the target vehicle based on the contour overlapping area to obtain vehicle contour data corresponding to the target vehicle;

S206, carrying out vehicle boundary analysis on the vehicle contour data to obtain corresponding vehicle boundary data;

s207, calibrating the observation field angle of the target vehicle through the vehicle boundary data to obtain an initial observation field angle range.

The gradation image data and the depth image data are subjected to data alignment processing to ensure that they correspond in the same coordinate system. This usually requires consideration of parameters inside and outside the camera, and distortion correction of the image. The data alignment process will obtain first aligned image data corresponding to the gray image data and second aligned image data corresponding to the depth image data. And performing first contour segmentation processing on the first aligned image data to extract contour information of the first vehicle. Contour segmentation may be implemented using an edge detection algorithm (e.g., canny operator) or an image segmentation algorithm (e.g., watershed algorithm). This will create a profile of the first vehicle. Similarly, a second contour segmentation process is performed on the second aligned image data to extract contour information of the second vehicle. Again, this may be done using edge detection or image segmentation algorithms. After the contours of the first vehicle and the second vehicle are obtained, analysis of the contour overlapping region may be performed. The purpose is to find the area where the two contours overlap. These areas may help determine relationships and intersections between the target vehicles. And analyzing the contour data of the target vehicle based on the contour overlapping area. This will help determine the profile characteristics of each vehicle, such as location, size, shape, etc. And carrying out vehicle boundary analysis on the contour data of the vehicle. This step involves determining the boundary of the vehicle, i.e. the outer edge of the vehicle contour. This may be achieved by edge detection, contour fitting or morphological operations, etc. And finally, calibrating the observation field angle of the target vehicle through the vehicle boundary data. This step involves measuring the angle and position of the vehicles in the image to determine their position and orientation in the field of view. This helps to establish an initial observation field angle range. For example, assume that a server is processing a set of vehicle images, including two vehicles. Through the data alignment process, the server ensures that the grayscale image and the depth image correspond in the same coordinate system. The server then extracts the contours of each vehicle using image segmentation techniques and finds their overlapping areas. By analyzing these coincident regions and contour data, the server determines the location and shape of each vehicle. Finally, the server performs calibration of the observation field angle through the vehicle boundary data, and an initial observation field angle range is obtained.

In a specific embodiment, as shown in fig. 3, the process of executing step S104 may specifically include the following steps:

S301, performing pixel gradient calculation on gray image data to obtain a corresponding pixel gradient data set;

S302, analyzing a maximum gradient region of the pixel gradient data set to obtain a maximum gradient region corresponding to the pixel gradient data set;

S303, performing non-maximum suppression processing on gray image data through pixel gradient data based on the maximum gradient region to obtain candidate image data;

S304, carrying out edge tracking processing on the candidate image data to obtain a corresponding strong edge pixel set, a weak edge pixel set and a non-edge pixel set;

S305, carrying out position relation analysis on the strong edge pixel set, the weak edge pixel set and the non-edge pixel set to obtain corresponding pixel position relation data;

S306, performing edge connection processing on the strong edge pixel set, the weak edge pixel set and the non-edge pixel set based on the pixel position relation data to obtain an edge connection image;

s307, extracting edge points of the edge connection image to obtain an edge point set, and simultaneously, performing binarization processing on the edge connection image to obtain an edge binarization image of the target vehicle.

The pixel gradient calculation is performed on the gradation image data. The pixel gradient represents the gray scale rate of change of each pixel in the image, and typically a convolution kernel such as Sobel, prewitt, or Scharr, is used to calculate the gradient in the horizontal and vertical directions. This will produce a gradient magnitude image in which each pixel corresponds to its local gradient. The pixel gradient data set is analyzed to determine a maximum gradient region. The region of maximum gradient generally represents edges or regions of high variation in the image, which is important for detecting the contour of the vehicle. Based on the maximum gradient region, the image is processed using a technique of Non-maximum suppression (Non-Maximum Suppression, NMS). The NMS helps refine the edges, leaving only the pixels with the largest gradient, resulting in candidate image data. And carrying out edge tracking processing on the candidate image data. This step helps to connect edge pixels and extract successive edges. Edge tracking may use different algorithms such as Canny edge detection or Hough transform. For the tracked edge pixel set, a positional relationship analysis is necessary. This may help identify the geometry of the edges, such as straight lines, curves or other shapes, and determine the relationship between them. Based on the pixel positional relationship data, edge connection processing is performed to connect adjacent edge pixels into a continuous edge. This can be achieved by analyzing the distance and direction between pixels. Finally, an edge point set is extracted from the connected edges, and binarization processing is performed to obtain an edge binarized image of the target vehicle. In this step, the pixels are classified as either strong edge pixels, weak edge pixels, or non-edge pixels, typically using a thresholding technique to accomplish binarization. For example, assuming that the server has a grayscale image of a vehicle, the server obtains a gradient magnitude image by pixel gradient calculation. The server then finds the main edge region in the image by maximum gradient region analysis. The server applies non-maximum suppression processing to obtain candidate image data. Through the edge tracking process, the server connects edges in the candidate image data into a continuous contour. Then, the server performs positional relationship analysis, and determines geometric features of the vehicle contour. Finally, through the edge connection process, the server obtains an edge binarized image of the vehicle in which pixels are classified into a portion belonging to the contour of the vehicle and a non-contour portion.

In a specific embodiment, as shown in fig. 4, the process of performing step S105 may specifically include the following steps:

s401, numbering edge points of the edge point set to obtain a plurality of edge point numbering data;

S402, randomly sampling the edge point number data to obtain a plurality of sampling edge point numbers;

s403, matching a plurality of corresponding sampling edge point data based on a plurality of sampling edge point numbers;

S404, carrying out parameter estimation on the plurality of sampling edge point data to obtain corresponding estimated parameter data;

s405, performing parameter projection mapping on the estimated parameter protector to obtain a parameter space corresponding to the edge point set, and simultaneously, performing peak analysis on the parameter space to obtain a parameter peak corresponding to the parameter space;

s406, performing parameter point conversion and fitting on the parameter peak value and the edge binarization image to obtain a target fitting image.

Specifically, the set of edge points is numbered so that each edge point is uniquely identified. This facilitates subsequent processing and tracking. Randomly sampling from the edge point set after numbering to obtain a plurality of sampling edge point numbers. The purpose of sampling is to reduce the computational complexity while still being able to represent the entire edge. And matching the sampling edge point number with the corresponding edge point data. This will provide each sampling edge point with its associated data. For each sampled edge point data, a parameter estimation is performed. This involves fitting a straight line, curve or other mathematical model to describe the shape and direction of the edge. The estimated parameter projections are mapped into a parameter space. The parameter space is typically a multidimensional space in which all edge point parameters are contained. Peak analysis is performed in the parameter space to find peaks in the parameter space. These peaks represent the primary cluster areas of edge point data, which help determine the shape and direction of the edge. Based on the peaks in the parameter space, these parameter points are converted back to an edge binarized image to obtain a target fit image. This process typically involves regenerating a fitted image and matching it to the original image to ensure correctness. For example, assume that the server has an edge binarized image that contains the contour of the vehicle. The server numbers edge points in the image and then randomly samples some points from them. The server matches the sampling points with their corresponding edge point data and estimates parameters such as position, slope, etc. for each sampling point. The server then projects these parameters into the parameter space and performs a peak analysis in the parameter space to find the dominant parameter peak. Finally, by converting these parameter points back into edge binarized images and fitting them, the server obtains a target fitted image that reflects the shape of the vehicle contour.

In a specific embodiment, the process of executing step S106 may specifically include the following steps:

(1) Carrying out semantic feature fusion on the target fitting image by the depth image data to obtain a depth semantic feature image corresponding to the target fitting image;

(2) Performing point cloud space construction on the target fitting image based on the depth semantic feature image to obtain point cloud space data corresponding to the target vehicle;

(3) And carrying out point cloud data analysis on the target vehicle based on the point cloud space data to obtain point cloud data of the vehicle to be registered, and simultaneously, obtaining reference point cloud data.

Specifically, semantic information is extracted from the depth image data. This can be achieved by semantic segmentation by a deep learning model such as convolutional neural network. Semantic segmentation may assign each pixel in the depth image to a particular semantic category, such as a vehicle surface, road, building, etc. This will generate a depth semantic feature image where each pixel contains not only depth information but also its semantic category information. Based on the depth semantic feature image, point cloud space data of the target vehicle can be constructed. This process involves converting the depth information of each pixel into one point in a point cloud and assigning each point a corresponding semantic class label. This will generate a point cloud data set that contains the three-dimensional shape of the target vehicle and semantic information related thereto. When the point cloud space data is obtained, the point cloud data analysis can be performed on the target vehicle. This includes detecting and segmenting the target vehicle point cloud, and extracting key features such as contours, geometries, etc. of the vehicle. This analysis may also be used to identify specific portions of the target vehicle, such as the head, tail, etc. Meanwhile, reference point cloud data, which is point cloud data of a vehicle having a known position and shape acquired in advance, or data acquired from other sensors or sources, may be acquired. The reference point cloud data is typically used as a comparison basis for pose estimation and registration of the target vehicle. For example, assuming that the server has a depth image of a vehicle traveling on a road, the server uses a depth learning model to perform semantic segmentation, assigning each pixel in the image to a class of vehicle, road, etc. This will generate a deep semantic feature image containing depth information and semantic category information for the vehicle. The server converts the depth semantic feature image into point cloud data, maps the depth information of each pixel to one point in the point cloud, and distributes a corresponding semantic label for each point. In this way, the server obtains the point cloud space data of the target vehicle, wherein the three-dimensional shape and semantic information of the vehicle are included. Then, the server analyzes the point cloud data, detects characteristics such as the outline and the geometric shape of the vehicle, and extracts key information. At the same time, the server obtains reference point cloud data, which may be data of a known vehicle model or other source.

In a specific embodiment, the process of executing step S107 may specifically include the following steps:

(1) Performing vehicle pose calculation on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain initial pose data corresponding to the target vehicle;

(2) Performing point cloud segmentation on point cloud data of a vehicle to be registered to obtain a plurality of sub-point cloud data sets;

(3) Respectively carrying out gesture transformation matrix matching on each sub-point cloud data set to obtain a target gesture transformation matrix corresponding to each sub-point cloud data set;

(4) Based on a target gesture transformation matrix corresponding to each sub-point cloud data set, respectively carrying out gesture coordinate transformation on each sub-point cloud data set to obtain transformed point cloud data;

(5) Tracking range analysis is carried out on the initial observation field angle range through a preset kd-tree algorithm, and an initial observation field angle tracking range is obtained;

(6) And carrying out iterative analysis on the initial observation field angle tracking range until the initial observation field angle tracking range meets the preset requirement, obtaining a corresponding target observation field angle tracking range, and carrying out dynamic tracking on the target vehicle based on the observation field angle tracking range.

Specifically, initial pose data of the target vehicle is calculated using the vehicle point cloud data to be registered and the reference point cloud data. This may be achieved by a point cloud registration algorithm, such as ITERATIVE CLOSEST POINT (ICP) algorithm. The ICP algorithm will attempt to align the point cloud to be registered with the reference point cloud to estimate the position and pose of the vehicle. The vehicle point cloud data to be registered is segmented into a plurality of sub point cloud data sets. This may be done by distinguishing between different parts of the vehicle or by dividing according to the point cloud density. And for each sub-point cloud data set, matching the gesture transformation matrix to find a gesture transformation matrix which is most matched with the reference point cloud. This may be accomplished using various point cloud registration algorithms to ensure that each sub-point cloud is aligned with a reference point cloud. Based on the found target pose transformation matrix, each sub-point cloud dataset is pose coordinate transformed to align them with the reference point cloud. This will generate transformed point cloud data. Tracking range analysis is performed on the initial observation field angle range by using a preset kd-Tree algorithm or other space analysis methods. This can help determine the position and direction of the target vehicle and narrow the search range to improve efficiency. And continuously adjusting pose estimation by iteratively analyzing the tracking range of the initial observation field angle until the preset requirement is met. When the target observation field angle tracking range is determined, the target vehicle can be dynamically tracked based on the range. For example, assuming a vehicle is traveling on a city street, the server obtains point cloud data of the vehicle using a lidar or a camera. The server aligns the point cloud to be registered with the reference point cloud of the urban street acquired in advance by using an ICP algorithm, so that initial pose data of the vehicle are obtained. The server then segments the vehicle point cloud data into multiple sub-point cloud data sets, such as a head, a body, and a tail. For each sub-point cloud dataset, the server uses ICP or other point cloud registration algorithm to find the optimal pose transformation matrix to align it with the reference point cloud. The server performs an attitude coordinate transformation on each sub-point cloud data set, and converts them into a coordinate system of the reference point cloud, so that they are completely aligned with the reference point cloud. In the tracking stage, the server uses kd-tree or other algorithms to analyze the space of the urban street, determines the position of the vehicle, and then iteratively adjusts pose estimation to reduce the search range and finally determine the position and the pose of the target vehicle. This information can be used for dynamic tracking to track the movement of vehicles on city streets.

The above describes the image dynamic tracking method based on the dynamic observation field angle in the embodiment of the present invention, and the following describes the image dynamic tracking device based on the dynamic observation field angle in the embodiment of the present invention, referring to fig. 5, one embodiment of the image dynamic tracking device based on the dynamic observation field angle in the embodiment of the present invention includes:

The acquisition module 501 is used for acquiring vehicle image data of a preset target vehicle within a preset range through an image acquisition terminal installed on the preset target vehicle;

The conversion module 502 is configured to perform image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data;

A calibration module 503, configured to calibrate an observation field angle of the target vehicle according to the grayscale image data and the depth image data, so as to obtain an initial observation field angle range;

the detection module 504 is configured to perform edge detection on the grayscale image data to obtain an edge binarized image and an edge point set of the target vehicle;

The processing module 505 is configured to perform random hough transform processing on the edge point set to obtain a parameter space corresponding to the edge point set, and perform image fitting on the edge binarized image based on the parameter space to obtain a target fitting image;

the analysis module 506 is configured to perform vehicle point cloud data analysis on the target vehicle through the target fitting image based on the depth image data to obtain vehicle point cloud data to be registered, and obtain reference point cloud data at the same time;

The tracking module 507 is configured to perform tracking range analysis on the initial observation field angle range based on the point cloud data of the vehicle to be registered and the reference point cloud data, obtain an observation field angle tracking range, and dynamically track the target vehicle based on the observation field angle tracking range.

Through the cooperative cooperation of the components, vehicle image data of the target vehicle in a preset range are acquired through an image acquisition terminal installed on the target vehicle; performing image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data; calibrating an observation field angle of a target vehicle through gray image data and depth image data to obtain an initial observation field angle range; performing edge detection on the gray image data to obtain an edge binarization image and an edge point set of the target vehicle; carrying out random Hough transformation on the edge point set to obtain a parameter space corresponding to the edge point set, and carrying out image fitting on the edge binarized image based on the parameter space to obtain a target fitting image; based on the depth image data, carrying out vehicle point cloud data analysis on a target vehicle through a target fitting image to obtain vehicle point cloud data to be registered, and meanwhile, obtaining reference point cloud data; and carrying out tracking range analysis on the initial observation field angle range based on the point cloud data of the vehicle to be registered and the reference point cloud data to obtain an observation field angle tracking range, and carrying out dynamic tracking on the target vehicle based on the observation field angle tracking range. According to the scheme, the image acquisition terminal is used for acquiring the vehicle image data and combining the depth information, so that the target vehicle tracking with high precision can be realized. The image is fitted through the depth information and the parameter space, and the tracking accuracy is improved. The kd-Tree improved algorithm is adopted to accelerate point cloud registration and track the range analysis of the observation field angle, so that the target tracking has instantaneity and is suitable for a high-speed moving scene. Through edge detection and random Hough transformation processing of gray image data and utilization of depth information, matching errors in the tracking process of a target vehicle can be effectively reduced, and tracking accuracy is improved. The target vehicle is calibrated by the gray image data and the depth information, so that the system can be automatically adapted to different viewing angle requirements, and is more flexible and adaptive.

Fig. 5 above describes the image dynamic tracking apparatus based on the dynamic observation field angle in the embodiment of the present invention in detail from the point of view of the modularized functional entity, and the image dynamic tracking device based on the dynamic observation field angle in the embodiment of the present invention is described in detail from the point of view of hardware processing below.

Fig. 6 is a schematic structural diagram of an image dynamic tracking apparatus based on a dynamic observation field angle according to an embodiment of the present invention, where the image dynamic tracking apparatus 600 based on a dynamic observation field angle may have relatively large differences due to different configurations or performances, and may include one or more processors (centralprocessingunits, CPUs) 610 (e.g., one or more processors) and a memory 620, one or more storage mediums 630 (e.g., one or more mass storage devices) storing applications 633 or data 632. Wherein the memory 620 and the storage medium 630 may be transitory or persistent storage. The program stored on the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations in the image dynamic tracking apparatus 600 based on the dynamic observation angle of view. Still further, the processor 610 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the image dynamic tracking apparatus 600 based on the dynamic observation angle of view.

The dynamic image tracking device 600 based on dynamically observed field of view may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input/output interfaces 660, and/or one or more operating systems 631, such as WindowsServe, macOSX, unix, linux, freeBSD, and the like. It will be appreciated by those skilled in the art that the dynamic image tracking apparatus structure based on dynamic viewing angles shown in fig. 6 does not constitute a limitation of the dynamic image tracking apparatus based on dynamic viewing angles, and may include more or less components than those illustrated, or may combine certain components, or may be arranged in different components.

The invention also provides an image dynamic tracking device based on the dynamic observation field angle, which comprises a memory and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the steps of the image dynamic tracking method based on the dynamic observation field angle in the above embodiments.

The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and may also be a volatile computer readable storage medium, where instructions are stored in the computer readable storage medium, when the instructions are executed on a computer, cause the computer to perform the steps of the dynamic tracking method for an image based on a dynamic observation field angle.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

The integrated units, if implemented in the form of software functional units and sold or passed as separate products, may be stored in a computer readable storage medium. Based on the understanding that the technical solution of the present invention may be embodied in essence or in a part contributing to the prior art or in whole or in part in the form of a software product stored in a storage medium, comprising instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (randomacceSmemory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1.An image dynamic tracking method based on dynamic observation field angle is characterized by comprising the following steps:

2. The method for dynamically tracking an image based on a dynamic observation field angle according to claim 1, wherein the performing image conversion on the vehicle image data to obtain gray image data and depth image data corresponding to the vehicle image data comprises:

3. The method for dynamically tracking an image based on a dynamic observation field of view according to claim 1, wherein the step of calibrating the observation field of view of the target vehicle by the grayscale image data and the depth image data to obtain an initial observation field of view range comprises:

4. The method for dynamically tracking an image based on a dynamic observation field angle according to claim 1, wherein the performing edge detection on the grayscale image data to obtain an edge binarized image and an edge point set of the target vehicle comprises:

5. The method for dynamically tracking an image based on a dynamic observation field angle according to claim 1, wherein the performing a random hough transform on the edge point set to obtain a parameter space corresponding to the edge point set, performing image fitting on the edge binarized image based on the parameter space to obtain a target fitting image, includes:

6. The method for dynamically tracking images based on dynamic observation field angles according to claim 1, wherein the step of analyzing vehicle point cloud data of the target vehicle based on the depth image data through the target fitting image to obtain vehicle point cloud data to be registered, and simultaneously obtaining reference point cloud data comprises the steps of:

carrying out semantic feature fusion on the target fitting image by the depth image data to obtain a depth semantic feature image corresponding to the target fitting image;

performing point cloud space construction on the target fitting image based on the depth semantic feature image to obtain point cloud space data corresponding to the target vehicle;

And carrying out point cloud data analysis on the target vehicle based on the point cloud space data to obtain the point cloud data of the vehicle to be registered, and simultaneously, obtaining the reference point cloud data.

7. The dynamic image tracking method based on a dynamic observation field angle according to claim 1, wherein the performing tracking range analysis on the initial observation field angle range based on the vehicle point cloud data to be registered and the reference point cloud data to obtain an observation field angle tracking range, and performing dynamic tracking on the target vehicle based on the observation field angle tracking range comprises:

8. An image dynamic tracking device based on a dynamic observation field angle, characterized in that the image dynamic tracking device based on a dynamic observation field angle comprises:

9. An image dynamic tracking apparatus based on a dynamic observation field angle, characterized by comprising: a memory and at least one processor, the memory having instructions stored therein;

The at least one processor invokes the instructions in the memory to cause the dynamic viewing angle based image dynamic tracking device to perform the dynamic viewing angle based image dynamic tracking method of any of claims 1-7.

10. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the dynamic image tracking method based on dynamic observation field angle according to any one of claims 1-7.