CN113688777B - Airport pavement plane real-time detection method based on embedded CPU - Google Patents
Airport pavement plane real-time detection method based on embedded CPU Download PDFInfo
- Publication number
- CN113688777B CN113688777B CN202111041254.4A CN202111041254A CN113688777B CN 113688777 B CN113688777 B CN 113688777B CN 202111041254 A CN202111041254 A CN 202111041254A CN 113688777 B CN113688777 B CN 113688777B
- Authority
- CN
- China
- Prior art keywords
- real
- tag
- network
- aircraft
- establishing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011897 real-time detection Methods 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000012549 training Methods 0.000 claims abstract description 56
- 238000001514 detection method Methods 0.000 claims abstract description 49
- 238000013135 deep learning Methods 0.000 claims abstract description 22
- 238000012360 testing method Methods 0.000 claims abstract description 13
- 230000004927 fusion Effects 0.000 claims abstract description 10
- 238000005070 sampling Methods 0.000 claims description 8
- 230000033001 locomotion Effects 0.000 claims description 7
- 238000012937 correction Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 4
- 238000011478 gradient descent method Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 4
- 101100377706 Escherichia phage T5 A2.2 gene Proteins 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 3
- 238000004364 calculation method Methods 0.000 abstract description 2
- 238000000605 extraction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of target detection, and particularly relates to an airport runway surface plane real-time detection method based on an embedded CPU. The specific technical scheme is as follows: the method comprises the steps of establishing an original training data set and two types of original tag sets of a left aircraft and a right aircraft, performing left-right overturning and repeated cutting on collected video pictures, copying corrected tag data, establishing a final training data set and a final tag set, establishing a deep learning target detection two types of network frames, training to obtain model parameters under the minimum test error, and establishing an airport taxiway aircraft real-time detection model. The built deep learning target detects the multi-scale fusion of the extracted features of the two types (left-hand aircraft and right-hand aircraft) of network frames, only outputs one prediction layer, simplifies model parameter while guaranteeing model prediction precision, and greatly reduces the consumption of calculation resources in training and testing processes.
Description
Technical Field
The invention belongs to the technical field of target detection, and particularly relates to an airport runway surface plane real-time detection method based on an embedded CPU.
Background
In recent years, with the continuous development of artificial intelligence, the wide application of deep learning in the field of computer vision has greatly updated image recognition and target detection technologies. The extraction of the target features is more accurate from the traditional manual feature extraction to the image feature extraction based on deep learning, and the precision of target detection and recognition is greatly improved.
The current target detection algorithms based on deep learning can be divided into two kinds, one is a first-order target detection algorithm, a representative algorithm is YOLO series, the other is a second-order target detection algorithm, and a representative algorithm is RCNN series. From the practical application aspect, the first-order target detection algorithm adopts the bounding box and the target class to simultaneously regress at one time, so that absolute advantages are occupied in the detection speed, and the first-order target detection algorithm is widely applied to the detection of video images. The detection network is established by establishing a trunk feature extraction network, a candidate frame, a category regression, a boundary frame regression and other modes, a detection model is obtained by utilizing a public VOC data set or a COCO data set training network, and the target detection of the video image is realized by sending each frame of image of the video stream into the trained detection model. Since the deep learning object detection model is mostly based on GPU operation, real-time performance of conventional pixel video stream detection on an embedded CPU is still difficult to guarantee.
The real-time detection of the conventional camera video by the target detection methods can only run by the GPU, and in the practical engineering application, the conventional camera video cannot be detected in real time by the embedded CPU; moreover, the detection method is limited by a training data set, has low recognition accuracy on the airport runway surface airplane, and can not recognize and distinguish the movement direction of the airplane.
Disclosure of Invention
In order to solve the technical problems in the background art, the invention aims to provide an airport runway surface plane real-time detection method based on an embedded CPU.
In order to achieve the aim of the invention, the invention adopts the following technical scheme: the real-time detection method of the airport runway surface plane based on the embedded CPU comprises the steps of carrying out video acquisition on an airport runway, establishing an original training data set of a left-row plane and a right-row plane and two types of original tag sets, carrying out left-right turning and repeated cutting on collected video pictures, copying corrected tag data, establishing a final training data set and a final tag set, setting up two types of network frames for deep learning target detection, obtaining model parameters under the minimum test errors according to the final training data set and the network weight parameters trained by the final tag set, and establishing a real-time detection model of the airport runway plane according to the network architecture and the final network training parameters.
Preferably: comprises the steps of,
A1, carrying out video acquisition on an airport taxiway, screening video pictures containing aircrafts, establishing an original training data set, and establishing two kinds of original tag sets of a left-row aircraft and a right-row aircraft;
A2, copying the pictures in the step A1, turning left and right, copying tag data, correcting the positions of tag frames, mutually replacing category names, cutting the pictures of the original training data set for multiple times, copying and correcting the tag data, and establishing a final training data set and a final tag set;
a3, constructing a deep learning target detection two-category network framework;
a4, initializing network parameters of the step A3, and training the network weight parameters of the step A3 by utilizing the final training data set and the final label set of the step A2 until the total loss is not reduced any more, so as to obtain model parameters under the minimum test error;
a5, based on the network architecture of the step A3 and the final network training parameters of the step A4, establishing a real-time detection model of the airport taxiway aircraft;
A6, the to-be-detected video enters the real-time detection model, and the model automatically detects the video image and locks the position of the aircraft to give the motion direction of the aircraft.
Preferably: in the step A2, a rectangular frame with a fixed size is adopted to carry out repeated cutting and label data copying correction on the picture of the original training data set.
Preferably: and in the step A4, one or more of Xavier normal distribution, random initialization, all-zero initialization and the like are initialized.
Preferably: said step A2 comprises the steps of,
A2.1, counting the size of an original tag set real tag frame in the step A1, recording the width W and the height H of the maximum real tag frame, and determining the width Wc and the height Hc of the cutting frame;
A2.2, calculating the center coordinates (XCI, YCi) of the real tag frame according to the tag data corresponding to the video pictures in the step A1, wherein XC is the abscissa of the center point, YC is the ordinate of the center point, and i is the serial number of the real tag frame of each video picture;
A2.3, randomly generating 10 coordinates (Xj, yj) in an area with an abscissa interval of [ XC i-Wc/2+wi/2,XCi+Wc/2-wi/2 ] and an ordinate interval of [ YC i-Hc/2+hi/2,YCi+Hc/2-hi/2 ] for each plane on each video picture, taking the 10 coordinates as the center coordinates of a cutting rectangular frame, cutting the video picture, copying tag data, and correcting the picture size and the position of the tag frame;
A2.4, saving the picture and the corrected label data which are cut each time, and establishing a final training data set and a final label set of the real-time detection model of the airport taxiway aircraft.
Preferably: the width Wc of the cutting frame is more than or equal to 2W and is an integral multiple of 32, and the height Hc of the cutting frame is more than or equal to 2H and is an integral multiple of 32.
Preferably: said step A3 comprises the steps of,
A3.1, constructing a deep learning target detection two-category network frame based on yolov-tiny target detection algorithm, wherein the two categories are a left-hand aircraft and a right-hand aircraft;
a3.2, changing the last layer of output channels of the backbone network into 256, carrying out one-time up-sampling and adding with the third to last convolution layer, continuing to carry out one-time up-sampling and adding with the output result of the fifth convolution layer to obtain a multi-scale fusion feature map;
a3.3, carrying out convolution (3 multiplied by 128) with the step length of 2 and convolution (3 multiplied by 256) on the multi-scale fusion feature map in sequence, so as to obtain an output layer with the dimensions of (Hc/32, wc/32 and 256), wherein Hc/32 is the height of the feature map, wc/32 is the width of the feature map, and 256 is the channel number of the feature map;
A3.4, performing (1×1×512) convolution with step length of 1 on the output layer of the step A3.3, and performing (3×3×21) convolution to obtain final prediction result output.
Preferably: said step A4 comprises the steps of,
A4.1, dividing the real label frame size of the label set into 3 types by adopting a Kmeans clustering algorithm, and arranging rectangular frames with the 3 types of sizes in order from small to large to obtain 3 candidate frames;
a4.2, encoding the real tag frame by using the candidate frames obtained in the step A4.1 and adopting the same encoding mode for the last three candidate frames corresponding to yolov-tini;
a4.3, randomly ordering training data and corresponding tag data, selecting a specific number of training pictures and corresponding tag frame codes each time, inputting the data to the deep learning target detection 2-class network built in the step A3, optimizing network internal parameters by applying a random gradient descent method based on a yolov-tiny loss function until total loss is not reduced, and outputting network parameters under the minimum test error.
Preferably: said step A5 comprises the steps of,
A5.1, establishing a video input interface;
A5.2, setting a video image detection area;
A5.3, establishing a network structure configuration file, and converting the final network weight parameters obtained in the step A4 into a network weight parameter file;
a5.4, loading a network structure configuration file and a network weight parameter file;
a5.5, performing non-maximum suppression on the model output value;
a5.6, outputting and displaying the result.
Corresponding to: an application of an airport runway plane real-time detection method based on an embedded CPU in airport runway plane speed measurement.
Compared with the prior art, the invention has the following beneficial effects:
(1) And the rectangular frame with the fixed size is adopted to carry out repeated cutting and label data copying correction on each picture of the original training data set, so that the training data is expanded to 10 times, the training data expansion is effectively carried out, and meanwhile, the precision of the model is improved.
(2) The built deep learning target detects the network frames of two categories (a left airplane and a right airplane), performs multi-scale fusion on the extracted features, only outputs one prediction layer, simplifies model parameter while guaranteeing model prediction precision, and greatly reduces the consumption of calculation resources in training and testing processes.
(3) The real-time detection model of the airport runway plane is built, the real-time detection of the plane can be carried out on the video shot by the conventional airport taxiways with about 200 ten thousand pixels under the embedded CPU, the movement direction of the plane can be distinguished, and the problems and defects that the real-time detection of the video shot by the conventional airport taxiways cannot be realized under the embedded CPU, the recognition precision of the plane on the airport runway plane is low, and the movement direction of the plane cannot be recognized and distinguished in the prior art are overcome.
Drawings
FIG. 1 is a flow chart of an implementation of the real-time detection method of an airport runway surface aircraft of the present invention;
FIG. 2 is a drawing illustrating an example of picture cropping according to the present invention (the region within the stippled frame represents the region where random coordinates need to be generated, 10 black dots represent the randomly generated coordinates, and 1 black solid line frame represents 10 crop frames);
FIG. 3 is a diagram of a deep learning object detection 2 category (left-hand aircraft, right-hand aircraft) network framework of the present invention;
FIG. 4 is a flow chart of the real-time detection of an airport runway surface aircraft of the present invention;
FIG. 5 is a step chart of a method for defining a tail in-head out list with a length not greater than N and storing data in embodiment 2 of the present invention;
fig. 6 is a step chart of solving the aircraft taxiing pixel value speed based on the detection result in embodiment 2 of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. The technical means used in the examples are conventional means well known to those skilled in the art unless otherwise indicated.
The invention discloses an airport runway surface plane real-time detection method based on an embedded CPU (Central processing Unit), which comprises the steps of carrying out video acquisition on an airport runway, establishing an original training data set and a 2-class original tag set of a left-going plane and a right-going plane, carrying out left-right turning and repeated cutting on collected video pictures, copying corrected tag data, establishing a final training data set and a final tag set, constructing a deep learning target detection 2-class network frame, obtaining model parameters under the minimum test error according to the network weight parameters trained by the final training data set and the final tag set, and establishing an airport runway plane real-time detection model according to the network frame and the final network training parameters.
The airport runway surface plane real-time detection method based on the embedded CPU comprises the following steps:
A1, carrying out video acquisition on an airport taxiway, screening video pictures containing airplanes and storing the video pictures to establish an original training data set; labeling the left aircraft and the right aircraft respectively by Labelimg label making software, and establishing an original label set of the left aircraft and the right aircraft (class 2).
A2, copying each picture in the step A1, turning left and right, copying corresponding tag data, correcting the positions of tag frames, replacing category names, cutting each picture of an original training data set for multiple times by adopting rectangular frames, copying and correcting the tag data, and establishing a final training data set and a final tag set. The rectangular frame is a rectangular frame of a fixed size.
A3, constructing a deep learning target detection 2-category (left-hand aircraft and right-hand aircraft) network frame.
And A4, initializing the network parameters of the step A3, and training the network weight parameters of the step A3 by utilizing the final training data set and the final label set of the step A2 until the total loss is not reduced any more, so as to obtain the model parameters under the minimum test error. It should be noted that, the initialization is one or more of Xavier normal distribution, random initialization, all-zero initialization, etc.
A5, based on the network architecture of the step A3 and the final network training parameters of the step A4, establishing a real-time detection model of the airport taxiway aircraft.
A6, the to-be-detected video enters the real-time detection model in the step A5, and the model automatically detects the video image and locks the position of the aircraft to give the motion direction of the aircraft.
Further, the step A2 comprises the following steps,
A2.1, counting the sizes of the real label frames of the original label set in the step A1, recording the width W and the height H of the maximum real label frame, and determining the width Wc and the height Hc of the cutting frame. The width Wc of the cutting frame is equal to or greater than 2W and is an integer multiple of 32, and the height Hc of the cutting frame is equal to or greater than 2H and is an integer multiple of 32.
A2.2, calculating the center coordinates (XCI, YCi) of the real tag frame of each video picture in the original training data set in the step A1 according to the corresponding tag data, wherein XC is the abscissa of the center point, YC is the ordinate of the center point, and i is the serial number of the real tag frame of each video picture.
A2.3, randomly generating 10 coordinate values ：(x1,y1)、(x2,y2)、(x3,y3)、(x4,y4)、(x5,y5)、(x6,y6)、(x7,y7)、(x8,y8)、(x9,y9)、(x10,y10), (Xj, yj) in an area with an abscissa interval of [ XC i-Wc/2+wi/2,XCi+Wc/2-wi/2 ] and an ordinate interval of [ YC i-Hc/2+hi/2,YCi+Hc/2-hi/2 ] for each plane on each video picture, cutting the video picture by taking each coordinate as the central coordinate of a cutting rectangular frame, copying tag data once every cutting, and correcting the size of the picture and the position of the tag frame.
A2.4, saving the picture cut each time and the label data file corresponding to the correction, so as to expand the original data set and the label set to 10 times, and establishing a final training data set and a final label set which are suitable for the real-time detection model of the airport taxiway aircraft.
Further, the step A3 comprises the following steps,
A3.1, constructing a deep learning target detection 2-category (left aircraft and right aircraft) network frame based on yolov-tiny target detection algorithm.
And A3.2, changing the last layer of output channels of the backbone network into 256, carrying out up-sampling once and adding with the third to last convolution layer, and then continuing up-sampling once and adding with the output result of the fifth convolution layer to obtain the multi-scale fusion characteristic diagram.
A3.3, carrying out (3 multiplied by n 1) convolution with the step length of 2 and (3 multiplied by n 2) convolution on the multi-scale fusion feature map in sequence, so as to obtain an output layer with the dimensions of (Hc/32, wc/32, 256), wherein Hc/32 is the height of the feature map, wc/32 is the width of the feature map, and 256 is the channel number of the feature map; the values of n1 and n2 are any value between 64 and 256. Preferably, n1 is 128 and n2 is 256.
A3.4, performing (1×1×512) convolution with step length of 1 on the output layer of the step A3.3, and performing (3×3×21) convolution to obtain final prediction result output.
Further, the step A4 comprises the following steps,
And A4.1, dividing the real label frame size of the label set into 3 types by adopting a Kmeans clustering algorithm, and arranging the rectangular frames with the 3 types of sizes in order from small to large to obtain 3 candidate frames.
And A4.2, encoding the real tag frame by using the candidate frames obtained in the step A4.1 and adopting the same encoding mode to correspond to the last three candidate frames of yolov-tini.
A4.3, randomly ordering training data and corresponding tag data, selecting a specific number of training pictures and corresponding tag frame codes each time, inputting the data to the deep learning target detection 2-class (left-row aircraft and right-row aircraft) network built in the step A3, optimizing network internal parameters based on yolov-tiny loss functions by using a random gradient descent method until total loss is not reduced, and outputting network parameters under the minimum test error.
Further, the step A5 comprises the following steps,
A5.1, establishing a video input interface.
A5.2, setting a video image detection area.
And A5.3, establishing a network structure configuration file, and converting the final network weight parameters obtained in the step A4 into a network weight parameter file. It should be noted that, the network weight parameter file is a weight file; the network fabric configuration file is a cfg file.
A5.4, loading a network structure configuration file and a network weight parameter file.
A5.5, performing non-maximum suppression on the model output value.
A5.6, outputting and displaying the result.
Example 1
Step A1: a certain taxiway of an airport is selected, a 200-ten-thousand-pixel camera is installed on one side near the taxiway, the installation position and the focal length are adjusted to enable the size of the passing airplane in a video picture to be suitable, and the video of the passing airplane of the taxiway is shot and transmitted to a computer for storage through a network. In order to obtain richer video data, the method can respectively collect 1-2 hours aiming at weather such as sunny days, overcast days, rainy days, snowy days, strong wind and the like and time periods such as morning, midday, evening and the like, and ensures that the collected video has different directions for taxiing the aircraft to pass through. Manually screening video frame pictures containing a sliding airplane in a video, saving the video frame pictures as pictures, opening each picture in sequence by using Labelimg label making software, respectively carrying out frame selection on a left-row airplane and a right-row airplane on the pictures to determine airplane position coordinates (x 1, y1, x2 and y 2), wherein (x 1, y 1) is a boundary frame upper left corner vertex coordinate, and (x 2, y 2) is a boundary frame lower right corner vertex coordinate, then carrying out attribute input (airplane-left row and airplane-right row), saving and outputting the same name as corresponding pictures as an xml label data file, and establishing 200 pieces of each of a picture set and a label set.
Step A2: firstly, copying each picture in the step A1, turning left and right, copying corresponding label data, correcting the positions of the label frames, and replacing category names, firstly, counting the sizes of the real label frames of the original label set in the step A1, obtaining the maximum real frame width W of 247 and the height H of 124, and determining the cutting rectangular frame width Wc of 512 (Wc is more than or equal to 2 XW and is a whole multiple of 32), and the height Hc of 256 (Hc is more than or equal to 2 XH and is a whole multiple of 32). Then for each picture of the original training dataset in1, the center coordinates (XC i,YCi) of the real frame are calculated from the corresponding label data. Then, for each plane on each graph, 10 coordinate values ：(x1,y1)、(x2,y2)、(x3,y3)、(x4,y4)、(x5,y5)、(x6,y6)、(x7,y7)、(x8,y8)、(x9,y9)、(x10,y10), are randomly generated in the section with the horizontal coordinate of [ XC i-Wc/2+wi/2,XCi+Wc/2-wi/2 ] and the vertical coordinate of [ YC i-Hc/2+hi/2,YCi+Hc/2-hi/2 ], and image clipping is performed by taking each coordinate as the central coordinate of the clipping rectangular frame, and label data is copied once every clipping, and correction is performed on the image size and the position of the label frame, and an image clipping example is shown in fig. 2. And then storing the image cut each time and the label data file corresponding to the correction, and 2000 final training pictures and the corresponding label data file (. Xml file).
Step A3: firstly, a deep learning target detection 2-category (left aircraft and right aircraft) network frame is built based on yolov-tiny target detection algorithm. And changing the convolution step length of the last layer of the main network into 2, changing the number of output channels into 256, then carrying out up-sampling once and adding with the third to last convolution layer, and then continuing up-sampling once and adding with the output result of the fifth convolution layer to obtain the multi-scale fusion characteristic diagram. And then (3×3×256) convolution with step length of 1 and (3×3×512) convolution with step length of 2 are sequentially carried out on the multi-scale fusion feature map, so as to obtain an output layer with dimension of (8, 16, 512). Finally, the last step of convolution with step length of 1 (1×1×21) is carried out on the output layer of the last step to obtain the final prediction result output, and the network framework structure is not shown in fig. 3.
Step A4: initializing network parameters of 3 by adopting an Xavier normal distribution, classifying the real label frame sizes of a label set into 3 types by adopting a Kmeans clustering algorithm, and arranging rectangular frames of 3 sizes in order from small to large to obtain 3 candidate frames, wherein the 3 candidate frames are respectively (37, 58), (72, 115), (118, 206), the first value in brackets is the height of the candidate frame, and the second value is the width of the candidate frame. And then, encoding the real tag frame by using the last three candidate frames corresponding to yolov-tini in the same encoding mode by using the candidate frame obtained in the last step. And finally randomly sequencing training data and corresponding tag data, inputting the data after 32 training pictures and corresponding tag frame codes to a deep learning target detection 2-class (left-hand plane and right-hand plane) network built in 3 each time, optimizing network internal parameters by applying a random gradient descent method based on a yolov-tini loss function until total loss is not reduced, outputting network parameters under the minimum test error, and storing as follows: the last_weight.h file (output file suffix name varies depending on the network frame used, this example application Keras frame).
Step A5: firstly, establishing a video input interface based on OpenCV, and setting a video picture detection area; and then, establishing a network structure configuration file (. Cfg file), and simultaneously converting the final network weight parameter file obtained in the step 4, namely the last_weight.h file, into a last_weight.weight file (if the file is the. Weight file, conversion is not needed). Then, a network structure configuration file (. Cfg file) and a network weight parameter file (. Weight) are loaded based on OpenCV, and non-maximum value suppression is carried out on the model output value based on OpenCV; and finally, connecting the result display output module. The real-time detection model structure is shown in fig. 4.
Step A6: and (3) inputting the to-be-detected video into the real-time detection model in the step (A5), automatically detecting the video picture by the model, outputting the position and the movement direction of the airplane, and displaying the video picture in real time. The real-time aircraft detection flow is shown in fig. 4.
Example 2
The invention also discloses an application of the real-time detection method of the airport pavement airplane based on the embedded CPU in the speed measurement of the airport pavement airplane, as shown in figure 6.
Step B1: a camera is arranged on one side of the airport taxiway to monitor the past taxiways and transmit video streams to a computer in real time.
Selecting a taxiway near a certain stand of an airport; selecting a part of area on the sliding road as a video shooting speed measuring area; a 200-ten-thousand camera is arranged opposite to the area; the camera shoots video pictures and is connected with the computer in real time; adjusting the video picture pixels to 768×1024, i.e. the video picture size is high h=768, and wide w=1024; video shooting and transmission are started.
Step B2: the method for detecting the airport pavement plane in real time based on the embedded CPU disclosed by the invention is used for detecting the video stream in the step B1 in real time.
Step B3: two empty lists (the length is not more than N, and the tail is in and out) are defined, and real-time detection results of the current continuous N frames of video images are respectively stored.
(1) Defining empty lists A and B;
(2) Converting the coordinates (x 0i,y0i,x1i,y1i) of the aircraft in the current frame detection result into a list, namely: x 0i,y0i,x1i,y1i, wherein x 0,y0,x1,y1 represents the left upper-corner abscissa, the ordinate, the right lower-corner abscissa and the ordinate of the aircraft respectively, the subscript i represents the ith aircraft, and if the detection result is empty, an empty list is stored;
(3) Defining empty lists An and Bn, grouping the ith detected aircraft according to different aircraft directions, putting the coordinate lists of all left aircraft into the An list, and putting the coordinate lists of all right aircraft into the Bn list in a similar way, wherein the subscript n represents a frame number;
(4) An is stored in A, bn is stored in B, and if no aircraft is detected in the frame, the empty lists of An and Bn are respectively stored in the list;
(5) Taking n=4, judging the length of the current A, B list, if the length is greater than N, deleting the first element in the list, moving to the next frame to jump to the (2) step, and implementing the steps specifically as shown in fig. 5.
Step B4: and defining an airplane position vertical direction offset Ymin=10 (according to video pixel size adjustment), and calculating the sliding speed v i of the currently detected airplane based on pixel values based on the real-time detection results of the head and tail frames in the A and B lists and Ymin, wherein the subscript i represents the detected airplane label.
(1) Assuming that the nth frame is currently detected, recording a program time T1 at the detection end time of the nth-3 frames, recording a program time T2 at the detection end time of the frames, and calculating the taxi time of the airplane under the continuous 4 frames at the moment, namely T=T2-T1;
(2) Assuming that the current detection process is performed on the nth frame video image, A= [ An-3, an-2, an-1, an ], B= [ Bn-3, bn-2, bn-1, bn ], each element in the outer layer circulation traversal An is set, and each element in the inner layer circulation traversal An-N+1 is set;
(3) Respectively calculating the abscissa and the ordinate of the center of the airplane in each element, namely: x ci＝(x0i+x1i)/2,yci＝(y0i+y1i)/2, wherein x ci represents the aircraft center abscissa, y ci represents the aircraft center ordinate, and subscript i represents the aircraft label;
(4) Assuming that m y cj are calculated in An-3, n y ci are calculated in An, judging if: y cj∈[yci-10,yci +10], then considering y cj、yci as the central ordinate of the same aircraft at different moments, calculating the pixel-based aircraft speed, i.e. v i＝(xci-xcj)/T, using the following formula, wherein the subscript i represents the aircraft label in the list An, and the subscript j represents the aircraft label in the list An-3;
(5) The aircraft speeds in the list Bn, v i＝(xcj-xci)/T, are calculated in the same way as in steps B2, B3, B4.
Step B5: the scaling factor k of the pixel size to the actual distance is calculated, the true aircraft taxi speed v ri＝kvi is calculated, and the subscript i indicates the detected aircraft label.
The actual length of the taxiway shot by the video picture is measured to be L=50 meters, the proportionality coefficient k=L/W=50/1024=0.049 is calculated, and the actual speed of the airplane is calculated as follows: v ri＝kvi＝0.049vi.
The above embodiments are only illustrative of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention, and various modifications, variations, alterations, substitutions made by those skilled in the art to the technical solution of the present invention should fall within the protection scope defined by the claims of the present invention without departing from the spirit of the design of the present invention.
Claims (8)
1. An airport runway surface plane real-time detection method based on an embedded CPU is characterized in that: the method comprises the steps of performing video acquisition on an airport taxiway, establishing an original training data set of a left-row airplane and a right-row airplane and two types of original tag sets, performing left-right turning and repeated cutting on collected video pictures, copying corrected tag data, establishing a final training data set and a final tag set, establishing two types of network frames for deep learning target detection, obtaining model parameters under the minimum test error according to the final training data set and the network weight parameters trained by the final tag set, and establishing an airport taxiway airplane real-time detection model according to the network frames and the final network training parameters;
Comprises the steps of,
A1, carrying out video acquisition on an airport taxiway, screening video pictures containing aircrafts, establishing an original training data set, and establishing an original tag set of two categories of a left-row aircraft and a right-row aircraft;
A2, copying the pictures in the step A1, turning left and right, copying tag data, correcting the positions of tag frames, mutually replacing category names, cutting the pictures of the original training data set for multiple times, copying and correcting the tag data, and establishing a final training data set and a final tag set;
a3, constructing a network framework of which the deep learning target detects two categories;
a4, initializing network parameters of the step A3, and training the network weight parameters of the step A3 by utilizing the final training data set and the final label set of the step A2 until the total loss is not reduced any more, so as to obtain model parameters under the minimum test error;
a5, based on the network architecture of the step A3 and the final network training parameters of the step A4, establishing a real-time detection model of the airport taxiway aircraft;
a6, the to-be-detected video enters the real-time detection model, the model automatically detects the video image and locks the position of the aircraft to give the motion direction of the aircraft;
said step A3 comprises the steps of,
A3.1, constructing a deep learning target detection two-category network framework based on yolov-tiny target detection algorithm;
a3.2, changing the last layer of output channels of the backbone network into 256, carrying out one-time up-sampling and adding with the third to last convolution layer, continuing to carry out one-time up-sampling and adding with the output result of the fifth convolution layer to obtain a multi-scale fusion feature map;
a3.3, carrying out (3 multiplied by 128) convolution and 3 multiplied by 256 convolution with the step length of 2 on the multi-scale fusion feature map in sequence, so as to obtain output layers with the dimensions of (Hc/32, wc/32, 256), wherein Wc and Hc are the width and the height of a cutting frame respectively, hc/32 is the height of the feature map, wc/32 is the width of the feature map, and 256 is the channel number of the feature map;
A3.4, performing 1×1×512 convolution with step length of 1 on the output layer of the step A3.3, and performing 3×3×21 convolution to obtain final prediction result output.
2. The method for detecting the airport pavement airplane in real time based on the embedded CPU according to claim 1, wherein the method comprises the following steps: in the step A2, a rectangular frame with a fixed size is adopted to carry out repeated cutting and label data copying correction on the picture of the original training data set.
3. The method for detecting the airport pavement airplane in real time based on the embedded CPU according to claim 1, wherein the method comprises the following steps: and in the step A4, one or more of Xavier normal distribution, random initialization and all-zero initialization are initialized.
4. The method for detecting the airport pavement airplane in real time based on the embedded CPU according to claim 1, wherein the method comprises the following steps: said step A2 comprises the steps of,
A2.1, counting the size of an original tag set real tag frame in the step A1, recording the width W and the height H of the maximum real tag frame, and determining the width Wc and the height Hc of the cutting frame;
A2.2, calculating the center coordinates (XCI, YCi) of the real tag frame according to the tag data corresponding to the video pictures in the step A1, wherein XC is the abscissa of the center point, YC is the ordinate of the center point, and i is the serial number of the real tag frame of each video picture;
A2.3, randomly generating 10 coordinates (Xj, yj) in an area with an abscissa interval of [ XC i-Wc/2+wi/2,XCi+Wc/2-wi/2 ] and an ordinate interval of [ YC i-Hc/2+hi/2,YCi+Hc/2-hi/2 ] for each plane on each video picture, taking the 10 coordinates as the center coordinates of a cutting rectangular frame, cutting the video picture, copying tag data, and correcting the picture size and the position of the tag frame;
A2.4, saving the picture and the corrected label data which are cut each time, and establishing a final training data set and a final label set of the real-time detection model of the airport taxiway aircraft.
5. The method for detecting the airport pavement airplane in real time based on the embedded CPU according to claim 4, wherein the method comprises the following steps: the width Wc of the cutting frame is more than or equal to 2W and is an integral multiple of 32, and the height Hc of the cutting frame is more than or equal to 2H and is an integral multiple of 32.
6. The method for detecting the airport pavement airplane in real time based on the embedded CPU according to claim 1, wherein the method comprises the following steps: said step A4 comprises the steps of,
A4.1, dividing the real label frame size of the label set into 3 types by adopting a Kmeans clustering algorithm, and arranging rectangular frames with the 3 types of sizes in order from small to large to obtain 3 candidate frames;
a4.2, encoding the real tag frame by using the candidate frames obtained in the step A4.1 and adopting the same encoding mode for the last three candidate frames corresponding to yolov-tini;
a4.3, randomly ordering training data and corresponding tag data, selecting a specific number of training pictures and corresponding tag frame codes each time, inputting the data to the deep learning target detection 2-class network built in the step A3, optimizing network internal parameters by applying a random gradient descent method based on a yolov-tiny loss function until total loss is not reduced, and outputting network parameters under the minimum test error.
7. The method for detecting the airport pavement airplane in real time based on the embedded CPU according to claim 1, wherein the method comprises the following steps: said step A5 comprises the steps of,
A5.1, establishing a video input interface;
A5.2, setting a video image detection area;
A5.3, establishing a network structure configuration file, and converting the final network weight parameters obtained in the step A4 into a network weight parameter file;
a5.4, loading a network structure configuration file and a network weight parameter file;
a5.5, performing non-maximum suppression on the model output value;
a5.6, outputting and displaying the result.
8. Use of an embedded CPU based real-time detection method for airport runway aircraft according to any of claims 1-7 for airport runway aircraft speed measurement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111041254.4A CN113688777B (en) | 2021-09-07 | 2021-09-07 | Airport pavement plane real-time detection method based on embedded CPU |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111041254.4A CN113688777B (en) | 2021-09-07 | 2021-09-07 | Airport pavement plane real-time detection method based on embedded CPU |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113688777A CN113688777A (en) | 2021-11-23 |
| CN113688777B true CN113688777B (en) | 2024-09-06 |
Family
ID=78585395
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111041254.4A Active CN113688777B (en) | 2021-09-07 | 2021-09-07 | Airport pavement plane real-time detection method based on embedded CPU |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688777B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110586A (en) * | 2019-03-18 | 2019-08-09 | 北京理工雷科电子信息技术有限公司 | The method and device of remote sensing airport Airplane detection based on deep learning |
| AU2020102091A4 (en) * | 2019-10-17 | 2020-10-08 | Wuhan University Of Science And Technology | Intelligent steel slag detection method and system based on convolutional neural network |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111222474B (en) * | 2020-01-09 | 2022-11-04 | 电子科技大学 | Method for detecting small target of high-resolution image with any scale |
| CN112464883B (en) * | 2020-12-11 | 2022-12-06 | 武汉工程大学 | Automatic detection and identification method and system for ship target in natural scene |
| CN112699967B (en) * | 2021-01-18 | 2024-03-12 | 武汉大学 | Remote airport target detection method based on improved deep neural network |
| CN113313695A (en) * | 2021-06-05 | 2021-08-27 | 西北工业大学 | Automatic deep learning defect detection and identification method based on small sample aeroengine blade CT image |
-
2021
- 2021-09-07 CN CN202111041254.4A patent/CN113688777B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110586A (en) * | 2019-03-18 | 2019-08-09 | 北京理工雷科电子信息技术有限公司 | The method and device of remote sensing airport Airplane detection based on deep learning |
| AU2020102091A4 (en) * | 2019-10-17 | 2020-10-08 | Wuhan University Of Science And Technology | Intelligent steel slag detection method and system based on convolutional neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113688777A (en) | 2021-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108596101B (en) | Remote sensing image multi-target detection method based on convolutional neural network | |
| CN111047554B (en) | Composite insulator overheating defect detection method based on instance segmentation | |
| CN109934163B (en) | Aerial image vehicle detection method based on scene prior and feature re-fusion | |
| CN109886066B (en) | Rapid target detection method based on multi-scale and multi-layer feature fusion | |
| CN106960195B (en) | Crowd counting method and device based on deep learning | |
| CN111680655A (en) | Video target detection method for aerial images of unmanned aerial vehicle | |
| CN108537824B (en) | Feature map enhanced network structure optimization method based on alternating deconvolution and convolution | |
| CN111967313B (en) | Unmanned aerial vehicle image annotation method assisted by deep learning target detection algorithm | |
| CN112381060B (en) | Building earthquake damage level classification method based on deep learning | |
| CN111144418B (en) | Railway track area segmentation and extraction method | |
| CN114581486B (en) | Template updating target tracking algorithm based on multi-layer characteristics of full convolution twin network | |
| CN112686172A (en) | Method and device for detecting foreign matters on airport runway and storage medium | |
| CN113780132A (en) | Lane line detection method based on convolutional neural network | |
| CN117197676B (en) | Target detection and identification method based on feature fusion | |
| Zhang et al. | Application research of YOLO v2 combined with color identification | |
| CN110866472A (en) | Unmanned aerial vehicle ground moving target identification and image enhancement system and method | |
| CN114332942A (en) | Night infrared pedestrian detection method and system based on improved YOLOv3 | |
| CN115240089A (en) | Vehicle detection method of aerial remote sensing image | |
| CN117333845A (en) | Real-time detection method for small target traffic sign based on improved YOLOv5s | |
| CN116342536A (en) | Aluminum strip surface defect detection method, system and equipment based on lightweight model | |
| Ouyang et al. | Aerial target detection based on the improved YOLOv3 algorithm | |
| CN113688777B (en) | Airport pavement plane real-time detection method based on embedded CPU | |
| Pillai et al. | Fine-Tuned EfficientNetB4 Transfer Learning Model for Weather Classification | |
| CN116630904B (en) | Small target vehicle detection method integrating non-adjacent jump connection and multi-scale residual error structure | |
| CN112861924B (en) | Visible light/infrared image multi-platform distributed fusion multi-target detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |