CN111340041B - License plate recognition method and device based on deep learning - Google Patents
License plate recognition method and device based on deep learning Download PDFInfo
- Publication number
- CN111340041B CN111340041B CN202010176205.0A CN202010176205A CN111340041B CN 111340041 B CN111340041 B CN 111340041B CN 202010176205 A CN202010176205 A CN 202010176205A CN 111340041 B CN111340041 B CN 111340041B
- Authority
- CN
- China
- Prior art keywords
- image
- gradient
- license plate
- normalized
- probability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013135 deep learning Methods 0.000 title claims abstract description 26
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 41
- 238000004364 calculation method Methods 0.000 claims abstract description 40
- 238000005070 sampling Methods 0.000 claims abstract description 25
- 238000012544 monitoring process Methods 0.000 claims description 13
- 238000006243 chemical reaction Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 5
- 230000001537 neural effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 101100190310 Emericella nidulans (strain FGSC A4 / ATCC 38163 / CBS 112.46 / NRRL 194 / M139) phiA gene Proteins 0.000 description 1
- 101150115185 binB gene Proteins 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Health & Medical Sciences (AREA)
- Character Discrimination (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention discloses a license plate recognition method and a license plate recognition device based on deep learning, which can calculate a gradient probability histogram corresponding to each video image after the video images are obtained by sampling, compare the gradient probability histogram distribution template of each video image with a gradient probability histogram distribution template of a clear license plate image and a gradient probability histogram distribution template of a fuzzy license plate image obtained by pre-calculation, determine the definition confidence coefficient of the video images, and further only recognize the license plate of the video image with higher definition confidence coefficient, so that the accuracy of license plate recognition can be improved. And when the final license plate recognition result is determined according to the plurality of video images, the definition confidence degrees corresponding to the video images can be combined for calculation, so that the accuracy of license plate recognition can be further improved. The number of the character bits included in the license plate recognition result output by the convolutional neural network in the embodiment is consistent with the number of the license plates, so that the condition of more or less recognition characters can be avoided, and the accuracy of license plate recognition is improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a license plate recognition method and device based on deep learning.
Background
In the field of video monitoring, license plate recognition is generally required in a monitoring video to track a vehicle and the like. In known methods, voting may be based on the identification scheme of the surveillance video. Specifically, video images are obtained by sampling in a monitoring video, license plate recognition is carried out on all the video images, and finally all recognition results are adopted for voting processing to obtain license plate recognition results.
However, in the above method, since there may be motion blur and defocus blur in the video image, this will result in low accuracy of license plate recognition. Therefore, in order to improve the accuracy of license plate recognition, a license plate recognition method is urgently needed.
Disclosure of Invention
The invention provides a license plate recognition method and device based on deep learning, and aims to improve the accuracy of license plate recognition. The specific technical scheme is as follows.
In a first aspect, an embodiment of the present invention provides a license plate recognition method based on deep learning, where the method includes:
sampling is carried out on the basis of a preset sampling period in a monitoring video containing a target license plate, so that a plurality of video images containing the target license plate are obtained;
carrying out gray level conversion on the multiple video images to obtain multiple gray level video images; normalizing the multiple gray level video images to a preset size to obtain multiple target video images;
performing convolution operation on each target video image through a preset x-direction filter to obtain an x-edge image corresponding to each target video image; performing convolution operation on each target video image through a preset y-direction filter to obtain a y-edge image corresponding to each target video image; calculating a gradient image corresponding to each target video image according to each x edge image and each corresponding y edge image;
normalizing the pixel value of each gradient image to obtain each normalized gradient image, carrying out histogram statistics on each normalized gradient image to obtain the statistical number of each pixel value in each normalized gradient image, calculating the probability value of each statistical number in the total pixel number of each normalized gradient image, and constructing a gradient probability histogram corresponding to each normalized gradient image containing each probability value;
acquiring a gradient probability histogram distribution template of a clear license plate image and a gradient probability histogram distribution template of a fuzzy license plate image which are obtained through pre-calculation, calculating a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the clear license plate image and a second cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the fuzzy license plate image aiming at each normalized gradient image, and determining a definition confidence coefficient corresponding to the normalized gradient image according to the sizes of the first cosine distance and the second cosine distance; the clear license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired clear image, and the fuzzy license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired fuzzy image;
inputting the video images corresponding to the normalized gradient images with the definition confidence degrees larger than a preset confidence degree threshold value into a convolutional neural network obtained by pre-training to obtain license plate recognition results corresponding to the video images; obtaining a final recognition result of the target license plate according to the license plate recognition result corresponding to each video image and the definition confidence coefficient of the normalized gradient image corresponding to each video image; the license plate recognition result comprises a plurality of characters at preset positions.
Optionally, the step of calculating a gradient image corresponding to each target video image according to each x-edge image and each corresponding y-edge image includes:
calculating the pixel value of any point in the gradient image corresponding to any target video image according to the following formulaG（x， y）：
GxThe pixel value of any point in the x-edge image corresponding to any target video image,Gythe pixel value of the point in the y edge image corresponding to any target video image is obtained;
the step of normalizing the pixel value of each gradient image to obtain each normalized gradient image comprises the following steps:
updating the pixel values of each of the gradient images according to the following formula:
optionally, the step of calculating the probability value of the statistical quantities in the total pixel quantity of the normalized gradient image includes:
calculating any statistical quantity according to the following formulabiProbability value of total pixel number of the normalized gradient imagepi：
bj is the number of pixel points with the pixel value of j in the normalized gradient image.
Optionally, the step of calculating a first cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the clear license plate image includes:
calculating a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and a gradient probability histogram distribution template of the clear license plate image according to the following formula:
xiis the probability value corresponding to the pixel value i in the gradient probability histogram corresponding to the normalized gradient image,yiand obtaining a probability value corresponding to the pixel value i in the gradient probability histogram distribution template of the clear license plate image.
Optionally, the step of determining the definition confidence corresponding to the normalized gradient image according to the first cosine distance and the second cosine distance includes:
when the first cosine distance is greater than the second cosine distance, determining the definition confidence coefficient corresponding to the normalized gradient image as the first cosine distance;
and when the first cosine distance is not greater than the second cosine distance, determining that the definition confidence coefficient corresponding to the normalized gradient image is 0.
Optionally, the step of obtaining a final recognition result of the target license plate according to the license plate recognition result corresponding to each of the video images and the sharpness confidence of the normalized gradient image corresponding to each of the video images includes:
according to the arrangement sequence of the license plate recognition results, aiming at h characters in the target license plate, when the preferred recognition results of h characters in partial images in the video image are all k, and the preferred posterior probability of each partial imageS k And when the sum of the product of the definition confidence coefficients of the corresponding normalized gradient images is maximum, determining that the final recognition result of the h-th character of the target license plate is k.
Optionally, the construction process of the gradient probability histogram distribution template of the clear license plate image and the gradient probability histogram distribution template of the fuzzy license plate image includes:
acquiring a sample video image, determining the sample video image with the definition greater than a preset threshold value as a clear image, and determining the sample video image with the definition not greater than the preset threshold value as a blurred image;
calculating a gradient probability histogram corresponding to each clear image and a gradient probability histogram corresponding to each fuzzy image;
calculating a first average value of probability values corresponding to the same pixel value in a gradient probability histogram corresponding to each clear image, and constructing and obtaining a clear license plate image gradient probability histogram distribution template containing each first average value; and calculating a second average value of the probability values corresponding to the same pixel values in the gradient probability histograms corresponding to the fuzzy images, and constructing a fuzzy license plate image gradient probability histogram distribution template containing the second average values.
Optionally, the convolutional neural network is trained by using a gradient back propagation algorithm, and a loss function of the convolutional neural network is as follows:
wherein T is the total number of character categories, M is the number of character digits contained in a license plate, and y is the number of characters contained in the license plate when the obtained classification result is correct j Is 1, otherwise is 0;S j preferred posterior probabilities output for the convolutional neural network:
a k estimating a resulting distance identified as a character k for the convolutional neural network,a j estimating a resulting distance identified as a character j for the convolutional neural network.
In a second aspect, an embodiment of the present invention provides a license plate recognition device based on deep learning, where the license plate recognition device includes:
the video sampling module is used for sampling a monitoring video containing a target license plate based on a preset sampling period to obtain a plurality of video images containing the target license plate;
the gray level conversion module is used for carrying out gray level conversion on the plurality of video images to obtain a plurality of gray level video images; normalizing the multiple gray level video images to a preset size to obtain multiple target video images;
the filtering operation module is used for performing convolution operation on each target video image through a preset x-direction filter to obtain an x-edge image corresponding to each target video image; performing convolution operation on each target video image through a preset y-direction filter to obtain a y-edge image corresponding to each target video image; calculating a gradient image corresponding to each target video image according to each x edge image and each corresponding y edge image;
a histogram statistic module, configured to normalize the pixel values of each gradient image to obtain each normalized gradient image, perform histogram statistics on the normalized gradient image for each normalized gradient image to obtain a statistical number of each pixel value in the normalized gradient image, calculate a probability value of each statistical number in the total pixel number of the normalized gradient image, and construct a gradient probability histogram corresponding to the normalized gradient image and including each probability value;
the definition calculation module is used for acquiring a gradient probability histogram distribution template of a clear license plate image and a gradient probability histogram distribution template of a fuzzy license plate image which are obtained through pre-calculation, calculating a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the clear license plate image and a second cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the fuzzy license plate image, and determining a definition confidence coefficient corresponding to the normalized gradient image according to the first cosine distance and the second cosine distance; the clear license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired clear image, and the fuzzy license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired fuzzy image;
the license plate recognition module is used for inputting the video image corresponding to the normalized gradient image with the definition confidence coefficient greater than a preset confidence coefficient threshold value into a convolutional neural network obtained by pre-training to obtain a license plate recognition result corresponding to each video image; obtaining a final recognition result of the target license plate according to the license plate recognition result corresponding to each video image and the definition confidence coefficient of the normalized gradient image corresponding to each video image; the license plate recognition result comprises a plurality of characters with preset positions.
Optionally, the filtering operation module is specifically configured to calculate a pixel value of any point in a gradient image corresponding to any target video image according to the following formulaG（x，y）：
GxThe pixel value of any point in the x-edge image corresponding to any target video image,Gythe pixel value of the point in the y edge image corresponding to any target video image is obtained;
the histogram statistic module is specifically configured to update the pixel value of each gradient image according to the following formula:
optionally, the histogram statistic module is specifically configured to calculate any statistical quantity according to the following formulabiProbability value of total pixel number of the normalized gradient imagepi：
bj is the number of pixel points with the pixel value of j in the normalized gradient image.
Optionally, the definition calculating module is specifically configured to calculate a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and the distribution template of the gradient probability histogram of the clear license plate image according to the following formula:
xiis the probability value corresponding to the pixel value i in the gradient probability histogram corresponding to the normalized gradient image,yiand obtaining a probability value corresponding to the pixel value i in the gradient probability histogram distribution template of the clear license plate image.
Optionally, the sharpness calculation module is specifically configured to determine, when the first cosine distance is greater than the second cosine distance, a sharpness confidence corresponding to the normalized gradient image as the first cosine distance; and when the first cosine distance is not greater than the second cosine distance, determining that the definition confidence coefficient corresponding to the normalized gradient image is 0.
Optionally, the license plate recognition module is specifically configured to, for the h character in the target license plate according to the arrangement sequence of the license plate recognition results, when the preferred recognition results of the h character in the partial images in the video image are all k, and the preferred posterior probability of each partial image isS k And when the sum of the product of the definition confidence coefficients of the corresponding normalized gradient images is maximum, determining that the final recognition result of the h-th character of the target license plate is k.
Optionally, the apparatus further comprises:
the image acquisition module is used for acquiring a sample video image, determining the sample video image with the definition greater than a preset threshold value as a clear image, and determining the sample video image with the definition not greater than the preset threshold value as a blurred image;
the histogram calculation module is used for calculating a gradient probability histogram corresponding to each clear image and a gradient probability histogram corresponding to each fuzzy image;
the histogram distribution module construction module is used for calculating a first average value of the probability values corresponding to the same pixel value in the gradient probability histogram corresponding to each clear image, and constructing a clear license plate image gradient probability histogram distribution template containing the first average values; and calculating a second average value of the probability values corresponding to the same pixel values in the gradient probability histograms corresponding to the fuzzy images, and constructing a fuzzy license plate image gradient probability histogram distribution template containing the second average values.
Optionally, the convolutional neural network is trained by using a gradient back propagation algorithm, and a loss function of the convolutional neural network is as follows:
wherein T is the total number of character categories, M is the number of character digits contained in a license plate, and y is the number of characters contained in the license plate when the obtained classification result is correct j Is 1, otherwise is 0. This running join is an expression of mathematics for T, because for each position character, only one y inside the running join is 1, and the rest are all 0; the above-described penalty function is a running sum of loss over M characters.S j Preferred posterior probabilities for convolutional neural network output:
a k estimating a resulting distance identified as a character k for the convolutional neural network,a j estimating a resulting distance identified as a character j for the convolutional neural network.
As can be seen from the above, the license plate recognition method and device based on deep learning provided by the embodiment of the invention can perform sampling based on a preset sampling period in a surveillance video including a target license plate to obtain a plurality of video images including the target license plate; carrying out gray level conversion on a plurality of video images to obtain a plurality of gray level video images; normalizing the multiple gray level video images to a preset size to obtain multiple target video images; performing convolution operation on each target video image through a preset x-direction filter to obtain an x-edge image corresponding to each target video image; performing convolution operation on each target video image through a preset y-direction filter to obtain a y-edge image corresponding to each target video image; calculating a gradient image corresponding to each target video image according to each x edge image and each corresponding y edge image; normalizing the pixel values of the gradient images to obtain normalized gradient images, performing histogram statistics on the normalized gradient images aiming at each normalized gradient image to obtain the statistical number of the pixel values in the normalized gradient images, calculating the probability value of the statistical number in the total pixel number of the normalized gradient images, and constructing a gradient probability histogram corresponding to the normalized gradient images containing the probability values; acquiring a gradient probability histogram distribution template of a clear license plate image and a gradient probability histogram distribution template of a fuzzy license plate image which are obtained by pre-calculation, calculating a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the clear license plate image and a second cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the fuzzy license plate image for each normalized gradient image, and determining a definition confidence coefficient corresponding to the normalized gradient image according to the sizes of the first cosine distance and the second cosine distance; the clear license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired clear image, and the fuzzy license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired fuzzy image; inputting the video image corresponding to the normalized gradient image with the definition confidence coefficient greater than the preset confidence coefficient threshold value into a convolutional neural network obtained by pre-training to obtain a license plate recognition result corresponding to each video image; obtaining a final recognition result of the target license plate according to the license plate recognition result corresponding to each video image and the definition confidence coefficient of the normalized gradient image corresponding to each video image; the license plate recognition result comprises a plurality of characters with preset bits, so that after video images are obtained through sampling, a gradient probability histogram corresponding to each video image can be calculated, the gradient probability histogram is compared with a gradient probability histogram distribution template of a clear license plate image and a gradient probability histogram distribution template of a fuzzy license plate image which are obtained through pre-calculation, the definition confidence coefficient of the video images is determined, and then license plate recognition can be carried out only on the video images with higher definition confidence coefficient, namely, the fuzzy images can be removed from all the video images, and license plate recognition is carried out only by utilizing the clear images, so that the accuracy of license plate recognition can be improved. And when the final license plate recognition result is determined according to the plurality of video images, the definition confidence degrees corresponding to the video images can be combined for calculation, so that the accuracy of license plate recognition can be further improved. The number of the character bits included in the license plate recognition result output by the convolutional neural network in the embodiment is consistent with the number of the license plates, so that the condition of more or less recognition characters can be avoided, and the accuracy of license plate recognition is improved. Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
The innovation points of the embodiment of the invention comprise:
after video images are obtained through sampling, a gradient probability histogram corresponding to each video image is calculated, and compared with a clear license plate image gradient probability histogram distribution template and a fuzzy license plate image gradient probability histogram distribution template which are obtained through pre-calculation, the definition confidence coefficient of the video images is determined, and then license plate recognition can be carried out on only the video images with the higher definition confidence coefficient, namely, the fuzzy images can be removed from all the video images, and license plate recognition is carried out only by utilizing the clear images, so that the accuracy of license plate recognition can be improved. And when the final license plate recognition result is determined according to the plurality of video images, the definition confidence degrees corresponding to the video images can be combined for calculation, so that the accuracy of license plate recognition can be further improved. The number of the character bits included in the license plate recognition result output by the convolutional neural network is consistent with the number of the license plates, so that the condition of more or less recognition characters can be avoided, and the accuracy of license plate recognition is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is to be understood that the drawings in the following description are of some embodiments of the invention only. For a person skilled in the art, without inventive effort, further figures can be obtained from these figures.
Fig. 1 is a schematic flowchart of a license plate recognition method based on deep learning according to an embodiment of the present invention.
FIG. 2 is a diagram of an x-direction filter and a y-direction filter according to an embodiment of the present invention.
Fig. 3 is another schematic flow chart of a license plate recognition method based on deep learning according to an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a license plate recognition device based on deep learning according to an embodiment of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely a few embodiments of the invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses a license plate recognition method and device based on deep learning, which can improve the accuracy of license plate recognition. The following provides a detailed description of embodiments of the invention.
Fig. 1 is a schematic flow chart of a license plate recognition method based on deep learning according to an embodiment of the present invention. The method is applied to the electronic equipment. The method specifically comprises the following steps.
S110: sampling is carried out on the basis of a preset sampling period in a monitoring video containing a target license plate, and a plurality of video images containing the target license plate are obtained.
In the embodiment of the invention, the foreground detection and tracking functions of the license plate recognition system can be called to recognize the monitoring video containing the same vehicle, and the license plate of the same vehicle can be called as a target license plate. The monitoring video can be a monitoring video collected in scenes such as a community entrance, a high-speed gate and the like.
The preset sampling period may be, for example, 100 milliseconds, 500 milliseconds, 1 second, or may be set according to different scenarios, which is not limited in the embodiment of the present invention.
S120: carrying out gray level conversion on the multiple video images to obtain multiple gray level video images; and normalizing the plurality of gray level video images to a preset size to obtain a plurality of target video images.
In the embodiment of the present invention, the video image obtained by sampling is a color image. In order to calculate the gradient probability histogram of each video image, the video image may be subjected to gray scale conversion, that is, the video image is converted into an image with image pixel values of 0 to 255, which is referred to as a gray scale video image.
After obtaining each gray scale video image, the size of each gray scale video image may be normalized, for example, each gray scale video image may be normalized to 32 × 240 pixels, and the normalized image may be referred to as a target video image. Therefore, gradient differences caused by different sizes of the gray-scale video images can be avoided, the influence on the license plate recognition result when the license plate recognition result is calculated according to the gradient probability histogram is further reduced, and the accuracy of the license plate recognition is improved.
S130: performing convolution operation on each target video image through a preset x-direction filter to obtain an x-edge image corresponding to each target video image; performing convolution operation on each target video image through a preset y-direction filter to obtain a y-edge image corresponding to each target video image; and calculating the gradient image corresponding to each target video image according to each x edge image and each corresponding y edge image.
In the embodiment of the invention, a sobel algorithm can be used for calculating the gradient probability histogram of each target video image. Specifically, an x-edge image and a y-edge image of each target video image may be calculated first. For example, assuming that the input target video image is I, the x-edge image Gx and the y-edge image Gy corresponding thereto are:
Gx = Sx*I，Gy = Sy*I
wherein, is a convolution operation; sx is an x-direction filter, and Sy is a y-direction filter. In one example, sx and Sy are shown in FIG. 2.
In one implementation, when the gradient image corresponding to each target video image is calculated according to each x-edge image and each corresponding y-edge image, the pixel value of any point in the gradient image corresponding to any target video image can be calculated according to the following formulaG（x，y）：
GxFor any pixel value of any point in the x-edge image corresponding to any target video image,Gythe pixel value of the point in the y-edge image corresponding to the target video image.
S140: normalizing the pixel values of the gradient images to obtain normalized gradient images, carrying out histogram statistics on the normalized gradient images aiming at each normalized gradient image to obtain the statistical number of the pixel values in the normalized gradient images, calculating the probability value of the statistical number in the total pixel number of the normalized gradient images, and constructing a gradient probability histogram corresponding to the normalized gradient images containing the probability values.
In an implementation manner, when the pixel values of each gradient image are normalized to obtain each normalized gradient image, the pixel values of each gradient image may be updated according to the following formula:
the pixel values of the gradient images are normalized, so that the problem of inaccurate matching caused by overlarge gradient variance can be solved, the calculation complexity of subsequent histogram statistics can be reduced, and the algorithm efficiency is improved.
After obtaining each normalized gradient image, histogram statistics can be further performed on each normalized gradient image to obtain the statistical number of each pixel value in each normalized gradient image. That is, for each normalized gradient image, 256 statistical quantities may be obtained, which may be represented as a quantity b1 of pixel values 0, a quantity b2 of pixel values 1, and so on, a quantity b255 of pixel values 255.
In an implementation manner, when the probability value of each statistical quantity in the total pixel number of the normalized gradient image is calculated for each normalized gradient image, any statistical quantity can be calculated according to the following formulabiProbability value of total pixel number of the normalized gradient imagepi：
bj is the number of pixel points with the pixel value j in the normalized gradient image.
S150: acquiring a gradient probability histogram distribution template of a clear license plate image and a gradient probability histogram distribution template of a fuzzy license plate image which are obtained through pre-calculation, calculating a first cosine distance between a gradient probability histogram corresponding to a normalized gradient image and the gradient probability histogram distribution template of the clear license plate image and a second cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the fuzzy license plate image aiming at each normalized gradient image, and determining a definition confidence coefficient corresponding to the normalized gradient image according to the sizes of the first cosine distance and the second cosine distance; the clear license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired clear image, and the fuzzy license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired fuzzy image.
In the embodiment of the invention, the license plate image can be collected in advance, and the manual work is divided into a clear category and a fuzzy category. And calculating a gradient probability histogram according to the mode, and averaging the obtained probability histogram to obtain a clear license plate image gradient probability histogram distribution template binC and a fuzzy license plate image gradient probability histogram distribution template binB.
Specifically, as shown in fig. 3, the construction process of the gradient probability histogram distribution template of the clear license plate image and the gradient probability histogram distribution template of the blurred license plate image may include the following steps.
S310: and acquiring a sample video image, determining the sample video image with the definition greater than a preset threshold as a clear image, and determining the sample video image with the definition not greater than the preset threshold as a blurred image.
The sample video image can be an image which is collected by monitoring equipment and contains a license plate. When the electronic device performs image classification, the preset threshold value can be determined according to an empirical value, and the specific value of the preset threshold value is not limited in the embodiment of the invention. Alternatively, the image classification may be performed manually, and this is all possible, and the image classification method is not limited in the embodiment of the present invention.
S320: and calculating a gradient probability histogram corresponding to each clear image and a gradient probability histogram corresponding to each fuzzy image.
Specifically, the gradient probability histogram corresponding to each sharp image and the gradient probability histogram corresponding to each blurred image may be calculated according to steps similar to steps S120 to S140, which are not described herein again.
S330: calculating a first average value of the probability values corresponding to the same pixel values in the gradient probability histograms corresponding to the clear images, and constructing a clear license plate image gradient probability histogram distribution template containing the first average values; and calculating a second average value of the probability values corresponding to the same pixel values in the gradient probability histogram corresponding to each fuzzy image, and constructing to obtain a fuzzy license plate image gradient probability histogram distribution template containing each second average value.
The clear license plate image gradient probability histogram distribution template and the fuzzy license plate image gradient probability histogram distribution template are obtained through calculation, and when license plate recognition is carried out, the image to be recognized can be subjected to definition division, so that license plate recognition is carried out only according to the clear image, and the accuracy of license plate recognition is improved.
In one implementation, when calculating, for each normalized gradient image, a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and a gradient probability histogram distribution template of a clear license plate image, the first cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the clear license plate image may be specifically calculated according to the following formula:
xiis the probability value corresponding to the pixel value i in the gradient probability histogram corresponding to the normalized gradient image,yiand obtaining a probability value corresponding to the pixel value i in the distribution template of the gradient probability histogram of the clear license plate image.
Similarly, when the second cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the distribution template of the gradient probability histogram of the blurred license plate image is calculated, the calculation can also be performed according to the formula, and at this time, the calculation is performed according to the formulayiNamely the probability value corresponding to the pixel value i in the gradient probability histogram distribution template of the fuzzy license plate image.
When the definition confidence corresponding to the normalized gradient image is determined according to the sizes of the first cosine distance and the second cosine distance, when the first cosine distance is greater than the second cosine distance, the definition confidence corresponding to the normalized gradient image is determined to be the first cosine distance; and when the first cosine distance is not greater than the second cosine distance, determining that the definition confidence coefficient corresponding to the normalized gradient image is 0.
The confidence degree describes the credibility of the definition of the current video image, and can be used for classification of the definition and the fuzzy image and also can be used for weighted fusion of subsequent identification results.
S160: inputting the video image corresponding to the normalized gradient image with the definition confidence coefficient greater than the preset confidence coefficient threshold value into a convolutional neural network obtained by pre-training to obtain a license plate recognition result corresponding to each video image; obtaining a final recognition result of the target license plate according to the license plate recognition result corresponding to each video image and the definition confidence coefficient of the normalized gradient image corresponding to each video image; the license plate recognition result comprises a plurality of characters at preset positions.
In the embodiment of the invention, the convolutional neural network for recognizing the license plate can be obtained by pre-training. For example, a convolutional neural network with googleNet as a backbone network can be constructed, a license plate image labeled with a license plate result is used as a training sample, and a gradient back propagation algorithm is adopted for training to obtain the convolutional neural network.
The loss function of the convolutional neural network may be:
wherein T is the total number of character categories, M is the number of character digits contained in a license plate, and y is the number of characters contained in the license plate when the obtained classification result is correct j Is 1, otherwise is 0;S j preferred posterior probabilities for convolutional neural network output:
a k the resulting distance identified as the character k is estimated for the convolutional neural network,a j the resulting distance identified as character j is estimated for the convolutional neural network.
When the license plate recognition is carried out, the video image corresponding to the normalized gradient image with the definition confidence coefficient larger than the preset confidence coefficient threshold value can be input into the convolutional neural network obtained through training, and the license plate recognition result corresponding to each video image is obtained. The convolutional neural network is an isometric character string recognition, and the output result contains a plurality of characters with preset bits, such as 7.
When the final recognition result of the target license plate is obtained according to the license plate recognition result corresponding to each video image and the definition confidence coefficient of the normalized gradient image corresponding to each video image, specifically, the h character in the target license plate can be pointed to according to the arrangement sequence of the license plate recognition results, when the preferred recognition results of the h character in partial images in the video images are all k, and the preferred posterior probability of each part of image isS k And when the sum of the products of the h characters and the definition confidence degrees of the corresponding normalized gradient images is maximum, determining that the final recognition result of the h character of the target license plate is k. Wherein the preferred posterior probability of each video image is output by the convolutional neural network.
That is to say, after image quality classification, the definition confidence corresponding to the license plate image of M video images entering the recognition process is conf 1-confM, the corresponding recognition result sequence is resi = { c1, c2, …, cT }, 0< = i < = M, and assuming that the number of character types is N, the recognition result of weighted fusion of the jth character in the license plate image is:
cj = k, max (scorek), 0< = k < = N, scorek being the sum of the products of the preferred posterior probability and the sharpness confidence for all sharp images with the recognition of a preferred result being the character class k.
As can be seen from the above, the license plate recognition method based on deep learning provided in the embodiments of the present invention can calculate the gradient probability histogram corresponding to each video image after the video images are obtained by sampling, compare the gradient probability histogram distribution template of the clear license plate image and the gradient probability histogram distribution template of the blurred license plate image obtained by pre-calculation, determine the sharpness confidence of the video images, and further perform license plate recognition on only the video images with a high sharpness confidence, that is, remove blurred images from all the video images, and perform license plate recognition only using the sharp images, thereby improving the accuracy of license plate recognition. And when the final license plate recognition result is determined according to the plurality of video images, the definition confidence degrees corresponding to the video images can be combined for calculation, so that the accuracy of license plate recognition can be further improved. The number of the character bits included in the license plate recognition result output by the convolutional neural network in the embodiment is consistent with the number of the license plates, so that the condition of more or less recognition characters can be avoided, and the accuracy of license plate recognition is improved.
As shown in fig. 4, a license plate recognition device based on deep learning according to an embodiment of the present invention may include:
the video sampling module 410 is used for sampling a monitoring video containing a target license plate based on a preset sampling period to obtain a plurality of video images containing the target license plate;
a gray level conversion module 420, configured to perform gray level conversion on the multiple video images to obtain multiple gray level video images; normalizing the multiple gray level video images to a preset size to obtain multiple target video images;
a filtering operation module 430, configured to perform convolution operation on each target video image through a preset x-direction filter, to obtain an x-edge image corresponding to each target video image; performing convolution operation on each target video image through a preset y-direction filter to obtain a y-edge image corresponding to each target video image; calculating a gradient image corresponding to each target video image according to each x edge image and each corresponding y edge image;
a histogram statistic module 440, configured to normalize the pixel values of each gradient image to obtain each normalized gradient image, perform histogram statistics on the normalized gradient image for each normalized gradient image to obtain a statistical number of each pixel value in the normalized gradient image, calculate a probability value of each statistical number in the total pixel number of the normalized gradient image, and construct a gradient probability histogram corresponding to the normalized gradient image and including each probability value;
a definition calculating module 450, configured to obtain a gradient probability histogram distribution template of a clear license plate image and a gradient probability histogram distribution template of a fuzzy license plate image, which are obtained through pre-calculation, calculate, for each normalized gradient image, a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the clear license plate image, and a second cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the fuzzy license plate image, and determine a definition confidence corresponding to the normalized gradient image according to the magnitudes of the first cosine distance and the second cosine distance; the clear license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired clear image, and the fuzzy license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired fuzzy image;
the license plate recognition module 460 is configured to input a video image corresponding to the normalized gradient image with the definition confidence greater than a preset confidence threshold into a convolutional neural network obtained through pre-training, so as to obtain a license plate recognition result corresponding to each video image; obtaining a final recognition result of the target license plate according to the license plate recognition result corresponding to each video image and the definition confidence coefficient of the normalized gradient image corresponding to each video image; the license plate recognition result comprises a plurality of characters at preset positions.
Optionally, the filtering operation module 430 is specifically configured to calculate a pixel value of any point in a gradient image corresponding to any target video image according to the following formulaG（x，y）：
GxIs the pixel value of any point in the x-edge image corresponding to any target video image,Gythe pixel value of the point in the y edge image corresponding to any target video image is obtained;
the histogram statistics module 440 is specifically configured to update the pixel value of each gradient image according to the following formula:
optionally, the histogram statistic module 440 is specifically configured to calculate any statistical quantity according to the following formulabiProbability value of total pixel number of the normalized gradient imagepi：
bj is the number of pixel points with the pixel value j in the normalized gradient image.
Optionally, the sharpness calculating module 450 is specifically configured to calculate a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and the distribution template of the gradient probability histogram of the sharp license plate image according to the following formula:
xiis the probability value corresponding to the pixel value i in the gradient probability histogram corresponding to the normalized gradient image,yiand obtaining a probability value corresponding to the pixel value i in the gradient probability histogram distribution template of the clear license plate image.
Optionally, the sharpness calculating module 450 is specifically configured to determine, when the first cosine distance is greater than the second cosine distance, that the sharpness confidence corresponding to the normalized gradient image is the first cosine distance; and when the first cosine distance is not greater than the second cosine distance, determining that the definition confidence coefficient corresponding to the normalized gradient image is 0.
Optionally, the license plate recognition module 460 is specifically configured to, for the h-th character in the target license plate according to the arrangement sequence of the license plate recognition results, when the preferred recognition results of the h-th character in the partial images in the video image are all k, and the preferred posterior probabilities of the partial imagesS k And when the sum of the products of the h characters and the definition confidence degrees of the corresponding normalized gradient images is maximum, determining that the final recognition result of the h character of the target license plate is k.
Optionally, the apparatus further comprises:
the image acquisition module is used for acquiring a sample video image, determining the sample video image with the definition greater than a preset threshold value as a clear image, and determining the sample video image with the definition not greater than the preset threshold value as a blurred image;
the histogram calculation module is used for calculating a gradient probability histogram corresponding to each clear image and a gradient probability histogram corresponding to each fuzzy image;
the histogram distribution module construction module is used for calculating a first average value of probability values corresponding to the same pixel value in a gradient probability histogram corresponding to each clear image, and constructing a clear license plate image gradient probability histogram distribution template containing each first average value; and calculating a second average value of the probability values corresponding to the same pixel value in the gradient probability histogram corresponding to each fuzzy image, and constructing to obtain a fuzzy license plate image gradient probability histogram distribution template containing each second average value.
Optionally, the convolutional neural network is trained by using a gradient back propagation algorithm, and a loss function of the convolutional neural network is as follows:
wherein T is the total number of character categories, M is the number of character digits contained in a license plate, and y is the number of characters contained in the license plate when the obtained classification result is correct j Is 1, otherwise is 0;S j preferred posterior probabilities output for the convolutional neural network:
a k estimating a resulting distance identified as a character k for the convolutional neural network,a j estimating a resulting distance identified as a character j for the convolutional neural network.
As can be seen from the above, the license plate recognition device based on deep learning provided in the embodiments of the present invention can calculate the gradient probability histogram corresponding to each video image after obtaining the video images through sampling, compare the gradient probability histogram distribution template of the clear license plate image and the gradient probability histogram distribution template of the blurred license plate image obtained through pre-calculation, determine the definition confidence of the video images, and further perform license plate recognition on only the video images with a high definition confidence, that is, remove the blurred images from all the video images, and perform license plate recognition only using the clear images, thereby improving the accuracy of license plate recognition. And when the final license plate recognition result is determined according to the plurality of video images, the definition confidence degrees corresponding to the video images can be combined for calculation, so that the accuracy of license plate recognition can be further improved. The number of the character bits included in the license plate recognition result output by the convolutional neural network in the embodiment is consistent with the number of the license plates, so that the condition of more or less recognition characters can be avoided, and the accuracy of license plate recognition is improved.
The above device embodiment corresponds to the method embodiment, and has the same technical effect as the method embodiment, and for the specific description, refer to the method embodiment. The device embodiment is obtained based on the method embodiment, and for specific description, reference may be made to the method embodiment section, which is not described herein again.
Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.
Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A license plate recognition method based on deep learning is characterized by comprising the following steps:
sampling is carried out on the basis of a preset sampling period in a monitoring video containing a target license plate, so that a plurality of video images containing the target license plate are obtained;
carrying out gray level conversion on the multiple video images to obtain multiple gray level video images; normalizing the multiple gray level video images to a preset size to obtain multiple target video images;
performing convolution operation on each target video image through a preset x-direction filter to obtain an x-edge image corresponding to each target video image; performing convolution operation on each target video image through a preset y-direction filter to obtain a y-edge image corresponding to each target video image; calculating a gradient image corresponding to each target video image according to each x edge image and each corresponding y edge image;
normalizing the pixel values of the gradient images to obtain normalized gradient images, performing histogram statistics on the normalized gradient images aiming at each normalized gradient image to obtain the statistical number of the pixel values in the normalized gradient images, calculating the probability value of the statistical number in the total pixel number of the normalized gradient images, and constructing a gradient probability histogram corresponding to the normalized gradient images and containing the probability value;
acquiring a gradient probability histogram distribution template of a clear license plate image and a gradient probability histogram distribution template of a fuzzy license plate image which are obtained through pre-calculation, calculating a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the clear license plate image and a second cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the fuzzy license plate image aiming at each normalized gradient image, and determining a definition confidence coefficient corresponding to the normalized gradient image according to the sizes of the first cosine distance and the second cosine distance; the clear license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired clear image, and the fuzzy license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired fuzzy image;
inputting the video images corresponding to the normalized gradient images with the definition confidence degrees larger than a preset confidence degree threshold value into a convolutional neural network obtained by pre-training to obtain license plate recognition results corresponding to the video images; obtaining a final recognition result of the target license plate according to the license plate recognition result corresponding to each video image and the definition confidence coefficient of the normalized gradient image corresponding to each video image; the license plate recognition result comprises a plurality of characters with preset positions.
2. The deep learning-based license plate recognition method of claim 1, wherein the step of calculating a gradient image corresponding to each target video image according to each x-edge image and each corresponding y-edge image comprises:
calculating the pixel value of any point in the gradient image corresponding to any target video image according to the following formulaG（x，y）：
GxFor any of the target video image pairsThe pixel value of any point in the corresponding x-edge image,Gythe pixel value of the point in the y edge image corresponding to any target video image is obtained;
the step of normalizing the pixel values of each gradient image to obtain each normalized gradient image comprises:
updating the pixel values of each of the gradient images according to the following formula:
3. the deep learning-based license plate recognition method of claim 1, wherein the step of calculating the probability value of each statistical quantity in the total pixel quantity of the normalized gradient image comprises:
calculating any statistical quantity according to the following formulabiProbability value of total pixel number of the normalized gradient imagepi：
bj is the number of pixel points with the pixel value j in the normalized gradient image.
4. The deep learning-based license plate recognition method of claim 1, wherein the step of calculating the first cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the clear license plate image comprises:
calculating a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and a gradient probability histogram distribution template of the clear license plate image according to the following formula:
xiis the probability value corresponding to the pixel value i in the gradient probability histogram corresponding to the normalized gradient image,yiand obtaining a probability value corresponding to the pixel value i in the gradient probability histogram distribution template of the clear license plate image.
5. The deep learning-based license plate recognition method of claim 1, wherein the step of determining the sharpness confidence corresponding to the normalized gradient image according to the first cosine distance and the second cosine distance comprises:
when the first cosine distance is greater than the second cosine distance, determining the definition confidence coefficient corresponding to the normalized gradient image as the first cosine distance;
and when the first cosine distance is not greater than the second cosine distance, determining that the definition confidence coefficient corresponding to the normalized gradient image is 0.
6. The deep learning-based license plate recognition method of claim 1, wherein the step of obtaining the final recognition result of the target license plate according to the license plate recognition result corresponding to each video image and the definition confidence of the normalized gradient image corresponding to each video image comprises:
according to the arrangement sequence of the license plate recognition results, aiming at the h character in the target license plate, when the preferred recognition results of the h character in partial images in the video image are all k, and the preferred posterior probability of each partial imageS k And when the sum of the product of the definition confidence coefficients of the corresponding normalized gradient images is maximum, determining that the final recognition result of the h-th character of the target license plate is k.
7. The deep learning-based license plate recognition method of claim 1, wherein the construction process of the gradient probability histogram distribution template of the clear license plate image and the gradient probability histogram distribution template of the fuzzy license plate image comprises:
acquiring a sample video image, determining the sample video image with the definition greater than a preset threshold value as a clear image, and determining the sample video image with the definition not greater than the preset threshold value as a blurred image;
calculating a gradient probability histogram corresponding to each clear image and a gradient probability histogram corresponding to each fuzzy image;
calculating a first average value of the probability values corresponding to the same pixel values in the gradient probability histogram corresponding to each clear image, and constructing and obtaining a clear license plate image gradient probability histogram distribution template containing each first average value; and calculating a second average value of the probability values corresponding to the same pixel values in the gradient probability histograms corresponding to the fuzzy images, and constructing a fuzzy license plate image gradient probability histogram distribution template containing the second average values.
8. The deep learning-based license plate recognition method of claim 1, wherein the convolutional neural network is trained by a gradient back propagation algorithm, and a loss function of the convolutional neural network is as follows:
wherein T is the total number of character categories, M is the number of character digits contained in a license plate, and y is the number of characters contained in the license plate when the obtained classification result is correct j Is 1, otherwise is 0;S j preferred posterior probabilities output for the convolutional neural network:
a k estimating a resulting distance identified as a character k for the convolutional neural network,a j for the convolutional neural networkThe resulting distance identified as character j is estimated.
9. A license plate recognition device based on deep learning, the device comprising:
the video sampling module is used for sampling a monitoring video containing a target license plate based on a preset sampling period to obtain a plurality of video images containing the target license plate;
the gray level conversion module is used for carrying out gray level conversion on the plurality of video images to obtain a plurality of gray level video images; normalizing the multiple gray level video images to a preset size to obtain multiple target video images;
the filtering operation module is used for performing convolution operation on each target video image through a preset x-direction filter to obtain an x-edge image corresponding to each target video image; respectively carrying out convolution operation on each target video image through a preset y-direction filter to obtain a y-edge image corresponding to each target video image; calculating a gradient image corresponding to each target video image according to each x edge image and each corresponding y edge image;
a histogram statistic module, configured to normalize the pixel values of each gradient image to obtain each normalized gradient image, perform histogram statistics on the normalized gradient image for each normalized gradient image to obtain a statistical number of each pixel value in the normalized gradient image, calculate a probability value of each statistical number in the total pixel number of the normalized gradient image, and construct a gradient probability histogram corresponding to the normalized gradient image and including each probability value;
the sharpness calculation module is used for acquiring a gradient probability histogram distribution template of a sharp license plate image and a gradient probability histogram distribution template of a fuzzy license plate image which are obtained by pre-calculation, calculating a first cosine distance between a gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the sharp license plate image and a second cosine distance between the gradient probability histogram corresponding to the normalized gradient image and the gradient probability histogram distribution template of the fuzzy license plate image aiming at each normalized gradient image, and determining the sharpness confidence coefficient corresponding to the normalized gradient image according to the sizes of the first cosine distance and the second cosine distance; the clear license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired clear image, and the fuzzy license plate image gradient probability histogram distribution template is obtained by calculation according to a pre-acquired fuzzy image;
the license plate recognition module is used for inputting the video image corresponding to the normalized gradient image with the definition confidence coefficient greater than a preset confidence coefficient threshold value into a convolutional neural network obtained by pre-training to obtain a license plate recognition result corresponding to each video image; obtaining a final recognition result of the target license plate according to the license plate recognition result corresponding to each video image and the definition confidence coefficient of the normalized gradient image corresponding to each video image; the license plate recognition result comprises a plurality of characters at preset positions.
10. The deep learning-based license plate recognition device of claim 9, wherein the filtering operation module is specifically configured to calculate a pixel value of any point in a gradient image corresponding to any one of the target video images according to the following formulaG （x，y）：
GxThe pixel value of any point in the x-edge image corresponding to any target video image,Gythe pixel value of the point in the y edge image corresponding to any target video image is obtained;
the histogram statistic module is specifically configured to update the pixel value of each gradient image according to the following formula:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010176205.0A CN111340041B (en) | 2020-03-13 | 2020-03-13 | License plate recognition method and device based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010176205.0A CN111340041B (en) | 2020-03-13 | 2020-03-13 | License plate recognition method and device based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111340041A CN111340041A (en) | 2020-06-26 |
| CN111340041B true CN111340041B (en) | 2023-03-24 |
Family
ID=71186110
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010176205.0A Active CN111340041B (en) | 2020-03-13 | 2020-03-13 | License plate recognition method and device based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111340041B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177527B (en) * | 2021-05-27 | 2022-09-23 | 安阳工学院 | Vehicle type recognition method and device |
| CN113505790B (en) * | 2021-07-15 | 2024-07-19 | 普联技术有限公司 | License plate processing method, device, equipment and computer readable storage medium |
| CN113763378B (en) * | 2021-09-22 | 2023-07-18 | 北京中电兴发科技有限公司 | Scoring method for license plate image non-reference quality analysis |
| CN114242209B (en) * | 2021-11-02 | 2024-07-05 | 深圳市智影医疗科技有限公司 | Medical image preprocessing method and system |
| CN117690120B (en) * | 2023-12-15 | 2024-10-29 | 北京卓视智通科技有限责任公司 | Vehicle monitoring method and system for mobile equipment snapshot license plate |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845478A (en) * | 2016-12-30 | 2017-06-13 | 同观科技（深圳）有限公司 | The secondary licence plate recognition method and device of a kind of character confidence level |
| WO2018028306A1 (en) * | 2016-08-11 | 2018-02-15 | 杭州海康威视数字技术股份有限公司 | Method and device for recognizing license plate number |
| CN108388896A (en) * | 2018-02-09 | 2018-08-10 | 杭州雄迈集成电路技术有限公司 | A kind of licence plate recognition method based on dynamic time sequence convolutional neural networks |
-
2020
- 2020-03-13 CN CN202010176205.0A patent/CN111340041B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018028306A1 (en) * | 2016-08-11 | 2018-02-15 | 杭州海康威视数字技术股份有限公司 | Method and device for recognizing license plate number |
| CN106845478A (en) * | 2016-12-30 | 2017-06-13 | 同观科技（深圳）有限公司 | The secondary licence plate recognition method and device of a kind of character confidence level |
| CN108388896A (en) * | 2018-02-09 | 2018-08-10 | 杭州雄迈集成电路技术有限公司 | A kind of licence plate recognition method based on dynamic time sequence convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 车辆牌照识别算法的研究与实现;牛进才等;《电子测量技术》;20180323(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111340041A (en) | 2020-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111340041B (en) | License plate recognition method and device based on deep learning | |
| CN111582150B (en) | Face quality assessment method, device and computer storage medium | |
| CN111784747B (en) | Multi-target vehicle tracking system and method based on key point detection and correction | |
| CN111310850B (en) | License plate detection model construction method and system, license plate detection method and system | |
| CN113591967A (en) | Image processing method, device and equipment and computer storage medium | |
| CN110956615B (en) | Image quality evaluation model training method and device, electronic equipment and storage medium | |
| CN111553310B (en) | Security inspection image acquisition method and system based on millimeter wave radar and security inspection equipment | |
| CN113989794B (en) | License plate detection and recognition method | |
| CN113780110A (en) | Method and device for detecting weak and small targets in image sequence in real time | |
| CN113780492A (en) | Two-dimensional code binarization method, device and equipment and readable storage medium | |
| CN113920168B (en) | Image tracking method in audio/video control equipment | |
| CN112990225B (en) | Image target identification method and device in complex environment | |
| CN114359333A (en) | Moving object extraction method and device, computer equipment and storage medium | |
| CN111667469B (en) | Lung disease classification method, device and equipment | |
| CN114187247A (en) | Ampoule bottle printing character defect detection method based on image registration | |
| CN115083008A (en) | Moving object detection method, device, equipment and storage medium | |
| CN110827319A (en) | Improved Staple target tracking method based on local sensitive histogram | |
| CN114663731B (en) | Training method and system of license plate detection model, and license plate detection method and system | |
| CN117423040A (en) | Visual garbage identification method for unmanned garbage sweeper based on improved YOLOv8 | |
| CN116152758A (en) | Intelligent real-time accident detection and vehicle tracking method | |
| CN116206297A (en) | Video stream real-time license plate recognition system and method based on cascade neural network | |
| CN113255472B (en) | Face quality evaluation method and system based on random embedding stability | |
| CN116229369B (en) | Method, device and equipment for detecting people flow and computer readable storage medium | |
| CN115713501B (en) | Detection processing method and system suitable for blurred pictures of camera | |
| CN112712004B (en) | Face detection system, face detection method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |