CN115272222A

CN115272222A - Method, device and equipment for processing road detection information and storage medium

Info

Publication number: CN115272222A
Application number: CN202210878033.0A
Authority: CN
Inventors: 李永杰; 许亮; 曹玉社; 李峰
Original assignee: Zhongkehai Micro Beijing Technology Co ltd
Current assignee: Zhongkehai Micro Beijing Technology Co ltd
Priority date: 2022-07-25
Filing date: 2022-07-25
Publication date: 2022-11-01

Abstract

The application relates to a method, a device, equipment and a storage medium for processing road detection information, which relate to the technical field of artificial intelligence, and the method comprises the following steps: the method comprises the steps of obtaining collected image information of a target road, carrying out pavement detection and identification processing through a pre-trained road detection model according to the collected image information, obtaining a pavement prediction result corresponding to the target road, wherein the pavement prediction result comprises a pavement disease type, a pavement signboard type, the number of diseases corresponding to the pavement disease type and the number of signboards corresponding to the pavement signboard type, determining geographic position information and pavement evenness information corresponding to the pavement disease type or the pavement signboard type based on the obtained positioning data and acceleration data, and determining a pavement detection result of the target road according to the geographic position information, the pavement evenness information and the collected image information, so that the detection efficiency of the pavement diseases and the pavement signboards is improved, and the problem caused by manual detection of the pavement condition in the prior art is solved.

Description

Method, device and equipment for processing road detection information and storage medium

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, a device, and a storage medium for processing road detection information.

Background

Along with infrastructure's ability promotion, the cover surface of highway road network is more and more wide, and the road surface condition of highway is great to driving experience and driving safety's influence, for ensureing driving safety, when the road surface condition such as collapse, crack, unevenness and road signboard damage appear, needs the staff to carry out timely restoration to the road.

The existing mode for detecting the condition of the road surface mainly records the condition of the road surface by watching and checking the condition of the road surface by workers along the road, but the existing mode for detecting the road surface has lower detection efficiency and large working strength.

Disclosure of Invention

In order to solve the technical problems described above or at least partially solve the technical problems, the present application provides a method, an apparatus, a device and a storage medium for processing road detection information.

In a first aspect, the present application provides a method for processing road detection information, including:

acquiring collected image information of a target road;

according to the collected image information, carrying out pavement detection and identification processing through a pre-trained road detection model to obtain a pavement prediction result corresponding to the target road, wherein the pavement prediction result comprises a pavement disease type, a pavement signboard type, the number of diseases corresponding to the pavement disease type and the number of signboards corresponding to the pavement signboard type;

determining geographical position information and road surface evenness information corresponding to the road surface disease type or the road surface signboard type based on the obtained positioning data and acceleration data;

and determining a road surface detection result of the target road according to the geographic position information, the road surface flatness information and the collected image information.

Optionally, the acquiring of the collected image information of the target road includes:

recording a target road through a camera to obtain road video information;

and extracting image frames from the road video information, and determining the extracted image frames as the acquired image information.

Optionally, the road detection model includes a road target detection model and a road target tracking model, and the acquiring image information is used to perform road detection and identification processing through a pre-trained road detection model to obtain a road prediction result corresponding to the target road, including:

carrying out pavement target detection on the collected image information through the pavement target detection model to obtain a pavement disease detection result and a pavement signboard detection result, wherein the pavement disease identification result comprises pavement disease information and a pavement disease type corresponding to the pavement disease information, and the pavement signboard detection result comprises the pavement signboard information and a pavement hazard type corresponding to the pavement signboard information;

and tracking and identifying the pavement disease information and the pavement signboard information through the pavement target tracking model based on the acquired image information to obtain the disease number of the pavement disease type and the signboard number of the pavement signboard type.

Optionally, the detecting the road surface target to the collected image information by the road surface target detection model to obtain a road surface disease detection result and a road surface signboard detection result includes:

inputting the collected image information into the road surface target detection model;

detecting and identifying the road surface target in the acquired image information through the road surface target detection model to obtain a road surface target identification result;

if the confidence coefficient of the diseases in the pavement target recognition result meets the condition of the preset confidence coefficient of the diseases, determining the pavement target information in the pavement target recognition result as the pavement disease information, and determining the pavement disease type corresponding to the pavement disease information;

and if the confidence coefficient of the signboard in the pavement target recognition result meets the condition of the preset confidence coefficient of the signboard, determining the pavement target information in the pavement target recognition result as the pavement signboard information, and determining the type of the pavement disease corresponding to the pavement signboard information.

Optionally, before the road detection recognition processing is performed by using the pre-trained road detection model, the method further includes:

extracting image data to be trained and verification data corresponding to the image data to be trained from the sample data set;

performing data enhancement processing according to the image data to be trained to obtain enhanced image data corresponding to the image data to be trained;

and performing model training according to the enhanced image data and the verification data to obtain the road detection model.

Optionally, the verifying data includes a pavement defect verifying type and/or a pavement signboard verifying type, and the performing model training according to the enhanced image data and the verifying data to obtain the road detection model includes:

performing model training according to the enhanced image data, the pavement disease verification type and the pavement signboard verification type to obtain a pavement target detection model;

determining the type of the pavement target diseases according to the pavement disease detection result output by the pavement target detection model, and determining the type of the pavement target signboard according to the pavement signboard detection result output by the pavement target detection model;

determining the verification quantity of the pavement diseases corresponding to the pavement target disease types and the verification quantity of the pavement signs corresponding to the pavement target sign types according to the enhanced image data;

and performing model training according to the verification quantity of the pavement diseases, the verification quantity of the pavement signboard and the enhanced image data to obtain a pavement target tracking model.

Optionally, before extracting, from the sample data set, the image data to be trained and the verification data corresponding to the image data to be trained, the method further includes:

determining the category name of the road surface disease and the category name of the road surface signboard corresponding to the collected image information;

and generating the sample data set based on the acquired image information, the category name of the road surface disease and the category name of the road surface signboard.

In a second aspect, the present application provides a processing apparatus for road detection information, including:

the acquisition image information acquisition module is used for acquiring acquisition image information of the target road;

the road surface prediction result determining module is used for carrying out road surface detection and recognition processing through a pre-trained road detection model according to the acquired image information to obtain a road surface prediction result corresponding to the target road, wherein the road surface prediction result comprises a road surface disease type, a road surface signboard type, the number of diseases corresponding to the road surface disease type and the number of signboards corresponding to the road surface signboard type;

the information acquisition module is used for acquiring geographical position information and road surface evenness information corresponding to the road surface disease type or the road surface signboard type based on the acquired positioning data and acceleration data;

and the road surface detection result determining module is used for determining the road surface detection result of the target road according to the geographical position information, the road surface flatness information and the collected image information.

In a third aspect, the present application provides a device for processing road detection information, which includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;

a memory for storing a computer program;

a processor, configured to implement the steps of the method for processing road detection information according to any one of the embodiments of the first aspect when executing the program stored in the memory.

In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the method for processing road detection information according to any one of the embodiments of the first aspect.

To sum up, the embodiment of the application obtains the collected image information of the target road, performs pavement detection and identification processing through a pre-trained road detection model according to the collected image information to obtain a pavement prediction result corresponding to the target road, where the pavement prediction result includes a pavement disease type, a pavement signboard type, a quantity of diseases corresponding to the pavement disease type and a quantity of diseases corresponding to the pavement signboard type, and obtains geographical position information and pavement evenness information corresponding to the pavement disease type or the pavement signboard type based on the obtained positioning data and acceleration data, and determines a pavement detection result of the target road according to the geographical position information, the pavement evenness information and the collected image information, so as to improve the detection efficiency of the pavement disease and the pavement signboard, and solve the problem caused by manual detection of the pavement condition in the prior art.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.

Fig. 1 is a schematic flowchart of a method for processing road detection information according to an embodiment of the present disclosure;

FIG. 2 is a flow chart illustrating steps of a method for processing road detection information according to an alternative embodiment of the present application;

fig. 3 is a schematic diagram of a method for processing road detection information according to an example of the present application;

fig. 4 is a schematic flowchart of a method for processing road detection information according to an alternative example of the present application;

fig. 5 is an overall framework diagram of a road detection information processing method according to an alternative example of the present application;

FIG. 6 is a schematic diagram of a road detection information process in an alternative example of the present application;

fig. 7 is a flowchart illustrating steps of a method for processing road detection information according to an alternative embodiment of the present application;

FIG. 8 is a schematic diagram of a model training process provided by an example of the present application;

fig. 9 is a block diagram of a road detection information processing apparatus according to an embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of a road detection information processing device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the correlation technique, the current mode of detecting the condition on highway road surface is generally to watch the inspection along the way through the staff to the condition on road surface is taken notes, if take notes the condition such as the road surface condition that appears collapsing, crack, unevenness and road sign damage, this detection mode efficiency is lower and working strength is big, and under the great, the faster road conditions of speed of a motor vehicle of highway traffic, the staff carries out road surface inspection work and still has certain safe risk.

One of the concepts of the embodiment of the application is to provide a method for processing road detection information, which includes obtaining collected image information of a target road, performing road detection recognition processing through a pre-trained road detection model according to the collected image information, and obtaining a road prediction result corresponding to the target road, where the road prediction result includes a road disease type, a road signboard type, a quantity of diseases corresponding to the road disease type and a quantity of diseases corresponding to the road signboard type, determining geographical position information and road flatness information corresponding to the road disease type or the road signboard type based on the obtained positioning data and acceleration data, and determining a road detection result of the target road according to the geographical position information, the road flatness information and the collected image information, so as to improve detection efficiency of the road disease and the road signboard, and solve problems caused by manual detection of a road condition in the prior art.

For the purpose of facilitating understanding of the embodiments of the present application, the following description will be further explained with reference to the accompanying drawings and specific embodiments, which are not intended to limit the embodiments of the present application.

Fig. 1 is a schematic flowchart of a method for processing road detection information according to an embodiment of the present disclosure. As shown in fig. 1, the method for processing road detection information provided by the present application may specifically include the following steps:

and step 110, acquiring the acquired image information of the target road.

Specifically, a road on which pavement disease detection and pavement signboard detection are required can be used as a target road, which is not limited in the embodiment of the present application; the acquired image information may include an image corresponding to the target road, which is not limited in the embodiment of the present application. For example, the image of the target road may be acquired by a camera, which may be a high-definition camera, as the acquired image information, which is not limited by this example.

And 120, carrying out road surface detection and identification processing through a pre-trained road detection model according to the acquired image information to obtain a road surface prediction result corresponding to the target road.

The pavement prediction result comprises a pavement disease type, a pavement signboard type, a disease number corresponding to the pavement disease type and a signboard number corresponding to the pavement signboard type.

Specifically, the road surface damage may include a road surface crack, which may include a pit, a repair, a transverse crack, a longitudinal crack, a chap, and the like, which is not limited in the embodiment of the present application; the pavement marking board may include an indication sign, a road indication sign, a prohibition sign, a warning sign, a tourist area sign, and the like, which is not limited in the embodiment of the present application. Specifically, after the acquired image information is acquired, the acquired image information can be input into a pre-trained road detection model to perform road detection and identification processing through the road detection model to obtain a road surface prediction result corresponding to a target road, and then the road surface disease type, the number of diseases corresponding to the road surface disease type, the type of road surface signboard, the number of signboards corresponding to the type of road surface signboard and the like can be determined according to the road surface prediction result, so that the identification and detection efficiency of the road surface diseases and the road surface signboard is improved, the problem of safety risks existing in the manual detection of the road surface in the existing scheme is solved, and the method has the positive effects of safety and high efficiency.

In specific implementation, after the collected image information is input into the road detection model, the collected image information may be detected and identified by the road detection model to obtain the road surface disease information of the target road, the road surface disease type corresponding to the road surface disease information, the road surface signboard information, and the road surface signboard type corresponding to the road surface signboard information, where the road surface disease information may include a position corresponding to the road surface disease type, and the road surface signboard information may include a position corresponding to the road surface signboard type. And then, determining the quantity of the pavement diseases corresponding to the pavement disease type of the target road and the quantity of the pavement signs corresponding to the pavement sign type, and determining a pavement prediction result corresponding to the target road based on the pavement disease type, the quantity of the pavement diseases corresponding to the pavement disease type, the pavement sign type and the quantity of the pavement signs corresponding to the pavement sign type.

And step 130, determining geographical position information and road flatness information corresponding to the road surface disease type or the road surface signboard type based on the acquired positioning data and acceleration data.

Specifically, the Positioning data may include Global Positioning System (GPS) data, and the Positioning data may be used to determine geographical location information corresponding to a road surface disease type or a road surface signboard type, which is not limited in the embodiment of the present application; the acceleration data can be used for determining the road surface evenness corresponding to the type of the road surface disease or the type of the road surface signboard, and the road surface evenness can be used as the road surface evenness information, which is not limited by the embodiment of the application. Specifically, after the number of the road surface diseases and the number of the road surface signboard are determined, positioning data and acceleration data can be acquired, and geographical position information and road surface evenness information corresponding to the types of the road surface diseases or geographical position information and road surface evenness information corresponding to the types of the road surface signboard can be determined based on the acquired positioning data and acceleration data.

For example, when acquiring image information of a target road, current position information may be acquired in real time through a Global Positioning System (GPS) module to serve as Positioning data, and the flatness of the current road surface may be detected in real time through an acceleration sensor module to obtain acceleration data, and then geographical position information corresponding to a road surface disease type or a road surface signboard type may be determined based on the acquired GPS Positioning data, and road surface flatness information corresponding to the road surface disease type or the road surface signboard type may be determined based on the acquired acceleration data.

And 140, determining a road surface detection and identification result of the target road according to the geographical position information, the road surface flatness information and the collected image information.

Specifically, the road surface detection result may include all road surface disease types appearing on the target road, a geographical position corresponding to the road surface disease type, a number of road surface diseases corresponding to the road surface disease type, a road surface signboard type, a geographical position corresponding to the road surface signboard type, a number of road surface signboards corresponding to the road surface signboard type, and the like, which is not limited in the embodiment of the present application. Specifically, the embodiment of the application can determine the pavement detection and identification result of the target road according to the geographic position information, the pavement evenness information and the collected image information, so that the pavement cracks and the pavement signboard of the road do not need to be manually detected and recorded, the identification and detection efficiency of the pavement diseases and the pavement signboard is improved, and the workload of workers is reduced.

As an example, the detected road surface diseases and/or road surface identification boards may be framed in the collected image information, and geographical location information corresponding to the road surface diseases and the road surface identification boards, road surface flatness information, road surface disease types corresponding to the road surface diseases, road surface identification board types corresponding to the road surface identification boards, the number of diseases corresponding to the road surface disease types and the number of identification boards corresponding to the road surface identification board types in the collected image information, and the like may be displayed as the road surface detection and recognition result of the target road. Of course, it may also be that all the road surface disease types, the number of diseases corresponding to the road surface disease types, the types of the road surface signboard and the number of signs corresponding to the types of the road surface signboard in the target road are counted as the road surface detection and identification result of the target road, which is not limited in this example. And subsequently, json format information can be generated based on the road surface detection and identification result and returned to the user.

Therefore, according to the embodiment of the application, the collected image information of the target road is obtained, the road surface detection recognition processing is carried out through the pre-trained road detection model according to the collected image information, the road surface prediction result corresponding to the target road is obtained, the road surface prediction result comprises the road surface disease type, the road surface signboard type, the disease number corresponding to the road surface disease type and the disease number corresponding to the road surface signboard type, the geographical position information and the road surface evenness information corresponding to the road surface disease type or the road surface signboard type are obtained based on the obtained positioning data and the obtained acceleration data, and the road surface detection result of the target road is determined according to the geographical position information, the road surface evenness information and the collected image information, so that the detection efficiency of the road surface disease and the road surface signboard is improved, and the problem caused by the manual detection of the road surface condition in the prior art is solved.

Referring to fig. 2, a schematic flowchart illustrating steps of a method for processing road detection information according to an alternative embodiment of the present application is shown. The method for processing the road detection information may specifically include the following steps:

step 210, acquiring the acquired image information of the target road.

In an optional embodiment, the acquiring the image information of the target road in the embodiment of the present application may specifically include: recording a target road through a camera to obtain road video information; and extracting image frames from the road video information, and determining the extracted image frames as the collected image information. Specifically, the road video information may include a plurality of image frames, and all the image frames in the road video information may be extracted as the collected image information, for example, the image frames may be extracted from the collected road video information through an open source computer vision library (OpenCv), and then the road surface detection and identification may be performed according to the collected image information to determine the road surface detection result of the target road, and the accuracy of the road surface detection may be effectively improved by performing the road surface detection and identification on all the image frames in the road video information. Certainly, a frame extraction method may also be preset, and the image frames are extracted from the road video information according to the preset frame extraction method, for example, the preset method may be to extract one frame from every five image frames, and the extracted image frames are used as the acquired image information, so that the detection efficiency can be improved and the consumption of model resources can be reduced by reducing the image frames.

As an example, referring to fig. 3, the vehicle may be installed with a front-end camera, a Network Video Recorder (NVR), a power supply, an edge box, and the like, wherein the front-end camera may be a ToF camera, which is not limited in this example; the edge box may be an edge computing device that may be deployed with a neural network model, such as a pre-trained road detection model may be built in, which is not limited in this example. When the road surface diseases and the road signboard of the target road need to be detected, the vehicle can run on the target road, and the front-end camera can record the target road so as to acquire the road video information of the target road in real time. The front-end camera can be connected with the vehicle-mounted NVR to send collected road video information to the vehicle-mounted NVR, then the vehicle-mounted NVR can receive the road video information sent by the front-end camera, can store the road video information, can extract image frames of the road video information to serve as collected image information to be transmitted to the edge box, and after the edge box receives the collected image information, the collected image information can be detected and identified through a built-in road detection model, so that a road surface detection result of a target road can be obtained.

Further, the vehicle NVR may also provide a network for the edge box, for example, a network hotspot (Wi-Fi) and/or a fifth generation Mobile Communication technology (5G) network may be provided for the edge box, and the edge box may be connected to the user equipment through the network provided by the vehicle NVR, as shown in fig. 3, in a case that the user equipment is a tablet, the edge box may be connected to the tablet through the Wi-Fi network and/or a 5G network, so that a road surface detection result of the target road may be subsequently sent to the user equipment, for example, the road surface detection result may be sent to the tablet through the Wi-Fi network and/or the 5G network.

And step 220, carrying out pavement target detection on the collected image information through the pavement target detection model to obtain a pavement disease detection result and a pavement signboard detection result.

The road detection model comprises a road target detection model and a road target tracking model, the road disease identification result comprises road disease information and a road disease type corresponding to the road disease information, and the road signboard detection result comprises road signboard information and a road hazard type corresponding to the road signboard information.

Specifically, the pavement target may include a pavement defect and a pavement signboard, which is not limited in the embodiment of the present application. In particular, the road detection model may include a road surface target detection model and a road surface target tracking model. The embodiment of the application can perform pavement target detection on the collected image information through the pavement target detection model, for example, the pavement disease and pavement signboard detection can be performed on the collected image information, so that when the pavement disease is detected, the pavement disease type corresponding to the pavement disease information and the pavement disease information is determined, the pavement disease detection result is obtained, and when the pavement signboard is detected, the pavement signboard type corresponding to the pavement signboard information and the pavement signboard information is determined, and the pavement signboard detection result is obtained.

For example, referring to fig. 4, the road surface disease and road signboard detection model may be a road surface target detection model, which is not limited by this example; the tracking model may be a road surface target tracking model, which is not limited in this example. Specifically, the collected image information may be input into a road surface defect and road signboard detection model, the road surface defect and the road signboard in the collected image information may be identified by the road surface defect and road signboard detection model, a road surface defect detection result and a road signboard detection result may be obtained, subsequently, a road surface target in the collected image information may be tracked and identified by the tracking model, for example, the road surface defect information in the road surface defect detection result may be tracked and identified by the tracking model, so as to obtain the number of defects of the road surface defect type, the road signboard information in the road signboard detection result may be tracked and identified by the tracking model, so as to obtain the number of signboards of the road signboard type, and the type and the corresponding number of each target appearing in a single frame image may be output, for example, the number of defects of the road surface defect type and the road surface defect type appearing in the collected image information may be output, and the number of the road signboard type and the road signboard type appearing in the collected image information may be output.

Optionally, the detecting the road surface target of the collected image information by the road surface target detecting model to obtain a road surface disease detection result and a road surface signboard detection result may specifically include the following sub-steps:

and a substep 2201, inputting the collected image information into the road surface target detection model.

Specifically, the embodiment of the application can input the collected image information into the pavement target detection model, so that the pavement target detection model can detect and identify the pavement target in the collected image information, the pavement disease and the pavement signboard of the target road can be rapidly identified, and the detection efficiency of the pavement hazard and the pavement signboard can be improved.

And a substep 2202, detecting and identifying the road surface target in the collected image information through the road surface target detection model to obtain a road surface target identification result.

Specifically, the embodiment of the application may perform detection and recognition on the road surface target in the collected image information through a road surface target detection model to obtain a road surface target recognition result, where the road surface target recognition result may include road surface disease information, a road surface disease type corresponding to the road surface disease information, a disease confidence coefficient corresponding to the road surface disease information, road surface signboard information, a road surface signboard type corresponding to the road surface signboard information, and a signboard confidence coefficient corresponding to the road surface signboard information.

In a specific implementation, the disease confidence and the signboard confidence may be both a numerical value, such as a probability numerical value, which is not limited in the embodiment of the present application. The road surface target detection model can detect and identify the road surface target in the collected image information to obtain the confidence coefficient corresponding to the road surface target, for example, when the road surface target is identified as a road surface disease, the confidence coefficient that the road surface target is the road surface disease can be predicted to obtain the disease confidence coefficient; similarly, when the road surface target is recognized as the road surface signboard, the confidence coefficient of the road surface target as the road surface signboard can be predicted, and the confidence coefficient of the signboard is obtained.

As an example, in a case that the road surface defect corresponding to the road surface target is a pit and the confidence of the defect corresponding to the pit may be 0.96, it may be determined that the probability that the road surface target is a pit is ninety-six percent, which is not limited by this example.

And a substep 2203, if the confidence coefficient of the road surface target in the recognition result of the road surface target meets the condition of preset confidence coefficient of the road surface disease, determining the road surface target information in the recognition result of the road surface target as the information of the road surface disease, and determining the type of the road surface disease corresponding to the information of the road surface disease.

In the substep 2204, if the confidence level of the signboard in the road surface target recognition result meets the preset signboard confidence level condition, determining the road surface target information in the road surface target recognition result as the road surface signboard information, and determining the type of the road surface disease corresponding to the road surface signboard information.

Specifically, after the road surface target recognition result is determined, the confidence coefficient may be extracted from the road surface target recognition result, and whether the confidence coefficient meets the preset confidence coefficient condition may be determined. Specifically, when the pavement target is a pavement disease, a disease confidence coefficient may be extracted from the pavement target recognition result, and then it may be determined whether the extracted disease confidence coefficient meets a preset disease confidence coefficient condition, and if it is determined that the disease confidence coefficient meets the preset confidence coefficient condition, the pavement target information in the pavement target recognition result may be determined as pavement disease information, and a pavement disease type corresponding to the pavement disease information may be determined. When the road surface target is a road surface signboard, the signboard confidence degree can be extracted from the road surface target recognition result. And then judging whether the extracted confidence coefficient of the signboard meets the preset signboard confidence coefficient condition or not, if the confidence coefficient of the signboard meets the preset signboard confidence coefficient condition, determining the pavement target information in the pavement target recognition result as the pavement signboard information, and determining the type of the pavement signboard corresponding to the pavement signboard information.

In a specific implementation, if the confidence level in the road surface target recognition result does not meet a preset confidence level condition, for example, the confidence level of a disease in the road surface target recognition result does not meet the preset disease confidence level condition, or the confidence level of a signboard in the road surface target recognition result does not meet the preset signboard confidence level condition, a prompt that the confidence level of the road surface target recognition result is low can be output, so that a user can be reminded that the confidence level of the current road surface target recognition result is low, and manual detection is recommended.

As an example, a confidence level judgment threshold may be preset, for example, a disease confidence level judgment threshold and a signboard confidence level judgment threshold may be preset, and the disease confidence level judgment threshold and the signboard confidence level judgment threshold may be the same or different, which is not limited in this example. When the road surface target is a road surface disease, a disease confidence coefficient can be extracted from the road surface target recognition result, then the disease confidence coefficient can be compared with a preset disease confidence coefficient judgment threshold, the condition that the disease confidence coefficient accords with the preset disease confidence coefficient condition can be determined when the disease confidence coefficient is not less than the confidence coefficient judgment threshold, and the condition that the disease confidence coefficient does not accord with the preset disease confidence coefficient condition can be determined when the disease confidence coefficient is less than the preset disease confidence coefficient judgment threshold. Similarly, when the road surface target is a road surface signboard, the signboard confidence coefficient can be extracted from the road surface target recognition result, then the signboard confidence coefficient can be compared with the preset signboard confidence coefficient judgment threshold, and when the signboard confidence coefficient is not less than the confidence coefficient judgment threshold, the signboard confidence coefficient is determined to be in accordance with the preset signboard confidence coefficient condition, and when the signboard confidence coefficient is less than the preset signboard confidence coefficient judgment threshold, the signboard confidence coefficient is determined to be not in accordance with the preset signboard confidence coefficient condition.

And 230, tracking and identifying the pavement disease information and the pavement signboard information through the pavement target tracking model based on the collected image information to obtain the disease number of the pavement disease type and the signboard number of the pavement signboard type.

Specifically, the embodiment of the application can track and identify the road surface disease information and the road surface signboard information through the road surface target tracking model based on the collected image information to obtain the disease number of the road surface disease type and the signboard number of the road surface signboard type. Specifically, for each pavement target contained in the collected image information, each pavement target can be numbered through a pavement target tracking model, for example, pavement diseases and pavement signboards in the collected image information can be numbered, then, all pavement diseases and pavement signboards in the collected image information of the target road can be tracked through the pavement target tracking model, pavement targets with the same number appearing in the target road can be deduplicated, for example, pavement diseases and pavement signboards with the same number can be deduplicated, so that all pavement diseases and pavement signboards in the target road can be determined, the number of diseases of pavement disease types and the number of signboards of pavement signboards types can be obtained, the pavement diseases and the pavement signboards in the collected image information can be identified and detected through a pavement target detection model, the identified pavement diseases and the identified pavement signboards can be tracked through the pavement target tracking model, the number of pavement diseases and the number of pavement signboards of the target road can be determined, the number of pavement diseases and the number of pavement signboards can be counted while the pavement diseases and the number of the pavement target signboards are automatically detected, and the problem of the existing road diseases and the problem of the artificial road can be solved.

For example, referring to FIG. 5, after model training is completed, such as after model training of the road surface target detection model and the road surface target tracking model is completed, the trained model models may be deployed into the edge computing device. When the road surface of the target road is detected, the road surface image of the target road captured in real time can be input into the edge computing equipment as the collected image information through the positioning module arranged in the vehicle, so that the edge computing equipment can carry out model reasoning on the collected image information through the built-in model, if the collected image information can be detected and identified through the road surface target detection model, the road surface target is obtained, and the geographical position information of the collected image can be determined through the positioning data to be used as the geographical position information of the road surface target. And then, analyzing and predicting the road surface target to obtain the class corresponding to the road surface target and the position of the road surface target in the collected image information, and taking the position of the road surface target in the collected image information as marking frame information. After the road surface target is determined, the road surface target can be tracked and identified through a road surface target tracking model, if a unique number can be set for each road surface target, then the type and the number of each road surface target appearing in the collected image information can be counted, and a detection identification image is output based on the type of the road surface target, the number of the road surface targets of the road surface target type and the collected image information.

As an example, referring to fig. 6, after the road surface target detection result is obtained by performing road surface target detection and recognition on the collected image information through the road surface target detection model, the detected road surface target (i.e., a disaster) can be identified in the collected image information, and detailed information of the road surface target can be displayed, such as the type of the road surface target (i.e., the type to which the disaster belongs), the coordinates of the road surface target in the image (i.e., the coordinates of the disaster in the image), the confidence level of the road surface target (i.e., the confidence level of the disaster), and the size of the road surface target in the collected image information (i.e., the size information of the disaster), can be displayed, then, each road surface target in the collected image information can be numbered through the road surface target tracking model, the number of the road surface targets in the collected image information can be counted (i.e., the number of the disasters in the image), and the number corresponding to each road surface target (i.e., the number of the disaster information) can be displayed in the collected image information.

Further, according to the embodiment of the application, after all the collected image information of the target road is identified and detected by the road surface target detection model and all the road surface targets of the collected image information are obtained, the road surface targets with the same number can be deduplicated by the road surface target tracking model, so that the types of all the road surface targets of the target road and the number of the road surface targets can be obtained.

And 240, determining the geographical position information and the road flatness information corresponding to the road surface disease type or the road surface signboard type based on the acquired positioning data and acceleration data.

As an example, referring to fig. 3, the vehicle may be equipped with an acceleration sensor and a GPS positioning module, wherein the acceleration sensor may acquire acceleration data of the vehicle in real time while the vehicle is running, so that the road flatness information may be determined according to the acceleration data, which is not limited in this example; the GPS positioning module may be configured to perform GPS positioning to obtain positioning data when the vehicle is running, which is not limited in this example; the acceleration sensor and the GPS positioning module can be connected with the edge box, so that data interaction can be carried out with the edge box. Specifically, the edge box can acquire acceleration data fed back by the acceleration sensor in real time, and when detecting that the collected image information contains a road target, the edge box acquires positioning data through the GPS positioning module, and then determines geographical position information and road flatness information corresponding to a road surface disease type or a road surface signboard type based on the acquired positioning data and the acceleration data.

And 250, determining a road surface detection result of the target road according to the geographical position information, the road surface flatness information and the collected image information.

In the specific implementation, the pavement detection result of the target road can be determined according to the geographical position information, the pavement evenness information and the collected image information, and the pavement detection result of the target road can be sent to a user, so that the detection efficiency of the pavement diseases and the pavement signboard is improved, and the problem caused by manual pavement condition detection in the prior art is solved.

In an optional implementation manner, the embodiment of the present application may update the road surface target detection model and the road surface target tracking model at regular time. Specifically, the model updating can be performed on the road surface target detection model and the road surface target tracking model based on the latest acquired collected image information or sample data set by executing the updating program at regular time, and the model updating can be persisted to the edge computing equipment where the model is located, so that the accuracy, generalization capability and timeliness of the model are ensured, and the accuracy of model prediction is ensured.

To sum up, the embodiment of the application acquires the collected image information of the target road, performs pavement target detection on the collected image information through the pavement target detection model to obtain a pavement disease detection result and a pavement signboard detection result, then performs tracking and identification on the pavement disease information and the pavement signboard information through the pavement target tracking model based on the collected image information to obtain the disease number of the pavement disease type and the signboard number of the pavement signboard type, and determines the pavement disease type or the geographic position information and the pavement evenness information corresponding to the pavement signboard type based on the acquired positioning data and acceleration data, and further determines the pavement detection result of the target road according to the geographic position information, the pavement evenness information and the collected image information, thereby improving the detection efficiency of the pavement disease and the pavement signboard and solving the problem caused by manual pavement condition detection in the prior art.

Referring to fig. 7, a schematic flowchart illustrating a method for processing road detection information according to an alternative embodiment of the present application is shown. Specifically, the method for processing road detection information provided in the embodiment of the present application may specifically include, in a model training stage, the following steps:

step 710, extracting image data to be trained and verification data corresponding to the image data to be trained from the sample data set.

Specifically, the image data to be trained and the verification data corresponding to the image data to be trained can be extracted from the sample data set. Specifically, a sample data set may be obtained in advance, the sample data set is divided into a training set and a verification set corresponding to the training set, then the training set may be used as the image data to be trained, the verification set corresponding to the training set may be used as the verification data corresponding to the image data to be trained, and then model training may be performed according to the image data to be trained and the verification data corresponding to the image data to be trained, so as to obtain the road detection model.

As an example, referring to fig. 8, in a model training phase, video data of a road may be acquired by a vehicle-mounted camera, and OpenCv may perform frame extraction on the video data according to a preset frame extraction manner, where the frame extraction manner may be to extract one video frame from every five video frames, or certainly may be to reserve all the video frames, so as to ensure accuracy of a model prediction result. The extracted video frames may then be data washed and annotated. Specifically, the extracted video frames may be subjected to data cleaning in a manual data cleaning manner to obtain cleaned video frames, for example, the video frames with higher similarity in the extracted video frames may be deleted, or the video frames with the content of the video frames in the extracted video frames as the video frames of the traffic lights waiting for the vehicle may be deleted, which is not limited in this example. The cleaned video frame may be labeled in a manual labeling manner to obtain a labeled video frame, for example, a road pavement crack and a road signboard included in the cleaned video frame may be labeled, which is not limited in this example. The marked data can be used as a sample data set, the sample data set can be divided into a training set and a verification set, the training set is used as image data to be trained, and the verification set is used as verification data corresponding to the image data to be trained.

In an optional implementation, before extracting, from a sample data set, image data to be trained and verification data corresponding to the image data to be trained, the embodiment of the present application may further include: determining the category name of the road surface disease and the category name of the road surface signboard corresponding to the collected image information; and generating the sample data set based on the acquired image information, the category name of the road surface disease and the category name of the road surface signboard. Specifically, a category name of a road surface disease corresponding to the acquired image information and a category name of a road surface signboard may be determined, and then, based on the category name of the road surface disease corresponding to the acquired image information and the category name of the road surface signboard, a sample labeling may be performed on the acquired image, for example, a road surface disease type corresponding to each road surface disease in the acquired image information and a road surface signboard type corresponding to the road surface signboard may be labeled, so as to generate a sample data set.

And 720, performing data enhancement processing according to the image data to be trained to obtain enhanced image data corresponding to the image data to be trained.

Specifically, after determining the image data to be trained, the embodiment of the present application may perform enhancement processing on the image data to be trained to obtain enhanced image data to be trained, and may use the enhanced image data as the enhanced image data corresponding to the image data to be trained, for example, the image data to be trained may be subjected to blur enhancement, gaussian noise enhancement, salt and pepper noise enhancement, contrast enhancement, and the like, which is not limited in this application.

And 730, performing model training according to the enhanced image data and the verification data to obtain the road detection model.

Specifically, model training may be performed according to the enhanced image data and the verification data, and a teacher may end the model training when the model training reaches an end condition, to obtain the road detection model.

As an example, referring to fig. 8, after performing enhancement processing according to image data to be trained to obtain enhanced image data, deep neural network training may be performed according to the enhanced image data, for example, an enhanced image may be input into the YOLO V3 neural network for model training, and then, the performance of the neural network may be tested through verification data verification, so that when the model training reaches a termination condition, the model training is terminated, and a road detection model and a training weight file corresponding to the road detection model are obtained.

In a specific implementation, the road detection model may include a road target detection model and a road target tracking model, which is not limited in this application.

In an optional embodiment, in the case that verification data includes a road surface defect verification type and/or a road surface signboard verification type, performing model training according to the enhanced image data and the verification data to obtain the road detection model specifically may include: performing model training according to the enhanced image data, the pavement disease verification type and the pavement signboard verification type to obtain a pavement target detection model; determining the type of the pavement target diseases according to the pavement disease detection result output by the pavement target detection model, and determining the type of the pavement target signboard according to the pavement signboard detection result output by the pavement target detection model; determining the verification quantity of the pavement diseases corresponding to the pavement target disease types and the verification quantity of the pavement signs corresponding to the pavement target sign types according to the enhanced image data; and performing model training according to the verification quantity of the pavement diseases, the verification quantity of the pavement signboard and the enhanced image data to obtain a pavement target tracking model.

After the road detection model is trained, the collected image information can be subjected to road surface detection and recognition through the road detection model, a road surface prediction result corresponding to the target road is obtained, the road surface prediction result can contain a road surface disease type, a road surface signboard type, the number of diseases corresponding to the road surface disease type and the number of signboards corresponding to the road surface signboard type, geographic position information and road surface evenness information corresponding to the road surface disease type or the road surface signboard type can be determined based on the obtained positioning data and acceleration data, and then the road surface detection result of the target road can be determined according to the geographic position information, the road surface evenness information and the collected image information, so that the detection efficiency of the road surface disease and the road surface signboard is improved, and the problem caused by the artificial detection of the road surface condition in the prior art is solved.

It should be noted that for simplicity of description, the method embodiments are described as a series of acts, but those skilled in the art should understand that the embodiments are not limited by the described order of acts, as some steps can be performed in other orders or simultaneously according to the embodiments.

As shown in fig. 9, an embodiment of the present application further provides a device 900 for processing road detection information, including:

a collected image information obtaining module 910, configured to obtain collected image information of a target road;

a road surface prediction result determining module 920, configured to perform road surface detection and recognition processing through a pre-trained road detection model according to the acquired image information to obtain a road surface prediction result corresponding to the target road, where the road surface prediction result includes a road surface disease type, a road surface signboard type, a disease number corresponding to the road surface disease type, and a signboard number corresponding to the road surface signboard type;

an information obtaining module 930, configured to obtain geographic position information and road flatness information corresponding to the road surface disease type or the road surface signboard type based on the obtained positioning data and acceleration data;

and a road surface detection result determining module 940, configured to determine a road surface detection result of the target road according to the geographic position information, the road surface flatness information, and the collected image information.

Optionally, the acquired image information obtaining module includes:

the road video determining submodule is used for recording a target road through the camera to obtain road video information;

and the image frame extraction submodule is used for extracting image frames from the road video information and determining the extracted image frames as the acquired image information.

Optionally, the road detection model includes a road target detection model and a road target tracking model, and the road prediction result determining module includes:

the pavement target detection result determining submodule is used for carrying out pavement target detection on the collected image information through the pavement target detection model to obtain a pavement disease detection result and a pavement signboard detection result, wherein the pavement disease identification result comprises pavement disease information and a pavement disease type corresponding to the pavement disease information, and the pavement signboard detection result comprises the pavement signboard information and a pavement damage type corresponding to the pavement signboard information;

and the quantity determining submodule is used for tracking and identifying the pavement disease information and the pavement signboard information through the pavement target tracking model based on the acquired image information to obtain the disease quantity of the pavement disease type and the signboard quantity of the pavement signboard type.

Optionally, the sub-module for determining the detection result of the road surface target includes:

the input unit is used for inputting the collected image information into the road surface target detection model;

the road surface target identification result determining unit is used for detecting and identifying the road surface target in the collected image information through the road surface target detection model to obtain a road surface target identification result;

the road surface disease determining unit is used for determining the road surface target information in the road surface target recognition result as the road surface disease information and determining the road surface disease type corresponding to the road surface disease information when the disease confidence coefficient in the road surface target recognition result meets the preset disease confidence coefficient condition;

and the pavement signboard determining unit is used for determining the pavement target information in the pavement target recognition result as the pavement signboard information and determining the pavement disease type corresponding to the pavement signboard information when the signboard confidence coefficient in the pavement target recognition result meets the preset signboard confidence coefficient condition.

Optionally, the processing apparatus of the road detection information further includes:

the extraction module is used for extracting the image data to be trained and the verification data corresponding to the image data to be trained from the sample data set;

the enhanced image data determining module is used for performing data enhancement processing according to the image data to be trained to obtain enhanced image data corresponding to the image data to be trained;

and the model training module is used for carrying out model training according to the enhanced image data and the verification data to obtain the road detection model.

Optionally, the verification data includes a pavement damage verification type and/or a pavement signboard verification type, and the model training module includes:

the pavement target detection model training submodule is used for carrying out model training according to the enhanced image data, the pavement disease verification type and the pavement signboard verification type to obtain a pavement target detection model;

the pavement target type determining submodule is used for determining the type of the pavement target diseases according to the pavement disease detection result output by the pavement target detection model and determining the type of the pavement target signboard according to the pavement signboard detection result output by the pavement target detection model;

the verification quantity determining submodule is used for determining the pavement disease verification quantity corresponding to the pavement target disease type and the pavement signboard verification quantity corresponding to the pavement target signboard type according to the enhanced image data;

and the pavement target tracking model training submodule is used for carrying out model training according to the pavement disease verification quantity, the pavement signboard verification quantity and the enhanced image data to obtain a pavement target tracking model.

the category name determining module is used for determining the category name of the road surface disease and the category name of the road surface signboard corresponding to the collected image information;

and the sample data set generating module is used for generating the sample data set based on the acquired image information and the pavement damage type data.

The road detection information processing device provided by the embodiment of the present application can execute the road detection information processing method provided by any embodiment of the present application, and has the corresponding functions and advantages of executing the road detection information processing method.

In a specific implementation, the processing device of the road detection information may be integrated into an apparatus, so that the apparatus may perform road detection and identification through a road detection model according to acquired image information of a target road to obtain a road prediction result, further determine geographic position information and road flatness information corresponding to a road surface disease type or a road surface signboard type based on the acquired positioning data and acceleration data, and determine a road surface detection result of the target road according to the geographic position information, the road flatness information and the acquired image information to serve as a processing device of the road detection information to realize road surface detection on the target road. The processing device of the road detection information may be composed of two or more physical entities, or may be composed of one physical entity, for example, the processing device of the road detection information may be a Personal Computer (PC), a Computer, a server, etc., which is not limited in this embodiment of the present application.

As shown in fig. 10, an embodiment of the present application provides a road detection information processing device, which includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, where the processor 111, the communication interface 112, and the memory 113 complete communication with each other through the communication bus 114; a memory 113 for storing a computer program; the processor 111 is configured to implement the steps of the method for processing the road detection information according to any one of the above-mentioned embodiments when executing the program stored in the memory 113. For example, the steps of the method for processing road detection information may include the steps of: acquiring collected image information of a target road; according to the collected image information, carrying out pavement detection and identification processing through a pre-trained road detection model to obtain a pavement prediction result corresponding to the target road, wherein the pavement prediction result comprises a pavement disease type, a pavement signboard type, the number of diseases corresponding to the pavement disease type and the number of signboards corresponding to the pavement signboard type; determining geographical position information and road surface evenness information corresponding to the road surface disease type or the road surface signboard type based on the obtained positioning data and acceleration data; and determining a road surface detection result of the target road according to the geographic position information, the road surface flatness information and the collected image information.

The present application also provides a computer readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the method for processing road detection information provided in any one of the method embodiments described above.

It is noted that, in this document, relational terms such as "first" and "second," and the like, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

The above description is merely exemplary of the present application and is presented to enable those skilled in the art to understand and practice the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing road detection information, comprising:

acquiring collected image information of a target road;

2. The method of claim 1, wherein the obtaining of the captured image information of the target road comprises:

recording a target road through a camera to obtain road video information;

3. The method according to claim 2, wherein the road detection model includes a road target detection model and a road target tracking model, and the obtaining of the road surface prediction result corresponding to the target road by performing road detection recognition processing through a pre-trained road detection model according to the collected image information comprises:

carrying out pavement target detection on the collected image information through the pavement target detection model to obtain a pavement disease detection result and a pavement signboard detection result, wherein the pavement disease identification result comprises pavement disease information and a pavement disease type corresponding to the pavement disease information, and the pavement signboard detection result comprises the pavement signboard information and a pavement damage type corresponding to the pavement signboard information;

4. The method of claim 3, wherein the performing, by the road surface target detection model, the road surface target detection on the collected image information to obtain a road surface disease detection result and a road surface signboard detection result comprises:

5. The method according to any one of claims 1 to 4, wherein before the road surface detection and identification processing by the pre-trained road detection model, the method further comprises:

6. The method according to claim 5, wherein the verification data includes a pavement damage verification type and/or a pavement marker verification type, and the performing model training according to the enhanced image data and the verification data to obtain the road detection model comprises:

and performing model training according to the pavement disease verification quantity, the pavement signboard verification quantity and the enhanced image data to obtain a pavement target tracking model.

7. The method according to claim 5, wherein before extracting, from the sample data set, image data to be trained and validation data corresponding to the image data to be trained, the method further comprises:

8. A processing apparatus of road detection information, comprising:

9. The processing equipment of the road detection information is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for implementing the steps of the method for processing road detection information according to any one of claims 1 to 7 when executing a program stored in the memory.

10. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of a method of processing road detection information according to any one of claims 1 to 7.