CN112907532B - Improved truck door falling detection method based on fast RCNN - Google Patents
Improved truck door falling detection method based on fast RCNN Download PDFInfo
- Publication number
- CN112907532B CN112907532B CN202110183109.3A CN202110183109A CN112907532B CN 112907532 B CN112907532 B CN 112907532B CN 202110183109 A CN202110183109 A CN 202110183109A CN 112907532 B CN112907532 B CN 112907532B
- Authority
- CN
- China
- Prior art keywords
- door
- candidate area
- frame
- truck
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30268—Vehicle interior
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
A freight car door falling detection method based on fast RCNN improvement relates to the field of railway freight car fault detection. The method aims to solve the problem that the detection precision is low or the overfitting risk is large due to the fact that the image is normalized by the RoiPooling when the fast RCNN detects the vehicle door falling fault at present. The specific process of the invention is as follows: inputting the images of the truck door of the truck to be detected into a trained fast RCNN network to obtain a detection result; the method comprises the following steps of utilizing a candidate region maximum filling layer (RoiMaxFill layer) to carry out maximum filling operation in a Faster RCNN network to normalize a candidate frame, wherein the specific process comprises the following steps: determining the equal division number of the large candidate region frame according to the height and the width of the small candidate region frame in all the image candidate region frames; and dividing the large candidate area frame according to the equal dividing number, and performing maximum filling operation (MaxFill) on other candidate area frames by taking the maximum divided candidate area frame as a target size to perform peripheral symmetric zero filling to obtain the candidate area frames with consistent sizes.
Description
Technical Field
The invention relates to the field of fault detection of rail wagons, in particular to a wagon door falling detection method based on improvement of fast RCNN.
Background
In recent years, driving accidents caused by falling-off of the vehicle door are increased remarkably. The falling of the car door not only can easily cause the derailment of the car, the separation of the train, the damage of the car, the damage of the turnout signal and other driving behaviors, but also can cause the casualty of passengers.
At present, a door drop fault is mainly detected by using a Faster RCNN network, images with different sizes are normalized by a RoiPooling operation before the fast RCNN network is connected with a full connection layer in the door drop detection process, but when images are normalized by the RoiPooling operation, a large-area discarded pixel value is too much, the door drop fault detection precision is low, and when a large-area image is used as a discarding condition, a small-area image has a high risk of overfitting.
Disclosure of Invention
The invention aims to solve the problem that the detection precision is low or the overfitting risk is high due to the fact that the image is normalized by the RoiPoling when the door falling fault is detected by the fast RCNN, and provides an improved truck door falling fault detection method based on the fast RCNN.
The method for detecting the falling fault of the truck door based on the improvement of the Faster RCNN comprises the following specific processes: inputting a truck door image to be detected into a trained fast RCNN network to obtain a detection result of whether the truck door falls off or not;
the method comprises the following steps of normalizing a candidate region frame by using a candidate region maximum filling operation in a Faster RCNN network, wherein the specific process comprises the following steps:
step1, regarding the candidate area frame which is larger than the first threshold value in all the image candidate area frames as a large candidate area frame, and regarding the candidate area frame which is smaller than the second threshold value as a small candidate area frame;
determining the equal fraction number of the large candidate area frame according to the height and the width of the small candidate area frame:
Nd＝Wd/(Ws/Nws)
Nhd＝Hd/(Hs/Nhs)
where Wd is the width of the large candidate region frame, Ws is the width of the small candidate region frame, Nws is the number of equally divided widths of the small candidate region frame determined in advance from experience, and Nd is the number of equally divided widths of the large candidate region frame. Hd is the height of the large candidate area frame, Hs is the height of the small candidate area frame, Nhs is the empirically predetermined number of equal divisions of the small candidate area frame, predetermined based on Hs, Nhd is the height equal division of the large candidate area frame;
and Step2, dividing the large candidate area frame according to the width equal dividing number of the large candidate area frame and the height equal dividing number of the large candidate area frame acquired in Step1, and performing peripheral symmetric zero filling on other candidate area frames by taking the maximum divided candidate area frame as a target size to obtain the candidate area frames with the same size.
Optionally, the trained fast RCNN network is obtained by the following method:
acquiring an image of a truck door part;
step two, dividing the acquired part images into a training set and a testing set;
step three, marking the training set;
and step four, training the fast RCNN network by using the marked training set to obtain the trained fast RCNN network.
Optionally, the door part in step one comprises: door bolt, door hasp, door fold.
Optionally, the first step is to acquire an image of the truck door component, and the specific process is as follows:
step one, linear array images are obtained;
step two, obtaining an image of the truck door part:
and according to the principle that the positions of the parts of the same vehicle type are approximately the same, intercepting the region where the parts are located from the linear array image to obtain a picture of the door part of the freight vehicle.
Optionally, the specific process of acquiring linear array images in the steps one by one is as follows:
the method comprises the steps of carrying a camera or a video camera by utilizing a fixed device, shooting a moving truck, shooting a whole-train image of the upper part of the truck, and scanning only one line of a train each time to generate a two-dimensional image.
Optionally, in the third step, a labelimg tool is used to mark the training set, and an xml file corresponding to each picture in the training is obtained after marking, where the xml file includes the marking information of each picture.
Optionally, the type of the training set labeled in step three includes: normal door bolt, abnormal door bolt, normal door hasp, abnormal door hasp, normal door hinge and abnormal door hinge.
Optionally, the first threshold in step1 is 60 pixels; the second threshold is 30 pixels;
optionally, the improved truck door drop-off detection system based on the Faster RCNN is used for realizing the improved truck door drop-off detection method based on the Faster RCNN.
The invention has the beneficial effects that:
according to the invention, the manual detection is replaced by an automatic image identification mode, so that the detection efficiency and accuracy of the falling fault of the truck door are improved. According to the method, the RoiMaxFill layer (the maximum filling layer of the candidate area) is used for replacing the RoiPooling layer, the equal-fraction number is determined according to the height and the width of the smaller image in the detected image for normalization, the overfitting risk is reduced, the image loss is reduced, and the image detection precision is improved.
Drawings
FIG. 1 is a flowchart illustrating normalization of candidate frames using a candidate region maximum fill layer in a fast RCNN network;
FIG. 2 is a line array picture taken by a camera;
FIG. 3 is a drawing of a part to be inspected;
FIG. 4 is a picture of part position;
Detailed Description
It should be noted that, in the case of conflict, the features included in the embodiments or the embodiments disclosed in the present application may be combined with each other.
The first embodiment is as follows: the specific process of the truck door falling detection method based on the fast RCNN improvement in the embodiment is as follows: inputting the images of the truck doors to be detected into a trained fast RCNN to obtain a detection result of whether the truck doors are all fallen off or not;
the method for normalizing the candidate frames by performing the maximum filling operation by using the maximum filling layer (RoiMaxFill layer) of the candidate areas in the Faster RCNN network comprises the following steps:
step1, regarding the candidate area frame which is larger than the first threshold value in all the image candidate area frames as a large candidate area frame, and regarding the candidate area frame which is smaller than the second threshold value as a small candidate area frame;
determining the equal fraction number of the large candidate area frame according to the height and the width of the small candidate area frame:
Nd＝Wd/(Ws/Nws)
Nhd＝Hd/(Hs/Nhs)
where Wd is the width of the large candidate region frame, Ws is the width of the small candidate region frame, Nws is the empirically predetermined number of equal divisions of the width of the small candidate region frame, the number of equal divisions being predetermined based on Ws, and Nd is the number of equal divisions of the width of the large candidate region frame. Hd is the height of the large candidate area frame, Hs is the height of the small candidate area frame, Nhs is the empirically predetermined number of equally divided small candidate area frames, predetermined on the basis of Hs, Nhd is the height equally divided number of the large candidate area frame;
wherein the first threshold is 60 pixels;
wherein the second threshold is 30 pixels;
aiming at the specific problem of vehicle door falling, only a large candidate area and a small candidate area are meaningful for identification and detection, and if the pixels of the candidate areas are 30-60, the interior of a program is automatically filtered and discarded.
In the door drop detection, the sizes of the large candidate region frames are all consistent in each image, and the sizes of the small candidate region frames are all consistent.
Wherein the number of equally divided heights and the number of equally divided widths of the small candidate region frames are empirically determined based on the small candidate region frames in the current image.
Wherein rounding is performed when an equal fraction of the large candidate region box is obtained.
And Step2, dividing the large candidate area frame according to the width equal dividing number of the large candidate area frame and the height equal dividing number of the large candidate area frame acquired in Step1, and performing peripheral symmetric zero filling on other candidate area frames by taking the maximum divided candidate area frame as a target size to obtain candidate area frames with consistent sizes.
The second embodiment is as follows: the trained fast RCNN network is obtained by the following method:
acquiring an image of a truck door part;
wherein, door parts include: a door bolt, a door hasp and a door hinge;
step two, dividing the acquired part images into a training set and a testing set;
step three, marking the training set;
step four, training a Faster RCNN network by using the marked training set to obtain a trained Faster RCNN network;
other steps are the same as those in the first embodiment.
The third concrete implementation mode: acquiring an image of a truck door part in the first step, wherein the method comprises the following steps:
step one, obtaining linear array images:
a camera or a video camera is carried by utilizing the fixing equipment to shoot the truck moving at high speed, the whole truck image at the upper part of the truck is shot, only one line of the train is scanned each time, seamless splicing is realized, and a two-dimensional image (as shown in figure 1) with large visual field and high precision is generated.
Step two, obtaining an image of the truck door part:
according to the principle that the positions of parts of the same vehicle type are approximately the same, intercepting the area where the parts are located from the linear array image to obtain a truck door part picture (as shown in figures 2 and 3);
the other steps are the same as those in the first to second embodiments.
The fourth concrete implementation mode: and in the third step, a labelimg tool is adopted to mark the training set, and an xml file corresponding to each picture in the training is obtained after marking, wherein the xml file comprises the marking information of each picture.
The other steps are the same as those in the first to third embodiments.
The fifth concrete implementation mode: the types of marking the training set in the third step include: normal door bolt, abnormal door bolt, normal door hasp, abnormal door hasp, normal door hinge and abnormal door hinge.
The other steps are the same as those in the first to fourth embodiments.
The sixth specific implementation mode: the improved wagon door falling detection system based on the Faster RCNN is used for realizing the improved wagon door falling detection method based on the Faster RCNN.
The other steps are the same as those in the first to fifth embodiments.
Example (b):
for the door drop problem, the same data set was used and the test was performed on original Faster Rcnn and modified version of Faster Rcnn, respectively. The improved version of Faster Rcnn improved the detection accuracy by 2% over the original fast Rcnn model, as shown in fig. 4.
Claims (9)
1. A truck door falling detection method based on fast RCNN improvement comprises the following specific processes: inputting the image of the door of the truck to be detected into a trained fast RCNN network to obtain a detection result of whether the door falls off; the method is characterized in that:
the method comprises the following steps of normalizing a candidate region frame by using a candidate region maximum filling operation in the Faster RCNN network, wherein the specific process comprises the following steps:
step1, taking the candidate area frame which is larger than the first threshold value in all the image candidate area frames as a large candidate area frame, and taking the candidate area frame which is smaller than the second threshold value as a small candidate area frame;
determining the equal fraction number of the large candidate area frame according to the height and the width of the small candidate area frame:
Nd = Wd / (Ws / Nws)
Nhd = Hd / (Hs / Nhs)
where Wd is the width of the large candidate region frame, Ws is the width of the small candidate region frame, Nws is the number of equally divided widths of the small candidate region frame determined in advance based on experience, Nd is the number of equally divided widths of the large candidate region frame, Hd is the height of the large candidate region frame, Hs is the height of the small candidate region frame, Nhs is the number of equally divided small candidate region frames determined in advance based on experience, and Nhd is the number of equally divided heights of the large candidate region frame;
dividing a large candidate area frame and a small candidate area frame according to a preset pixel threshold;
and Step2, dividing the large candidate area frame according to the width equal dividing number and the height equal dividing number of the large candidate area frame acquired in Step1, and performing peripheral symmetric zero filling on other candidate area frames by taking the maximum divided candidate area frame as a target size to obtain the candidate area frames with the same size.
2. The improved truck door drop detection method based on Faster RCNN according to claim 1, wherein: the trained Faster RCNN network is obtained by the following method:
acquiring an image of a truck door part;
step two, dividing the acquired part images into a training set and a testing set;
step three, marking the training set;
and step four, training the fast RCNN network by using the marked training set to obtain the trained fast RCNN network.
3. The improved truck door drop detection method based on Faster RCNN according to claim 2, wherein: the door part in the first step comprises: door bolt, door hasp, door fold.
4. The improved truck door drop detection method based on Faster RCNN according to claim 3, wherein: acquiring the truck door part image in the first step, wherein the specific process is as follows:
step one, linear array images are obtained;
step two, obtaining an image of the truck door part:
and according to the principle that the positions of the parts of the same vehicle type are approximately the same, intercepting the area where the parts are located from the linear array image to obtain the picture of the parts of the door of the truck.
5. The improved truck door drop detection method based on Faster RCNN according to claim 4, wherein: the specific process of acquiring the linear array images in the steps one by one is as follows:
the method comprises the steps of carrying a camera or a video camera by utilizing a fixed device, shooting a moving truck, shooting a whole-train image of the upper part of the truck, and scanning only one line of a train each time to generate a two-dimensional image.
6. The improved truck door drop detection method based on Faster RCNN according to claim 5, wherein: and in the third step, a labelimg tool is adopted to mark the training set, and an xml file corresponding to each picture in the training is obtained after marking, wherein the xml file comprises the marking information of each picture.
7. The improved truck door drop detection method based on Faster RCNN according to claim 6, wherein: the type of marking the training set in the third step includes: normal door bolt, abnormal door bolt, normal door hasp, abnormal door hasp, normal door hinge and abnormal door hinge.
8. The improved freight car door drop-off detection method based on Faster RCNN according to claim 7, wherein: the first threshold in step1 is 60 pixels; the second threshold is 30 pixels.
9. Freight train door detection system that drops based on fast RCNN improves, its characterized in that: the method for implementing improved truck door drop-out detection based on Faster RCNN according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110183109.3A CN112907532B (en) | 2021-02-10 | 2021-02-10 | Improved truck door falling detection method based on fast RCNN |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110183109.3A CN112907532B (en) | 2021-02-10 | 2021-02-10 | Improved truck door falling detection method based on fast RCNN |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112907532A CN112907532A (en) | 2021-06-04 |
| CN112907532B true CN112907532B (en) | 2022-03-08 |
Family
ID=76123341
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110183109.3A Active CN112907532B (en) | 2021-02-10 | 2021-02-10 | Improved truck door falling detection method based on fast RCNN |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112907532B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115273044B (en) * | 2022-07-15 | 2023-04-07 | 哈尔滨市科佳通用机电股份有限公司 | Vehicle door damage fault identification and detection method based on improved graph convolution network |
| CN116385953B (en) * | 2023-01-11 | 2023-12-15 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon door hinge breaking fault image identification method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019109793A1 (en) * | 2017-12-08 | 2019-06-13 | 腾讯科技（深圳）有限公司 | Human head region recognition method, device and apparatus |
| CN111079747A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon bogie side frame fracture fault image identification method |
| CN111080611A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon bolster spring fracture fault image identification method |
| CN112329737A (en) * | 2020-12-01 | 2021-02-05 | 哈尔滨理工大学 | Vehicle detection method based on improved Faster RCNN |
| CN112330631A (en) * | 2020-11-05 | 2021-02-05 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon brake beam pillar rivet pin collar loss fault detection method |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018165753A1 (en) * | 2017-03-14 | 2018-09-20 | University Of Manitoba | Structure defect detection using machine learning algorithms |
| CN109767427A (en) * | 2018-12-25 | 2019-05-17 | 北京交通大学 | The detection method of train rail fastener defect |
| CN111382766A (en) * | 2018-12-29 | 2020-07-07 | 中国科学院沈阳计算技术研究所有限公司 | Equipment fault detection method based on fast R-CNN |
| CN110210463B (en) * | 2019-07-03 | 2023-07-11 | 中国人民解放军海军航空大学 | Precise ROI-fast R-CNN-based radar target image detection method |
| CN111444939B (en) * | 2020-02-19 | 2022-06-28 | 山东大学 | Small-scale equipment component detection method based on weak supervision cooperative learning in open scene of power field |
| CN112102281B (en) * | 2020-09-11 | 2021-07-06 | 哈尔滨市科佳通用机电股份有限公司 | Truck brake cylinder fault detection method based on improved Faster Rcnn |
-
2021
- 2021-02-10 CN CN202110183109.3A patent/CN112907532B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019109793A1 (en) * | 2017-12-08 | 2019-06-13 | 腾讯科技（深圳）有限公司 | Human head region recognition method, device and apparatus |
| CN111079747A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon bogie side frame fracture fault image identification method |
| CN111080611A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon bolster spring fracture fault image identification method |
| CN112330631A (en) * | 2020-11-05 | 2021-02-05 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon brake beam pillar rivet pin collar loss fault detection method |
| CN112329737A (en) * | 2020-12-01 | 2021-02-05 | 哈尔滨理工大学 | Vehicle detection method based on improved Faster RCNN |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112907532A (en) | 2021-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2697738B1 (en) | Method and system of rail component detection using vision technology | |
| CN111079819B (en) | Method for judging state of coupler knuckle pin of railway wagon based on image recognition and deep learning | |
| CN110254468B (en) | Intelligent online detection device and detection method for track surface defects | |
| EP3321860B1 (en) | Method and system for identifying train number and model, and safety check method and system | |
| CN102759347B (en) | Online in-process quality control device and method for high-speed rail contact networks and composed high-speed rail contact network detection system thereof | |
| CN112907532B (en) | Improved truck door falling detection method based on fast RCNN | |
| CN111079746B (en) | Railway wagon axle box spring fault image identification method | |
| CN109238756B (en) | Dynamic image detection equipment and detection method for freight car operation fault | |
| CN111080598A (en) | Bolt and nut missing detection method for coupler yoke key safety crane | |
| KR102017870B1 (en) | Real-time line defect detection system | |
| CN111080611A (en) | Railway wagon bolster spring fracture fault image identification method | |
| CN109711285B (en) | Training and testing method and device for recognition model | |
| JP6737638B2 (en) | Appearance inspection device for railway vehicles | |
| CN111080607B (en) | Rolling bearing oil slinging fault detection method based on image recognition | |
| CN113221839B (en) | Automatic truck image identification method and system | |
| CN111080614A (en) | Method for identifying damage to rim and tread of railway wagon wheel | |
| CN113112501A (en) | Vehicle-mounted track inspection device and method based on deep learning | |
| CN113487561B (en) | Pantograph foreign matter detection method and device based on gray gradient abnormal voting | |
| CN112061989A (en) | Method for loading and unloading a load with a crane system and crane system | |
| CN112528861A (en) | Foreign matter detection method and device applied to track bed in railway tunnel | |
| CN111091557B (en) | Method and system for detecting breakage fault of flange of bearing saddle of railway wagon | |
| CN115527170A (en) | Method and system for identifying closing fault of door stopper handle of automatic freight car derailing brake device | |
| CN112001908B (en) | Railway freight car sleeper beam hole carried foreign matter detection method | |
| Edwards et al. | Advancements in railroad track inspection using machine-vision technology | |
| CN112330600B (en) | Fault identification method for vehicle end link line fracture based on image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |