CN112382068B - Station waiting line crossing detection system based on BIM and DNN - Google Patents
Station waiting line crossing detection system based on BIM and DNN Download PDFInfo
- Publication number
- CN112382068B CN112382068B CN202011205657.3A CN202011205657A CN112382068B CN 112382068 B CN112382068 B CN 112382068B CN 202011205657 A CN202011205657 A CN 202011205657A CN 112382068 B CN112382068 B CN 112382068B
- Authority
- CN
- China
- Prior art keywords
- crossing
- real
- pedestrian
- waiting line
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 40
- 238000004364 calculation method Methods 0.000 claims abstract description 13
- 230000003247 decreasing effect Effects 0.000 claims abstract description 5
- 239000013598 vector Substances 0.000 claims description 27
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 231100001261 hazardous Toxicity 0.000 claims description 8
- 238000005516 engineering process Methods 0.000 claims description 6
- 238000012800 visualization Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000012549 training Methods 0.000 description 9
- 238000000034 method Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 230000008447 perception Effects 0.000 description 4
- 208000009144 Pure autonomic failure Diseases 0.000 description 3
- 239000013312 porous aromatic framework Substances 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003449 preventive effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
- G08B21/22—Status alarms responsive to presence or absence of persons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/13—Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Artificial Intelligence (AREA)
- Emergency Management (AREA)
- Business, Economics & Management (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Civil Engineering (AREA)
- Structural Engineering (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Architecture (AREA)
- Signal Processing (AREA)
- Train Traffic Observation, Control, And Security (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a station waiting line crossing detection system based on BIM and DNN. The system comprises: the platform building information model module is used for constructing a platform building information model of a station; the crossing detection module is used for detecting whether pedestrians exist in a dangerous area defined by the safe waiting line; the data calculation module is used for calculating an included angle between the real-time orientation of the pedestrians in the non-dangerous area and the safe waiting line and a real-time moving speed component towards the safe waiting line direction; the first crossing prediction module is used for acquiring data from the data calculation module, if the included angle does not belong to the set angle range, predicting that the pedestrian has no crossing tendency, otherwise, further judging whether the real-time moving speed component is zero, if the included angle is zero, predicting that the pedestrian has no crossing tendency, otherwise, further judging whether the real-time moving speed component sequence is decreased progressively, if so, predicting that the pedestrian has no crossing tendency, otherwise, predicting that the pedestrian has the crossing tendency. The invention improves the safety monitoring performance of the platform.
Description
Technical Field
The invention relates to the technical field of BIM and artificial intelligence, in particular to a station waiting line crossing detection system based on BIM and DNN.
Background
Rail transit is becoming the main means of transportation of people's daily trip gradually, including subway, ordinary train, high-speed railway. For preventing the accident of falling, the platform all is provided with the warning line of waiting to send the staff to patrol. Due to the fact that under the rail transit scene, the passenger flow is large, and the effect of manual inspection is not good. Some methods judge whether the passengers cross the waiting line by calculating the distance between the passengers and the warning line, and have the defects that the crossing behavior which occurs is only detected, the crossing tendency prediction of the pedestrians in a non-dangerous area cannot be realized, and the buffer time is provided for preventing the passengers from crossing the line and taking preventive measures.
Disclosure of Invention
The invention aims to provide a station waiting line crossing detection system based on BIM and DNN, aiming at the defects in the prior art.
A station waiting line crossing detection system based on BIM and DNN comprises:
the platform building information model module is used for constructing a platform building information model of a station;
the crossing detection module is used for detecting whether pedestrians exist in a dangerous area defined by the safe waiting line;
the data calculation module is used for calculating an included angle between the real-time orientation of the pedestrians in the non-dangerous area and the safe waiting line and a real-time moving speed component towards the safe waiting line direction;
and the first crossing prediction module is used for acquiring data from the data calculation module, if the included angle does not belong to a set angle range, predicting that the pedestrians have no crossing tendency, otherwise, further judging whether the real-time moving speed component is zero, if so, predicting that the pedestrians have no crossing tendency, otherwise, further judging whether the real-time moving speed component sequence is decreased progressively, if so, predicting that the pedestrians have no crossing tendency, and otherwise, predicting that the pedestrians have the crossing tendency.
The set angle range satisfies:
the pedestrian faces towards the safe waiting line in real time corresponding to any angle in the set angle range, and the smaller included angle between the straight line where the pedestrian faces in real time and the safe waiting line is larger than a first angle.
The system further comprises:
the second crossing prediction module is used for calculating a real-time crossing tendency index of the pedestrian in the non-dangerous area, and predicting that the pedestrian has a crossing tendency if the real-time crossing tendency index continuously exceeds a set threshold, wherein the real-time crossing tendency index is as follows:
wherein,
，
t represents the number of calculations of the real-time moving velocity component,
the included angle between the real-time orientation of the pedestrians in the non-dangerous area and the safe waiting line,
for the real-time moving velocity component in the direction of the safe waiting line,
is the real-time distance between the pedestrian and the safe waiting line,
，
in order to balance the parameters of the weight,
is a constant number of the base numbers,
count according to the following rule: the safe waiting line is taken as a zero-degree axis, the counter-clockwise rotation is 0-180 degrees from the safe waiting line to the track side, and the clockwise rotation is 0-180 degrees from the safe waiting line to the side far away from the track.
The system further comprises:
the alarm module is used for giving an alarm when a dangerous condition occurs, and the dangerous condition comprises: pedestrians are present in the hazardous area.
The hazardous condition further comprises:
the prediction result of the first cross prediction module is a cross-over-inclined dangerous situation, and the prediction result of the second cross prediction module is a cross-over-inclined dangerous situation.
The hazardous condition further comprises:
setting a first sliding time window aiming at the prediction result of the first cross prediction module, and setting a second sliding time window aiming at the prediction result of the second cross prediction module, wherein the first sliding time window and the second sliding time window have the same initial time, size and step length;
the prediction result of the first crossing prediction module in the first sliding time window is a first number with crossing tendency, the prediction result of the second crossing prediction module in the second sliding time window is a second number with crossing tendency, and if the first number and the second number both reach set conditions, the dangerous condition is judged.
The pedestrian detection is performed based on a pedestrian key point detection depth neural network, and the detected pedestrian key points comprise left and right shoulder key points and left and right foot key points.
The method for acquiring the real-time orientation of the pedestrian comprises the following steps:
connecting the left shoulder key point with the right shoulder key point to obtain a first connecting line;
defining a direction for the first connecting line as a left shoulder key point pointing to a right shoulder key point to obtain a first vector;
and on the platform building information model ground coordinate system, rotating the first vector by 90 degrees anticlockwise to obtain a second vector, wherein the unit vector of the second vector is the real-time orientation of the pedestrian.
The system further comprises:
and the real-time tracking module is used for tracking the pedestrians in real time.
The system further comprises:
and the visualization module is used for visualizing the building information model of the station platform area by combining the WebGIS technology.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention detects the line-crossing personnel in the dangerous area and simultaneously detects the line-crossing tendency of the pedestrian in the non-dangerous area, thereby providing buffer time for preventing line crossing and taking preventive measures and improving the safety monitoring performance of the platform.
2. The continuous speed components are related, the speed components cannot be instantly increased to be particularly large or reduced to be particularly small, so that the speed components are reduced to zero after being accumulated to a certain degree, the line crossing tendency is quantized by the second crossing prediction module, the accuracy of line crossing prediction is improved, and meanwhile false detection caused by instant speed is prevented.
3. The BIM technology is combined, the information integration capability of the platform information model is improved, and the WebGIS technology is combined to provide a better visualization effect.
Drawings
FIG. 1 is a system framework diagram of the present invention;
fig. 2 is a schematic diagram of real-time orientation of a pedestrian.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides a station waiting line crossing detection system based on BIM and DNN. FIG. 1 is a block diagram of the system of the present invention. The following description will be made by way of specific examples.
Example 1:
BIM and DNN-based station waiting line crossing detection system comprises: the platform building information model module, the crossing detection module, the data calculation module and the first crossing prediction module.
And the platform building information model module is used for constructing a station platform building information model.
The method comprises the steps of firstly, constructing a platform Building Information Model (BIM) and an Information exchange module, namely, establishing an organic complex of a three-dimensional Building space model and Building Information on the basis of Building Information data of a station platform. The data in the platform building information model comprises platform geographical environment information such as track position information, platform position information and safe waiting line position information and camera position information. Through the information exchange module, the platform building information model can acquire platform scene information, pedestrian detection results and crossing tendency detection results sensed by the camera in real time. The information exchange module is an access module of a database of the BIM, and may be in various forms, for example: can be various information exchange forms such as RESTful, MQ and the like, and an implementer can decide which implementation mode is specifically adopted according to needs. The pedestrian tracking detection result and the crossing tendency prediction result are visually processed by using the WebGIS, and a supervisor can search, inquire and analyze on Web, so that the supervisor can conveniently warn passengers crossing the waiting line and take corresponding countermeasures.
A plurality of cameras are deployed on the station platform, the cameras at least cover the area near the safe waiting line, and the pedestrian moving condition near the safe waiting line can be monitored. In order to reduce the use difficulty of the system and improve the user experience, the position information of the safe waiting line in the embodiment is obtained through automatic detection, and is specifically realized through a waiting line sensing module. And the safe waiting line sensing module is used for recognizing and extracting the waiting line, and the final result is projected to the BIM ground so as to detect whether the safe line is crossed or not through the position relation between the safe waiting line and the key points of the two feet of the passengers.
The safe waiting line sensing module is realized by utilizing a safe waiting line detection neural network. The training method of the neural network for detecting the safe waiting line comprises the following steps: and selecting an image containing a waiting line in the platform area as a training data set, and labeling the data set, wherein the lane label is 0, the waiting line label is 1, and the other categories are labeled 2. Considering that the waiting line is relative to the lane, the lane marking is used for reducing the time required by network training, improving the perception precision of the network and perceiving the waiting line more accurately and rapidly. Where 80% of the data set was randomly selected as the training set and the remaining 20% as the validation set. Inputting image data acquired by a platform camera and labeled label data into a network for training, performing feature extraction by a waiting line perception Encoder Encoder1, and outputting a first feature map Featuremap 1; and then, the Featuremap1 is up-sampled by a waiting line perception Decoder Decoder1, Featuremap1 generated for Encoder1 is input, and finally the network outputs a semantic segmentation graph. The Loss function is trained using a cross entropy Loss function. Therefore, the waiting line in the platform can be extracted through the safe waiting line detection neural network, and the result is projected to the BIM ground plane, so that the subsequent detection of the line crossing behavior of the waiting passenger is facilitated.
And the crossing detection module is used for detecting whether pedestrians exist in the dangerous area defined by the safe waiting line. The pedestrian detection is performed based on a pedestrian key point detection depth neural network, and the detected pedestrian key points comprise left and right shoulder key points and left and right foot key points.
And (3) carrying out image acquisition on the platform environment by utilizing a common camera of the platform, and taking the image acquisition as the input of a pedestrian key point detection neural network. Since there are many passengers on the platform and other vehicles, and false detection are likely to occur during human body detection, it is necessary to distinguish the instances of the passengers in the waiting area of the platform so as to accurately identify the behavior of the passengers crossing the waiting line in the following.
The pedestrian key point detection neural network is implemented based on a partial affinity field and comprises two branches, wherein the first branch outputs a key point heat map, namely PCM, and the second branch outputs a joint affinity field PAFs. The platform camera is used for collecting images, and the left shoulder, the right shoulder, the left foot, the right foot and the like of a person are selected as key points. Then, image tag data is produced. The manufacturing of the heat map label of the key point comprises two steps, wherein in the first step, the selected key point is marked, and the position of the key point is marked, namely X, Y coordinates; and secondly, convolving the marked human body scatter diagram with a Gaussian kernel to obtain two-foot key points. The size of the Gaussian kernel can be automatically adjusted and selected by an implementer according to the height of the camera. PAFs employ a series of 2D vector fields to encode the position and orientation of a limb in the image domain. I.e. each pixel belonging to a torso corresponds to a 2D vector representing the direction from one keypoint to another keypoint of the torso. The tag for the joint affinity field was made as follows: the joint affinity field is encoded according to empirical human limb width, for example, encoding an affinity field of appropriate width pointing from the neck keypoints to the left and right hip center points. In this embodiment, the direction from the left foot key point to the right foot key point and the direction from the left shoulder key point to the right shoulder key point may be encoded, or the direction from the left shoulder key point to the left foot key point and the direction from the right shoulder key point to the right foot key point may be encoded and represented by unit vectors, and the data label should include the key point position and the affinity vector.
The training method of the pedestrian key point detection neural network is as follows. Training of the first branch: the image collected by the camera is normalized, that is, the value range of the picture matrix is changed into a floating point number between [0,1] so that the model can be converged better. The labels are also normalized. The keypoint detection Encoder2 and the keypoint detection Decoder2 are trained end-to-end from the captured images and thermodynamic tag data. The Encoder2 extracts the features of the image, inputs the image data after normalization processing, and outputs a second feature map Featuremap 2; the Decoder2 upsamples the Featuremap2 to finally generate a keypoint thermodynamic diagram (heatmap), which is input into the Featuremap2 generated by the Encoder2 and output into the keypoint thermodynamic diagram. The Loss function of the Loss is used for training. It should be noted that, due to the characteristics of the thermodynamic diagram itself, the thermodynamic diagram output by the network has pixel values conforming to a gaussian distribution, and the value range is between [0,1 ]. Training of the second branch: the relational vector spectrum Encoder Encoder3, the relational vector spectrum Decoder Decoder3 are trained with the acquired image and tag data. The Encoder3 extracts the features of the image, inputs the image data after normalization processing, and outputs a third feature map Featuremap 3; the Decoder3 performs upsampling and feature extraction on the Featuremap3, and finally generates a joint affinity field diagram, which is input into the Featuremap3 generated by the Encoder 3. The Loss function is a mean square error Loss function. The matching method for detecting the postures of multiple persons by using the PAFs obtains the connection weight values between each adjacent component in an integral mode, and applies a matching algorithm of condition constraint, namely, a final matching relation is obtained through maximum weight matching according to the condition that two edges do not share a node. So far, the key points belonging to each passenger in the platform are obtained through the pedestrian key point detection neural network, and the foot key points are projected to the platform BIM model.
And the crossing detection module is used for detecting whether pedestrians exist in the dangerous area defined by the safe waiting line. The BIM is combined with a station platform BIM model, a safe waiting line and passenger two-foot key points are projected into the BIM, and the purpose is to obtain the position relation between the passenger two-foot key points and the waiting line more clearly and visually. And judging whether the passenger crosses the safe waiting line or not according to the position relation between the waiting line in the BIM and the key points of the feet of the passenger. When the coordinates of the two-foot key points of the passengers are positioned outside the waiting line, the passengers waiting for the bus are considered to cross the safe waiting line, and early warning can be sent out.
The invention considers that the passenger may have the risk of crossing the safe waiting line, therefore, the passenger who does not cross the waiting line is further detected and analyzed whether the passenger has the risk of crossing the line. The analysis is mainly performed by the orientation and the rate of movement of the passenger. For the passengers without the cross-line behavior, the orientation of the passengers is judged according to the key points of the left and right shoulders of the passengers, the orientation of the passengers is a perpendicular line which passes through the middle point of the connecting line of the key points of the left and right shoulders of the passengers and is determined according to the left and right shoulders of the passengers, and the direction is recorded as an orientation vector.
And the data calculation module is used for calculating an included angle between the real-time orientation of the pedestrians in the non-dangerous area and the safe waiting line and a real-time moving speed component towards the direction of the safe waiting line.
The method for acquiring the real-time orientation of the pedestrian comprises the following steps: connecting the left shoulder key point with the right shoulder key point to obtain a first connecting line; defining a direction for the first connecting line as a left shoulder key point pointing to a right shoulder key point to obtain a first vector; and on the platform building information model ground coordinate system, rotating the first vector by 90 degrees anticlockwise to obtain a second vector, wherein the unit vector of the second vector is the real-time orientation of the pedestrian. Firstly, making a vertical line towards a waiting line through the center point of key points of left and right shoulders of a passenger, wherein if the passenger faces the waiting line, a vertical line unit vector pointing to the waiting line is the real-time orientation of the pedestrian, and if the passenger faces away from the waiting line, a vertical line unit vector facing away from the waiting line is the real-time orientation of the pedestrian. Then the included angle between the real-time orientation vector of the passenger (namely the pedestrian) and the safe waiting line is calculated. The included angles are counted according to the following rules: the safe waiting line is taken as a zero-degree axis, the counter-clockwise rotation is 0-180 degrees from the safe waiting line to the track side, and the clockwise rotation is 0-180 degrees from the safe waiting line to the side far away from the track. Fig. 2 is a schematic diagram of real-time orientation of a pedestrian.
The real-time moving speed component calculation method comprises the following steps: respectively calculating the position of the same key point between two adjacent frames, and then calculating the instantaneous speed and the instantaneous speed direction according to the frame frequency. And obtaining a real-time moving speed component towards the safe waiting line direction according to the projection and the included angle of the key point in the platform building model. In the invention, the tracking module is used for tracking the passengers in real time so as to confirm whether the passengers are key points of the same person. Preferably, whether the two frames are the same key point or not can be judged through the IOU of the passenger surrounding frame between the adjacent frames, and repeated detection of the same target of different frames is avoided. The passenger enclosure may be obtained from the detected keypoint locations. The IOU method is to find the intersection ratio of two key points, and when the value of IOU is greater than 0.8, it is considered as the same target. It is well known that the IOU of neighboring key points track the detection of passengers and the implementer can adjust the threshold on his own.
The first crossing prediction module is used for acquiring data from the data calculation module, if the included angle does not belong to the set angle range, predicting that the pedestrian has no crossing tendency, otherwise, further judging whether the real-time moving speed component is zero, if the included angle is zero, predicting that the pedestrian has no crossing tendency, otherwise, further judging whether the real-time moving speed component sequence is decreased progressively, if so, predicting that the pedestrian has no crossing tendency, otherwise, predicting that the pedestrian has the crossing tendency. The set angle range satisfies: the pedestrian faces towards the safe waiting line in real time corresponding to any angle in the set angle range, and the smaller included angle between the straight line where the pedestrian faces in real time and the safe waiting line is larger than a first angle.
In the present embodiment, the angular range is defined by the angle a in fig. 2. And taking the perpendicular bisector in the figure 2 as a symmetry axis, if the smaller included angle is smaller than the first angle a, the passenger is considered to be right opposite to the waiting line, otherwise, the passenger is considered to be back opposite to the waiting line. It should be noted that the passenger orientation vector refers to a vector that is perpendicular to the line connecting the left and right shoulders of the passenger through the midpoint of the line connecting the left and right shoulders, and the specific direction is as described above.
Generally, passengers are considered to have no risk of crossing when facing away from the waiting line, and therefore, when facing away from the waiting line, passengers are considered to have no risk of crossing the safe waiting line; when the passenger is facing the waiting line, the passenger moving speed is further analyzed. After the movement rate of the passenger is obtained, the speed change of the key point is compared according to time, and whether the passenger has the risk of crossing the waiting line is predicted based on the movement rate change of the passenger. If the rate of movement of the passenger is zero or the rate is decreasing, it is considered that there is no risk of crossing the waiting line; if the moving speed of key points of the feet of the passengers is larger than zero and is approximately uniform or gradually changed, the passengers think that the passengers have the risk of crossing the safe waiting line, and the staff can timely give an early warning to prevent the passengers from crossing the waiting line.
The system further comprises: the alarm module is used for giving an alarm when a dangerous condition occurs, and the dangerous condition comprises: pedestrians are present in the hazardous area. The hazardous conditions further include: the prediction result of the first cross prediction module is a cross-inclined dangerous situation.
In order to obtain a real-time over-line tendency prediction quantization value and improve over-line tendency prediction, the invention is provided with a second cross prediction module. The second crossing prediction module is used for calculating a real-time crossing tendency index of the pedestrian in the non-dangerous area, and predicting that the pedestrian has a crossing tendency if the real-time crossing tendency index continuously exceeds a set threshold, wherein the real-time crossing tendency index is as follows:
wherein,
，
t represents the number of calculations of the real-time moving velocity component,
the included angle between the real-time orientation of the pedestrians in the non-dangerous area and the safe waiting line,
in real time for the direction of a safe waiting lineThe component of the speed of movement is,
is the real-time distance between the pedestrian and the safe waiting line,
，
in order to balance the parameters of the weight,
is a constant number of the base numbers,
count according to the following rule: the safe waiting line is taken as a zero-degree axis, the counter-clockwise rotation is 0-180 degrees from the safe waiting line to the track side, and the clockwise rotation is 0-180 degrees from the safe waiting line to the side far away from the track. Preferably, the first and second electrodes are formed of a metal,
taking out the mixture of 0.5 percent,
taking out the mixture of 0.8 percent,
take 0.8 and set the threshold to 0.45.
Thus, the dangerous situations also include: and the prediction result of the second crossing prediction module is a crossing-inclined dangerous situation.
In combination with the first cross prediction module and the second cross prediction module, the risk condition further includes: setting a first sliding time window aiming at the prediction result of the first cross prediction module, and setting a second sliding time window aiming at the prediction result of the second cross prediction module, wherein the first sliding time window and the second sliding time window have the same initial time, size and step length; the prediction result of the first crossing prediction module in the first sliding time window is a first number with crossing tendency, the prediction result of the second crossing prediction module in the second sliding time window is a second number with crossing tendency, and if the first number and the second number both reach set conditions, the dangerous condition is judged. It should be clear to the implementer that when the first cross-over prediction module analyzes the velocity component, the forward velocity component sequence should be analyzed to obtain the prediction result at the current time. And the two cross prediction modules are combined, so that the cross prediction precision is improved.
The system further comprises: and the visualization module is used for visualizing the building information model of the station platform area by combining the WebGIS technology. In order to facilitate management and more intuitively present information, the platform BIM model is combined to be visualized through a WebGIS technology, the platform three-dimensional model and related information are updated in real time by calling the information exchange module, the camera perception information can be checked, and the current waiting line crossing detection result information is displayed. Through the visualization of the WebGIS, a manager can search, inquire and analyze on the Web, so that the manager can know the pedestrian crossing trend in the whole shooting area in time and further manage the pedestrian crossing trend.
The above embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the present invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. A station waiting line crossing detection system based on BIM and DNN is characterized by comprising:
the platform building information model module is used for constructing a platform building information model of a station;
the crossing detection module is used for detecting whether pedestrians exist in a dangerous area defined by the safe waiting line;
the data calculation module is used for calculating an included angle between the real-time orientation of the pedestrians in the non-dangerous area and the safe waiting line and a real-time moving speed component towards the safe waiting line direction;
the first crossing prediction module is used for acquiring data from the data calculation module, if the included angle does not belong to a set angle range, predicting that the pedestrian has no crossing tendency, otherwise, further judging whether the real-time moving speed component is zero, if the included angle is zero, predicting that the pedestrian has no crossing tendency, otherwise, further judging whether the real-time moving speed component sequence is decreased progressively, if so, predicting that the pedestrian has no crossing tendency, otherwise, predicting that the pedestrian has the crossing tendency;
the second crossing prediction module is used for calculating a real-time crossing tendency index of the pedestrian in the non-dangerous area, and predicting that the pedestrian has a crossing tendency if the real-time crossing tendency index continuously exceeds a set threshold, wherein the real-time crossing tendency index is as follows:
wherein,
，
t represents the number of calculations of the real-time moving velocity component,
the included angle between the real-time orientation of the pedestrians in the non-dangerous area and the safe waiting line,
for the real-time moving velocity component in the direction of the safe waiting line,
is the real-time distance between the pedestrian and the safe waiting line,
，
in order to balance the parameters of the weight,
is a constant number of the base number,
count according to the following rule: the safe waiting line is taken as a zero-degree axis, the counter-clockwise rotation is 0-180 degrees from the safe waiting line to the track side, and the clockwise rotation is 0-180 degrees from the safe waiting line to the side far away from the track.
2. The system of claim 1, the set angle range satisfying:
the pedestrian faces towards the safe waiting line in real time corresponding to any angle in the set angle range, and the smaller included angle between the straight line where the pedestrian faces in real time and the safe waiting line is larger than a first angle.
3. The system of claim 1, further comprising:
the alarm module is used for giving an alarm when a dangerous condition occurs, and the dangerous condition comprises: pedestrians are present in the hazardous area.
4. The system of claim 3, wherein the hazardous condition further comprises:
the prediction result of the first cross prediction module is a cross-over-inclined dangerous situation, and the prediction result of the second cross prediction module is a cross-over-inclined dangerous situation.
5. The system of claim 3, wherein the hazardous condition further comprises:
setting a first sliding time window aiming at the prediction result of the first cross prediction module, and setting a second sliding time window aiming at the prediction result of the second cross prediction module, wherein the first sliding time window and the second sliding time window have the same initial time, size and step length;
the prediction result of the first crossing prediction module in the first sliding time window is a first number with crossing tendency, the prediction result of the second crossing prediction module in the second sliding time window is a second number with crossing tendency, and if the first number and the second number both reach set conditions, the dangerous condition is judged.
6. The system of claim 1, in which the pedestrian detection is performed based on a pedestrian keypoint detection deep neural network, the detected pedestrian keypoints including left and right shoulder keypoints, left and right foot keypoints.
7. The system of claim 1, wherein the real-time orientation of the pedestrian is obtained by:
connecting the left shoulder key point with the right shoulder key point to obtain a first connecting line;
defining a direction for the first connecting line as a left shoulder key point pointing to a right shoulder key point to obtain a first vector;
and on the platform building information model ground coordinate system, rotating the first vector by 90 degrees anticlockwise to obtain a second vector, wherein the unit vector of the second vector is the real-time orientation of the pedestrian.
8. The system of claim 1, further comprising:
and the real-time tracking module is used for tracking the pedestrians in real time.
9. The system of claim 1, further comprising:
and the visualization module is used for visualizing the building information model of the station platform area by combining the WebGIS technology.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011205657.3A CN112382068B (en) | 2020-11-02 | 2020-11-02 | Station waiting line crossing detection system based on BIM and DNN |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011205657.3A CN112382068B (en) | 2020-11-02 | 2020-11-02 | Station waiting line crossing detection system based on BIM and DNN |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112382068A CN112382068A (en) | 2021-02-19 |
| CN112382068B true CN112382068B (en) | 2022-09-16 |
Family
ID=74577533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011205657.3A Expired - Fee Related CN112382068B (en) | 2020-11-02 | 2020-11-02 | Station waiting line crossing detection system based on BIM and DNN |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112382068B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113486857B (en) * | 2021-08-03 | 2023-05-12 | 云南大学 | YOLOv 4-based ascending safety detection method and system |
| CN114326448B (en) * | 2021-12-30 | 2024-07-02 | 广东威力电器有限公司 | Intelligent toilet cover control method and intelligent toilet cover control system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001006097A (en) * | 1999-06-25 | 2001-01-12 | Fujitsu Ten Ltd | Device for supporting driving for vehicle |
| WO2008127465A1 (en) * | 2007-04-11 | 2008-10-23 | Nec Laboratories America, Inc. | Real-time driving danger level prediction |
| KR20130050763A (en) * | 2011-11-08 | 2013-05-16 | 엘지이노텍 주식회사 | A parking assisting system |
Family Cites Families (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4400316B2 (en) * | 2004-06-02 | 2010-01-20 | 日産自動車株式会社 | Driving intention estimation device, vehicle driving assistance device, and vehicle equipped with vehicle driving assistance device |
| CN1858815B (en) * | 2005-09-27 | 2010-04-14 | 华为技术有限公司 | Position monitoring method and system |
| ATE463813T1 (en) * | 2006-10-09 | 2010-04-15 | Ericsson Telefon Ab L M | METHOD AND SYSTEM FOR DETERMINING A THREAT TO A BORDER |
| JP4888275B2 (en) * | 2007-08-08 | 2012-02-29 | 三菱電機株式会社 | Intrusion detection device |
| CN101894271B (en) * | 2010-07-28 | 2012-11-07 | 重庆大学 | Visual computing and prewarning method of deviation angle and distance of automobile from lane line |
| JP5762730B2 (en) * | 2010-12-09 | 2015-08-12 | パナソニック株式会社 | Human detection device and human detection method |
| CN202018710U (en) * | 2011-02-24 | 2011-10-26 | 中国电信集团系统集成有限责任公司内蒙古分公司 | Guard line crossing alarm system based on 3G (third generation) intelligent video analysis |
| CN103456024B (en) * | 2012-06-02 | 2016-03-09 | 浙江西谷数字技术有限公司 | A kind of moving target gets over line determination methods |
| KR102433791B1 (en) * | 2015-11-20 | 2022-08-19 | 주식회사 에이치엘클레무브 | Lane Departure Warning Apparatus and Method |
| CN105930795A (en) * | 2016-04-20 | 2016-09-07 | 东北大学 | Walking state identification method based on space vector between human body skeleton joints |
| US20170372580A1 (en) * | 2016-06-23 | 2017-12-28 | OnPoint Systems, LLC | Device and Method For Containing and Tracking A Subject Using Satellite Positioning Data |
| CN107762516A (en) * | 2016-08-19 | 2018-03-06 | 张鹏 | A kind of determination method that can be realized to opencast mining boundary |
| CN106406189A (en) * | 2016-11-28 | 2017-02-15 | 中国农业大学 | Electric fence monitoring method for unmanned aerial vehicle plant protecting operations |
| KR101840042B1 (en) * | 2017-03-03 | 2018-03-19 | 주식회사 에스원 | Multi-Imaginary Fence Line Setting Method and Trespassing Sensing System |
| CN107832672B (en) * | 2017-10-12 | 2020-07-07 | 北京航空航天大学 | Pedestrian re-identification method for designing multi-loss function by utilizing attitude information |
| ES1219669Y (en) * | 2018-10-04 | 2019-01-18 | Yuste Cayetano Castellano | ANTISALT CEILING FOR SAFETY VALVES |
| CN109686048B (en) * | 2019-01-02 | 2022-03-18 | 平安科技（深圳）有限公司 | Early warning method and device for line crossing of train platform, computer equipment and storage medium |
| CN109886196A (en) * | 2019-02-21 | 2019-06-14 | 中水北方勘测设计研究有限责任公司 | Personnel track traceability system and method based on BIM plus GIS video monitoring |
| CN110047237B (en) * | 2019-04-15 | 2021-01-26 | 商洛学院 | Intelligent golden eye identification area intrusion alarm method and device |
| CN111756859A (en) * | 2020-07-06 | 2020-10-09 | 中兴飞流信息科技有限公司 | Platform intelligent landmark system based on video AI |
-
2020
- 2020-11-02 CN CN202011205657.3A patent/CN112382068B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001006097A (en) * | 1999-06-25 | 2001-01-12 | Fujitsu Ten Ltd | Device for supporting driving for vehicle |
| WO2008127465A1 (en) * | 2007-04-11 | 2008-10-23 | Nec Laboratories America, Inc. | Real-time driving danger level prediction |
| KR20130050763A (en) * | 2011-11-08 | 2013-05-16 | 엘지이노텍 주식회사 | A parking assisting system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112382068A (en) | 2021-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106980829B (en) | Abnormal behaviour automatic testing method of fighting based on video analysis | |
| CN105574501B (en) | A kind of stream of people's video detecting analysis system | |
| CA3094424A1 (en) | Safety monitoring and early-warning method for man-machine interaction behavior of underground conveyor belt operator | |
| CN106128022A (en) | A kind of wisdom gold eyeball identification violent action alarm method and device | |
| CN112382068B (en) | Station waiting line crossing detection system based on BIM and DNN | |
| CN112116510A (en) | Intelligent park management system and method based on Internet of things | |
| CN112960014B (en) | Rail transit operation safety online real-time monitoring and early warning management cloud platform based on artificial intelligence | |
| CN112434566B (en) | Passenger flow statistics method and device, electronic equipment and storage medium | |
| CN112071084A (en) | Method and system for judging illegal parking by utilizing deep learning | |
| Jwa et al. | Kalman filter based railway tracking from mobile Lidar data | |
| CN114529880A (en) | Urban rail foreign matter intrusion detection method, device and system and storage medium | |
| CN111582069B (en) | Track obstacle zero sample classification method and device for air-based monitoring platform | |
| CN114248819A (en) | Railway intrusion foreign matter unmanned aerial vehicle detection method, device and system based on deep learning | |
| CN113076825A (en) | Transformer substation worker climbing safety monitoring method | |
| Qin et al. | An improved deep learning algorithm for obstacle detection in complex rail transit environments | |
| Sumalee et al. | Probabilistic fusion of vehicle features for reidentification and travel time estimation using video image data | |
| CN113569801B (en) | Distribution construction site live equipment and live area identification method and device thereof | |
| CN113505704B (en) | Personnel safety detection method, system, equipment and storage medium for image recognition | |
| CN117612109A (en) | Step-by-step detection method for non-specific foreign matter invasion limit in railway running track range | |
| Raj et al. | Estimating social distance in public places for COVID-19 protocol using region CNN | |
| CN114708544A (en) | Intelligent violation monitoring helmet based on edge calculation and monitoring method thereof | |
| Klammsteiner et al. | Vision Based Stationary Railway Track Monitoring System | |
| Khan et al. | Multiple moving vehicle speed estimation using Blob analysis | |
| Gao et al. | Molten metal hazards monitoring and early warning system based on convolutional neural network | |
| CN114979567B (en) | Object and region interaction method and system applied to video intelligent monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220830 Address after: Room 301, No. 433, Songhu Road, Yangpu District, Shanghai, 200438 Applicant after: Luban Software Co.,Ltd. Address before: 69 Chuangye Avenue, hi tech Zone, Yantai City, Shandong Province Applicant before: Chen Songshan |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220916 |