[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN118230261A - Intelligent construction site construction safety early warning method and system based on image data - Google Patents

Intelligent construction site construction safety early warning method and system based on image data Download PDF

Info

Publication number
CN118230261A
CN118230261A CN202410660416.XA CN202410660416A CN118230261A CN 118230261 A CN118230261 A CN 118230261A CN 202410660416 A CN202410660416 A CN 202410660416A CN 118230261 A CN118230261 A CN 118230261A
Authority
CN
China
Prior art keywords
image data
construction
early warning
monitoring
similarity calculation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410660416.XA
Other languages
Chinese (zh)
Other versions
CN118230261B (en
Inventor
杨晓娇
何跃川
杨光
黄宇豪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan Institute of Building Research
Original Assignee
Sichuan Institute of Building Research
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan Institute of Building Research filed Critical Sichuan Institute of Building Research
Priority to CN202410660416.XA priority Critical patent/CN118230261B/en
Publication of CN118230261A publication Critical patent/CN118230261A/en
Application granted granted Critical
Publication of CN118230261B publication Critical patent/CN118230261B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an intelligent construction site construction safety early warning method and system based on image data, and relates to the technical field of image data processing. The system comprises a monitoring camera, a memory and an FPGA DPU unit. The method comprises the following steps: dividing a construction function monitoring area; training a similarity calculation model, and constructing a standard image database and a safety early warning information entry table; acquiring real-time monitoring image data of a target construction function monitoring area; calculating the similarity between the monitoring image to be analyzed and the standard image to be analyzed; and searching corresponding safety early warning information from a safety early warning information entry table of the target construction function monitoring area according to the serial number of the standard image data corresponding to the maximum similarity, sending the searched safety early warning information to a building site integrated management platform, and broadcasting the searched safety early warning information in a voice mode in the target construction function monitoring area. The invention can carry out comprehensive and accurate safety monitoring and safety early warning on the whole construction site, and ensures the timely transmission and processing of safety information.

Description

Intelligent construction site construction safety early warning method and system based on image data
Technical Field
The invention relates to the technical field of image data processing, in particular to an intelligent construction site construction safety early warning method and system based on image data.
Background
At present, building construction safety management faces a plurality of challenges, such as accident frequency, difficulty in timely finding potential safety hazards and the like, and brings great pressure to construction site management. The traditional safety management method mainly relies on manual inspection and monitoring, and has the following problems: large building sites are often huge and complex, manual inspection is difficult to achieve full coverage, and missed inspection or blind areas are easy to occur; because the construction site environment is complex, potential safety hazards are often hidden among various materials and equipment, and manual inspection is difficult to find in time; the traditional safety early warning system generally depends on manual judgment, and early warning information transmission speed is low, so that real-time response is difficult to achieve.
In order to solve the problems, intelligent construction site safety early warning technologies based on image data are emerging in recent years. The technologies collect real-time monitoring image data of a construction site through the monitoring camera, realize automatic detection and early warning of potential safety hazards by utilizing the computer vision and deep learning technology, and achieve a certain progress. However, the prior art still has the following limitations: the current image recognition technology has limited accuracy in a complex environment, is easily influenced by factors such as light rays, angles and the like, and has higher false alarm rate; because the model has higher calculation complexity, the real-time performance is insufficient, and the real-time performance requirement of the site safety early warning cannot be met; most of the prior art focuses on the detection of potential safety hazards, and lacks comprehensive monitoring and early warning functions for various construction activities of a construction site. Based on the above, we provide a novel intelligent construction site construction safety early warning method and system based on image data, which aims to overcome the limitation of the prior art and improve the safety management efficiency and accuracy.
Disclosure of Invention
The invention provides an intelligent construction site construction safety pre-warning method and system based on image data, which can solve the problems.
In order to solve the problems, the technical scheme adopted by the invention is as follows:
in a first aspect, the invention provides an intelligent construction site construction safety pre-warning method based on image data, which comprises the following steps:
s1, dividing a construction site into a plurality of construction function monitoring areas according to a plurality of different construction functions of the construction site;
S2, constructing and training a similarity calculation model based on a deep learning model for each construction function monitoring area, and constructing a corresponding standard image database and a safety early warning information item table, wherein standard image data in the standard image database corresponds to safety early warning information in the safety early warning information item table one by one and has the same serial number;
S3, acquiring real-time monitoring image data of a target construction function monitoring area, and preprocessing to obtain monitoring image data to be analyzed;
S4, randomly acquiring standard image data which is not subjected to similarity calculation from a standard image database of the target construction function monitoring area, and preprocessing to obtain standard image data to be analyzed;
S5, inputting the monitoring image data to be analyzed and the standard image data to be analyzed into a trained similarity calculation model of the target construction function monitoring area, and calculating the similarity of the monitoring image data to be analyzed and the standard image data to be analyzed;
S6, judging whether standard image data which is not calculated in similarity exists in a standard image database of the target construction function monitoring area, if so, jumping to the step S4, otherwise, executing the step S7;
s7, judging whether the maximum similarity reaches a similarity threshold, if so, completing the judgment of the current real-time monitoring image data, and jumping to the step S3, otherwise, executing the step S8;
And S8, searching corresponding safety early warning information from a safety early warning information entry table of the target construction function monitoring area according to the serial number of the standard image data corresponding to the maximum similarity, sending the searched safety early warning information to a building site integrated management platform, broadcasting the searched safety early warning information in a voice mode in the target construction function monitoring area, and then jumping to the step S3.
As a further description of the above technical solution: allowing more than two construction function monitoring areas to exist, corresponding to the same physical construction site unit area, and when the situation occurs, allowing a set of monitoring equipment to be adopted, and acquiring monitoring image data corresponding to different construction functions from the construction site unit area; for a certain construction function monitoring area, the standard image database of the construction function monitoring area consists of a plurality of image data which completely meet the requirements of a construction site.
As a further description of the above technical solution: in step S2, the method for constructing and training the similarity calculation model includes the following steps:
S21, collecting a plurality of image groups, wherein the construction sequence points corresponding to the image groups are different, each image group comprises a plurality of samples corresponding to the same construction sequence point, and each sample comprises standard image data and historical monitoring image data corresponding to the same construction sequence point;
S22, for each image group, constructing a training set and a verification set, wherein the numerical ratio of samples in the training set to samples in the verification set is 8:2, each sample comprises a standard image and a history monitoring image, and each sample is provided with a label and used for indicating whether the samples belong to the same construction sequence point or not;
S23, constructing a similarity calculation model based on a Siamese network structure, wherein the similarity calculation model comprises an image input layer, a convolution layer, a pooling layer, a full connection layer and a distance calculation layer; the image input layer is used for inputting two images at a time, and the size of each image is a fixed size; the convolution layer is used for extracting the characteristics of the image by adopting a convolution kernel; the pooling layer is used for reducing the size of the feature map by adopting average pooling and retaining the most important features; the full connection layer is used for flattening the output of the pooling layer into a vector, and combining and abstracting the features through full connection to obtain two final feature vectors; the distance calculation layer is used for calculating the distance between the two final feature vectors based on a Manhattan distance calculation method and taking the distance as a similarity score between the two images;
S24, training the similarity calculation model by using a training sample, updating model parameters to minimize a loss function by iterating the training sample, verifying the performance of the similarity calculation model by using a verification sample, and adjusting the super parameters of the model according to a verification result;
And S25, after training is completed, preserving parameters of the similarity calculation model.
As a further description of the above technical solution: in step S24, the loss function used in the training process of the similarity calculation model is a contrast loss function, and the formula is:
Wherein L is contrast loss, and N is sample number; y i is a label of the sample i, which indicates whether the two images belong to the same construction sequence point, that is, if the two images in the sample i belong to the same construction sequence point, y i =1, otherwise, y i=0;di is a manhattan distance between feature vectors of the sample i, which indicates similarity between image data; w i is the weight parameter of sample i to balance the losses between the same class and different classes; alpha is a balance parameter of the sample weight and is used for adjusting the influence degree of the sample weight; beta is a constant term for adjusting the magnitude of the overall loss; m is a hyper-parameter representing the maximum distance between the standard image and the history monitor image that is considered similar in sample i, if the distance between the standard image and the history monitor image is less than m, they are considered to be of the same class, otherwise they are of different classes.
As a further description of the above technical solution:
The step S24 specifically includes the following steps:
S241, initializing similarity calculation model parameters;
S242, randomly selecting a sample i from the training set, wherein the sample i comprises standard image data x s and historical monitoring image data x h;
S243, inputting the standard image data x s and the history monitoring image data x h into a similarity calculation model, and calculating the characteristic vectors f s and f h and the Manhattan distance d i between the characteristic vectors f s and f h;
S244, calculating a contrast loss L by using a contrast loss function formula based on the Manhattan distance d i between the feature vectors f s and f h and the labels y i of the corresponding samples;
S245, calculating the gradient of the contrast loss function with respect to the similarity calculation model parameters by using a back propagation algorithm, and updating the parameters of the similarity calculation model by using a gradient descent method;
S246, repeating the steps S242 to S245 until all samples in the training set are traversed, or N training rounds are completed;
s247, evaluating the performance of the current similarity calculation model by using the verification set;
s248, adjusting the super parameters of the similarity calculation model according to the verification result.
As a further description of the above technical solution:
Step S245 includes:
(1) From the contrast loss L, the eigenvectors f s and f h, the Manhattan distance d i, the sample tag y i, and the sample i weight w i, the gradient of the loss function with respect to the eigenvectors f s and f h is calculated 、/>The calculation formula is as follows:
Wherein sign represents a sign taking function;
(2) Gradient with respect to eigenvectors f s and f h according to the loss function 、/>Calculating the gradient of the similarity calculation model parameters by using a back propagation algorithm;
(3) And updating the similarity calculation model parameters by adopting a gradient descent method according to the gradient of the similarity calculation model parameters.
As a further description of the above technical solution: step S247 specifically includes: for each sample in the verification set, predicting by using a similarity calculation model to obtain a prediction result; calculating an accuracy rate P and a recall rate R according to the prediction result and the real label, wherein the formula is as follows:
Wherein TP represents the number of true positives, FP represents the number of false positives, FN represents the number of false negatives;
and averaging the accuracy and recall of all samples to obtain the performance index of the similarity calculation model.
As a further description of the above technical solution: the methods of preprocessing the image data in steps S3 and S4 are the same, and include resizing, normalization, enhancement, and conversion to tensors.
As a further description of the above technical solution: the method further comprises the steps of obtaining real-time wind speed and wind direction data, setting a wind direction range, a first wind speed data threshold value and a second wind speed data threshold value, wherein the first wind speed data threshold value is larger than the second wind speed data, and sending wind condition early warning to the building site comprehensive management platform when the real-time wind speed data in the real-time wind speed and wind direction data is larger than the first wind speed data threshold value or when the real-time wind speed data in the real-time wind speed and wind direction data is larger than the second wind speed data threshold value and the real-time wind direction in the real-time wind speed and wind direction data is located in the wind direction range.
In a second aspect, the invention provides an intelligent construction site construction safety early warning system based on image data, which comprises a monitoring camera, a memory and an FPGA DPU unit; the monitoring cameras are distributed in each construction function monitoring area of the construction site and are used for collecting real-time monitoring image data; the storage stores a construction site construction safety early warning program, and the FPGA DPU unit executes the intelligent construction site construction safety early warning method based on the image data in the first aspect by executing the construction site construction safety early warning program.
Compared with the prior art, the invention has the beneficial effects that:
1) By dividing the construction site into different construction function monitoring areas, the system can comprehensively and safely monitor the whole construction site, and the safety of the construction site is improved.
2) And each construction function monitoring area is pertinently constructed with a similarity calculation model and a standard image database, and safety early warning information is customized according to the characteristics of each area, so that the early warning accuracy and reliability are improved.
3) The system can acquire and process the monitoring image data in real time, discover potential safety hazards in time and perform early warning processing, so that the possibility of accident occurrence is reduced.
4) And a similarity calculation model is built through the deep learning model, so that intelligent image similarity calculation is realized, and the processing efficiency and accuracy of the system are improved.
5) The system can automatically judge the similarity of the monitoring image data and the standard image data, and automatically trigger early warning or alarm according to the preset threshold value, thereby reducing the requirement of manual intervention and improving the management efficiency.
6) The safety early warning information is sent to the building site integrated management platform and is broadcasted to the monitoring area through voice, so that timely transmission and processing of the safety information are guaranteed, and timely countermeasures are facilitated.
7) The monitoring cameras are distributed in each construction function monitoring area of the construction site, so that the comprehensive monitoring of the construction site is realized, and the safety management level of the construction site is improved; the FPGA DPU unit can quickly execute the construction site safety early warning programs, so that the quick processing and early warning of the monitoring image data are realized, and the early warning processing efficiency is improved; the FPGADPU unit has lower delay and high performance, can meet the safety early warning processing requirement with higher real-time requirement, and ensures the response speed of the system; the system adopts the FPGA DPU unit, has stronger flexibility and expandability, can meet the requirements of different scale construction sites, and supports the upgrading and the expansion of the system; by the application of the intelligent early warning system, the construction management of the construction site is intelligent, the construction efficiency and the safety are improved, and the management cost and the risk are reduced.
In order to make the above objects, features and advantages of the present invention more comprehensible, embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a block diagram of an embodiment intelligent worksite construction safety precaution system;
FIG. 2 is a flow chart of an embodiment of an intelligent worksite construction safety precaution method based on image data;
FIG. 3 is a flow chart of a method of constructing and training a similarity calculation model according to an embodiment;
FIG. 4 is a flow chart of an embodiment of training an updated similarity calculation model based on gradient descent.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention.
Referring to fig. 1, the embodiment provides an intelligent construction site construction safety early warning system based on image data, which comprises a monitoring camera, a switch, an ethernet optical fiber MAC and an FPGA DPU unit; the monitoring camera is in data communication connection with the FPGA DPU unit through the switch and the Ethernet optical fiber MAC, the FPGA DPU is used for carrying out image reasoning and prediction on the acquired images, and the wireless communication base station can send alarm feedback information to the building site integrated management platform, and the carrier of the building site integrated management platform can be a PC (personal computer) or a smart phone.
The monitoring cameras are distributed in each construction function monitoring area of the construction site and are used for collecting real-time monitoring image data; the FPGA DPU unit can execute a construction site construction safety early warning program so as to realize intelligent construction site construction safety early warning based on image data.
The construction site construction safety early warning program, the image data and all intermediate data can be stored in an independent mobile memory.
As shown in fig. 2, the embodiment provides an intelligent construction site construction safety early warning method based on image data, which specifically includes the following steps:
S1, dividing a construction site into a plurality of construction function monitoring areas according to a plurality of different construction functions of the construction site.
The types of construction functions that they have may be different for different building construction sites, examples of construction function monitoring areas are given below (note that the construction function monitoring areas may be increased or decreased according to different construction sites):
construction area: the overall condition of the job site is monitored, including various construction activities and operations.
Lifting area: and monitoring operations such as hoisting and lifting, and preventing unexpected falling or collision of hoisting equipment and hoisted articles.
High-rise operation area: the overhead operation is monitored, including the use of various overhead operation equipment and the safe operation of workers.
Traffic area: and monitoring the traffic route of the construction site, the traveling condition of vehicles and pedestrians, and preventing traffic accidents.
Material stacking area: monitoring the stacking and carrying process of the materials, and avoiding potential safety hazards caused by improper stacking or improper carrying of the materials.
Dangerous goods area: the storage and use conditions of special materials such as dangerous chemicals, inflammable and explosive materials and the like are monitored, and accidents such as chemical leakage or fire explosion and the like are prevented.
Construction equipment area: the use condition of various construction equipment is monitored, and the operation safety of large-scale equipment such as an excavator, a road roller, a drilling machine and the like is monitored.
Edge area: monitoring the operation of side construction such as side slope and overhead, and preventing workers from falling or articles from falling to hurt people.
Special operation area: and setting monitoring points for safety monitoring according to specific conditions in special operation areas such as a blasting operation area, a deep foundation pit operation area and the like.
S2, constructing and training a similarity calculation model based on a deep learning model for each construction function monitoring area, and constructing a corresponding standard image database and a safety early warning information item table, wherein standard image data in the standard image database corresponds to safety early warning information in the safety early warning information item table one by one and has the same serial number.
The safety early warning information item table comprises safety problems and early warning information description which do not meet the requirements of standard image data, and when the similarity between the implemented monitoring image data and the corresponding standard image data is too low, the safety problems are described as existing, and early warning is needed. An example of the security pre-warning information entry table is as follows:
safety precaution information item table 1-construction area
Sequence number Safety problem Early warning information
1 Safety helmet not worn by worker Reminding worker to wear safety helmet
2 Temporary wire confusion and stumbling risk Notice the detour to keep the channel clean
3 Construction machinery is not parked or improperly parked Reminding an operator to stop and park correctly
Safety precaution information item table 2-lifting area
Sequence number Safety problem Early warning information
1 Overweight or unstable lifting articles Warning workers of distance from the lifting area
2 Damage or overload of hooks Stopping hoisting operation and overhauling equipment
Safety precaution information item table 3-overhead working area
Sequence number Safety problem Early warning information
1 Lack of protective rails or safety nets Reminding workers to strengthen safety protection
2 The worker is not belted or not used correctly Safety belt for warning worker to use correctly
Safety precaution information item table 4-traffic area
Sequence number Safety problem Early warning information
1 Overspeed or scram and disorderly putting of vehicle Reminding driver of traffic safety
2 Pedestrian intrusion prohibition area or construction area Safety warning sign for reminding pedestrians to pay attention to
Safety precaution information item table 5-material stacking area
Sequence number Safety problem Early warning information
1 Unstable or ultra-high material stacking Alerting workers to distance from dangerous areas
2 Material stacking shielding emergency exit or passage Reminding worker cleaning channel
Safety precaution information item table 6-dangerous goods area
Sequence number Safety problem Early warning information
1 Leakage of chemicals or abnormal odors Immediate evacuation and notification to emergency department
2 Broken or unsealed safety container Immediate handling of leaks and replacement of containers
Safety precaution information entry table 7-construction equipment area
Sequence number Safety problem Early warning information
1 Malfunction or improper operation of equipment Alerting operators to operating specifications
2 Equipment malfunction or abnormal sound Shutdown overhauling equipment
Safety precaution information item table 8-border area
Sequence number Safety problem Early warning information
1 Lack of guard rails or warning signs Warning workers away from bordering areas
2 Construction slope looseness or collapse Immediate evacuation of dangerous areas
Safety precaution information item table 9-special work area
Sequence number Safety problem Early warning information
1 Countdown of blasting operation Evacuation of dangerous areas, maintenance of a safe distance
2 Collapse risk of deep foundation pit Inhibit entry, warn workers to keep away from
For a certain construction function monitoring area, the standard image database of the construction function monitoring area consists of a plurality of image data which completely meet the requirements of a construction site. The standard image database is exemplified as follows:
1) Standard image database-construction area
Standard image data 1: the workers in the construction site wear the safety helmet, and the road in the construction site is smooth and free from barriers.
Standard image data 2: the workers perform construction operation in a specified area, and the wires are orderly and orderly without tripping risks.
Standard image data 3: the construction machine is parked neatly, and the safety mark is indicated and set by staff.
2) Standard image database-lifting area
Standard image data 1: and in the hoisting operation, the suspended object is stable, and the lifting hook is intact.
Standard image data 2: the lifting device is parked in a designated area without personnel and obstacles around.
3) Standard image database-high-altitude operation area
Standard image data 1: the high working area is provided with a perfect safety protection railing and a safety net.
Standard image data 2: the worker wears the safety belt correctly and performs high-place operation according to the specification.
4) Standard image database-traffic area
Standard image data 1: the vehicles in the traffic area run at a specified speed, and the vehicles are parked orderly.
Standard image data 2: pedestrians enter the construction site from the designated channel, and obvious safety warning signs are arranged around the pedestrians.
5) Standard image database-material stacking area
Standard image data 1: the materials are stacked neatly and stably, and an emergency outlet or a channel is not blocked.
Standard image data 2: the material stacking area of the construction site is clean and tidy, and workers can arrange the material stacking area around the construction site.
6) Standard image database-dangerous area
Standard image data 1: the container in the dangerous chemical storage area is intact and has no leakage phenomenon.
Standard image data 2: obvious safety warning signs and protective measures are arranged around the dangerous goods storage area.
7) Standard image database-construction equipment area
Standard image data 1: the construction equipment operates normally without abnormal sound and fault phenomena.
Standard image data 2: and operating the equipment by an operator according to the specification, and stopping the equipment in a designated area.
8) Standard image database-edge area
Standard image data 1: the border area is provided with a firm protective railing and a warning sign, and workers are far away from the border area.
Standard image data 2: the construction slope is firm and stable, and no personnel or barriers exist around the construction slope.
9) Standard image database-special working area
Standard image data 1: before blasting operation, obvious warning signs and warning lines are arranged around the blasting machine, and people are evacuated from dangerous areas.
Standard image data 2: the protection railing and the warning sign are arranged around the deep foundation pit operation area, and collapse phenomenon is not seen.
As shown in fig. 3, the method for constructing and training the similarity calculation model includes the following steps:
S21, collecting a plurality of image groups, wherein the construction sequence points corresponding to the image groups are different, each image group comprises a plurality of samples corresponding to the same construction sequence point, and each sample comprises standard image data and historical monitoring image data corresponding to the same construction sequence point.
S22, for each image group, constructing a training set and a verification set, wherein the numerical ratio of samples in the training set to samples in the verification set is 8:2, each sample comprises a standard image and a historical monitoring image, and each sample is provided with a label for indicating whether the samples belong to the same construction sequence point.
S23, constructing a similarity calculation model based on a Siamese network structure, wherein the similarity calculation model comprises an image input layer, a convolution layer, a pooling layer, a full connection layer and a distance calculation layer; the image input layer is used for inputting two images at a time, and the size of each image is a fixed size; the convolution layer is used for extracting the characteristics of the image by adopting a convolution kernel; the pooling layer is used for reducing the size of the feature map by adopting average pooling and retaining the most important features; the full connection layer is used for flattening the output of the pooling layer into a vector, and combining and abstracting the features through full connection to obtain two final feature vectors; the distance calculation layer is used for calculating the distance between the two final feature vectors based on the Manhattan distance calculation method and taking the distance as a similarity score between the two images.
S24, training the similarity calculation model by using a training sample, as shown in FIG. 4, specifically comprising the following steps:
S241, initializing similarity calculation model parameters;
S242, randomly selecting a sample i from the training set, wherein the sample i comprises standard image data x s and historical monitoring image data x h;
S243, inputting the standard image data x s and the history monitoring image data x h into a similarity calculation model, and calculating the characteristic vectors f s and f h and the Manhattan distance d i between the characteristic vectors f s and f h;
S244, calculating a contrast loss L by using a contrast loss function formula based on the Manhattan distance d i between the feature vectors f s and f h and the labels y i of the corresponding samples;
the contrast loss function formula is:
Wherein L is contrast loss, and N is sample number; y i is a label of the sample i, which indicates whether the two images belong to the same construction sequence point, that is, if the two images in the sample i belong to the same construction sequence point, y i =1, otherwise, y i=0;di is a manhattan distance between feature vectors of the sample i, which indicates similarity between image data; w i is the weight parameter of sample i to balance the losses between the same class and different classes; alpha is a balance parameter of the sample weight and is used for adjusting the influence degree of the sample weight; beta is a constant term for adjusting the magnitude of the overall loss; m is a hyper-parameter representing the maximum distance between the standard image and the history monitoring image that is considered similar in sample i, if the distance between the standard image and the history monitoring image is less than m, they are considered to be of the same class, otherwise they are of different classes;
S245, updating parameters of a similarity calculation model, wherein the parameters are specifically as follows:
(1) From the contrast loss L, the eigenvectors f s and f h, the Manhattan distance d i, the sample tag y i, and the sample i weight w i, the gradient of the loss function with respect to the eigenvectors f s and f h is calculated 、/>The calculation formula is as follows:
Wherein sign represents a sign taking function;
(2) Gradient with respect to eigenvectors f s and f h according to the loss function 、/>Calculating the gradient of the similarity calculation model parameters by using a back propagation algorithm;
(3) Updating the similarity calculation model parameters by adopting a gradient descent method according to the gradient of the similarity calculation model parameters;
S246, repeating the steps S242 to S245 until all samples in the training set are traversed;
s247, evaluating the performance of the current similarity calculation model by using the verification set;
For each sample in the verification set, predicting by using a similarity calculation model to obtain a prediction result; calculating an accuracy rate P and a recall rate R according to the prediction result and the real label, wherein the formula is as follows:
Wherein TP represents the number of true positives, FP represents the number of false positives, FN represents the number of false negatives;
Then, the accuracy and recall of all samples are averaged to be used as the performance index of a similarity calculation model, so that the processing has the advantages of simplicity, intuitiveness, good comprehensiveness, strong robustness, wide applicability and the like; the average accuracy and recall can be integrated to account for the performance of the model on all samples, and is not limited to a single sample or a particular class, thus enabling a more comprehensive assessment of the performance of the similarity calculation model. The average accuracy and recall have intuitive interpretation and calculation methods, so that engineers and decision makers can understand and use the methods conveniently; average accuracy and recall can provide a more robust performance assessment in the event of sample imbalance, as they take into account the weights of all samples.
S248, adjusting the super parameters of the similarity calculation model according to the verification result.
And S25, after training is completed, preserving parameters of the similarity calculation model.
And S3, acquiring real-time monitoring image data of the target construction function monitoring area, and preprocessing to obtain monitoring image data to be analyzed so as to match with a similarity calculation model.
The pretreatment method comprises the following steps:
resizing the: the images are resized to a fixed size of the model input in order to ensure that all input images have the same size for model processing.
Normalization: the pixel values of the image are normalized to fall within the range of 0to 1 or-1 to 1, which helps to improve the stability and convergence speed of the model.
Enhancement: and (3) performing enhancement operations on the image according to the need, such as random clipping, rotation, overturning and the like, so as to increase the diversity of data and the generalization capability of the model.
Conversion to tensors: the image is converted into a tensor format acceptable to the model, and the invention converts the pixel values of the image into a floating point tensor.
S4, randomly acquiring standard image data which is not subjected to similarity calculation from a standard image database of the target construction function monitoring area, and preprocessing to obtain standard image data to be analyzed, wherein the preprocessing method is the same as that in the step S3.
S5, inputting the monitoring image data to be analyzed and the standard image data to be analyzed into a trained similarity calculation model of the target construction function monitoring area, and calculating the similarity of the monitoring image data to be analyzed and the standard image data to be analyzed.
S6, judging whether standard image data which is not calculated in similarity exists in a standard image database of the target construction function monitoring area, if so, jumping to the step S4, otherwise, executing the step S7.
After the steps are executed, the construction sequence point corresponding to the current real-time monitoring image data of the target construction function monitoring area can be quickly found through the maximum similarity.
And S7, judging whether the maximum similarity reaches a similarity threshold, if so, completing the judgment of the current real-time monitoring image data, and jumping to the step S3, otherwise, executing the step S8.
If the similarity between the current real-time monitoring image data of the target construction function monitoring area and the standard image data of the same construction sequence point reaches a similarity threshold value, the current construction operation of the construction sequence point is indicated, the construction requirement is met, otherwise, the construction requirement is not met, and potential safety and quality hazards exist.
The user can modify the similarity threshold according to the actual condition of the construction site, if the similarity threshold is larger, the safety requirement on the construction site is higher, otherwise, the similarity threshold is lower.
S8, searching corresponding safety early warning information from a safety early warning information entry table of the target construction function monitoring area according to the serial number of the standard image data corresponding to the maximum similarity, sending the searched safety early warning information to a building site integrated management platform, broadcasting the searched safety early warning information in the target construction function monitoring area in a voice mode, and then jumping to the step S3 to achieve circulation of construction safety early warning of the building site.
In this embodiment, by identifying the most similar standard image and searching the corresponding safety early warning information, the system can timely find the potential safety hazard in the construction site and perform early warning, thereby being beneficial to preventing accidents or reducing the accident risk. The safety early warning information is sent to the comprehensive management platform of the construction site, so that the centralized management and monitoring of the safety management information can be realized, and relevant management personnel can know the safety condition of the construction site in time and take corresponding measures. The voice broadcasting of the safety early warning information is carried out in the target construction function monitoring area, so that the perception of the safety early warning information by the construction site workers can be improved, the safety consciousness is further enhanced, and the workers can take timely action to avoid potential danger. By the method for matching the similarity, the safety early warning is triggered only when the monitored image is highly similar to the standard image, so that the false alarm rate can be reduced, and the accuracy and the reliability of the early warning can be improved. Through the automatic process, the system can rapidly and accurately identify potential safety hazards and send early warning information, thereby reducing the workload of management personnel and improving the early warning processing efficiency.
In an alternative embodiment of the invention, more than two construction function monitoring areas are allowed to exist, corresponding to the same physical construction function unit area, when the situation occurs, a set of monitoring equipment is allowed to be adopted, and monitoring image data corresponding to different construction functions are acquired from the construction function unit area; for a certain construction function monitoring area, the standard image database of the construction function monitoring area consists of a plurality of image data which completely meet the requirements of a construction site.
The method allows a plurality of construction functions to share one set of monitoring equipment, and acquires monitoring image data from the same physical building site unit area, so that equipment investment and maintenance cost can be reduced, and resource utilization rate can be improved. The monitoring image data of different construction functions can be obtained from the same position at the same time, so that the arrangement and adjustment time of equipment are reduced, and the monitoring efficiency and flexibility are improved. Although the monitoring images from the same physical area serve different construction function monitoring areas, the standard image databases of the monitoring images are independent, so that the standard image databases meeting the requirements can be respectively established according to the requirements of the construction functions, and the separation and unified management of information are realized. The standard image database of each construction function monitoring area contains image data meeting the requirements of the construction site, so that real-time monitoring images can be more accurately matched, the false alarm rate is reduced, and the reliability of safety early warning is improved. The new construction function monitoring area is allowed to be added in the same physical area, only the standard image database is required to be expanded, no additional monitoring equipment is required, and the flexibility and the expandability of the system are improved.
In an optional embodiment of the present invention, the intelligent site construction safety precaution method based on image data further includes obtaining real-time wind speed and wind direction data, setting a wind direction range, a first wind speed and wind direction data threshold value, and a second wind speed and wind direction data threshold value, wherein the first wind speed and wind direction data threshold value is greater than the second wind speed and wind direction data, and when the real-time wind speed and wind direction data in the real-time wind speed and wind direction data is greater than the first wind speed and wind direction data threshold value, or when the real-time wind speed and wind direction data in the real-time wind speed and wind direction data is greater than the second wind speed and wind direction data in the real-time wind speed and wind direction data is within the wind direction range, sending the wind condition precaution to the site integrated management platform.
The wind speed and wind direction data are monitored in real time, a wind speed threshold value and a wind direction range are set, possible risk situations of a construction site can be early warned in time, and construction site accidents caused by large wind power are avoided. The two different wind speed data thresholds are set, so that the sensitivity of wind speed early warning can be adjusted according to actual conditions, and false alarm caused by instant wind speed fluctuation is avoided. Considering the wind direction range, when the wind speed reaches the threshold value, the early warning can be triggered only when the wind direction is in the designated range, so that false alarm or missing alarm caused by wind direction change is avoided. Once the wind speed or the wind direction exceeds a set threshold value, the system immediately sends wind condition early warning to the comprehensive construction site management platform, so that a construction site manager can take measures to cope with risks rapidly.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. The intelligent construction site construction safety early warning method based on the image data is characterized by comprising the following steps of:
s1, dividing a construction site into a plurality of construction function monitoring areas according to a plurality of different construction functions of the construction site;
S2, constructing and training a similarity calculation model based on a deep learning model for each construction function monitoring area, and constructing a corresponding standard image database and a safety early warning information item table, wherein standard image data in the standard image database corresponds to safety early warning information in the safety early warning information item table one by one and has the same serial number;
S3, acquiring real-time monitoring image data of a target construction function monitoring area, and preprocessing to obtain monitoring image data to be analyzed;
S4, randomly acquiring standard image data which is not subjected to similarity calculation from a standard image database of the target construction function monitoring area, and preprocessing to obtain standard image data to be analyzed;
S5, inputting the monitoring image data to be analyzed and the standard image data to be analyzed into a trained similarity calculation model of the target construction function monitoring area, and calculating the similarity of the monitoring image data to be analyzed and the standard image data to be analyzed;
S6, judging whether standard image data which is not calculated in similarity exists in a standard image database of the target construction function monitoring area, if so, jumping to the step S4, otherwise, executing the step S7;
s7, judging whether the maximum similarity reaches a similarity threshold, if so, completing the judgment of the current real-time monitoring image data, and jumping to the step S3, otherwise, executing the step S8;
And S8, searching corresponding safety early warning information from a safety early warning information entry table of the target construction function monitoring area according to the serial number of the standard image data corresponding to the maximum similarity, sending the searched safety early warning information to a building site integrated management platform, broadcasting the searched safety early warning information in a voice mode in the target construction function monitoring area, and then jumping to the step S3.
2. The intelligent construction safety pre-warning method based on image data according to claim 1, wherein more than two construction function monitoring areas are allowed to exist, the two construction function monitoring areas correspond to the same physical construction function unit area, and when the situation occurs, one set of monitoring equipment is allowed to be adopted, and monitoring image data corresponding to different construction functions are acquired from the construction function unit area; for a certain construction function monitoring area, the standard image database of the construction function monitoring area consists of a plurality of image data which completely meet the requirements of a construction site.
3. The intelligent construction safety precaution method based on image data according to claim 1, wherein in step S2, the method for constructing and training the similarity calculation model comprises the following steps:
S21, collecting a plurality of image groups, wherein the construction sequence points corresponding to the image groups are different, each image group comprises a plurality of samples corresponding to the same construction sequence point, and each sample comprises standard image data and historical monitoring image data corresponding to the same construction sequence point;
S22, for each image group, constructing a training set and a verification set, wherein the numerical ratio of samples in the training set to samples in the verification set is 8:2, each sample comprises a standard image and a history monitoring image, and each sample is provided with a label and used for indicating whether the samples belong to the same construction sequence point or not;
S23, constructing a similarity calculation model based on a Siamese network structure, wherein the similarity calculation model comprises an image input layer, a convolution layer, a pooling layer, a full connection layer and a distance calculation layer; the image input layer is used for inputting two images at a time, and the size of each image is a fixed size; the convolution layer is used for extracting the characteristics of the image by adopting a convolution kernel; the pooling layer is used for reducing the size of the feature map by adopting average pooling and retaining the most important features; the full connection layer is used for flattening the output of the pooling layer into a vector, and combining and abstracting the features through full connection to obtain two final feature vectors; the distance calculation layer is used for calculating the distance between the two final feature vectors based on a Manhattan distance calculation method and taking the distance as a similarity score between the two images;
S24, training the similarity calculation model by using a training sample, updating model parameters to minimize a loss function by iterating the training sample, verifying the performance of the similarity calculation model by using a verification sample, and adjusting the super parameters of the model according to a verification result;
And S25, after training is completed, preserving parameters of the similarity calculation model.
4. The intelligent construction safety precaution method based on image data according to claim 3, wherein in step S24, a loss function used in the training process of the similarity calculation model is a contrast loss function, and the formula is:
Wherein L is contrast loss, and N is sample number; y i is a label of the sample i, which indicates whether the two images belong to the same construction sequence point, that is, if the two images in the sample i belong to the same construction sequence point, y i =1, otherwise, y i=0;di is a manhattan distance between feature vectors of the sample i, which indicates similarity between image data; w i is the weight parameter of sample i to balance the losses between the same class and different classes; alpha is a balance parameter of the sample weight and is used for adjusting the influence degree of the sample weight; beta is a constant term for adjusting the magnitude of the overall loss; m is a hyper-parameter representing the maximum distance between the standard image and the history monitor image that is considered similar in sample i, if the distance between the standard image and the history monitor image is less than m, they are considered to be of the same class, otherwise they are of different classes.
5. The intelligent construction site construction safety precaution method based on image data as claimed in claim 4, wherein the step S24 specifically comprises the following steps:
S241, initializing similarity calculation model parameters;
S242, randomly selecting a sample i from the training set, wherein the sample i comprises standard image data x s and historical monitoring image data x h;
S243, inputting the standard image data x s and the history monitoring image data x h into a similarity calculation model, and calculating the characteristic vectors f s and f h and the Manhattan distance d i between the characteristic vectors f s and f h;
S244, calculating a contrast loss L by using a contrast loss function formula based on the Manhattan distance d i between the feature vectors f s and f h and the labels y i of the corresponding samples;
S245, calculating the gradient of the contrast loss function with respect to the similarity calculation model parameters by using a back propagation algorithm, and updating the parameters of the similarity calculation model by using a gradient descent method;
S246, repeating the steps S242 to S245 until all samples in the training set are traversed, or N training rounds are completed;
s247, evaluating the performance of the current similarity calculation model by using the verification set;
s248, adjusting the super parameters of the similarity calculation model according to the verification result.
6. The intelligent construction safety precaution method based on image data according to claim 5, wherein step S245 comprises:
(1) From the contrast loss L, the eigenvectors f s and f h, the Manhattan distance d i, the sample tag y i, and the sample i weight w i, the gradient of the loss function with respect to the eigenvectors f s and f h is calculated 、/>The calculation formula is as follows:
Wherein sign represents a sign taking function;
(2) Gradient with respect to eigenvectors f s and f h according to the loss function 、/>Calculating the gradient of the similarity calculation model parameters by using a back propagation algorithm;
(3) And updating the similarity calculation model parameters by adopting a gradient descent method according to the gradient of the similarity calculation model parameters.
7. The intelligent construction site construction safety precaution method based on image data as claimed in claim 6, wherein the step S247 specifically comprises: for each sample in the verification set, predicting by using a similarity calculation model to obtain a prediction result; calculating an accuracy rate P and a recall rate R according to the prediction result and the real label, wherein the formula is as follows:
Wherein TP represents the number of true positives, FP represents the number of false positives, FN represents the number of false negatives;
and averaging the accuracy and recall of all samples to obtain the performance index of the similarity calculation model.
8. The intelligent construction safety precaution method based on image data according to claim 7, characterized in that the preprocessing methods of the image data in steps S3 and S4 are the same, and all include resizing, normalization, enhancement and conversion into tensors.
9. The intelligent building site construction safety precaution method based on image data according to any one of claims 1 to 8, further comprising the steps of acquiring real-time wind speed and wind direction data, setting a wind direction range, a first wind speed data threshold value and a second wind speed data threshold value, wherein the first wind speed data threshold value is larger than the second wind speed data, and sending wind condition precaution to a building site comprehensive management platform when the real-time wind speed and wind direction data in the real-time wind speed and wind direction data is larger than the first wind speed data threshold value or when the real-time wind speed and wind direction data in the real-time wind speed and wind direction data is larger than the second wind speed data threshold value and the real-time wind direction in the real-time wind speed and wind direction data is located in the wind direction range.
10. The intelligent construction site construction safety early warning system based on the image data is characterized by comprising a monitoring camera, a memory and an FPGA DPU unit; the monitoring cameras are distributed in each construction function monitoring area of the construction site and are used for collecting real-time monitoring image data; the intelligent construction safety early warning method based on the image data of any one of claims 1-8 is executed by the FPGA DPU unit through executing the construction safety early warning program.
CN202410660416.XA 2024-05-27 2024-05-27 Intelligent construction site construction safety early warning method and system based on image data Active CN118230261B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410660416.XA CN118230261B (en) 2024-05-27 2024-05-27 Intelligent construction site construction safety early warning method and system based on image data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410660416.XA CN118230261B (en) 2024-05-27 2024-05-27 Intelligent construction site construction safety early warning method and system based on image data

Publications (2)

Publication Number Publication Date
CN118230261A true CN118230261A (en) 2024-06-21
CN118230261B CN118230261B (en) 2024-07-19

Family

ID=91512279

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410660416.XA Active CN118230261B (en) 2024-05-27 2024-05-27 Intelligent construction site construction safety early warning method and system based on image data

Country Status (1)

Country Link
CN (1) CN118230261B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118506290A (en) * 2024-07-19 2024-08-16 贵州交建信息科技有限公司 AI (advanced technology attachment) -recognition-based beam field construction safety quality monitoring method and system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111488804A (en) * 2020-03-19 2020-08-04 山西大学 Labor insurance product wearing condition detection and identity identification method based on deep learning
CN115063730A (en) * 2022-07-13 2022-09-16 山东建筑大学 Video track analysis-based method and system for early warning of intrusion of workers into borderline area
CN116664845A (en) * 2023-07-28 2023-08-29 山东建筑大学 Intelligent engineering image segmentation method and system based on inter-block contrast attention mechanism
CN116797814A (en) * 2022-12-28 2023-09-22 中建新疆建工集团第三建设工程有限公司 Intelligent building site safety management system
CN117768618A (en) * 2023-12-27 2024-03-26 齐丰科技股份有限公司 Method for analyzing personnel violation based on video image
US20240125888A1 (en) * 2021-10-06 2024-04-18 Guozhen Zhu Method, apparatus, and system for wireless human and non-human motion detection

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111488804A (en) * 2020-03-19 2020-08-04 山西大学 Labor insurance product wearing condition detection and identity identification method based on deep learning
US20240125888A1 (en) * 2021-10-06 2024-04-18 Guozhen Zhu Method, apparatus, and system for wireless human and non-human motion detection
CN115063730A (en) * 2022-07-13 2022-09-16 山东建筑大学 Video track analysis-based method and system for early warning of intrusion of workers into borderline area
CN116797814A (en) * 2022-12-28 2023-09-22 中建新疆建工集团第三建设工程有限公司 Intelligent building site safety management system
CN116664845A (en) * 2023-07-28 2023-08-29 山东建筑大学 Intelligent engineering image segmentation method and system based on inter-block contrast attention mechanism
CN117768618A (en) * 2023-12-27 2024-03-26 齐丰科技股份有限公司 Method for analyzing personnel violation based on video image

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118506290A (en) * 2024-07-19 2024-08-16 贵州交建信息科技有限公司 AI (advanced technology attachment) -recognition-based beam field construction safety quality monitoring method and system
CN118506290B (en) * 2024-07-19 2024-10-01 贵州交建信息科技有限公司 AI (advanced technology attachment) -recognition-based beam field construction safety quality monitoring method and system

Also Published As

Publication number Publication date
CN118230261B (en) 2024-07-19

Similar Documents

Publication Publication Date Title
Hamid et al. Causes of fatal construction accidents in Malaysia
CN111083640B (en) Intelligent supervision method and system for construction site
CN118230261B (en) Intelligent construction site construction safety early warning method and system based on image data
CN107146024A (en) The three-dimensional dynamic safety management method of building enterprise's overall process
CN113724105A (en) Building construction site monitoring system and monitoring method thereof
CN114997682B (en) Construction site safety monitoring system and method based on big data
CN102436721B (en) Intelligent early-warning emergency system for wind power safety and method thereof
JP2021149668A (en) Construction site safety management device
CN105956796A (en) Power operation risk assessment method and system
CN211877114U (en) Construction site supervisory system based on 5G
CN117910811B (en) Intelligent fire control management method and system based on multi-mode AI large model
CN117973705B (en) Data processing and analyzing system and method for intelligent building site monitoring system
CN111506011A (en) Construction safety monitoring method and device
CN108761570A (en) A kind of machine level ground thunder and lightning security protection system and method
CN116882722B (en) Intelligent building site management method and system based on Internet of things
CN113506416A (en) Engineering abnormity early warning method and system based on intelligent visual analysis
KR20170081880A (en) Method and apparatus for creating safety management information in shipbuilding
Choi et al. A risk estimation approach based on deep learning in shipbuilding industry
CN116862244A (en) Industrial field vision AI analysis and safety pre-warning system and method
CN112381435A (en) Gridding directional pushing management method for dynamic risk in hydropower station operation process
CN108388873A (en) A kind of supervisory systems of machine vision, method and client computer, storage medium
CN114819419B (en) Emergency command method and system based on big data and readable storage medium
Luo et al. Requirements for autonomous crane safety monitoring
CN113592360B (en) Electric power high-altitude operation strong wind early warning method and system
CN116739343A (en) Safety evaluation method and system for power grid construction project site

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant