WO2024124970A1 - 一种用于复杂环境行为识别的监控装置及监控方法 - Google Patents
一种用于复杂环境行为识别的监控装置及监控方法 Download PDFInfo
- Publication number
- WO2024124970A1 WO2024124970A1 PCT/CN2023/115727 CN2023115727W WO2024124970A1 WO 2024124970 A1 WO2024124970 A1 WO 2024124970A1 CN 2023115727 W CN2023115727 W CN 2023115727W WO 2024124970 A1 WO2024124970 A1 WO 2024124970A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- behavior
- module
- face recognition
- information
- camera
- Prior art date
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 23
- 206010000117 Abnormal behaviour Diseases 0.000 claims abstract description 42
- 230000001815 facial effect Effects 0.000 claims abstract description 13
- 230000006399 behavior Effects 0.000 claims description 198
- 238000012806 monitoring device Methods 0.000 claims description 27
- 238000013527 convolutional neural network Methods 0.000 claims description 21
- 238000012549 training Methods 0.000 claims description 15
- 238000013135 deep learning Methods 0.000 claims description 13
- 230000003542 behavioural effect Effects 0.000 claims description 8
- 230000000391 smoking effect Effects 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000013136 deep learning model Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 230000035945 sensitivity Effects 0.000 claims description 5
- 238000012790 confirmation Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000007689 inspection Methods 0.000 abstract description 6
- 238000010276 construction Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 101000827703 Homo sapiens Polyphosphoinositide phosphatase Proteins 0.000 description 2
- 102100023591 Polyphosphoinositide phosphatase Human genes 0.000 description 2
- 101100012902 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) FIG2 gene Proteins 0.000 description 2
- 101100233916 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) KAR5 gene Proteins 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 1
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
Definitions
- the present invention relates to the field of monitoring and identification technology, and in particular to a monitoring device and a monitoring method for complex environment behavior identification.
- the common monitoring device is a computer connected to a surveillance camera to identify behavior information.
- the computer simulates the motion contour of the object and compares it with the database to achieve behavior recognition.
- the recognition target is clear and the recognition environment is relatively simple, but in a complex environment, the recognition rate is low and the false alarm rate is high, which cannot meet the management requirements.
- the early warning information and display are insufficient, and the real-time warning effect cannot be achieved.
- the present invention provides a monitoring device and a monitoring method for behavior recognition in complex environments, which can solve the current technical problems of lack of automatic behavior recognition function in complex environments, automatic analysis and judgment of unsafe behaviors on site, active voice alarm, and inability to realize intelligent patrol and early warning.
- the present invention provides the following technical solutions:
- a monitoring device for complex environment behavior recognition comprising a supercomputer, a server, a monitoring camera, a face recognition camera, a PC computer, a large display screen, a switch, a behavior recognition module, a face recognition module and an early warning module;
- the monitoring camera is connected to the behavior recognition module
- the face recognition camera is connected to the face recognition module
- the supercomputer is connected to the behavior recognition module, the switch and the early warning module;
- the machine is connected to the switch, and the large display screen is connected to the PC computer;
- the behavior recognition module has built-in behavior recognition algorithms and convolutional neural networks, which are used to make behavior judgments on the video stream transmitted by the surveillance camera, analyze the behaviors in combination with the deep learning model library, and calculate the results;
- the face recognition module is used to perform face recognition comparison based on the face photo data of the face recognition camera, verify the identity information of the person entering the workplace, and link the warning module to identify and warn strangers;
- the early warning module is used to broadcast abnormal behaviors and send display information according to the early warning information of the behavior recognition module and the face recognition module;
- the PC computer is used to obtain the display information of the early warning module through the switch, and control the large display screen to display abnormal behavior of the display information.
- the behavior recognition module includes an abnormal behavior judgment module and an abnormal behavior verification module;
- the abnormal behavior judgment module is used to judge whether there is abnormal behavior in the workplace through the smoking algorithm, the mobile phone playing algorithm, the not wearing a helmet algorithm and the not wearing a seat belt algorithm;
- the abnormal behavior verification module is used to verify false alarm information of the abnormal behavior judgment information output by the abnormal behavior judgment module.
- the monitoring device further includes a storage module and a parameter configuration module
- the storage module is used to receive information transmitted by the switch and store output results of the behavior recognition module and the face recognition module;
- the parameter configuration module is used to adjust the parameters of the face recognition camera and the surveillance camera, and the parameters include angle, focal length, confidence and sensitivity.
- the face recognition camera and the surveillance camera both include infrared sensors.
- the warning module includes a plurality of smart speakers and a display information generation module
- the display information generation module is used to edit text content according to the warning information of the behavior recognition module and the face recognition module to generate broadcast information or display information for warning of abnormal behavior;
- the smart speaker is set in multiple operation areas, and the smart speaker is used to broadcast the broadcast information corresponding to abnormal behavior in each operation area in the form of text-to-speech.
- the comparison result of the face recognition comparison performed by the face recognition module is displayed on the large display screen, and the display information on the large display screen includes name, gender, snapshot photo and work group; when the face recognition module cannot recognize the identity information of the workplace personnel and determines that the person is a stranger, the face recognition module links with the early warning module to identify and warn the stranger, and sends a warning sound to the stranger through the large display screen.
- the convolutional neural network in the behavior recognition module includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer;
- the training steps of the convolutional neural network include: manually calibrating samples of each type of target violation behavior and normal behavior image material samples in a complex environment through samples collected from video images, inputting the image material samples into a deep learning algorithm model to perform classification training on unsafe behaviors, and continuously iterating the training until convergence, outputting an algorithm model that can identify various types of unsafe behaviors as a convolutional neural network.
- the present application also provides a monitoring method for complex environment behavior recognition, comprising the following steps:
- the steps of recording surveillance video and simultaneously identifying unsafe behaviors include:
- the monitoring video recording step wherein the monitoring camera transmits the monitoring video of each working area in the working field to the supercomputer for storage;
- the face recognition database is established by storing the identity information and photos of the personnel in the workplace in the face recognition database, wherein the identity information of the personnel includes name, gender and team to which they belong.
- the face recognition module performs face recognition comparison based on the face photo data of the face recognition camera to verify the identity information of the personnel entering the workplace until all target personnel in the monitored area have been monitored, and displays the comparison results in conjunction with the large display screen;
- the step of recording the motion trajectory is that the supercomputer converts the monitoring video stream into frames of pictures through editing software, performs motion trajectory outlining according to the monitored object in the monitoring video, depicts the motion trajectory, and establishes a behavior model to record the motion trajectory;
- Behavior recognition step wherein the behavior recognition module recognizes unsafe behaviors of operators based on the video stream generated in real time by the monitoring camera, and when unsafe behaviors are detected, the behavior recognition module is linked with the early warning module to issue early warnings by category and region;
- the supercomputer saves the surveillance video, audio and captured photos recorded by the surveillance camera in the form of a compressed package.
- the step of continuously performing behavior judgment and outputting judgment results includes:
- the face recognition camera compares the acquired facial features with the data in the face recognition database one by one to confirm the personnel information, and outputs the face image with the highest degree of conformity in the face recognition database as the person
- the face recognition module compares the personnel information of the person with the person in the workplace, and stores the captured person image in the database. It determines whether to allow entry based on the compared personnel information. If the face recognition module cannot recognize the identity information of the person in the workplace, it determines that the person is a stranger and works in conjunction with the warning module to issue a voice warning.
- the supercomputer monitors in real time through the video stream of the surveillance camera whether the operator has any unsafe behavior.
- the early warning model is linked to make real-time voice warnings by area and category, and the picture of the unsafe behavior is displayed on the large display screen in real time, and the supercomputer switches to the surveillance camera that detects the unsafe behavior, and the picture of the surveillance camera is displayed on the large display screen.
- the present invention is a behavior recognition technology based on scaffolding in a complex environment.
- the built-in behavior recognition algorithm on the supercomputer is a synthetic algorithm, such as a collection of algorithms for smoking, playing with mobile phones, not wearing a helmet, and not wearing a seat belt as an algorithm model, which effectively identifies abnormal behaviors.
- the behavior recognition module is embedded with a behavior recognition algorithm and a convolutional neural network, which is used to make behavioral judgments on the video stream transmitted by the surveillance camera, and analyzes the behavior in combination with a deep learning model library, which can greatly increase the accuracy of behavior recognition.
- the site can provide real-time voice warnings classified by region and category, and display abnormal behaviors such as pictures of unsafe behaviors on the large screen in real time, thereby improving the inspection efficiency of patrol personnel, truly achieving the first-time discovery, disposal and resolution of unsafe behaviors at the construction site, and realizing intelligent recognition of multiple unsafe behaviors in complex environments.
- FIG2 is a flow chart of an identification method for deep learning algorithm model training in a complex environment according to an embodiment of the present invention
- FIG3 is a structural block diagram of a monitoring device for complex environment behavior recognition according to an embodiment of the present invention.
- FIG4 is a flow chart of a monitoring method for complex environment behavior recognition according to an embodiment of the present invention.
- FIG. 5 is a flow chart of the steps of recording surveillance video and simultaneously identifying unsafe behaviors according to an embodiment of the present invention
- Example 1 of the present invention a monitoring device and method based on deep-learning behavior recognition technology are provided, namely, a monitoring device and a monitoring method for complex environment behavior recognition.
- a deep learning algorithm model is constructed, and the algorithm and computing resources are reasonably scheduled to realize real-time collection and intelligent analysis of unstructured data such as videos and pictures, and real-time warnings are linked to broadcast and large-screen.
- the video monitoring of the operation site in the complex scaffolding operation environment is traceable, and real-time monitoring and warning of violations such as not wearing a safety helmet, not tying a safety rope, illegal use, smoking, and using mobile phones are realized. Face recognition is performed on people entering the area, and voice warnings are implemented for strangers.
- FIG. 1 is a principle block diagram of a monitoring device for complex environment behavior recognition of the present invention.
- the monitoring device for complex environment behavior recognition is based on scaffold complex environment behavior recognition technology, which includes a server, a PC computer, a supercomputer, a face recognition camera, a surveillance camera, a behavior recognition module, a face recognition module, an early warning module, and a database.
- the supercomputer receives information obtained by other modules, the behavior recognition module identifies unsafe behaviors, the face recognition module identifies facial features, and the database stores the results of behavior and face recognition.
- Both the face recognition camera and the surveillance camera can record surveillance videos, and the convolutional neural network can quickly process information.
- the supercomputer is connected to a server, a behavior recognition module, a face recognition module and an early warning module, and has an embedded behavior recognition algorithm and a convolutional neural network.
- the behavior recognition module is connected to an intelligent speaker, a surveillance camera, a PC computer, a large display screen and a supercomputer.
- the face recognition modules are all embedded with a convolutional neural network and are connected to a PC computer.
- the face recognition module includes an infrared lens and a fill light.
- the early warning module is connected to an intelligent speaker and a surveillance camera, and supports the recognition of the camera IP address to link the early warning signal.
- the face recognition camera and surveillance camera include infrared sensors, which automatically sense and switch to day mode or night mode.
- Special cameras and surveillance cameras include infrared sensors, automatically sense day and night, automatically filter light spots, and perform deep learning training models based on the postures of people in the environment to reduce false alarms.
- the early warning modules are linked with the corresponding surveillance cameras respectively, and can broadcast different voice information for different abnormal behaviors in different areas.
- the described voice information can be broadcast in the form of MP3 or text-to-speech, and the text content can be edited by oneself.
- the monitoring method for complex environment behavior recognition is used to identify unsafe behaviors of operators, including not wearing a helmet, smoking, playing with mobile phones, not wearing seat belts, etc. Once an operator exhibits unsafe behavior, the warning module is immediately activated, and the intelligent speaker in the corresponding area broadcasts the corresponding unsafe behavior voice, where the voice setting can be broadcast according to the voice or text-to-speech function.
- the unsafe behavior algorithm is built into the supercomputer, which analyzes the video stream of the monitoring camera in real time. Once the confidence of the operator's unsafe behavior reaches the set threshold, a warning event occurs.
- the supercomputer implementation process includes the following specific steps:
- Step 2 Training of deep learning algorithm model in complex environment, as shown in Figure 2, the flow chart of the identification method.
- the deep learning algorithm model marks various unsafe behaviors based on the on-site pictures of the video stream, builds a behavior algorithm model, and embeds the algorithm model into the supercomputer; by adjusting the camera angle, focal length, recognition sensitivity, and accuracy, the recognition effect is optimized.
- the model can be updated and iterated by increasing the data set, becoming more accurate, and will be continuously optimized through learning in the scaffolding scenario, making the recognition model more suitable for the current scenario.
- Step 3 Record surveillance video and identify unsafe behaviors.
- the face recognition module uses a face matching algorithm built into the face recognition camera to add a person's photo to the model database, and add the person's information including name, gender, department, etc. to the model library.
- the face matching step can be repeated until all targets in the monitoring area are fully monitored, and the face matching step is linked to the large display screen.
- the supercomputer converts the surveillance video stream into frames of pictures through editing software, traces the motion trajectory according to the monitored object in the surveillance video, depicts the motion trajectory, and establishes a behavior model;
- Behavior recognition the behavior recognition module recognizes unsafe behaviors of operators based on the video stream generated in real time by the monitoring camera. Once unsafe behaviors are detected, the module will work together with the early warning module to issue early warnings by intelligent speakers in different categories and regions.
- (e) Backup file creation the supercomputer stores the surveillance video, audio and captured photos recorded by the surveillance camera in the form of a compressed package in a storage hard disk;
- Step 4 Behavioral judgment and output.
- Unsafe behavior analysis The supercomputer is embedded with a behavioral algorithm model, which establishes an unsafe behavior data model. It monitors in real time through the video stream of the surveillance camera whether its operators have any unsafe behavior. Once unsafe behavior is detected, the early warning model is immediately linked to provide real-time voice warnings by area and category, and the picture of the unsafe behavior is displayed on the large screen in real time. The system immediately switches to the surveillance camera that detected the unsafe behavior, and the picture of the surveillance camera is displayed on the large screen.
- Figure 2 is a flow chart of the identification method for deep learning algorithm model training in a complex environment.
- the deep learning algorithm model annotates unsafe behaviors based on the on-site pictures of the video stream, builds a behavioral algorithm model, and embeds the algorithm model into the supercomputer; the model can be updated and iterated by increasing the data set, becoming more accurate, and will be more suitable for the current scenario through continuous optimization.
- Convolutional neural networks are constructed based on unsafe behaviors, and the basic parameters of the model are designed and set.
- the generated data sets are then input into the model for training and evaluation.
- samples collected from video images are manually calibrated for each type of target violation behavior and normal behavior image material samples in the complex environment of the scaffolding. All behaviors are manually calibrated, and the current unsafe behaviors are classified and trained using the model algorithm.
- An algorithm model that can identify various types of unsafe behaviors is output. Based on the complex environment of the scaffolding, the algorithm is continuously iterated to adapt to the complex environment of the scaffolding.
- the behavior recognition module is embedded in the supercomputer and performs behavior recognition based on the video stream according to the deep learning algorithm model.
- the continuous practice of unsafe behavior can be identified in 1 second at the fastest setting or can last for 1s-10s according to the scenario.
- the present invention is based on the behavior recognition technology in the complex environment of the scaffold.
- the built-in behavior recognition algorithm on the supercomputer is a synthetic algorithm, that is, smoking, playing with mobile phones, not wearing a helmet and not wearing a seat belt are combined into one algorithm model, saving the GPU resources of the supercomputer.
- Deep learning model training based on the on-site environment of the scaffold can greatly increase the accuracy of behavior recognition.
- the site can issue real-time voice warnings by region and category, and display pictures of the unsafe behavior on the large screen in real time, and immediately switch to the surveillance camera that detected the unsafe behavior.
- the surveillance camera picture is displayed on the large screen, and there is no need to manually switch the surveillance camera picture.
- Example 2 includes all the technical features of Example 1.
- a monitoring device 100 for behavior recognition in complex environments includes a supercomputer 1, a server 2, a monitoring camera 3, a face recognition camera 4, a PC computer 5, a large display screen 6, a switch 7, a behavior recognition module 8, a face recognition module 9 and an early warning module 10;
- the monitoring camera 3 is connected to the behavior recognition module 8
- the face recognition camera 4 is connected to the face recognition module 9
- the supercomputer 1 is connected to the behavior recognition module 8, the switch 7 and the early warning module 10
- the face recognition module 9, the server 2 and the PC computer 5 are connected to the switch 7, and the large display screen 6 is connected to the PC computer 5.
- the behavior recognition module 8 has built-in behavior recognition algorithms and convolutional neural networks, which are used to make behavior judgments on the video stream transmitted by the surveillance camera 3, analyze the behaviors in combination with a deep learning model library, and calculate the results.
- the face recognition module 9 includes a face recognition comparison module 91, which is used to perform face recognition comparison based on the face photo data of the face recognition camera 4, verify the identity information of the person entering the workplace, and link the warning module 10 to identify and warn strangers.
- a face recognition comparison module 91 which is used to perform face recognition comparison based on the face photo data of the face recognition camera 4, verify the identity information of the person entering the workplace, and link the warning module 10 to identify and warn strangers.
- the warning module 10 is used to detect the warning information of the behavior recognition module 8 and the face recognition module 9. Broadcast abnormal behavior and send display information.
- the PC computer 5 is used to obtain the display information of the early warning module 10 through the switch 7, and control the large display screen 6 to display abnormal behavior of the display information.
- the behavior recognition module 8 includes an abnormal behavior judgment module 81 and an abnormal behavior verification module 82 .
- the abnormal behavior judgment module 81 is used to judge whether there is abnormal behavior in the workplace through a smoking algorithm, a mobile phone playing algorithm, a helmet-not-wearing algorithm, and a seatbelt-not-fastening algorithm.
- the abnormal behavior verification module 82 is used to verify the false alarm information of the abnormal behavior judgment information output by the abnormal behavior judgment module 81 .
- the monitoring device 100 further includes a storage module 11 and a parameter configuration module 12;
- the storage module 11 is preferably a database set in the supercomputer 1, or a database set in the server 2.
- the parameter configuration module 12 can be set in the supercomputer 1 or in the server 2, or can be set separately.
- the storage module 11 is used to receive information transmitted by the switch 7 and store output results of the behavior recognition module 8 and the face recognition module 9 .
- the parameter configuration module 12 is used to adjust the parameters of the face recognition camera 4 and the monitoring camera 3, and the parameters include angle, focal length, confidence and sensitivity.
- the face recognition camera 4 and the monitoring camera 3 both include infrared sensors.
- the face recognition camera 4 includes an infrared lens and a fill light.
- the early warning module 10 includes a plurality of smart speakers 101 and a display information generation module 102 .
- the display information generation module 102 is used to edit text content according to the warning information of the behavior recognition module 8 and the face recognition module 9 to generate broadcast information or display information for warning of abnormal behavior.
- the smart speaker 101 is set in multiple operation areas, and the smart speaker 101 is used to broadcast the broadcast information corresponding to the abnormal behavior in each operation area in the form of text-to-speech.
- the face recognition module 9 performs face recognition comparison and the comparison result is displayed on the large display screen 6, and the display information on the large display screen 6 includes name, gender, snapshot photo and team; when the face recognition module 9 cannot identify the identity information of the workplace personnel and determines that the person is a stranger, the face recognition module 9 links the warning module 10 to identify and warn the stranger, and the stranger passes the large display screen 6 Sound a warning.
- the convolutional neural network in the behavior recognition module 8 includes an input layer, a convolution layer, a pooling layer, a full-link layer, and an output layer;
- the training steps of the convolutional neural network include: manually calibrating the image material samples of each type of target violation behavior and normal behavior in a complex environment through samples collected from video images, inputting the image material samples into the deep learning algorithm model to perform classification training on unsafe behaviors, and continuously iterating the training until convergence, outputting an algorithm model that can identify various types of unsafe behaviors as a convolutional neural network.
- the present application also provides a monitoring method for complex environment behavior recognition, comprising the following steps:
- the steps of recording surveillance video and simultaneously identifying unsafe behaviors include:
- the monitoring camera 3 transmits the monitoring video of each working area in the working field to the supercomputer 1 and stores it;
- S32 a step of establishing a face recognition database, storing the identity information of the personnel in the workplace and the photos of the personnel in the face recognition database, wherein the identity information of the personnel includes name, gender and team, and the face recognition module 9 performs face recognition comparison based on the face photo data of the face recognition camera 4 to verify the identity information of the personnel entering the workplace until all target personnel in the monitored area are monitored, and displays the comparison results in conjunction with the display screen 6;
- S33 a step of recording a motion trajectory
- the supercomputer 1 converts the surveillance video stream into frames of pictures through editing software, performs motion trajectory outlining according to the monitored object in the surveillance video, depicts the motion trajectory, and establishes a behavior model to record the motion trajectory;
- the behavior recognition module 8 recognizes the unsafe behavior of the operator according to the video stream generated by the monitoring camera in real time, and when unsafe behavior is detected, it is linked with the early warning module 10 to classify and warn in different areas;
- the supercomputer 1 saves the surveillance video, audio and captured photos recorded by the surveillance camera 3 in the form of a compressed package.
- the step of continuously performing behavior judgment and outputting judgment results includes:
- the face recognition camera 4 compares the acquired facial features with the data in the face recognition database one by one to confirm the personnel information, outputs the face image with the highest degree of conformity in the face recognition database as the personnel information result of the personnel comparison, and stores the captured personnel image in the database, and determines whether to allow entry based on the compared personnel information result. If the face recognition module 9 cannot recognize the identity information of the workplace personnel, it is determined to be a stranger, and it is linked with the warning module 10 to issue a voice warning;
- unsafe behavior analysis step the supercomputer 1 monitors in real time whether the operator has any unsafe behavior through the video stream of the monitoring camera.
- the early warning model is linked to make real-time voice warnings by area and category, and the picture of the unsafe behavior is displayed on the large display screen 6 in real time, and the supercomputer 1 switches to the monitoring camera 3 that detects the unsafe behavior, and the picture of the monitoring camera 3 is displayed on the large display screen 6.
- the present invention is a behavior recognition technology based on scaffolding in a complex environment.
- the built-in behavior recognition algorithm on the supercomputer is a synthetic algorithm, such as a collection of algorithms for smoking, playing with mobile phones, not wearing a helmet and not wearing a seat belt as an algorithm model, which effectively identifies abnormal behaviors.
- the behavior recognition module is embedded with a behavior recognition algorithm and a convolutional neural network, which is used to make behavioral judgments on the video stream transmitted by the surveillance camera, and analyzes the behavior in combination with a deep learning model library, which can greatly increase the accuracy of behavior recognition.
- the site can provide real-time voice warnings classified by region and category, and display abnormal behaviors such as pictures of unsafe behaviors on the large screen in real time, thereby improving the inspection efficiency of patrol personnel, truly achieving the first-time discovery, disposal and resolution of unsafe behaviors at the construction site, and realizing intelligent recognition of multiple unsafe behaviors in complex environments.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Social Psychology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Psychiatry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Alarm Systems (AREA)
Abstract
本发明公开了一种用于复杂环境行为识别的监控装置及监控方法，所述装置包括超级计算机、服务器、监控摄像头、人脸识别摄像机、PC计算机、显示大屏、交换机、行为识别模块、人脸识别模块和预警模块。所述人脸识别模块用于根据所述人脸识别摄像机的人脸照片数据进行人脸识别对比，验证进入作业场所人员身份信息，并联动所述预警模块对陌生人进行识别预警；所述预警模块用于根据所述行为识别模块和所述人脸识别模块的预警信息对异常行为进行广播并发送展示信息；所述PC计算机用于通过所述交换机获取所述预警模块的展示信息，并控制所述显示大屏对展示信息进行异常行为展示。当出现不安全行为时，实时分区域分类别的语音预警，提高巡视效率。
Description
本发明涉及监控识别技术领域，特别是一种用于复杂环境行为识别的监控装置及监控方法。
最近几年随着社会经济的飞速发展，现场安全管控工作仅仅依靠文件要求、现场检查等手段来提高作业现场安全管理水平，但对于复杂的施工现场“点多、面广、周期不定”的特点，不能全过程对作业现场进行监督，同时还需要耗费大批人力物力。复杂施工现场的不安全行为识别越来越被重视，脚手架内涉及高处作业、吊装作业、动火作业等多种危险作业，且可能存在交叉作业，风险较高，脚手架空间狭窄，结构复杂，光线暗且光照不均，管理难度高，迫切需要一种能智能识别异常行为的技术手段，常见的监控装置是计算机与监控摄像头连接以识别行为信息，计算机通过模拟物体的运动轮廓，与数据库将进行对比实现行为识别，识别目标明确识别环境相对简单，但在复杂的环境下识别率低误报率高，无法满足管理要求。同时当施工现场人员发生不安全行为时，其预警信息和展示不足，不能达到实时警示的效果。亟需基于复杂环境下人工智能技术，实现行为识别功能，对现场不安全行为自动研判、主动语音报警，实现了智能巡检预警的突破。
发明内容
为了克服现有技术的不足，本发明提供一种用于复杂环境行为识别的监控装置及监控方法，能够解决目前缺少基于复杂环境下实现行为自动识别功能，对现场不安全行为自动研判、主动语音报警，无法实现智能巡检预警的技术问题。
为实现上述目的，本发明提供如下技术方案：
在本申请一实施例中，提供一种用于复杂环境行为识别的监控装置，所述监控装置包括超级计算机、服务器、监控摄像头、人脸识别摄像机、PC计算机、显示大屏、交换机、行为识别模块、人脸识别模块和预警模块；所述监控摄像头连接至所述行为识别模块，所述人脸识别摄像机连接至所述人脸识别模块，所述超级计算机连接至所述行为识别模块、所述交换机和所述预警模块；所述人脸识别模块、所述服务器和所述PC计算
机连接至所述交换机，所述显示大屏连接至所述PC计算机；
所述行为识别模块内嵌行为识别算法和卷积神经网络，用于对所述监控摄像头传输的视频流进行行为判断，结合深度学习模型库对行为进行分析，计算结果；
所述人脸识别模块用于根据所述人脸识别摄像机的人脸照片数据进行人脸识别对比，验证进入作业场所人员身份信息，并联动所述预警模块对陌生人进行识别预警；
所述预警模块用于根据所述行为识别模块和所述人脸识别模块的预警信息对异常行为进行广播并发送展示信息；
所述PC计算机用于通过所述交换机获取所述预警模块的展示信息，并控制所述显示大屏对展示信息进行异常行为展示。
在本实施例中，所述行为识别模块包括异常行为判断模块和异常行为验证模块；
所述异常行为判断模块用于通过抽烟算法、玩手机算法、不戴安全帽算法和不系安全带算法判断是否在作业场内存在异常行为；
所述异常行为验证模块用于对所述异常行为判断模块输出的异常行为判断信息进行误报信息核对验证。
在本实施例中，所述监控装置还包括存储模块和参数配置模块；
所述存储模块用于接收所述交换机传输的信息，并存储所述行为识别模块和所述人脸识别模块的输出结果；
所述参数配置模块用于对所述人脸识别摄像机和所述监控摄像头的参数进行调整，所述参数包括角度、焦距、置信度和灵敏度。
在本实施例中，所述人脸识别摄像机和所述监控摄像头均包括红外线传感器。
在本实施例中，所述预警模块包括多个智能音箱和展示信息生成模块；
所述展示信息生成模块用于根据所述行为识别模块和所述人脸识别模块的预警信息自行编辑文字内容生成对异常行为进行预警的广播信息或展示信息；
所述智能音箱设置在多个作业区域内，所述智能音箱用于对每一作业区域内的异常行为对应的所述广播信息通过文字转语音的形式进行播报。
在本实施例中，所述人脸识别模块进行人脸识别对比的比对结果显示在所述显示大屏上，所述显示大屏上的展示信息包括姓名、性别、抓拍照片和所属班组；当所述人脸识别模块无法识别出为作业场所人员身份信息时判定为陌生人，所述人脸识别模块联动所述预警模块对陌生人进行识别预警，并对陌生人通过所述显示大屏发出预警声音。
在本实施例中，所述行为识别模块内的卷积神经网络包括输入层、卷积层、池化层、全链接层和输出层；
所述卷积神经网络的训练步骤包括：通过视频图像收集的样本，对复杂环境下每类目标违章行为和正常行为图像素材样本进行人工标定，将图像素材样本输入至深度学习算法模型中进行对不安全行为分类训练，通过不断迭代训练直至收敛，输出可识别各类不安全行为的算法模型作为卷积神经网络。
本申请还提供一种用于复杂环境行为识别的监控方法，包括以下步骤：
设置前文所述的用于复杂环境行为识别的监控装置；
训练在复杂环境下可识别各类不安全行为的卷积神经网络；
录制监控视频并同步对不安全行为进行识别；
持续进行行为判断并输出判断结果。
在本实施例中，所述录制监控视频并同步对不安全行为进行识别步骤，包括：
录制监控视频步骤，所述监控摄像头把作业场内每一作业区域内的监控视频分别传输至所述超级计算机内并存储；
人脸识别数据库的建立步骤，将作业场所人员身份信息及人员的照片存储在人脸识别数据库内，所述人员身份信息包括姓名、性别和所属班组，所述人脸识别模块根据所述人脸识别摄像机的人脸照片数据进行人脸识别对比，验证进入作业场所人员身份信息，直至被监测区域内所有目标人员被监测完为止，并与所述显示大屏联动展示对比结果；
记录运动轨迹步骤，所述超级计算机通过剪辑软件将监控视频流转成一帧帧的图片，根据所述监控视频中的监测对象进行运动轨迹描边，描绘出运动轨迹，建立行为模型记录运动轨迹；
行为识别步骤，所述行为识别模块根据所述监控摄像机实时产生的视频流识别作业人员的不安全行为，当监测到不安全行为时，与所述预警模块进行联动，分类分区域预警；
备份文件建立步骤，所述超级计算机将监控摄像头录制的监控视频、音频以及抓拍到的照片以压缩包的形式进行保存。
在本实施例中，所述持续进行行为判断并输出判断结果步骤，包括：
人员信息确认步骤，所述人脸识别摄像机把获取的脸部特征与人脸识别数据库内的数据逐一比对确认人员信息，输出所述人脸识别数据库中符合度最高的人脸图像作为人
员比对的人员信息结果，并把抓拍到的人员图片存储进数据库中，根据比对出的人员信息结果确定是否准许进入，如果所述人脸识别模块无法识别出为作业场所人员身份信息时判定为陌生人，并与所述预警模块联动，进行语音预警；
不安全行为分析步骤，所述超级计算机通过监控摄像机的视频流实时监测作业人员是否存在不安全行为，当监测到不安全行为时，联动所述预警模型，分区域分类别进行实时语音预警，并在显示大屏上实时展示不安全行为的图片，且切换到监测到不安全行为的监控摄像头，该监控摄像头画面在所述显示大屏上展示。
本发明的有益效果是：
本发明是基于脚手架的复杂环境下的行为识别技术，在超级计算机上内置行为识别算法为合成算法，如抽烟、玩手机、不戴安全帽和不系安全带算法的集合为一个算法模型，有效识别异常行为。所述行为识别模块内嵌行为识别算法和卷积神经网络，用于对所述监控摄像头传输的视频流进行行为判断，结合深度学习模型库对行为进行分析，可大大增加行为识别的准确率。当施工现场出现不安全行为时，现场可实时分区域分类别的语音预警，且在显示大屏上实时展示不安全行为的图片等异常行为，提高巡检人员的巡视效率，真正做到施工现场不安全行为第一时间发现、处置并解决，实现了在复杂环境下的多种不安全行为的智能识别。
为了更清楚地说明本发明实施的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例的用于复杂环境行为识别的监控装置的原理框图；
图2为本发明实施例的在复杂环境下的深度学习算法模型训练的标识方法流程图；
图3为本发明实施例的用于复杂环境行为识别的监控装置的结构框图；
图4为本发明实施例的用于复杂环境行为识别的监控方法的流程图；
图5为本发明实施例的录制监控视频并同步对不安全行为进行识别步骤的流程图；
图6为本发明实施例的持续进行行为判断并输出判断结果步骤的流程图。
下面结合附图和实施例，对本发明的具体实施方式作进一步描述。以下实施例仅用于更加清楚地说明本发明的技术方案，而不能以此来限制本发明的保护范围。
实施例1
本发明实施例1中提供一种基于可深度学习的行为识别技术的监控装置及方法，即用于复杂环境行为识别的监控装置及监控方法。根据脚手架作业区域应用需求，构建深度学习算法模型，合理调度算法、算力资源，实现对视频、图片等非结构化数据进行实时采集与智能分析，联动广播、大屏实时预警。实现在脚手架复杂作业环境中对作业现场的视频监控可追溯，实现了未戴安全帽、未系安全绳及违规使用、吸烟、使用手机等违规行为的实时监测预警，并对进入区域的人员进行人脸识别，针对陌生人实现语音预警。
如图1所示，图1为本发明的用于复杂环境行为识别的监控装置的原理框图，所述用于复杂环境行为识别的监控装置基于脚手架复杂环境行为识别技术，其包括服务器、PC计算机、超级计算机、人脸识别摄像机、监控摄像头、行为识别模块、人脸识别模块、预警模块、数据库，超级计算机、行为识别模块、人脸识别模块，超级计算机接收其他模块得到的信息，行为识别模块识别不安全行为，人脸识别模块识别人脸特征，数据库存储行为和人脸识别的结果，人脸识别摄像机和监控摄像头均可录制监控视频，卷积神经网络可快速处理信息，超级计算机通过根据监控摄像头的监控画面进行算法比对以识别人员行为结果判断是否存在不安全行为，一旦确认为不安全行为，与预警模块联动，在对应区域发出对应异常行为的警示声音，并在显示大屏上展示不安全行为预警信息，且其监控摄像头将自动跳转至预警的监控摄像头画面。
所述超级计算机连接有服务器、行为识别模块、人脸识别模块和预警模块，并内嵌行为识别算法和卷积神经网络，所述行为识别模块连接有智能音响、监控摄像头、PC计算机、显示大屏和超级计算机，所述人脸识别模块均内嵌卷积神经网络，并连接PC计算机，所述人脸识别模块包括红外镜头和补光灯，所述预警模块连接智能音响和监控摄像头，支持识别摄像头IP地址联动预警信号。
所述人脸识别摄像机和监控摄像头包括红外线传感器，自动感应并切换白天模式或晚上模式。针对脚手架环境复杂，空间狭窄，光线暗且光照不均匀的特点，所述人脸识
别摄像机和监控摄像头包括红外线传感器，自动感应白天和夜晚，自动过滤光斑，基于环境内人员姿态进行深度学习训练模型，减少误报。
所述预警模块分别与对应监控摄像头进行联动，可分区域进行对不同的异常行为分别播报不同的语音信息，所描述的语音信息可先择使用mp3等格式播报或者通过文字转语音的形式进行播报，其文字内容可自行编辑。
所述人脸识别摄像机的人脸识别模块在摄像机内，人脸比对结果可在显示大屏上显示，展示信息可包括姓名、性别、抓拍照片、班组等内容，当人脸比对出陌生人通过大屏幕发出预警声音。
在脚手架狭窄空间内通过调整摄像机角度、焦距、置信度、灵敏度等参数调整，增加识别效率，减少误报，以达到最佳识别效果。
所述用于复杂环境行为识别的监控方法用于识别作业人员的不安全行为，包括不戴安全帽、抽烟、玩手机、不系安全带等，一旦作业人员出现不安全行为的情况则立即启动预警模块，对应区域的智能音响则播报对应的不安全行为语音，其中语音设定可根据语音或者文字转语音的功能进行播报。不安全行为算法内置于超级电脑内，实时分析监测摄像机的视频流，当作业人员的不安全行为的置信度一旦达到设定的阈值则发生预警事件。
具体的，所述超级计算机实施过程具体步骤如下：
第一步：初始化监控设备，若干监控摄像头、一台人脸识别摄像头、一台超级计算机、交换机、PC计算机、显示大屏、若干智能音响组成了一整套监控系统，智能音响根据多区域进行划分，人脸识别摄像头设置于入口处，根据入口量设置人员识别摄像机数。
第二步：复杂环境下的深度学习算法模型的训练，如图2所示，标识方法流程图。深度学习的算法模型基于视频流的现场图片对各类不安全行为进行标注，构建出行为算法模型，并将算法模型内嵌至超级计算机内；通过调整摄像机角度、焦距、识别灵敏度、置性度等让识别效果达到最佳，模型可通过数据集的增加进行更新迭代，变得更加准确，会通过脚手架场景下不断的学习优化，使识别模型更加适用于当前的场景。
第三步：录制监控视频及不安全行为识别。
(a)录制监控视频,所述监控摄像头设置在脚手架不同层和不同通道的位置录制监控视频，所述监控摄像头把所述监控视频分别传输给所述超级计算机内，所述超级计算
机会把该监控摄像头录制的视频存储到超级计算机的存储硬盘中；
(b)人脸识别数据库的建立，所述人脸识别模块在人脸识别摄像机内置的人脸比对算法，将人员的照片加入到模型数据库内，并将人员的信息包括姓名、性别、部门等添加进模型库，人脸比对的步骤可重复至监测区域内所有目标数监测完全为止，并与显示大屏联动；
(c)记录运动轨迹，所述超级计算机通过剪辑软件将监控视频流转成一帧帧的图片，根据所述监控视频中的监测对象进行运动轨迹描边，描绘出运动轨迹，建立行为模型；
(d)行为识别，所述行为识别模块根据所述监控摄像机实时产生的视频流识别作业人员的不安全行为，一旦识别监测到不安全行为，则与预警模块进行联动，通过智能音响分类分区域预警；
(e)备份文件建立：所述超级计算机将监控摄像头录制的监控视频、音频以及抓拍到的照片以压缩包的形式存入存储硬盘内；
第四步：行为判断及输出。
(a)人员信息确认：所述人脸识别摄像机把所述脸部特征与人脸识别数据库内的数据逐一比对确认人员信息，所述人脸识别数据库中最符合的人脸图像作为人员比对的人员信息结果，并把抓拍到的人员图片存储进数据库中，根据比对出的人员信息结果确定是否准许进入，如果比对未成功则与所述预警模块联动，进行语音预警。
(b)不安全行为分析：所述超级计算机内嵌有行为算法模型，算法模型建立起不安全行为数据模型，通过监控摄像机的视频流实时监测其作业人员是否存在不安全行为，一旦监测到不安全行为，则立即联动预警模型，分区域分类别进行实时语音预警，并在显示大屏上实时展示不安全行为的图片，且立即切换到监测到不安全行为的监控摄像头，该监控摄像头画面在大屏上展示。
如图2所示，图2是在复杂环境下的深度学习算法模型训练的标识方法流程图。深度学习的算法模型基于视频流的现场图片对其不安全行为进行标注，构建出行为算法模型，并将算法模型内嵌至超级计算机内；模型可通过数据集的增加进行更新迭代，变得更加准确，会通过不断的优化，更加适用于当前的场景。
基于不安全行为构建卷积神经网络，并设计和设置模型的基础参数，并将生成的数据集输入模型中进行训练和评估。首先通过视频图像收集的样本，针对在脚手架的复杂环境下每类目标违章行为和正常行为图像素材样本进行人工标定，对素材样本中的每种
行为均进行手动标定，利用模型算法将目前不安全行为分类训练，输出可识别各类不安全行为的算法模型，基于脚手架复杂环境，通过算法不断迭代以适应脚手架的复杂环境。
行为识别模块嵌入在超级计算机内，根据深度学习的算法模型基于视频流进行行为识别，不安全行为的持续实践最快设定可1s识别或持续1s-10s均可根据场景进行设定。
本发明的有益效果是：
本发明是基于脚手架的复杂环境下的行为识别技术，其超级计算机上内置行为识别算法为合成算法，即抽烟、玩手机、不戴安全帽和不系安全带集合为一个算法模型，节省了超级计算机的GPU资源。根据脚手架的现场环境进行深度学习模型训练，可大大增加行为识别的准确率。
当施工现场出现不安全行为时，现场可实时分区域分类别的语音预警，且在显示大屏上实时展示不安全行为的图片，且立即切换到监测到不安全行为的监控摄像头，该监控摄像头画面在大屏上展示，无需手动切换监控摄像头画面。提高巡检人员的巡视效率，真正做到施工现场不安全行为第一时间发现、处置并解决，利用深度学习方法，实现了在复杂环境下的多种不安全行为的智能识别。
实施例2
基于与实施例1相同的发明构思，在实施例2中包含实施例1的全部技术特征。
如图1、图3所示，在本实施例中，提供一种用于复杂环境行为识别的监控装置100，所述监控装置100包括超级计算机1、服务器2、监控摄像头3、人脸识别摄像机4、PC计算机5、显示大屏6、交换机7、行为识别模块8、人脸识别模块9和预警模块10；所述监控摄像头3连接至所述行为识别模块8，所述人脸识别摄像机4连接至所述人脸识别模块9，所述超级计算机1连接至所述行为识别模块8、所述交换机7和所述预警模块10；所述人脸识别模块9、所述服务器2和所述PC计算机5连接至所述交换机7，所述显示大屏6连接至所述PC计算机5。
所述行为识别模块8内嵌行为识别算法和卷积神经网络，用于对所述监控摄像头3传输的视频流进行行为判断，结合深度学习模型库对行为进行分析，计算结果。
所述人脸识别模块9包括人脸识别对比模块91，用于根据所述人脸识别摄像机4的人脸照片数据进行人脸识别对比，验证进入作业场所人员身份信息，并联动所述预警模块10对陌生人进行识别预警。
所述预警模块10用于根据所述行为识别模块8和所述人脸识别模块9的预警信息
对异常行为进行广播并发送展示信息。
所述PC计算机5用于通过所述交换机7获取所述预警模块10的展示信息，并控制所述显示大屏6对展示信息进行异常行为展示。
如图3所示，在本实施例中，所述行为识别模块8包括异常行为判断模块81和异常行为验证模块82。
所述异常行为判断模块81用于通过抽烟算法、玩手机算法、不戴安全帽算法和不系安全带算法判断是否在作业场内存在异常行为。
所述异常行为验证模块82用于对所述异常行为判断模块81输出的异常行为判断信息进行误报信息核对验证。
如图3所示，在本实施例中，所述监控装置100还包括存储模块11和参数配置模块12；所述存储模块11优选为设置在超级计算机1中的数据库，或者是设置在服务器2中的数据库。所述参数配置模块12可设置在超级计算机1中或设置在服务器2中，也可单独设置。
所述存储模块11用于接收所述交换机7传输的信息，并存储所述行为识别模块8和所述人脸识别模块9的输出结果。
所述参数配置模块12用于对所述人脸识别摄像机4和所述监控摄像头3的参数进行调整，所述参数包括角度、焦距、置信度和灵敏度。
在本实施例中，所述人脸识别摄像机4和所述监控摄像头3均包括红外线传感器。为了适应复杂环境光线不好的情况，所述人脸识别摄像机4包括红外镜头和补光灯。
如图3所示，在本实施例中，所述预警模块10包括多个智能音箱101和展示信息生成模块102。
所述展示信息生成模块102用于根据所述行为识别模块8和所述人脸识别模块9的预警信息自行编辑文字内容生成对异常行为进行预警的广播信息或展示信息。
所述智能音箱101设置在多个作业区域内，所述智能音箱101用于对每一作业区域内的异常行为对应的所述广播信息通过文字转语音的形式进行播报。
在本实施例中，所述人脸识别模块9进行人脸识别对比的比对结果显示在所述显示大屏6上，所述显示大屏6上的展示信息包括姓名、性别、抓拍照片和所属班组；当所述人脸识别模块9无法识别出为作业场所人员身份信息时判定为陌生人，所述人脸识别模块9联动所述预警模块10对陌生人进行识别预警，并对陌生人通过所述显示大屏6
发出预警声音。
如图2所示，在本实施例中，所述行为识别模块8内的卷积神经网络包括输入层、卷积层、池化层、全链接层和输出层；
如图2所示，所述卷积神经网络的训练步骤包括：通过视频图像收集的样本，对复杂环境下每类目标违章行为和正常行为图像素材样本进行人工标定，将图像素材样本输入至深度学习算法模型中进行对不安全行为分类训练，通过不断迭代训练直至收敛，输出可识别各类不安全行为的算法模型作为卷积神经网络。
如图4所示，本申请还提供一种用于复杂环境行为识别的监控方法，包括以下步骤：
S1、设置前文所述的用于复杂环境行为识别的监控装置100；
S2、训练在复杂环境下可识别各类不安全行为的卷积神经网络；
S3、录制监控视频并同步对不安全行为进行识别；
S4、持续进行行为判断并输出判断结果。
如图5所示，在本实施例中，所述录制监控视频并同步对不安全行为进行识别步骤，包括：
S31、录制监控视频步骤，所述监控摄像头3把作业场内每一作业区域内的监控视频分别传输至所述超级计算机1内并存储；
S32、人脸识别数据库的建立步骤，将作业场所人员身份信息及人员的照片存储在人脸识别数据库内，所述人员身份信息包括姓名、性别和所属班组，所述人脸识别模块9根据所述人脸识别摄像机4的人脸照片数据进行人脸识别对比，验证进入作业场所人员身份信息，直至被监测区域内所有目标人员被监测完为止，并与所述显示大屏6联动展示对比结果；
S33、记录运动轨迹步骤，所述超级计算机1通过剪辑软件将监控视频流转成一帧帧的图片，根据所述监控视频中的监测对象进行运动轨迹描边，描绘出运动轨迹，建立行为模型记录运动轨迹；
S34、行为识别步骤，所述行为识别模块8根据所述监控摄像机实时产生的视频流识别作业人员的不安全行为，当监测到不安全行为时，与所述预警模块10进行联动，分类分区域预警；
S35、备份文件建立步骤，所述超级计算机1将监控摄像头3录制的监控视频、音频以及抓拍到的照片以压缩包的形式进行保存。
如图6所示，在本实施例中，所述持续进行行为判断并输出判断结果步骤，包括：
S41、人员信息确认步骤，所述人脸识别摄像机4把获取的脸部特征与人脸识别数据库内的数据逐一比对确认人员信息，输出所述人脸识别数据库中符合度最高的人脸图像作为人员比对的人员信息结果，并把抓拍到的人员图片存储进数据库中，根据比对出的人员信息结果确定是否准许进入，如果所述人脸识别模块9无法识别出为作业场所人员身份信息时判定为陌生人，并与所述预警模块10联动，进行语音预警；
S42、不安全行为分析步骤，所述超级计算机1通过监控摄像机的视频流实时监测作业人员是否存在不安全行为，当监测到不安全行为时，联动所述预警模型，分区域分类别进行实时语音预警，并在显示大屏6上实时展示不安全行为的图片，且切换到监测到不安全行为的监控摄像头3，该监控摄像头3画面在所述显示大屏6上展示。
本发明的有益效果是：
本发明是基于脚手架的复杂环境下的行为识别技术，在超级计算机上内置行为识别算法为合成算法，如抽烟、玩手机、不戴安全帽和不系安全带算法的集合为一个算法模型，有效识别异常行为。所述行为识别模块内嵌行为识别算法和卷积神经网络，用于对所述监控摄像头传输的视频流进行行为判断，结合深度学习模型库对行为进行分析，可大大增加行为识别的准确率。当施工现场出现不安全行为时，现场可实时分区域分类别的语音预警，且在显示大屏上实时展示不安全行为的图片等异常行为，提高巡检人员的巡视效率，真正做到施工现场不安全行为第一时间发现、处置并解决，实现了在复杂环境下的多种不安全行为的智能识别。
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明技术原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。
Claims (10)
- 一种用于复杂环境行为识别的监控装置，其特征在于：所述监控装置包括超级计算机、服务器、监控摄像头、人脸识别摄像机、PC计算机、显示大屏、交换机、行为识别模块、人脸识别模块和预警模块；所述监控摄像头连接至所述行为识别模块，所述人脸识别摄像机连接至所述人脸识别模块，所述超级计算机连接至所述行为识别模块、所述交换机和所述预警模块；所述人脸识别模块、所述服务器和所述PC计算机连接至所述交换机，所述显示大屏连接至所述PC计算机；所述行为识别模块内嵌行为识别算法和卷积神经网络，用于对所述监控摄像头传输的视频流进行行为判断，结合深度学习模型库对行为进行分析，计算结果；所述人脸识别模块用于根据所述人脸识别摄像机的人脸照片数据进行人脸识别对比，验证进入作业场所人员身份信息，并联动所述预警模块对陌生人进行识别预警；所述预警模块用于根据所述行为识别模块和所述人脸识别模块的预警信息对异常行为进行广播并发送展示信息；所述PC计算机用于通过所述交换机获取所述预警模块的展示信息，并控制所述显示大屏对展示信息进行异常行为展示。
- 根据权利要求1所述的一种用于复杂环境行为识别的监控装置，其特征在于：所述行为识别模块包括异常行为判断模块和异常行为验证模块；所述异常行为判断模块用于通过抽烟算法、玩手机算法、不戴安全帽算法和不系安全带算法判断是否在作业场内存在异常行为；所述异常行为验证模块用于对所述异常行为判断模块输出的异常行为判断信息进行误报信息核对验证。
- 根据权利要求1所述的一种用于复杂环境行为识别的监控装置，其特征在于：所述监控装置还包括存储模块和参数配置模块；所述存储模块用于接收所述交换机传输的信息，并存储所述行为识别模块和所述人脸识别模块的输出结果；所述参数配置模块用于对所述人脸识别摄像机和所述监控摄像头的参数进行调整，所述参数包括角度、焦距、置信度和灵敏度。
- 根据权利要求1所述的一种用于复杂环境行为识别的监控装置，其特征在于：所述人脸识别摄像机和所述监控摄像头均包括红外线传感器。
- 根据权利要求1所述的一种用于复杂环境行为识别的监控装置，其特征在于：所述 预警模块包括多个智能音箱和展示信息生成模块；所述展示信息生成模块用于根据所述行为识别模块和所述人脸识别模块的预警信息自行编辑文字内容生成对异常行为进行预警的广播信息或展示信息；所述智能音箱设置在多个作业区域内，所述智能音箱用于对每一作业区域内的异常行为对应的所述广播信息通过文字转语音的形式进行播报。
- 根据权利要求1所述的一种用于复杂环境行为识别的监控装置，其特征在于：所述人脸识别模块进行人脸识别对比的比对结果显示在所述显示大屏上，所述显示大屏上的展示信息包括姓名、性别、抓拍照片和所属班组；当所述人脸识别模块无法识别出为作业场所人员身份信息时判定为陌生人，所述人脸识别模块联动所述预警模块对陌生人进行识别预警，并对陌生人通过所述显示大屏发出预警声音。
- 根据权利要求1所述的一种用于复杂环境行为识别的监控装置，其特征在于：所述行为识别模块内的卷积神经网络包括输入层、卷积层、池化层、全链接层和输出层；所述卷积神经网络的训练步骤包括：通过视频图像收集的样本，对复杂环境下每类目标违章行为和正常行为图像素材样本进行人工标定，将图像素材样本输入至深度学习算法模型中进行对不安全行为分类训练，通过不断迭代训练直至收敛，输出可识别各类不安全行为的算法模型作为卷积神经网络。
- 一种用于复杂环境行为识别的监控方法，其特征在于，包括：设置权利要求1至7任一项所述的用于复杂环境行为识别的监控装置；训练在复杂环境下可识别各类不安全行为的卷积神经网络；录制监控视频并同步对不安全行为进行识别；持续进行行为判断并输出判断结果。
- 根据权利要求8所述的一种用于复杂环境行为识别的监控方法，其特征在于，所述录制监控视频并同步对不安全行为进行识别步骤，包括：录制监控视频步骤，所述监控摄像头把作业场内每一作业区域内的监控视频分别传输至所述超级计算机内并存储；人脸识别数据库的建立步骤，将作业场所人员身份信息及人员的照片存储在人脸识别数据库内，所述人员身份信息包括姓名、性别和所属班组，所述人脸识别模块根据所述人脸识别摄像机的人脸照片数据进行人脸识别对比，验证进入作业场所人员身份信息，直至被监测区域内所有目标人员被监测完为止，并与所述显示大屏联动展示对比结果；记录运动轨迹步骤，所述超级计算机通过剪辑软件将监控视频流转成一帧帧的图片，根据所述监控视频中的监测对象进行运动轨迹描边，描绘出运动轨迹，建立行为模型记录运动轨迹；行为识别步骤，所述行为识别模块根据所述监控摄像机实时产生的视频流识别作业人员的不安全行为，当监测到不安全行为时，与所述预警模块进行联动，分类分区域预警；备份文件建立步骤，所述超级计算机将监控摄像头录制的监控视频、音频以及抓拍到的照片以压缩包的形式进行保存。
- 根据权利要求8所述的一种用于复杂环境行为识别的监控方法，其特征在于，所述持续进行行为判断并输出判断结果步骤，包括：人员信息确认步骤，所述人脸识别摄像机把获取的脸部特征与人脸识别数据库内的数据逐一比对确认人员信息，输出所述人脸识别数据库中符合度最高的人脸图像作为人员比对的人员信息结果，并把抓拍到的人员图片存储进数据库中，根据比对出的人员信息结果确定是否准许进入，如果所述人脸识别模块无法识别出为作业场所人员身份信息时判定为陌生人，并与所述预警模块联动，进行语音预警；不安全行为分析步骤，所述超级计算机通过监控摄像机的视频流实时监测作业人员是否存在不安全行为，当监测到不安全行为时，联动所述预警模型，分区域分类别进行实时语音预警，并在显示大屏上实时展示不安全行为的图片，且切换到监测到不安全行为的监控摄像头，该监控摄像头画面在所述显示大屏上展示。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211596339.3A CN116129490A (zh) | 2022-12-13 | 2022-12-13 | 一种用于复杂环境行为识别的监控装置及监控方法 |
| CN202211596339.3 | 2022-12-13 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024124970A1 true WO2024124970A1 (zh) | 2024-06-20 |
Family
ID=86299956
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2023/115727 WO2024124970A1 (zh) | 2022-12-13 | 2023-08-30 | 一种用于复杂环境行为识别的监控装置及监控方法 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN116129490A (zh) |
| WO (1) | WO2024124970A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118692015A (zh) * | 2024-08-28 | 2024-09-24 | 山东特联信息科技有限公司 | 基于ai视频的液化石油气瓶智能充装监管系统及方法 |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116129490A (zh) * | 2022-12-13 | 2023-05-16 | 上海船舶工艺研究所(中国船舶集团有限公司第十一研究所) | 一种用于复杂环境行为识别的监控装置及监控方法 |
| CN117975361A (zh) * | 2024-01-29 | 2024-05-03 | 北京易丰嘉诚科技有限公司 | 一种大数据安防监控系统 |
| CN118072375B (zh) * | 2024-04-17 | 2024-06-21 | 云燕科技(山东)股份有限公司 | 一种人脸图像采集系统 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110176117A (zh) * | 2019-06-17 | 2019-08-27 | 广东翔翼科技信息有限公司 | 一种基于行为识别技术的监控装置及监控方法 |
| CN114071070A (zh) * | 2020-08-08 | 2022-02-18 | 徐州市五岳通信科技有限公司 | 一种基于智慧人脸识别及增强现实的动态监控预警系统 |
| CN114926778A (zh) * | 2022-04-18 | 2022-08-19 | 中航西安飞机工业集团股份有限公司 | 一种生产环境下的安全帽与人员身份识别系统 |
| CN116129490A (zh) * | 2022-12-13 | 2023-05-16 | 上海船舶工艺研究所(中国船舶集团有限公司第十一研究所) | 一种用于复杂环境行为识别的监控装置及监控方法 |
-
2022
- 2022-12-13 CN CN202211596339.3A patent/CN116129490A/zh active Pending
-
2023
- 2023-08-30 WO PCT/CN2023/115727 patent/WO2024124970A1/zh unknown
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110176117A (zh) * | 2019-06-17 | 2019-08-27 | 广东翔翼科技信息有限公司 | 一种基于行为识别技术的监控装置及监控方法 |
| CN114071070A (zh) * | 2020-08-08 | 2022-02-18 | 徐州市五岳通信科技有限公司 | 一种基于智慧人脸识别及增强现实的动态监控预警系统 |
| CN114926778A (zh) * | 2022-04-18 | 2022-08-19 | 中航西安飞机工业集团股份有限公司 | 一种生产环境下的安全帽与人员身份识别系统 |
| CN116129490A (zh) * | 2022-12-13 | 2023-05-16 | 上海船舶工艺研究所(中国船舶集团有限公司第十一研究所) | 一种用于复杂环境行为识别的监控装置及监控方法 |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118692015A (zh) * | 2024-08-28 | 2024-09-24 | 山东特联信息科技有限公司 | 基于ai视频的液化石油气瓶智能充装监管系统及方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116129490A (zh) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2024124970A1 (zh) | 一种用于复杂环境行为识别的监控装置及监控方法 | |
| CN109300471B (zh) | 融合声音采集识别的场区智能视频监控方法、装置及系统 | |
| CN211293956U (zh) | 基于ai的施工现场异常行为人的识别和告警系统 | |
| TWI749113B (zh) | 在視訊監控系統中產生警示之方法、系統及電腦程式產品 | |
| JP6474107B2 (ja) | 映像監視システム、映像処理装置、映像処理方法および映像処理プログラム | |
| CN118072255B (zh) | 一种智慧园区多源数据动态监控和实时分析系统及方法 | |
| CN110516529A (zh) | 一种基于深度学习图像处理的投喂检测方法和系统 | |
| CN110705482A (zh) | 一种基于视频ai智能分析的人员行为告警提示系统 | |
| CN111414873A (zh) | 基于安全帽佩戴状态的告警提示方法、装置和告警系统 | |
| KR20200052418A (ko) | 딥러닝 기반의 자동 폭력 감지 시스템 | |
| CN113282778A (zh) | 一种品质异常记录方法、装置、ar设备、系统及介质 | |
| KR20210097500A (ko) | 화재감지장치 및 이를 포함하는 화재감지시스템 | |
| CN110287917A (zh) | 基建工地的安全管控系统和方法 | |
| CN111178241A (zh) | 一种基于视频分析的智能监控系统及方法 | |
| CN111223011A (zh) | 一种基于视频分析的餐饮企业食品安全监管方法及系统 | |
| CN110602453B (zh) | 一种物联网大数据智能视频安防监控系统 | |
| CN115019462A (zh) | 视频处理方法、装置、存储介质及设备 | |
| KR20230097854A (ko) | 발전소내 작업자의 위험행동 인지방법 및 시스템 | |
| CN104767969A (zh) | 铁路道口用作业智能化安全监控系统和方法 | |
| CN210091231U (zh) | 一种智慧园区管理系统 | |
| CN118171809A (zh) | 一种智慧工地安全预警方法 | |
| CN116208633A (zh) | 一种人工智能服务平台系统、方法、设备及介质 | |
| CN114363552A (zh) | 一种基于图像识别的远程运维视频会议系统 | |
| Mohammed et al. | Implementation of human detection on raspberry pi for smart surveillance | |
| TWI784718B (zh) | 廠區告警事件處理方法與系統 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23902177 Country of ref document: EP Kind code of ref document: A1 |