CN110674761A - Regional behavior early warning method and system - Google Patents
Regional behavior early warning method and system Download PDFInfo
- Publication number
- CN110674761A CN110674761A CN201910921181.4A CN201910921181A CN110674761A CN 110674761 A CN110674761 A CN 110674761A CN 201910921181 A CN201910921181 A CN 201910921181A CN 110674761 A CN110674761 A CN 110674761A
- Authority
- CN
- China
- Prior art keywords
- information
- person
- future
- scene
- behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000006399 behavior Effects 0.000 claims abstract description 73
- 230000000694 effects Effects 0.000 claims abstract description 60
- 238000013528 artificial neural network Methods 0.000 claims abstract description 40
- 230000033001 locomotion Effects 0.000 claims abstract description 40
- 230000015654 memory Effects 0.000 claims abstract description 29
- 238000012544 monitoring process Methods 0.000 claims description 23
- 230000003993 interaction Effects 0.000 claims description 18
- 230000000007 visual effect Effects 0.000 claims description 11
- 230000006835 compression Effects 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 6
- 230000003542 behavioural effect Effects 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 description 18
- 230000009471 action Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 11
- 206010000117 Abnormal behaviour Diseases 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 8
- 230000001815 facial effect Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000007619 statistical method Methods 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 241001622623 Coeliadinae Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
- G06Q50/265—Personal security, identity or safety
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Multimedia (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Tourism & Hospitality (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computer Security & Cryptography (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- Educational Administration (AREA)
- Economics (AREA)
- Primary Health Care (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Alarm Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a regional behavior early warning method and a system, wherein the embodiment of the invention inputs figure image information collected in a region into a set first neural network, and outputs and obtains scene information, figure limb information, relationship information between figures and the scene and relationship information between the figures and objects in the scene; abstracting the obtained information into an information tensor, and respectively inputting the information tensor into a set memory network model and a set second neural network, wherein the memory network model outputs the future motion trail information of the figure, and the second neural network outputs the future activity tag and the future target position information of the figure; and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person. The embodiment of the invention adopts a plurality of neural networks to accurately predict the information related to the future activities of the people so as to judge whether the behavior of the people is suspicious, thereby improving the early warning accuracy and improving the early warning effect.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a regional behavior early warning method and a regional behavior early warning system.
Background
With the great progress of algorithm optimization and prediction accuracy of the deep neural network in recent years, the deep learning is applied to various fields of life to almost achieve good effect, and a large amount of labor cost is saved. With the increase of urban residents in recent years, the regional security early warning faces unprecedented challenges, and good security effects are difficult to obtain by means of old-fashioned monitoring and artificial checking.
In order to improve the effect of the regional behavior early warning, a deep neural network can be introduced for realization, and specifically, the following modes are available for realization.
The patent application with the publication number of CN109064698A discloses a resident security early warning system, which identifies whether people entering a residential area are residents of a non-residential area or not through human faces; for the residents in the non-local community, continuously acquiring the path track and the behavior pattern of the residents in the non-local community in the community, wherein the path track and the behavior pattern at least comprise the behavior determination of the residents in and out of the non-local community; accumulating the total number of units of the residents of the non-local community entering and exiting the community within a set time period, and acquiring the residence time of each unit of the residents of the non-local community in the community; when the total number of units of residents in the non-local community entering and exiting the community within a set time period is not less than N, judging whether the residents in the non-local community have abnormal behaviors or not according to the residence time of the residents in each unit of the non-local community, and determining the grade of the abnormal behaviors, wherein N is an integer and is not less than 3; and sending out an alarm according to different abnormal behavior levels.
The patent application with publication number CN109189078A discloses a household safety protection robot and method based on deep reinforcement learning, which share the behavior state information of a target person, the position of the target person and the position of an obstacle in the environment through deep learning, and output the moving track of the target person, thereby realizing locking and tracking of the target person.
Patent application publication No. CN105975633A discloses a method and apparatus for obtaining motion trajectory. Predicting the motion track of a preset figure according to the current geographic position and the motion state of the preset figure corresponding to the preset figure information; and starting the target shooting equipment on the motion trail.
Patent application with publication number CN108877121A discloses an artificial intelligence early warning system based on cloud platform, mainly through the face collection module who locates the district entrance for gather the face of discrepancy district, and form face identification information and send cloud ware, compare through the photo with criminal, form warning signal.
It can be seen that, when the above scheme is used for realizing the regional behavior early warning, the region is a cell as an example, and a face recognition unit is mostly arranged at a position such as a cell entrance guard, so that the identity of a person entering the cell is determined through face recognition, or the identity is compared with a photo of a criminal through a cloud platform, and the early warning is realized. However, once the relevant personnel enter the cell, the whereabouts of the relevant personnel can be recorded only through the traditional monitoring camera, and abnormal behaviors are judged through artificial inspection or a simple statistical method to give an early warning. The manual inspection of video monitoring consumes a large amount of labor cost, continuous monitoring cannot be achieved when personnel change posts and the spirit is not concentrated, the inspection of video monitoring by adopting a statistical method is not accurate, and the statistical standard value is difficult to determine. No matter which mode is adopted, the early warning accuracy rate is low.
Therefore, how to realize the regional behavior early warning on the basis of improving the early warning accuracy and the early warning effect is an urgent problem to be solved.
Disclosure of Invention
In view of this, the embodiment of the present invention provides a regional behavior early warning method, which can improve early warning accuracy and improve early warning effect.
The embodiment of the invention also provides a regional behavior early warning system, which can improve the early warning accuracy and improve the early warning effect.
The embodiment of the invention is realized as follows:
a regional behavior early warning method comprises the following steps:
inputting the image information of the person collected in the area into a first set neural network, and outputting the information of the scene in the area, the limb information of the person, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene;
abstracting scene information, figure limb information, relationship information between a figure and a scene, and relationship information between the figure and an object in the scene in the area into an information tensor;
inputting the abstract information tensor into a set memory network model and a set second neural network, so that the memory network model outputs the future motion trail information of the person, and the second neural network outputs the future activity label and the future target position information of the person;
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
Before the acquiring of the person image information in the region, the method further comprises:
and identifying the person appearing in the set monitoring camera by adopting a face and identity identification mode, and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
The determining whether the person behavior is suspicious further comprises:
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label of the person, the future purpose information and the identity information of the person.
The abstract information tensor is input into a set memory network model, and the information of the future motion trail of the figure obtained through output is as follows:
and performing compression coding on the abstract information tensor by using a memory network model to obtain a visual characteristic tensor, decoding by using the memory network model, and outputting characteristic coordinate information of a future motion trail of a person on an actual coordinate plane.
The first neural network is updated in real time; updating the memory network model in real time; the second neural network is updated in real time.
The method further comprises the following steps:
and if the person behavior is judged to be suspicious, early warning is carried out.
A regional behavioral early warning system, comprising: a character behavior module, a character interaction module, a track generation module and an activity prediction module, wherein,
the character behavior module is used for setting a first neural network, setting a memory network model and a second neural network;
the character interaction module is used for inputting character image information acquired in the region into a set first neural network, outputting and obtaining scene information, character limb information, relationship information between characters and the scene and relationship information between the characters and objects in the scene in the region, and abstracting the scene information, the character limb information, the relationship information between the characters and the scene and the relationship information between the characters and the objects in the scene into an information tensor;
the track generation module is used for inputting the abstract information tensor into a set memory network model so as to enable the memory network model to output the information of the future movement track of the person;
an activity prediction module for inputting the abstracted information tensor into a second neural network, the second neural network outputting future activity labels and future destination location information of the person;
and the abnormity alarm module is used for judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
The system further comprises:
and the face and identity recognition module is used for recognizing the person appearing in the set monitoring camera and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
The track generation module is further configured to perform compression coding on the abstract information tensor through a memory network model, obtain a visual characteristic tensor, decode the visual characteristic tensor through the memory network model, and output characteristic coordinate information of a future motion track of a person on an actual coordinate plane.
And the abnormity alarm module is used for early warning when the person behavior is judged to be suspicious.
As can be seen from the above, the embodiment of the present invention inputs the person image information collected in the area into the first neural network, and outputs the information to obtain the scene information, the person limb information, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene in the area; abstracting the obtained information into an information tensor, and respectively inputting the information tensor into a set memory network model and a set second neural network, wherein the memory network model outputs the future motion trail information of the figure, and the second neural network outputs the future activity tag and the future target position information of the figure; and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person. The embodiment of the invention adopts a plurality of neural networks to accurately predict the information related to the future activities of the people so as to judge whether the behavior of the people is suspicious, thereby improving the early warning accuracy and improving the early warning effect.
Drawings
Fig. 1 is a flowchart of a regional behavior early warning method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a regional behavior early warning system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an execution process of a face and identity recognition module according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a process of a character behavior module according to an embodiment of the present invention;
FIG. 5 is a schematic processing flow diagram of a character interaction module according to an embodiment of the present invention;
FIG. 6 is a block diagram of an activity prediction module according to an embodiment of the present invention;
fig. 7 is a schematic processing flow diagram of an exception alarm module according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and examples.
It can be seen from the background art that the main reasons for the low accuracy and poor warning effect of the warning during the regional behavior warning are that the warning is implemented only by using the face recognition unit to determine suspicious people, or using the surveillance camera technology to track the trace of people, and manually or statistically determining whether the behavior of people is suspicious. Furthermore, most behavior predictions about people described in the background art are trajectory predictions, and the main purpose is to predict the next traveling direction of a suspicious person, so as to perform real-time tracking, which does not involve future behavior trajectory prediction of people and has low early warning accuracy.
With the development of Artificial Intelligence (AI) technology, AI technology is widely used. Along with the continuous proposition of more advanced network models and algorithms, the accuracy of AI application is improved, and the artificial inspection can be replaced in various fields in the future. Therefore, in order to overcome the problems generated in the regional behavior early warning in the background technology, the embodiment of the invention applies the most advanced action track and behavior prediction of video characters to the regional behavior early warning, replaces the manual monitoring and statistical method in the background technology, not only can determine the identity of the characters, but also can train a plurality of deep neural networks through a large amount of training data, predict the abnormal behaviors of the characters in real time through the deep neural networks, achieve early warning and prevent the abnormal behaviors in the bud. Specifically, the embodiment of the invention inputs the image information of the person collected in the area into the set first neural network, and outputs the information to obtain the scene information, the limb information of the person, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene; abstracting the obtained information into an information tensor, and respectively inputting the information tensor into a set memory network model and a set second neural network, wherein the memory network model outputs the future motion trail information of the figure, and the second neural network outputs the future activity tag and the future target position information of the figure; and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
Here, the memory network model is a Long Short Term Memory (LSTM) network, and the neural network employs CNN, which is described in detail below as an example.
Therefore, the embodiment of the invention adopts a plurality of neural networks to accurately predict the information related to the future activities of the person so as to judge whether the behavior of the person is suspicious, thereby improving the early warning accuracy and improving the early warning effect.
Fig. 1 is a flowchart of a regional behavior early warning method provided in an embodiment of the present invention, which includes the following specific steps:
step 101, inputting the image information of the person collected in the area into a set first CNN, and outputting the information of the scene in the area, the information of the limbs of the person, the information of the relationship between the person and the scene, and the information of the relationship between the person and the object in the scene;
103, inputting the abstract information tensor into a set LSTM network and a set second CNN, so that the LSTM network outputs to obtain the future motion track information of the person, and the second CNN outputs the future activity label and the future target position information of the person;
and step 104, judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
In the method, before the capturing of the person image information in the region, the method further includes:
and identifying the person appearing in the set monitoring camera by adopting a face and identity identification mode, and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
In this case, the determining whether the person behavior is suspicious further includes:
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label of the person, the future purpose information and the identity information of the person.
In the method, the abstract information tensor is input into a set LSTM network, and the information of the future motion trail of the person obtained by output is:
and carrying out compression coding on the abstract information tensor by using an LSTM network to obtain a visual characteristic tensor, then adopting the LSTM network to decode, and outputting characteristic coordinate information of a future motion track of a person on an actual coordinate plane.
In the method, the first CNN is updated in real time; the LSTM is updated in real time; the second CNN is updated in real time. Therefore, the plurality of deep neural networks of the embodiment of the invention are actually a network which can continuously learn and optimize, and the recorded information can be used for the training process of the plurality of neural networks, so that the accuracy of network judgment is improved.
In this method, the method further comprises:
if the person behavior is judged to be suspicious, early warning is carried out
Fig. 2 is a schematic structural diagram of a regional behavior early warning system provided in an embodiment of the present invention, including: a character behavior module, a character interaction module, a track generation module and an activity prediction module, wherein,
the character behavior module is used for setting a first CNN, and setting an LSTM and a second CNN;
the character interaction module is used for inputting character image information acquired in the region into a set first neural network CNN, outputting obtained scene information, character limb information, relationship information between characters and the scene and relationship information between the characters and objects in the scene in the region, and abstracting the scene information, the character limb information, the relationship information between the characters and the scene and the relationship information between the characters and the objects in the scene into an information tensor;
the track generation module is used for inputting the abstract information tensor into a set LSTM network so as to enable the LSTM network to output the information of the future motion track of the person;
the activity prediction module is used for inputting the abstract information tensor into a set second CNN, and the second CNN outputs future activity labels and future target position information of the person;
and the abnormity alarm module is used for judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
The system further comprises: and the face and identity recognition module is used for recognizing the person appearing in the set monitoring camera and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
In the system, the track generation module is further configured to perform compression coding on the abstract information tensor through an LSTM network to obtain a visual feature tensor, then decode the visual feature tensor through the LSTM network, and output feature coordinate information of a future motion track of a person on an actual coordinate plane.
In the system, the first CNN is updated in real time; the LSTM is updated in real time; the second CNN is updated in real time.
In the system, the abnormity warning module is used for carrying out early warning when the person behavior is judged to be suspicious.
Fig. 3 is a schematic diagram of an execution process of the face and identity recognition module according to the embodiment of the present invention. The face and identity recognition module is generally arranged in an area, such as a main channel and an inlet/outlet of a large-scale activity place, and mainly comprises a camera unit, a face acquisition unit, a facial feature extraction unit, a face matching unit and an identity confirmation unit. Specifically, as shown in fig. 3, wherein:
and step 305, the identity confirmation unit confirms whether the person is the person inside the area or the person outside the area according to the matching result.
In the embodiment of the present invention, the character behavior module is configured to set the first CNN, set the LSTM, and the second CNN, and when the first CNN, the LSTM, and the second CNN are set, the visual information of each person appearing in the monitoring camera scene is encoded, and in this process, the figure shape and the body motion are modeled instead of abstracting the character into one point, as shown in fig. 4, fig. 4 is a schematic processing procedure diagram of the character behavior module provided in the embodiment of the present invention. In order to model the appearance of a person in a regional scene, an object detection model with RolAlign is adopted to extract the CNN characteristic of the set size of each person bounding box; to obtain a physical motion model of a person, a microsoft image recognition (MSCOCO) network trained on a microsoft image recognition dataset was used to extract person motion keypoint information.
In the embodiment of the invention, the character interaction module is responsible for viewing the interaction between the character and the surrounding environment, including the interaction between the character and the scene and the interaction between the character and the object in the scene. The interaction between the person and the scene is to identify the scene near the person. As shown in fig. 5, fig. 5 is a schematic processing flow diagram of a character interaction module according to an embodiment of the present invention. Firstly, deriving pixel level scene semantic classification of each frame by using a trained scene segmentation model, and dividing parts such as roads, sidewalks and the like in a scene; and then selecting the set size to determine the environment area needing to be identified by the model. For example, the set size is 3, the range of 3 × 3 around the person is selected as the observation region, the information obtained at different times is input to the LSTM, the relationship information between the person and the scene is finally obtained, and then the scene information, the person limb information, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene in the region are abstracted into one information tensor.
In the embodiment of the invention, the track generation module inputs the abstract information tensor into a set LSTM network, so that the LSTM network outputs the future motion track information of the person, the information is compressed into a visual feature tensor Q by an LSTM network encoder, and then the future motion track information of the person is obtained by decryption by a decryptor of the LSTM network.
In the embodiment of the invention, the activity prediction module has two tasks and determines the type and the occurrence place of the future activity of the person. Accordingly, it includes two parts, activity location prediction and activity tag prediction of the manhattan mesh. The role of activity tag prediction is to guess what the final purpose of the character in the picture is, and predict the activity at some moment in the future. The active tags are not limited to one at a time, such as a person may walk and carry an item at the same time. The function of activity location prediction is to correct errors for the trajectory generation module, which determines the final destination of the character to compensate for the deviation between the trajectory generation module and the activity tag prediction. The method comprises two tasks of position classification and position regression. The purpose of the location classification is to predict a grid block where the final location coordinate is located, as shown in fig. 6, fig. 6 is a schematic structural diagram of an activity prediction module provided in the embodiment of the present invention. The goal of position regression is to predict the deviation of the grid block center (the dot in the figure) from the final position coordinates (the end of the arrow).
The abnormal alarm module in the embodiment of the present invention may analyze whether the behavior of the person is suspicious according to the results of the face and identity recognition module and the activity prediction module, so as to perform early warning on suspicious behavior of suspicious people, as shown in fig. 7, fig. 7 is a schematic processing flow diagram of the abnormal alarm module provided in the embodiment of the present invention, and the specific steps are as follows:
step 701, obtaining personnel identity information from a face and identity recognition module;
step 704, determining whether the behavior of the person is abnormal, if so, executing step 705; otherwise, returning to step 703 to continue execution;
By adopting the embodiment of the invention, the monitored personnel comprise personnel in the area and external personnel. For the cell environment, the method comprises the steps of providing a cell environment, wherein the cell environment comprises an owner, a property worker, an external person and the like; for villa environments, including owners, visitors, maintenance workers, security guards, unknown personnel and the like; the storehouse environment comprises custodians, pickup members, delivery members, external personnel and the like; for the bank environment, the bank environment comprises staff, business handling personnel, security personnel, suspicious personnel and the like; the market environment comprises guests, salesmen, cleaning personnel, security guards and the like.
By adopting the embodiment of the invention, the predicted character future activity information can comprise: outside personnel enter the corridor along with the owner, stand by to observe the work and rest rules of residents in the residential area, and check the structure of the building, monitor the camera position and the access passage of the residential area; burglary and destruction behaviors such as turning into a courtyard wall, prying open a door lock and the like; illegal entry, theft of confidential materials, technology, documents, etc.; the theft and robbery actions are carried out in banks and malls and are not limited to the above suspicious actions.
The notification objects for the criminal behavior early warning by adopting the embodiment of the invention comprise security personnel, policemen and owners, and are not limited to the above personnel. The real-time monitoring and early warning method includes common early warning methods such as short message early warning, telephone early warning, alarm bell early warning, internet of things (IOT) device early warning, video early warning, and the like, but is not limited to the above early warning methods.
By adopting the embodiment of the invention, the area comprises but is not limited to a community environment, a villa environment, a storehouse environment, a bank environment and a shopping mall environment. The concept of the area is also applicable to environments which are complicated for other people and are easy to steal and robbery.
The following description will be made by taking several specific examples
The first embodiment is as follows: the area is designated as a cell. By adopting the embodiment of the invention, in a community environment, early warning is given to the fact that outside personnel enter a corridor along with a owner in advance, the residence is sufficient to observe the work and rest rules of residents in the community, and abnormal behaviors such as building structures, monitoring camera positions, community access channels and the like are checked.
The specific process is as follows:
the first step is that the entrance or the main trunk of the cell detects that external people enter the cell through a face and identity recognition module.
And secondly, predicting future action paths and behaviors of external personnel through a character behavior module, a character interaction module, a track generation module and an activity prediction module according to video monitoring.
And step three, the abnormity alarm module finds that the abnormity appears to trail the owner to the corridor according to the information, and the abnormity alarm module can be used for standing and observing the working and rest rules of residents in the residential area, or checking the abnormal behaviors of the building structure, monitoring the camera position, the residential area access passage and the like. And giving an alarm in advance, and reminding security personnel to closely observe subsequent actions of external personnel. If necessary, the person is triggered to ask for questions.
Example two: the area is determined as a villa, and early warning is carried out for theft and destruction behaviors such as turning into a yard wall, prying open a door lock and the like outside the single villa by adopting the embodiment of the invention.
The specific process is as follows:
the method comprises the first step of arranging a camera on a high-altitude wall of the single villa and judging whether a person outside the villa yard is a master or not through face recognition.
And secondly, predicting the future action path and the future action of the person through a person behavior module, a person interaction module, a track generation module and an activity prediction module according to the monitoring of the video.
And in the third step, the abnormity alarm module alarms to the main person and security personnel in advance through the APP at the mobile phone end according to the information, if abnormal behaviors are found, and the automatic video playing function is started. And the IOT sound equipment in the villa can be started simultaneously to play the warning voice.
Example three: the area is set to be in a factory storeroom, and the embodiment of the invention is adopted to carry out early warning on illegal entry and actions of stealing confidential materials, technologies, files and the like.
The specific process is as follows:
the first step is that suspicious people are detected to enter through face recognition at an entrance or a main trunk of a storehouse, and the identity of the suspicious people is judged firstly.
And secondly, predicting the future action path and the future action of the person through a person behavior module, a person interaction module, a track generation module and an activity prediction module according to the monitoring of the video.
And thirdly, the abnormity alarm module gives an alarm in advance according to the information, such as the suspicion that confidential materials, technologies and files are stolen, so as to remind security personnel to closely observe the condition of the camera at the position, and the entrance and the exit of the storehouse are blocked to implement capture.
Example four: the area is set in a bank, and the embodiment of the invention is adopted to carry out early warning on the behaviors of robbery and theft to be implemented, remind security guards to pay attention and alarm to public security if necessary.
The specific process is as follows:
the first step is that each person entering the bank is detected through face recognition at the entrance and the main corner of the bank, the identity of the person is judged firstly, and if the person is the escaped person recorded on a case, an alarm is triggered immediately.
And secondly, predicting the future action path and the future action of the person through a person behavior module, a person interaction module, a track generation module and an activity prediction module according to the monitoring of the video.
And thirdly, the abnormity alarm module is communicated with wireless equipment on the security personnel in advance according to the information if the suspicion of robbery and theft is found, and the security personnel is reminded to closely observe the movement of the security personnel. And directly informing the police to alarm according to different safety levels when necessary.
Example five: the area is set in a large-scale shopping mall, and the embodiment of the invention is adopted to early warn the robbery and the theft to be implemented, remind security personnel to pay attention and alarm to public security if necessary.
The specific process is as follows:
the method comprises the first step of detecting each person entering the shopping mall at the entrance and the main corner of the large shopping mall through face recognition, firstly judging the identity of the person, and immediately triggering alarm if the person is the escaping person recorded on a case.
And secondly, predicting the future action path and the future action of the person through a person behavior module, a person interaction module, a track generation module and an activity prediction module according to the monitoring of the video.
And thirdly, the abnormity alarm module is communicated with wireless equipment on the security personnel in advance according to the information if the suspicion of robbery and theft is found, and the security personnel is reminded to track and observe the personnel.
And step four, if the person implements a criminal behavior in the early warning process, the person can play the track of the person in real time through a large screen of a networked mall, and security guards at all places of the mall are reminded to block.
Therefore, the embodiment of the invention provides intelligent monitoring in the area, not only provides more advanced and effective future behavior prediction and early warning for suspicious personnel, but also reduces the heavy workload of security personnel in the area to check a plurality of monitoring videos.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A regional behavior early warning method is characterized by comprising the following steps:
inputting the image information of the person collected in the area into a first set neural network, and outputting the information of the scene in the area, the limb information of the person, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene;
abstracting scene information, figure limb information, relationship information between a figure and a scene, and relationship information between the figure and an object in the scene in the area into an information tensor;
inputting the abstract information tensor into a set memory network model and a set second neural network, so that the memory network model outputs the future motion trail information of the person, and the second neural network outputs the future activity label and the future target position information of the person;
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
2. The method of claim 1, wherein prior to capturing the image information of the person within the region, the method further comprises:
and identifying the person appearing in the set monitoring camera by adopting a face and identity identification mode, and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
3. The method of claim 2, wherein determining whether the person behavior is suspicious further comprises:
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label of the person, the future purpose information and the identity information of the person.
4. The method of claim 1, wherein the abstract information tensor is input into the set memory network model, and the future motion trail information of the person is output as follows:
and performing compression coding on the abstract information tensor by using a memory network model to obtain a visual characteristic tensor, decoding by using the memory network model, and outputting characteristic coordinate information of a future motion trail of a person on an actual coordinate plane.
5. The method of claim 1, in which the first neural network is updated in real-time; updating the memory network model in real time; the second neural network is updated in real time.
6. The method of claim 1, wherein the method further comprises:
and if the person behavior is judged to be suspicious, early warning is carried out.
7. A regional behavioral early warning system, comprising: a character behavior module, a character interaction module, a track generation module and an activity prediction module, wherein,
the character behavior module is used for setting a first neural network, setting a memory network model and a second neural network;
the character interaction module is used for inputting character image information acquired in the region into a set first neural network, outputting and obtaining scene information, character limb information, relationship information between characters and the scene and relationship information between the characters and objects in the scene in the region, and abstracting the scene information, the character limb information, the relationship information between the characters and the scene and the relationship information between the characters and the objects in the scene into an information tensor;
the track generation module is used for inputting the abstract information tensor into a set memory network model so as to enable the memory network model to output the information of the future movement track of the person;
an activity prediction module for inputting the abstracted information tensor into a second neural network, the second neural network outputting future activity labels and future destination location information of the person;
and the abnormity alarm module is used for judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
8. The system of claim 7, wherein the system further comprises:
and the face and identity recognition module is used for recognizing the person appearing in the set monitoring camera and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
9. The system of claim 7, wherein the trajectory generation module is further configured to perform compression coding on the abstract information tensor by using a memory network model to obtain a visual feature tensor, and then decode the visual feature tensor by using the memory network model to output feature coordinate information of a future movement trajectory of the person on an actual coordinate plane.
10. The system of claim 7, wherein the anomaly alarm module is configured to perform an early warning when the person is determined to be suspicious.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910921181.4A CN110674761B (en) | 2019-09-27 | 2019-09-27 | Regional behavior early warning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910921181.4A CN110674761B (en) | 2019-09-27 | 2019-09-27 | Regional behavior early warning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110674761A true CN110674761A (en) | 2020-01-10 |
| CN110674761B CN110674761B (en) | 2022-07-15 |
Family
ID=69079531
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910921181.4A Active CN110674761B (en) | 2019-09-27 | 2019-09-27 | Regional behavior early warning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110674761B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111935461A (en) * | 2020-09-11 | 2020-11-13 | 合肥创兆电子科技有限公司 | Based on intelligent security control system |
| CN113376656A (en) * | 2021-06-07 | 2021-09-10 | 重庆大学 | Multi-robot enclosure system based on LSTM prediction |
| CN113449558A (en) * | 2020-03-26 | 2021-09-28 | 上海依图网络科技有限公司 | Method and device for monitoring abnormal behaviors of personnel |
| CN115273370A (en) * | 2022-09-28 | 2022-11-01 | 苏州美集供应链管理股份有限公司 | System and method for monitoring personnel in real time based on visual scanning and track recognition |
| CN115294515A (en) * | 2022-07-05 | 2022-11-04 | 南京邮电大学 | Artificial intelligence-based comprehensive anti-theft management method and system |
| CN118629137A (en) * | 2024-08-08 | 2024-09-10 | 江苏蓝盾智能科技有限公司 | Artificial intelligence-oriented access control monitoring system |
| CN118644094A (en) * | 2024-08-19 | 2024-09-13 | 杭州万斛泉科技有限公司 | Intelligent community user behavior intelligent monitoring and early warning method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509880A (en) * | 2018-03-21 | 2018-09-07 | 南京邮电大学 | A kind of video personage behavior method for recognizing semantics |
| CN109410496A (en) * | 2018-10-25 | 2019-03-01 | 北京交通大学 | Attack early warning method, apparatus and electronic equipment |
-
2019
- 2019-09-27 CN CN201910921181.4A patent/CN110674761B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509880A (en) * | 2018-03-21 | 2018-09-07 | 南京邮电大学 | A kind of video personage behavior method for recognizing semantics |
| CN109410496A (en) * | 2018-10-25 | 2019-03-01 | 北京交通大学 | Attack early warning method, apparatus and electronic equipment |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449558A (en) * | 2020-03-26 | 2021-09-28 | 上海依图网络科技有限公司 | Method and device for monitoring abnormal behaviors of personnel |
| CN111935461A (en) * | 2020-09-11 | 2020-11-13 | 合肥创兆电子科技有限公司 | Based on intelligent security control system |
| CN113376656A (en) * | 2021-06-07 | 2021-09-10 | 重庆大学 | Multi-robot enclosure system based on LSTM prediction |
| CN115294515A (en) * | 2022-07-05 | 2022-11-04 | 南京邮电大学 | Artificial intelligence-based comprehensive anti-theft management method and system |
| CN115273370A (en) * | 2022-09-28 | 2022-11-01 | 苏州美集供应链管理股份有限公司 | System and method for monitoring personnel in real time based on visual scanning and track recognition |
| CN115273370B (en) * | 2022-09-28 | 2023-01-03 | 苏州美集供应链管理股份有限公司 | System and method for monitoring personnel in real time based on visual scanning and track recognition |
| CN118629137A (en) * | 2024-08-08 | 2024-09-10 | 江苏蓝盾智能科技有限公司 | Artificial intelligence-oriented access control monitoring system |
| CN118629137B (en) * | 2024-08-08 | 2024-11-05 | 江苏蓝盾智能科技有限公司 | Artificial intelligence-oriented access control monitoring system |
| CN118644094A (en) * | 2024-08-19 | 2024-09-13 | 杭州万斛泉科技有限公司 | Intelligent community user behavior intelligent monitoring and early warning method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110674761B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110674761B (en) | Regional behavior early warning method and system | |
| US9911294B2 (en) | Warning system and method using spatio-temporal situation data | |
| WO2020168960A1 (en) | Video analysis method and apparatus | |
| JP6905850B2 (en) | Image processing system, imaging device, learning model creation method, information processing device | |
| CN112233300A (en) | Community passing epidemic prevention monitoring joint defense system and method based on artificial intelligence | |
| Lee et al. | ArchCam: Real time expert system for suspicious behaviour detection in ATM site | |
| CN110490126B (en) | Safe deposit box safety control system based on artificial intelligence | |
| CN111814510A (en) | Detection method and device for remnant body | |
| Gayathri et al. | Intelligent smart home security system: A deep learning approach | |
| Zin et al. | A Markov random walk model for loitering people detection | |
| Van Den Hengel et al. | Activity topology estimation for large networks of cameras | |
| Shivthare et al. | Suspicious activity detection network for video surveillance using machine learning | |
| Gnanavel et al. | Smart Surveillance System and Prediction of Abnormal Activity in ATM Using Deep Learning | |
| CN115294515B (en) | Comprehensive anti-theft management method and system based on artificial intelligence | |
| CN116778657A (en) | Method and system for intelligently identifying intrusion behavior | |
| KR102397839B1 (en) | A captioning sensor apparatus based on image analysis and a method for operating it | |
| JP6739119B6 (en) | Risk determination program and system | |
| KR20220031258A (en) | A method for providing active security control service based on learning data corresponding to counseling event | |
| Prezioso et al. | Integrating Object Detection and Advanced Analytics for Smart City Crowd Management | |
| KR20220064213A (en) | Program for operation of security monitoring device | |
| CN117746218B (en) | Security management method and device for smart campus | |
| CN118334559B (en) | Intelligent campus security management method and system based on face recognition | |
| Kapoor et al. | A study on various methods used in automatic video surveillance | |
| Blauensteiner et al. | A socio-technical approach for event detection in security critical infrastructure | |
| CN118942159A (en) | Digital twinning-based park visitor safety early warning method, system and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |