CN115994953A - Electric power site safety monitoring and tracking method and system - Google Patents

Abstract

The invention discloses a power field safety supervision tracking method, which relates to the technical field of relay protection operation and maintenance and is used for solving the problems of inaccurate existing manual supervision and lack of positioning data, and the method comprises the following steps: calculating internal parameter and external parameter of multiple view cameras; performing personnel positioning according to the internal parameter and the external parameter; receiving a video sent by the camera, extracting skeleton characteristics according to the video, and obtaining a behavior recognition result according to the skeleton characteristics; identifying a preset target in the camera and tracking a track to obtain a target detection result; and outputting the behavior identification result, the target detection result and the operator positioning result as security monitoring information. The invention also discloses a power field security monitoring and tracking system. According to the invention, the multi-view camera is used for positioning and behavior recognition, so that three-dimensional positioning and tracking of personnel and the like on an electric power operation site are realized.

Description

Method and system for power site safety monitoring and tracking

technical field

The invention relates to the technical field of relay protection operation and maintenance, in particular to a method and system for on-site safety monitoring and tracking of electric power in a power distribution operation environment.

Background technique

In engineering industries such as construction, communication, and electric power, operators need to frequently perform outdoor operations. Due to the complex outdoor environment, factors such as high pressure, high altitude, and deep pits will bring safety hazards to operators; once a safety accident occurs, it will cause huge loss of personnel and property. Therefore, it is necessary to identify and track the on-site behavior of operators during the operation to ensure that operators use compliant operating procedures, and to identify, warn and locate dangerous behaviors.

The current supervision method is usually completed by manual verification, and the surveillance video also relies on manual guards, which cannot achieve real-time identification of operator behavior and early warning of dangerous behavior. In order to ensure the real-time identification and tracking of the behavior of workers in outdoor operation scenarios, there are currently researches based on human body image recognition to easily obtain the results of current personnel's behavior recognition. However, affected by factors such as occlusion, detection distance, and detection angle, human body image recognition is prone to missing or image size reduction problems.

Therefore, in order to ensure the safety of personnel and equipment in the operation scene, and to solve the current technical problems of inaccurate behavior recognition and inability to obtain the three-dimensional coordinates of the operator, it is urgent to construct an operation demand-oriented power site safety monitoring and tracking method.

Contents of the invention

In order to overcome the deficiencies of the prior art, one of the objectives of the present invention is to provide a power site safety monitoring and tracking method, which realizes power site safety monitoring and tracking by performing three-dimensional positioning and tracking on operators.

One of purpose of the present invention adopts following technical scheme to realize:

A power site safety monitoring and tracking method, comprising the following steps:

Calculating internal parameters and external parameters of a plurality of viewing angle cameras; performing personnel positioning according to the internal parameters and the external parameters;

receiving the video sent by the camera, performing skeleton feature extraction according to the video, and matching the skeleton feature to obtain a behavior recognition result;

Recognizing the preset target in the camera and performing trajectory tracking to obtain a target detection result;

Outputting the behavior recognition result, the target detection result, and the operator positioning result as safety monitoring information.

Further, calculate the internal parameter parameters of multiple viewing angle cameras, including:

receiving the video recorded by the camera including the checkerboard;

Input the video into the camera imaging model to obtain internal parameters and distortion parameters.

Further, calculate the extrinsic parameters of multiple perspective cameras, including:

receiving the image sequence sent by the camera;

According to the image sequence, the camera checkerboard coordinate pose is output through a pose algorithm;

A camera is selected as the world coordinate system, and three-dimensional coordinate transformation is performed on the checkerboard coordinate poses of other cameras to obtain the world coordinate system pose as the extrinsic parameter.

Further, the extraction of the skeleton feature includes:

preprocessing the video;

The preprocessed video is input to the openpose human pose estimation network to obtain a skeleton feature map;

The bone feature map is input into the ST-GCN graph convolutional neural network to obtain 256-dimensional features.

Further, obtaining a behavior recognition result according to the skeleton feature matching includes the following steps:

Concatenate the 256-dimensional features of multiple perspective cameras and perform 2 times of downsampling;

Input the SoftMax classifier through the 256-dimensional feature after downsampling;

Output behavior categories.

Further, the behavior categories include standing, walking, running, climbing, and jumping.

Further, identifying a preset target in the camera and performing trajectory tracking to obtain a target detection result includes the following steps:

Preprocessing the video in the camera;

The preprocessed video input detection neural network model VSSA-Net is obtained to obtain the target category;

The target category is input into the target tracking model ByteTrack to track the motion trajectory, and the detection result of the target motion trajectory is obtained.

Further, the target detection result also includes target positioning, and the detection of target positioning includes:

According to the pixel coordinates of the same target in multiple cameras, the world coordinate system coordinates of the same target are calculated through a multi-view geometric algorithm.

Further, the target categories include: human heads, safety helmets, birds, and small inspection robots.

The second object of the present invention is to provide an electric power on-site safety monitoring and tracking system, which can identify operators and other targets through cameras with multiple viewing angles, thereby realizing on-site safety monitoring.

Two of the purpose of the present invention adopts following technical scheme to realize:

A power site safety monitoring and tracking system, characterized in that the system includes a multi-camera internal and external reference calibration unit, a behavior recognition unit based on skeleton features, and a target tracking unit; the multi-camera internal and external reference calibration unit is used to record the The relative pose transformation and calculation of internal parameters and external parameter parameters; the behavior recognition unit is used to identify the behavior of the operator in the camera video; the target tracking unit is used to locate and track the preset target and identification.

Compared with the prior art, the beneficial effects of the present invention are:

The present invention records the video of the work site through a multi-view camera, which can effectively obtain the precise positioning of the operator, avoid the interference caused by problems such as occlusion and distance to the camera, realize three-dimensional coordinate positioning by calculating internal and external parameters, and intuitively display the work of the operator through feature recognition situation; in addition, the present invention also identifies targets other than operators, and can display the conditions of the work site in detail; the safety monitoring information provided by the present invention is detailed and accurate, which can ensure safe and stable on-site operations.

Description of drawings

Fig. 1 is the flow chart of embodiment one electric field safety supervision tracking method;

Fig. 2 is a flow chart of the behavior recognition process in Embodiment 1.

Detailed ways

The present invention will be described in more detail below in conjunction with the accompanying drawings. It should be noted that the following description of the present invention with reference to the accompanying drawings is only illustrative rather than limiting. Various embodiments can be combined with each other to form other embodiments not shown in the following description.

Embodiment one

Embodiment 1 provides a safety monitoring and tracking method for an electric power site, which aims to perform three-dimensional positioning through multi-view cameras, and realize tracking of personnel and other things on the job site.

It should be noted that the methods described in this embodiment are all based on multiple viewing angle cameras. The above-mentioned cameras can be cameras at several different job site viewing angles, or a camera that can realize multiple viewing angles. In order to provide multi-view image information, and realize the coordinate positioning of personnel on the job site by calculating internal and external parameters.

Please refer to Fig. 1, a power site safety monitoring and tracking method includes the following steps:

S1. Calculating the internal parameters and external parameters of multiple view cameras; performing personnel positioning according to the internal parameters and the external parameters;

The above internal reference refers to the calculation of the focal length and distortion parameters of each camera itself. Each camera needs to record a video containing a checkerboard, and ensure that all corners of the checkerboard are visible to the camera, and detect the corners of the checkerboard. And use the camera imaging model combined with the camera image distortion model to calculate the internal reference matrix of each camera and the distortion parameters of the camera lens. Specifically, calculate the internal parameter parameters of multiple viewing angle cameras, including:

receiving the video recorded by the camera including the checkerboard;

Input the video into the camera imaging model to obtain internal parameters and distortion parameters.

The imaging model of the above-mentioned camera is a pinhole camera model in this embodiment, and the corresponding internal parameter matrix is as follows:

Where f _x , c _x , f _y , c _y are the corresponding internal parameters;

The distortion model generally refers to the radial distortion and tangential distortion caused by the camera lens, as follows:

x _distorted ＝x(1+k ₁ r ² +k ₂ r ⁴ +k ₃ r ⁶ )+2p ₁ xy+p ₂ (r ² +2x ² )

y _distorted = y(1+k ₁ r ² +k ₂ r ⁴ +k ₃ r ⁶ )+2p ₂ xy+p ₁ (r ² +2y ² )

Where (x, y) are normalized coordinates before distortion, (x _distorted , y _distorted ) are normalized coordinates after distortion, k ₁ , k ₂ , k ₃ are radial distortion parameters, p ₁ , p ₂ is the tangential distortion parameter;

Through checkerboard calibration for each camera, input recorded video, output internal reference and distortion parameters

In S1, the external parameter parameters of multiple view cameras are calculated, including:

receiving the image sequence sent by the camera;

According to the image sequence, the camera checkerboard coordinate pose is output through a pose algorithm;

A camera is selected as the world coordinate system, and three-dimensional coordinate transformation is performed on the checkerboard coordinate poses of other cameras to obtain the world coordinate system pose as the extrinsic parameter.

接着以相机C₁坐标系为世界坐标系，通过三维坐标变换得到其他相机在世界坐标系下的位姿

也即外参。变换过程满足The calculation of the external parameters in this embodiment adopts the EasyMoCap algorithm. Specifically, the checkerboard is used to set up ground control points relative to multiple cameras to ensure that the control points are visible to all cameras at the same time; then input the recorded image sequences of each camera, and output each camera The pose in the established checkerboard coordinate system. Define the checkerboard coordinate system as O, each camera coordinate system as C _i , i=1, 2,...n, then the pose of the camera in the checkerboard coordinate system is obtained through calibration as

Then take the camera C ₁ coordinate system as the world coordinate system, and obtain the poses of other cameras in the world coordinate system through three-dimensional coordinate transformation

That is, external reference. The conversion process satisfies

S21. Receive the video sent by the camera, perform skeleton feature extraction according to the video, and obtain a behavior recognition result according to the skeleton feature matching;

The above-mentioned skeleton feature behavior recognition is used to detect and record the time-series behavior of the electric field workers.

Specifically, referring to Fig. 2, the extraction of the skeleton features and the matching of the behavior recognition results include:

S211. Preprocess the video;

The above preprocessing is used to preprocess the image streams of multiple cameras. Specifically, the video streams from multiple cameras are aligned according to timestamps, and 10 frames of images in the same time window are respectively intercepted and normalized. Crop uniformly to a size of 256×256 pixels.

S212. Input the preprocessed video into the openpose human pose estimation network to obtain a skeleton feature map;

S213. Input the bone feature map into the ST-GCN graph convolutional neural network to obtain 256-dimensional features.

S214. Concatenating the 256-dimensional features of multiple view cameras, and performing down-sampling twice;

The above concatenation takes setting up 4 cameras around the scene as an example, respectively concatenating 256-dimensional features from 4 perspectives, and re-obtaining 256-dimensional features after 2 times of downsampling, and inputting them into the SoftMax classifier.

S215. Input the down-sampled 256-dimensional feature into the SoftMax classifier;

S216. Output the behavior category.

In this embodiment, the behavior categories include standing, walking, running, climbing, and jumping. Of course, according to actual needs, the behavior category can also be increased or decreased.

S22. Identifying the preset target in the camera and performing trajectory tracking to obtain a target detection result;

The aforementioned preset targets in this embodiment include, but are not limited to: personnel heads, safety helmets, birds, and small inspection robots.

Target tracking is used to detect and track specific small targets on the power site, so as to provide a basis for safety judgments at the work site.

In this embodiment, the Tracking-by-detection strategy is used as an overall strategy for implementing target tracking. This strategy refers to finding the target object in a single frame image through target detection, and then associating the same detected target between different frames to achieve tracking. Therefore, the whole module includes the small target detection network model VSSA-Net and the target tracking model ByteTrack. Combined to realize the small target tracking function.

Specifically, identifying a preset target in the camera and performing trajectory tracking to obtain a target detection result includes the following steps:

Preprocessing the video in the camera; the preprocessing mainly includes normalizing the picture and picture data in the video sequence and scaling to 640×640 resolution.

Input the preprocessed video into the detection neural network model VSSA-Net, detect the specific preset target existing in the current picture, and obtain the target category; in addition, for the tracked target that appears in multiple cameras at the same time, also need Matching is performed based on the detection categories output by the detection model.

Input the target category into the target tracking model ByteTrack to track the trajectory of the target, track the trajectory of the target in the corresponding camera view, output the historical trajectory of the tracked target, and obtain the detection result of the target trajectory.

The reason for choosing the VSSA-Net model is that it is an efficient single-shot detector with the characteristics of multi-scale fusion. It uses densely connected deconvolution layers and skip connections to obtain multi-scale feature maps. Compared with ordinary target detectors, it improves The detection performance of the small target, that is, the preset target mentioned above, is improved. For the ByteTrack model used for target tracking, the model tracks by associating almost all detection frames instead of only high-score detection frames, and uses the similarity between the detection frame and the trajectory to determine the real target object and filter the background.

In order to further increase the accuracy of target detection, the target detection result also includes target location, and the detection of the target location includes:

According to the pixel coordinates of the same target in multiple cameras, the world coordinate system coordinates of the same target are calculated through a multi-view geometric algorithm.

Specifically, the coordinates [x, y, z] ^T of the target in the world coordinate system are calculated using the multi-view geometry method, denoted as P _w , by using the pixel coordinates of the same target under different cameras, combined with the camera calibration information. Specifically, it is necessary to use objects that are stable and non-deformable, such as the detection results of hard hats. First, take the center point M of the detection frame and treat it as the same point in space. The pixel coordinates of point M in different camera images are marked as pixel coordinates (u _i , v _i ), where i represents the camera number. Transform the coordinate P _w from the camera coordinate system to the respective camera coordinate system P _i =[ _xi ,y _i ,zi _] ^T as follows:

是此前计算的相机外参，另外公式中涉及到齐次坐标与非齐次坐标的转换，需要在P_w与P_i向量末尾添加1；in

is the previously calculated camera extrinsic parameter, and the formula involves the conversion of homogeneous coordinates and non-homogeneous coordinates, and 1 needs to be added at the end of the P _w and P _i vectors;

The projection model by the camera should have:

where K _i is the previously calculated camera internal reference;

Simultaneously combining the camera equations can be solved to get P _w ;

S3. Outputting the behavior recognition result, the target detection result, and the operator positioning result as safety monitoring information.

Through the behavior recognition results, target detection results and operator positioning results output in S3, the three-dimensional positioning and tracking of operators in the operation scene can be realized, and safety warning basis can be provided for supervisors or warning systems, effectively avoiding safety hazards at the operation site. question.

Embodiment two

The second embodiment is an explanation and description of a power site safety monitoring and tracking system.

A power site security monitoring and tracking system, the system includes a multi-camera internal and external reference calibration unit, a behavior recognition unit based on skeleton features, and a target tracking unit; the multi-camera internal and external reference calibration unit is used to record the relative position between multiple cameras Pose transformation and calculation of internal parameters and external parameters; the behavior identification unit is used to identify the behavior of the operator in the camera video; the target tracking unit is used to locate, track and identify preset targets.

The camera internal reference calibration module calibrates the focal length and distortion parameters of each camera. First, each camera records a video containing a checkerboard, and ensures that all corners of the checkerboard are visible to the camera; then, the internal parameter matrix of each camera is calculated by using the camera imaging model combined with the camera distortion model. The camera extrinsic calibration unit can calculate the relative pose transformation between multiple cameras.

The behavior recognition unit based on skeleton features can also include a serially connected preprocessing module, feature extraction module, and fusion output module; for the operating logic and processing methods of each module, please refer to the relevant description in Embodiment 1.

The target tracking unit may include interconnected image preprocessing sub-modules, small target detection sub-modules, target tracking sub-modules, and multi-view positioning sub-modules. For the specific operation logic and processing methods of this unit, please refer to the relevant description in Embodiment 1.

Those skilled in the art can make various other corresponding changes and deformations according to the above-described technical solutions and concepts, and all these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims (10)

1. A method for tracking safety on the spot of electric power, is characterized in that, comprises the following steps: Calculating internal parameters and external parameters of a plurality of viewing angle cameras; performing personnel positioning according to the internal parameters and the external parameters; receiving the video sent by the camera, performing skeleton feature extraction according to the video, and matching the skeleton feature to obtain a behavior recognition result; Recognizing the preset target in the camera and performing trajectory tracking to obtain a target detection result; Outputting the behavior recognition result, the target detection result and the operator positioning result as safety monitoring information. 2. The electric power site safety monitoring and tracking method according to claim 1, wherein the calculation of internal parameters of a plurality of viewing angle cameras includes: receiving the video recorded by the camera including the checkerboard; The video is input into the camera imaging model to obtain internal reference parameters and distortion parameters. 3. The power site safety monitoring and tracking method according to claim 1 or 2, wherein calculating the external parameter parameters of a plurality of viewing angle cameras includes: receiving the image sequence sent by the camera; According to the image sequence, the camera checkerboard coordinate pose is output through a pose algorithm; A camera is selected as the world coordinate system, and three-dimensional coordinate transformation is performed on the checkerboard coordinate poses of other cameras to obtain the world coordinate system pose as the extrinsic parameter. 4. The electric power site safety monitoring and tracking method according to claim 1, wherein the extraction of the skeleton features includes: preprocessing the video; The preprocessed video is input to the openpose human pose estimation network to obtain a skeleton feature map; The bone feature map is input into the ST-GCN graph convolutional neural network to obtain 256-dimensional features. 5. The electric power site safety monitoring and tracking method as claimed in claim 4, wherein obtaining the behavior recognition result according to the skeleton feature matching comprises the following steps: Concatenate the 256-dimensional features of multiple perspective cameras and perform 2 times of downsampling; Input the SoftMax classifier through the 256-dimensional feature after downsampling; Output behavior categories. 6. The power site safety monitoring and tracking method according to claim 5, wherein the behavior categories include standing, walking, running, climbing, and jumping over. 7. The electric power site safety monitoring and tracking method according to claim 1, wherein identifying a preset target in the camera and performing trajectory tracking to obtain a target detection result comprises the following steps: Preprocessing the video in the camera; The preprocessed video input detection neural network model VSSA-Net is obtained to obtain the target category; The target category is input into the target tracking model ByteTrack to track the motion trajectory, and the detection result of the target motion trajectory is obtained. 8. The power site safety monitoring and tracking method according to claim 1 or 7, wherein the target detection result also includes target location, and the detection of the target location includes: According to the pixel coordinates of the same target in multiple cameras, the world coordinate system coordinates of the same target are calculated through a multi-view geometric algorithm. 9. The electric power site safety monitoring and tracking method according to claim 7, wherein the target categories include: personnel heads, safety helmets, birds, and small inspection robots. 10. A power site safety monitoring and tracking system, characterized in that the system includes a multi-camera internal and external reference calibration unit, a behavior recognition unit based on skeleton features, and a target tracking unit; the multi-camera internal and external reference calibration unit is used to record multiple The relative pose transformation between the cameras and the calculation of internal parameters and external parameters; the behavior recognition unit is used to identify the behavior of the operator in the camera video; the target tracking unit is used to locate the preset target , tracking and identification.

Images