WO2024114500A1

WO2024114500A1 - Human body pose recognition method and device

Info

Publication number: WO2024114500A1
Application number: PCT/CN2023/133598
Authority: WO
Inventors: 殷政; 张力文; 栾元杰; 金子杰
Original assignee: 天翼数字生活科技有限公司
Priority date: 2022-11-30
Filing date: 2023-11-23
Publication date: 2024-06-06
Also published as: CN115909497A

Abstract

The present application discloses a human body pose recognition method and device. The method comprises: combining a preset tracker and a human body detection network to perform human body detection on a target person image frame so as to obtain a target human body box, wherein the preset tracker is a network that tracks a human body on the basis of an IOU and a threshold value; predicting human body key points in the target human body box by using a preset key point prediction network, wherein the preset key point prediction network comprises a reference pose estimation network and an interference part classification network; and recognizing a human body pose in the target person image frame according to the human body key points to obtain a human body pose recognition result. The present application can solve the technical problem in the prior art that high recognition precision and high recognition speed cannot be achieved at the same time.

Description

Human body posture recognition method and device

Technical Field

The present application relates to the field of machine vision technology, and in particular to a method and device for human body posture recognition.

Background technique

The human posture recognition task is mainly to obtain the coordinates of the joints of various parts of the human body from the video or picture captured by the camera. Human posture recognition currently has many application fields, including virtual reality, human-computer interaction, sports training and analysis, abnormal behavior detection, etc.

Current human posture recognition algorithms can be divided into two categories: one is top-down, which first detects each human bounding box and then predicts each person's skeleton key points through a key point regression network. The other is bottom-up, which directly predicts all human skeleton key points and then connects the key points into different human bodies. The former consumes a lot of computing cost in the human target detection stage and the overall algorithm is slow; while the latter has poor anti-interference ability and low algorithm recognition accuracy.

Summary of the invention

The present application provides a method and device for human posture recognition, which are used to solve the technical problem that the prior art cannot take into account both high recognition accuracy and fast recognition speed at the same time.

In view of this, the first aspect of the present application provides a human body posture recognition method, comprising:

Combining a preset tracker and a human body detection network to perform human body detection on the target person image frame to obtain a target human body frame, wherein the preset tracker is a network that tracks the human body based on IOU and threshold;

A preset key point prediction network is used to predict key points of a human body in the target human body frame, wherein the preset key point prediction network includes a reference posture estimation network and an interference part classification network;

The human body posture in the target person image frame is recognized according to the human body key points to obtain a human body posture recognition result.

Preferably, the method combines a preset tracker and a human body detection network to perform human body detection on the target person image frame to obtain a target human body frame, wherein the preset tracker is a network for tracking the human body based on IOU and threshold, and also includes:

Acquire multiple person image frames in the surveillance video;

A preprocessing operation is performed on the character image frame to obtain a target character image frame, wherein the preprocessing includes clipping, mean reduction and normalization.

Preferably, the preset tracker and the human body detection network are combined to perform human body detection on the target person image frame to obtain the target human body frame, and the preset tracker is a network that tracks the human body based on IOU and threshold, including:

The preset tracker is used to determine whether the historical IOU of the previous frame is less than the threshold. If so, the human body detection network is used to perform human body detection on the target person image frame, obtain the target person frame, and update the current IOU;

If not, the extended frame obtained by expanding the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels is used as the target human body frame, and the current IOU is updated.

Preferably, the preset key point prediction network is used to predict the key points of the human body in the target human body frame, and the preset key point prediction network includes a reference posture estimation network and an interference part classification network, including:

Obtaining benchmark key points in the target human frame through a benchmark pose estimation network;

Performing interference analysis on the human body torso in the target human body frame according to the interference part classification network to obtain an interference vector;

A weighted calculation is performed based on the reference key points and the interfering vectors to obtain the key points of the human body.

Preferably, the step of recognizing the human body posture in the target person image frame according to the human body key points to obtain the human body posture recognition result includes:

Connecting the human body key points into a human body skeleton graph based on a key point matching algorithm;

The human body posture in the target person image frame is recognized according to the human body skeleton diagram to obtain a human body posture recognition result.

A second aspect of the present application provides a human posture recognition device, comprising:

A human frame detection unit, used to perform human body detection on a target person image frame in combination with a preset tracker and a human body detection network to obtain a target human body frame, wherein the preset tracker is a network that tracks a human body based on IOU and a threshold;

A key point prediction unit, used to predict the key points of the human body in the target human body frame by using a preset key point prediction network, wherein the preset key point prediction network includes a reference posture estimation network and an interference part classification network;

The human body posture recognition unit is used to recognize the human body posture in the target person image frame according to the human body key points to obtain a human body posture recognition result.

Preferably, it also includes:

An image frame acquisition unit, used to acquire multiple person image frames in the surveillance video;

An image preprocessing unit is used to perform a preprocessing operation on the character image frame to obtain a target character The image frame, the preprocessing includes clipping, mean subtraction and normalization.

Preferably, it also includes:

The image preprocessing unit is used to perform a preprocessing operation on the character image frame to obtain a target character image frame, wherein the preprocessing includes clipping, mean reduction and normalization.

Preferably, the human body frame detection unit comprises:

The first judgment subunit is used to judge whether the historical IOU of the previous frame is less than the threshold through the preset tracker. If so, the human body detection network is used to perform human body detection on the target person image frame to obtain the target person frame and update the current IOU;

The second judgment subunit is used to, if not, expand the minimum circumscribed rectangle of the human body key points of the previous frame by a preset number of pixels to obtain an expanded frame as the target human body frame, and update the current IOU.

Preferably, the key point prediction unit comprises:

A reference prediction subunit, used for obtaining reference key points in the target human body frame through a reference pose estimation network;

An interference analysis subunit, configured to perform interference analysis on the human body trunk in the target human body frame according to an interference part classification network to obtain an interference vector;

The weighted calculation subunit is used to perform weighted calculation based on the reference key points and the interfering vectors to obtain the key points of the human body.

Preferably, the human body posture recognition unit comprises:

A connecting subunit, used for connecting the human body key points into a human body skeleton graph based on a key point matching algorithm;

The recognition subunit is used to recognize the human body posture in the target person image frame according to the human body skeleton diagram to obtain a human body posture recognition result.

It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

In the present application, a method for human posture recognition is provided, including: performing human body detection on a target person image frame in combination with a preset tracker and a human body detection network to obtain a target person frame, wherein the preset tracker is a network for tracking a human body based on IOU and a threshold; using a preset key point prediction network to predict human body key points in the target person frame, wherein the preset key point prediction network includes a reference posture estimation network and an interference part classification network; recognizing the human body posture in the target person image frame according to the human body key points to obtain a human body posture recognition result.

The human body posture recognition method provided in this application uses a tracker to cooperate with a human body detection network to detect human body postures. Box detection, tracking the human body based on IOU and threshold can reduce the number of human body detections, reduce redundant calculations, and speed up processing; in the key point prediction process, an interference part classification network is added to perform interference analysis, and key point prediction based on the interference analysis results can improve prediction accuracy; by making targeted improvements to the human posture recognition process, high recognition accuracy and fast recognition speed can be taken into account at the same time. Therefore, this application can solve the technical problem that the existing technology cannot take into account high recognition accuracy and fast recognition speed at the same time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a flow chart of a method for human body posture recognition provided by an embodiment of the present application;

FIG2 is another schematic flow chart of a method for human body posture recognition provided by an embodiment of the present application;

FIG3 is a schematic diagram of the structure of a human posture recognition device provided in an embodiment of the present application;

FIG4 is a schematic diagram of a human frame detection process of the MMpose network model provided in an embodiment of the present application;

FIG5 is a schematic diagram of a human frame detection process combining a preset tracker and a human body detection network provided in an embodiment of the present application;

FIG6 is a schematic diagram of a key point prediction process of a preset key point prediction network provided in an embodiment of the present application.

Detailed ways

In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

For ease of understanding, please refer to FIG. 1 , which shows a first embodiment of a human body posture recognition method provided by the present application, including:

Step 101: Perform human body detection on a target person image frame by combining a preset tracker and a human body detection network to obtain a target human body frame. The preset tracker is a network that tracks a human body based on IOU and a threshold.

The preset tracker is a network that tracks the human body in the image frame based on IOU (Intersection over Union) and the corresponding threshold. The threshold can be set according to actual experience or actual conditions, and is not limited here. IOU is a standard for measuring the accuracy of detecting corresponding objects in a specific data set. As long as a prediction range (bounding boxes, hereinafter referred to as bbox) is obtained in the output, the task can be measured by IoU. By tracking the human body, the preset tracker can avoid multiple human detection networks. The large amount of calculation brought by repeated detection can speed up the algorithm processing speed.

The specific detection principle is that when the preset tracker can track a specific human body, the human body detection network only needs to detect the human body frame once until the tracked human body is lost, that is, it exceeds the preset tracking range, then restart the human body detection network to perform human body frame detection. Such a mechanism can greatly reduce the amount of calculation and improve the running speed.

The improved algorithm of the embodiment of the present application is based on the MMpose network model, which is a top-down type algorithm. Its human frame detection mechanism is to obtain the human body boundary box in the picture through the detection network, and then directly use the human body area map as the input map of the human skeleton key point prediction network for key point prediction calculation. This method is to constantly reread the human body frame detection, resulting in a large amount of calculation, which slows down the running process.

Step 102: predicting the key points of the human body in the target human body frame using a preset key point prediction network, wherein the preset key point prediction network includes a reference posture estimation network and an interference part classification network.

The preset key point prediction network includes two important network layers, namely the reference pose estimation network and the interference part classification network. The reference pose estimation network can directly predict the key points of the human body in the target human body frame. However, in multi-person scenes, there will be overlaps of multiple different parts of the human body. This part of the torso will interfere with the recognition of the subject, resulting in inaccurate prediction of key points. Therefore, in this embodiment, an interference part classification network is added to extract the vector features of each part classification, thereby optimizing the prediction of key points. Specifically, in order not to increase the computational burden of the overall algorithm, a lightweight network can be selected to construct an interference part classification network, which can both improve the prediction accuracy and reduce the amount of calculation.

Step 103: Recognize the human body posture in the target person image frame according to the human body key points to obtain a human body posture recognition result.

The predicted key points of the human body are marked, and the points can be connected according to the marks to form the human skeleton lines. According to the direction of the lines or the geometric shapes formed by the lines, the human body posture in the image frame of the target person can be identified to obtain the human body posture recognition result.

The human posture recognition method provided in the embodiment of the present application performs human frame detection through a tracker in cooperation with a human body detection network. Tracking the human body according to IOU and threshold can reduce the number of human body detections, reduce redundant calculations, and speed up processing. In the key point prediction process, an interference part classification network is added to perform interference analysis, and key point prediction based on the interference analysis results can improve prediction accuracy. By making targeted improvements to the human posture recognition process, both high recognition accuracy and fast recognition speed can be taken into account. Therefore, the embodiment of the present application can solve the technical problem that the prior art cannot take both high recognition accuracy and fast recognition speed into account.

For ease of understanding, please refer to FIG. 2 . The present application provides a second embodiment of a human posture recognition method, including:

Step 201: Acquire multiple character image frames in a surveillance video.

Step 202: preprocess the character image frame to obtain a target character image frame, wherein the preprocessing includes clipping, mean reduction and normalization.

The monitoring device can continuously obtain the video stream, and the monitoring video is selected from the video stream. Through FFmpeg video decoding, multiple image frames can be extracted from the monitoring video. There is a time sequence relationship between adjacent frames, and continuous frames can be detected and key points predicted, so as to judge the human posture and even the behavior of the person. This embodiment mainly analyzes human body movements, so it is sufficient to select the image frame with the person.

The preprocessing operation is to improve the quality of the image frame and facilitate the subsequent human frame detection and key point prediction. In addition to the clipping process, mean reduction process and normalization process proposed in this embodiment, other preprocessing processes can be added according to actual conditions, which are not limited here.

Step 203: determine whether the historical IOU of the previous frame is less than the threshold value through the preset tracker. If so, perform human body detection on the target person image frame through the human body detection network to obtain the target person frame and update the current IOU.

Step 204: If not, the expanded frame obtained by expanding the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels is used as the target human body frame, and the current IOU is updated.

If each image frame is detected once using the human body detection network, then a surveillance video needs to call the human body detection network multiple times, which brings a large amount of calculation. Therefore, whether the human body detection network is used to detect the current target human body frame needs to be judged through historical IOU, that is, the preset tracker is started for judgment and processing, and then it is determined whether the human body detection network is needed to process the frame image.

If the historical IOU of the previous frame is less than the threshold, it means that the overlap between the minimum bounding rectangle of the human key points identified in the previous image frame and the minimum bounding rectangle of the human key points identified in the previous frame is small, then the IOU calculated based on the two minimum bounding rectangles will be less than the threshold, and the detection target may have moved out of the previous image frame, and the human body has left the movement range allowed by the tracker. Therefore, the current frame no longer uses the minimum bounding rectangle of the previous frame to extract the target human body frame, but needs to perform human body detection on the target person image frame through the human body detection network to obtain the target human body frame.

If the historical IOU of the previous frame is not less than the threshold, it means that there is a large overlap between the minimum bounding rectangle of the human key points identified in the previous frame and the minimum bounding rectangle of the human key points identified in the previous frame, so the historical IOU calculated based on the two minimum bounding rectangles is greater than or equal to Threshold, the detection target is still in the image frame, and the current frame can use the minimum circumscribed rectangle of the human key points identified at the previous moment to expand the preset number of pixels to obtain the target human frame in the current frame, without redundant detection through the human detection network.

It should be noted that the first frame image needs to directly use the human body frame detected by the human body detection network, and then use the preset key point prediction network to identify the human body key points. At this time, the expanded frame obtained by expanding the minimum circumscribed rectangle of the human body key points by a certain number of pixels can be regarded as the initial bbox. The human body frame at this time does not have the corresponding IOU calculated by the human body frame at the previous moment, so the IOU of the first frame image processing is directly defined as 1.

It should be explained that bbox is the extended box obtained by expanding the minimum circumscribed rectangle of the human key points by a preset number of pixels, not the target human box extracted from the image frame. Therefore, each bbox is obtained after the human key point prediction is completed and is used to calculate the current IOU corresponding to the current frame. The current IOU is used to determine whether the next frame needs to use the human detection network for target detection.

It should be noted that the expansion is in pixels, and the specific number of pixels to be expanded, that is, the preset number can be set according to actual conditions and is not limited here. Moreover, the expansion refers to expanding outward from the four sides of the minimum circumscribed rectangle at the same time, not just one side.

Please refer to 4, the process of detecting the human body frame is a traditional method, which directly detects the human body frame through the human body detection network (Person detector), and the detection mechanism of this application please refer to Figure 5, where Tracker is a preset tracker that can track the human body, and Pose landmarks is a subsequent preset key point prediction network.

Step 205: Obtain benchmark key points in the target human frame through a benchmark pose estimation network.

Step 206: Perform interference analysis on the human body torso in the target human body frame according to the interference part classification network to obtain an interference vector.

Step 207: Perform weighted calculation based on the reference key points and the interfering vectors to obtain the human body key points.

It should be noted that the preset key point prediction network includes a reference posture estimation network and an interference part classification network. In order to improve the prediction accuracy of the subject target key points when multiple torsos overlap, the present embodiment adds an interference part classification network in the key point prediction process. Please refer to Figure 6, where Image is the target human frame diagram obtained by the above detection. In addition to inputting the reference posture estimation network for key point prediction, the target human frame also needs to input the interference part classification network for interference analysis; then the results of the two are calculated through channel weighting to obtain a more accurate human key point prediction result. In addition, after each current frame image predicts the human key point, it is necessary to extract the minimum circumscribed rectangle of the human key point, and expand the preset number of pixels to obtain the extended frame, and calculate the current IOU with the extended frame corresponding to the previous frame.

The interference part classification network mainly predicts whether each joint point in the frame belongs to the main target, or in other words, the process of classifying the joint points belonging to the main target and the joint points not belonging to the main target. The lightweight network MobileNetV2 is used as the backbone network of the interference part classification network to extract features suitable for the interference part classification task and obtain the interference vector of each joint point:
V＝[ _v0p0 _， _v1p1 _， ..... _vkpk _] ， _vi∈ {0，1}，i∈[0，k]

Among them, _pi is each joint point of the human body, _vi is a binary variable, indicating whether the i-th joint point is an interference joint point. The value 0 means a non-interference joint point, the value 1 means an interference joint point, and k is the number of joint points.

This embodiment makes appropriate modifications to MobileNetV2 to adapt to the interference part classification task, and replaces the 1000-dimensional fully connected classifier used for image classification in the network MobileNetV2 with a 1×1 convolution with k output channels. The loss function of the preset key point prediction network based on the above structure is expressed as:
L＝L _lm +λL _ic

Wherein, L _lm is the loss function of the reference posture estimation network, L _ic is the loss function of the interference part classification network, and λ is the balance factor. Therefore, this embodiment can effectively improve the problem of poor key point prediction effect caused by multi-person interference at a relatively low computational cost.

Step 208: Connect the key points of the human body into a human skeleton graph based on a key point matching algorithm.

Step 209: Recognize the human body posture in the target person image frame according to the human body skeleton diagram to obtain a human body posture recognition result.

The key point matching algorithm can select a suitable algorithm according to the matching accuracy, and is not limited here. The human skeleton image includes lines connected by key points and the original target person image frame, which are superimposed and displayed to facilitate the correspondence of human body parts.

The method provided in this embodiment not only retains the high precision of the Top-down type, but also achieves a faster speed than the Top-down type algorithm, reduces the amount of calculation and power consumption, and makes the algorithm more practical engineering significance.

For ease of understanding, please refer to FIG. 3 . The present application also provides an embodiment of a human posture recognition device, including:

A human frame detection unit 301 is used to perform human body detection on a target person image frame in combination with a preset tracker and a human body detection network to obtain a target human body frame, wherein the preset tracker is a network for tracking a human body based on IOU and a threshold;

A key point prediction unit 302 is used to predict the key points of the human body in the target human body frame using a preset key point prediction network, wherein the preset key point prediction network includes a reference posture estimation network and an interference part classification network;

The human body posture recognition unit 303 is used to recognize the human body posture in the target person image frame according to the human body key points to obtain the human body posture recognition result.

Furthermore, it also includes:

An image frame acquisition unit 304 is used to acquire multiple person image frames in the surveillance video;

The image preprocessing unit 305 is used to perform preprocessing operations on the character image frame to obtain a target character image frame, and the preprocessing includes clipping, mean reduction and normalization.

Furthermore, the human frame detection unit 301 includes:

The first judgment subunit 3011 is used to judge whether the historical IOU of the previous frame is less than the threshold value through the preset tracker. If so, the human body detection is performed on the target person image frame through the human body detection network to obtain the target person frame and update the current IOU;

The second judgment subunit 3012 is used to, if not, expand the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels to obtain an expanded frame as the target human body frame, and update the current IOU.

Furthermore, the key point prediction unit 302 includes:

A reference prediction subunit 3021 is used to obtain reference key points in the target human frame through a reference pose estimation network;

The interference analysis subunit 3022 is used to perform interference analysis on the human body torso in the target human body frame according to the interference part classification network to obtain an interference vector;

The weighted calculation subunit 3023 is used to perform weighted calculation based on the reference key points and the interfering vectors to obtain the human body key points.

Furthermore, the human body posture recognition unit 303 includes:

A connecting subunit 3031 is used to connect the key points of the human body into a human skeleton graph based on a key point matching algorithm;

The recognition subunit 3032 is used to recognize the human body posture in the target person image frame according to the human body skeleton diagram to obtain a human body posture recognition result.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be It can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, or all or part of the technical solution. The computer software product is stored in a storage medium, including a number of instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A human body posture recognition method, characterized by comprising:

Combining a preset tracker and a human body detection network to perform human body detection on the target person image frame to obtain a target human body frame, wherein the preset tracker is a network that tracks the human body based on IOU and threshold;

A preset key point prediction network is used to predict key points of a human body in the target human body frame, wherein the preset key point prediction network includes a reference posture estimation network and an interference part classification network;

The human body posture in the target person image frame is recognized according to the human body key points to obtain a human body posture recognition result.
The human posture recognition method according to claim 1 is characterized in that the preset tracker and the human body detection network are combined to perform human body detection on the target person image frame to obtain the target human body frame, and the preset tracker is a network that tracks the human body based on IOU and threshold, and the method also includes:

Acquire multiple person image frames in the surveillance video;

A preprocessing operation is performed on the character image frame to obtain a target character image frame, wherein the preprocessing includes clipping, mean reduction and normalization.
The human posture recognition method according to claim 1 is characterized in that the preset tracker and the human body detection network are combined to perform human body detection on the target person image frame to obtain the target human body frame, and the preset tracker is a network that tracks the human body based on IOU and threshold, including:

The preset tracker is used to determine whether the historical IOU of the previous frame is less than the threshold. If so, the human body detection network is used to perform human body detection on the target person image frame, obtain the target person frame, and update the current IOU;

If not, the extended frame obtained by expanding the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels is used as the target human body frame, and the current IOU is updated.
The human body posture recognition method according to claim 1 is characterized in that the human body key points are predicted in the target human body frame using a preset key point prediction network, the preset key point prediction network includes a reference posture estimation network and an interference part classification network, including:

Obtaining benchmark key points in the target human frame through a benchmark pose estimation network;

Performing interference analysis on the human body torso in the target human body frame according to the interference part classification network to obtain an interference vector;

A weighted calculation is performed based on the reference key points and the interfering vectors to obtain the key points of the human body.
The human body posture recognition method according to claim 1 is characterized in that the human body posture in the target person image frame is recognized according to the human body key points to obtain the human body posture recognition Results, including:

Connecting the human body key points into a human body skeleton graph based on a key point matching algorithm;

The human body posture in the target person image frame is recognized according to the human body skeleton diagram to obtain a human body posture recognition result.
A human body posture recognition device, characterized by comprising:

A human frame detection unit, used to perform human body detection on a target person image frame in combination with a preset tracker and a human body detection network to obtain a target human body frame, wherein the preset tracker is a network that tracks a human body based on IOU and a threshold;

A key point prediction unit, used to predict the key points of the human body in the target human body frame by using a preset key point prediction network, wherein the preset key point prediction network includes a reference posture estimation network and an interference part classification network;

The human body posture recognition unit is used to recognize the human body posture in the target person image frame according to the human body key points to obtain a human body posture recognition result.
The human body posture recognition device according to claim 6, characterized in that it also includes:

An image frame acquisition unit, used to acquire multiple person image frames in the surveillance video;

The image preprocessing unit is used to perform a preprocessing operation on the character image frame to obtain a target character image frame, wherein the preprocessing includes clipping, mean reduction and normalization.
The human body posture recognition device according to claim 6, characterized in that the human body frame detection unit comprises:

The first judgment subunit is used to judge whether the historical IOU of the previous frame is less than the threshold through the preset tracker. If so, the human body detection network is used to perform human body detection on the target person image frame to obtain the target person frame and update the current IOU;

The second judgment subunit is used to, if not, expand the minimum circumscribed rectangle of the human body key points of the previous frame by a preset number of pixels to obtain an expanded frame as the target human body frame, and update the current IOU.
The human body posture recognition device according to claim 6, characterized in that the key point prediction unit comprises:

A reference prediction subunit, used for obtaining reference key points in the target human body frame through a reference pose estimation network;

An interference analysis subunit, used for performing interference analysis on the human body trunk in the target human body frame according to the interference part classification network to obtain an interference vector;

A weighted calculation subunit is used to perform weighted calculation based on the reference key point and the interfering vector. Calculate and get the key points of the human body.
The human body posture recognition device according to claim 6, characterized in that the human body posture recognition unit comprises:

A connecting subunit, used for connecting the human body key points into a human body skeleton graph based on a key point matching algorithm;

The recognition subunit is used to recognize the human body posture in the target person image frame according to the human body skeleton diagram to obtain a human body posture recognition result.