CN102609956B - Editing method for human motions in videos - Google Patents

Abstract

The invention discloses an editing method for human motions in videos, which supports users to edit human motions in videos so as to obtain videos containing new human motions. The method concretely comprises the following three steps (namely, human posture analysis, posture data editing, and image deformation): firstly, by using an immune evolution based hierarchical posture analysis method, through global posture optimization, posture sequence smoothing and local attitude optimization calculation, obtaining the posture data of a human motion inputted in a video; then, editing the calculated posture data of the human motion in a way of sketch interaction so as to generate new posture data of the human motion; and finally, deforming each frame of image inputted in the video by using a model-driven moving least square image deformation method so as to generate a video containing a new human motion. By using the method disclosed by the invention, a human motion in a video can be edited so as to obtain a video containing a new human motion, therefore, the method has important applications in the fields such as film post-production and advertisement design.

Description

Human motion editing method in video

Technical Field

The invention relates to a method for editing human body movement in a video, belongs to the field of computer image video processing research, and particularly relates to a method for supporting a user to edit the human body movement in the video.

Background

Image editing techniques have been widely applied in real life, for example, using an editing tool such as Photoshop to implement various editing operations such as tone adjustment, structure adjustment, image restoration, image denoising, image deformation, etc. on an image. However, currently existing editing tools for video, such as Adobe's Effects and Apple's Shake, can only implement basic operations such as matting and foreground extraction. Editing of video content, such as the appearance and movement of characters in video, remains a very challenging task. However, video editing has important applications in the fields of advertisement design, movie post-production and the like, and the potential application value makes video editing one of the important subjects in the field of computer image and video processing research.

Currently, the existing research of video editing mainly focuses on three aspects of basic motion editing, motion special effect processing, specific object editing and the like. As in document 1: SCHOLLZ, V, EL-ABED, S, SEIDEL, H-P, AND MAGNOR, M.A. editing object is viewer in video sequences CGF,2009.28(6): 1632-1643 implements some basic video editing methods, such as removing AND adding objects, motion editing of objects, non-rigid object deformation, key frame interpolation, AND processing of camera motion, etc., AND then such operations can only implement basic editing of video AND cannot meet the requirements such as editing the appearance AND posture of human body; document 2: WANG, J., DRUCKER, S.M., AGRAWALA, M., AND COHEN, M.F. the vehicle animation filter ACM TOG,2006,25(3): 1169-; document 3: LEYVAND, T, COHEN-OR, D, DROR, G, AND LISCHINSKI, D.Data-drive enhancement of facial features ACM TOG,2008,27(3): 1-9 learns a two-dimensional deformation function from a face image training set, can deform the face in an input image to increase the attractiveness of the expression, but the method can only process the face image and cannot be expanded into human body editing for example. In addition, in recent years, studies on human appearance editing in images and videos have also appeared, as in document 4: ZHOU, S, FU, H, LIU, L, COHEN-OR, D, AND HAN, X.Parametric rehabilitation of human bodies in images, ACMTOG,2010,29(4) 1-10 introduce a parameterized three-dimensional human body model, AND realize the direct editing of the external appearance of the human body by adjusting the appearance attribute of the human body; document 5: ARJUN Jain, Thorsten Thorm ahlen, Hans-Peter Seidel and clinical theory, movieReshape: Tracking and rehaping of Humans in video. ACMTOG,2010,29(5) extended the work of document 4 to video, enabling editing of the human appearance in video, but this method was not able to edit the pose of the human body.

In summary, the current video editing methods mainly focus on basic motion editing, motion special effect processing, specific object editing, and the like, and editing for human bodies is also limited to editing of human body appearances, and research on human body gestures and motion editing in videos is not yet visible. This is mainly because video human motion editing is a very challenging task, including the following reasons: firstly, keeping consistency of human skeleton and appearance between video frames is a primary difficulty in video human motion editing; secondly, how to provide an effective user interface is an important challenge facing research. However, video human motion editing has important application in the fields of advertisement design and movie production, and human posture and motion editing technology can support artists to describe storylines through movie editing in a more free artistic manner. The important academic value and the application prospect enable the video human posture editing to become the research content of the invention.

Disclosure of Invention

The purpose of the invention is as follows: the invention provides a human motion editing method in a video aiming at the defects of the prior art.

The technical scheme is as follows: the invention discloses a human motion editing method in a video, which comprises the following steps:

step one, analyzing human body postures: calculating the human body posture of each frame of image in the input video by adopting a hierarchical posture optimization method based on immune evolution to obtain posture data of human body motion;

secondly, editing the attitude data: editing the posture data of the human body movement obtained in the step one in a sketch interaction mode to generate new posture data of the human body movement;

step three, image deformation: and D, taking the new posture data of the human body motion in the step two as a control condition, and adopting a model-driven moving least square image deformation method to deform each frame of image in the input video to generate the video containing the new human body motion.

The hierarchical posture analysis method based on immune evolution in the step one belongs to a generating posture analysis method, a human body model is defined explicitly, and the human body posture estimation is realized by optimizing the fitness between the projection of the human body model and the image characteristics. The layered optimization idea is to obtain the posture data of the human motion in the input video through global posture optimization, posture sequence smoothing and local posture optimization calculation. Due to the utilization of the time sequence information of the movement and the local optimization of the posture, the accuracy of the posture analysis is effectively improved. The method comprises the following steps:

step 21, global attitude optimization: for each frame image z of the input video_tCalculating to obtain a candidate posture set containing N candidate postures by adopting an immune evolution posture optimization method

Wherein,

the method comprises the steps that a candidate gesture is obtained, i is the serial number of the candidate gesture, i is 1, the integer N is the total number of the candidate gestures, and T is 1, the integer T is the total frame number of an input video;

step 22, smoothing the attitude sequence: from the candidate attitude set, using a dynamic programming based attitude sequence smoothing methodTo select one candidate pose as image z_tEstimated attitude of

Wherein h (t) represents the estimated attitude

In a set of candidate poses

Number of (1), h (t) e [1, N]；

Step 23, local attitude optimization: using estimated attitude

Repositioning image z_tHuman body contour of (1); calculating the local posture of the human body by adopting an immune evolution posture optimization method to obtain an image z_tPosture data y of human body motion_t。

The immune evolution posture optimization method in the step 21 is an optimization method integrating an evolution mechanism and an immune mechanism, improves the convergence and the local optimization capability of posture optimization by introducing a posture vaccine and an immune operator, and specifically comprises the following steps:

step 31, initialization: randomly generating M posture individuals within the constraint range defined by vaccine V

Is recorded as a population A₀(ii) a Wherein,

is a posture individual, M is the serial number of the posture individual, M is the total number of the posture individual in the population, namely the scale of the population, 0 represents the algebra of the population, y represents the algebra of the population_jJ, J being the dimension of the posed individual, J1. -; the vaccine V defines the posture individual

Is the value range of each dimension, namely min (y)_j)<y_j<max(y_j)，min(y_j) And max (y)_j) Are each y_jMinimum and maximum values of the values;

step 32, iteratively optimizing population A₀The method comprises the following steps:

step 321, calculating the kth generation population A_kThe fitness of each posture individual in the population, if the current population A_kIf the posture of the individual is the best posture, stopping running and outputting a result, otherwise, continuing; the fitness of the attitude individual refers to the distance between an attitude model corresponding to the attitude individual and the image characteristics, and the fitness of the attitude individual is calculated by adopting the contour and the edge characteristics

The calculation method is <math> <mrow> <mi>E</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mi>&Sigma;</mi> <mrow> <mo>(</mo> <msup> <mi>w</mi> <mi>s</mi> </msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>+</mo> <msup> <mi>w</mi> <mi>e</mi> </msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Wherein,is the k generation population A_kOne of the posture individuals is in a state of,

for the contour-based distance calculation formula,

is an edge-based distance calculation formula; the best posture individual refers to fitnessThe attitude individuals are smaller than a threshold value e, and the value range of the threshold value e is 0.001-0.002;

step 322, genetic operator: adopting Gaussian cross mutation operator in genetic operator to pair the kth generation population A_kEach posture individual in the group B is subjected to cross variation operation to obtain an intermediate population B_k(ii) a Wherein

(y_t′)^m,kThe posture individuals obtained by the cross mutation operation;

step 323, vaccination: for intermediate population B_kEach posture individual in (a) is inoculated with the vaccine V to obtain an antibody population C_k(ii) a Wherein

(y_t′′)^m,kPosture individuals obtained by vaccination with V; the vaccination V refers to the individual in posture (y)_t′)^m,k，(y_t′)^m,k∈B_kIf, if

ThenIf it is not ${\left(y_t^{\text{'}}\right)}_j^{m,k}>\max \left(y_j\right),$ Then ${\left(y_t^{\text{'}}\right)}_j^{m,k}=\max \left(y_j\right);$

Step 324, immunoselection: for antibody population C_kPerforming immunodetection and annealing selection to obtain temporary population D_k(ii) a The immunoassay refers to the antibody population C_kEach posture of (y)_t′′)^m,kIf the individual is in posture (y)_t′′)^m,kIs improved, i.e. E ((y)_t′′)^m,k,z_t)-E((y_t′)^m,k,z_t)>0, then the individual (y)_t′′)^m,kAdding temporary population D_k(ii) a The annealing selection refers to the antibody population C_kEach posture of (y)_t′′)^m,kIf the individual is in posture (y)_t′′)^m,kIs reduced, i.e. E ((y)_t′′)^m,k,z_t)-E((y_t′)^m,k,z_t)<0, then with probability P ((y)_t′′)^m,k) Will pose to the individual (y)_t′′)^m,kAdding temporary population D_kProbability of occurrence <math> <mrow> <mi>P</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>)</mo> </mrow> <mo>=</mo> <msup> <mi>e</mi> <mfrac> <mrow> <mi>E</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>T</mi> <mi>k</mi> </msub> </mfrac> </msup> <mo>/</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msup> <mi>e</mi> <mfrac> <mrow> <mi>E</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>T</mi> <mi>k</mi> </msub> </mfrac> </msup> <mo>;</mo> </mrow> </math> Wherein, E ((y)_t′′)^m,k,z_t) Is the fitness function in step 321, T_kFor temperature-controlled sequences, T_k＝ln(_T0k+1)，T₀Is the initial temperature, k is the algebra of the population;

step 325, constructing a new population: the following steps are adopted to construct a new generation male parent population A_k+1: statistics of temporary population D_kThe number of middle posture individuals M'; for population A_kAnd a temporary population D_kDeleting the attitude individuals with the total number of M + M ', wherein the same attitude individuals in the M + M' attitude individuals are deleted; calculating the fitness of the remaining posture individuals, and selecting the first M posture individuals with the highest fitness to form a new generation male parent population A_k+1；

Step 326, when the iteration number k is smaller than the threshold C, k equals k +1, and the process returns to step 321; otherwise, executing the step 33, wherein the value range of the threshold value C is 80-120;

step 33, outputting: computing the kth generation population A_kThe first N attitude individuals with the highest fitness are selected to form a candidate attitude set

The gesture sequence in step 22 is smooth, and the accuracy of gesture analysis can be improved by introducing motion time sequence information to constrain gesture estimation, and the method specifically comprises the following steps:

step 41, from the set of candidate poses

To select a candidate pose

As image z_tThe estimated attitude of the vehicle; an estimated pose sequence H (1) H (2.) H (t) is constructed for the input video, where H (t) e [1, N ]]Representing an image z_tThe sequence number of the estimated attitude;

step 42, calculating image feature constraint f_h(t): calculating an image z_tSet of candidate poses of

Each candidate gesture in

Is adapted to

The image feature constraint calculation method is as follows: <math> <mrow> <msub> <mi>f</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </msub> <mo>=</mo> <mi>E</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>/</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <mi>E</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mi>i</mi> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>;</mo> </mrow> </math>

step 43, calculate the space-time constraint O_h(t)h(t+1): calculating an image z_tEstimated attitude of

And image z_t+1Estimated attitude of

Space-time constraints between <math> <mrow> <msub> <mi>O</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </msub> <mo>=</mo> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mi>Dis</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>,</mo> <msubsup> <mi>y</mi> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>)</mo> </mrow> <mo>/</mo> <msup> <mi>&sigma;</mi> <mn>2</mn> </msup> <mo>)</mo> </mrow> <mo>;</mo> </mrow> </math> Wherein, the sigma is a nuclear parameter,is a postureAnd posture

The distance between the two sensors is calculated by the following method: <math> <mrow> <mi>Dis</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>,</mo> <msubsup> <mi>y</mi> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>=</mo> <mn>0</mn> </mrow> <mi>J</mi> </munderover> <msub> <mi>w</mi> <mi>j</mi> </msub> <msup> <mrow> <mo>|</mo> <mo>|</mo> <mi>log</mi> <msup> <mrow> <mo>(</mo> <msubsup> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mi>t</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </msup> <msubsup> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <mo>,</mo> </mrow> </math> wherein y is_jJ1, J, w for each dimension of pose_jIs the weight of each dimension.

Step 44, establishing an evaluation function: constraint f based on image features_h(t)And space-time constraint O_h(t)h(t+1)Establishing an evaluation function of the estimated attitude sequence H ═ H (1) H (2.. H (T).)

Wherein alpha is a weight control parameter, the value of alpha is inversely proportional to the human posture change speed in the input video, and the value range is 0.5-2.

Step 45, solving the attitude sequence: solving the attitude sequence H by adopting a dynamic programming method, and calculating to obtain each frame of image z_tEstimated attitude of

In the local posture optimization in the step 23, the human body contour in the pre-estimated posture repositioning image is firstly utilized, then the immune evolution posture optimization method is adopted to optimize the local posture of the human body, and the precision of the posture analysis is improved, and the method comprises the following steps:

step 51, contour repositioning: using estimated attitude

Generating a corresponding pose model and projecting the pose model onto a corresponding image z_tAn image plane of (a); carrying out image erosion and expansion operation on a projection region of the attitude model, marking a region obtained by the erosion operation as a foreground, and marking a part except the region obtained by the expansion operation as a background; calculating to obtain an image z by utilizing an image segmentation method according to the marked foreground and background_tHuman body contour s in_t′；

Step 52, population initialization: using image z_tEstimated attitude of

Obtaining M attitude individuals conforming to normal distribution by adopting Gaussian prediction method

In combination with

Initializing population A₀；

Step 53, local attitude optimization: using the body contour s obtained in step 51_tCalculating the fitness of the attitude individuals, and calculating by adopting an immune evolution attitude optimization method to obtain a candidate attitude set

Step 54, gesture generation: from a set of candidate poses

Selecting the posture individual y with the highest fitness_tAs image z_tPosture data of the human body motion.

Editing the attitude data in the second stepAnd editing the posture data of the human body motion obtained in the first step by adopting a sketch interaction mode to generate new posture data of the human body motion, wherein the new posture data of the human body motion is used as a constraint condition of image deformation in the third step. The sketch interaction mode accords with the interaction habit of the user and has good usability. The method specifically comprises the following steps: posture data y of the human body motion obtained by the posture analysis in the step one_tSelecting K key postures y by adopting a sketch interaction mode_k，k＝1,...,K，K<T; for each key gesture y_kSpecifying a new position of a terminal joint point of a key part of a human body by adopting a sketch interaction mode; solving by adopting a multi-priority inverse kinematics method to obtain new human body posture data y_k'; with y_k' Quaternary number interpolation method is adopted for key frames to analyze the posture data y of the human motion obtained in the step-posture analysis_tInterpolation is carried out to generate new posture data y of human body movement_t′。

In the third step, the model-driven moving least square image deformation method refers to a moving least square image deformation method which takes the edited new posture data of human motion as a deformation control condition and respectively adopts a line segment deformation-based and control point deformation-based moving least square image deformation method to deform each frame of an input video so as to obtain a video containing new human motion, and specifically comprises the following steps:

step 71, analyzing the posture data y of the human body motion obtained in the step one_tSkeletonally drawn to image z_tObtaining a line segment set b in the image_t,r(ii) a Editing the posture data y of the human body motion obtained in the step two postures_t' skeletal rendering to image z_tObtaining a line segment set b in the image_t′_,r(ii) a Wherein R is 1, R is the number of human skeleton segments;

step 72, collect line segment b_t,rAs the initial position of the control line segment, set b of line segments_t′_,rUsing, as target positions for control line segments, line-segment-deformation-basedMoving least squares image deformation method, for image z_tCarrying out deformation to obtain a deformed video frame image z_t′；

Step 73, adopting a Gaussian background modeling method to calculate the deformed video frame image z_t' human body contour s_t''; using the edited posture data y of human body movement_t' generating a corresponding pose model, projecting the pose model to the deformed video frame image z_t' in the image plane; calculating to obtain the outer contour u of the posture model projection_t；

Step 74, contour s of human body_t'' equidistant sampling obtains sampling point set

Outer contour u projected on model_tEquidistant sampling is carried out to obtain a sampling point set

Wherein Q is 1.., Q is the number of sampling points;

step 75, set of sampling points

As the initial position of the control point, the sampling point set

As the target position of the control point, the moving least square image deformation method based on the control point deformation is utilized to carry out the deformation on the video frame image z_t' deformation to get each frame of image z of video containing new human motion_t′′。

Has the advantages that: the invention discloses a method for editing human body movement in a video, which supports a user to edit the human body movement in the video to obtain the video containing new human body movement. The invention has the following characteristics: 1. the user is supported to adopt a natural sketch interaction mode to conveniently edit the human body movement in the video to obtain the video containing the new human body movement; 2, the immune evolution hierarchical posture analysis method has good convergence and global optimality, and can automatically and accurately analyze the human body posture in the video; 3, the moving least square image deformation method driven by the model can effectively ensure the authenticity of the human body posture and the appearance after the image is deformed, so that the visual effect of the generated new video is more natural. The invention has important application in the fields of movie post production, advertisement design and the like.

Drawings

The foregoing and/or other advantages of the invention will become further apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.

Fig. 1 is a main flow chart of the present invention.

FIG. 2 is a flowchart of the immune evolution posture optimization method of the present invention.

Fig. 3a, fig. 3b and fig. 3c are diagrams of human body posture data results obtained by adopting the layered posture analysis method of the invention for three-pin images of an input video.

Fig. 4a is the posture data of the human body movement obtained by the posture analysis of the present invention, and fig. 4b is the new posture data of the human body movement obtained by editing the posture data.

The specific implementation mode is as follows:

the invention discloses a scheme for performing posture modeling and editing and motion generation by adopting a hand-drawn sketch, which comprises the following steps of:

step one, analyzing human body postures: calculating the human body posture of each frame of image in the input video by adopting a hierarchical posture optimization method based on immune evolution to obtain posture data of human body motion;

secondly, editing the attitude data: editing the posture data of the human body movement obtained in the step one in a sketch interaction mode to generate new posture data of the human body movement;

step three, image deformation: and D, taking the new posture data of the human body motion in the step two as a control condition, and adopting a model-driven moving least square image deformation method to deform each frame of image in the input video to generate the video containing the new human body motion.

More specifically, the invention discloses a method for editing human body movement in a video, which supports a user to edit the human body movement in the video to obtain a video containing new human body movement, and the method mainly relates to three key technologies of human body posture analysis, posture data editing and image deformation, and the processing flow is shown in fig. 1. Firstly, analyzing human body postures, and calculating by adopting a hierarchical posture analysis method based on immune evolution through a hierarchical processing strategy of global posture optimization, posture sequence smoothing and local posture optimization to obtain posture data of human body movement in an input video; secondly, editing posture data, namely editing the calculated posture data of the human body motion in a sketch interaction mode to generate new posture data of the human body motion; and thirdly, image deformation, namely deforming each frame of image in the input video by adopting a model-driven moving least square image deformation method to generate a video containing new human body motion. The main processes of the embodiments are described below:

1. human body posture analysis

The human body posture of each frame of image of the input video is calculated by adopting a hierarchical posture analysis method based on immune evolution, and the method specifically comprises three steps of global optimization, posture sequence smoothing and local posture optimization.

1.1 Global attitude optimization

Global pose optimization belongs to a generative pose analysis method, which explicitly defines a human body model by optimizing the human body modelThe fitness between the projection and the image features of the model realizes the estimation of the human body posture. The invention uses SCAPE model (document 6: ANGULOV, D., SRINIVASAN, P., KOLLER, D., THRUN, S., RODGERS, J., AND DAVIS, J.2005.SCAPE: Shape compatibility AND Evaluation of peer. ACM TOG,2005,24(3):408-416.), AND uses contour AND edge features to calculate fitness of individuals (document 7: Leonid Sigal, Alexandru O.Balan, Michael J.Black, HUMANEVA: Synchronized de AND motion Capture Dataset AND base Algorine for Evaluation of organized Human motion. int JComp Vis,2010,87: 4-27), AND the optimization method is immune attitude optimization method (document 8: Joule formation, Du Pond, Liu et al., immune recognition, published by Lei., Liu scientific Co., Ltd.). For each frame image z of the input video_tCalculating to obtain a candidate posture set containing N candidate postures by adopting an immune evolution posture optimization methodThe flow chart of the immune evolution posture optimization method is shown in fig. 2, and specifically comprises the following steps:

step 31, initialization: randomly generating M posture individuals within the constraint range defined by vaccine VIs recorded as a population A₀(ii) a Wherein,

is a posture individual, M is the serial number of the posture individual, M is the total number of the posture individuals in the population, 0 represents the algebra of the population, y_jJ, J being the dimension of the posed individual, J1. -; the vaccine V defines the posture individual

Is the value range of each dimension, namely min (y)_j)<y_j<max(y_j)，min(y_j) And max (y)_j) Are each y_jMinimum and maximum values of the values; in the implementation of the invention, the dimension J of the posture individuals is 69, and the total number M of the posture individuals in the population is 200 min (y)_j) And max (y)_j) Obtained by the statistics of the motion data.

Step 32, iteratively optimizing population A₀The method comprises the following steps: (ii) a

Step 321, calculating the kth generation population A_kThe fitness of each posture individual in the population, if the current population A_kIf the posture of the individual is the best posture, stopping running and outputting a result, otherwise, continuing; the fitness of the attitude individual refers to the distance between an attitude model corresponding to the attitude individual and the image characteristics, and the fitness of the attitude individual is calculated by adopting the contour and the edge characteristics

The calculation method is <math> <mrow> <mi>E</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mi>&Sigma;</mi> <mrow> <mo>(</mo> <msup> <mi>w</mi> <mi>s</mi> </msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>+</mo> <msup> <mi>w</mi> <mi>e</mi> </msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Wherein,

is the k generation population A_kOne of the posture individuals is in a state of,

for contour-based distance calculation, superscript s is an abbreviation for the contour english word silhouette,for the edge-based distance calculation formula, superscript e is an abbreviation for the edge english word edge; the best posture individual refers to fitness

The posture individuals are smaller than a threshold value e, and the threshold value e is 0.0015;

step 322, genetic operator: (iii) applying Research of Computers,2008,25(010):2911-_kEach posture individual in the group B is subjected to cross variation operation to obtain an intermediate population B_k(ii) a Wherein

(y_t′)^m,kThe posture individuals obtained by the cross mutation operation;

step 323, vaccination: for intermediate population B_kEach posture individual in (a) is inoculated with the vaccine V to obtain an antibody population C_k(ii) a Wherein

(y_t′′)^m,kPosture individuals obtained by vaccination with V; the vaccination V refers to the individual in posture (y)_t′)^m,k，(y_t′)^m,k∈B_kIf, ifThen

If it is not ${\left(y_t^{\text{'}}\right)}_j^{m,k}>\max \left(y_j\right),$ Then ${\left(y_t^{\text{'}}\right)}_j^{m,k}=\max \left(y_j\right);$

Step 324, immunoselection: for antibody population C_kPerforming immunodetection and annealing selection to obtain temporary population D_k(ii) a The immunoassay refers to the antibody population C_kEach posture of (y)_t′′)^m,kIf the individual is in posture (y)_t′′)^m,kIs improved, i.e.E((y_t′′)^m,k,z_t)-E((y_t′)^m,k,z_t)>0, then the individual (y)_t′′)^m,kAdding temporary population D_k(ii) a The annealing selection refers to the antibody population C_kEach posture of (y)_t′′)^m,kIf the individual is in posture (y)_t′′)^m,kIs reduced, i.e. E ((y)_t′′)^m,k,z_t)-E((y_t′)^m,k,z_t)<0, then with probability P ((y)_t′′)^m,k) Will pose to the individual (y)_t′′)^m,kAdding temporary population D_kProbability of occurrence <math> <mrow> <mi>P</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>)</mo> </mrow> <mo>=</mo> <msup> <mi>e</mi> <mfrac> <mrow> <mi>E</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>T</mi> <mi>k</mi> </msub> </mfrac> </msup> <mo>/</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msup> <mi>e</mi> <mfrac> <mrow> <mi>E</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>T</mi> <mi>k</mi> </msub> </mfrac> </msup> <mo>;</mo> </mrow> </math> Wherein, E ((y)_t′′)^m,k,z_t) Is the fitness function in step 321, T_kFor temperature-controlled sequences, T_k＝ln(T₀k+1)，T₀Is the initial temperature, k is the algebra of the population; initial temperature T in the implementation of the invention₀＝1000。

Step 325, constructing a new population: the following steps are adopted to construct a new generation male parent population A_k+1: statistics of temporary population D_kThe number of middle posture individuals M'; for population A_kAnd a temporary population D_kDeleting the attitude individuals with the total number of M + M ', wherein the same attitude individuals in the M + M' attitude individuals are deleted; calculating the fitness of the remaining posture individuals, and selecting the first M posture individuals with the highest fitness to form a new generation male parent population A_k+1；

Step 326, when the iteration number k is smaller than the threshold C, k equals k +1, and the process returns to step 321; otherwise, executing step 33; in the implementation of the invention, the threshold value C is 100;

step 33, outputting: computing the kth generation population A_kThe first N attitude individuals with the highest fitness are selected to form a candidate attitude setIn the implementation of the present invention, the size N of the candidate pose set is 20.

1.2 pose sequence smoothing

The calculation process of the gesture sequence smoothing is to collect the candidate gestures

To select a candidate pose

As image z_tThe estimated attitude of the system is to construct an estimated attitude sequence H-H (1) H (2.) H (T) and establish an evaluation function thereof, and then optimize the estimated attitude sequence by adopting a dynamic programming method (document 10: Panjingui, consider iron, Licheng law and other translation, algorithm introduction, mechanical industry publisher, 2006, page 191 and 212) to obtain the optimal estimated attitude sequence. The gesture sequence smoothly utilizes the time sequence information of the movement, can improve the accuracy of human body gesture analysis, and specifically comprises the following steps:

step 41, from the set of candidate poses

To select a candidate pose

As image z_tThe estimated attitude of the vehicle; an estimated pose sequence H (1) H (2.) H (t) is constructed for the input video, where H (t) e [1, N ]]Representing an image z_tThe sequence number of the estimated attitude;

step 42, calculating image feature constraint f_h(t): calculating an image z_tSet of candidate poses ofEach candidate gesture inIs adapted to

The image feature constraint calculation method is as follows: <math> <mrow> <msub> <mi>f</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </msub> <mo>=</mo> <mi>E</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>/</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <mi>E</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mi>i</mi> </msubsup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>;</mo> </mrow> </math>

step 43, calculate the space-time constraint O_h(t)h(t+1): calculating an image z_tEstimated attitude ofAnd image z_t+1Estimated attitude of

Space-time constraints between <math> <mrow> <msub> <mi>O</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </msub> <mo>=</mo> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mi>Dis</mi> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>,</mo> <msubsup> <mi>y</mi> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>t</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </msubsup> <mo>)</mo> </mrow> <mo>/</mo> <msup> <mi>&sigma;</mi> <mn>2</mn> </msup> <mo>)</mo> </mrow> <mo>;</mo> </mrow> </math> Wherein, the sigma is a nuclear parameter,

is a posture

And posture

The distance between the two sensors is calculated by the following method: <math> <mrow> <mi>P</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>)</mo> </mrow> <mo>=</mo> <msup> <mi>e</mi> <mfrac> <mrow> <mi>E</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>T</mi> <mi>k</mi> </msub> </mfrac> </msup> <mo>/</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msup> <mi>e</mi> <mfrac> <mrow> <mi>E</mi> <mrow> <mo>(</mo> <msup> <mrow> <mo>(</mo> <msubsup> <mi>y</mi> <mi>t</mi> <mrow> <mo>'</mo> <mo>'</mo> </mrow> </msubsup> <mo>)</mo> </mrow> <mrow> <mi>m</mi> <mo>,</mo> <mi>k</mi> </mrow> </msup> <mo>,</mo> <msub> <mi>z</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>T</mi> <mi>k</mi> </msub> </mfrac> </msup> <mo>;</mo> </mrow> </math> wherein y is_jJ1, J, w for each dimension of pose_jA weight for each dimension; in the implementation of the invention, the kernel parameter σ is 10, and different weights w are set for different joint points_jWherein the corresponding dimensionalities of the key joint points including the root joint, the knee joint, the elbow joint, the cross joint and the head-shoulder joint are set to be 1, and the weights of other joints are set to be 0;

step 44, establishing an evaluation function: constraint f based on image features_h(t)And space-time constraint O_h(t)h(t+1)Establishing an evaluation function of the estimated attitude sequence H ═ H (1) H (2.. H (T).)

Wherein alpha is a weight control parameter, the value is related to the human posture change speed in the input video, for the video with high human motion speed, such as running, the value of a is 2, for the video with low motion speed, such as walking, the value of a is 0.5, and most other videos have the value of a being 1. In the implementation of the invention, take α ═ 0.5;

step 45, solving the attitude sequence: solving the attitude sequence H by adopting a dynamic programming method, and calculating to obtain each frame of image z_tEstimated attitude of

1.3 local pose optimization

The basic idea of local posture optimization is to optimize the local posture of a human body by improving the accuracy of image characteristic calculation and adopting an immune evolution posture optimization method, and the processing flow is as follows: the method comprises the following steps of (1) projecting and marking a video frame image based on a posture model of a pre-estimated posture, calculating an accurate human body contour by adopting a graph cutting method, and realizing local optimization by utilizing an immune evolution posture optimization method, wherein the method specifically comprises the following steps:

step 51, wheelContour repositioning: using estimated attitude

Generating a corresponding pose model and projecting the pose model onto a corresponding image z_tAn image plane of (a); carrying out image erosion and expansion operation on a projection region of the attitude model, marking a region obtained by the erosion operation as a foreground, and marking a part except the region obtained by the expansion operation as a background; calculating to obtain an image z by utilizing an image segmentation method according to the marked foreground and background_tHuman body contour s in_t′；

Step 52, population initialization: using image z_tEstimated attitude of

Obtaining M attitude individuals conforming to normal distribution by adopting Gaussian prediction method

In combination with

Initializing population A₀；

Step 53, local attitude optimization: using the body contour s obtained in step 51_tCalculating the fitness of the attitude individuals, and calculating by adopting an immune evolution attitude optimization method to obtain a candidate attitude set

Step 54, gesture generation: from a set of candidate poses

Selecting the posture individual y with the highest fitness_tAs image z_tPosture data of the human body motion.

Fig. 3a, fig. 3b and fig. 3c are diagrams of human body posture data results obtained by adopting the layered posture analysis method of the invention for three-pin images of an input video.

2. Pose data editing

The invention applies the sketch interaction mode to the posture data editing, and improves the operability of the user. The process of the posture editing comprises the steps of firstly, specifying the key posture of posture data of human motion, then specifying the position of a human target joint point of the key posture in a sketch interaction mode, further solving a new human posture by adopting an inverse kinematics method, and finally interpolating original posture data to obtain new posture data of the human motion, wherein the process specifically comprises the following steps:

step 61, obtaining the posture data y of the human body movement by the posture analysis in the step one_tSelecting K key postures y by adopting a sketch interaction mode_k，k＝1,...,K，K<T; in the present invention, the key pose number K is typically taken.

Step 62, for each key posture yk, a new position of the terminal joint point of the key part of the human body is specified in a sketch interaction mode;

step 63, solving and obtaining new human body posture data y by using a multi-priority inverse kinematics method (document: 11: Paolo Baillocher, Ronan Boulic, An inverse kinematic architecture engineering for describing An architecture number of gradient priorities, the visual Computer,2004,20(6):402-_k′；

Step 64, with y_k' Quaternary number interpolation method is adopted for key frames to analyze the posture data y of the human motion obtained in the step-posture analysis_tInterpolation is carried out to generate new posture data y of human body movement_t′。

3. Image deformation

The idea of image deformation is to use the edited new posture data of human body motion as deformation control conditions, AND sequentially use the line segment deformation-based AND control point deformation-based moving least square image deformation method (document 12: M simple, M., HEIDELBERGER, b., TESCHNER, M., AND general, m.e. spatial transformations based on shape matching. acm TOG,2005,24(3):471 478), to deform each frame image of the input video to obtain a video containing new human body motion, which specifically includes the following steps:

step 71, analyzing the posture data y of the human body motion obtained in the step one_tSkeletonally drawn to image z_tObtaining a line segment set b in the image_t,r(ii) a Editing the posture data y of the human body motion obtained in the step two postures_t' skeletal rendering to image z_tObtaining a line segment set b in the image_t′_,r(ii) a Wherein R is 1, R is the number of human skeleton segments; in the implementation of the invention, the number R of skeleton segments of the human body is 25.

Step 72, collect line segment b_t,rAs the initial position of the control line segment, set b of line segments_t′_,rAs the target position of the control line segment, the image z is subjected to a moving least square image deformation method based on line segment deformation_tCarrying out deformation to obtain a deformed video frame image z_t′；

Step 73, calculating the deformed video frame image z by adopting a Gaussian background modeling method (document 13: C.Stauffer and W.E.L.Grimson, adaptive background mix for real-time tracking, Proc.IEEE CVPR, June1999:246-_t' human body contour s_t''; using the edited posture data y of human body movement_t' generating a corresponding pose model, projecting the pose model to the deformed video frame image z_t' in the image plane; calculating to obtain the outer contour u of the posture model projection_t；

Step 74, contour s of human body_t'' equidistant sampling obtains sampling point set

Outer contour u projected on model_tSampling at equal intervals to obtain sampling pointsCollection

Wherein Q is 1.., Q is the number of sampling points; in the implementation of the present invention, the number of sampling points Q is 150.

Step 75, set of sampling pointsAs the initial position of the control point, the sampling point set

As the target position of the control point, the moving least square image deformation method based on the control point deformation is utilized to carry out the deformation on the video frame image z_t' deformation to get each frame of image z of video containing new human motion_t′′。

The present invention provides a method for editing human body movement in video, and a plurality of methods and ways for implementing the technical solution are provided, and the above description is only a preferred embodiment of the present invention, it should be noted that, for those skilled in the art, a plurality of improvements and embellishments can be made without departing from the principle of the present invention, and these improvements and embellishments should also be regarded as the protection scope of the present invention. All the components not specified in the present embodiment can be realized by the prior art.

Claims (5)

1. A human motion editing method in a video is characterized by comprising the following steps:

step one, analyzing human body postures: calculating the human body posture of each frame of image in the input video by adopting a hierarchical posture optimization method based on immune evolution to obtain posture data of human body motion;

secondly, editing the attitude data: editing the posture data of the human body movement obtained in the step one in a sketch interaction mode to generate new posture data of the human body movement;

step three, image deformation: taking the new posture data of the human body motion in the step two as a control condition, and adopting a model-driven moving least square image deformation method to deform each frame of image in the input video to generate a video containing the new human body motion;

the first step comprises the following steps:

step 21, global attitude optimization: for each frame image z of the input video_tCalculating to obtain a candidate posture set containing N candidate postures by adopting an immune evolution posture optimization method

Wherein,the method comprises the steps that a candidate gesture is obtained, i is the serial number of the candidate gesture, i is 1, the integer N is the total number of the candidate gestures, and T is 1, the integer T is the total frame number of an input video;

step 22, smoothing the attitude sequence: from a set of candidate poses

To select one candidate pose as image z_tEstimated attitude of

Wherein h (t) represents the estimated attitude

In a set of candidate poses

Number of (1), h (t) e [1, N]；

Step 23, local attitude optimization: using estimated attitude

Repositioning image z_tHuman body contour of (1); by using the posture of immune evolutionCalculating the local posture of the human body by a state optimization method to obtain an image z_tPosture data y of human body motion_t；

Step 21 comprises the steps of:

step 31, initialization: randomly generating M posture individuals within the constraint range defined by vaccine V

Is recorded as a population A₀(ii) a Wherein,

is a posture individual, M is the serial number of the posture individual, M is the total number of the posture individuals in the population, 0 represents the algebra of the population, y_jJ, J being the dimension of the posed individual, J1. -; the vaccine V defines the posture individualIs the value range of each dimension, namely min (y)_j)<y_j<max(y_j)，min(y_j) And max (y)_j) Are each y_jMinimum and maximum values of the values;

step 32, iteratively optimizing population A₀The method comprises the following steps:

step 321, calculating the kth generation population A_kThe fitness of each posture individual in the population, if the current population A_kIf the posture of the individual is the best posture, stopping running and outputting a result, otherwise, continuing; the fitness of the attitude individual refers to the distance between an attitude model corresponding to the attitude individual and the image characteristics, and is expressed as

The calculation method is

Wherein,

is the k generation population A_kOne of the posture individuals is in a state of,for the contour-based distance calculation formula,

is an edge-based distance calculation formula; the best posture individual refers to fitness

The attitude individuals are smaller than a threshold value e, and the value range of the threshold value e is 0.001-0.002;

step 322, genetic operator: for the kth generation population A_kEach posture individual in the group B is subjected to cross variation operation to obtain an intermediate population B_k(ii) a Wherein

(y_t′)^m,kThe posture individuals obtained by the cross mutation operation;

step 323, vaccination: for intermediate population B_kEach posture individual in (a) is inoculated with the vaccine V to obtain an antibody population C_k(ii) a Wherein(y_t′′)^m,kThe posture individuals obtained by vaccination V;

the vaccination V refers to the individual in posture (y)_t′)^m,k，(y_t′)^m,k∈B_kIf, if

Then

If it is not

Then

Step 324, immunoselection: for antibody population C_kPerforming immunodetection and annealing selection to obtain temporary population D_k(ii) a The immunoassay refers to the antibody population C_kEach posture of (y)_t′′)^m,kIf the individual is in posture (y)_t′′)^m,kIs improved, i.e. E ((y)_t′′)^m,k,z_t)-E((y_t′)^m,k,z_t)>0, then the individual (y)_t′′)^m,kAdding temporary population D_k(ii) a The annealing selection refers to the antibody population C_kEach posture of (y)_t′′)^m,kIf the individual is in posture (y)_t′′)^m,kIs reduced, i.e. E ((y)_t′′)^m,k,z_t)-E((y_t′)^m,k,z_t)<0, then with probability P ((y)_t′′)^m,k) Will pose to the individual (y)_t′′)^m,kAdding temporary population D_kProbability of occurrence

Wherein E ((y)_t′′)^m,k,z_t) Is the fitness function in step 321, T_kFor temperature-controlled sequences, T_k＝ln(T₀k+1)，T₀Is the initial temperature, k is the algebra of the population;

step 325, constructing a new population: the following steps are adopted to construct a new generation male parent population A_k+1: statistics of temporary population D_kThe number of middle posture individuals M'; for population A_kAnd a temporary population D_kDeleting the attitude individuals with the total number of M + M ', wherein the same attitude individuals in the M + M' attitude individuals are deleted; calculating the fitness of the remaining posture individuals, and selecting the first M posture individuals with the highest fitness to form a new generation male parent population A_k+1；

Step 326, when the iteration number k is smaller than the threshold C, k equals k +1, and the process returns to step 321; otherwise, executing the step 33, wherein the value range of the threshold value C is 80-120;

step 33, outputting: computing the kth generation population A_kThe first N attitude individuals with the highest fitness are selected to form a candidate attitude set

2. The editing method for human motion in video according to claim 1, wherein the step 22 comprises the following steps:

step 41, from the set of candidate poses

To select one candidate pose as image z_tThe estimated attitude of the vehicle; an estimated pose sequence H (1) H (2.) H (t) is constructed for the input video, where H (t) e [1, N ]]Representing an image z_tThe sequence number of the estimated attitude;

step 42, calculating image feature constraint f_h(t): calculating an image z_tSet of candidate poses of

Each candidate gesture in

Is adapted to

The image feature constraint calculation method is as follows:

step 43, calculate the space-time constraint O_h(t)h(t+1): calculating an image z_tEstimated attitude of

And image z_t+1Estimated attitude of

Space-time constraints between

Wherein sigma is a nuclear parameter, and the value range is 1-10;

is a posture

And posture

The distance between the two sensors is calculated by the following method:

wherein y is_jJ1, J, w for each dimension of pose_jA weight for each dimension;

step 44, establishing an evaluation function: constraint f based on image features_h(t)And space-time constraint O_h(t)h(t+1)Establishing an evaluation function of the estimated attitude sequence H ═ H (1) H (2.. H (T).)

Wherein alpha is a weight control parameter, and the value range of alpha is 0.5-2;

step 45, solving the attitude sequence: solving the attitude sequence H, and calculating to obtain each frame of image z_tEstimated attitude of

3. The editing method for human motion in video according to claim 2, wherein the step 23 comprises the following steps:

step 51, contour repositioning: using estimated attitude

Generating a corresponding pose model and projecting the pose model onto a corresponding image z_tAn image plane of (a); carrying out image erosion and expansion operation on a projection region of the attitude model, marking a region obtained by the erosion operation as a foreground, and marking a part except the region obtained by the expansion operation as a background; from the marked foreground and background, image z is computed_tHuman body contour s in_t′；

Step 52, population initialization: using image z_tEstimated attitude of

Obtaining M attitude individuals conforming to normal distribution by adopting Gaussian prediction method

In combination withInitializing a population;

step 53, local attitude optimization: using the body contour s obtained in step 51_tCalculating the fitness of the attitude individuals, and calculating by adopting an immune evolution attitude optimization method to obtain a new candidate attitude set;

step 54, gesture generation: selecting attitude individual y with highest fitness from new candidate attitude set_tAs image z_tPosture data of the human body motion.

4. The editing method for human motion in video according to claim 3, wherein the second step comprises: analyzing the step-attitude to obtainPosture data y of human body motion_tSelecting L key postures y by adopting a sketch interaction mode_l，l＝1,...,L，L<T; for each key gesture y_lSpecifying a new position of a terminal joint point of a key part of a human body by adopting a sketch interaction mode; solving to obtain new human body posture data y_l'; with y_l' Quaternary number interpolation method is adopted for key frames to analyze the posture data y of the human motion obtained in the step-posture analysis_tInterpolation is carried out to generate new posture data y of human body movement_t′。

5. The editing method for human motion in video according to claim 4, wherein the third step comprises the following steps:

step 71, analyzing the posture data y of the human body motion obtained in the step one_tSkeletonally drawn to image z_tObtaining a line segment set b in the image_t,r(ii) a Editing the posture data y of the human body motion obtained in the step two postures_t' skeletal rendering to image z_tObtaining a line segment set b in the image_t′_,r(ii) a Wherein R is 1, R is the number of human skeleton segments;

step 72, collect line segment b_t,rAs the initial position of the control line segment, set b of line segments_t′_,rAs the target position of the control line segment, the image z is subjected to a moving least square image deformation method based on line segment deformation_tCarrying out deformation to obtain a deformed video frame image z_t′；

Step 73, calculating the deformed video frame image z_t' human body contour s_t''; using the edited posture data y of human body movement_t' generating a corresponding pose model, projecting the pose model to the deformed video frame image z_t' in the image plane; calculating to obtain the outer contour u of the posture model projection_t；

Step 74, contour s of human body_t'' equidistant sampling obtains sampling point set

Outer contour u projected on model_tEquidistant sampling is carried out to obtain a sampling point set

Wherein Q is 1.., Q is the number of sampling points;

step 75, set of sampling points

As the initial position of the control point, the sampling point set

As the target position of the control point, the moving least square image deformation method based on the control point deformation is utilized to carry out the deformation on the video frame image z_t' deformation to get each frame of image z of video containing new human motion_t′′。

Images