CN111161200A - Human body posture migration method based on attention mechanism - Google Patents
Human body posture migration method based on attention mechanism Download PDFInfo
- Publication number
- CN111161200A CN111161200A CN201911332748.0A CN201911332748A CN111161200A CN 111161200 A CN111161200 A CN 111161200A CN 201911332748 A CN201911332748 A CN 201911332748A CN 111161200 A CN111161200 A CN 111161200A
- Authority
- CN
- China
- Prior art keywords
- image
- discriminator
- posture
- attention
- generated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013508 migration Methods 0.000 title claims abstract description 27
- 230000005012 migration Effects 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000007246 mechanism Effects 0.000 title claims abstract description 24
- 230000036544 posture Effects 0.000 claims abstract description 67
- 238000012549 training Methods 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 23
- 238000012360 testing method Methods 0.000 claims description 8
- 239000008186 active pharmaceutical agent Substances 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 239000013256 coordination polymer Substances 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000008447 perception Effects 0.000 claims description 3
- 230000003042 antagnostic effect Effects 0.000 claims 1
- 230000015572 biosynthetic process Effects 0.000 abstract description 10
- 238000003786 synthesis reaction Methods 0.000 abstract description 10
- 238000012545 processing Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the field of image synthesis, and aims to realize image synthesis guided by postures and enhance the definition of a generated image and the coincidence degree of the image and a target posture. The invention adopts the technical scheme that a human body posture migration method based on an attention mechanism comprises the following steps: an image preprocessing step: forming training data; attention coding under posture guidance; network building and training: adopting a generated confrontation network model, wherein the network model is divided into a generator and a discriminator; putting the generated image into a discriminator, wherein the discriminator forces the generator to generate a picture closer to reality by distinguishing a real image from the generated image; and finally, the trained generation confrontation network is used for completing the human posture migration. The invention is mainly applied to the image processing occasion.
Description
Technical Field
The invention belongs to the field of image synthesis, and particularly relates to an image synthesis technology aiming at human body posture migration based on an attention mechanism. In particular to a human body posture migration method based on an attention mechanism.
Background
Human pose migration is the generation of an image of a particular person making a specified gesture, which can be used to generate a data set for pedestrian re-recognition and like tasks, to resolve these tasks in a data-driven fashion. In view of their importance, more and more researchers are beginning to focus on the human pose migration task. Unlike image synthesis tasks, human pose migration is a conditional image synthesis task. Given an image containing a person and a fixed pose, the task wishes to generate an image of the person making the specified pose.
Most of the existing human body posture migration methods adopt a codec structure, and under the guidance of an input image and a target two-dimensional posture, certain joint points of human body joints are used for coding to learn the conversion from the input image to the target two-dimensional posture. The mainstream human posture transfer technology mainly comprises two types: the conditional variational self-encoder and the conditional generation countermeasure network. The conditional variational self-encoder can well express the transformation relation among postures, but pictures generated by the method are not clear enough. The conditional generation countermeasure network can produce a clearer picture, but the problem of pixel misalignment caused by posture change cannot be well solved, so that the image with a more complex posture is poor in performance.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to:
1) aiming at the problem of pixel misalignment caused by attitude migration which is difficult to process by the conventional method, the invention utilizes an attention mechanism to reform the interior of an image generator so as to realize attitude-guided image synthesis.
2) In order to fully utilize the image information and generate a clear picture, the invention adopts a framework for generating the countermeasure network and simultaneously enhances the definition of the generated image and the coincidence degree of the image and the target posture.
The invention adopts the technical scheme that a human body posture migration method based on an attention mechanism comprises the following steps:
an image preprocessing step: forming training data;
attention coding under gesture guidance: for image feature CIAnd an attitude feature CPThe gesture features are used for guiding the image features to be transformed by using a self-attention mechanism, and attention codes under gesture guidance are obtained;
network building and training: generating a confrontation network model, wherein the network model is divided into a generator and a discriminator, the generator part firstly carries out down-sampling convolution module to encode the picture into high-dimensional image characteristics, then carries out attention encoding under the guidance of the posture, finishes the conversion of the image characteristics through multiple encoding, and finally converts the image characteristics into the picture through an up-sampling convolution module; putting the generated image into a discriminator, wherein the discriminator forces the generator to generate a picture closer to reality by distinguishing a real image from the generated image; and finally, the trained generation confrontation network is used for completing the human posture migration.
The image preprocessing comprises the following specific steps: firstly, extracting the postures of a person by using a trained joint detector HPE, then dividing a fixed person and corresponding postures into a group, arranging and combining pictures in each group to form training data, and collecting 263632 groups of training data and 12000 groups of test data for a reference data set Market-1501; for the depfashinon dataset, 101966 sets of training data and 8570 sets of test data were collected.
The attention coding under the posture guidance comprises the following specific steps: firstly, mapping the attitude characteristics into Key and Value respectively through convolution of 1 x 1, wherein the Key and the Value represent information of the attitude characteristics and are in one-to-one correspondence; then multiplying the translated Key with Value to obtain an attention diagram; finally, the image characteristics and the attention map are formed to obtain an attention code under the guidance of the posture;
after the attention coding is obtained, the image feature and the pose feature are spliced for better integration, and after the feedback of the image feature is obtained, the pose feature can further guide the image feature to carry out the subsequent transformation.
The input of the generator is a conditional image IcConditional image corresponding to pose PcAnd target attitude PtOutput as a generated image IgAfter the image is generated, the generated image is put into a discriminator; the discriminator takes the form of a double discriminator: texture discriminator DAAnd a shape discriminator DS(ii) a Texture discriminator DAInput generated image IgAnd a conditional image IcFor judging whether the texture between the two images is consistent, the input is (I)c,It)，(Ic,Ig) A doublet of the conditional image and the target image or the generated image, respectively; shape discriminator DSInputting a generated image and a target posture for judging whether the generated image conforms to the target posture, wherein the input is (P)t,It)，(Pt,Ig) A target pose and a target image or a binary set of generated images, respectively.
The loss function for generating the countermeasure network model comprises three parts:
1) generating a loss function L for a countermeasure networkCGANThe loss function is used to constrain the relationship between the generator and the arbiter to make the two more balanced, and the loss function contains two parts of the penalty, corresponding to the two arbiters, and the total loss function is defined as follows:
wherein
Respectively representing the distribution of the human body posture, the distribution of the real image and the distribution of the generated image;
2) distance loss LL1The loss is the distance between pixel points of the generated image and the target image, the generated image is closer to the target image by reducing the loss function, and the loss function is defined as follows:
LL1＝‖Ig-It‖1, (2)
3) loss of perception LpercepThe perceptual loss is used to reduce the structural difference between the generated image and the target image and make the generated image more natural, and is defined as follows:
wherein
Representing the VGG-19 network model pre-trained on ImageNet data set
The output of the layer(s) is,
to represent
Ith feature map in the layer output.
The final overall loss function is shown in equation (4):
Lfull＝αLCGAN+βLL1+γLpercep, (4)
wherein α, gamma respectively represent LCGAN，LL1，LpercepThe weights of the three parts.
The invention has the characteristics and beneficial effects that:
the invention provides an attention mechanism-based image synthesis system for human body posture migration. Given a picture containing a person and an arbitrary pose, the system can generate a picture of the person making the specified pose. The system introduces an attention mechanism, changes the attention mechanism into an attention mechanism more suitable for posture guidance of the task, and solves the problem of picture pixel misalignment caused in the posture migration process. And meanwhile, the optimal result is obtained on both the Market-1501 data set and the DeepFashinon data set.
Description of the drawings:
FIG. 1 is a system block diagram of a human pose migration technique based on an attention mechanism.
FIG. 2 gives a result graph generated for an arbitrary pose on a Market-1501 data set.
Fig. 3 is a graph of the results generated given an arbitrary pose on the depfashinon dataset.
Figure 4 is a qualitative comparison of the present system with the four other best algorithms currently under this task.
Detailed Description
In order to solve the problems in the prior art, the invention provides an image generation method which is closer to a human thinking mode, and the guiding synthesis of the image pixels by the gestures is completed based on an attention mechanism. The former method mostly adopts the form of human body segmentation, divides the human body into a plurality of parts, carries out rigid body transformation on each part, and then further splices and synthesizes the final result. The methods can better process the condition that the difference between the condition posture and the target posture is small, but the problem of pixel misalignment caused by posture conversion is highlighted when the difference is large. In order to solve the problem of pixel misalignment caused by posture conversion, the invention guides the image characteristics by the posture characteristics based on an attention mechanism, gradually converts the image characteristics from an initial posture to a specified posture, and gradually solves the problem of pixel misalignment. Meanwhile, the invention can generate sufficiently clear pictures by using the framework for generating the countermeasure network.
The invention provides a human body posture migration technology based on an attention mechanism. The technical scheme uses a Market-1501 data set and a DeepFashinon data set as processing objects, and the whole system comprises three parts: data preprocessing, attention coding under the guidance of postures, and network building and training. In order to better complete the task of human body posture migration and generate pictures meeting requirements, network design and network training are two main problems to be solved. The specific technical scheme is as follows:
step one, preprocessing image data:
for the pictures in the two data sets, firstly, the Pose of a person is extracted by using an HPE (Human position Estimation) joint point detector, then, the fixed person and the corresponding Pose are divided into a group, and the pictures in each group are arranged and combined to form training data. For a Market-1501 data set (a reference data set for pedestrian re-recognition work and a reference data set for human posture migration work), 263632 groups of training data and 12000 groups of test data are collected; for the DeepFashinon dataset (containing 80 million pictures, containing different angles, different scenes, buyer show, etc.), we collected 101966 sets of training data and 8570 sets of test data.
Step two, encoding attention under the guidance of the posture:
for image feature CIAnd an attitude feature CPThe invention leads the posture characteristic to guide the image characteristic transformation by transforming a self-attention mechanism. Firstly, mapping the attitude characteristics into Key and Value respectively through convolution of 1 x 1, wherein the Key and the Value represent information of the attitude characteristics and are in one-to-one correspondence; then multiplying the translated Key with Value to obtain an attention diagram; and finally, the image features and the attention map are combined to obtain the attention code under the guidance of the posture.
After the attention coding is obtained, the image feature and the posture feature are spliced for better integration. After feedback of the image features is obtained, the pose features may further guide the image features for subsequent transformation.
Step three, network building and training:
the invention adopts a framework for generating the countermeasure network, and is divided into a generator and a discriminator. The generator part firstly carries out a downsampling convolution module to code the picture into high-dimensional image characteristics, then uses an attention coding module under the guidance of the posture to complete the conversion of the image characteristics through multiple times of coding, and finally converts the image characteristics into the picture through an upsampling convolution module. The input of the generator is a conditional image IcConditional image corresponding to pose PcAnd target attitude PtOutput as a generated image Ig. After the image is generated, the generated image is put into a discriminator which distinguishes a real image ItAnd generation ofImage IgTo force the generator to generate a picture that is closer to the real one. The invention adopts the form of double discriminators: texture discriminator DAAnd a shape discriminator DS. Texture discriminator DAInput generated image IgAnd a conditional image IcFor judging whether the texture between the two images is consistent, the input is (I)c,It)，(Ic,Ig) A doublet of the conditional image and the target image or the generated image, respectively; shape discriminator DSInputting a generated image and a target posture for judging whether the generated image conforms to the target posture, wherein the input is (P)t,It)，(Pt,Ig) A target pose and a target image or a binary set of generated images, respectively. The loss function of the network model comprises three parts:
1. generating a loss function L for a countermeasure networkCGAN. The loss function is used to constrain the relationship between the generator and the arbiter to make the two more balanced. Corresponding to the two discriminators, the loss function contains two parts of the opposing loss, the total loss function being defined as follows:
wherein
Respectively representing the distribution of the human body posture, the distribution of the real image and the distribution of the generated image.
2. Distance loss LL1. The loss is a distance between pixel points of the generated image and the target image, and the generated image can be closer to the target image by reducing the loss function. The loss function is defined as follows:
LL1＝‖Ig-It‖1, (2)
3. loss of perception Lpercep. Perceptual loss is used to reduce the structural difference between the generated image and the target image and to make the generated image more natural. The perceptual loss is defined as follows:
wherein
Representing the VGG-19 network model pre-trained on ImageNet data set
The output of the layer(s) is,
to represent
Ith feature map in the layer output.
The final overall loss function is shown in equation (4):
Lfull＝αLCGAN+βLL1+γLpercep, (4)
wherein α, gamma respectively represent LCGAN，LL1，LpercepThe weights of the three parts.
The present invention will be described in further detail below with reference to the accompanying drawings and specific experiments.
Fig. 1 is a system framework diagram of a human body posture migration technique based on an attention mechanism according to the present invention, which mainly includes the following steps:
step one, preprocessing image data:
from each group of pictures in the Market-1501 data set and the DeepFashinon data set, the postures of the pictures are extracted by the joint point detector, and the input and output paired images shown in the figure 1 are formed by combining two pictures of the same person with different postures. For the Market-1501 data set, we collected 263632 sets of training data and 12000 sets of test data; for the DeepFashinon dataset, we collected 101966 sets of training data and 8570 sets of test data.
Step two, encoding attention under the guidance of the posture:
the structure of the generator of the system is shown in FIG. 1, and the attention coding module under each posture guidance is shown in the small diagram at the lower right corner in FIG. 1. The generator comprises two encoders and a decoder, wherein the two encoders respectively apply the conditional image IcConditional image corresponding to pose PcAnd target attitude PtThe concatenation is used as input. Both encoders have the same structure, i.e. the downsampled convolutional layer, and the decoder is the upsampled convolutional layer. And the image characteristics are migrated through the attention coding module under the posture guidance provided by the invention. The input to each module is an image feature and a pose feature. For example, the input to the tth module is an image feature
And attitude characteristics
Outputting transformed image features after passing through a module
And attitude characteristics
After the last module, only the transformed image features are required
The final image is generated in the decoder. All experimental results in the invention are tested when T is 6, namely 6 gesture-guided attention coding modules are available.
Step three, network building and training:
the constructed network comprises a generator and two discriminators. The structure of the discriminator is a common Convolutional Neural Network (CNN). For the texture discriminator, the input at each time is the condition image and the target image (I)c,It) And conditional image and generated image (I)c,Ig) The score is output as a score for judging the consistency of the texture. For the shape discriminator, the input at each time is the target image and the target pose (I)t,Pt) And generating an image and a target pose (I)g,Pt) And outputting a score as a score for judging the gesture consistency.
During the course of training, approximately 9 million iterations were performed using the Adam optimizer. The learning rate is initially set to 2 × 10-4Linear decay guided learning rate was 0 after 6 ten thousand iterations for both datasets we used 6 pose guided attention coding modules with slightly different hyper-parameters α, gamma settings, Market-1501 settings were 5, 10, respectively, on the deepfast dataset 5, 1, respectively.
Figure 4 sets forth a qualitative comparison of the results of the present system with the four methods currently performing optimally on this task. Wherein PG2VUnet, defem, and PATN are methods in the 2017 top conference NIPS, the 2018 top conference CVPR, and the 2019 top conference CVPR, respectively. It can be seen that the system can produce clearer pictures and can well process samples with larger posture change. Meanwhile, the pictures generated by the system can well ensure the texture information in the condition images and ensure better face information.
Table 1 shows a quantitative comparison of the results of the present system with the four methods currently performing optimally on this task.
TABLE 1 quantitative comparison of the present System and the four best algorithms currently under this task
In table 1, SSIM is structural similarity index, i.e. structural loss, which is used to measure the structural similarity between two pictures. Because the Market-1501 data set contains various complex backgrounds, a mask-SSIM (small Scale integration) is adopted as a measurement index; IS the inclusion Score, i.e., the Score derived by the pre-trained inclusion net neural network, used to measure the performance of generating the network synthesis picture. It can be seen that our results are the best system at present in terms of performance for the human pose migration task.
Claims (6)
1. A human body posture migration method based on an attention mechanism is characterized by comprising the following steps:
an image preprocessing step: forming training data;
attention coding under gesture guidance: for image feature CIAnd an attitude feature CPThe gesture features are used for guiding the image features to be transformed by using a self-attention mechanism, and attention codes under gesture guidance are obtained;
network building and training: generating a confrontation network model, wherein the network model is divided into a generator and a discriminator, the generator part firstly carries out down-sampling convolution module to encode the picture into high-dimensional image characteristics, then carries out attention encoding under the guidance of the posture, finishes the conversion of the image characteristics through multiple encoding, and finally converts the image characteristics into the picture through an up-sampling convolution module; putting the generated image into a discriminator, wherein the discriminator forces the generator to generate a picture closer to reality by distinguishing a real image from the generated image; and finally, the trained generation confrontation network is used for completing the human posture migration.
2. The human body posture migration method based on the attention mechanism as claimed in claim 1, wherein the image preprocessing comprises the following specific steps: firstly, extracting the postures of a person by using a trained joint detector HPE, then dividing a fixed person and corresponding postures into a group, arranging and combining pictures in each group to form training data, and collecting 263632 groups of training data and 12000 groups of test data for a reference data set Market-1501; for the depfashinon dataset, 101966 sets of training data and 8570 sets of test data were collected.
3. The human body posture migration method based on the attention mechanism as claimed in claim 1, wherein the attention coding under the posture guidance comprises the following steps: firstly, mapping the attitude characteristics into Key and Value respectively through convolution of 1 x 1, wherein the Key and the Value represent information of the attitude characteristics and are in one-to-one correspondence; then multiplying the translated Key with Value to obtain an attention diagram; finally, the image characteristics and the attention map are formed to obtain an attention code under the guidance of the posture; after the attention coding is obtained, the image feature and the pose feature are spliced for better integration, and after the feedback of the image feature is obtained, the pose feature can further guide the image feature to carry out the subsequent transformation.
4. The method of claim 1, wherein the generator inputs the condition image IcConditional image corresponding to pose PcAnd target attitude PtOutput as a generated image IgAfter the image is generated, the generated image is put into a discriminator; the discriminator takes the form of a double discriminator: texture discriminator DAAnd a shape discriminator DS(ii) a Texture discriminator DAInput generated image IgAnd a conditional image IcFor judging whether the texture between the two images is consistent, the input is (I)c,It)，(Ic,Ig) A doublet of the conditional image and the target image or the generated image, respectively; shape discriminator DSInputting a generated image and a target posture for judging whether the generated image conforms to the target posture, wherein the input is (P)t,It)，(Pt,Ig) A target pose and a target image or a binary set of generated images, respectively.
5. The method of human pose migration based on attention mechanism of claim 4, wherein the penalty function for generating the antagonistic network model comprises three parts:
1) generating a loss function L for a countermeasure networkCGANThe loss function is used for constraining the relation between the generator and the discriminator to make the two more balanced, corresponding to the two discriminators, the loss function comprises two parts of countermeasure loss and a total loss functionThe numbers are defined as follows:
wherein
Respectively representing the distribution of the human body posture, the distribution of the real image and the distribution of the generated image;
2) distance loss LL1The loss is the distance between pixel points of the generated image and the target image, the generated image is closer to the target image by reducing the loss function, and the loss function is defined as follows:
LL1＝‖Ig-It‖1, (2)
3) loss of perception LpercepThe perceptual loss is used to reduce the structural difference between the generated image and the target image and make the generated image more natural, and is defined as follows:
wherein
Representing the VGG-19 network model pre-trained on ImageNet data set
The output of the layer(s) is,
to represent
Ith feature map in the layer output.
6. The method for human pose migration based on attention mechanism as claimed in claim 4 wherein the final overall loss function is shown in equation (4):
Lfull＝αLCGAN+βLL1+γLpercep, (4)
wherein α, gamma respectively represent LCGAN，LL1，LpercepThe weights of the three parts.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911332748.0A CN111161200A (en) | 2019-12-22 | 2019-12-22 | Human body posture migration method based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911332748.0A CN111161200A (en) | 2019-12-22 | 2019-12-22 | Human body posture migration method based on attention mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111161200A true CN111161200A (en) | 2020-05-15 |
Family
ID=70557725
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911332748.0A Pending CN111161200A (en) | 2019-12-22 | 2019-12-22 | Human body posture migration method based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111161200A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111681182A (en) * | 2020-06-04 | 2020-09-18 | Oppo广东移动通信有限公司 | Picture restoration method and device, terminal equipment and storage medium |
| CN111696027A (en) * | 2020-05-20 | 2020-09-22 | 电子科技大学 | Multi-modal image style migration method based on adaptive attention mechanism |
| CN111739115A (en) * | 2020-06-23 | 2020-10-02 | 中国科学院自动化研究所 | Unsupervised human body posture migration method, system and device based on cycle consistency |
| CN112116673A (en) * | 2020-07-29 | 2020-12-22 | 西安交通大学 | Virtual human body image generation method and system based on structural similarity under posture guidance and electronic equipment |
| CN112149645A (en) * | 2020-11-10 | 2020-12-29 | 西北工业大学 | Human body posture key point identification method based on generation of confrontation learning and graph neural network |
| CN113408351A (en) * | 2021-05-18 | 2021-09-17 | 河南大学 | Pedestrian re-recognition method for generating confrontation network based on attitude guidance |
| CN113538608A (en) * | 2021-01-25 | 2021-10-22 | 哈尔滨工业大学（深圳） | Controllable character image generation method based on generation countermeasure network |
| CN113706650A (en) * | 2021-08-27 | 2021-11-26 | 深圳龙岗智能视听研究院 | Image generation method based on attention mechanism and flow model |
| CN113936073A (en) * | 2021-11-02 | 2022-01-14 | 哈尔滨理工大学 | AtISTANet compressed sensing magnetic resonance reconstruction method based on attention mechanism |
| CN114399829A (en) * | 2022-03-25 | 2022-04-26 | 浙江壹体科技有限公司 | Posture migration method based on generative countermeasure network, electronic device and medium |
| CN114401446A (en) * | 2021-12-16 | 2022-04-26 | 广州方硅信息技术有限公司 | Human body posture migration method, device, system, electronic equipment and storage medium |
| CN114783039A (en) * | 2022-06-22 | 2022-07-22 | 南京信息工程大学 | Motion migration method driven by 3D human body model |
| CN114863005A (en) * | 2022-04-19 | 2022-08-05 | 佛山虎牙虎信科技有限公司 | Rendering method and device for limb special effect, storage medium and equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108564119A (en) * | 2018-04-04 | 2018-09-21 | 华中科技大学 | A kind of any attitude pedestrian Picture Generation Method |
| WO2019015466A1 (en) * | 2017-07-17 | 2019-01-24 | 广州广电运通金融电子股份有限公司 | Method and apparatus for verifying person and certificate |
-
2019
- 2019-12-22 CN CN201911332748.0A patent/CN111161200A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019015466A1 (en) * | 2017-07-17 | 2019-01-24 | 广州广电运通金融电子股份有限公司 | Method and apparatus for verifying person and certificate |
| CN108564119A (en) * | 2018-04-04 | 2018-09-21 | 华中科技大学 | A kind of any attitude pedestrian Picture Generation Method |
Non-Patent Citations (1)
| Title |
|---|
| JINSONG ZHANG等: ""Attention-guided GANs for human pose transfer"" * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696027A (en) * | 2020-05-20 | 2020-09-22 | 电子科技大学 | Multi-modal image style migration method based on adaptive attention mechanism |
| CN111696027B (en) * | 2020-05-20 | 2023-04-07 | 电子科技大学 | Multi-modal image style migration method based on adaptive attention mechanism |
| CN111681182A (en) * | 2020-06-04 | 2020-09-18 | Oppo广东移动通信有限公司 | Picture restoration method and device, terminal equipment and storage medium |
| CN111739115A (en) * | 2020-06-23 | 2020-10-02 | 中国科学院自动化研究所 | Unsupervised human body posture migration method, system and device based on cycle consistency |
| CN111739115B (en) * | 2020-06-23 | 2021-03-16 | 中国科学院自动化研究所 | Unsupervised human body posture migration method, system and device based on cycle consistency |
| CN112116673B (en) * | 2020-07-29 | 2022-12-09 | 西安交通大学 | Virtual human body image generation method and system based on structural similarity under posture guidance and electronic equipment |
| CN112116673A (en) * | 2020-07-29 | 2020-12-22 | 西安交通大学 | Virtual human body image generation method and system based on structural similarity under posture guidance and electronic equipment |
| CN112149645A (en) * | 2020-11-10 | 2020-12-29 | 西北工业大学 | Human body posture key point identification method based on generation of confrontation learning and graph neural network |
| CN113538608A (en) * | 2021-01-25 | 2021-10-22 | 哈尔滨工业大学（深圳） | Controllable character image generation method based on generation countermeasure network |
| CN113538608B (en) * | 2021-01-25 | 2023-08-01 | 哈尔滨工业大学（深圳） | Controllable figure image generation method based on generation countermeasure network |
| CN113408351A (en) * | 2021-05-18 | 2021-09-17 | 河南大学 | Pedestrian re-recognition method for generating confrontation network based on attitude guidance |
| CN113706650A (en) * | 2021-08-27 | 2021-11-26 | 深圳龙岗智能视听研究院 | Image generation method based on attention mechanism and flow model |
| CN113936073A (en) * | 2021-11-02 | 2022-01-14 | 哈尔滨理工大学 | AtISTANet compressed sensing magnetic resonance reconstruction method based on attention mechanism |
| CN113936073B (en) * | 2021-11-02 | 2024-05-14 | 哈尔滨理工大学 | ATTISTANET compressed sensing magnetic resonance reconstruction method based on attention mechanism |
| CN114401446A (en) * | 2021-12-16 | 2022-04-26 | 广州方硅信息技术有限公司 | Human body posture migration method, device, system, electronic equipment and storage medium |
| CN114399829B (en) * | 2022-03-25 | 2022-07-05 | 浙江壹体科技有限公司 | Posture migration method based on generative countermeasure network, electronic device and medium |
| CN114399829A (en) * | 2022-03-25 | 2022-04-26 | 浙江壹体科技有限公司 | Posture migration method based on generative countermeasure network, electronic device and medium |
| CN114863005A (en) * | 2022-04-19 | 2022-08-05 | 佛山虎牙虎信科技有限公司 | Rendering method and device for limb special effect, storage medium and equipment |
| CN114783039A (en) * | 2022-06-22 | 2022-07-22 | 南京信息工程大学 | Motion migration method driven by 3D human body model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111161200A (en) | Human body posture migration method based on attention mechanism | |
| CN110706302B (en) | System and method for synthesizing images by text | |
| CN111275518B (en) | Video virtual fitting method and device based on mixed optical flow | |
| CN110399850B (en) | Continuous sign language recognition method based on deep neural network | |
| CN110263912A (en) | A kind of image answering method based on multiple target association depth reasoning | |
| CN109670576B (en) | Multi-scale visual attention image description method | |
| CN113255457A (en) | Animation character facial expression generation method and system based on facial expression recognition | |
| CN113780059B (en) | Continuous sign language identification method based on multiple feature points | |
| CN115237255B (en) | Natural image co-pointing target positioning system and method based on eye movement and voice | |
| CN112818764A (en) | Low-resolution image facial expression recognition method based on feature reconstruction model | |
| CN117033609B (en) | Text visual question-answering method, device, computer equipment and storage medium | |
| CN111401156B (en) | Image identification method based on Gabor convolution neural network | |
| CN111724458A (en) | Voice-driven three-dimensional human face animation generation method and network structure | |
| CN113140020A (en) | Method for generating image based on text of countermeasure network generated by accompanying supervision | |
| CN112037239B (en) | Text guidance image segmentation method based on multi-level explicit relation selection | |
| CN118038139A (en) | Multi-mode small sample image classification method based on large model fine tuning | |
| CN115908639A (en) | Transformer-based scene image character modification method and device, electronic equipment and storage medium | |
| CN113888399B (en) | Face age synthesis method based on style fusion and domain selection structure | |
| CN113076918B (en) | Video-based facial expression cloning method | |
| CN111767842B (en) | Micro-expression type discrimination method based on transfer learning and self-encoder data enhancement | |
| CN114944002B (en) | Text description-assisted gesture-aware facial expression recognition method | |
| CN114783039B (en) | Motion migration method driven by 3D human body model | |
| CN113780350B (en) | ViLBERT and BiLSTM-based image description method | |
| CN116311472A (en) | Micro-expression recognition method and device based on multi-level graph convolution network | |
| Alves et al. | Enhancing Brazilian Sign Language Recognition through Skeleton Image Representation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200515 |
|
| RJ01 | Rejection of invention patent application after publication |