CN118587362A - Digital human image generation method and system based on virtual concert hall - Google Patents
Digital human image generation method and system based on virtual concert hall Download PDFInfo
- Publication number
- CN118587362A CN118587362A CN202410775826.9A CN202410775826A CN118587362A CN 118587362 A CN118587362 A CN 118587362A CN 202410775826 A CN202410775826 A CN 202410775826A CN 118587362 A CN118587362 A CN 118587362A
- Authority
- CN
- China
- Prior art keywords
- digital
- digital human
- human image
- generation model
- skin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 239000011159 matrix material Substances 0.000 claims abstract description 74
- 210000000988 bone and bone Anatomy 0.000 claims abstract description 42
- 238000009877 rendering Methods 0.000 claims description 17
- 210000002805 bone matrix Anatomy 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 6
- 238000012935 Averaging Methods 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 15
- 230000008569 process Effects 0.000 abstract description 7
- 230000033001 locomotion Effects 0.000 description 8
- 230000000007 visual effect Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 230000003993 interaction Effects 0.000 description 5
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000003278 mimic effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000027455 binding Effects 0.000 description 1
- 238000009739 binding Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a digital human image generation method and a digital human image generation system based on a virtual studio, which relate to the technical field of digital human image generation, and can generate digital human images in the virtual studio, thereby meeting the application requirements under complex scenes; by finely processing bone and skin information, digital human images with highly realistic characteristics can be generated; by independent processing of the skeleton generation model and the skin generation model, the actions and the appearance of the digital person can be controlled more flexibly, and the flexibility and the accuracy of the generation process are improved. The scheme is as follows: creating a three-dimensional virtual concert hall; importing a digital human image generation model into the model; acquiring a digital human image; inputting the digital human image into a skeleton generation model to obtain a skeleton matrix; inputting the digital human image into a skin generation model to obtain a skin matrix; and inputting the skeleton matrix and the skin matrix into a digital human image generation model to obtain the digital human image. The invention is used for digital human image generation.
Description
Technical Field
The invention relates to the technical field of digital human image generation, in particular to a digital human image generation method and system based on a virtual studio.
Background
Currently, with the rapid development of digital media and virtual reality technology, more and more industries are exploring the use of virtual environments to enhance user experience. In the fields of television broadcasting, movie production, online education, live game broadcasting, etc., virtual concert hall technology has been applied to some extent. By using a green screen background and special visual effects, a starring or actor can present rich and colorful scenes in a limited space, thereby greatly improving the space and imagination of content creation.
However, existing virtual concert techniques focus primarily on background replacement and visual effects, and do not provide adequate support for the generation and interaction of digital persons (i.e., virtual characters). Although some systems can realize basic three-dimensional model rendering, the actions and behaviors of the models are often very limited, and cannot meet the application requirements in complex scenes. Furthermore, prior art digital mannequins typically require manual control, and lack real-time motion capture and feedback capabilities.
Disclosure of Invention
The invention provides a digital person image generation method and a system based on a virtual studio, which can generate digital person images in the virtual studio, thereby meeting the application requirements in complex scenes; by finely processing bone and skin information, digital human images with highly realistic characteristics can be generated; by independent processing of the skeleton generation model and the skin generation model, the actions and the appearance of the digital person can be controlled more flexibly, and the flexibility and the accuracy of the generation process are improved.
In order to achieve the above purpose, the invention adopts the following technical scheme:
the first aspect of the present invention provides a digital person image generating method based on a virtual concert hall, comprising:
creating a three-dimensional virtual concert hall; and a three-dimensional scene is preset in the virtual concert hall.
And importing a digital human image generation model in the virtual studio.
A digital person image is acquired.
And inputting the digital human image into a skeleton generation model to obtain a skeleton matrix.
And inputting the digital human image into a skin generation model to obtain a skin matrix.
And inputting the skeleton matrix and the skin matrix into the digital human image generation model to obtain the digital human image.
Further, the method for generating the digital human image based on the virtual studio further comprises the steps of:
and positioning and posture adjustment are carried out on the digital human image generation model so that the digital human image generation model is adapted to the three-dimensional scene of the virtual concert hall.
Further, in the digital person image generating method based on the virtual concert hall, the skeleton generating model comprises:
The bone generation model includes a bone keypoint detection model.
Further, the method for generating the digital human image based on the virtual concert hall inputs the digital human image into a skeleton generation model to obtain a skeleton matrix, and the method comprises the following steps:
and extracting bone key points through the bone key point detection model.
And sequentially selecting two adjacent bone key points from the bone key points.
Initializing a preset number of three-dimensional zero matrices.
And filling areas corresponding to the bone key points adjacent to two positions in each three-dimensional zero matrix with preset pixel values so as to convert each three-dimensional zero matrix into the bone matrix corresponding to the bone key points adjacent to each two positions.
Further, the method for generating the digital human image based on the virtual concert hall inputs the digital human image into a skin generation model to obtain a skin matrix, and the method comprises the following steps:
Generating a grid according to the digital human image, and determining skin weights of grid vertices influenced by the skeleton key points; the mesh simulates the skin shape of a human body.
And based on the grid connectivity, averaging adjacent edge features of the grid vertexes to obtain an edge axis.
And after forward transmission of the adjacent edge features, predicting coordinate values of the grid vertexes according to the skeleton key points and the skin weights.
And determining a digital human skin matrix according to the edge axis and the predicted grid vertex coordinate value.
Further, the digital person image generating method based on the virtual concert hall comprises the following steps:
the digital person image generation model, the bone generation model, and the skin generation model are constructed from a grid convolution and a bone convolution.
Further, the method for generating a digital human image based on the virtual concert hall further comprises, after inputting the skeleton matrix and the skin matrix to the digital human image generation model to obtain a digital human image:
And rendering the images of the digital person image generation model in the virtual studio in real time.
A second aspect of the present invention provides a digital person image generating system based on a virtual concert hall, comprising:
The creation module is used for creating a three-dimensional virtual studio; and a three-dimensional scene is preset in the virtual concert hall.
And the importing module is used for importing a digital human image generating model into the virtual studio.
And the acquisition module is used for acquiring the digital human image.
And the first input module is used for inputting the digital human image into a skeleton generation model to obtain a skeleton matrix.
And the second input module is used for inputting the digital human image into the skin generation model to obtain a skin matrix.
And the third input module is used for inputting the skeleton matrix and the skin matrix into the digital human image generation model to obtain the digital human image.
Further, the digital person image generating system based on the virtual concert hall further comprises:
and the positioning and adjusting module is used for positioning and adjusting the digital human image generating model so as to adapt the digital human image generating model to the three-dimensional scene of the virtual concert hall.
Further, the digital person image generating system based on the virtual concert hall further comprises:
And the rendering module is used for rendering the images of the digital person image generation model in the virtual studio in real time.
The invention provides a digital human image generation method and a system based on a virtual concert hall, comprising the following steps: creating a three-dimensional virtual concert hall; importing a digital human image generation model in a virtual studio; acquiring a digital human image; inputting the digital human image into a skeleton generation model to obtain a skeleton matrix; inputting the digital human image into a skin generation model to obtain a skin matrix; the skeleton matrix and the skin matrix are input into the digital human image generation model to obtain the digital human image, and compared with the prior art, the digital human image generation method and the digital human image generation device can generate the digital human image in the virtual studio, so that the application requirements under a complex scene are met; by finely processing bone and skin information, digital human images with highly realistic characteristics can be generated; by independent processing of the skeleton generation model and the skin generation model, the actions and the appearance of the digital person can be controlled more flexibly, and the flexibility and the accuracy of the generation process are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are used in the description of the embodiments will be briefly described below, which are only for the purpose of illustrating the embodiments and are not to be construed as limiting the present invention.
Fig. 1 is a schematic flow chart of a digital person image generating method based on a virtual studio in an embodiment of the invention;
FIG. 2 is a flow chart of another digital person image generating method based on a virtual studio according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a digital human image generating system based on a virtual studio according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a digital human image generating system based on a virtual studio according to another embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs; the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention and the terms "comprising" and "having" and any variations thereof, as described in the specification and claims of the invention and the above description of the drawings, are intended to cover a non-exclusive inclusion.
In the description of embodiments of the present invention, the technical terms "first," "second," and the like are used merely to distinguish between different objects and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated, a particular order or a primary or secondary relationship. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more unless explicitly defined otherwise.
In the description of the embodiments of the present invention, the term "and/or" is merely an association relationship describing an association object, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
In the description of the embodiments of the present invention, the term "plurality" means two or more (including two), and similarly, "plural sets" means two or more (including two), and "plural sheets" means two or more (including two).
In the description of the embodiments of the present invention, the orientation or positional relationship indicated by the technical terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. are based on the orientation or positional relationship shown in the drawings, and are merely for convenience of description and simplification of the description, and do not indicate or imply that the apparatus or element referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the embodiments of the present invention.
In the description of the embodiments of the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured" and the like should be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally formed; or may be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the examples of the present invention will be understood by those skilled in the art according to the specific circumstances.
Example 1
The embodiment of the invention provides a digital human image generation method based on a virtual studio, which is shown in fig. 1 and comprises the following steps:
s101, creating a three-dimensional virtual concert hall; a three-dimensional scene is preset in the virtual concert hall.
The three-dimensional virtual studio is a virtual environment created by computer graphics and virtual reality technology, can simulate a real-world studio scene, and provides a three-dimensional space for interaction and display of digital content, virtual characters or virtual objects. Creating a three-dimensional virtual studio generally includes the steps of:
demand analysis: and determining the purpose and design requirements of the virtual concert hall, and collecting the three-dimensional model and texture resources of the required scene.
Scene modeling: three-dimensional modeling software (such as Blender, maya, 3ds Max and the like) is used for creating a three-dimensional model of elements such as ground, background, props and the like, refining model details, increasing realism such as lamplight, shadow and reflection effects, and importing the required three-dimensional model into a virtual reality engine or rendering software.
And (3) light setting: and setting light according to actual lighting requirements, simulating natural light or artificial light sources, and adjusting the intensity, color and range of the light so as to obtain an ideal light and shadow effect.
Camera settings: setting the position and angle of the virtual camera, simulating the real camera visual angle, and adjusting the parameters of the camera, such as focal length, aperture, white balance, and the like.
Integration and testing: and integrating all elements in the virtual reality engine, ensuring the integrity and interactivity of the scene, and testing the visual effects under different visual angles and illumination conditions.
Animation and interaction: if dynamic effects are desired, keyframe animations can be added or skeletal bindings can be used to effect movement of objects, interaction logic with the virtual environment can be set, such as collision responses of objects or responses of user operations.
Rendering and outputting: rendering according to the required resolution and frame rate, outputting an imaging file or directly viewing in a virtual reality environment.
The preset three-dimensional scene can be customized according to specific requirements, such as news broadcasting, sports events, concerts or other types of program designs. Through fine modeling and real textures, a virtual environment with immersion feeling can be created, and a viewer is difficult to distinguish between virtual and reality. In addition, by integrating real-time rendering technology and interactive logic, the virtual studio can respond to the operation of the audience, and more personalized and interactive viewing experience is provided.
S102, importing a digital human image generation model into the virtual studio.
Among them, the digital human image generation model is a computer graphics technology for creating highly realistic virtual characters that can be used in the fields of movies, games, virtual reality, and other visual effects. The digital human image generation model has highly detailed appearance and behavioral characteristics that can mimic the appearance and actions of a real human.
S103, acquiring a digital human image.
Wherein, the digital human image refers to a two-dimensional or three-dimensional image representing a human figure created through computer graphics technology. These images may be static or dynamic, they may closely mimic the appearance and motion of a real person, or may be stylized or cartoonized representations. There are a number of ways to obtain digital people, depending on your needs and purpose. The following are some common methods:
Three-dimensional modeling and rendering: three-dimensional modeling software (such as Blender, 3ds Max, maya, etc.) is used to create a three-dimensional model, which is then rendered to obtain a high quality image. This approach can fully control the appearance and environment of a digital person.
Photo modeling: by taking a photograph of a real person model or using an existing portrait photograph, it can be used as a starting point for creating a digital portrait image. Using these photos you can edit and modify by image editing software (e.g. Photoshop) or use special software (e.g. ZBrush) to extract three-dimensional information from it.
Scanning: and capturing three-dimensional data of the real model by using a three-dimensional scanner, and then processing and repairing the data in a computer to finally obtain the digital human model.
Motion capture: if you need dynamic images, the motion capture system can be used to capture real person motions and then apply these motions to the digital person model.
And (3) synthesis: the digital human model is synthesized into other backgrounds or scenes using techniques of computer graphics to obtain the desired image.
AI generation: training the neural network to generate a digital human image using a deep learning technique. For example, extremely realistic digital person images may be generated using the generation countermeasure network (GANs).
S104, inputting the digital human image into a skeleton generation model to obtain a skeleton matrix.
Wherein the bone generation model (Bone Generation Model) generally refers to techniques used in computer graphics to create and manage three-dimensional model bone structures. Such skeletal structures, also known as "skeletons" or "skeletal models," are the basis for controlling how various portions of the three-dimensional model move and deform. The skeleton generation model can be applied to the fields of animation production, game role design, virtual reality application and the like.
S105, inputting the digital human image into a skin generation model to obtain a skin matrix.
Among other things, skin generation model (Skin Generation Model) is commonly referred to in computer graphics as a model for creating and simulating the skin surface of an organism. Such models may be used in the fields of video games, movie special effects, virtual reality, medical simulations, etc. to provide highly realistic visual effects.
S106, inputting the skeleton matrix and the skin matrix into a digital human image generation model to obtain the digital human image.
In particular, the bone matrix and the skin matrix are combined, typically by attaching the skin matrix to a skeletal structure driven by the bone matrix. Thus, as the bone matrix changes, the skin matrix changes as well to maintain its apparent consistency. And inputting the combined bone and skin data into a digital human image generation model, and finally obtaining the digital human image after the digital human image generation model is processed.
The embodiment of the invention provides a digital human image generation method based on a virtual studio, which comprises the following steps: creating a three-dimensional virtual concert hall; importing a digital human image generation model in a virtual studio; acquiring a digital human image; inputting the digital human image into a skeleton generation model to obtain a skeleton matrix; inputting the digital human image into a skin generation model to obtain a skin matrix; the skeleton matrix and the skin matrix are input into a digital human image generation model to obtain the digital human image, and compared with the prior art, the embodiment of the invention can generate the digital human image in a virtual studio, thereby meeting the application requirements under complex scenes; by finely processing bone and skin information, digital human images with highly realistic characteristics can be generated; by independent processing of the skeleton generation model and the skin generation model, the actions and the appearance of the digital person can be controlled more flexibly, and the flexibility and the accuracy of the generation process are improved.
Example two
The embodiment of the invention provides a digital human image generation method based on a virtual studio, which is shown in fig. 2 and comprises the following steps:
S201, creating a three-dimensional virtual concert hall; a three-dimensional scene is preset in the virtual concert hall.
S202, importing a digital human image generation model into a virtual studio.
S203, positioning and posture adjustment are carried out on the digital human image generation model so that the digital human image generation model is adapted to the three-dimensional scene of the virtual concert hall.
Specifically, after the digital person image generation model is imported into the virtual studio, the position of the digital person image generation model is adjusted in three-dimensional space so as to be in a proper position in the virtual studio. And adjusting the posture of the digital human image generation model to coordinate with the scene of the virtual concert hall.
S204, acquiring a digital human image.
S205, inputting the digital human image into a skeleton generation model to obtain a skeleton matrix.
Specifically, the bone generation model comprises a bone key point detection model, and bone key points are extracted through the bone key point detection model; sequentially selecting two adjacent bone key points from the bone key points; initializing a preset number of three-dimensional zero matrixes; and filling the areas corresponding to the bone key points adjacent to two positions in each three-dimensional zero matrix with preset pixel values so as to convert each three-dimensional zero matrix into the bone matrix corresponding to the bone key points adjacent to each two positions.
S206, inputting the digital human image into a skin generation model to obtain a skin matrix.
Specifically, generating a grid according to the digital human image, and determining skin weights of skeleton key points affecting grid vertices; the grid simulates the skin shape of a human body; based on the grid connectivity, averaging adjacent edge features of grid vertices to obtain an edge axis; after forward transmission of adjacent edge features, predicting coordinate values of grid vertexes according to skeleton key points and skin weights; a digital human skin matrix is determined based on the edge axes and the predicted mesh vertex coordinate values.
The digital human image generation model, the skeleton generation model and the skin generation model are built according to grid convolution and skeleton convolution.
S207, inputting the skeleton matrix and the skin matrix into a digital human image generation model to obtain a digital human image.
S208, rendering the images of the digital person image generation model in the virtual studio in real time.
Specifically, images of the digital human image generation model in the virtual concert hall are rendered in real-time by a rendering engine using graphics-acceleration hardware (GPU). Including receiving input data (e.g., bone transforms, camera movements, etc.), calculating illumination, shadows, reflections, etc., and outputting final pixel data.
The embodiment of the invention provides a digital human image generation method based on a virtual studio, which comprises the following steps: creating a three-dimensional virtual concert hall; importing a digital human image generation model in a virtual studio; positioning and posture adjustment are carried out on the digital human image generation model; acquiring a digital human image; inputting the digital human image into a skeleton generation model to obtain a skeleton matrix; inputting the digital human image into a skin generation model to obtain a skin matrix; inputting the skeleton matrix and the skin matrix into a digital human image generation model to obtain a digital human image; compared with the prior art, the embodiment of the invention can generate the digital human image in the virtual studio, thereby meeting the application requirements in complex scenes; by finely processing bone and skin information, digital human images with highly realistic characteristics can be generated; by independent processing of the skeleton generation model and the skin generation model, the actions and the appearance of the digital person can be controlled more flexibly, and the flexibility and the accuracy of the generation process are improved.
Example III
The embodiment of the invention provides a digital human image generating system based on a virtual studio, which is shown in fig. 3 and comprises the following steps:
A creation module 31 for creating a three-dimensional virtual concert hall; a three-dimensional scene is preset in the virtual concert hall.
An importing module 32 for importing the digital person image generating model in the virtual studio.
An acquisition module 33 for acquiring a digital person image.
A first input module 34 for inputting the digital human image into a bone generation model to obtain a bone matrix.
A second input module 35 for inputting the digital human image into the skin generation model to obtain a skin matrix.
A third input module 36 is configured to input the skeleton matrix and the skin matrix into the digital human image generation model to obtain a digital human image.
The embodiment of the invention provides a digital human image generation system based on a virtual studio, which comprises the following steps: the creation module is used for creating a three-dimensional virtual studio; the importing module is used for importing a digital human image generating model in the virtual studio; the acquisition module is used for acquiring the digital human image; the first input module is used for inputting the digital human image into the skeleton generation model to obtain a skeleton matrix; the second input module is used for inputting the digital human image into the skin generation model to obtain a skin matrix; the third input module is used for inputting the skeleton matrix and the skin matrix into the digital human image generation model to obtain the digital human image, and compared with the prior art, the embodiment of the invention can generate the digital human image in the virtual studio, thereby meeting the application requirements under complex scenes; by finely processing bone and skin information, digital human images with highly realistic characteristics can be generated; by independent processing of the skeleton generation model and the skin generation model, the actions and the appearance of the digital person can be controlled more flexibly, and the flexibility and the accuracy of the generation process are improved.
Example IV
The embodiment of the invention provides a digital human image generating system based on a virtual studio, which is shown in fig. 4 and comprises the following steps:
A creation module 41 for creating a three-dimensional virtual concert hall; a three-dimensional scene is preset in the virtual concert hall.
An importing module 42 for importing the digital person image generating model in the virtual studio.
The positioning and adjusting module 43 is used for positioning and adjusting the digital human image generating model so as to adapt the digital human image generating model to the three-dimensional scene of the virtual concert hall.
An acquisition module 44 for acquiring a digital person image.
A first input module 45 is configured to input a digital human image into the bone generation model to obtain a bone matrix.
A second input module 46 for inputting the digital human image into the skin generation model to obtain a skin matrix.
The third input module 47 is configured to input the skeleton matrix and the skin matrix into the digital human image generation model to obtain a digital human image.
And a rendering module 48 for rendering in real time the images of the digital person image generation model in the virtual concert hall.
The embodiment of the invention provides a digital human image generation system based on a virtual studio, which comprises the following steps: the creation module is used for creating a three-dimensional virtual studio; the importing module is used for importing a digital human image generating model in the virtual studio; the positioning and adjusting module is used for positioning and adjusting the digital human image generating model so as to enable the digital human image generating model to adapt to the three-dimensional scene of the virtual concert hall; the acquisition module is used for acquiring the digital human image; the first input module is used for inputting the digital human image into the skeleton generation model to obtain a skeleton matrix; the second input module is used for inputting the digital human image into the skin generation model to obtain a skin matrix; the third input module is used for inputting the skeleton matrix and the skin matrix into the digital human image generation model to obtain a digital human image; the rendering module is used for rendering the image of the digital person image generation model in the virtual studio in real time, and compared with the prior art, the embodiment of the invention can generate the digital person image in the virtual studio, thereby meeting the application requirements under complex scenes; by finely processing bone and skin information, digital human images with highly realistic characteristics can be generated; by independent processing of the skeleton generation model and the skin generation model, the actions and the appearance of the digital person can be controlled more flexibly, and the flexibility and the accuracy of the generation process are improved.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limited thereto; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention, and are intended to be included within the scope of the appended claims and description. In particular, the technical features mentioned in the respective embodiments may be combined in any manner as long as there is no structural conflict. The present invention is not limited to the specific embodiments disclosed herein, but encompasses all technical solutions falling within the scope of the claims.
Claims (10)
1. A digital person image generation method based on a virtual concert hall, comprising:
Creating a three-dimensional virtual concert hall; a three-dimensional scene is preset in the virtual concert hall;
Importing a digital human image generation model in the virtual studio;
acquiring a digital human image;
inputting the digital human image into a skeleton generation model to obtain a skeleton matrix;
Inputting the digital human image into a skin generation model to obtain a skin matrix;
and inputting the skeleton matrix and the skin matrix into the digital human image generation model to obtain the digital human image.
2. The virtual studio-based digital person image generation method according to claim 1, further comprising, after importing a digital person image generation model in said virtual studio:
and positioning and posture adjustment are carried out on the digital human image generation model so that the digital human image generation model is adapted to the three-dimensional scene of the virtual concert hall.
3. The virtual studio-based digital person image generation method according to claim 1, wherein said skeleton generation model comprises:
The bone generation model includes a bone keypoint detection model.
4. A digital person image generation method based on a virtual concert hall according to claim 1 or 3, wherein inputting the digital person image into a skeleton generation model to obtain a skeleton matrix comprises:
Extracting bone key points through the bone key point detection model;
sequentially selecting two adjacent bone key points from the bone key points;
Initializing a preset number of three-dimensional zero matrixes;
And filling areas corresponding to the bone key points adjacent to two positions in each three-dimensional zero matrix with preset pixel values so as to convert each three-dimensional zero matrix into the bone matrix corresponding to the bone key points adjacent to each two positions.
5. The virtual studio-based digital person image generation method according to claim 1, wherein inputting the digital person image into a skin generation model to obtain a skin matrix comprises:
generating a grid according to the digital human image, and determining skin weights of grid vertices influenced by the skeleton key points; the grid simulates the skin shape of a human body;
Based on the grid connectivity, averaging adjacent edge features of the grid vertices to obtain edge axes;
after forward transmission of the adjacent edge features, predicting coordinate values of the grid vertexes according to the skeleton key points and the skin weights;
and determining a digital human skin matrix according to the edge axis and the predicted grid vertex coordinate value.
6. The virtual studio-based digital person image generation method according to claim 1, comprising:
the digital person image generation model, the bone generation model, and the skin generation model are constructed from a grid convolution and a bone convolution.
7. The virtual studio-based digital person image generation method according to claim 1, wherein after inputting the skeleton matrix and the skin matrix into the digital person image generation model, obtaining a digital person image, further comprising:
And rendering the images of the digital person image generation model in the virtual studio in real time.
8. A digital person image generation system based on a virtual concert hall, comprising:
The creation module is used for creating a three-dimensional virtual studio; a three-dimensional scene is preset in the virtual concert hall;
The importing module is used for importing a digital human image generating model into the virtual studio;
The acquisition module is used for acquiring the digital human image;
the first input module is used for inputting the digital human image into a skeleton generation model to obtain a skeleton matrix;
the second input module is used for inputting the digital human image into a skin generation model to obtain a skin matrix;
And the third input module is used for inputting the skeleton matrix and the skin matrix into the digital human image generation model to obtain the digital human image.
9. The virtual studio-based digital person image generation system according to claim 8, further comprising:
and the positioning and adjusting module is used for positioning and adjusting the digital human image generating model so as to adapt the digital human image generating model to the three-dimensional scene of the virtual concert hall.
10. The virtual studio-based digital person image generation system according to claim 8, further comprising:
And the rendering module is used for rendering the images of the digital person image generation model in the virtual studio in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410775826.9A CN118587362A (en) | 2024-06-17 | 2024-06-17 | Digital human image generation method and system based on virtual concert hall |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410775826.9A CN118587362A (en) | 2024-06-17 | 2024-06-17 | Digital human image generation method and system based on virtual concert hall |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118587362A true CN118587362A (en) | 2024-09-03 |
Family
ID=92527648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410775826.9A Pending CN118587362A (en) | 2024-06-17 | 2024-06-17 | Digital human image generation method and system based on virtual concert hall |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118587362A (en) |
-
2024
- 2024-06-17 CN CN202410775826.9A patent/CN118587362A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Roussou et al. | Photorealism and non-photorealism in virtual heritage representation | |
| US11488348B1 (en) | Computing virtual screen imagery based on a stage environment, camera position, and/or camera settings | |
| US12112418B2 (en) | Volumetric capture and mesh-tracking based machine learning 4D face/body deformation training | |
| Zhao et al. | Computer-aided graphic design for virtual reality-oriented 3D animation scenes | |
| CN113781660A (en) | Method and device for rendering and processing virtual scene on line in live broadcast room | |
| CN117788689A (en) | Interactive virtual cloud exhibition hall construction method and system based on three-dimensional modeling | |
| WO2022055367A1 (en) | Method for emulating defocus of sharp rendered images | |
| KR101273531B1 (en) | Between Real image and CG Composed Animation authoring method and system by using motion controlled camera | |
| EP3980975B1 (en) | Method of inferring microdetail on skin animation | |
| KR101859318B1 (en) | Video content production methods using 360 degree virtual camera | |
| CN118587362A (en) | Digital human image generation method and system based on virtual concert hall | |
| US20230121860A1 (en) | Interactive image generation | |
| CN117475068A (en) | Model data processing method and device, storage medium and electronic equipment | |
| CN116681854A (en) | Virtual city generation method and device based on target detection and building reconstruction | |
| JP2023153534A (en) | Image processing apparatus, image processing method, and program | |
| JPH11316853A (en) | Device and method for producing virtual reality feeling | |
| WO2022055368A1 (en) | Method for applying a vignette effect to rendered images | |
| KR102622709B1 (en) | Method and Apparatus for generating 360 degree image including 3-dimensional virtual object based on 2-dimensional image | |
| Booth et al. | Computers: Computers animate films and video: Computers permit spectacular visual effects and take the drudgery out of film cartooning | |
| CN118411453B (en) | Digital human-computer interaction method and system | |
| Yusufu | Research on 3D Animation Production System of Industrial Internet of Things under Computer Artificial Intelligence Technology | |
| JP7390265B2 (en) | Virtual viewpoint video rendering device, method and program | |
| CN117527994A (en) | Visual presentation method and system for space simulation shooting | |
| Wang | Design of Electrified Railroad Simulation System based on Virtual Reality Technology | |
| CN116934940A (en) | Method for generating model map by using panorama based on ray tracing technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |