CN117726728A - Avatar generation method, device, electronic equipment and storage medium - Google Patents

Abstract

The application provides an avatar generation method, an avatar generation device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a description text for creating an avatar; tracking an editing intention for the avatar based on the acquired descriptive text, and generating a reply text; and displaying a reply text and an avatar generated according to the editing intention, wherein the reply text is used for indicating the generated avatar to carry out editing response on the descriptive text. According to the interactive virtual image generation scheme, the operation of a user can be simplified, and interactive response can be timely made.

Description

Virtual image generation method, device, electronic equipment and storage medium

Technical field

The present application relates to the field of computer application technology, and in particular to a virtual image generation method, device, electronic equipment and storage medium.

Background technique

"Face pinching" provides the function of customizing virtual characters and is an important module in role-playing games, augmented reality, and the metaverse. It allows users to adjust the virtual character's skeletal position, makeup attributes and other character parameters to meet the user's personalized customization needs and greatly enhance the user's immersive experience.

However, although hundreds of character parameters provide great freedom in character creation, they also increase the user's operational burden. Currently, virtual character automatic generation technology is developing rapidly, allowing users to generate virtual characters through simple input, such as inputting images, text, etc., helping users save time and further improve playability.

Face pinching solutions in related technologies are usually non-interactive, requiring the user to fully describe the desired character image at one time, and then directly generate character customization results. However, fully describing the character image places a high burden on the user, and when text When the description content is long and complex, it is easy for the generated results to deviate too much from the description.

Contents of the invention

In view of this, embodiments of the present application provide at least a virtual image generation method, device, electronic device, and storage medium to overcome at least one of the above-mentioned disadvantages.

In a first aspect, exemplary embodiments of the present application provide a method for generating an avatar. The method includes: obtaining a description text for creating an avatar; based on the obtained description text, tracking the editing intention for the avatar, and generating Reply text; display the reply text and the avatar generated according to the editing intention, and the reply text is used to indicate the generated avatar's editing response to the description text.

In a second aspect, embodiments of the present application also provide a device for generating a virtual image. The device includes: an acquisition module to acquire description text for creating a virtual image; and an identification module to track the virtual image based on the acquired description text. Edit the intention and generate a reply text; a display module displays the reply text and the avatar generated according to the editing intention, and the reply text is used to indicate the generated avatar's editing response to the description text. .

In a third aspect, embodiments of the present application also provide an electronic device, a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processing The processor communicates with the storage medium through a bus, and the processor executes the machine-readable instructions to perform the steps of the above virtual image generation method.

In a fourth aspect, embodiments of the present application further provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the above virtual image generation method when run by a processor.

The virtual image generation method, device, electronic device and storage medium provided by the embodiments of the present application can simplify the player's operation to reduce the user's input burden, and can improve the processing efficiency of character creation through timely interactive response, thus contributing to Provide users with a more natural, convenient, and accurate interaction solution for character creation.

In order to make the above-mentioned objects, features and advantages of the present application more obvious and understandable, preferred embodiments are given below and described in detail with reference to the attached drawings.

Description of the drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore do not It should be regarded as a limitation of the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 shows a flow chart of a virtual image generation method provided by an exemplary embodiment of the present application;

Figure 2 shows a flowchart of language identification steps for description text provided by an exemplary embodiment of the present application;

Figure 3 shows a flow chart of multiple rounds of interactions provided by an exemplary embodiment of the present application;

Figure 4 shows a flow chart of the steps of training a correlation prediction model provided by an exemplary embodiment of the present application;

Figure 5 shows a flow chart of the steps of training a differentiable renderer provided by an exemplary embodiment of the present application;

Figure 6 shows a flow chart of the steps of determining a priori loss provided by an exemplary embodiment of the present application;

Figure 7 shows a schematic structural diagram of a virtual image generation device provided by an exemplary embodiment of the present application;

FIG. 8 shows a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the technical solutions in the embodiments of the present application The drawings are for illustration and description purposes only and are not intended to limit the scope of the present application. Additionally, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented in accordance with some embodiments of the application. It should be understood that the operations of the flowchart may be implemented out of sequence, and steps without logical context may be implemented in reverse order or simultaneously. In addition, those skilled in the art can add one or more other operations to the flow chart, or remove one or more operations from the flow chart under the guidance of the content of this application.

The terms "a", "an", "the" and "said" are used in this specification to indicate the existence of one or more elements/components/etc.; the terms "include" and "have" are used to indicate an open-ended Inclusive is intended and means that there may be additional elements/components/etc. in addition to the listed elements/components/etc.; the terms "first" and "second" etc. are used as labels only and do not refer to The number of its objects is limited.

It should be understood that in the embodiments of this application, "at least one" refers to one or more, and "multiple" refers to two or more. "And/or" is just an association relationship that describes related objects. It means that there can be three kinds of relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. Condition. The character "/" generally indicates that the related objects are in an "or" relationship. "Including A, B and/or C" means including any one, any two or three of A, B and C.

It should be understood that in the embodiment of the present application, "B corresponding to A", "B corresponding to A", "A corresponding to B" or "B corresponding to A" means that B is associated with A, B can be determined based on A. Determining B based on A does not mean determining B only based on A, but can also determine B based on A and/or other information.

In addition, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the application provided in the appended drawings is not intended to limit the scope of the claimed application, but rather to represent selected embodiments of the application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without any creative work fall within the scope of protection of this application.

"Face pinching" provides the function of customizing virtual characters and is an important module in role-playing games, augmented reality, and the metaverse. It allows users to adjust the virtual character's skeletal position, makeup attributes and other character parameters to meet the user's personalized customization needs and greatly enhance the user's immersive experience.

However, although hundreds of character parameters provide great freedom in character creation, they also increase the user's operational burden. Currently, virtual character automatic generation technology is developing rapidly, allowing users to generate virtual characters through simple input, for example, generating virtual characters based on input images or text, helping users save time and further improve playability.

Face pinching solutions in related technologies are usually single-round and non-interactive. For example, the user is required to give a complete description of the desired character image at one time, and then directly generate the character customization results. However, the above-mentioned face pinching has the following defects:

(1) Completely describing the character image at one time brings a greater input burden to the user. For situations where the user is not familiar with the face pinching scheme, the user cannot enter an accurate description at one time, and may require the user to try repeatedly to obtain the ideal generation result. In addition, if the user's input is not complete and accurate enough, it may cause deviation in the generated results.

(2) When the character image generated by the algorithm deviates from the description or the user's expectations, further adjustments cannot be made and only a long and complete description can be re-entered.

(3) When the text description content is long and complex, it also brings great challenges to the algorithm, and it is easy for the generated results to deviate too much from the description.

In addition, face pinching solutions in related technologies have problems such as too long computation time and non-robust computation effects, and are prone to generate abnormal results that do not conform to the natural distribution of faces. The above problems arise mainly due to the inefficient parameter search method adopted in the relevant scheme, which results in low computational efficiency. In addition, the related scheme generates parameters in an unconstrained space and lacks the constraints of a priori distribution, so it is easy to produce abnormal results that are not consistent with the natural distribution of faces.

In response to at least one of the above problems, this application proposes an interactive virtual image generation solution to reduce the user's input burden and help make the character creation process more natural, convenient, and accurate.

First, the names involved in the embodiments of this application are introduced.

Terminal Equipment:

The terminal device involved in the embodiments of this application mainly refers to an electronic device that can provide a user interface (User Interface) to realize human-computer interaction. In an exemplary application scenario, the terminal device can be used to provide game screens (such as games). Related settings/configuration interfaces, interfaces that present game scenes), and smart devices that can control and operate virtual characters. Terminal devices can include but are not limited to any of the following devices: smartphones, tablets, portable computers , desktop computers, game consoles, personal digital assistants (PDAs), e-book readers, MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer IV) players, etc. Applications supporting game scenes, such as applications supporting three-dimensional game scenes, are installed and run on the terminal device. The application may include but is not limited to virtual reality applications, three-dimensional map programs, military simulation programs, MOBA games (Multiplayer Online Battle Arena, multiplayer online tactical competitive games), multiplayer gun battle survival games, third-person shooters (TPS) , Third-Personal Shooting Game). Optionally, the application may be a stand-alone version of the application, such as a stand-alone version of a 3D (Three Dimensions, three-dimensional graphics) game program, or may be a network online version of the application.

GUI:

It is an interface display format for human-computer communication that allows users to use input devices such as mouse, keyboard and/or game controller to manipulate icons, logos or menu options on the screen. It also allows users to control icons, logos or menu options on the touch screen of a touch terminal. Touch to manipulate on-screen icons or menu options to select commands, launch programs, or perform other tasks. In the game scene, the game scene interface and the game configuration interface can be displayed in the graphical user interface.

Virtual scene:

It is the virtual environment displayed (or provided) when the application is running on the terminal device or server. Optionally, the virtual scene is a simulation environment of the real world, or a semi-simulation and semi-fictional virtual environment, or a purely fictitious virtual environment. The virtual scene can be any one of a two-dimensional virtual environment, a 2.5-dimensional virtual environment, and a three-dimensional virtual environment. The virtual environment can be the sky, land, ocean, etc. The virtual scene is a scene in which the user controls the complete game logic of the virtual character. Optionally, the virtual scene is also used for a virtual environment battle between at least two virtual objects. In the virtual scene, there is a space for at least two virtual characters to play. The virtual resource used.

Virtual character:

Refers to a virtual character in a virtual environment (such as a game scene). The virtual character can be a virtual character controlled by a player, including but not limited to at least one of a virtual character, a virtual animal, and an animation character. It can be a non-player controlled virtual character. A virtual character (NPC) can also be a virtual object, such as a static object in a virtual scene, for example, virtual props in the virtual scene, virtual tasks, a location in the virtual environment, terrain, houses, bridges, vegetation, etc. Static objects are often not directly controlled by the player, but can respond to the interactive behavior of the virtual character in the scene (such as attacking, demolishing, etc.) and make corresponding performances. For example, the virtual character can demolish, pick up, drag, and drop buildings. Construction etc. Optionally, virtual objects may also be unable to respond to the interactive behavior of virtual characters. For example, virtual objects may also be buildings, doors, windows, plants, etc. in the game scene, but virtual characters cannot interact with them. For example, virtual characters Windows cannot be damaged or dismantled. Optionally, when the virtual environment is a three-dimensional virtual environment, the virtual characters may be three-dimensional virtual models. Each virtual character has its own shape and volume in the three-dimensional virtual environment and occupies a part of the space in the three-dimensional virtual environment. Optionally, the virtual character is a three-dimensional character constructed based on three-dimensional human skeleton technology, and the virtual character achieves different external images by wearing different skins. In some implementations, the virtual character can also be implemented using a 2.5-dimensional or 2-dimensional model, which is not limited in the embodiments of the present application.

There may be multiple virtual characters in the virtual scene. The virtual characters are virtual characters controlled by players (that is, characters controlled by players through input devices and touch screens), or artificial characters set up in virtual environment battles through training. Artificial Intelligence (AI). Optionally, the virtual character is a virtual character competing in a game scene. Optionally, the number of virtual characters in the game scene battle is preset, or is dynamically determined based on the number of terminal devices participating in the virtual battle, which is not limited in the embodiments of the present application. In one possible implementation, the user can control the virtual character to move in the virtual scene, for example, control the virtual character to run, jump, crawl, etc., and can also control the virtual character to use skills, virtual props, etc. provided by the application. Fight with other virtual characters.

In an optional implementation, the terminal device may be a local terminal device. Taking games as an example, the local terminal device stores game programs and is used to present game screens. The local terminal device is used to interact with players through a graphical user interface, that is, conventionally downloading, installing and running game programs through electronic devices. The local terminal device may provide the graphical user interface to the player in a variety of ways. For example, it may be rendered and displayed on the display screen of the terminal device, or provided to the player through holographic projection. For example, the local terminal device may include a display screen and a processor. The display screen is used to present a graphical user interface. The graphical user interface includes a game scene screen and a game configuration interface. The processor is used to run the game and generate a graphical user interface. and controlling the display of the graphical user interface on the display screen.

Introduce the applicable application scenarios of this application. This application can be applied to the field of game technology. In the game, multiple players participating in the game join the same virtual game.

Before entering the virtual battle, players can choose different character attributes for their virtual characters in the virtual battle, such as identity attributes. By assigning different character attributes to determine different camps, players can compete in different matches of this virtual battle. In the stage, the task assigned by the game is performed to win the game competition. For example, multiple virtual characters with the attributes of character A "eliminate" the virtual characters with the attributes of character B in the game stage to win the game competition. Here, when entering the virtual battle, character attributes can also be randomly assigned to each virtual character participating in this virtual battle.

An implementation environment provided by an embodiment of the present application may include: a first terminal device, a server, and a second terminal device. The first terminal device and the second terminal device communicate with the server respectively to implement data communication. In this embodiment, the first terminal device and the second terminal device are each installed with an application program that executes the virtual image generation method provided by this application, and the server is a server that executes the virtual image generation method provided by this application. Through the application program, the first terminal device and the second terminal device can respectively communicate with the server.

Taking the first terminal device as an example, the first terminal device establishes communication with the server by running an application program. In an optional implementation, the server establishes a virtual battle based on the application's game request. The parameters of the virtual battle can be determined according to the parameters in the received game request. For example, the parameters of the virtual battle can include the number of people participating in the virtual battle, the level of the characters participating in the virtual battle, etc. When the first terminal device receives a response from the game server, the game scene corresponding to the virtual battle is displayed through the graphical user interface of the first terminal device. The first terminal device is a device controlled by the first user. The graphical user of the first terminal device The virtual character displayed in the interface is the player character (ie, the first virtual character) controlled by the first user. The first user inputs character operation instructions through the graphical user interface to control the player character to perform corresponding operations in the game scene.

Taking the second terminal device as an example, the second terminal device establishes communication with the server by running an application program. In an optional implementation, the server establishes a virtual battle based on the application's game request. The parameters of the virtual battle can be determined according to the parameters in the received game request. For example, the parameters of the virtual battle can include the number of people participating in the virtual battle, the level of the characters participating in the virtual battle, etc. When the second terminal device receives the response from the server, the game scene corresponding to the virtual battle is displayed through the graphical user interface of the second terminal device. The second terminal device is a device controlled by the second user, and the virtual character displayed in the graphical user interface of the second terminal device is the player character (ie, the second virtual character) controlled by the second user. The second user uses the graphical user interface to Input character operation instructions to control the player character to perform corresponding operations in the virtual scene.

The server performs data calculation based on the game data reported by receiving the first terminal device and the second terminal device, and synchronizes the calculated game data to the first terminal device and the second terminal device, so that the first terminal device and the second terminal device The device controls the graphical user interface to render corresponding game scenes and/or virtual characters based on the synchronized data sent by the game server.

In this embodiment, the first virtual character controlled by the first terminal device and the second virtual character controlled by the second terminal device are virtual characters in the same virtual battle. Wherein, the first virtual character controlled by the first terminal device and the second virtual character controlled by the second terminal device may have the same character attributes, or may have different character attributes. The first virtual character controlled by the first terminal device and the second virtual character controlled by the second terminal device may have the same character attributes. The second virtual character controlled by the second terminal device may belong to the same camp or different camps.

It should be noted that a virtual battle can include two or more virtual characters, and different virtual characters can correspond to different terminal devices. That is to say, in a virtual battle, there are more than two terminal devices. Send and synchronize game data with the game server respectively.

The virtual image generation method provided by the embodiment of the present application can be applied to virtual reality applications, three-dimensional map programs, military simulation programs, multiplayer online tactical competitive games (MOBA), multiplayer gunfight survival games, third-person battle games, first-person Any kind of battle game.

The virtual image generation method in one embodiment of the present application can be run on a local terminal device or a server. When the method is run on the server, the method can be implemented and executed based on a cloud interaction system, where the cloud interaction system includes a server and a client device.

In an optional implementation, various cloud applications, such as cloud games, can be run under the cloud interactive system. Take cloud gaming as an example. Cloud gaming refers to a gaming method based on cloud computing. In the running mode of cloud games, the running body of the game program and the game screen presentation body are separated. The storage and operation of the information display method are completed on the cloud game server. The role of the client device is used to receive data, transmission and presentation of the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a mobile terminal, a television, a computer, a handheld computer, etc.; however, the information processing is done in the cloud. Cloud gaming server. When playing a game, the player operates the client device to send operating instructions to the cloud game server. The cloud game server runs the game according to the operating instructions, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the cloud game server performs operations through the client device. Decode and output game screen.

In an optional implementation, taking a game as an example, the local terminal device stores the game program and is used to present the game screen. The local terminal device is used to interact with players through a graphical user interface, that is, conventionally downloading, installing and running game programs through electronic devices. The local terminal device may provide the graphical user interface to the player in a variety of ways. For example, it may be rendered and displayed on the display screen of the terminal, or provided to the player through holographic projection. For example, the local terminal device may include a display screen and a processor. The display screen is used to present a graphical user interface. The graphical user interface includes a game screen. The processor is used to run the game, generate the graphical user interface, and control the graphical user interface. displayed on the display screen.

In a possible implementation, an embodiment of the present invention provides a virtual image generation method that provides a graphical user interface through a terminal device, where the terminal device may be the aforementioned local terminal device or the aforementioned Client devices in cloud interactive systems.

In order to facilitate understanding of the present application, the virtual image generation method, device, electronic device and storage medium provided by the embodiments of the present application are introduced in detail below.

Please refer to FIG. 1 , which is a flow chart of an avatar generation method provided in an exemplary embodiment of the present application. It is usually applied in a game server, such as the cloud game server described above, but the present application is not limited thereto.

In the process of creating a virtual image in the exemplary embodiment of the present application, at least one round of interaction may be included. Exemplarily, each round of interaction may include but is not limited to: obtaining the description text used to create the virtual image in this round of interaction, and then displaying The results of this round of interaction, such as displaying the reply text and avatar generated by the description text of this round of interaction. The user can choose to end the interaction based on the avatar and reply text displayed in this round of interaction, or input the description text of the next round of interaction, and then display the avatar and reply text based on the description text input in the next round, so Repeat until the interaction is complete, resulting in the final created avatar. Through the above interactive virtual image creation solution, users are allowed to complete the creation and modification of characters through multiple rounds of natural language dialogue, providing users with a better character image creation experience.

The following describes one of the multiple rounds of interactions used to create an avatar with reference to Figure 1. The details are as follows:

Step S101: Obtain the description text used to create the avatar.

In the embodiment of the present application, the above description text can be obtained through multiple ways, and the present application does not limit this.

In a preferred example of the present application, the description text can be obtained through input performed on the avatar editing interface.

For example, a graphical user interface can be provided through a terminal device, and an avatar editing interface of the game is displayed on the graphical user interface. The avatar editing interface can be used to create and/or adjust the avatar.

For example, the above-mentioned avatar editing interface may include a text input area. At this time, based on the user's input information in the text input area, the description text for creating the avatar is obtained.

It should be understood that the above description text can be obtained through various input methods on the avatar editing interface, and this application does not limit this. For example, the voice data input by the user can also be collected based on the voice input control on the avatar editing interface, and the text content corresponding to the voice data can be obtained by identifying the collected voice data, and the text content can be determined as the description text. .

In addition to the above methods of obtaining description text, description text can also be received from other terminal devices, or description text can be obtained from the network.

In this embodiment of the present application, the description text may be text used to describe the overall features or local features of the avatar, or it may be a vague or precise description of the avatar. For example, the above description text may be a short text used to describe the appearance characteristics of the avatar, for example, a natural sentence to reduce the user's input burden.

Here, the virtual image may refer to the image presented in the virtual scene. From the perspective of the structure of the virtual image, the virtual image can be the image of a three-dimensional model or a flat image. In terms of the type of avatar, an avatar can be a virtual image formed by simulating a human image, a virtual image formed by simulating an animal image, or a virtual image formed based on images in cartoons and comics.

For example, the virtual image is generally a virtual character image, and the virtual character can be a person, an animal, etc. The created virtual image can be the overall image of the virtual character or a partial image of the virtual character. For example, the virtual image can refer to the facial image of the virtual character.

Step S102: Based on the obtained description text, track the editing intention for the avatar and generate a reply text.

Here, the corresponding editing intention and reply text can be determined through language recognition of the description text. For example, a large language model (LLM, Large Language Model) can be pre-trained, and the description text is used as the input of the LLM to obtain the editing intention and reply text corresponding to the description text.

Alternatively, another large-scale language model can be pre-trained, and the description text can be used as the input of the large-scale language model to obtain the editing intention corresponding to the description text, and then the corresponding reply text can be generated based on the determined editing intention.

The above reply text is related to the identified editing intention, or in other words, the reply text can reflect the avatar generation algorithm's understanding of the intention of the description text.

In a preferred embodiment of the present application, a large language model is introduced into the virtual image creation system, and a memory mechanism is designed for it, that is, a large language model with a memory mechanism is formed. The above large language model with a memory mechanism is It can track the user's editing process to achieve a more accurate understanding and analysis of complex dialogue processes.

The following is an introduction to the process of intent understanding based on a large language model with a memory mechanism in conjunction with Figure 2.

FIG. 2 shows a flowchart of language identification steps for description text provided by an exemplary embodiment of the present application.

As shown in Figure 2, in step S201, the interactive editing memory used to create the avatar is extracted.

Here, the above-mentioned interactive editing memory is used to indicate the interaction history for the creation process of the avatar, that is, the editing history of the avatar in the historical interaction rounds before the current round of interaction. For the first round of interaction, the interactive editing memory extracted in step S201 is empty. In this case, the editing intention and reply text are determined based only on the obtained description text.

In a first embodiment, the above-mentioned interactive editing memory may include historical conversations.

For example, the historical dialogue includes at least one dialogue group. Here, one round of interaction may correspond to one dialogue group. Each dialogue group may include, but is not limited to, the description text in the round of interaction and its corresponding reply text.

In a second embodiment, the interactive editing memory may include editing attributes and related editing strengths corresponding to the editing attributes.

For example, the editing attributes include the preset character image parameters in the editing state among the multiple character image parameters. Here, multiple character image parameters for characterizing the external image of the virtual image can be predefined for the virtual image. For example, it can include but It is not limited to character image parameters used to characterize face shape, body shape, clothing, hair color, hairstyle, skin color, etc. Under any of the above types of parameters, multiple character image parameters can also be included. As an example, for face shape, it can Including but not limited to character image parameters such as round face, square face, long face, etc.

Here, being in an editing state may refer to the character image parameters that have been edited in the process of creating the avatar. For example, the character image parameters involved in the historical interaction rounds before this round of interaction, that is, in the historical interaction rounds with Edit the character image parameters associated with the intent. Here, the editing intention identified in each round of interaction is used to characterize the editing object of this round of interaction, and the editing object may refer to the character image parameters that need to be adjusted in this round of interaction.

For example, the correlation editing strength is used to characterize the degree of correlation between the preset character image parameters and the description text, or in other words, the correlation editing strength is used to characterize the correlation degree between the preset character image parameters and the identified editing intention.

In a third embodiment, the above-mentioned interactive editing memory may simultaneously include historical conversations, editing attributes, and related editing intensity corresponding to the editing attributes.

In step S202, based on the description text and interactive editing memory, the editing intention and reply text for the avatar are obtained.

For example, the interactive editing memory and description text used to create the avatar can be input into a pre-trained large language model to obtain the editing intention and reply text.

Regarding the above first embodiment of interactive editing memory, historical dialogues can be extracted from the dialogue memory structure.

For example, the dialogue memory structure stores dialogue groups in historical interaction rounds before this round of interaction. The initial state of the dialogue memory structure is empty. As the interaction rounds proceed, the content stored in the dialogue memory structure is processed. renew.

Specifically, the dialogue memory structure can be updated in the following way: whenever a round of interaction ends, the description text corresponding to the round of interaction and its corresponding reply text are added to the dialogue memory structure for storage.

In a preferred embodiment of the present application, while obtaining the editing intention and reply text for the avatar, the related editing strength can also be obtained. The related editing strength obtained here is used to characterize the relationship between the target character image parameters and the description text. The degree of association, the target character image parameter is determined according to the editing intention recognized in this round of interaction, that is, the character image parameter associated with the editing intention of this round of interaction.

Illustratively, the interactive editing memory and description text used to create the avatar can be input into a pre-trained large language model to obtain editing intent, related editing intensity, and reply text.

Here, the large language model involved in the above embodiments can be trained in various ways, and this part will not be described again in this application.

Regarding the second embodiment of the above-mentioned interactive editing memory, the editing attributes and the related editing intensity corresponding to the editing attributes can be extracted from the editing state memory structure.

For example, the editing state memory structure stores the editing attributes involved in the historical interaction rounds before this round of interaction and their corresponding related editing strengths, that is, the character image parameters edited in the historical interaction rounds and their corresponding Relevant to the editing intensity, the initial state of the editing state memory structure is empty. As the interaction round proceeds, the content stored in the editing state memory structure is updated.

Specifically, the editing state memory structure can be updated in the following way: whenever a round of interaction ends, search the editing state memory structure to see whether there is an editing intention corresponding to the round of interaction, that is, search whether there is an editing intention corresponding to the round of interaction. Editing properties. If it exists, the relevant editing intensity corresponding to the searched editing intention is updated. For example, the above update can refer to increasing the value of the relevant editing intensity to reflect the higher degree of correlation. If it does not exist, then the corresponding round of interaction is The editing intention and related editing strength are added to the editing state memory structure.

Returning to Figure 1, step S103: display the reply text and the avatar generated according to the editing intention.

Here, the reply text is used to indicate the editing response of the generated avatar to the description text. For example, the editing response is used to indicate the description of the creation or adjustment of the avatar in this round of interaction. In other words, the creation or adjustment of the avatar in this round of interaction is consistent with the content recorded in the reply text, so that users who want to create an avatar can be informed of the creation and adjustment process of the algorithm in a timely manner. In addition, it can also help guide the user to input description text in the next round of interaction to a certain extent, thereby improving the processing efficiency of the avatar.

Regarding the above situation of displaying the virtual image editing interface through the terminal device, the above virtual image editing interface may include an image display area and a system response area in addition to a text input area.

For example, the virtual image generated according to the editing intention can be presented in the image display area, and the reply text can be displayed in the system response area. In a preferred example, during the first round of interactive input of description text, only a text input area can be provided on the avatar editing interface for receiving the description text, and the image display area and system response are not displayed on the avatar editing interface. area, or, while providing a text input area for the first round of interaction, the image display area and the system response area can be displayed, but no content will be displayed in the above two areas.

Here, the description text, avatar and reply text related to this round of interaction can be displayed in the text input area, image display area and system response area. Preferably, a historical input viewing control, a historical image viewing control, and a historical editing viewing control can also be displayed on the avatar editing interface. For example, the historical input viewing control can be displayed at a position associated with the text input area, in response to For the operation of the historical input viewing control, the description text obtained in the historical rounds is displayed in the text input area, and the description text obtained in the current round of interaction and the historical rounds can also be displayed in the text input area at the same time.

Correspondingly, the historical image viewing control can be displayed at a location associated with the image display area, and the historical editing viewing control can be displayed at a location associated with the system response area. Here, the operations for the historical image viewing control and the historical editing viewing control are as follows: The operation of the historical input viewing control is similar, and this part will not be described again in this application.

The multi-round interaction process of creating a virtual image is introduced below with reference to Figure 3.

Figure 3 shows a flow chart of multiple rounds of interaction provided by an exemplary embodiment of the present application.

As shown in Figure 3, in step S301, description text for creating an avatar is obtained.

The multi-round interaction process is explained below through a non-limiting example. For example, the description text for this round of interaction input by the user on the avatar editing interface is “The girl doesn’t looks cute enough. You should make some modifications.” Here, the embodiment of the present application does not limit the language of the description text.

In step S302, based on the description text and interactive editing memory, the editing intention, related editing intensity and reply text are obtained.

For the situation where this round of interaction is not the first round of interaction, for example, the following content can be stored in the dialogue memory structure:

‘user’: Make the skin fair.

‘system’:…

…

The above list is the historical dialogue stored in the dialogue memory structure, where 'user' refers to the description text in the corresponding interaction round, and 'system' refers to the reply text in the corresponding interaction round.

For example, the editing status memory structure can store the following content:

‘round face’: 0.5

‘fair skin’: 0.5

…

The above list is the editing attributes stored in the editing state memory structure and their corresponding related editing strengths. Among them, 'round face' and 'fair skin' refer to the editing attributes "round face" and "fair skin" respectively, and the corresponding values of the editing attributes. Used to characterize the relative editing intensity corresponding to the editing attribute. For example, the larger the value, the higher the degree of correlation, and the smaller the value, the lower the degree of correlation.

Following the above example, the historical dialogue, editing attributes and related editing intensity in the editing state can be extracted from the dialogue memory structure and the editing state memory structure, and filled into the pre-designed prompt together with the description text input by the user. For example, Prompt can include the following items:

‘user input’:

'histery':

'state':

Among them, 'user input' is filled with description text, 'histery' is filled with historical conversations, and 'state' is filled with editing attributes and related editing intensity in the editing state.

Input the filled Prompt into the pre-trained large language model. For example, you can get the parsed instructions in json form, as follows:

Target: Big Eyes

Strength: 0.5

Response: "I increased the size of the eyes a bit.Dose she looks better?"

Among them, the content of Target corresponds to the parsed editing intention T _k , that is, Big Eyes, which represents the target character image parameters that need to be edited for this round of interaction inferred from the description text and interactive editing memory. Here, the editing intention may be a text instruction for subsequent editing of the avatar, for example, used to indicate an editing operation performed on the avatar obtained in the previous round of interaction.

The content of Strength corresponds to the parsed relevant editing strength s, that is, 0.5, which represents the association strength between the inferred editing intention of this round of interaction and the description text.

The content of Response corresponds to the reply text R _k presented to the user by the avatar creation system, that is, "Iincreased the size of the eyes a bit.Dose she looks better?".

After this round of interaction ends, the conversation memory structure can be updated according to the description text y _k and the reply text R _k , and the editing status memory structure can be updated according to the editing intention T _k and the related editing intensity s. Here, the subscript k indicates that the current round of interaction is the kth round, where 1≤k≤M, and M is a positive integer, which may be, for example, the preset maximum number of rounds of interaction.

In step S303, the initial latent variable is obtained.

Here, the optimized latent variables can be obtained through multiple iterations of the initial latent variables, and then the character image parameters used to generate the virtual image can be obtained.

In a preferred embodiment of the present application, the above-mentioned initial latent variables are used to represent the projection of character image parameters in a low-dimensional space.

Here, the initial latent variables z _k are the optimized character image parameters corresponding to the latest iteration. Projection into low-dimensional space. For the case of non-first iteration, the latest iteration refers to the last iteration of the current iteration. For the case of non-first round of interaction but the first iteration, the latest iteration refers to the last iteration of the previous round of the current interaction round. iterate.

For the first iteration of the first round of interaction, the above-mentioned initial latent variable is the projection of the character image parameters corresponding to the baseline avatar in the low-dimensional space. For example, the baseline avatar includes a virtual character that matches the description text of the first round of interaction. For example, the avatar that has the highest matching degree with the description text of the first round of interaction can be selected from multiple pre-created candidate avatars and determined as the above-mentioned reference avatar.

For the case where the virtual image is a facial image of a virtual character, the above character image parameters may include face pinching parameters, and the face pinching parameters may include parameters used to characterize the head features of the virtual character. Examples may include but are not limited to the following. At least one of the items: parameters characterizing facial features (eye shape/size, nose shape/size, lip shape/size), parameters characterizing head shape, hairstyle, and hair color characteristics.

In step S304, initial image parameters are obtained by decoding the initial latent variables.

Here, the initial latent variable can be decoded back to the original space (ie, character image parameter space) through the back-projection matrix to obtain the initial image parameters. The above-mentioned projection and back-projection processes between the low-dimensional space and the original space are common knowledge in the art, and this part will not be described again in this application.

In an optional embodiment, the initial image parameters obtained after the above decoding can be directly used as candidate image parameters to perform subsequent step S307. In addition, a correlation vector can also be introduced to determine candidate image parameters based on the initial image parameters and the correlation vector.

Specifically, in step S305, a correlation vector is obtained.

For example, the correlation between multiple character image parameters and the editing intention determined by this round of interaction can be predicted to obtain a correlation vector. Here, each element in the correlation vector is used to characterize the relationship between the corresponding character image parameter and the editing intention. the strength of the correlation between them.

For example, a pre-trained correlation prediction model can be used to input the parsed editing intention T _k into the correlation prediction model to predict the correlation between each character image parameter and the editing intention, thereby forming a Correlation vector r _k .

The following describes the training process of the correlation prediction model with reference to Figure 4.

FIG. 4 shows a flowchart of steps of training a correlation prediction model provided by an exemplary embodiment of the present application.

As shown in Figure 4, in step S401, a first training sample is obtained.

For example, the first training sample can be obtained in the following way: a large language model writes multiple description text samples for the avatar according to the example, and the large language model rough annotation and manual fine annotation are used to obtain the description text sample corresponding to each description text sample. Corresponding text correlation labels, so as to form a first training sample from the plurality of description text samples and corresponding text correlation labels.

In step S402, the description text sample of the current iteration is input into the correlation prediction model to obtain a predicted correlation vector.

In step S403, the loss function is calculated.

For example, a loss function can be calculated between the predicted correlation vector and the textual correlation label corresponding to the description text sample. Here, various methods can be used to calculate the loss function, and this application does not limit this.

In step S404, network parameters are optimized.

For example, the loss function can be used to update the network parameters in the correlation prediction model through gradient descent.

In step S405, it is determined whether the current iteration reaches the maximum number of iterations.

If the maximum number of iterations is not reached, the number of iterations is increased by 1, and execution returns to step S402 to continue training the correlation prediction model.

If the maximum number of iterations is reached, step S406 is executed: save the model parameters. At this time, a trained correlation prediction model is obtained, that is, a correlation prediction model constructed based on the above optimized network parameters.

Returning to Figure 3, in step S306, candidate image parameters are obtained based on the initial image parameters and the correlation vector.

For example, it can be calculated by comparing the correlation vector r _k and the initial image parameters (e.g., ) is weighted. For example, an element in the correlation vector r _k can be multiplied by a corresponding image parameter in the initial image parameter to obtain the candidate image parameter x _k corresponding to the initial latent variable z _k .

In the embodiment of the present application, by introducing the correlation vector r _k , the virtual image can be finely adjusted to avoid modification of irrelevant areas. For example, in the above process, according to the text instructions of the editing intention, the correlation of the image parameters of each character is predicted through the correlation prediction model, for example, the relevant facial features area and makeup attributes are predicted, and then the predicted relevant Regions and properties can be edited.

In the embodiment of this application, the editing intention T _k , the relevant editing intensity s, and the correlation vector r _k are input into the FR-T2P model to obtain the character image parameters after iterative optimization (that is, after editing) In the above process, by continuously iterating the hidden variable of the projection of the character image parameters in the low-dimensional space, the encoding distance between the rendered image corresponding to the Cain variable and the text instruction of the editing intention is finally made as close as possible.

Specifically, in step S307, the rendered image of the game engine is obtained based on the candidate image parameters.

For example, the candidate image parameters x _k can be input into a pre-trained neural renderer network to obtain the corresponding rendered image. The rendered image may refer to the image of the virtual image rendered by the game engine.

For example, the neural renderer network can be a differentiable renderer, for example, trained according to pre-prepared data (character image parameters and game engine rendering image) to imitate the rendering process of the game engine, thereby realizing the rendering process. Differentiable.

In the process of generating character image parameters based on editing intention (i.e., textual instructions), a neural renderer network with makeup is pre-trained, thereby achieving the generation of character image parameters in a completely stochastic gradient descent manner. The process of training a differentiable renderer is introduced below with reference to Figure 5.

FIG. 5 shows a flowchart of steps for training a differentiable renderer provided by an exemplary embodiment of the present application.

As shown in Figure 5, in step S501, a second training sample is obtained.

For example, the second training sample may include randomly sampled sample image parameters and avatar images rendered by a game engine. The randomly sampled image parameters are used as input, and the avatar image rendered by the game engine is used as an output. The differentiable renderer Conduct training.

For the case where the virtual image refers to the facial image of the virtual character, continuous facial parameters and discrete makeup parameters can be randomly sampled and the facial image of the corresponding game virtual character rendered by the game engine can be used as training data.

In step S502, the sample image parameters are input to the differentiable renderer to obtain a predicted rendering image of the game engine.

For the above example of face pinching, the continuous facial parameters can be input into the differentiable renderer to render the predicted facial image of the character.

In step S503, the loss function is calculated.

For example, a loss function can be calculated between a predicted rendered image rendered by a differentiable renderer and a rendered image image rendered by a game engine. Here, various methods can be used to calculate the loss function, and this application does not limit this.

In step S504, network parameters are optimized.

For example, a loss function can be used to update network parameters in a differentiable renderer via gradient descent.

In step S505, it is determined whether the current iteration reaches the maximum number of iterations.

If the maximum number of iterations is not reached, the number of iterations is increased by 1, and step S502 is returned to continue training the differentiable renderer.

If the maximum number of iterations is reached, step S506 is executed: save the model parameters. At this time, a trained differentiable renderer is obtained, that is, a differentiable renderer built based on the above optimized network parameters.

In the embodiment of the present application, the optimization goal is to reduce the deviation between the editing intention and the rendered image, and the candidate image parameters are optimized through multiple iterations to obtain optimized character image parameters.

Specifically, returning to Figure 3, in step S308, the rendered image is input into CLIP to obtain image coding.

For example, the cross-modal coding model CLIP (Contrastive Language-Image Pre-Training) can be pre-trained to determine the image encoding based on the rendered image of the game engine corresponding to the candidate image parameter of the current iteration.

In step S309, based on the editing intention, the text encoding is obtained.

For example, the editing intention T _k (ie, text instruction) can be input into the pre-trained CLIP to obtain text encoding.

In step S310, the intent understanding loss is obtained according to the determined image encoding and text encoding.

For example, the cosine distance between the image encoding and the text encoding can be calculated, and the cosine distance is determined as the intent understanding loss CLIP Loss.

In the embodiment of this application, the initial latent variable z _k will be optimized based on the gradient descent method to minimize the above-mentioned cosine distance, that is, the candidate image parameters will be optimized with the intention understanding loss as the optimization goal.

In a preferred embodiment of the present application, the above-mentioned related editing intensity s obtained based on the description text will also control the editing intensity by affecting the CLIP loss weight during the above-mentioned optimization process. For example, the product of the relevant editing intensity s and the intention understanding loss CLIP Loss can be calculated, so that the candidate image parameters are optimized with the intention understanding loss under the influence of the relevant editing intensity (ie, the above-mentioned product) as the optimization target.

In addition to the most important optimization objective (CLIP Loss) mentioned above in the gradient descent method, in a preferred embodiment of the present application, a priori regularization is also applied to the initial latent variable z _k .

Specifically, in step S311, a priori loss is obtained.

For example, in the single-round generation process of text-to-character image parameters, a prior distribution is introduced and a complete gradient descent framework is used to achieve more robust and faster generation of character image parameters.

FIG. 6 shows a flowchart of steps for determining a priori loss provided by an exemplary embodiment of the present application.

As shown in Figure 6, in step S601, multiple groups of samples are obtained from the avatar sample set.

Here, multiple sets of samples can be obtained from a public virtual character image image data set, and each set of samples includes multiple character image parameters.

In step S602, the prior distribution statistical values corresponding to multiple groups of samples are counted.

For example, the prior distribution statistical values of the character image parameters of multiple groups of samples may include but are not limited to the mean μ _Z and the covariance A _Z . Here, the method of calculating the mean and covariance is common knowledge in the art, and this application does not limit it.

In step S603, the prior loss is determined based on the prior distribution statistical value.

For example, the prior loss can be calculated by the following formula:

priorLoss＝||A _Z ·(z-μ _Z )|| ²

Among them, priorLoss represents the prior loss, and z represents the character image parameter of the current iteration. For example, the above z can also represent the initial latent variable of the current iteration.

In a preferred embodiment, a priori constraints can be imposed during the optimization process, and the candidate image parameters can be optimized with intent understanding loss and a priori loss as optimization goals.

In this process, the prior distribution is introduced, so that the parameter optimization space is limited to the prior distribution space and additional prior distribution constraints are imposed, thereby avoiding the generation of results that do not conform to the real face distribution.

Returning to Figure 3, in step S312, it is determined whether the optimization conditions are met, that is, whether the optimization process is completed.

For example, whether the optimization process ends can be determined by determining whether the maximum number of iterations is reached.

If the optimization conditions are not met (for example, the maximum number of iterations is not reached), the hidden variable adjustment value is determined, the initial hidden variable is updated based on the hidden variable adjustment value, and the execution step S303 is returned to proceed again based on the updated initial hidden variable. iterate.

For example, the latent variable adjustment value can be obtained in the following manner: inputting the intent understanding loss and the prior loss to the optimizer to determine the latent variable adjustment value through the optimizer, and the latent variable adjustment value represents the image for at least one character image The degree and direction of parameter adjustment can be expressed as ±ΔZ.

If the optimization conditions are met (for example, the maximum number of iterations is reached), step S313 is executed: determine the character image parameters.

For example, under the above optimization goal, the initial hidden variable z _k is continuously iteratively optimized using gradient descent, and the optimal hidden variable is obtained by convergence. Then through back-projection, the character image parameters can be obtained/>

In step S314, the avatar and the reply text are displayed.

For example, based on the character image parameters determined above Generate an avatar corresponding to the editing intention. For example, the determined avatar and the reply text can be displayed simultaneously on the avatar editing interface where the description text is input.

In step S315, it is determined whether the conversation ends.

Here, the dialogue end condition can be set in advance (for example, the maximum number of interaction rounds is reached). When it is detected that the set dialogue end condition is met, it is determined that the dialogue has ended. When it is detected that the set dialogue end condition is not met, it is determined that the dialogue has not been completed. Finish.

In addition to the above method, a control button for triggering the end of the dialogue can also be set on the avatar editing interface. When an operation on the control button is detected, the end of the dialogue is determined. If no operation on the control button is detected, then Make sure the conversation is not over.

If it is determined that the conversation has not ended, the process returns to step S301 and continues to receive the description text input by the user in the next round of interaction.

If it is determined that the dialogue is over, the avatar displayed in the current round of interaction is determined to be the avatar created by the user.

Based on the above multiple rounds of interaction, it is possible to create or adjust the avatar through natural language dialogue, without the need to manually adjust parameters repeatedly, and without knowing the specific meaning of each parameter. Fine adjustments can also be achieved to avoid irrelevant problems. The area is changed, so that the time cost spent by the user in the avatar adjustment process can be greatly reduced.

Based on the same application concept, the embodiments of this application also provide a virtual image generation device corresponding to the method provided in the above embodiments. Since the problem-solving principle of the device in the embodiments of this application is the same as the virtual image generation method in the above embodiments of this application, are similar, so the implementation of the device can refer to the implementation of the method, and repeated details will not be repeated.

Figure 7 is a schematic structural diagram of a virtual image generation device provided by an exemplary embodiment of the present application.

As shown in Figure 7, the virtual image generation device 200 includes:

The acquisition module 210 obtains the description text used to create the virtual image;

The identification module 220 tracks the editing intention for the avatar based on the obtained description text, and generates a reply text;

The display module 230 displays the reply text and the avatar generated according to the editing intention, and the reply text is used to indicate the editing response of the generated avatar to the description text.

In a possible implementation of the present application, the identification module 220 is also used to: extract the interactive editing memory used to create the virtual image, the interactive editing memory is used to indicate the interaction history of the creation process of the virtual image; based on the The description text and the interactive editing memory are used to obtain the editing intention for the avatar and the reply text.

In a possible implementation of the present application, the creation process includes at least one round of interaction. In each round of interaction, the corresponding reply text and avatar are generated based on the description text of the round of interaction, wherein the interaction The editing memory includes historical dialogues, the historical dialogues include at least one dialogue group, each dialogue group includes description text and its corresponding reply text in a round of interaction, and/or, the interactive editing memory includes editing attributes and editing attributes. Corresponding related editing strength, the editing attribute includes the preset character image parameter in the editing state among the plurality of character image parameters, and the related editing strength is used to represent the degree of association between the preset character image parameter and the description text.

In a possible implementation of the present application, the identification module 220 is also used to extract historical dialogues from the dialogue memory structure, where the identification module 220 updates the dialogue memory structure in the following manner: whenever a round of interaction ends, Add the description text corresponding to this round of interaction and its corresponding reply text to the dialogue memory structure.

In a possible implementation of the present application, the display module 230 generates a virtual image according to the editing intention in the following manner: based on the candidate image parameters, obtains the rendering image of the game engine; to narrow the gap between the editing intention and the rendering image. The deviation of is the optimization target, and the candidate image parameters are optimized through multiple iterations to obtain optimized character image parameters; according to the character image parameters, a virtual image corresponding to the editing intention is generated.

In a possible implementation of the present application, the display module 230 obtains the candidate image parameters in the following manner: obtaining the initial latent variable of the current iteration, which represents the projection of the character image parameters in a low-dimensional space; The initial latent variable is decoded to obtain the candidate image parameters of the current iteration; where the initial latent variable is the projection of the optimized character image parameters corresponding to the latest iteration in the low-dimensional space. In the first iteration of the first round of interaction, the The initial latent variable is the projection of the character image parameters corresponding to the base virtual image in the low-dimensional space. The base virtual image includes the virtual image that matches the description text of the first round of interaction.

In a possible implementation of the present application, the display module 230 obtains the candidate image parameters in the following manner: predicting the correlation between multiple character image parameters and the editing intention to obtain a correlation vector, in which Each element is used to represent the correlation strength between the corresponding character image parameters and the editing intention; the initial image parameters are obtained by decoding the initial latent variables, which represent the projection of the character image parameters in the low-dimensional space; according to The initial image parameters and the correlation vector are used to obtain candidate image parameters; wherein, the initial latent variable is the projection of the optimized character image parameters corresponding to the latest iteration in the low-dimensional space. In the first iteration of the first round of interaction When , the initial latent variable is the projection of the character image parameters corresponding to the reference virtual image in the low-dimensional space, and the reference virtual image includes the virtual image that matches the description text of the first round of interaction.

In a possible implementation of the present application, the display module 230 optimizes the candidate image parameters in the following manner: determining the image encoding based on the rendered image of the game engine corresponding to the candidate image parameters of the current iteration; based on the editing According to the intention, the text encoding is determined; according to the determined image encoding and text encoding, the intention understanding loss is obtained; with the intention understanding loss as the optimization target, the candidate image parameters are optimized.

In a possible implementation of the present application, the display module 230 is also used to: obtain multiple groups of samples from the virtual image sample set, each group of samples includes multiple character image parameters; and count the prior distributions corresponding to the multiple groups of samples. Statistical value; determine the prior loss according to the prior distribution statistical value; wherein the multiple iterations use the intention understanding loss and the prior loss as optimization goals to optimize the candidate image parameters.

In a possible implementation of the present application, the identification module 220 is also configured to: while obtaining the editing intention for the avatar based on the description text, also obtain the relevant editing intensity, which is used to characterize the target. The degree of association between the character image parameters and the description text, the target character image parameters are determined according to the editing intention; wherein, the loss of intention understanding under the influence of the relevant editing intensity is the optimization goal, and the candidate image is Parameters are optimized.

In a possible implementation of the present application, the recognition module 220 obtains the editing intention, related editing strength and reply text in the following manner: inputting the interactive editing memory used to create the avatar and the description text into a pre-trained large language model to obtain the editing intent, relevant editing intensity, and the reply text.

In a possible implementation of the present application, the interactive editing memory includes editing attributes and related editing intensity corresponding to the editing attributes, wherein the identification module 220 extracts the interactive editing memory for creating the avatar in the following manner: from the editing state The editing attributes and the relevant editing intensity corresponding to the editing attributes are extracted from the memory structure. The identification module 220 updates the editing state memory structure in the following manner: whenever a round of interaction ends, search the editing state memory structure to see whether there is any interaction with that round. If the corresponding editing intention exists, the relevant editing intensity corresponding to the searched editing intention is updated. If it does not exist, the editing intention and the relevant editing intensity corresponding to this round of interaction are added to the editing state memory structure.

In a possible implementation of the present application, the acquisition module 210 is also used to: display the avatar editing interface of the game. The avatar editing interface includes a text input area, an image display area and a system response area, based on where the user is. The input information in the text input area is obtained to obtain the description text, and/or the display module 230 is also used to: present the virtual image generated according to the editing intention in the image display area, and when the system responds The reply text is displayed in the area.

In a possible implementation manner of the present application, the virtual image includes a facial image of a virtual character, and the character image parameters include face pinching parameters.

Based on the above device, a virtual image can be generated through interface interaction and respond to the interaction in a timely manner.

Please refer to FIG. 8 , which is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application. As shown in FIG. 8 , the electronic device 300 includes a processor 310 , a memory 320 and a bus 330 .

The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 and the memory 320 communicate through the bus 330, and the machine-readable instructions are When the processor 310 is executed, it may execute the steps of the virtual image generation method in any of the above embodiments, specifically as follows:

Obtain the description text used to create the avatar; based on the obtained description text, track the editing intention for the avatar, and generate a reply text; display the reply text and the avatar generated according to the editing intention, and the reply The text is used to indicate the generated avatar's editing response to the description text.

Based on the above electronic devices, a virtual image can be generated through interface interaction and respond to the interaction in a timely manner.

Embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is run by a processor, the computer program can perform the steps of the virtual image generation method in any of the above embodiments, specifically as follows:

Obtain the description text used to create the avatar; based on the obtained description text, track the editing intention for the avatar, and generate a reply text; display the reply text and the avatar generated according to the editing intention, and the reply The text is used to indicate the generated avatar's editing response to the description text.

Based on the above computer-readable storage medium, a virtual image can be generated through interface interaction and respond to the interaction in a timely manner.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program code. .

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, and they should be covered by within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (17)

1. A method for generating a virtual image, characterized in that the method includes: Get the description text used to create the avatar; Based on the obtained description text, track the editing intention for the avatar and generate a reply text; The reply text and the avatar generated according to the editing intention are displayed, and the reply text is used to indicate the editing response of the generated avatar to the description text. 2. The method according to claim 1, wherein the step of tracking the editing intention for the avatar based on the obtained description text and generating a reply text includes: Extracting an interactive editing memory used to create the avatar, the interactive editing memory being used to indicate an interaction history for the creation process of the avatar; Based on the description text and the interactive editing memory, the editing intention for the avatar and the reply text are obtained. 3. The method according to claim 2, characterized in that the creation process includes at least one round of interaction, and in each round of interaction, the corresponding reply text and avatar are generated based on the description text of the round of interaction, Wherein, the interactive editing memory includes historical dialogues, and the historical dialogues include at least one dialogue group, and each dialogue group includes the description text and its corresponding reply text in a round of interaction, And/or, the interactive editing memory includes editing attributes and related editing strengths corresponding to the editing attributes. The editing attributes include preset character image parameters in an editing state among multiple character image parameters, and the related editing strengths are used to represent Preset the degree of correlation between the character image parameters and the description text. 4. The method according to claim 3, wherein the step of extracting the interactive editing memory used to create the avatar includes: Extract historical dialogues from dialogue memory structures, Wherein, the dialogue memory structure is updated in the following manner: whenever a round of interaction ends, the description text corresponding to the round of interaction and its corresponding reply text are added to the dialogue memory structure. 5. The method according to claim 1, characterized in that the avatar is generated according to the editing intention in the following manner: Based on the candidate image parameters, obtain the rendered image of the game engine; With the optimization goal of reducing the deviation between the editing intention and the rendered image, the candidate image parameters are optimized through multiple iterations to obtain optimized character image parameters; According to the character image parameters, an avatar corresponding to the editing intention is generated. 6. The method according to claim 5, characterized in that the candidate image parameters are obtained in the following manner: Obtain the initial latent variable of the current iteration, which represents the projection of the character image parameters in the low-dimensional space; By decoding the initial latent variable, the candidate image parameters of the current iteration are obtained; Wherein, the initial latent variable is the projection of the optimized character image parameters corresponding to the latest iteration in the low-dimensional space. In the first iteration of the first round of interaction, the initial latent variable is the character image parameter corresponding to the benchmark virtual image in A projection of the low-dimensional space, the baseline avatar includes an avatar that matches the description text of the first round of interaction. 7. The method according to claim 5, characterized in that the candidate image parameters are obtained in the following manner: Predict the correlation between multiple character image parameters and the editing intention to obtain a correlation vector, where each element in the correlation vector is used to characterize the correlation strength between the corresponding character image parameter and the editing intention; By decoding the initial latent variables, the initial image parameters are obtained, and the initial latent variables represent the projection of the character image parameters in the low-dimensional space; Obtain candidate image parameters according to the initial image parameters and the correlation vector; Wherein, the initial latent variable is the projection of the optimized character image parameters corresponding to the latest iteration in the low-dimensional space. In the first iteration of the first round of interaction, the initial latent variable is the character image parameter corresponding to the benchmark virtual image in A projection of the low-dimensional space, the baseline avatar includes an avatar that matches the description text of the first round of interaction. 8. The method according to any one of claims 5-7, characterized in that the candidate image parameters are optimized in the following manner: Determine the image encoding based on the rendered image of the game engine corresponding to the candidate image parameter of the current iteration; Based on the editorial intent, determine text encoding; Intent understanding loss is obtained based on the determined image encoding and text encoding; Taking the intention understanding loss as the optimization target, the candidate image parameters are optimized. 9. The method according to claim 8, characterized in that the method further comprises: Obtain multiple groups of samples from the avatar sample set, each group of samples includes multiple character image parameters; Statistics of prior distribution statistical values corresponding to the multiple groups of samples; Determine the prior loss based on the prior distribution statistical value; Wherein, the multiple iterations use the intention understanding loss and the prior loss as optimization targets to optimize the candidate image parameters. 10. The method according to claim 8, characterized in that the method further comprises: While obtaining the editing intention for the virtual image based on the description text, the relevant editing intensity is also obtained. The relevant editing intensity is used to represent the degree of association between the target character image parameters and the description text. The target character image parameters are based on Determined by the editorial intent; Wherein, the candidate image parameters are optimized with the loss of intention understanding under the influence of the relevant editing intensity as the optimization target. 11. The method according to claim 10, characterized in that the editing intention, related editing intensity and reply text are obtained in the following manner: The interactive editing memory used to create the avatar and the description text are input into a pre-trained large-scale language model to obtain the editing intention, related editing intensity and the reply text. 12. The method according to claim 10, characterized in that the interactive editing memory includes editing attributes and related editing intensity corresponding to the editing attributes, Among them, the interactive editing memory used to create the avatar is extracted in the following ways: Extract the editing attributes and the related editing intensity corresponding to the editing attributes from the editing status memory structure, Wherein, the editing status memory structure is updated in the following manner: Whenever a round of interaction ends, search the editing state memory structure to see if there is an editing intention corresponding to the round of interaction. If it exists, update the relevant editing intensity corresponding to the searched editing intention. If it does not exist, the editing intention and related editing intensity corresponding to this round of interaction are added to the editing state memory structure. 13. The method of claim 1, wherein the step of obtaining description text for creating an avatar includes: Display the virtual image editing interface of the game, the virtual image editing interface includes a text input area, an image display area and a system response area, Obtain the description text based on the user's input information in the text input area, And/or, the step of displaying the reply text and the avatar generated according to the editing intention includes: Present the virtual image generated according to the editing intention in the image display area, Display the reply text in the system response area. 14. The method according to claim 5, wherein the virtual image includes a facial image of a virtual character, and the character image parameters include face pinching parameters. 15. A virtual image generating device, characterized in that the device includes: Get the module to get the description text used to create the avatar; The identification module, based on the obtained description text, tracks the editing intention for the avatar and generates a reply text; A display module displays the reply text and the avatar generated according to the editing intention, and the reply text is used to indicate the editing response of the generated avatar to the description text. 16. An electronic device, characterized in that it includes: a processor, a storage medium and a bus, the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and The storage media communicate with each other through a bus, and the processor executes the machine-readable instructions to perform the steps of the method according to any one of claims 1 to 14. 17. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the method according to any one of claims 1 to 14 are executed. .