CN117425044A

CN117425044A - Video generation method, first device, second device and storage medium

Info

Publication number: CN117425044A
Application number: CN202311347975.7A
Authority: CN
Inventors: 李锐; 文锋
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-10-17
Filing date: 2023-10-17
Publication date: 2024-01-19

Abstract

The embodiment of the application provides a video generation method, first equipment, second equipment, electronic equipment and a computer readable storage medium, and relates to the technical field of streaming media processing. The method comprises the following steps: acquiring real-time pose information of a dynamic capture object in a dynamic capture space; synchronously mapping the pose information to a virtual object in a first virtual space; the real-time states of the synchronous target virtual object in the first virtual space and the second virtual space respectively; and responding to the real-time pose information of the virtual object in the first virtual space, interacting the target virtual object in the first virtual space in a real-time state, generating a real-time virtual reality picture, and generating a video stream according to the real-time virtual reality picture. According to the method and the device for capturing the video, the interaction between the dynamic capturing object and the target virtual object through the second device is achieved, color bars and fixing sites are not required to be advanced, and video generation efficiency is improved.

Description

Video generation method, first device, second device and storage medium

Technical Field

The present application relates to the field of streaming media processing technologies, and in particular, to a video generating method, a first device, a second device, an electronic device, a computer readable storage medium, and a computer program product.

Background

In the related art, in the process of motion capture, a mark point is required to be set in advance at a place of a virtual object needing interaction in a motion capture space, for example, a position needing the motion capture object to move is attached to the ground to assist the motion capture object to perform, and in the process of actual performance, the motion capture object performs through the mark point in the motion capture space.

In an actual project, because marking points need to be set in advance, a moving object needs to be color-arrayed in advance, the points are fixed, then performance is performed according to the fixed points in actual live broadcast, the flow is complex, meanwhile, dynamic points cannot be determined, for example, when the moving object needs to follow a dynamically moving virtual character, the moving object is difficult to perform particularly, because a real-time dynamic object does not exist in an actual field, the moving object needs to imagine matching pictures to perform non-physical performance, and therefore the requirement on the moving object is high.

Disclosure of Invention

Embodiments of the present application provide a video generating method, a first device, a second device, an electronic device, a computer readable storage medium, and a computer program product, which can solve the above-mentioned problems of the prior art. The technical scheme is as follows:

According to a first aspect of embodiments of the present application, there is provided a video generating method, applied to a first device, the method including:

acquiring real-time pose information of a dynamic capture object in a dynamic capture space;

synchronously mapping the pose information to a virtual object in a first virtual space; the first virtual space is a virtual space with a virtual scenery;

the real-time state of the synchronous target virtual object in the first virtual space and the second virtual space respectively, wherein the second virtual space is a virtual space obtained by mapping the dynamic capture space by the second equipment; the second device is used for displaying an augmented reality picture to the dynamic capture object in real time, wherein the augmented reality picture is a picture for interacting with a target virtual object in a real-time state in the second virtual space based on real-time pose information of the dynamic capture object in a dynamic capture space;

generating a real-time virtual reality picture in response to interaction between the virtual object and a target virtual object in the first virtual space, wherein the virtual reality picture is a picture for interacting with the target virtual object in the first virtual space in a real-time state based on real-time pose information of the virtual object in the first virtual space;

And generating a video stream according to the real-time virtual reality picture.

According to a second aspect of embodiments of the present application, there is provided a video generating method, applied to a second device, the method including:

the method comprises the steps that real-time states of a synchronous target virtual object in a first virtual space and a second virtual space are respectively achieved, so that a first device generates a video stream, the first virtual space is a virtual space with a virtual scenery, and the second virtual space is a virtual space obtained by mapping a dynamic capturing space by the second device;

acquiring real-time pose information of a dynamic capture object in a dynamic capture space, and generating and displaying an augmented reality picture in response to interaction between the dynamic capture object and a target virtual object in the second virtual space, wherein the augmented reality picture is a picture for interacting with the target virtual object in a real-time state in the second virtual space based on the real-time pose information of the dynamic capture object in the dynamic capture space;

the video stream is generated according to a virtual reality picture, wherein the virtual reality picture is a picture for interacting with a target virtual object in a real-time state in a first virtual space based on real-time pose information of the virtual object in the first virtual space;

The pose information of the virtual object is obtained by synchronously mapping real-time pose information of the dynamic capture object in the dynamic capture space to the first virtual space.

According to a third aspect of embodiments of the present application, there is provided a first device comprising:

the pose obtaining module is used for obtaining real-time pose information of the dynamic capture object in the dynamic capture space;

the pose mapping module is used for synchronously mapping the pose information to the virtual object in the first virtual space; the first virtual space is a virtual space with a virtual scenery;

the first state synchronization module is used for synchronizing real-time states of a target virtual object in the first virtual space and the second virtual space respectively, and the second virtual space is a virtual space obtained by mapping the dynamic capture space by the second equipment; the second device is used for displaying an augmented reality picture to the dynamic capture object in real time, wherein the augmented reality picture is a picture for interacting with a target virtual object in a real-time state in the second virtual space based on real-time pose information of the dynamic capture object in a dynamic capture space;

the virtual reality picture generation module is used for responding to interaction between the virtual object and the target virtual object in the first virtual space to generate a real-time virtual reality picture, wherein the virtual reality picture is based on real-time pose information of the virtual object in the first virtual space and is used for interacting with the target virtual object in the first virtual space in a real-time state;

And the video stream generating module is used for generating a video stream according to the real-time virtual reality picture.

According to a fourth aspect of embodiments of the present application, there is provided a second apparatus comprising:

the second state synchronization module is used for synchronizing real-time states of the target virtual object in a first virtual space and a second virtual space respectively, so that the first device generates a video stream, the first virtual space is a virtual space with a virtual scenery, and the second virtual space is a virtual space obtained by mapping the dynamic capture space by the second device;

the augmented reality picture generation module is used for obtaining real-time pose information of the dynamic capture object in the dynamic capture space, responding to interaction between the dynamic capture object and the target virtual object in the second virtual space, and generating and displaying an augmented reality picture, wherein the augmented reality picture is a picture for interacting with the target virtual object in the real-time state in the second virtual space based on the real-time pose information of the dynamic capture object in the dynamic capture space;

According to another aspect of embodiments of the present application, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory, the processor executing the computer program to implement the steps of the method provided in the first or second aspect above.

According to a further aspect of embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method provided by the first or second aspect described above.

According to an aspect of embodiments of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method provided by the first or second aspect described above.

The beneficial effects that technical scheme that this application embodiment provided brought are:

the method and the device are applied to streaming media processing scenes, the dynamic capturing objects are different from the pictures seen by audiences, the dynamic capturing objects see the augmented reality pictures displayed by the second equipment, the virtual reality pictures seen by the audiences are generated by responding to interaction of virtual objects on target virtual objects in the first virtual space, and particularly the pictures of interaction of the target virtual objects in the first virtual space in real-time state based on real-time pose information of the virtual objects in the first virtual space. Because real-time pose information of the dynamic capture object in the dynamic capture space and real-time pose information of the virtual object in the first virtual space are corresponding, and real-time states of the target virtual object in the first virtual space and the second virtual space are synchronous, when the dynamic capture object interacts with the target virtual object in the second virtual space, a viewer can synchronously see a picture of the interaction of the virtual object with the target virtual object in the first virtual space, and the dynamic capture object interacts with the target virtual object through the second device, color bars and fixed sites do not need to be advanced, and video generation efficiency is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.

FIG. 1a is a schematic diagram of an dynamic capture space according to an embodiment of the present disclosure;

fig. 1b is a schematic architecture diagram of a live video system according to an embodiment of the present application;

fig. 2 is a flow chart of a video generating method according to an embodiment of the present application;

fig. 3 is a schematic diagram of a virtual space obtained by scanning a conference room according to an embodiment of the present application;

fig. 4 is a schematic diagram of an augmented reality screen according to an embodiment of the present application;

fig. 5a is an interaction schematic diagram of a video generating method according to an embodiment of the present application;

fig. 5b is an interaction schematic diagram of another video generating method according to an embodiment of the present application;

fig. 6 is a flowchart of a video generating method at a first device side according to an embodiment of the present application;

fig. 7 is a flowchart of a video generating method at a second device side according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an interface for unifying the coordinate system of the dynamic capture space and the coordinate system of the second virtual space according to the embodiment of the present application

Fig. 9 is a flowchart of a video generating method according to an embodiment of the present application;

Fig. 10 is an interaction schematic diagram of a live video scene provided in an embodiment of the present application;

fig. 11 is a schematic structural diagram of a first device according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of a second device according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below with reference to the drawings in the present application. It should be understood that the embodiments described below with reference to the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and the technical solutions of the embodiments of the present application are not limited.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and "comprising," when used in this application, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof, all of which may be included in the present application. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates that at least one of the items defined by the term, e.g., "a and/or B" may be implemented as "a", or as "B", or as "a and B".

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Several terms which are referred to in this application are first introduced and explained:

the augmented reality (Augmented Reality, AR) technology is a technology of skillfully fusing virtual information with a real world, and widely uses various technical means such as multimedia, three-dimensional modeling, real-time following and registration, intelligent interaction, sensing and the like, and applies virtual information such as characters, images, three-dimensional models, music, videos and the like generated by a computer to the real world after simulation, wherein the two kinds of information are mutually complemented, so that the 'enhancement' of the real world is realized.

Virtual Reality (VR), also known as Virtual Reality or ambience technology, is a computer simulation system that can create and experience a Virtual world by using a computer to create a simulated environment into which a user is immersed. The virtual reality technology is to use data in real life, combine electronic signals generated by computer technology with various output devices to convert the electronic signals into phenomena which can be perceived by people, wherein the phenomena can be real and cut objects in reality or substances which can not be seen by naked eyes, and the phenomena are shown by a three-dimensional model. These phenomena are not directly visible but are simulated by computer technology in the real world, and are therefore referred to as virtual reality.

Virtual man: is a digital character image which is created by digital technology and is close to the human image.

Dynamic catching space: in order for a virtual person to move up as a real person, motion capture techniques are required to capture the motion of the real person. However, in order to perform motion capture, a specific spatial range is generally required to be set, as shown in fig. 1a, which shows a schematic diagram of a motion capture space, it can be seen that a spatial range surrounded by a plurality of cameras is the motion capture space, and the plurality of cameras capture motions of a real person from different angles to obtain motion capture data.

In an actual project, because marking points need to be set in advance, a moving object needs to be color-arrayed in advance, the points are fixed, then performance is performed according to the fixed points in actual shooting, the flow is complex, dynamic points cannot be determined, for example, when the moving object needs to follow a dynamically moving virtual character, the moving object is difficult to perform, because no real-time dynamic object exists in an actual field, the moving object needs to imagine matching with a picture to perform no-real-object performance, and therefore the requirement on the moving object is high.

The present application provides a video generating method, a first device, a second device, an electronic device, a computer readable storage medium and a computer program product, which aim to solve the above technical problems in the prior art.

The technical solutions of the embodiments of the present application and technical effects produced by the technical solutions of the present application are described below by describing several exemplary embodiments. It should be noted that the following embodiments may be referred to, or combined with each other, and the description will not be repeated for the same terms, similar features, similar implementation steps, and the like in different embodiments.

Referring to fig. 1b, fig. 1b is a schematic architecture diagram of a live video system 100 according to an embodiment of the present application, in order to support an exemplary application, a terminal (a terminal 400-1 and a terminal 400-2 are shown in an exemplary manner) are connected to a server 200 through a network 300, where the network 300 may be a wide area network or a local area network, or a combination of the two, and a wireless link is used to implement data transmission.

The terminal may be a device with a data acquisition (e.g., video acquisition) function, such as a second device, a virtual VR device, an MR device, etc.; as another example, the second device may be a smart phone with AR function, a tablet computer, a notebook computer, an AR glasses, or other various types of user terminals, or may be a desktop computer with AR function, a television, or a combination of any two or more of these data processing devices; the server 200 may be a server supporting various services, which is configured separately, may be configured as a server cluster, may be a cloud server, or the like.

In practical application, the data acquisition equipment on the terminal acquires pose information of the dynamic capture object in the dynamic capture space, and it can be understood that the pose information includes position information and pose information, namely, the position and the pose of the dynamic capture object (also can be a specific part of the dynamic capture object, such as a hand, an arm, a head, and the like) in the dynamic capture space. Mapping pose information to virtual objects in a first virtual space, which in embodiments of the present application is a virtual space with a virtual scenery, such as a virtual football field, a virtual space station, etc., in some embodiments, the first virtual space may be a space in which a dynamic capture object controls a virtual object to play a game, and in example a running cool game, the virtual scenery may be one or more of a virtual floor, a virtual fire ring, a virtual trap, a virtual post. By way of example, the present application may utilize machine learning/deep learning algorithms, such as artificial neural networks, confidence networks, reinforcement learning, and migration learning, to pre-construct an object model of a virtual object in a game, and implement rendering processing on a virtual scene, so that each virtual object included in a virtual layout displayed on a game interface is more realistic, and actions of each virtual object are smoother, and detailed descriptions of specific implementation processes of the present application are omitted.

It should be noted that, for the game described in the above embodiments of the present application, the game may be an offline game, that is, the game is installed and operated by an object on an electronic device such as a mobile phone, a tablet computer, a desktop computer, a notebook computer, etc., and in the process of game operation, the electronic device may execute the video generation method provided in the present application, so as to meet the requirement of generating a game video. In the game running process, the function implementation of supporting the game interface to display different virtual scenery scenes can be realized locally on the electronic equipment, namely, the electronic equipment calls corresponding functional components stored locally to support the realization of corresponding scenery updating, and the specific realization process is not described in detail herein.

Since the virtual object is driven by the motion capture data (i.e., real-time pose information) of the motion capture object, the virtual object performs an action that is completely identical to the motion capture object. The terminal of the embodiment of the application also displays the augmented reality image to the dynamic capturing object in real time, so that the dynamic capturing object can see the real scene and the virtual object in the second virtual space, the second virtual space of the embodiment of the application is the virtual space obtained by scanning (mapping) the dynamic capturing space by the terminal, therefore, the sizes of the virtual space and the dynamic capturing space are consistent, the target virtual object is arranged at the appointed position in the virtual space, and the dynamic capturing object is displayed by the terminal, so that the problem that the dynamic capturing object predicts the position of the virtual object by imagination and interacts with the virtual object in the prior art is avoided.

The embodiment of the application provides a video generation method, as shown in fig. 2, which includes:

s101, acquiring real-time pose information of a dynamic capture object in a dynamic capture space.

It should be noted that, when the embodiments of the present application are applied to a live scene, the dynamic capture object may be a host.

According to the embodiment of the application, the dynamic capture object is located in the dynamic capture space, at least one camera capable of capturing motion of the dynamic capture object is arranged around the dynamic capture space, and the motion of the dynamic capture object in the dynamic capture space in real time can be captured by the camera to corresponding pose information and sent to the first device. It is understood that pose information includes information in two dimensions, position and pose. In some embodiments, the pose information of the dynamic capture object may refer to pose information of a specific part of the dynamic capture object, and taking a hand as an example, the pose information of the hand may include coordinates of the hand in the dynamic capture space and gestures.

It should be understood that, in the embodiments of the present application, when relevant data (e.g., pose information) is collected and processed in an example application, the informed consent or independent consent of the personal information body should be obtained strictly according to the requirements of relevant national laws and regulations, and the subsequent data use and processing actions should be performed within the scope of the laws and regulations and the authorization of the personal information body.

In this embodiment of the present application, a coordinate system of the dynamic capturing space may be pre-constructed, where an origin of the coordinate system may be located at a center of a ground of the dynamic capturing space, and the coordinate may be a right-hand rectangular coordinate system (right-hand system), where positive directions of an x-axis, a y-axis and a z-axis are defined as follows: the right hand is placed at the original point, so that the thumb, the index finger and the middle finger are mutually right-angled, the thumb points to the positive direction of the x axis, and when the index finger points to the positive direction of the y axis, the direction pointed by the middle finger is the positive direction of the z axis. The right-hand (left-hand) coordinate system may also be determined as follows: this coordinate system is referred to as the right hand (left hand) coordinate system if the thumb of the right hand (left hand) is pointing in the forward direction of the first axis (x-axis) and the remaining fingers are held in the direction of rotation of the second axis (y-axis) about the first axis, coinciding with the third axis (z-axis).

S102, mapping the pose information to the virtual objects in the first virtual space synchronously.

In the embodiment of the application, the first virtual space is pre-rendered, and the first virtual space is a virtual space with a virtual scenery, and the virtual layout is related to an actual application environment, for example, a virtual court, a virtual space station and the like. According to the embodiment of the invention, the mapping relation between the dynamic capturing space and the first virtual space can be pre-established, so that the pose information of the dynamic capturing object can be mapped to the virtual object in the first virtual space according to the mapping relation, and the dynamic capturing object can control the movement of the virtual object.

S103, synchronizing real-time states of the target virtual object in the first virtual space and the second virtual space respectively.

The embodiment of the application also needs to pre-select and construct a virtual space corresponding to the dynamic capture space, namely a second virtual space. The second virtual space is obtained by scanning the dynamic capture space. When the second virtual space is constructed, the second equipment is slowly moved around the capturing space and scanned four sides, so that the second equipment can obtain more data of the relative positions among sampling points, and the precision is improved by scanning the key area for multiple times. Referring to fig. 3, which is an exemplary diagram illustrating a virtual space obtained by scanning a conference room, it can be seen that the virtual space well restores the actual layout of the conference room.

According to the method and the device for displaying the augmented reality picture to the dynamic capture object through the second device, it can be understood that the augmented reality picture is generated by superposing the virtual scene on the real scene, namely, the dynamic capture object can see the real limb, the real environment and the virtual object superposed on the display environment through the second device. Please refer to fig. 4, which schematically illustrates a schematic diagram of an augmented reality image provided by an embodiment of the present application, it can be seen from the diagram that a real hand and a virtual keyboard of a dynamic capture object are displayed in the image, and the dynamic capture object can simultaneously observe the hand and the virtual keyboard of the dynamic capture object through the image displayed by the second device, so that the purpose of accurate and virtual keyboard interaction can be achieved. It should be appreciated that in different interaction scenarios, the dynamic capture object may interact with different limbs and limbs of the target virtual object, e.g. in chess playing, decryption, etc., the dynamic capture object interacts with the target virtual object (e.g. virtual chess pieces) by hand, and in kicking scenarios, the dynamic capture object interacts with the target virtual object (e.g. virtual football) by legs and feet.

The augmented reality picture in the embodiment of the application is a picture for interacting with a target virtual object in a real-time state in a second virtual space based on real-time pose information of the dynamic capture object in the dynamic capture space. Taking fig. 4 as an example, the augmented reality image seen by the capturing object from the second device is an image that the finger of the capturing object interacts with the virtual keyboard, and since the virtual keyboard is displayed based on reality, the virtual keyboard has a definite position in reality, when the distance between the finger of the capturing object and a virtual key of the virtual keyboard is reduced to a preset threshold value, the capturing object is considered to interact with the virtual key, the capturing object will intuitively see the image that the virtual key is pressed, and it should be noted that the character corresponding to the virtual key can be synchronously displayed on the second device—the virtual reality image can display a text box, and each time when the virtual key is pressed, the character corresponding to the virtual key can be displayed in the text box, in some embodiments, the character corresponding to the virtual key can also be displayed not on the second device, but on other devices externally connected to the second device, and the other devices can be the first device.

The embodiment of the application needs to synchronize real-time states of the target virtual object in the first virtual space and the second virtual space respectively. It should be appreciated that the object is able to see the target virtual object in the first virtual space in order to interact with the target virtual object in the second virtual space, and what the viewer needs to see is the target virtual object in the first virtual space, i.e. the interaction result of the target virtual object in the virtual scenery, so when the object interacts with the target virtual object in the second virtual space and changes the state of the target virtual object, the state of the target virtual object needs to be synchronized to the first virtual space to generate the video stream.

In some embodiments, in addition to the situation where the real-time state of the target virtual object in the second virtual space is synchronized to the first virtual space, there is also a situation where the real-time state of the target virtual object in the first virtual space is synchronized to the second virtual space. Taking the first virtual space as a virtual football field and taking a target virtual object as a virtual football field as an example, if the original virtual football is located at the midfield position of the virtual football field, but when the position of the angle ball needs to be updated, the position of the virtual football field needs to be updated in the virtual football field, and meanwhile, the position of the virtual football field in the second virtual space is synchronously adjusted according to the updated position of the virtual football field.

The embodiments of the present application do not limit the type of the state of the target virtual object, and may include, for example, a position, a switch state, a color, a size, a shape, and the like.

And S104, responding to interaction between the virtual object and the target virtual object in the first virtual space, and generating a real-time virtual reality picture.

S105, generating a video stream according to the real-time virtual reality picture.

The method and the device are applied to streaming media processing scenes, the dynamic capturing objects are different from the pictures seen by audiences, the dynamic capturing objects see the augmented reality pictures displayed by the second equipment, the virtual reality pictures seen by the audiences are generated by responding to virtual objects to interact with target virtual objects in the first virtual space, and specifically, the pictures of interaction with the target virtual objects in real-time state in the first virtual space are generated based on real-time pose information of the virtual objects in the first virtual space. Because the real-time pose information of the dynamic capture object in the dynamic capture space and the real-time pose information of the virtual object in the first virtual space are corresponding, and the real-time states of the target virtual object in the first virtual space and the second virtual space are synchronous, the method realizes that when the dynamic capture object interacts with the target virtual object in the second virtual space, a spectator can synchronously see the picture of the interaction of the virtual object with the target virtual object in the first virtual space.

It should be noted that, in the embodiment of the present application, the video stream generated according to the real-time virtual reality picture may be used for live broadcast, or may be used for recorded broadcast, if the video stream is used for live broadcast, the video stream will be transmitted to the terminal of the viewer watching the live broadcast in real time, and if the video stream is used for recorded broadcast, the video stream will be stored in a preset database.

In a streaming media processing scene, a second device is worn in a dynamic capturing space by a dynamic capturing object, a target virtual object to be interacted is displayed in the second virtual space, and the second virtual space is a scanning result of the dynamic capturing space, so that the dynamic capturing object can see the target virtual object in the second virtual space through the second device and interact with the target virtual object, the performance of the dynamic capturing object is more natural, an interactive picture is an augmented reality picture, the real-time pose information of the dynamic capturing object in the dynamic capturing space is synchronously mapped to the virtual object in the first virtual space, the first virtual space is a virtual space with a virtual scenery, real-time pose information of the real-time target virtual object in the first virtual space and the second virtual space is determined based on the real-time pose information of the virtual object in the first virtual space, the picture of interaction of the target virtual object in the real-time state in the first virtual space, namely, a virtual reality picture is not needed, the dynamic space accurate interaction is achieved, and the interaction position of an interaction mark can be directly and even cancelled in advance.

On the basis of the foregoing embodiments, as an optional embodiment, the real-time states of the synchronization target virtual objects in the first virtual space and the second virtual space respectively include two modes:

in a first mode, a real-time state of the target virtual object in the second virtual space, which is sent by the second device, is obtained, and the real-time state of the target virtual object in the first virtual space is determined according to the mapping relation between the first virtual space and the second virtual space.

The first mode is directed to the case that the target virtual object in the second virtual space is a master object and the target virtual object in the first virtual space is a slave object. The second virtual space is created and maintained by the second device, the real-time state of the target virtual object in the second virtual space can be adjusted by the dynamic capture object through the second device, for example, the position, the shape, the size and the like of the target virtual object are adjusted, the dynamic capture object can also update actions in the dynamic capture space, the second device captures the actions of the dynamic capture object, the real-time pose of the dynamic capture object is determined, and the target virtual object is interacted based on the real-time pose of the dynamic capture object, so that the position of the target virtual object in the second virtual space is obtained. The second device sends the real-time state of the target virtual object in the second virtual space to the first device, and the first device determines the real-time state of the target virtual object in the first virtual space according to the mapping relation between the first virtual space and the second virtual space.

Referring to fig. 5a, an interactive schematic diagram of a video generating method according to an embodiment of the present application is shown, as shown in the figure, including:

s201, the second equipment collects real-time pose information of the dynamic capture object in the dynamic capture space and sends the real-time pose information to the first equipment;

s202, the first equipment synchronizes pose information to a virtual object in a first virtual space;

s203, the second device adjusts the real-time state of the target virtual object in the second virtual space and sends the adjusted real-time state to the first device;

s204, the first device adjusts the real-time state of the target virtual object in the first virtual space according to the adjusted real-time state of the target virtual object in the second virtual space;

s205, the first device responds to interaction between the virtual object and the target virtual object in the first virtual space, generates a real-time virtual reality picture, and generates a video stream according to the real-time virtual reality picture.

The timing relationship between steps S202 and S203 is not limited, and step S202 may be performed before step S203, may be performed after step S203, or may be performed simultaneously.

The second device may adjust the real-time state of the target virtual object in the second virtual space, which may be that the second device sets the state of the target virtual object so that the second device adjusts the real-time state of the target virtual object in the second virtual space, or that the second device responds to the interaction of the dynamic capturing object with the target virtual object in the second virtual space, and updates the real-time state of the target virtual object.

In a second mode, determining the real-time state of the target virtual object in the first virtual space, and sending the real-time state of the target virtual object in the first virtual space to a second device, so that the second device determines the real-time state of the target virtual object in the second virtual space according to the mapping relation between the first virtual space and the second virtual space.

The second mode is directed to a case where the target virtual object in the first virtual space is a master object and the target virtual object in the first virtual space is a slave object. The first virtual space is created and maintained by the first device, the dynamic capture object can adjust the real-time state of the target virtual object in the first virtual space through the first device, for example, adjust the position, the shape, the size and the like of the target virtual object, the first device sends the real-time state of the target virtual object in the first virtual space to the second device, and the second device determines the real-time state of the target virtual object in the second virtual space according to the mapping relation between the first virtual space and the second virtual space.

It should be noted that, although the target virtual object in the first virtual space may be the main object, because the virtual reality image is generated based on interaction between the virtual object and the target virtual object in the first virtual space, and the pose information of the virtual object is derived from the real-time pose information of the auto-capture object in the auto-capture space, the real-time state of the target virtual object in the first virtual space is changed based on interaction between the virtual object and the target virtual object most of the time, and the first device often updates the real-time state of the target virtual object in the first virtual space before live broadcast begins.

Referring to fig. 5b, an interactive schematic diagram of a video generating method according to an embodiment of the present application is shown, as shown in the figure, including:

s301, the second equipment collects real-time pose information of the dynamic capture object in the dynamic capture space and sends the real-time pose information to the first equipment;

s302, the first equipment synchronizes the pose information to a virtual object in a first virtual space;

s303, the first device determines the real-time state of the target virtual object in the first virtual space and sends a second device;

s304, the second device synchronizes the real-time state of the target virtual object in the first virtual space to the second virtual space;

s305, the second device responds to the interaction of the dynamic capture object on the target virtual object in the second virtual space, updates the real-time state of the target virtual object in the second virtual space and sends the real-time state to the first device;

s305, the first device synchronizes the real-time state of the target virtual object in the second virtual space to the first virtual space;

and S306, the first equipment responds to interaction between the virtual object and the target virtual object in the first virtual space, generates a real-time virtual reality picture, and generates a video stream according to the real-time virtual reality picture.

The timing relationship between steps S302 and S303 is not limited, and step S302 may be performed before step S303, may be performed after step S303, or may be performed simultaneously.

The mapping relationship between the first virtual space and the second virtual space is not particularly limited, and may be, for example, a mapping relationship of a coordinate system, a mapping relationship of a rendering effect, and the like.

On the basis of the foregoing embodiments, as an optional embodiment, generating a real-time virtual reality screen in response to interaction between the virtual object and the target virtual object in the first virtual space includes:

responding to interaction between the virtual object and a target virtual object in the first virtual space, and if the change degree of the state of the target virtual object in the first virtual space at the current moment in comparison with the previous moment is larger than a preset threshold value, generating a virtual reality picture at the current moment based on the current pose information of the virtual object in the first virtual space and the interaction result of the target virtual object at the current moment in the first virtual space;

and if the change degree of the state of the target virtual object in the first virtual space at the current moment is not greater than a preset threshold value compared with the change degree of the state of the target virtual object at the previous moment, taking the virtual reality picture at the previous moment as the virtual reality picture at the current moment.

The embodiment of the application provides an optimization scheme for live broadcast flow, specifically, the application generates a new virtual reality picture only when the state of a target virtual object in a first virtual space changes greatly, if the state of the target virtual object changes little, a last moment virtual reality picture is used, at this time, the first device sends indication information to a client corresponding to a spectator, and the indication information is used for indicating the client to take the last moment virtual reality picture as a current moment virtual picture, so that the sent flow bandwidth is saved.

In some embodiments, a first thread and a second thread that are independent of each other may be provided in the first device, where the first thread is used to synchronize real-time states of the target virtual object in two virtual spaces, and the second thread is used to generate a real-time virtual reality picture and a video stream, and the second thread sends the video stream at a fixed frame rate, so that the first thread is prevented from being blocked and affecting the sending of the video stream.

On the basis of the above embodiments, as an alternative embodiment, the state includes an object identifier of the target virtual object, a name of a function to be executed, and a parameter to be transferred into the function.

In practical application, the transmission structure of the state contains the following three information:

objectPath: object identification of the target virtual object;

functionName: names of functions to be executed

parameters: parameters of the function are to be entered.

The code information for one specific state is shown below:

in the code, the object identification of the target virtual object is represented by a storage path/Game/third PersonBP/Maps/third PersonExampleMap. Through the functions, the rotation of the target virtual object can be set. Similarly, other states of the target virtual object may be set by other functions.

On the basis of the foregoing embodiments, as an optional embodiment, the real-time states of the synchronization target virtual objects in the first virtual space and the second virtual space respectively, before further include:

And sending the object identification of the target virtual object to the second equipment, so that the second equipment deploys the target virtual object to an AR layer according to the object identification of the target virtual object, and renders the target virtual object in the second virtual space.

It should be noted that the number of virtual objects is very large in practical application, but the second device generally does not display all the virtual objects at a moment, but only displays the virtual objects deployed in an AR layer (AR layer), in this embodiment of the present application, the first device indicates the object identifier of the target virtual object to the second device, and the second device deploys the target virtual object to the AR layer according to the object identifier of the target virtual object, and renders the target virtual object in the second virtual space.

Referring to fig. 6, a flowchart of a video generating method at a first device side according to an embodiment of the present application is shown, where the flowchart includes:

s401, sending an object identifier of the target virtual object to second equipment so that the second equipment renders the target virtual object in the second virtual space;

s402, acquiring real-time pose information of a dynamic capture object in a dynamic capture space;

S403, synchronously mapping the pose information to a virtual object in a first virtual space;

s404, synchronizing real-time states of the target virtual object in the first virtual space and the second virtual space respectively;

s405, responding to interaction between the virtual object and a target virtual object in the first virtual space, and if the change degree of the state of the target virtual object in the first virtual space at the current moment is larger than a preset threshold value compared with the state of the target virtual object in the first virtual space at the previous moment, generating a virtual reality picture at the current moment based on the current pose information of the virtual object in the first virtual space; otherwise, taking the virtual reality picture at the previous moment as the virtual reality picture at the current moment;

s406, generating a video stream according to the real-time virtual reality picture and sending the video stream to a client of a spectator.

Referring to fig. 7, a flowchart of a video generating method at the second device side according to an embodiment of the present application is shown schematically, where:

s501, synchronizing real-time states of the target virtual object in the first virtual space and the second virtual space respectively, so that the first device generates a video stream.

The second device in this embodiment needs to synchronize real-time states of the target virtual object in the first virtual space and the second virtual space, so as to ensure that the state changes of the target virtual object in different virtual spaces are synchronous, that is, the state changes of the target virtual object observed by the capturing object and the state of the target virtual object observed by the audience are synchronous.

The first virtual space is a virtual space with a virtual scenery, so as to improve live broadcasting effect, enable spectators to see a more wonderful picture, namely a virtual reality picture, wherein the video stream is generated according to the virtual reality picture, the virtual reality picture is based on real-time pose information of the virtual object in the first virtual space, the real-time target virtual object in the first virtual space is interacted, and the pose information of the virtual object is obtained by synchronously mapping the real-time pose information of the dynamic capture object in the dynamic capture space to the first virtual space. That is, the first virtual space of the embodiment of the present application includes a virtual layout, a virtual object that synchronously maps real-time pose information of the dynamic capture object in the dynamic capture space, and a state synchronized with the target virtual object in the second virtual space.

The second virtual space is a virtual space obtained by mapping the dynamic capture space by the second equipment. According to the method and the device, the second equipment finely scans the dynamic capturing space, the virtual space which is as consistent as possible is obtained through three-dimensional reconstruction, and the coordinate system of the second virtual space is unified with the coordinate system of the dynamic capturing space, so that when the position of the target virtual object is set on the basis of the dynamic capturing space, the position of the target virtual object in the second virtual space can be quickly obtained, and the interaction between the dynamic capturing object and the target virtual object is facilitated through the second equipment.

S502, acquiring real-time pose information of the dynamic capture object in the dynamic capture space, and generating and displaying an augmented reality picture in response to interaction between the dynamic capture object and the target virtual object in the second virtual space.

According to the method and the device for capturing the real-time position and posture information of the dynamic capture object in the dynamic capture space, the second device can display the position of the target virtual object in the second virtual space to the dynamic capture object, so that the real-time state of the target virtual object after interaction with the dynamic capture object can be further determined by obtaining the real-time position and posture information of the dynamic capture object in the dynamic capture space, and an augmented reality picture is further generated. The augmented reality picture in the embodiment of the application is a picture for interacting with the target virtual object in the real-time state in the second virtual space based on the real-time pose information of the dynamic capture object in the dynamic capture space.

In the streaming media processing scene, the second equipment is worn in the dynamic capturing space by the dynamic capturing object, the target virtual object to be interacted is displayed in the second virtual space, the second virtual space is the scanning result of the dynamic capturing space, so that the dynamic capturing object can see the target virtual object in the second virtual space through the second equipment and interact with the target virtual object, the performance of the dynamic capturing object is more natural, the interacted picture is an augmented reality picture, the real-time pose information of the dynamic capturing object in the dynamic capturing space is synchronously mapped to the virtual object in the first virtual space, the first virtual space is the virtual space with virtual scenery, the real-time pose information of the real-time virtual object in the first virtual space is determined by synchronizing the real-time states of the target virtual object in the first virtual space and the second virtual space, the method and the device have the advantages that the dynamic space accurate interaction is achieved, the color arrangement time is greatly reduced or even cancelled, the interaction can be directly performed without marking the interaction position in advance, and because the coordinate systems of the second virtual space and the dynamic capturing space are unified in advance, when the position of the target virtual object is set in the second virtual space, the position of the target virtual object in the augmented reality picture can be quickly and intuitively determined, namely, the position of the target virtual object observed by the dynamic capturing object through the second device is improved, the efficiency of arranging the target virtual object is improved, and the method and the device are particularly suitable for the requirement that the dynamic capturing object is required to be interacted with the target virtual object in a live broadcast scene.

mapping the dynamic capture space to obtain the second virtual space;

establishing a coordinate system of the dynamic capturing space, establishing a coordinate system of the second virtual space according to the coordinate system of the dynamic capturing space, and determining a conversion relation between the coordinate system of the second virtual space and the coordinate system of the dynamic capturing space.

According to the embodiment of the application, the whole dynamic capturing space is scanned, the second equipment obtains data of the relative positions of more sampling points by slowly moving around the dynamic capturing space, after the environmental information of the dynamic capturing space is obtained, a mark point is made in the dynamic capturing space and used for determining the central position of the dynamic capturing space, and the direction and position information of a coordinate system are determined based on the mark point, so that the coordinate system consistent with the coordinate system of the second virtual space is ensured.

According to the embodiment of the application, the relative positions of the sampling points can be obtained through the dynamic capturing camera on the second device, and in practical application, a tool with a fixed relative position is used for carrying out 8-shaped waving in the dynamic capturing space, the position of the dynamic capturing camera is ensured to be unchanged, and after a period of time, the accurate relative position of the dynamic capturing camera in the dynamic capturing space can be obtained.

The embodiment of the application needs to verify whether the directions of the dynamic capture space and the second virtual space and the positions of the coordinate system agree. Specifically, a coordinate system model can be placed at the origin of the coordinate system of the second virtual space and points to three axial positive directions, and the origin and the positive directions of the coordinate system are shown to the dynamic capture object in the second device.

Please refer to fig. 8, which illustrates an interface schematic diagram of unifying a coordinate system of a dynamic capturing space and a coordinate system of a second virtual space in the embodiment of the present application, and as illustrated in the drawing, a scanned coordinate system frame 201 of the dynamic capturing space, a scanned coordinate system frame 202 of the second virtual space, and draggable controls 203 in different directions are displayed, and an operator realizes an effect of moving the second virtual space in the corresponding directions by dragging the controls 203 in the corresponding directions, so as to coincide positions of the two coordinate system frames.

It should be noted that, in the related art, the live barrage is usually displayed on the computer of the host (live capturing object), which causes that when the host wants to watch the barrage, the live capturing needs to be paused first, then the barrage is watched, and after the audience opinion is known, the live capturing is performed again, so that the host process is not smooth, or the host can live without looking at the audience opinion, and the live capturing effect is affected.

On the basis of the above embodiments, when the embodiments of the present application are applied to live scenes, the second device is further configured to display a barrage sent by a viewer. Specifically, after the audience inputs the barrage information, the terminal sends the barrage information to the first device, and the first device forwards the barrage information to the second device for display, so that the dynamic capture object is helped to adjust the interaction strategy according to the barrage information in real time. According to the live broadcast method and device, the capability of the second device for displaying information is utilized, bullet screen information is displayed in the second device, a host can watch the bullet screen while interacting with a target virtual object conveniently, smoothness of live broadcast is guaranteed, the host can adjust live broadcast strategies according to the bullet screen conveniently in time, efficiency is improved on an interaction level, and meanwhile live broadcast effects are improved.

Referring to fig. 9, a flowchart of a video generating method according to an embodiment of the present application is shown, and as shown in the drawing, the embodiment of the present application mainly includes three stages:

the preparation stage: the second equipment scans the dynamic capturing space to obtain a second virtual space, sets the central position of the second virtual space, captures real-time pose information of the dynamic capturing object in the dynamic capturing space (also called sweeping), sets the central position of the dynamic capturing space, and unifies the coordinate systems of the second virtual space and the dynamic capturing space according to the central positions of the second virtual space and the dynamic capturing space, so that the pose information of the subsequent dynamic capturing object in the dynamic capturing space and the coordinates of the target virtual object in the second virtual space are unified.

Engineering setting: and synchronously setting the target virtual objects needing interaction.

The target virtual object in the embodiment of the application is an object which needs to be seen by both a passive capturing object and a spectator, and the target virtual object can be a static grid object, a dynamic character object with bones, a virtual control and the like. According to the method and the device, the second virtual space is layered through the AR layer tool, and the virtual object needing interaction, namely the target virtual object, is arranged in the AR layer. The method and the device for real-time state determination of the target virtual object in the second virtual space can be configured to the second device or the first device, if the second device has the ownership of the target virtual object, the second device sends the real-time state of the target virtual object in the second virtual space to the first device, the first device determines the real-time state of the target virtual object in the first virtual space according to the mapping relation between the first virtual space and the second virtual space, and if the first device has the ownership of the target virtual object, the first device determines the real-time state of the target virtual object in the first virtual space and sends the real-time state of the target virtual object in the first virtual space to the second device, so that the second device determines the real-time state of the target virtual object in the second virtual space according to the mapping relation between the first virtual space and the second virtual space.

Live broadcasting stage: accurate dynamic interaction is realized mainly through data synchronization of the second equipment.

In the live broadcasting stage, the second device packages and sends real-time state of the target virtual object in the second virtual space and real-time pose information of the dynamic capturing object to the first device for synchronization, and the first device also can respond to interaction between the virtual object and the target virtual object in the first virtual space to generate a real-time virtual reality picture. The live broadcast picture seen by the final audience comprises a virtual object, a virtual object and a virtual scenery.

In the following, an interactive schematic diagram of a live game scene in the embodiment of the present application will be described with reference to a specific embodiment, as shown in fig. 10, an active capturing space 301 is deployed first, when an active capturing object (i.e. a host) 302 moves in the active capturing space, pose information of the active capturing object 302 in real time is acquired by an active capturing camera, the active capturing space 301 is scanned by a second device 303 to obtain a second virtual space 304, a coordinate system of the active capturing space 301 and a coordinate system of the second virtual space 304 are unified, a position of a virtual football 306 to be interacted is set in the second virtual space 304 by the second device 303 or a first device 305, the first device 305 creates a virtual court, the embodiment of the present application provides a virtual shooting game, and the first device creates the virtual court and can also provide scenario content, thereby enriching game experience, the second device 303 and the first device 305 synchronize the positions of the virtual football 305 in the second virtual space 304 and the virtual football 307 respectively, the second device 303 obtains real-time pose information of the dynamic capturing object in the dynamic capturing space 301, responds to the interaction between the dynamic capturing object 302 and the virtual football 306 in the second virtual space, generates and displays an augmented reality picture, thereby avoiding the defects that the dynamic capturing object needs to interact with virtual objects and color bars in advance, the first device 305 maps the real-time pose information of the dynamic capturing object 302 to the virtual football 308 of the virtual football field, the first device 305 responds to the interaction between the virtual football 308 and the virtual football 306 in the virtual football field, generates a real-time virtual reality picture, specifically, can simulate the motion trail of the virtual football according to the shooting force and the shooting angle of the virtual football, if the motion trail falls into the virtual football gate in the virtual football field, the virtual reality picture can also display score, goal prompt information and the like, the first device 305 generates a video stream according to the real-time virtual reality picture, the audience can see the live broadcast picture of the virtual football player playing the ball through the terminal 309, the audience can release bullet screen information through the terminal 309, the first device 305 forwards the bullet screen information to the second device 303, the second device 303 displays the bullet screen information to the host, and the host can timely adjust the live broadcast strategy according to the bullet screen information, such as adjusting the position of the virtual football, the ball playing mode, the virtual scenery and the like, so that the live broadcast effect is improved.

The embodiment of the application provides a first device, as shown in fig. 11, where the first device may include: a pose acquisition module 401, a pose mapping module 402, a first state synchronization module 403, a virtual reality picture generation module 404, and a video stream generation module 405, wherein,

the pose obtaining module 401 is configured to obtain real-time pose information of a dynamic capture object in a dynamic capture space;

a pose mapping module 402, configured to map the pose information to a virtual object in a first virtual space synchronously; the first virtual space is a virtual space with a virtual scenery;

a first state synchronization module 403, configured to synchronize real-time states of a target virtual object in the first virtual space and a second virtual space, where the second virtual space is a virtual space obtained by mapping the dynamic capture space by a second device; the second device is used for displaying an augmented reality picture to the dynamic capture object in real time, wherein the augmented reality picture is a picture for interacting with a target virtual object in a real-time state in the second virtual space based on real-time pose information of the dynamic capture object in a dynamic capture space;

a virtual reality picture generation module 404, configured to generate a real-time virtual reality picture in response to interaction between the virtual object and a target virtual object in the first virtual space, where the virtual reality picture is a picture that interacts with the target virtual object in the first virtual space in a real-time state based on real-time pose information of the virtual object in the first virtual space;

The video stream generating module 405 is configured to generate a video stream according to the real-time virtual reality picture.

The embodiment of the application provides a second device, as shown in fig. 12, where the second device may include: a second state synchronization module 501 and an augmented reality picture generation module 502, wherein

A second state synchronization module 501, configured to synchronize real-time states of a target virtual object in a first virtual space and a second virtual space, respectively, so that a first device generates a video stream, where the first virtual space is a virtual space with a virtual scenery, and the second virtual space is a virtual space obtained by mapping a dynamic capture space by the second device;

the augmented reality image generating module 502 is configured to obtain real-time pose information of a dynamic capture object in a dynamic capture space, and generate and display an augmented reality image in response to interaction between the dynamic capture object and a target virtual object in the second virtual space, where the augmented reality image is an image that interacts with the target virtual object in a real-time state in the second virtual space based on the real-time pose information of the dynamic capture object in the dynamic capture space;

The first device and the second device in the embodiments of the present application may correspondingly execute the video generating method on the first device side and the video generating method on the second device side provided in the embodiments of the present application, and the implementation principle is similar, and actions executed by each module in the apparatus in each embodiment of the present application correspond to steps in the method in each embodiment of the present application, and detailed functional descriptions of each module in the apparatus may be referred to the descriptions in the corresponding methods shown in the foregoing, which are not repeated herein.

An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement steps of a video generating method, and compared with the related art, the steps of the method may be implemented: the method and the device are applied to streaming media processing scenes, the dynamic capturing objects are different from the pictures seen by audiences, the dynamic capturing objects see the augmented reality pictures displayed by the second equipment, the virtual reality pictures seen by the audiences are generated by responding to virtual objects to interact with target virtual objects in the first virtual space, and specifically, the pictures of interaction with the target virtual objects in real-time state in the first virtual space are generated based on real-time pose information of the virtual objects in the first virtual space. Because the real-time pose information of the dynamic capture object in the dynamic capture space and the real-time pose information of the virtual object in the first virtual space are corresponding, and the real-time states of the target virtual object in the first virtual space and the second virtual space are synchronous, the method realizes that when the dynamic capture object interacts with the target virtual object in the second virtual space, a spectator can synchronously see the picture of the interaction of the virtual object with the target virtual object in the first virtual space.

In an alternative embodiment, there is provided an electronic device, as shown in fig. 13, the electronic device 4000 shown in fig. 13 includes: a processor 4001 and a memory 4003. Wherein the processor 4001 is coupled to the memory 4003, such as via a bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004, the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data, etc. It should be noted that, in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.

The processor 4001 may be a CPU (Central Processing Unit ), general purpose processor, DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor 4001 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

Bus 4002 may include a path to transfer information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture ) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 13, but not only one bus or one type of bus.

Memory 4003 may be, but is not limited to, ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, RAM (Random Access Memory ) or other type of dynamic storage device that can store information and instructions, EEPROM (Electrically Erasable Programmable Read Only Memory ), CD-ROM (Compact Disc Read Only Memory, compact disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer.

The memory 4003 is used for storing a computer program that executes an embodiment of the present application, and is controlled to be executed by the processor 4001. The processor 4001 is configured to execute a computer program stored in the memory 4003 to realize the steps shown in the foregoing method embodiment.

Embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, where the computer program, when executed by a processor, may implement the steps and corresponding content of the foregoing method embodiments.

The embodiments of the present application also provide a computer program product, which includes a computer program, where the computer program can implement the steps of the foregoing method embodiments and corresponding content when executed by a processor.

The terms "first," "second," "third," "fourth," "1," "2," and the like in the description and in the claims of this application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the present application described herein may be implemented in other sequences than those illustrated or otherwise described.

It should be understood that, although the flowcharts of the embodiments of the present application indicate the respective operation steps by arrows, the order of implementation of these steps is not limited to the order indicated by the arrows. In some implementations of embodiments of the present application, the implementation steps in the flowcharts may be performed in other orders as desired, unless explicitly stated herein. Furthermore, some or all of the steps in the flowcharts may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or phases may be performed at the same time, or each of these sub-steps or phases may be performed at different times, respectively. In the case of different execution time, the execution sequence of the sub-steps or stages may be flexibly configured according to the requirement, which is not limited in the embodiment of the present application.

The foregoing is merely an optional implementation manner of the implementation scenario of the application, and it should be noted that, for those skilled in the art, other similar implementation manners based on the technical ideas of the application are adopted without departing from the technical ideas of the application, and also belong to the protection scope of the embodiments of the application.

Claims

1. A method of video generation, for use with a first device, the method comprising:

2. The method of claim 1, wherein synchronizing real-time states of the target virtual object in the first virtual space and the second virtual space, respectively, comprises:

acquiring a real-time state of the target virtual object in the second virtual space, which is sent by the second device, and determining the real-time state of the target virtual object in the first virtual space according to the mapping relation between the first virtual space and the second virtual space; or alternatively

The real-time state of the target virtual object in the first virtual space is determined, and the real-time state of the target virtual object in the first virtual space is sent to second equipment, so that the second equipment determines the real-time state of the target virtual object in the second virtual space according to the mapping relation between the first virtual space and the second virtual space.

3. The method of claim 2, wherein generating a real-time virtual reality screen in response to the virtual object interacting with a target virtual object of the first virtual space comprises:

4. A method according to any of claims 1-3, characterized in that the status comprises an object identification of the target virtual object, a name of a function to be executed and a parameter to be entered into the function.

5. The method of claim 1, wherein the synchronizing the real-time status of the target virtual object in the first virtual space and the second virtual space, respectively, further comprises, before:

6. A video generation method, applied to a second device, the method comprising:

the video stream is generated according to a virtual reality picture, wherein the virtual reality picture is a picture which is based on real-time pose information of a virtual object in a first virtual space and is used for interacting with a target virtual object in a real-time state in the first virtual space;

7. The method of claim 6, wherein the synchronizing the real-time status of the target virtual object in the first virtual space and the second virtual space, respectively, further comprises:

mapping the dynamic capture space to obtain the second virtual space;

8. A first device, comprising:

9. A second device, comprising:

10. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to carry out the steps of the method according to any one of claims 1-7.

11. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the video generation method of any of claims 1-7.

12. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the video generation method of any of claims 1-7.