WO2025002075A1 - Video generation method and apparatus, electronic device, and storage medium - Google Patents

Abstract

Embodiments of the present disclosure provide a video generation method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining an initial camera movement sequence on the basis of an initial camera movement template, wherein the initial camera movement sequence comprises at least one camera movement node corresponding to an initial video, and the camera movement node is used for representing a camera movement feature of a shot picture during video playback; acquiring a video content feature of a video to be processed; obtaining an optimization camera movement sequence on the basis of the video content feature and the initial camera movement sequence, wherein the optimization camera movement sequence comprises at least one optimization camera movement node, and a camera movement feature corresponding to the optimization camera movement node matches the video content feature; and processing said video on the basis of the optimization camera movement sequence, and generating an output video, wherein a shot picture of the output video is provided with the camera movement feature corresponding to the optimization camera movement node. The output video is generated on the basis of the optimization camera movement sequence and said video, so as to solve the problem of the mismatch between a camera movement special effect generated by the initial camera movement template and the content of said video, thereby improving the visual impression of the camera movement special effect.

Description

Video generation method, device, electronic device and storage medium

This application claims priority to the Chinese invention patent application entitled “Video generation method, device, electronic device and storage medium” and application number 2023107702269, filed on June 27, 2023. The entire contents of this application are incorporated by reference into this application.

Technical Field

The embodiments of the present disclosure relate to the field of Internet technology, and in particular to a video generation method, device, electronic device, and storage medium.

Background Art

Currently, video platforms and video processing applications provide users with a variety of special effects templates to improve the efficiency and quality of users' homemade video works. Among them, the camera template is a special effect template that processes the video to add simulated camera effects to the video and improve the video's viewing experience.

However, in the actual application of the camera movement template, there is a problem that the camera movement effect provided by the camera movement template does not match the content of the video to be processed actually shot by the user, which affects the visual perception of the camera movement special effects.

Summary of the invention

In a first aspect, an embodiment of the present disclosure provides a video generation method, comprising: obtaining an initial camera movement sequence according to an initial camera movement template, wherein the initial camera movement sequence includes at least one camera movement node corresponding to the initial video, and the camera movement node is used to characterize the camera movement characteristics of the camera lens when the video is played; obtaining video content characteristics of the video to be processed, and obtaining an optimized camera movement sequence according to the video content characteristics and the initial camera movement sequence, wherein the optimized camera movement sequence includes at least one optimized camera movement node, and the camera movement characteristics corresponding to the optimized camera movement node match the video content characteristics; processing the video to be processed based on the optimized camera movement sequence to generate an output video, wherein the camera lens of the output video has the camera movement nodes corresponding to the optimized camera movement nodes. Camera movement characteristics.

In a second aspect, an embodiment of the present disclosure provides a video generating device, comprising: a loading module, used to obtain an initial camera movement sequence according to an initial camera movement template, wherein the initial camera movement sequence includes at least one camera movement node corresponding to the initial video, and the camera movement node is used to characterize the camera movement characteristics of the camera lens when the video is played; a processing module, used to obtain the video content characteristics of the video to be processed, and obtain an optimized camera movement sequence according to the video content characteristics and the initial camera movement sequence, wherein the optimized camera movement sequence includes at least one optimized camera movement node, and the camera movement characteristics corresponding to the optimized camera movement node match the video content characteristics; an output module, used to process the video to be processed based on the optimized camera movement sequence to generate an output video, and the camera lens of the output video has the camera movement characteristics corresponding to the optimized camera movement node.

In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the video generation method described in the first aspect and various possible designs of the first aspect.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video generation method described in the first aspect and various possible designs of the first aspect is implemented.

In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video generation method described in the first aspect and various possible designs of the first aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments or current solutions of the present disclosure, the drawings required for use in the description of the embodiments or current solutions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a diagram of an application scenario of a video generation method provided by an embodiment of the present disclosure;

FIG2 is a flow chart of a video generation method according to an embodiment of the present disclosure;

FIG3 is a flow chart of a specific implementation method of step S102 in the embodiment shown in FIG2 ;

FIG4 is a schematic diagram of a process for determining video content features provided by an embodiment of the present disclosure;

FIG5 is a flow chart of a possible implementation of step S103 in the embodiment shown in FIG2 ;

FIG6 is a schematic diagram of an optimized camera movement node generated based on a first content feature provided by an embodiment of the present disclosure;

FIG. 7 is a flow chart of another possible implementation of step S103 in the embodiment shown in FIG. 2 ;

FIG8 is a schematic diagram of an optimized camera movement node generated based on a second content feature provided by an embodiment of the present disclosure;

FIG9 is a flow chart of another possible implementation of step S103 in the embodiment shown in FIG2 ;

FIG10 is a flowchart of a specific implementation method of step S1037 in the embodiment shown in FIG9 ;

FIG11 is a second flow chart of a video generation method provided by an embodiment of the present disclosure;

FIG12 is a flowchart of a specific implementation method of step S205 in the embodiment shown in FIG11 ;

FIG13 is a flowchart of a specific implementation method of step S206 in the embodiment shown in FIG11 ;

FIG14 is a structural block diagram of a video generating device provided by an embodiment of the present disclosure;

FIG15 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

FIG. 16 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure.

DETAILED DESCRIPTION

The embodiments of the present disclosure provide a video generation method, device, electronic device and storage medium to overcome the problem that the camera movement effect provided by the camera movement template does not match the content of the video to be processed actually shot by the user.

The video generation method, device, electronic device and storage medium provided in this embodiment obtain an initial camera movement sequence according to an initial camera movement template, wherein the initial camera movement sequence includes at least one camera movement node corresponding to the initial video, and the camera movement node is used to represent the video playback time. The camera movement characteristics of the lens picture; obtain the video content characteristics of the video to be processed, and obtain the optimized camera movement sequence according to the video content characteristics and the initial camera movement sequence, wherein the optimized camera movement sequence includes at least one optimized camera movement node, and the camera movement characteristics corresponding to the optimized camera movement node match the video content characteristics; process the video to be processed based on the optimized camera movement sequence to generate an output video, wherein the lens picture of the output video has the camera movement characteristics corresponding to the optimized camera movement node. By using the video content characteristics of the video to be processed, the initial camera movement template is set to obtain an optimized camera movement sequence composed of optimized camera movement nodes, and an output video with camera movement effects is generated based on the optimized camera movement sequence. Since the camera movement characteristics corresponding to the optimized camera movement nodes in the optimized camera movement sequence match the video content characteristics of the video to be processed, the output video is generated based on the optimized camera movement sequence and the video to be processed, thereby avoiding the problem of the camera movement special effects generated by the initial camera movement template and the content of the video to be processed not matching, and improving the visual perception of the camera movement special effects.

In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

The application scenarios of the embodiments of the present disclosure are explained below:

FIG1 is a diagram of an application scenario of the video generation method provided by the embodiment of the present disclosure. The video generation method provided by the embodiment of the present disclosure can be applied to an application scenario of adding camera effects to a video. Specifically, as shown in FIG1 , the method provided by the embodiment of the present disclosure can be applied to electronic devices such as terminal devices and servers. Taking a terminal device as an example, a terminal device such as The video generation method provided in this embodiment is implemented by running a video processing application (Application, APP) on a smart phone. Specifically, as shown in FIG1 , after loading a pre-generated video to be processed 11, the terminal device triggers a camera template control 12 provided by the video processing application to add camera effects to the video to be processed, thereby generating an output video 13 with a camera effect on the lens screen. The camera effect includes, for example, movement and zooming of the lens screen.

In the related art, the camera movement effect provided by the camera movement template is achieved by manually or automatically cutting the image screen of the initial video to generate multiple moving lens pictures to simulate the real camera movement. Therefore, the camera movement effect achieved by the camera movement template can match the image screen of the initial video. However, for the video to be processed that needs to apply the camera movement template to obtain the same camera movement effect, the corresponding video content is different from the initial video, which leads to the problem that the camera movement effect provided by the camera movement template does not match the content of the video to be processed actually shot by the user. The camera movement special effects of the output video generated by applying the camera movement template will cause problems such as the inability to focus on the portrait in the center of the lens picture, the lens picture zooming too large or too small, etc., affecting the visual perception.

The embodiment of the present disclosure provides a video generation method to solve the above-mentioned problem.

Referring to FIG. 2 , FIG. 2 is a flow chart of a video generation method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a terminal device or a server, and the video generation method includes:

Step S101: obtaining an initial camera movement sequence according to an initial camera movement template. The initial camera movement sequence includes at least one camera movement node corresponding to an initial video, and the camera movement node is used to represent the camera movement characteristics of a lens picture during video playback.

First, let's briefly introduce the concept of camera movement. Camera movement refers to the process of shooting a video by pushing, pulling, shaking, moving, following, swinging, raising, lowering, etc., so that the camera lens of the shooting device is in motion to complete the video shooting. At the same time, it ensures the focus of the shooting subject (such as a dancing person) in the shooting picture, so that the movement of the shooting subject remains coherent, thereby achieving the effect of highlighting the shooting subject and improving the visual perception. It is a common video shooting method.

As for the camera movement effect, it refers to processing the video that has been shot. Based on the original video, the video screen of the original video is dynamically trimmed. Image processing steps such as scaling and rotation are performed to form a continuous, moving lens screen, thereby simulating the real camera movement effect. Among them, the camera movement template is the data used to realize the above-mentioned camera movement special effects. By processing the video to be processed through the camera movement template, the processed video to be processed (i.e., the output video) can have the corresponding camera movement special effects.

Further, the initial camera movement template in the present embodiment is generated based on the initial video. Wherein, the initial camera movement template includes an initial camera movement sequence, and one or some independent camera movement effects are expressed through the initial camera movement sequence. Specifically, the camera movement sequence is composed of at least one orderly camera movement node, and the camera movement node is used to characterize the camera movement characteristics of the lens picture when the video is played, such as the lens picture shaking left and right, shaking up and down, zooming, etc., so as to achieve an independent camera movement effect. The camera movement characteristics also include specific parameters of the above-mentioned lens picture movement, such as shaking amplitude, frequency, etc. Each camera movement node in the camera movement sequence corresponds to a timestamp, and the timestamp may include a start timestamp and an end timestamp, which respectively correspond to the start time and end time of the camera movement effect represented by the camera movement node. The initial camera movement sequence corresponds to one or more camera movement effects in the initial video, and the camera movement nodes in the initial camera movement sequence characterize the camera movement characteristics of the lens picture when the initial video is played. Further, by parsing the model data corresponding to the initial camera movement model, the initial camera movement sequence in the initial camera movement model can be obtained. The initial camera movement sequence in the initial camera movement model can be manually set by the user through key frames (K frames) based on the content of the initial video. It can also be automatically generated by the terminal device or server after content recognition based on the content of the initial video. The specific generation process of the initial camera movement model will not be introduced here.

The above steps can be performed based on a trigger operation within the application. For example, after the user clicks on the control of the camera template A (initial camera template), the terminal device loads the template data corresponding to the camera simulation A, and then obtains the initial camera sequence. The specific interactive operation process is not specifically limited here and will not be repeated.

Step S102: Obtain video content features of the video to be processed.

For example, the terminal device can load the video to be processed by responding to the user's instruction. The video to be processed can be a video shot by the terminal device, or a video shot or generated by other devices. More specifically, the content of the video to be processed can be various, for example Such as dance videos, landscape videos, cartoon videos, etc., no specific restrictions are made here. After the terminal device loads the video to be processed, the corresponding video content features can be obtained by performing content recognition on the video to be processed. Among them, the video content features are information used to characterize the video content of the video to be processed. In one possible implementation, the video content features can be implemented in the form of identifiers. For example, the video content feature is "0", indicating that no face appears in the video to be processed. The video content feature is "1", indicating that a face appears in the video to be processed. In another possible implementation, the video content features can be implemented in the form of more complex feature vectors and feature matrices to characterize more complex content. For example, the outline of a portrait in the image to be processed, the size of a specific object, etc.

In a possible implementation, as shown in FIG3 , a specific implementation of step S102 includes:

Step S1021: According to the timestamp of at least one camera movement node in the initial camera movement sequence, obtain the corresponding target video frame in the video to be processed.

Step S1023: Perform content detection on the target video frame to obtain video content features.

Exemplarily, the initial camera movement sequence includes at least one camera movement node. The camera movement node corresponds to at least one timestamp, and the timestamp is used to characterize the marking time of the camera movement node in the initial camera movement sequence in the initial video. More specifically, the marking time can be the camera movement start time, the camera movement end time, etc. Afterwards, based on the timestamp of at least one camera movement node, the corresponding video frame, that is, the target video frame, is read in the video frame to be processed. Content detection is performed on the target video frame to obtain the video frame content feature corresponding to the target video frame. Afterwards, the video frame content feature corresponding to one of the target video frames can be used as the video content feature. It is also possible to use a collection of video frame content features corresponding to multiple target video frames as the video content feature.

FIG4 is a schematic diagram of a process for determining video content features provided by an embodiment of the present disclosure. The above process will be further described with reference to FIG4. As shown in FIG4, exemplarily, the initial camera movement sequence includes camera movement node #1, camera movement node #2, and camera movement node #3. Among them, the timestamp of camera movement node #1 is [00:02, 00:03], which indicates that the start time of the camera movement effect corresponding to camera movement node #1 is 00:02 (0 minutes and 2 seconds, the same below), and the end time is 00:03. The timestamp of camera movement node #2 is [00:04, 00:05], which indicates that the start time of the camera movement effect corresponding to camera movement node #2 is 00:02 (0 minutes and 2 seconds, the same below), and the end time is 00:03. The start time is 00:04 and the end time is 00:05. The timestamp of camera node #3 is [00:06, 00:08], which indicates that the start time of the camera effect corresponding to camera node #3 is 00:06 and the end time is 00:08. Afterwards, according to the start times corresponding to camera nodes #1, #2 and #3, i.e. 00:02, 00:04 and 00:06, the target video frames at the corresponding playback timestamps are obtained from the video to be processed, which are video frame P1, video frame P2 and video frame P3 respectively. Afterwards, content detection is performed on video frames P1, video frames P2 and video frames P3 respectively to obtain corresponding video frame content features F_1, video frame content features F_2 and video frame content features F_3. Finally, the set of the above video frame content features F_1, video frame content features F_2 and video frame content features F_3 (shown as [F_1, F_2, F_3] in the figure) is used as the video content features.

Step S103: obtaining an optimized camera movement sequence according to the video content features and the initial camera movement sequence, wherein the optimized camera movement sequence includes at least one optimized camera movement node, and the camera movement features corresponding to the optimized camera movement node match the video content features.

Exemplarily, after obtaining the video content feature and the initial camera movement sequence, at least one initial camera movement node in the initial camera movement sequence is optimized according to the video content feature to generate a corresponding optimized camera movement node, thereby obtaining an optimized camera movement sequence. Wherein, similar to the initial camera movement node in the initial camera movement sequence, the optimized camera movement node is also used to characterize the camera movement feature of the lens picture during video playback. However, since the optimized camera movement node is generated after being set based on the initial camera movement node in the initial camera movement sequence and based on the video content feature, the generated optimized camera movement node can match the video content feature. Wherein, the optimized camera movement node can match the video content feature. For example, it includes: the motion amplitude corresponding to the optimized camera movement node (an exemplary implementation of the camera movement feature) is matched with the scale of the scene in the video to be processed (an exemplary implementation of the video content feature). Another example includes: the center point of the lens picture corresponding to the optimized camera movement node (an exemplary implementation of the camera movement effect) is matched with the portrait position in the video to be processed (an exemplary implementation of the video content feature). Therefore, when the video to be processed is inconsistent with the initial video in content, the adaptive optimization of the camera movement effect is achieved by optimizing the camera movement node.

In a possible implementation, the video content feature includes a first content feature, and the first content feature represents the position of the portrait in the video to be processed. As shown in FIG. 5 , step S103 A possible implementation includes:

Step S1031: determining a lens focus area of a lens picture corresponding to at least one lens movement node in an initial lens movement sequence during the movement process according to the first content feature.

Step S1032: configuring corresponding camera movement nodes according to the lens focus area during the movement of the lens image, and generating optimized camera movement nodes.

Step S1033: Generate an optimized camera movement sequence according to the optimized camera movement nodes.

Exemplarily, the camera movement node has a first parameter, and the first parameter is used to determine the center point of the lens picture corresponding to the camera movement node during the movement process, that is, the lens focus area. When the video content feature is a first content feature that characterizes the position of the portrait in the video to be processed, the terminal device can determine the position of the portrait in the video to be processed through the first content feature. Afterwards, the first parameter of the camera movement node is set based on the position of the portrait, and the optimized camera movement node is generated, so that in the camera movement special effect generated by the camera movement node, the center point of the lens picture during the movement process is the position of the portrait, thereby realizing the focus of the portrait in the video to be processed. Compared with the solution of directly using the initial camera movement sequence (initial camera movement template) to process the video to be processed, it can avoid the problem that the center of the lens picture is inconsistent with the position of the character in the video to be processed when adding camera movement special effects such as "zoom camera movement" and "shaking camera movement", and improve the visual effect of the camera movement special effect.

Referring to the introduction of the video content features in the previous embodiment, the video content features may be a set composed of a plurality of video frame content features, and the video frame content features correspond to a video frame at a certain playback position in the video to be processed (for example, the starting position of the camera movement special effect). Therefore, the first content feature may include the coordinates of the position of the portrait in the video at a plurality of playback positions in the video to be processed. More specifically, for example, the coordinates of the position of the center point of the character, or the coordinates of the center of the character's face. Afterwards, based on the position represented by the first content feature, the first parameter of at least one camera movement node in the initial camera movement sequence is set. Exemplarily, when the first content feature includes a plurality of video frame content features, each video frame content feature represents a portrait position, and the camera movement node corresponding to the video frame content feature is set according to each video frame content feature (the video frame content feature is determined based on the timestamp of the camera movement node, so the two have a corresponding relationship). Thereby, when the character in the video to be processed is in a moving state (the position of the portrait changes), the center point of the lens picture is actually focused on the character position, further highlighting the Create portraits and improve the visual effect of camera movement.

FIG6 is a schematic diagram of an optimized camera movement node generated based on a first content feature provided by an embodiment of the present disclosure. As shown in FIG6 , the camera movement effect achieved by the optimized camera movement node #1 generated based on the first content feature representing the position of the portrait in the video to be processed is "zooming camera movement". In the process of achieving this camera movement effect, during the zooming process, the center of the lens picture is aligned with the reference point P1 where the portrait is located in the video to be processed, thereby achieving the effect of focusing on the portrait.

In another possible implementation, the video content feature includes a second content feature, and the second content feature represents the scale of the scene in the to-be-processed video. As shown in FIG. 7 , another possible implementation of step S103 includes:

Step S1034: determining the movement amplitude of the shot image corresponding to at least one camera movement node in the initial camera movement sequence during the movement process according to the second content feature.

Step S1035: configuring corresponding camera movement nodes according to the movement amplitude of the lens image during the movement process, and generating optimized camera movement nodes.

Step S1036: Generate an optimized camera movement sequence according to the optimized camera movement nodes.

Exemplarily, the camera movement node has a second parameter, and the second parameter is used to determine the movement amplitude of the lens picture corresponding to the camera movement node during the movement process. When the video content feature is a second content feature that characterizes the scale of the scene in the video to be processed, the terminal device can determine the scale of the scene in the video to be processed through the second content feature. More specifically, that is, close view, medium view, long view, etc. Of course, in other implementations, the scale of the scene represented by the second content feature can be a continuously changing value, which is used to represent a more refined scene scale, and examples are not given one by one here. Afterwards, the second parameter of the camera movement node is set based on the scale of the scene, and the optimized camera movement node is generated, so that the shaking amplitude of the lens picture in the camera movement special effect generated by the camera movement node during the shaking process matches the scene. Specifically, the farther the scene is, the larger the relative movement amplitude is; and the farther the scene is, the smaller the relative movement amplitude is, so as to avoid the dizziness caused by the inappropriate camera movement shaking amplitude when the camera movement special effects such as "shaking camera movement" are used, and the viewing effect of the camera movement special effects is improved.

FIG8 is a schematic diagram of an optimized camera movement node generated based on a second content feature provided by an embodiment of the present disclosure. The optimized camera movement node #2 generated by the second content feature realizes the camera movement effect of "left and right shaking camera movement". In the process of realizing this camera movement effect, when the scale of the scene in the video to be processed is 03 (representing the distant view), the shaking amplitude is a during the movement of the lens image P1 to the lens image P2. When the scale of the scene in the video to be processed is 01 (representing the close view), the shaking amplitude is b during the movement of the lens image P3 to the lens image P4. Among them, illustratively, b = 0.2a. The second parameter of the camera movement node is set by the first content feature to achieve the effect of adaptive amplitude adjustment of the left and right shaking camera movement.

In another possible implementation, the video content feature includes a third content feature, and the third content feature represents a target object in the video to be processed. As shown in FIG. 9 , another possible implementation of step S103 includes:

Step S1037: determining the change frequency of the lens picture corresponding to at least one camera movement node in the initial camera movement sequence during the camera movement process according to the third content feature, and the change frequency is less than a preset frequency threshold.

Step S1038: configuring corresponding camera movement nodes according to the frequency of changes of the lens image during the camera movement process, and generating optimized camera movement nodes.

Step S1039: Generate an optimized camera movement sequence according to the optimized camera movement nodes.

Exemplarily, the camera movement node has a third parameter, and the third parameter is used to determine the frequency of change of the lens picture corresponding to the camera movement node during the camera movement process. When the video content feature is a third content feature that characterizes the target object in the video to be processed, the terminal device can determine whether the target object exists in the video to be processed through the third content feature. For example, whether there are important parts such as face and hands, and the position and size of the above parts. Afterwards, the second parameter of the camera movement node is set based on the target object to generate an optimized camera movement node, thereby realizing the control of the shaking frequency of the lens picture in the camera movement special effect generated by the camera movement node. Specifically, for example, for camera movement special effects such as "shaking camera movement", in a possible implementation method, when it is determined based on the third content feature that there are portrait faces and fingers (target objects) in the video to be processed, the high-frequency shaking is reduced. For example, the shaking frequency (changing frequency) is reduced to below a preset frequency value by the third parameter, thereby improving the focusing effect of the target object, avoiding the problem that the user cannot see the target object such as the facial expression of the portrait and the finger movement due to too frequent shaking.

In another possible implementation, as shown in FIG10 , a specific implementation of step S1037 includes:

Step S1037A: Obtain the outline size of the target object.

Step S1037B: Obtain the corresponding target motion frequency according to the outline size, wherein the target motion frequency is inversely proportional to the outline size.

Step S1037C: according to the target motion frequency, a third parameter is set, where the third parameter is used to determine the frequency of change of the lens image corresponding to the camera movement node during the camera movement process.

Exemplarily, after obtaining the outline size of the target object (e.g., the area of the facial outline) based on the third content feature, the corresponding shaking frequency (target motion frequency) is set based on the outline size of the target object. Specifically, for example, when the outline size of the facial outline is larger, the shaking frequency is lower. Conversely, when the outline size is smaller, the shaking frequency is higher (but not greater than the initial value of the third parameter). This achieves shaking frequency control in the camera movement special effect, reduces the visual impact of high-frequency shaking on target objects such as faces and fingers in the video, and improves the visual effect of camera movement.

It should be noted that the specific implementation of the above video content feature can be one of the above first content feature, the second content feature, and the third content feature, or multiple of the above first content feature, the second content feature, and the third content feature. That is, the camera movement node can be configured by at least two of the first parameter, the second parameter, and the third parameter to generate an optimized camera movement node and a corresponding optimized camera movement sequence. The specific implementation method can be set as needed and will not be repeated here.

Step S104: Process the video to be processed based on the optimized camera movement sequence to generate an output video, wherein the shot images of the output video have camera movement features corresponding to the optimized camera movement nodes.

Exemplarily, after obtaining the optimized camera movement sequence, camera movement special effects are generated at corresponding positions of the video to be processed based on each optimized camera movement node in the optimized camera movement sequence, thereby generating a special effects video with one or more camera movement special effects, i.e., an output video. Since the camera movement special effects in the output video are generated based on the optimized optimized camera movement sequence, the lens images of the output video have camera movement features corresponding to the optimized camera movement nodes, so that the camera movement special effects in the output video match the video content.

In this embodiment, the initial camera movement sequence is obtained according to the initial camera movement template. The initial camera movement sequence includes at least one camera movement node corresponding to the initial video, and the camera movement node is used to characterize the camera movement characteristics of the lens picture when the video is played. The video content characteristics of the video to be processed are obtained, and the optimized camera movement sequence is obtained according to the video content characteristics and the initial camera movement sequence. The optimized camera movement sequence includes at least one optimized camera movement node, and the camera movement characteristics corresponding to the optimized camera movement node match the video content characteristics. The video to be processed is processed based on the optimized camera movement sequence to generate an output video, and the lens picture of the output video has the camera movement characteristics corresponding to the optimized camera movement node. By using the video content characteristics of the video to be processed, the initial camera movement template is set to obtain an optimized camera movement sequence composed of optimized camera movement nodes. And based on the optimized camera movement sequence, an output video with a camera movement effect is generated, because the camera movement characteristics corresponding to the optimized camera movement nodes in the optimized camera movement sequence match the video content characteristics of the video to be processed. Therefore, the output video is generated based on the optimized camera movement sequence and the video to be processed, avoiding the problem that the camera movement special effects generated by the initial camera movement template do not match the content of the video to be processed, and improving the visual perception of the camera movement special effects.

Referring to FIG11 , FIG11 is a second flow chart of the video generation method provided by the embodiment of the present disclosure. Based on the embodiment shown in FIG2 , this embodiment further refines step S103 , and the video generation method includes:

Step S201: obtaining an initial camera movement sequence according to an initial camera movement template, wherein the initial camera movement sequence includes at least one camera movement node corresponding to an initial video, and the camera movement node is used to represent camera movement features of a camera shot during video playback.

Step S202: Obtain a first video content feature of the video to be processed.

Step S203: Acquire the second video content feature of the initial video.

Step S204: obtaining feature deviation information according to the second video content feature and the first video content feature, wherein the feature deviation information represents the feature deviation amount in video content between the video to be processed and the initial video.

Exemplarily, the initial video is the video used to generate the initial camera movement sequence, that is, the video content matches the camera movement special effects generated by the initial camera movement sequence. In one possible implementation, the template data corresponding to the initial camera movement template includes the initial video corresponding to the initial camera movement template. Afterwards, the initial video is subjected to content detection to obtain the video content features corresponding to the initial video, that is, the second video content features. Among them, the specific implementation method of obtaining the second video content features is similar to that of obtaining the video content features of the video to be processed (the second video content features). The implementation method of obtaining the video content feature of the video to be processed is the same as that of the video content feature of the video to be processed. Please refer to the introduction of the specific implementation method of obtaining the video content feature of the video to be processed in the embodiment shown in Figure 2, which will not be repeated here.

Furthermore, after obtaining the first video content feature and the second video content feature, feature comparison can be performed based on the first video content feature and the second video content feature to obtain feature deviation information. The feature deviation amount represents the feature deviation amount between the initial video and the video to be processed in terms of video content. Specifically, for example, the first video content feature represents the first position where the portrait is located in the video to be processed, and the second video content feature represents the second position where the portrait is located in the initial video. The feature deviation information representing the deviation amount between the first position and the second position is obtained through the first video content feature and the second video content feature.

Step S205: setting node parameters of at least one camera movement node in the initial camera movement sequence according to the feature deviation information to obtain an optimized camera movement sequence.

Exemplarily, after obtaining the characteristic deviation information, the node parameters of the camera movement node in the initial camera movement sequence are corrected based on the characteristic deviation amount represented by the characteristic deviation information, so that the camera movement node can match the video content feature (first video content feature) of the video to be processed, so as to achieve the purpose of camera movement node optimization, and then obtain an optimized camera movement sequence.

In a possible implementation, the feature deviation information includes a position deviation, and the position deviation represents a deviation between a first position where the portrait is located in the initial video and a second position where the portrait is located in the video to be processed. As shown in FIG. 12 , a specific implementation of step S205 includes:

Step S2051: Obtain the target camera movement node corresponding to the position deviation in the initial camera movement sequence.

Step S2052: according to the position deviation, the center point of the lens picture corresponding to the target camera movement node during the camera movement process is set to obtain an optimized camera movement sequence.

Exemplarily, since the feature deviation information is obtained based on the feature comparison between the first video content feature and the second video content feature, and the first video content feature and the second video content feature are obtained based on the video frame corresponding to the same camera movement node. Therefore, the feature deviation information corresponds to a camera movement node in the initial camera movement sequence. In this embodiment, the camera movement node corresponding to the feature deviation information (position deviation amount) is the target camera movement node.

Furthermore, when the characteristic deviation information is a position deviation amount representing the deviation between a first position where a person portrait is located in the initial video and a second position where a person portrait is located in the video to be processed, by The characteristic deviation information can realize the adjustment of the center point of the picture of the lens picture corresponding to the camera movement node during the camera movement process. Specifically, for example, the position deviation amount is a deviation vector, through which the center point of the picture of the lens picture corresponding to the target camera movement node during the camera movement process is set. Thereby, the center point of the picture generates an offset corresponding to the deviation vector, and the optimization of the target camera movement node is realized. Further, with the optimization of at least one target camera movement node, the optimization of the initial camera movement sequence is realized at the same time, thereby generating an optimized camera movement sequence corresponding to the initial camera movement sequence. The camera movement effect generated by the optimized optimized camera movement sequence can be focused on the position where the portrait is located in the video to be processed, thereby improving the visual perception experience. Among them, the step of setting the center point of the picture of the lens picture corresponding to the target camera movement node during the camera movement process in this embodiment can be realized by setting the first parameter of the target camera movement node. The specific implementation method can refer to the relevant introduction in the embodiment shown in Figure 2, which will not be repeated here.

In another possible implementation, the feature deviation information includes a scale coefficient, and the scale coefficient represents the ratio of the scene scale in the initial video to the scene scale in the video to be processed. The specific implementation of step S205 includes:

Step S2053: Obtain the target camera movement node corresponding to the scale coefficient in the initial camera movement sequence.

Step S2054: according to the scale coefficient, the movement amplitude of the lens picture corresponding to the target camera movement node during the movement process is set to obtain an optimized camera movement sequence.

Exemplarily, it is similar to the above-mentioned process of obtaining an optimized camera movement sequence based on the position deviation. When the characteristic deviation information is a scale coefficient that characterizes the ratio of the scene scale in the initial video to the scene scale in the video to be processed, the target camera movement node corresponding to the scale coefficient is first obtained. The implementation method is similar to the step of obtaining the target camera movement node corresponding to the position deviation in the above-mentioned embodiment, and will not be repeated. Afterwards, according to the scale coefficient, the movement amplitude of the lens picture corresponding to the target camera movement node during the movement process is set to achieve the setting of the shaking amplitude of the lens picture in the camera movement special effects such as "shaking camera movement". Thereby, the optimization of the target camera movement node is achieved, and the optimized camera movement sequence corresponding to the initial camera movement sequence is generated. The movement amplitude of the camera movement effect generated by the optimized optimized camera movement sequence matches the scene scale. Thereby, the visual fatigue and dizziness caused by large-scale movement in the close view and the problem of unclear camera movement special effects caused by too small movement in the distant view are avoided, and the visual perception experience is improved. Among them, in this embodiment, the step of setting the movement amplitude of the lens picture corresponding to the target camera movement node during the movement process is as follows: The step can be realized by setting the second parameter of the target camera movement node. The specific implementation method can be found in the relevant introduction of the embodiment shown in FIG2, which will not be repeated here.

Optionally, in a possible implementation, the initial video has a first play duration, and the video to be processed has a second play duration. When the second play duration is greater than the first play duration, after step S205, the method further includes:

Step S206: Generate an extended camera movement sequence according to at least one camera movement node in the optimized camera movement sequence. The camera movement duration of the lens picture corresponding to the extended camera movement sequence is not less than the difference between the second playback duration and the first playback duration.

Step S207: Process the video to be processed based on the optimized camera movement sequence and the extended camera movement sequence in sequence to generate an output video.

Exemplarily, when the playback duration of the initial video is longer than the playback duration of the video to be processed, the playback duration of the initial camera movement template generated based on the initial video will be shorter than the video to be processed, thereby causing the camera movement special effects generated by the initial camera movement sequence to be unable to cover the entire video to be processed. In response to this problem, this embodiment generates a delayed camera movement sequence by obtaining at least one camera movement node from the generated optimized camera movement sequence. After that, the video to be processed is processed based on the optimized camera movement sequence and the extended camera movement sequence in turn, so that the camera movement special effects generated by the optimized camera movement sequence and the extended camera movement sequence can cover the entire video to be processed, avoiding the existence of blank special effects segments without camera movement special effects in the generated output video. At the same time, since the extended camera movement sequence is generated based on the camera movement nodes in the optimized camera movement sequence, it can be ensured that the extended camera movement sequence matches the remaining video clips in the video to be processed that are not covered by the optimized camera movement sequence, thereby improving the camera movement special effects of the output video.

Exemplarily, as shown in FIG13 , the specific implementation of step S206 includes:

Step S2061: Obtain identification features of the first video segment in the video to be processed, wherein the first video segment is a video segment after a first playback duration in the video to be processed, and the identification features include video content features and/or music melody features of the first video segment.

Step S2062: According to the identification feature, a second video segment is obtained from the video to be processed, where the second video segment is a video segment before the first playback duration in the video to be processed. The identification feature of the second video segment is similar to the identification feature of the first video segment.

Step S2063: According to at least one continuous camera movement node corresponding to the second video clip, Generates an extended camera sequence.

Exemplarily, when the playback duration of the video to be processed is longer than that of the initial video, the video to be processed can be divided into two parts, namely, the part after the first playback duration and the part before the first playback duration. Among them, for the part after the first playback duration, it can be divided into one or more sub-segments, namely, the first video segment, based on the duration of the camera movement node. Afterwards, feature extraction is performed on the first video segment to obtain the identification feature corresponding to the first video segment, wherein the identification feature, for example, includes the video content feature and/or the music melody feature of the first video segment. Afterwards, based on the identification feature, the part before the first playback duration is searched to obtain a second video segment similar to the identification feature of the first video segment; wherein the judgment method of similar identification features includes: the feature distance between the identification feature of the second video segment and the identification feature of the first video segment is less than the feature threshold. Afterwards, according to the timestamp of the second video segment, at least one corresponding continuous multiplexed camera movement node is obtained. Afterwards, the above process is repeated until the multiplexed camera movement nodes corresponding to each first video segment are obtained, and the above multiplexed camera movement nodes are spliced to generate an extended camera movement sequence. Since the identification features of the second video clip are similar to the video features of the first video clip, the multiplexed camera movement node selected for the first video clip can also match the video content features of the first video clip to a certain extent. Therefore, the generated extended camera movement sequence can inherit the effect of the optimized camera movement sequence, achieve the adaptation of the video segments in the processed video that are not covered by the optimized camera movement sequence, and improve the visual effect of the overall camera movement special effect of the generated output video.

In this embodiment, the implementation of step S201-step S202 is the same as the implementation of step S101-step S102 in the embodiment shown in FIG. 2 of the present disclosure, and will not be described in detail here.

Corresponding to the video generation method of the above embodiment, FIG14 is a structural block diagram of a video generation device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Referring to FIG14 , the video generation device 3 includes:

A loading module 31 is used to obtain an initial camera movement sequence according to an initial camera movement template, wherein the initial camera movement sequence includes at least one camera movement node corresponding to an initial video, and the camera movement node is used to represent the camera movement characteristics of a lens picture during video playback;

The processing module 32 is used to obtain the video content features of the video to be processed, and obtain an optimized camera sequence according to the video content features and the initial camera sequence. The optimized camera sequence includes One less optimized camera movement node, the camera movement features corresponding to the optimized camera movement node match the video content features;

The output module 33 is used to process the video to be processed based on the optimized camera movement sequence to generate an output video, and the lens images of the output video have the camera movement features corresponding to the optimized camera movement nodes.

In one embodiment of the present disclosure, the video content feature includes a first content feature, and the first content feature characterizes the position of the portrait in the video to be processed; when the processing module 32 obtains the optimized camera movement sequence according to the video content feature and the initial camera movement sequence, it is specifically used to: generate a first parameter of at least one camera movement node in the initial camera movement sequence according to the first content feature, wherein the first parameter is used to determine the center point of the lens picture corresponding to the camera movement node during the movement process; configure the corresponding camera movement node according to the first parameter to generate an optimized camera movement node; and generate an optimized camera movement sequence according to the optimized camera movement node.

In one embodiment of the present disclosure, the video content feature includes a second content feature, and the second content feature characterizes the scale of the scene in the video to be processed; when the processing module 32 obtains the optimized camera movement sequence according to the video content feature and the initial camera movement sequence, it is specifically used to: generate a second parameter of at least one camera movement node in the initial camera movement sequence according to the second content feature, wherein the second parameter is used to determine the movement amplitude of the lens picture corresponding to the camera movement node during the movement process; configure the corresponding camera movement node according to the second parameter to generate an optimized camera movement node; and generate an optimized camera movement sequence according to the optimized camera movement node.

In one embodiment of the present disclosure, the video content feature includes a third content feature, and the third content feature characterizes the facial contour in the video to be processed; when the processing module 32 obtains the optimized camera movement sequence according to the video content feature and the initial camera movement sequence, it is specifically used to: generate a third parameter of at least one camera movement node in the initial camera movement sequence according to the third content feature, wherein the third parameter is used to determine the frequency of change of the lens picture corresponding to the camera movement node during the camera movement process; configure the corresponding camera movement node according to the third parameter to generate an optimized camera movement node; and generate an optimized camera movement sequence according to the optimized camera movement node.

In one embodiment of the present disclosure, when the processing module 32 generates the third parameter of at least one camera movement node in the initial camera movement sequence according to the third content feature, it is specifically used to: obtain the contour size of the facial contour; obtain the corresponding target motion frequency according to the contour size, wherein the target motion frequency is inversely proportional to the contour size; and set the third parameter.

In one embodiment of the present disclosure, when the processing module 32 obtains the optimized camera movement sequence based on the video content features and the initial camera movement sequence, it is specifically used to: obtain the video content features of the initial video; obtain feature deviation information based on the video content features of the initial video and the video content features of the video to be processed, the feature deviation information representing the feature deviation amount in video content between the video to be processed and the initial video; and set the node parameters of at least one camera movement node in the initial camera movement sequence based on the feature deviation information to obtain the optimized camera movement sequence.

In one embodiment of the present disclosure, the characteristic deviation information includes a position deviation, and the position deviation represents a deviation between a first position of a portrait in an initial video and a second position of the portrait in a video to be processed; when the processing module 32 sets the node parameters of at least one camera movement node in the initial camera movement sequence according to the characteristic deviation information to obtain an optimized camera movement sequence, it is specifically used to: obtain a target camera movement node corresponding to the position deviation in the initial camera movement sequence; and set the center point of the lens picture corresponding to the target camera movement node during the camera movement process according to the position deviation to obtain an optimized camera movement sequence.

In one embodiment of the present disclosure, the characteristic deviation information includes a scale coefficient, which represents the ratio of the scene scale in the initial video to the scene scale in the video to be processed; when the processing module 32 sets the node parameters of at least one camera movement node in the initial camera movement sequence according to the characteristic deviation information to obtain the optimized camera movement sequence, it is specifically used to: obtain the target camera movement node corresponding to the scale coefficient in the initial camera movement sequence; according to the scale coefficient, set the movement amplitude of the lens picture corresponding to the target camera movement node during the movement process to obtain the optimized camera movement sequence.

In one embodiment of the present disclosure, when the processing module 32 obtains the video content features of the video to be processed, it is specifically used to: obtain the corresponding target video frame in the video to be processed according to the timestamp of at least one camera movement node in the initial camera movement sequence; perform content detection on the target video frame to obtain the video content features.

In one embodiment of the present disclosure, the initial video has a first playback duration, and the video to be processed has a second playback duration. When the second playback duration is greater than the first playback duration, the processing module 32 is further configured to: generate an extended camera movement sequence according to at least one camera movement node in the optimized camera movement sequence, wherein the camera movement duration of the lens picture corresponding to the extended camera movement sequence is not less than the difference between the second playback duration and the first playback duration; and the processing module 32 processes the camera movement sequence based on the optimized camera movement sequence. When the video to be processed generates an output video, it is specifically used to: process the video to be processed based on the optimized camera movement sequence and the extended camera movement sequence in sequence to generate an output video.

In one embodiment of the present disclosure, when the processing module 32 generates an extended camera movement sequence based on at least one camera movement node in the optimized camera movement sequence, it is specifically used to: obtain identification features of a first video clip in the video to be processed, wherein the first video clip is a video segment after a first playback duration in the video to be processed, and the identification features include video content features and/or music melody features of the first video clip; according to the identification features, obtain a second video clip from the video to be processed, the second video clip is a video segment before the first playback duration in the video to be processed, and the identification features of the second video clip are similar to the identification features of the first video clip; and generate an extended camera movement sequence based on at least one continuous camera movement node corresponding to the second video clip.

The loading module 31, processing module 32 and output module 33 are connected in sequence. The video generating device 3 provided in this embodiment can implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

FIG. 15 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 15 , the electronic device 4 includes:

A processor 41, and a memory 42 communicatively connected to the processor 41;

The memory 42 stores computer executable instructions;

The processor 41 executes the computer-executable instructions stored in the memory 42 to implement the video generation method in the embodiments shown in FIG. 2 to FIG. 13 .

Optionally, the processor 41 and the memory 42 are connected via a bus 43 .

The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the embodiments corresponding to Figures 2 to 13, and no further details will be given here.

An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the video generation method provided in any one of the embodiments corresponding to Figures 2 to 13 of the present disclosure.

An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the methods in the embodiments shown in FIG. 2 to FIG. 13 .

Referring to FIG. 16 , an electronic device 900 suitable for implementing the embodiments of the present disclosure is shown. The electronic device 900 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (Portable Media Players, PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG16 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 16 , the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.

Typically, the following devices may be connected to the I/O interface 905: input devices 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 16 shows an electronic device 900 having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is processed When the processing device 901 executes, the above functions defined in the method of the embodiment of the present disclosure are performed.

It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

Computer program code for performing operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be written entirely in The program may be executed on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store information for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

Although the subject matter has been described in language specific to structural features and/or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. In a first aspect, the present disclosure provides a video generation method, comprising:

According to the initial camera movement template, an initial camera movement sequence is obtained, wherein the initial camera movement sequence includes at least one camera movement node corresponding to the initial video, and the camera movement node is used to characterize the camera movement characteristics of the lens screen when the video is played; the video content characteristics of the video to be processed are obtained, and the initial camera movement sequence includes at least one camera movement node corresponding to the initial video, and the camera movement node is used to characterize the camera movement characteristics of the lens screen when the video is played; The video content features and the initial camera movement sequence are combined to obtain an optimized camera movement sequence, wherein the optimized camera movement sequence includes at least one optimized camera movement node, and the camera movement features corresponding to the optimized camera movement node match the video content features; based on the optimized camera movement sequence, the video to be processed is processed to generate an output video, and the shot images of the output video have the camera movement features corresponding to the optimized camera movement node.

In one or more possible implementations, the video content feature includes a first content feature, and the first content feature represents the position of the portrait in the video to be processed; the optimized camera movement sequence is obtained according to the video content feature and the initial camera movement sequence, including: determining the lens focus area of the lens picture corresponding to at least one camera movement node in the initial camera movement sequence during the movement process according to the first content feature; configuring the corresponding camera movement node according to the lens focus area of the lens picture during the movement process, and generating an optimized camera movement node; generating the optimized camera movement sequence according to the optimized camera movement node.

In one or more possible implementations, the video content feature includes a second content feature, and the second content feature characterizes the scale of the scene in the video to be processed; obtaining the optimized camera movement sequence based on the video content feature and the initial camera movement sequence includes: determining, based on the second content feature, the motion amplitude of the lens picture corresponding to at least one camera movement node in the initial camera movement sequence during the motion process; configuring the corresponding camera movement node according to the motion amplitude of the lens picture during the motion process, and generating an optimized camera movement node; generating the optimized camera movement sequence based on the optimized camera movement node.

In one or more possible implementations, the video content feature includes a third content feature, and the third content feature represents the target object in the video to be processed; the optimized camera movement sequence is obtained based on the video content feature and the initial camera movement sequence, including: determining the change frequency of the lens picture corresponding to at least one camera movement node in the initial camera movement sequence during the camera movement process based on the third content feature, and the change frequency is less than a preset frequency threshold; configuring the corresponding camera movement node according to the change frequency of the lens picture during the camera movement process, and generating an optimized camera movement node; generating the optimized camera movement sequence based on the optimized camera movement node.

In one or more possible implementations, generating a third parameter of at least one camera movement node in the initial camera movement sequence according to the third content feature includes: obtaining The outline size of the target object; according to the outline size, obtaining a corresponding target motion frequency, wherein the target motion frequency is inversely proportional to the outline size; according to the target motion frequency, setting the third parameter.

In one or more possible implementations, obtaining an optimized camera movement sequence based on the video content features and the initial camera movement sequence includes: acquiring video content features of the initial video; obtaining feature deviation information based on the video content features of the initial video and the video content features of the video to be processed, wherein the feature deviation information represents the amount of feature deviation in video content between the video to be processed and the initial video; and setting node parameters of at least one camera movement node in the initial camera movement sequence based on the feature deviation information to obtain the optimized camera movement sequence.

In one or more possible implementations, the characteristic deviation information includes a position deviation, and the position deviation represents a deviation between a first position of a portrait in the initial video and a second position of the portrait in the video to be processed; according to the characteristic deviation information, node parameters of at least one camera movement node in the initial camera movement sequence are set to obtain the optimized camera movement sequence, including: obtaining a target camera movement node corresponding to the position deviation in the initial camera movement sequence; according to the position deviation, setting the center point of the lens picture corresponding to the target camera movement node during the camera movement process to obtain the optimized camera movement sequence.

In one or more possible implementations, the feature deviation information includes a scale coefficient, and the scale coefficient represents the ratio of the scene scale in the initial video to the scene scale in the video to be processed; according to the feature deviation information, the node parameters of at least one camera movement node in the initial camera movement sequence are set to obtain the optimized camera movement sequence, including: obtaining the target camera movement node corresponding to the scale coefficient in the initial camera movement sequence; according to the scale coefficient, setting the movement amplitude of the lens picture corresponding to the target camera movement node during the movement process to obtain the optimized camera movement sequence.

In one or more possible implementations, the obtaining of video content features of the video to be processed includes: obtaining a corresponding target video frame in the video to be processed based on a timestamp of at least one of the camera movement nodes in the initial camera movement sequence; and performing content detection on the target video frame to obtain the video content features.

In one or more possible implementations, the initial video has a first playback time. The video to be processed has a second playback duration. When the second playback duration is greater than the first playback duration, the method further comprises: generating an extended camera movement sequence according to at least one camera movement node in the optimized camera movement sequence, and the camera movement duration of the lens picture corresponding to the extended camera movement sequence is not less than the difference between the second playback duration and the first playback duration; processing the video to be processed based on the optimized camera movement sequence to generate an output video comprises: processing the video to be processed based on the optimized camera movement sequence and the extended camera movement sequence in sequence to generate an output video.

In one or more possible implementations, generating an extended camera movement sequence based on at least one camera movement node in the optimized camera movement sequence includes: obtaining identification features of a first video clip in the video to be processed, wherein the first video clip is a video segment after a first playback duration in the video to be processed, and the identification features include video content features and/or music melody features of the first video clip; obtaining a second video clip from the video to be processed based on the identification features, wherein the second video clip is a video segment before a first playback duration in the video to be processed, and the identification features of the second video clip are similar to those of the first video clip; generating the extended camera movement sequence based on at least one continuous camera movement node corresponding to the second video clip.

Claims (15)

A video generation method, comprising: According to the initial camera movement template, an initial camera movement sequence is obtained, wherein the initial camera movement sequence includes at least one camera movement node corresponding to the initial video, and the camera movement node is used to represent the camera movement characteristics of the lens picture when the video is played; Acquire video content features of the video to be processed, and obtain an optimized camera movement sequence according to the video content features and the initial camera movement sequence, wherein the optimized camera movement sequence includes at least one optimized camera movement node, and the camera movement features corresponding to the optimized camera movement node match the video content features; The video to be processed is processed based on the optimized camera movement sequence to generate an output video, wherein the shot images of the output video have camera movement features corresponding to the optimized camera movement nodes. The method according to claim 1, wherein the video content feature comprises a first content feature, the first content feature characterizing the position of the human figure in the video to be processed; The step of obtaining an optimized camera movement sequence according to the video content feature and the initial camera movement sequence includes: Determining, according to the first content feature, a lens focus area of a lens picture corresponding to at least one lens movement node in the initial lens movement sequence during the movement process; According to the lens focus area of the lens picture during the movement, a corresponding lens movement node is configured to generate an optimized lens movement node; The optimized camera movement sequence is generated according to the optimized camera movement node. The method according to claim 1, wherein the video content feature comprises a second content feature, and the second content feature represents the scale of the scene in the video to be processed; The step of obtaining an optimized camera movement sequence according to the video content feature and the initial camera movement sequence includes: Determining, according to the second content feature, a movement amplitude of a shot picture corresponding to at least one camera movement node in the initial camera movement sequence during the movement process; According to the movement amplitude of the lens picture during the movement process, the corresponding camera movement node is configured. point, generate optimized camera movement nodes; The optimized camera movement sequence is generated according to the optimized camera movement node. The method according to claim 1, wherein the video content feature comprises a third content feature, and the third content feature represents a target object in the video to be processed; The step of obtaining an optimized camera movement sequence according to the video content feature and the initial camera movement sequence includes: Determine, according to the third content feature, a change frequency of a shot picture corresponding to at least one camera movement node in the initial camera movement sequence during the camera movement process, wherein the change frequency is less than a preset frequency threshold; According to the change frequency of the lens picture during the lens movement process, a corresponding lens movement node is configured to generate an optimized lens movement node; The optimized camera movement sequence is generated according to the optimized camera movement node. The method according to claim 4, wherein generating a third parameter of at least one camera node in the initial camera sequence according to the third content feature comprises: Obtaining the outline size of the target object; According to the outline size, a corresponding target motion frequency is obtained, wherein the target motion frequency is inversely proportional to the outline size; The third parameter is set according to the target motion frequency. The method according to claim 1, wherein obtaining the optimized camera movement sequence according to the video content features and the initial camera movement sequence comprises: Acquire video content features of the initial video; According to the video content features of the initial video and the video content features of the video to be processed, feature deviation information is obtained, wherein the feature deviation information represents the feature deviation amount in video content between the video to be processed and the initial video; According to the characteristic deviation information, node parameters of at least one camera movement node in the initial camera movement sequence are set to obtain the optimized camera movement sequence. The method according to claim 6, wherein the feature deviation information includes a position deviation amount, wherein the position deviation amount represents a deviation between a first position where the portrait in the initial video is located and a second position where the portrait in the video to be processed is located; According to the characteristic deviation information, setting a node parameter of at least one camera movement node in the initial camera movement sequence to obtain the optimized camera movement sequence includes: Obtaining a target camera movement node corresponding to the position deviation in the initial camera movement sequence; According to the position deviation, the center point of the lens picture corresponding to the target camera movement node during the camera movement process is set to obtain the optimized camera movement sequence. The method according to claim 6, wherein the feature deviation information includes a scale coefficient, and the scale coefficient represents the ratio of the scene scale in the initial video to the scene scale in the video to be processed; According to the characteristic deviation information, setting a node parameter of at least one camera movement node in the initial camera movement sequence to obtain the optimized camera movement sequence includes: Obtaining a target camera movement node corresponding to the scale coefficient in the initial camera movement sequence; According to the scale coefficient, the movement amplitude of the lens picture corresponding to the target camera movement node during the movement process is set to obtain the optimized camera movement sequence. The method according to claim 1, wherein obtaining video content features of the video to be processed comprises: Acquire a corresponding target video frame in the video to be processed according to a timestamp of at least one of the camera movement nodes in the initial camera movement sequence; Content detection is performed on the target video frame to obtain the video content feature. The method according to claim 1, wherein the initial video has a first playback duration, and the video to be processed has a second playback duration, and when the second playback duration is greater than the first playback duration, the method further comprises: generating an extended camera movement sequence according to at least one camera movement node in the optimized camera movement sequence, wherein the camera movement duration of the shot picture corresponding to the extended camera movement sequence is not less than the difference between the second playback duration and the first playback duration; The step of processing the video to be processed based on the optimized camera movement sequence to generate an output video includes: The video to be processed is processed based on the optimized camera movement sequence and the extended camera movement sequence in sequence to generate an output video. The method according to claim 10, wherein the optimization of the camera sequence At least one camera movement node in the column generates an extended camera movement sequence, including: Acquire identification features of a first video segment in the video to be processed, wherein the first video segment is a video segment after a first playback duration in the video to be processed, and the identification features include video content features and/or music melody features of the first video segment; According to the identification feature, a second video segment is obtained from the video to be processed, where the second video segment is a video segment before the first playback duration in the video to be processed, and the identification feature of the second video segment is similar to the identification feature of the first video segment; The extended camera movement sequence is generated according to at least one continuous camera movement node corresponding to the second video clip. A video generating device, comprising: A loading module, used to obtain an initial camera movement sequence according to an initial camera movement template, wherein the initial camera movement sequence includes at least one camera movement node corresponding to an initial video, and the camera movement node is used to represent the camera movement characteristics of a lens picture during video playback; A processing module, used for obtaining video content features of a video to be processed, and obtaining an optimized camera movement sequence according to the video content features and the initial camera movement sequence, wherein the optimized camera movement sequence includes at least one optimized camera movement node, and the camera movement features corresponding to the optimized camera movement node match the video content features; An output module is used to process the video to be processed based on the optimized camera movement sequence to generate an output video, wherein the shot images of the output video have the camera movement features corresponding to the optimized camera movement nodes. An electronic device comprises: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the video generating method according to any one of claims 1 to 11. A computer-readable storage medium having computer-executable instructions stored therein, wherein when a processor executes the computer-executable instructions, the video generation method according to any one of claims 1 to 11 is implemented. A computer program product tangibly stored in The computer storage medium contains computer executable instructions, which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 11.