CN115174822A

CN115174822A - Video special effect adding method and device

Info

Publication number: CN115174822A
Application number: CN202210703378.2A
Authority: CN
Inventors: 李立锋; 谢韬
Original assignee: Migu Cultural Technology Co Ltd; China Mobile Communications Group Co Ltd
Current assignee: Migu Cultural Technology Co Ltd; China Mobile Communications Group Co Ltd
Priority date: 2022-06-21
Filing date: 2022-06-21
Publication date: 2022-10-11

Abstract

The invention discloses a video special effect adding method and a device, wherein the method comprises the following steps: acquiring optical flow information of any lens unit; determining an object displaced in a lens unit according to the optical flow information, and acquiring basic information and track information of the object; and determining special effect parameters corresponding to the lens unit according to the basic information and the track information of the object and a preset special effect parameter corresponding rule so as to transmit the special effect parameters to a special effect engine to finish the special effect rendering of the video. By utilizing the method and the device, the special effect can be automatically added to the video according to the object presented in the video in real time, and the added special effect is related to the track information and the basic information of the object, so that the added special effect is ensured to accord with the video presentation effect. The added special effect does not need to depend on a professional tool, and the technical threshold of adding the special effect is also reduced.

Description

Video special effect adding method and device

Technical Field

The embodiment of the invention relates to the technical field of video special effect processing, in particular to a video special effect adding method and device.

Background

The video special effect can improve the effect of watching the video by a user, and the video special effect addition is generally realized by adopting the following modes:

by using an AE (After Effects, nonlinear special effect making software), various different types of special Effects such as dynamic pictures, 2D Effects, 3D Effects and the like can be directly added to the video, but the AE tool has a higher threshold, longer making time and higher professional degree.

The method has the advantages that the paste image class is used for special effect addition, the presented effect is based on template solidification, although the operation is simple, the effect generated on each object in the video is almost consistent, and the characteristics of the object cannot be highlighted.

When an AI is used to generate a special effect, although the threshold for use is relatively low, the versatility is very poor, and the special effect processing can be generally performed only for a certain object class. When an object other than the specific object class is added, problems such as drift and disappearance of the special effect occur.

Disclosure of Invention

In view of the above problems, embodiments of the present invention are proposed to provide a video special effect adding method and apparatus that overcome or at least partially solve the above problems.

According to an aspect of an embodiment of the present invention, there is provided a video special effect adding method, including:

carrying out shot segmentation on the video to obtain a plurality of shot units generated by shot switching;

acquiring optical flow information of any lens unit;

determining an object displaced in a lens unit according to the optical flow information, and acquiring basic information and track information of the object;

and determining special effect parameters corresponding to the lens unit according to the basic information and the track information of the object and a preset special effect parameter corresponding rule so as to transmit the special effect parameters to a special effect engine to finish the special effect rendering of the video.

According to another aspect of the embodiments of the present invention, there is provided a video special effect adding apparatus, including:

the segmentation module is suitable for carrying out shot segmentation on the video to obtain a plurality of shot units generated by shot switching;

the optical flow acquisition module is suitable for acquiring optical flow information of any lens unit;

the track acquisition module is suitable for determining an object displaced in a lens unit according to the optical flow information and acquiring basic information and track information of the object;

and the rendering module is suitable for determining the special effect parameters corresponding to the lens unit according to the basic information and the track information of the object and the preset special effect parameter corresponding rule so as to transmit the special effect parameters to the special effect engine to finish the special effect rendering of the video.

According to still another aspect of an embodiment of the present invention, there is provided a computing device including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the video special effect adding method.

According to still another aspect of the embodiments of the present invention, a computer storage medium is provided, where at least one executable instruction is stored, and the executable instruction causes a processor to perform an operation corresponding to the video special effect adding method.

According to the video special effect adding method and device provided by the embodiment of the invention, the special effect can be automatically added according to the object presented in the video in real time, and the added special effect is related to the track information and the basic information of the object, so that the added special effect is ensured to be in accordance with the video presenting effect. The added special effect does not need to depend on a professional tool, and the technical threshold of adding the special effect is also reduced.

The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and in order that the technical solutions of the embodiments of the present invention can be clearly understood, the embodiments of the present invention can be implemented according to the content of the description, and the above and other objects, features, and advantages of the embodiments of the present invention can be more clearly understood, the following detailed description of the embodiments of the present invention is given.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the embodiments of the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

fig. 1 shows a flow diagram of a video effect addition method according to an embodiment of the invention;

FIG. 2 shows a video frame optical flow information presentation diagram;

FIG. 3 is a schematic diagram illustrating object trajectory information presentation in a video;

FIG. 4 is a diagram illustrating the recognition result of an object in a video;

fig. 5 is a diagram illustrating correspondence between special effect parameters and basic information and track information of an object;

FIG. 6 is a graph showing track speed variation and track direction variation;

fig. 7 is a schematic structural diagram of a video special effects adding apparatus according to an embodiment of the present invention;

FIG. 8 illustrates a schematic structural diagram of a computing device in accordance with one embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Fig. 1 shows a flow chart of a video effect adding method according to an embodiment of the invention, as shown in fig. 1, the method comprises the following steps:

step S101, performing shot segmentation on the video to obtain a plurality of shot units generated by shot switching.

In the embodiment, it is considered that there is shot switching in the video, and if a shot is switched from one object a to another object b, the special effect addition to the original object a cannot be directly applied to the object b, so before the special effect is added, the shot segmentation is performed on the video, the special effect addition to the video is changed into the special effect addition to a plurality of shot units, and the added special effect is ensured to be more attached to each object in the video content.

Specifically, when a video is shot-cut, whether the similarity is smaller than a similarity threshold may be determined by calculating the similarity of adjacent video frames in the video, and the similarity threshold may be set according to an implementation situation, which is not limited herein. If the SIFT algorithm is used for extracting key special effects in two adjacent video frames to calculate the similarity, if the similarity is smaller than a similarity threshold value, the shot is switched, and shot segmentation is carried out on the basis of the adjacent video frames to obtain a plurality of shot units generated according to the shot switching.

In step S102, for any lens unit, optical flow information of the lens unit is acquired.

For any one of a plurality of lens units generated by lens division, optical flow information of the lens unit is acquired. The optical flow information may be obtained based on optical flow methods, such as Horn-Schunck, lucas-Kanade, farneback, and the like, and is not limited herein.

Step S103 specifies an object displaced in a lens unit based on the optical flow information, and acquires basic information and trajectory information of the object.

From the optical flow information, the pixel ratio of the optical flow change can be calculated. The optical flow information of all pixels in a video frame is sensed, the coordinate change of the pixel (that is, the optical flow change) is determined, and the pixel ratio of the optical flow change is obtained by comparing the pixel where the optical flow change occurs with all the pixels. And judging whether the pixel ratio exceeds a preset ratio threshold, if so, determining that the lens in the lens unit is displaced, and correspondingly acquiring lens displacement information in the lens unit. Here, the preset duty ratio threshold may be set according to implementation, such as 99%, and is not limited herein.

The lens displacement information includes a lens displacement speed, a lens displacement direction, and the like. When lens displacement information is obtained, classification can be carried out according to pixel displacement speed in a video frame, pixels with the same displacement speed are classified into one class, if the pixels are ranked according to the pixel displacement speed, the pixels with the same displacement speed are ranked in the same rank, and the number of pixel repetition with the same displacement speed is obtained through statistics. The pixel moving speed corresponding to the maximum value of the pixel repetition number is taken as the lens moving speed, and at this time, the opposite direction of the pixel moving direction is taken as the lens moving direction.

Taking fig. 2 as an example for explanation, each arrow in the picture in fig. 2 represents a displacement direction, an arrow length represents a displacement speed, and the longer the arrow, the larger the displacement speed. The arrows may be determined from the optical flow information, and each arrow in the picture may be determined from the optical flow information of each pixel. The light-colored long arrow at the position of the airship (hereinafter, the airship a) above the picture represents the displacement direction and the displacement speed of the airship a; an oblique upper arrow at the position of the airship (hereinafter referred to as the airship B) below the picture represents the displacement direction and the displacement speed of the airship B; each arrow in the right direction in the picture is the relative displacement direction and displacement speed of the static object caused by lens displacement; the left arrow on the right of the camera in the picture represents the displacement direction and displacement speed of the camera (lens). The acquiring of the left arrow specifically includes: the arrows in the picture may be sorted according to the length of the arrow (i.e. the displacement speed), for example, sorted from large to small, and the same length of the arrow is grouped together to obtain the number of pixel repetitions with the same displacement speed. For convenience of understanding, the displacement speeds in fig. 2 are sequenced, and for example, the arrows in fig. 2 are used as the numbers, so that 8 light long arrows of the airship a, 9 obliquely upward arrows of the airship B, and 57 arrows of the static object in the right direction can be obtained in sequence. The arrow with the largest number of repetitions is the arrow of the static object in the right direction, and the displacement direction of the lens can be determined to be the opposite direction to the arrow of the static object in the left direction; the displacement speed of the lens is the same as that, that is, the arrow length of the lens is the same as the arrow length of the static object in the right direction. Further, when the displacement speed of the pixels in the video frame is classified, the number of the pixels is large, and the processing amount is large; the problems of deviation and the like are easy to occur for some scenes such as close-up, close-up and the like. In view of the above, it is possible to recognize an object in a lens unit and process the object that has been displaced as a whole, and by taking fig. 2 as an example, it is possible to obtain 1 displacement velocity of the airship a and 1 displacement velocity of the airship B in this order, thereby simplifying the amount of processing and improving the processing accuracy. Each object in the unit of a lens can be obtained by performing object recognition based on optical flow information, or by dividing the object and acquiring basic information of the object. The basic information includes, for example, object name, object color, object outline, object size, etc. As in fig. 2, it is possible to identify 2 object airship, segment the object outline, determine the size of the object in lens units, the color presented in lens units, and the like.

From the lens displacement information and the optical flow information, object displacement information can be calculated. If the lens is static, namely the displacement speed is 0, the obtained optical flow information, namely the optical flow information of the object, can directly determine the displacement information of the object; if the lens is moving, that is, if there is a displacement velocity, the displacement information of the lens calculated as illustrated in fig. 2 needs to be added to the displacement information of the lens determined based on the optical flow information, that is, the displacement information determined based on the optical flow information needs to be added to the deviation caused by the movement of the lens to obtain the displacement information of the object. Taking fig. 2 as an example, the displacement information of the airship a needs to be obtained by adding the light-color long arrow of the airship a to the left arrow of the lens, adding vectors, and adjusting the direction and length of the light-color long arrow. Or, an optical flow picture relative reference mode can also be adopted, such as extracting a characteristic point A1 of an object position in a previous frame and a characteristic point A2 of an object position in a next frame in adjacent video frames; the feature point B1 of the non-object position in the previous frame in the adjacent video frames and the corresponding feature point B2 of the non-object position in the next frame are matched through the non-object position feature points, namely the feature point B1 is matched with the feature point B2, the coordinates of the adjacent video frames of the anchor points are matched and overlapped with the non-object position feature points, and the direction and the distance between the A1 and the A2 are the displacement direction and the displacement distance of the object. The above is an example, and the specific implementation can be set according to the implementation, and is not limited herein.

And determining the track information of the object in the lens unit according to the object displacement information. The track information includes, for example, track type, track length, track speed, track direction, etc. The object displacement information in each video frame in the lens unit is counted to obtain the track length and the track direction of the object, the track speed is further calculated, and the tracks in the video frames are classified to obtain the track type. Specifically, according to displacement information of the object in all video frames in a lens unit, a motion trajectory of the object in the lens unit can be acquired. As shown in fig. 3, the left side is a video frame, the line in the right side is the motion trajectory of each vehicle, and the displacement information of all video frames in the unit of a lens is obtained through statistics. The motion trajectory can be classified to obtain the trajectory type by using classification algorithms such as a classification model, an SVM, machine learning and the like. The trace types are shown in table 1 below:

type of track	Scenario examples or illustrations
		Circle (C)	Ballet spin
Arc line	Semicircle
		Parabola line	Volleyball
Straight line	Boxing, sprinting
		S line	Wandering away
Direction changing	Ball impact quilt rebounding
		Direction change to and fro	Float up and down

TABLE 1

As shown in table 1, the first column is a track type, and the second column is an example or description of a corresponding scenario, that is, a scenario in which the corresponding track type occurs. Table 1 is merely an example, and the type of the track may be set according to the implementation, which is not limited herein. The track information includes, in addition to the track type, track length, track speed, track direction, etc., the track length and the track speed being based on the size in the video picture. The trajectory direction is substantially the same as the displacement direction. After obtaining the basic information and the track information of the object, the track information of the object may be associated with the basic information. As shown in fig. 4, the object subjected to displacement is identified as an automobile, the basic information and the trajectory information of the associated object can obtain that the trajectory type of the automobile is a straight line, the trajectory length is 1920 pixels, the trajectory speed is 800 pixels/s, the trajectory direction is 0 degrees, the name of the object is the automobile, and the color is #2B323C (here, the average color value or the local color value of the object may be taken, a plurality of different color values may be reserved according to local features, different special effects are added correspondingly to the subsequent steps, and the like), the contour is obtained by performing contour extraction on object segmentation, and the size of the object is 200 × 150 pixels and the like. Further, the embodiment mainly adds special effects to objects which are displaced, and static objects which are not displaced do not need to be identified.

And step S104, determining a special effect parameter corresponding to the lens unit according to the basic information and the track information of the object and a preset special effect parameter corresponding rule, so that the special effect parameter is transmitted to a special effect engine to finish the special effect rendering of the video.

In this embodiment, a preset special effect parameter correspondence rule may be preset, where a correspondence between a special effect parameter and basic information and trajectory information of an object may be established for each special effect style, so as to obtain a preset special effect parameter correspondence rule. According to the basic information of the object in the video and the change of the track information, the dynamic special effect adjustment can be realized, and the current state of the moving object in the video can be self-adapted. As shown in fig. 5, fig. 5 includes various special effect patterns, such as streamer, lightning, flame, rain, cloud, and the like, where a dotted arrow represents a corresponding relationship, and the special effect pattern corresponds to a track classification, a corresponding relationship between a special effect parameter (the special effect adjustment parameter in fig. 5) and information such as basic information and track information. For illustration only in fig. 5, the special effect parameters may include parameters such as brightness, contrast, vibration amplitude, saturation, repulsion, gravity, particle size, original particle shape, and optical flow density, in addition to the rendering manner (pasting image following, particle rendering, filter, etc.), special effect color (same color system with the object, inverse color system with the object, random color, and set as xxx color), special effect position (attached to the object, overlapped with the object, corresponding object coordinate: increasing or decreasing x and y coordinate positions), special effect transparency, special effect density, special effect strength, special effect display frequency, and special effect size shown in fig. 5, which are set according to the implementation and are not limited herein. The object is taken as an example in fig. 5, but the object in this embodiment may also include various objects such as a human image and an animal and plant. The corresponding relationship between each special effect parameter and the basic information and trajectory information of the object may be many-to-one, for example, the rendering mode of the particles and the filter may be used for one object at the same time, and the specific corresponding relationship is set according to the implementation, and is not limited herein.

Further, when the video includes text information and audio information, such as subtitles, BGM background music, episode music, and the like, the text information and the audio information in the video may be acquired, and the emotion information of the video is determined according to the text information and the audio information. For example, the method includes recognizing subtitles in a video by technologies such as OCR and the like to obtain a corresponding text, pre-training a text-emotion classification model by a text-emotion classification algorithm such as NLP and the like, or establishing an emotion word stock and the like to carry out emotion classification on the text to obtain emotion information. The emotional information includes, for example, an emotion classification, a score, etc. The audio information can also be classified by a music-emotion classification algorithm, such as a classification algorithm using an SVM, machine learning, deep learning, and the like, so as to obtain emotion information corresponding to the audio information. And determining special effect parameters corresponding to the lens units according to the emotional information of the video and a preset special effect parameter corresponding rule. As shown in fig. 5, the related parameters further include emotion and lines, which correspond to emotion information.

The special effects are illustrated in table 2 below as an example:

TABLE 2

Wherein, taking the special effect style as streamer and the track type as straight line as examples, if someone breaks a fist in the corresponding video, the special effect of the bomb map 1101 is displayed on the fist; when strong anger is also included in the video, the special effect of the flame map 1104 appears full screen. Taking the special effect pattern as streamer and the track type as circle as examples, the rendering mode may include two different modes, i.e., particle and filter, and the special effect position may adopt different positions according to the size of the object, for example, the size of the object is greater than 200 × 200 pixels, the special effect position is attached to the object, the size of the object is less than 200 × 200 pixels, the special effect position overlaps the object, and the like.

Furthermore, in order to control the consistency and integrity of the special effects of each lens unit of the video conveniently, the video presents a global overall special effect, and whether the preset special effect triggering condition is met or not can be judged according to the basic information and the track information of the object. The preset special effect triggering conditions include that the track speed variation value is larger than the preset speed variation value, the track direction variation value is larger than the preset direction variation value, the track length is within the preset length special effect range value, the track speed is within the preset speed special effect range value, the special effect type number is within the preset type range value, the special effect total number is within the preset special effect number range value, the object size is within the preset object size range value, and the like. Setting different preset special effect triggering conditions for different special effect effects, setting corresponding preset variable special effect parameters for the object when the basic information and the track information of the object accord with the corresponding preset special effect triggering conditions, adjusting the special effect to different degrees, and transmitting the preset variable special effect parameters and the special effect parameters to a special effect engine to finish the special effect rendering of the video. As shown in table 3 below:

TABLE 3

Wherein, according to different track lengths of the objects, different preset variable special effect parameters are corresponded, for example, the track length is 1250 pixels, and the weighting is 0.8 corresponding to the general effect of the special effect; the track length is 2000 pixels, corresponding to the effect of few special effects, the weight is 0.5, and the like. The special effect types in the same shot comprise 3 types, and the weight is 0.8 corresponding to the general effect of the special effect; the special effect types in the same shot comprise 2 types, corresponding to the effect of a small number of special effects, the weight is 0.5, and the like. Other things are the same, the number of the special effects is further reduced by setting the preset special effect triggering conditions, and the excessive special effects are further prevented by limiting the number and the types. Preset values such as a preset length special effect range value, a preset speed special effect range value, a preset type range value, a preset special effect quantity range value, a preset object size range value and the like in the conditions that the track length is within a preset length special effect range value, the track speed is within a preset speed special effect range value, the special effect type quantity is within a preset type range value, the special effect total quantity is within a preset special effect quantity range value, the object size is within a preset object size range value and the like in the preset special effect trigger condition are set according to implementation conditions, and are not limited herein.

Further, in this embodiment, the preset special effect triggering condition further includes that the track speed variation value is greater than the preset speed variation value, and the track direction variation value is greater than the preset direction variation value. The trajectory speed variation value and the trajectory direction variation value may be determined by acquiring optical flow information of the object in units of video frames or seconds and determining the trajectory speed variation value and the trajectory direction variation value. Data in which the trajectory speed and the trajectory direction of the object vary greatly is determined based on the time sequence (video frame time sequence, etc.), and as shown in fig. 6, the trajectory speed in the gray area in the left image varies greatly and the trajectory direction in the gray area in the right image varies greatly. And judging whether the track speed variation value is greater than the preset speed variation value or the track direction variation value is greater than the preset direction variation value through a preset special effect triggering condition, and correspondingly setting a preset variation special effect parameter. The preset change special effect parameter can set the special effect of adding particles and the like. A plurality of conditions in the preset special effect triggering conditions can be judged simultaneously, and one or more conditions can be selected for judgment, so that the special effect continuity and overall harmony in the video are guaranteed.

The preset variable special effect parameter can be transmitted to the special effect engine together with the special effect parameter, the preset variable special effect parameter can be set into a format meeting the requirement of the special effect engine, the preset variable special effect parameter and the special effect parameter are transmitted through an interface provided by the special effect engine, and the special effect engine carries out special effect rendering on the video according to the received preset variable special effect parameter and the received special effect parameter.

According to the video special effect adding method provided by the embodiment of the invention, the special effect can be automatically added according to the object presented in the video in real time, and the added special effect is related to the track information and the basic information of the object, so that the added special effect is ensured to accord with the video presenting effect. The added special effect does not need to depend on a professional tool, and the technical threshold of special effect addition is also reduced.

Fig. 7 is a schematic structural diagram illustrating a video special effect adding apparatus according to an embodiment of the present invention. As shown in fig. 7, the video special effects adding apparatus includes:

the segmentation module 710 is adapted to perform shot segmentation on the video to obtain a plurality of shot units generated by shot switching;

an optical flow acquisition module 720 adapted to acquire optical flow information of any lens unit for the lens unit;

a track obtaining module 730, adapted to determine an object displaced in the lens unit according to the optical flow information, and obtain basic information and track information of the object;

the rendering module 740 is adapted to determine a special effect parameter corresponding to the lens unit according to the basic information and the track information of the object and a preset special effect parameter correspondence rule, so as to transmit the special effect parameter to a special effect engine to complete special effect rendering on a video.

Optionally, the segmentation module 710 is further adapted to:

calculating the similarity of adjacent video frames in the video, and judging whether the similarity is smaller than a similarity threshold value;

if yes, determining that shot switching occurs in the video, and performing shot segmentation based on adjacent video frames to obtain a plurality of shot units generated by the shot switching.

Optionally, the trajectory acquisition module 730 is further adapted to:

calculating the pixel ratio of the optical flow change according to the optical flow information;

judging whether the pixel proportion exceeds a preset proportion threshold value or not;

if so, acquiring lens displacement information in the lens unit;

identifying and obtaining an object in a lens unit according to the optical flow information, and acquiring basic information of the object; the basic information comprises an object name, an object color, an object outline and/or an object size;

calculating object displacement information according to the lens displacement information and the optical flow information;

determining the track information of the object in the lens unit according to the object displacement information; the track information includes a track type, a track length, a track speed, and/or a track direction.

Optionally, the lens shift information includes a lens shift speed and a lens shift direction;

the optical flow acquisition module 720 is further adapted to:

classifying according to the pixel displacement speed to obtain the pixel repetition number with the same displacement speed;

the pixel moving speed corresponding to the maximum value in the pixel repetition number is taken as the lens displacement speed, and the opposite direction of the pixel moving direction is taken as the lens displacement direction.

Optionally, the trajectory acquisition module 730 is further adapted to:

and counting the object displacement information in each video frame in the lens unit to obtain the track length and/or track direction of the object, calculating to obtain track speed, and classifying tracks in the video frames to obtain track types.

Optionally, the rendering module 740 is further adapted to:

judging whether a preset special effect triggering condition is met or not according to the basic information and/or the track information of the object; the preset special effect triggering conditions comprise: the track speed variation value is greater than a preset speed variation value, the track direction variation value is greater than a preset direction variation value, the track length is within a preset length special effect range value, the track speed is within a preset speed special effect range value, the special effect type number is within a preset type range value, the special effect total number is within a preset special effect number range value, and/or the object size is within a preset object size range value;

if so, setting a corresponding preset variable special effect parameter for the object, so as to transmit the preset variable special effect parameter and the special effect parameter to a special effect engine to finish the special effect rendering of the video.

Optionally, the apparatus further comprises: the emotion acquiring module 750 is suitable for acquiring text information and/or audio information in the video and determining emotion information of the video according to the text information and/or the audio information; and determining a special effect parameter corresponding to the lens unit according to the emotion information of the video and a preset special effect parameter corresponding rule so as to transmit the special effect parameter to a special effect engine to finish the special effect rendering of the video.

The descriptions of the modules refer to the corresponding descriptions in the method embodiments, and are not repeated herein.

The embodiment of the invention also provides a nonvolatile computer storage medium, wherein the computer storage medium stores at least one executable instruction, and the executable instruction can execute the video special effect adding method in any method embodiment.

Fig. 8 is a schematic structural diagram of a computing device according to an embodiment of the present invention, and a specific embodiment of the present invention does not limit a specific implementation of the computing device.

As shown in fig. 8, the computing device may include: a processor (processor) 802, a Communications Interface 804, a memory 806, and a communication bus 808.

The method is characterized in that:

the processor 802, communication interface 804, and memory 806 communicate with one another via a communication bus 808.

A communication interface 804 for communicating with network elements of other devices, such as clients or other servers.

The processor 802 is configured to execute the program 810, and may specifically execute relevant steps in the above-described video special effect adding method embodiment.

In particular, the program 810 may include program code comprising computer operating instructions.

The processor 802 may be a central processing unit CPU, or an Application Specific Integrated Circuit ASIC (Application Specific Integrated Circuit), or one or more Integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

The memory 806 stores a program 810. The memory 806 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 810 may be specifically configured to cause the processor 802 to perform a video special effects addition method in any of the method embodiments described above. For specific implementation of each step in the program 810, reference may be made to corresponding steps and corresponding descriptions in units in the above-described video special effect adding embodiment, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.

The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of embodiments of the present invention as described herein, and any descriptions of specific languages are provided above to disclose preferred embodiments of the invention.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that is, the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Moreover, those of skill in the art will appreciate that while some embodiments herein include some features included in other embodiments, not others, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components according to embodiments of the present invention. Embodiments of the invention may also be implemented as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing embodiments of the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specified otherwise.

Claims

1. A video special effect adding method, comprising:

acquiring optical flow information of any lens unit;

determining an object displaced in the lens unit according to the optical flow information, and acquiring basic information and track information of the object;

2. The method of claim 1, wherein the shot segmentation of the video to obtain a plurality of shot units generated by shot cut further comprises:

calculating the similarity of adjacent video frames in the video, and judging whether the similarity is smaller than a similarity threshold value or not;

3. The method according to claim 1, wherein the determining an object displaced in the lens unit according to the optical flow information, and the acquiring basic information and trajectory information of the object further comprises:

calculating the pixel ratio of optical flow change according to the optical flow information;

if so, acquiring lens displacement information in the lens unit;

4. The method of claim 3, wherein the lens shift information comprises a lens shift speed and a lens shift direction;

the acquiring lens displacement information in the lens unit further includes:

classifying according to the pixel displacement speed to obtain the pixel repetition quantity with the same displacement speed;

5. The method according to claim 3, wherein the determining trajectory information of the object in the lens units according to the object displacement information further comprises:

6. The method according to claim 1, wherein determining the special effect parameter corresponding to the lens unit according to the basic information and the track information of the object and a preset special effect parameter correspondence rule, so that transmitting the special effect parameter to a special effect engine to complete special effect rendering of the video further comprises:

7. The method according to any one of claims 1-6, further comprising:

acquiring text information and/or audio information in the video, and determining emotion information of the video according to the text information and/or the audio information;

and determining a special effect parameter corresponding to the lens unit according to the emotion information of the video and a preset special effect parameter corresponding rule so as to transmit the special effect parameter to a special effect engine to finish the special effect rendering of the video.

8. A video special effect adding apparatus, characterized in that the apparatus comprises:

the track acquisition module is suitable for determining an object which is displaced in the lens unit according to the optical flow information and acquiring basic information and track information of the object;

and the rendering module is suitable for determining the special effect parameters corresponding to the lens units according to the basic information and the track information of the object and a preset special effect parameter corresponding rule so as to transmit the special effect parameters to a special effect engine to finish the special effect rendering of the video.

9. A computing device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the video special effect adding method according to any one of claims 1-7.

10. A computer storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the video special effect adding method according to any one of claims 1 to 7.