WO2023051185A1 - 图像处理方法、装置、电子设备及存储介质 - Google Patents
图像处理方法、装置、电子设备及存储介质 Download PDFInfo
- Publication number
- WO2023051185A1 WO2023051185A1 PCT/CN2022/117167 CN2022117167W WO2023051185A1 WO 2023051185 A1 WO2023051185 A1 WO 2023051185A1 CN 2022117167 W CN2022117167 W CN 2022117167W WO 2023051185 A1 WO2023051185 A1 WO 2023051185A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- target
- special effect
- video frame
- model
- image
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 238000000034 method Methods 0.000 claims abstract description 50
- 238000012545 processing Methods 0.000 claims abstract description 31
- 230000004044 response Effects 0.000 claims abstract description 12
- 238000007499 fusion processing Methods 0.000 claims abstract description 10
- 230000003321 amplification Effects 0.000 claims abstract 2
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract 2
- 230000000694 effects Effects 0.000 claims description 186
- 230000008569 process Effects 0.000 claims description 19
- 230000003068 static effect Effects 0.000 claims description 5
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 32
- 238000010586 diagram Methods 0.000 description 13
- 241000282326 Felis catus Species 0.000 description 10
- 230000001960 triggered effect Effects 0.000 description 9
- 238000004590 computer program Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 241001125929 Trisopterus luscus Species 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000013307 optical fiber Substances 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000000481 breast Anatomy 0.000 description 1
- 238000013216 cat model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Definitions
- Embodiments of the present disclosure relate to the technical field of image processing, for example, to an image processing method, device, electronic equipment, and storage medium.
- the present disclosure provides an image processing method, device, electronic equipment, and storage medium, so as to realize the richness and interest of video shooting content, thereby improving the technical effect of user experience.
- an embodiment of the present disclosure provides an image processing method, the method including:
- an embodiment of the present disclosure further provides an image processing device, which includes:
- the display video frame determination module is configured to respond to the detection of the trigger special effect display function, add a virtual model to the collected image to be processed, and obtain the display video frame;
- a three-dimensional special effect determination module configured to enlarge and display the image area of the virtual model in the display video frame, and process the virtual model as a target three-dimensional special effect model in response to determining that a stop zoom-in condition is detected;
- the target video frame determination module is configured to fuse and process the target three-dimensional special effect model and the target object in the image to be processed, and display the target video frame.
- an embodiment of the present disclosure further provides an electronic device, and the electronic device includes:
- processors one or more processors
- storage means configured to store one or more programs
- the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of the embodiments of the present disclosure.
- an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, the computer-executable instructions are configured to perform the image processing described in any one of the embodiments of the present disclosure when executed by a computer processor method.
- FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure
- FIG. 2 is a schematic diagram of adding virtual special effects to an image to be processed provided by an embodiment of the present disclosure
- FIG. 3 is a schematic diagram of enlarging an image area corresponding to a virtual special effect provided by an embodiment of the present disclosure
- FIG. 4 is a schematic diagram of processing a virtual special effect into a target 3D special effect and merging the target 3D special effect with a target object provided by an embodiment of the present disclosure
- FIG. 5 is a schematic flowchart of an image processing method provided by another embodiment of the present disclosure.
- FIG. 6 is a schematic structural diagram of an image processing device provided by a third embodiment of the present disclosure.
- FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
- the term “comprise” and its variations are open-ended, ie “including but not limited to”.
- the term “based on” is “based at least in part on”.
- the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one further embodiment”; the term “some embodiments” means “at least some embodiments.” Relevant definitions of other terms will be given in the description below.
- the technical solution of the present disclosure can be applied to any scene where a video can be shot and played.
- the special effects disclosed in this technical solution can be added to the calling user; or, in a live broadcast scene, the special effects disclosed in this technical solution can be added to the host user; of course, it can also be applied in the short video shooting process, you can
- the technical solution of the present disclosure is executed during the process of photographing the subject.
- FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure.
- the embodiment of the present disclosure is applicable to any scene of video display supported by the Internet, and is used to add special effects to the target object during video shooting, and then display video frames including special effects, and the method can be executed by an image processing device , the device may be implemented in the form of software and/or hardware.
- the electronic device may be a mobile terminal, a PC terminal, or a server.
- the scene of arbitrary image display is usually implemented by the cooperation of the client and the server.
- the method provided in this embodiment can be executed by the server, the client, or the cooperation of the client and the server.
- the device for executing the image processing method may be integrated into application software supporting image processing functions, and the software may be installed in electronic equipment.
- the electronic device may be a mobile terminal or a PC.
- the application software may be a type of software for image/video processing, and its specific application software will not be described here one by one, as long as the image/video processing can be realized.
- the display interface may include buttons for target objects and adding special effects.
- the special effect button When the user triggers the special effect button, at least one special effect to be added may pop up, and the user may select one of the multiple special effects to be added as the target special effect.
- the server may determine to add the corresponding special effect to the object in the captured image.
- the image to be processed may be an image collected based on application software, and in a specific application scenario, the image to be processed may be collected in real time or periodically.
- the camera device collects images including the target object in the target scene in real time, and at this time, the image collected by the camera device may be used as an image to be processed.
- the virtual model can be a model added to it, and the model can be a virtual model corresponding to a special effect to be displayed.
- the special effect to be displayed is a cat
- the color of the cat is black
- the virtual model can be an augmented reality corresponding to a big black cat ( Augmented Reality, AR) special effect model, see the content marked as 1 in Figure 2.
- Virtual models can be added anywhere in the image to be processed.
- the image to be processed after adding the special effect model can be used as a display video frame.
- the video frame corresponding to the triggering special effect display function can be used as the image to be processed, and a virtual model is added to the image to be processed during the video shooting process to obtain the display video frame.
- the user can trigger a button for adding a special effect on the display interface, that is, the special effect display function is triggered.
- the special effect display function is triggered.
- multiple special effects to be added may be displayed, and the user may determine one special effect from the multiple special effects to be added.
- the image corresponding to the triggering special effect display function can be used as the image to be processed, and the virtual model corresponding to the special effect can be added to the image to be processed to obtain a display video frame.
- the area occupied by the virtual model in the image to be processed may be used as the image area of the virtual model.
- the enlarged display may be to enlarge and display the virtual model according to a preset ratio. For example, the image area is enlarged by 10% each time, or the virtual model is enlarged and displayed according to a preset time. For example, every 50ms, it will be enlarged by 10% on the original basis. That is to say, the enlarged image area of the virtual model is displayed as a video frame in the video.
- the zoom-in stop condition can be understood as a condition for no longer zooming in when the image area is zoomed in to a certain extent.
- the zoom-in please refer to the content marked 1 in Fig. 3 .
- the target 3D special effect model corresponds to the virtual model.
- the 3D target special effect can be a big cat model as shown in Figure 4.
- the big cat target 3D special effect model is a model close to a real pet.
- the pet in the target 3D special effect model Many parts of the pet can be moved, for example, the pupils in the eyes can be rotated, the eyes can be closed, and the tail can be shaken.
- the image area corresponding to the virtual model can be determined, and the image area can be enlarged and displayed.
- the target three-dimensional special effect model corresponding to the virtual model may be determined.
- a plurality of virtual special effects of pets and target three-dimensional special effects corresponding to the virtual special effects can be pre-stored in the server, which can be set according to actual needs in a specific application process.
- the target display video frame may be a video frame obtained after the target three-dimensional special effect is displayed to the target object on the image to be processed.
- the subject in the image to be processed may be photographed, for example, a user, and the user may be used as a target object in the image to be processed.
- Fusion processing may be the effect of fusing the target 3D special effect model onto the body of the target object so that the target 3D special effect model matches the target object. See FIG. 4 . Fusion processing may be that the target object embraces the target 3D special effect model.
- the target three-dimensional model and the target object may be fused together to obtain a final displayed target video frame.
- the display video frame corresponding to the stop zoom-in condition can be directly jumped to the target video frame, that is, there is a case of video frame jumping.
- the special effect display function when recording a video, if the special effect display function is triggered, the image in the target scene can be captured, that is, the screen-opening panorama effect, and virtual special effects can be added at this time.
- the kitten special effect that is, the AR virtual special effect shown in Figure 2
- the enlarged display can be to enlarge the AR special effect itself, or push the AR special effect into the lens, or
- the camera focuses and advances the AR special effect, that is, focuses on the kitten special effect, and obtains the video frame shown in Figure 3.
- the target 3D special effect corresponding to the AR special effect can be determined, that is, the target 3D special effect can be fused with the target object to obtain and display the target video frame, see the effect shown in Figure 4 .
- a virtual model can be added to the collected image to be processed to obtain a display video frame, and the image area of the virtual model in the display video frame is enlarged and displayed, and detected
- the virtual model is processed into the target 3D special effect model, and the target 3D special effect model is fused with the target object in the image to be processed to obtain the final displayed video frame, which improves the richness and The video content is interesting, thereby improving the user experience.
- FIG. 5 is a schematic flowchart of an image processing method provided by another embodiment of the present disclosure.
- technical terms that are the same as or corresponding to those in the foregoing embodiments will not be repeated here.
- the method includes:
- triggering the special effect display function includes at least one of the following conditions: detecting that the captured image to be processed includes target gesture information; detecting a triggering special effect display button.
- the image to be processed including the target object can be collected in real time, and it can be determined whether the target object in the image to be processed is the target pose information.
- the target posture information is used to trigger special effect display.
- the target gesture information may be pouting, a specific gesture, etc.
- the specific gesture may be a gesture of making a fist, a gesture of facing the palm of the hand, and the like. If it is detected that the pose information of the target object in the image to be processed is the target pose information, it is considered that the special effect display function is triggered. Another way may be to display a special effect display button on the display interface. If the user triggers the special effect display button, it can be understood that a corresponding special effect needs to be added to the target object in the image to be processed.
- the posture information of the target object can be detected in real time or at intervals, and when the posture information of the detected target object is consistent with the preset posture information, or trigger the After the special effect display button, it can be considered that the target special effect needs to be added to the target object.
- the target effect may be an avatar.
- a virtual model may be added to the target object, that is, a special effect may be added.
- adding a virtual model to the collected image to be processed to obtain a display video frame includes: adding the virtual model to the position to be displayed corresponding to the target object in the image to be processed to obtain the display video frame.
- the position to be displayed may be any position of the body part of the target object.
- the position to be displayed may be on the hand, on the arm, on the shoulder, on the top of the head, etc.
- the position to be displayed may also be a position corresponding to the target posture information.
- the target posture information is gesture information
- the position to be displayed may be a finger, a palm, and the like.
- the determined virtual model can be added to the position to be displayed corresponding to the target object in the image to be processed, so as to obtain a video frame in which the virtual model is added for the target object.
- the position to be displayed is the palm
- the virtual model can be added to the palm of the target object, and then the image processing of the virtual model on the palm can be enlarged.
- the zoom-in process can be zooming in on the virtual model, or zooming in on the lens, so that the image area corresponding to the virtual model can be zoomed in, so that the user can enjoy the enlarged image of the virtual model.
- zooming in on the image area of the virtual model See Figure 3 for a schematic diagram.
- adding a virtual model to the collected image to be processed to obtain a display video frame includes: if the target object in the image to be processed is the target pose information, then in the image corresponding to the target pose information Add the virtual model to the part to obtain the display video frame.
- the target object in the image to be processed triggers the target pose information
- special effects need to be added to the target object in the image to be processed.
- the target posture information may be a specific posture, for example, a posture with arms stretched forward and palms up.
- the target gesture may be a pouting gesture or the like.
- adding the virtual model to the target object can be, the virtual model can be added to the palm, or added to the beeping mouth.
- the target pose information can be determined, and the virtual model can be added to the position to be displayed corresponding to the target pose information.
- the target posture is a pouting posture
- the virtual model may be added to the mouth; if the target posture information is a palm-up posture, the virtual model may be added to the palm.
- gradual enlargement can be understood as, if the initial image ratio is 30%, and the end image ratio is 100%, the image can be enlarged according to the ratio of 1% each time, and this kind of enlargement can be understood as gradual enlargement.
- the gradual magnification can be 30% to 50% in a single step.
- the image area displaying the virtual model in the video frame may be gradually enlarged and displayed.
- the gradual zoom-in display may be zoomed in proportionally, and may be zoomed in by 20% each time on the basis of the previous zoom-in.
- processing the virtual model as a target three-dimensional special effect model includes: if it is detected that the zoom-in duration of the virtual model reaches a preset zoom-in duration threshold, processing the virtual model The virtual model is processed as a target three-dimensional special effects model; or, if it is detected that the enlarged ratio of the virtual model reaches a preset zoom ratio threshold, the virtual model is processed as a target three-dimensional special effects model.
- the preset zoom-in duration threshold is preset, for example, the zoom-in duration threshold may be 1 second.
- the target three-dimensional special effect model is a model corresponding to the virtual model.
- the virtual model is an A special effect model, and the target three-dimensional special effect model is an AR object corresponding to the AR special effect model.
- the AR object can be a static model or a dynamic model.
- the static model may be an AR real object.
- the real object may be an object corresponding to the virtual model.
- the virtual model is an AR pet model, and the AR object may be a real pet.
- the dynamic model can be a model in which the AR pet can move, just like a real pet. It should be noted that the virtual model can be any virtual model, not limited to the pet model.
- the preset magnification ratio threshold is preset, that is, the maximum magnification ratio.
- the preset magnification ratio threshold is 300%, and if the magnification ratio of the image area reaches the preset magnification ratio threshold, the magnification may be stopped.
- the image area when zooming in on the image area of the virtual model, the image area can be zoomed in step by step, and the actual zoom-in duration can be recorded during the zoom-in process.
- the actual zoom-in time reaches the preset zoom-in time threshold, it indicates that the virtual model needs to be processed into a target three-dimensional special effect model.
- the enlarging ratio can be recorded, and when it is detected that the enlarging ratio reaches the preset enlarging ratio threshold, it indicates that the virtual model needs to be processed into a target three-dimensional special effect.
- the target display position may be any position on the pre-selected target image, for example, if the image to be processed includes a table, the ground, etc., the table and the ground may be used as the target display positions.
- the target position may also be any position preset on the body of the target object, for example, the shoulder or the bosom.
- the target three-dimensional special effect model may be placed at the preset target display position of the target object in the image to be processed, so as to obtain the target video frame, and display the target video frame. It is also possible to place the target three-dimensional special effect at any position displayed on the image to be processed. For example, if the virtual special effect is a bird special effect, the target three-dimensional special effect corresponding to the bird special effect can be displayed in the sky in the image to be processed place.
- the target object can be a user, a pet, or any other subject.
- the virtual special effect is a small cat AR special effect
- the target 3D special effect is an AR real big cat.
- the AR can be determined.
- the real big cat, the big cat has dynamic special effects, that is, it is the same as the real cat, and the paws, tail, eyes, mouth, etc. can all be moved. You can place the real-life special effects of the big cat in the arms of the target object, and at the same time, the target object is holding the big cat, and the video frame obtained at this time can be used as the display video frame, see Figure 4.
- the displaying the target video frame includes: transitioning from the video frame corresponding to the stop zoom-in condition to the target video frame through a preset animation special effect, so as to display the target video frame .
- the preset animation special effect may be a special effect inserted into a video frame, may be a special effect inserted into a transitional picture, or may be a feint special effect.
- the target video frame may be directly jumped from the corresponding video frame when the zoom-in stop condition is met, so as to display the target video frame.
- a virtual model when the special effect display function is triggered, a virtual model can be added to the target object in the collected image to be processed, and the image area corresponding to the virtual model can be enlarged and displayed.
- the virtual model When zooming in, the virtual model can be processed as the target 3D special effect model.
- the target 3D special effect model can be fused with the target object in the image to be processed to obtain the target special effect image.
- FIG. 6 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. As shown in FIG. 6 , the device includes: a display video frame determination module 310 , a 3D special effect determination module 320 and a target video frame determination module 330 .
- the display video frame determination module 310 is configured to add a virtual model to the collected image to be processed to obtain a display video frame when it is detected that the special effect display function is triggered;
- the image area of the virtual model is enlarged and displayed, and when it is detected that the zoom-in stop condition is reached, the virtual model is processed as a target three-dimensional special effect model;
- the target video frame determination module 330 is configured to set the target three-dimensional special effect model and the target three-dimensional special effect model
- the target object in the image to be processed is fused to display the target video frame.
- triggering the special effect display function includes at least one of the following conditions: detecting that the collected image to be processed includes target posture information; detecting a triggering special effect display button.
- the display video frame determining module is configured to add the virtual model to the position to be displayed corresponding to the target object in the image to be processed, so as to obtain the display video frame.
- the display video frame determination module is configured to add the virtual model to the part corresponding to the target posture information if the target object in the image to be processed is the target posture information , to obtain the displayed video frame.
- the three-dimensional special effect determination module is configured to gradually enlarge and display the image area of the virtual model in the display video frame.
- the three-dimensional special effect determining module is configured to process the virtual model as a target three-dimensional special effect model if it is detected that the enlarged time of the virtual model reaches the preset zoom-in time threshold; or, if it is detected that The enlarged ratio of the virtual model reaches a preset enlarged ratio threshold, and the virtual model is processed as a target three-dimensional special effect model.
- the target video frame determination module is set to:
- the target three-dimensional special effect model is placed at the target display position of the target object in the image to be processed, the target video frame is obtained, and the target video frame is displayed.
- the target video frame determination module is set to:
- the video frame corresponding to when the zoom-in stop condition is reached is transitioned to the target video frame through a preset animation special effect, so as to display the target video frame.
- the virtual model is an AR special effect model
- the target three-dimensional special effect model is an AR object corresponding to the AR special effect model
- the target three-dimensional special effect model is a static model or a dynamic model. kind of.
- a virtual model can be added to the collected image to be processed to obtain a display video frame, and the image area of the virtual model in the display video frame is enlarged and displayed, and detected
- the virtual model is processed into the target 3D special effect model, and the target 3D special effect model is fused with the target object in the image to be processed to obtain the final displayed video frame, which improves the richness and The video content is interesting, thereby improving the user experience.
- the image processing device provided by the embodiment of the present disclosure can execute the image processing method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.
- FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
- the terminal equipment in the embodiments of the present disclosure may include but not limited to mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers and the like.
- the electronic device shown in FIG. 7 is only an example, and should not limit the functions and application scope of the embodiments of the present disclosure.
- an electronic device 400 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 401, which may be randomly accessed according to a program stored in a read-only memory (ROM) 402 or loaded from a storage device 406. Various appropriate actions and processes are executed by programs in the memory (RAM) 403 . In the RAM 403, various programs and data necessary for the operation of the electronic device 400 are also stored.
- the processing device 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404.
- An edit/output (I/O) interface 405 is also connected to the bus 404 .
- I/O interface 405 editing devices 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration an output device 407 such as a computer; a storage device 406 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409.
- the communication means 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. While FIG. 7 shows electronic device 400 having various means, it should be understood that implementing or possessing all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.
- embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program code for executing the method shown in the flowchart.
- the computer program may be downloaded and installed from a network via communication means 409, or from storage means 406, or from ROM 402.
- the processing device 401 When the computer program is executed by the processing device 401, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.
- the electronic device provided by the embodiment of the present disclosure belongs to the same idea as the image processing method provided by the above embodiment, and the technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same benefits as the above embodiment Effect.
- An embodiment of the present disclosure provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the image processing method provided in the foregoing embodiments is implemented.
- the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
- a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
- a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
- a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
- a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device .
- Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to electrical wires, optical fiber cables, radio frequency (RF), etc., or any suitable combination of the foregoing.
- the client and the server can communicate using any currently known or future network protocols such as Hypertext Transfer Protocol (HyperText Transfer Protocol, HTTP), and can communicate with digital data in any form or medium Communications (eg, communication networks) are interconnected.
- Examples of communication networks include local area networks (LANs), wide area networks (WANs), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
- the above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
- the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device:
- a virtual model is added to the collected image to be processed to obtain a display video frame
- Computer program code for carrying out operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Included are conventional procedural programming languages - such as the "C" language or similar programming languages.
- the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
- the remote computer may be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (such as through an Internet Service Provider). Internet connection).
- LAN local area network
- WAN wide area network
- Internet Service Provider such as AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
- each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
- the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
- each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
- the units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the unit does not constitute a limitation of the unit itself under certain circumstances, for example, the first obtaining unit may also be described as "a unit for obtaining at least two Internet Protocol addresses".
- FPGAs Field Programmable Gate Arrays
- ASICs Application Specific Integrated Circuits
- ASSPs Application Specific Standard Products
- SOCs System on Chips
- CPLD Complex Programmable Logical device
- a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
- a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
- a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing.
- machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
- RAM random access memory
- ROM read only memory
- EPROM or flash memory erasable programmable read only memory
- CD-ROM compact disk read only memory
- magnetic storage or any suitable combination of the foregoing.
- Example 1 provides an image processing method, the method including:
- a virtual model is added to the collected image to be processed to obtain a display video frame
- Example 2 provides an image processing method, and the method further includes:
- triggering the special effect display function includes at least one of the following conditions:
- the collected images to be processed include target pose information
- a key that triggers a special effect display is detected.
- Example 3 provides an image processing method, and the method further includes:
- adding a virtual model to the collected image to be processed to obtain a display video frame includes:
- Example 4 provides an image processing method, and the method further includes:
- adding a virtual model to the collected image to be processed to obtain a display video frame includes:
- the target object in the image to be processed is target pose information
- Example 5 provides an image processing method, and the method further includes:
- the enlarging and displaying the image area of the virtual model in the display video frame includes:
- the image area of the virtual model in the display video frame is gradually enlarged and displayed.
- Example 6 provides an image processing method, and the method further includes:
- processing the virtual model as a target three-dimensional special effect model includes:
- the virtual model is processed as a target three-dimensional special effect model.
- Example 7 provides an image processing method, and the method further includes:
- the fusion processing of the target three-dimensional special effect model and the target object in the image to be processed to display the target video frame includes:
- the target three-dimensional special effect model is placed at the target display position of the target object in the image to be processed, the target video frame is obtained, and the target video frame is displayed.
- Example 8 provides an image processing method, and the method further includes:
- the displaying the target video frame includes:
- the video frame corresponding to when the zoom-in stop condition is reached is transitioned to the target video frame through a preset animation special effect, so as to display the target video frame.
- Example 9 provides an image processing method, and the method further includes:
- the virtual model is an AR special effect model
- the target 3D special effect model is an AR object corresponding to the AR special effect model
- the target 3D special effect model is one of a static model or a dynamic model.
- Example 10 provides an image processing device, including:
- the display video frame determination module is configured to add a virtual model to the collected image to be processed to obtain a display video frame when a triggering special effect display function is detected;
- the three-dimensional special effect determination module is configured to enlarge and display the image area of the virtual model in the display video frame, and process the virtual model as a target three-dimensional special effect model when it is detected that the zoom-in stop condition is reached;
- the target video frame determination module is configured to fuse the target three-dimensional special effect model and the target object in the image to be processed, and display the target video frame
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Claims (12)
- 一种图像处理方法,包括:响应于确定检测到触发特效显示功能,为采集的待处理图像添加虚拟模型,得到展示视频帧;将所述展示视频帧中所述虚拟模型的图像区域放大显示,并响应于确定检测到达到停止放大条件,将所述虚拟模型处理为目标三维特效模型;将所述目标三维特效模型以及所述待处理图像中的目标对象融合处理,展示目标视频帧。
- 根据权利要求1所述的方法,其中,所述触发特效显示功能包括下述条件中的至少一种:检测到采集的待处理图像中包括目标姿态信息;检测到触发特效显示按键。
- 根据权利要求1所述的方法,其中,所述为采集的待处理图像添加虚拟模型,得到展示视频帧,包括:将所述虚拟模型添加至所述待处理图像中与目标对象所对应的待展示位置处,得到所述展示视频帧。
- 根据权利要求1所述的方法,其中,所述为采集的待处理图像添加虚拟模型,得到展示视频帧,包括:响应于确定所述待处理图像中目标对象为目标姿态信息,在与所述目标姿态信息相对应的部位上添加所述虚拟模型,得到所述展示视频帧。
- 根据权利要求1所述的方法,其中,所述将所述展示视频帧中所述虚拟模型的图像区域放大显示,包括:将所述展示视频帧中所述虚拟模型的图像区域逐步放大显示。
- 根据权利要求1所述的方法,其中,所述响应于确定检测到达到停止放大条件,将所述虚拟模型处理为目标三维特效模型,包括:响应于确定检测到所述虚拟模型放大的时长达到预设放大时长阈值,将所述虚拟模型处理为目标三维特效模型;或,响应于确定检测到所述虚拟模型放大的比例达到预设放大比例阈值,将所述虚拟模型处理为目标三维特效模型。
- 根据权利要求1所述的方法,其中,所述将所述目标三维特效模型以及所述待处理图像中的目标对象融合处理,展示目标视频帧,包括:将所述目标三维特效模型放置在所述待处理图像中目标对象的目标展示位置处,得到所述目标视频帧,并展示所述目标视频帧。
- 根据权利要求7所述的方法,其中,所述展示所述目标视频帧,包括:通过预先设置的动画特效将从达到停止放大条件时所对应的视频帧过渡至所述目标视频帧,以展示所述目标视频帧。
- 根据权利要求1-8中任一所述的方法,其中,所述虚拟模型为增强现实AR特效模型, 所述目标三维特效模型为与所述AR特效模型相对应的AR实物,且所述目标三维特效模型为静态模型或动态模型中的一种。
- 一种图像处理装置,包括:展示视频帧确定模块,设置为响应于确定检测到触发特效显示功能,为采集的待处理图像添加虚拟模型,得到展示视频帧;三维特效确定模块,设置为将所述展示视频帧中所述虚拟模型的图像区域放大显示,并响应于确定检测到达到停止放大条件,将所述虚拟模型处理为目标三维特效模型;目标视频帧确定模块,设置为将所述目标三维特效模型以及所述待处理图像中的目标对象融合处理,展示目标视频帧。
- 一种电子设备,包括:一个或多个处理器;存储装置,设置为存储一个或多个程序,当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现如权利要求1-9中任一所述的图像处理方法。
- 一种包含计算机可执行指令的存储介质,所述计算机可执行指令在由计算机处理器执行时设置为执行如权利要求1-9中任一所述的图像处理方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111151627.3A CN113850746B (zh) | 2021-09-29 | 2021-09-29 | 图像处理方法、装置、电子设备及存储介质 |
CN202111151627.3 | 2021-09-29 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023051185A1 true WO2023051185A1 (zh) | 2023-04-06 |
Family
ID=78977168
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/117167 WO2023051185A1 (zh) | 2021-09-29 | 2022-09-06 | 图像处理方法、装置、电子设备及存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN113850746B (zh) |
WO (1) | WO2023051185A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2025020813A1 (zh) * | 2023-07-24 | 2025-01-30 | 北京字跳网络技术有限公司 | 一种连麦方法及装置 |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113744414B (zh) * | 2021-09-06 | 2022-06-28 | 北京百度网讯科技有限公司 | 图像处理方法、装置、设备和存储介质 |
CN113850746B (zh) * | 2021-09-29 | 2024-11-22 | 北京字跳网络技术有限公司 | 图像处理方法、装置、电子设备及存储介质 |
CN114331823A (zh) * | 2021-12-29 | 2022-04-12 | 北京字跳网络技术有限公司 | 图像处理方法、装置、电子设备及存储介质 |
CN114401443B (zh) * | 2022-01-24 | 2023-09-01 | 脸萌有限公司 | 特效视频处理方法、装置、电子设备及存储介质 |
CN116630488A (zh) * | 2022-02-10 | 2023-08-22 | 北京字跳网络技术有限公司 | 视频图像处理方法、装置、电子设备及存储介质 |
CN114677386A (zh) * | 2022-03-25 | 2022-06-28 | 北京字跳网络技术有限公司 | 特效图像处理方法、装置、电子设备及存储介质 |
CN114697703B (zh) * | 2022-04-01 | 2024-03-22 | 北京字跳网络技术有限公司 | 视频数据生成方法、装置、电子设备及存储介质 |
CN114900621B (zh) * | 2022-04-29 | 2024-08-13 | 北京字跳网络技术有限公司 | 特效视频确定方法、装置、电子设备及存储介质 |
CN114842120B (zh) * | 2022-05-19 | 2024-08-20 | 北京字跳网络技术有限公司 | 一种图像渲染处理方法、装置、设备及介质 |
CN116126182A (zh) * | 2022-09-08 | 2023-05-16 | 北京字跳网络技术有限公司 | 特效处理方法、装置、电子设备及存储介质 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104394313A (zh) * | 2014-10-27 | 2015-03-04 | 成都理想境界科技有限公司 | 特效视频生成方法及装置 |
CN109727303A (zh) * | 2018-12-29 | 2019-05-07 | 广州华多网络科技有限公司 | 视频展示方法、系统、计算机设备、存储介质和终端 |
CN109803165A (zh) * | 2019-02-01 | 2019-05-24 | 北京达佳互联信息技术有限公司 | 视频处理的方法、装置、终端及存储介质 |
CN112333491A (zh) * | 2020-09-23 | 2021-02-05 | 字节跳动有限公司 | 视频处理方法、显示装置和存储介质 |
CN112544070A (zh) * | 2020-03-02 | 2021-03-23 | 深圳市大疆创新科技有限公司 | 视频的处理方法和装置 |
US20210118236A1 (en) * | 2019-10-15 | 2021-04-22 | Beijing Sensetime Technology Development Co., Ltd. | Method and apparatus for presenting augmented reality data, device and storage medium |
CN113850746A (zh) * | 2021-09-29 | 2021-12-28 | 北京字跳网络技术有限公司 | 图像处理方法、装置、电子设备及存储介质 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101021952A (zh) * | 2007-03-23 | 2007-08-22 | 北京中星微电子有限公司 | 一种实现三维视频特效的方法及装置 |
US9754416B2 (en) * | 2014-12-23 | 2017-09-05 | Intel Corporation | Systems and methods for contextually augmented video creation and sharing |
CN112188074B (zh) * | 2019-07-01 | 2022-08-05 | 北京小米移动软件有限公司 | 图像处理方法及装置、电子设备、可读存储介质 |
CN111225231B (zh) * | 2020-02-25 | 2022-11-22 | 广州方硅信息技术有限公司 | 虚拟礼物的显示方法、装置、设备及存储介质 |
CN111565332A (zh) * | 2020-04-27 | 2020-08-21 | 北京字节跳动网络技术有限公司 | 视频传输方法、电子设备和计算机可读介质 |
CN111526412A (zh) * | 2020-04-30 | 2020-08-11 | 广州华多网络科技有限公司 | 全景直播方法、装置、设备及存储介质 |
CN111935491B (zh) * | 2020-06-28 | 2023-04-07 | 百度在线网络技术(北京)有限公司 | 直播的特效处理方法、装置以及服务器 |
CN111880709A (zh) * | 2020-07-31 | 2020-11-03 | 北京市商汤科技开发有限公司 | 一种展示方法、装置、计算机设备及存储介质 |
CN112732152B (zh) * | 2021-01-27 | 2022-05-24 | 腾讯科技(深圳)有限公司 | 直播处理方法、装置、电子设备及存储介质 |
CN113453034B (zh) * | 2021-06-29 | 2023-07-25 | 上海商汤智能科技有限公司 | 数据展示方法、装置、电子设备以及计算机可读存储介质 |
-
2021
- 2021-09-29 CN CN202111151627.3A patent/CN113850746B/zh active Active
-
2022
- 2022-09-06 WO PCT/CN2022/117167 patent/WO2023051185A1/zh active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104394313A (zh) * | 2014-10-27 | 2015-03-04 | 成都理想境界科技有限公司 | 特效视频生成方法及装置 |
CN109727303A (zh) * | 2018-12-29 | 2019-05-07 | 广州华多网络科技有限公司 | 视频展示方法、系统、计算机设备、存储介质和终端 |
CN109803165A (zh) * | 2019-02-01 | 2019-05-24 | 北京达佳互联信息技术有限公司 | 视频处理的方法、装置、终端及存储介质 |
US20210118236A1 (en) * | 2019-10-15 | 2021-04-22 | Beijing Sensetime Technology Development Co., Ltd. | Method and apparatus for presenting augmented reality data, device and storage medium |
CN112544070A (zh) * | 2020-03-02 | 2021-03-23 | 深圳市大疆创新科技有限公司 | 视频的处理方法和装置 |
CN112333491A (zh) * | 2020-09-23 | 2021-02-05 | 字节跳动有限公司 | 视频处理方法、显示装置和存储介质 |
CN113850746A (zh) * | 2021-09-29 | 2021-12-28 | 北京字跳网络技术有限公司 | 图像处理方法、装置、电子设备及存储介质 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2025020813A1 (zh) * | 2023-07-24 | 2025-01-30 | 北京字跳网络技术有限公司 | 一种连麦方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN113850746B (zh) | 2024-11-22 |
CN113850746A (zh) | 2021-12-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2023051185A1 (zh) | 图像处理方法、装置、电子设备及存储介质 | |
WO2022171024A1 (zh) | 图像显示方法、装置、设备及介质 | |
CN113076048B (zh) | 视频的展示方法、装置、电子设备和存储介质 | |
CN110070496B (zh) | 图像特效的生成方法、装置和硬件装置 | |
WO2022100735A1 (zh) | 视频处理方法、装置、电子设备及存储介质 | |
WO2023284708A1 (zh) | 一种视频处理方法、装置、电子设备和存储介质 | |
WO2022188305A1 (zh) | 信息展示方法及装置、电子设备、存储介质及计算机程序 | |
CN112035046B (zh) | 榜单信息显示方法、装置、电子设备及存储介质 | |
CN114245028B (zh) | 图像展示方法、装置、电子设备及存储介质 | |
US20220159197A1 (en) | Image special effect processing method and apparatus, and electronic device and computer readable storage medium | |
US12019669B2 (en) | Method, apparatus, device, readable storage medium and product for media content processing | |
CN115002359B (zh) | 视频处理方法、装置、电子设备及存储介质 | |
CN114598815B (zh) | 一种拍摄方法、装置、电子设备和存储介质 | |
WO2022037484A1 (zh) | 图像处理方法、装置、设备及存储介质 | |
WO2023040749A1 (zh) | 图像处理方法、装置、电子设备及存储介质 | |
WO2023169305A1 (zh) | 特效视频生成方法、装置、电子设备及存储介质 | |
WO2024165010A1 (zh) | 信息生成方法、信息显示方法、装置、设备和存储介质 | |
US20220272283A1 (en) | Image special effect processing method, apparatus, and electronic device, and computer-readable storage medium | |
CN114697568B (zh) | 特效视频确定方法、装置、电子设备及存储介质 | |
WO2021089002A1 (zh) | 多媒体信息处理方法、装置、电子设备及介质 | |
CN115278107A (zh) | 视频处理方法、装置、电子设备及存储介质 | |
WO2021027632A1 (zh) | 图像特效处理方法、装置、电子设备和计算机可读存储介质 | |
WO2023143240A1 (zh) | 图像处理方法、装置、设备、存储介质和程序产品 | |
WO2021073204A1 (zh) | 对象的显示方法、装置、电子设备及计算机可读存储介质 | |
GB2600341A (en) | Image special effect processing method and apparatus, electronic device and computer-readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22874576 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18695813 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 08/07/2024) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 22874576 Country of ref document: EP Kind code of ref document: A1 |