[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2022007565A1 - 增强现实的图像处理方法、装置、电子设备及存储介质 - Google Patents

增强现实的图像处理方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2022007565A1
WO2022007565A1 PCT/CN2021/098456 CN2021098456W WO2022007565A1 WO 2022007565 A1 WO2022007565 A1 WO 2022007565A1 CN 2021098456 W CN2021098456 W CN 2021098456W WO 2022007565 A1 WO2022007565 A1 WO 2022007565A1
Authority
WO
WIPO (PCT)
Prior art keywords
augmented reality
target
audio data
model
reality model
Prior art date
Application number
PCT/CN2021/098456
Other languages
English (en)
French (fr)
Inventor
李云珠
张璟聪
宋旭晨
杨建朝
王国晖
陈志立
罗琳捷
杨骁�
李昊泽
刘晶
Original Assignee
北京字节跳动网络技术有限公司
字节跳动有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司, 字节跳动有限公司 filed Critical 北京字节跳动网络技术有限公司
Priority to EP21837648.1A priority Critical patent/EP4167192A4/en
Priority to JP2023501106A priority patent/JP2023533295A/ja
Publication of WO2022007565A1 publication Critical patent/WO2022007565A1/zh
Priority to US18/053,476 priority patent/US11756276B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/205—3D [Three Dimensional] animation driven by audio data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2012—Colour editing, changing, or manipulating; Use of colour codes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2021—Shape modification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • Embodiments of the present invention relate to virtual reality technology, and in particular, to an augmented reality image processing method, device, electronic device, and storage medium.
  • Augmented Reality is a technology in which real information and virtual information are superimposed.
  • the computer system processes real information, and generates virtual information that matches and contains virtual objects, sounds or texts according to the real information; then, the virtual information is superimposed on the human-computer interaction interface that displays real information, thereby enhancing the user's understanding of the real world. perception.
  • augmented reality models can only be displayed in a preset fixed manner, which is single, lacks interaction, and is not easy to use.
  • the present invention provides an augmented reality image processing method, device, electronic device and storage medium, so as to improve the interactivity of the augmented reality model and the ease of use.
  • an embodiment of the present invention provides an augmented reality image processing method, including:
  • the augmented reality model is driven according to the playback progress and audio features of the target audio data.
  • an embodiment of the present invention further provides an augmented reality image processing device, including:
  • the target image acquisition module is used to acquire the target image in response to the image acquisition instruction triggered by the user, and the target image contains the target object;
  • the augmented reality model acquisition module is used to acquire the augmented reality model of the target object, and output the augmented reality model in combination with the target object;
  • a target audio acquisition module used to acquire the target audio data selected by the user
  • an audio feature determination module used for determining audio features with timing according to the target audio data
  • the output module is used to drive the augmented reality model according to the playback progress and audio features of the target audio data when the target audio data is output.
  • an embodiment of the present invention further provides an electronic device, where the electronic device includes:
  • processors one or more processors
  • the one or more processors When one or more programs are executed by one or more processors, the one or more processors implement the augmented reality image processing method as shown in the embodiments of the present disclosure.
  • embodiments of the present disclosure further provide a storage medium containing computer-executable instructions, and the computer-executable instructions, when executed by a computer processor, are used to execute the augmented reality image processing method shown in the embodiments of the present disclosure .
  • an embodiment of the present invention further provides a computer program product, where the computer program product includes: a computer program, where the computer program is stored in a readable storage medium, and one or more processors of the electronic device may The readable storage medium reads the computer program, and the one or more processors execute the computer program, so that the electronic device executes the augmented reality image processing method as shown in the embodiments of the present disclosure.
  • an embodiment of the present disclosure further provides a computer program, where the computer program is stored in a readable storage medium, and one or more processors of a device can read the computer from the readable storage medium A program, wherein the one or more processors execute the computer program, so that the electronic device executes the augmented reality image processing method as shown in the embodiments of the present disclosure.
  • the augmented reality image processing solution disclosed in the embodiments of the present disclosure can respond to an image acquisition instruction triggered by a user to acquire a target image, where the target image contains a target object; acquire an augmented reality model of the target object, and output the augmented reality model in combination with the target object; For the target audio data selected by the user, the time-series audio features are determined according to the target audio data; when outputting the target audio data, the augmented reality model is driven according to the playback progress and audio features of the target audio data. Compared with the current augmented reality model, it lacks interactivity and has poor ease of use.
  • the augmented reality image processing solution disclosed in the embodiments of the present disclosure can drive the output of the augmented display model in combination with the audio features of the target audio data selected by the user when the augmented reality model is output, so that the user can participate in the display process of the augmented display model , by selecting different target audio data, the augmented reality model is driven to display according to the audio features of the target audio data, which improves the usability.
  • Embodiment 1 is a flowchart of an image processing method for augmented reality in Embodiment 1 of the present invention
  • FIG. 2 is a schematic structural diagram of an image processing apparatus for augmented reality in Embodiment 2 of the present invention
  • FIG. 3 is a schematic structural diagram of an electronic device in Embodiment 3 of the present invention.
  • Embodiment 1 is a flowchart of an augmented reality image processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of displaying an augmented reality model.
  • the method can be executed by an electronic device that implements augmented reality, and the electronic device can For smartphones, tablet computers, etc., the specific steps include the following:
  • Step 110 in response to an image acquisition instruction triggered by the user, acquire a target image, where the target image includes a target object.
  • the user can issue an image acquisition instruction in the preset application.
  • the electronic device acquires the target image through the camera.
  • the user can open the preset application under the premise of the intention to use the augmented reality model.
  • the target object can be an object with an augmented reality model such as a landmark building.
  • Landmark buildings can be buildings with a unique design in an area.
  • the preset application may be a photographing application of an electronic device, or an application having an augmented reality function.
  • the camera of the electronic device acquires the target image, and the electronic device displays the acquired target image on the preview page.
  • the preview page can provide users with real-time images obtained by the camera.
  • Step 120 Acquire an augmented reality model of the target object, and output the augmented reality model in combination with the target object.
  • the augmented reality model is mapped to the target object in the target image, and the augmented reality model is output in combination with the target object.
  • an interface for manually adjusting the size of the augmented reality model can be provided for the user. If the combination of machines is not effective, such as the augmented reality model cannot be accurately combined with the target object, the size of the augmented reality model can be adjusted through this interface, so that the user can adjust the size of the augmented reality model. Adjust the size of augmented reality models to improve ease of use.
  • obtaining an augmented reality model of the target object can be implemented in the following manner:
  • the target object identifier is determined according to the current position information of the electronic device and the shooting angle, and the augmented reality model represented by the target object identifier is determined as the augmented reality model of the target object.
  • GPS Global Positioning System
  • the orientation of the electronic device is obtained through the gyroscope, and the orientation is used as the shooting angle. According to the positioning information and the shooting angle, it is determined whether there is a landmark building within a certain shooting range. If there is a landmark building, the augmented reality model of the landmark building is used as the augmented reality model of the target object.
  • acquiring the augmented reality model of the target object may be implemented in the following manner:
  • the basic image of the target object is determined from the target image, and the augmented reality model of the target object is determined according to the basic image.
  • the base image may be an image of the target object in the target image.
  • the basic image may be analyzed by the network-side server, so as to determine what the target object in the basic image is.
  • the target objects can be objects such as buildings, vehicles, clothing, shoes and hats.
  • the above two implementation manners can be used to find the augmented reality model of the target object, and can also be used to check whether the augmented reality model of the target object is accurate.
  • Step 130 Acquire the target audio data selected by the user, and determine audio features with time sequence according to the target audio data.
  • the target audio data may be audio data such as songs or recordings selected by the user.
  • the above audio data may be locally stored audio data. It can also be audio data selected by the user in the playlist provided by the server.
  • the target audio data may also be audio data input by the user in real time.
  • audio feature detection is performed on the target audio data to obtain time-series audio features of the target audio data, where the audio features include one or a combination of accents, beats or beats.
  • Audio feature detection is used to analyze beat patterns, accents, and rebeat locations or patterns in target audio data.
  • a beat is a unit representing the tempo of target audio data.
  • a series of beats with a certain strength and weakness are repeated at regular intervals. Such as 2/4 beats (four or two beats), 4/4 beats (four or four beats), 3/4 beats (four or three beats), etc.
  • the beat changes periodically over time. Accents are the louder notes in the target audio.
  • Rebeat refers to the strong beat in the beat.
  • Step 140 When outputting the target audio data, drive the augmented reality model according to the playback progress and audio characteristics of the target audio data.
  • the target audio data is output through the audio output module of the electronic device. While outputting the target audio data, the augmented reality model is driven according to the current playback progress and the target audio features.
  • the overall color of the augmented reality model may be driven to change.
  • the color change is different for heavy beats, taps, or accents.
  • the augmented reality model includes a plurality of model units.
  • Driving the augmented reality model according to the playback progress and audio features of the target audio data includes: driving model units in the augmented reality model according to the playback progress and audio features of the target audio data.
  • the augmented reality model can be composed of multiple model units, and each model unit can be a cube unit. Splicing multiple cube elements to form an augmented reality model of the target object. Multiple cubes can be processed in parallel by shaders. According to the playback progress and audio characteristics of the target audio data, the model unit in the augmented reality model is driven.
  • drive model units in the augmented reality model according to the playback progress and audio features of the target audio data including:
  • Step 1 Determine the target time and target amplitude for the morphological change of the model unit according to the audio feature with time sequence.
  • the time at which the tap, the overbeat and the stress are located in the beat can be determined as the target time, and the target amplitude is determined according to the degree of the tap, the overshot and the stress.
  • Step 2 If the playback progress of the target audio data is the target time, drive the model unit in the augmented reality model according to the target amplitude.
  • the driving mode of the model unit can include convex motion, color change or transparency change, etc.
  • model units in the augmented reality model are driven according to the target amplitude, including:
  • the preset model units in the augmented reality model are driven to perform a bulging action; or, multiple model units in the augmented reality model are driven to perform color changes; or, multiple model units in the augmented reality model are driven to perform transparency changes.
  • the preset model When driving a preset model unit in the augmented reality model to perform a protruding action, the preset model may be a randomly selected model unit. Determine the bulge amplitude of the bulge action according to the target amplitude.
  • multiple model units When driving multiple model units in the augmented reality model to perform color change, multiple model units may be randomly selected from all model units to perform color change or to perform color change on all model units.
  • Transparency refers to the transparency of the model element texture graphics.
  • the texture image can be a solid color or the actual texture pattern of the target object.
  • the method further includes:
  • the emotional feature of the target audio data determines the first deformation feature of the augmented reality model according to the emotional feature, and the first deformation feature is used to drive the shape of the augmented reality model to be consistent with the emotion expressed by the emotional feature; when outputting the target audio data, according to the first deformation feature Deformable features drive augmented reality models.
  • the emotional feature may be determined from the song title of the target audio data. Or, the emotional feature is determined according to the text content entered by the user in the target audio.
  • the emotional feature is used to represent the emotional bias of the audio content of the target audio data, such as cheerfulness, melancholy, etc.
  • Deformation features corresponding to different emotional features can be preconfigured. For example, in the first deformation feature corresponding to the cheerful emotion feature, the bulge amplitude of the model unit is higher, and the bulge frequency is faster. For example, in the first deformation feature corresponding to the melancholic emotion feature, the bulge amplitude of the model unit is lower, and the bulge frequency is slower.
  • the emotional feature is sadness
  • the TV tower model is in a bent shape. If the building model is deformed, delete the original image of the building by cropping, and use the model to cover the original image area.
  • the method further includes:
  • the user's limb movements can be obtained through limb detection.
  • the user's expression can be obtained through face recognition.
  • a second deformation feature is generated based on body movements or expressions. Exemplarily, after the TV tower is photographed, if the facial expression of the person is sad, the second deformation feature is a stooped shape, and the TV tower model is in a stooped shape at this time. If the building model is deformed, delete the original image of the building by cropping, and use the model to cover the original image area.
  • the augmented reality image processing method disclosed in the embodiments of the present disclosure can respond to an image acquisition instruction triggered by a user to acquire a target image, where the target image contains a target object; acquire an augmented reality model of the target object, and output the augmented reality model in combination with the target object; For the target audio data selected by the user, the time-series audio features are determined according to the target audio data; when outputting the target audio data, the augmented reality model is driven according to the playback progress and audio features of the target audio data. Compared with the current augmented reality model, it lacks interactivity and has poor ease of use.
  • the augmented reality image processing method disclosed in the embodiment of the present disclosure can drive the output of the augmented display model in combination with the audio features of the target audio data selected by the user when the augmented reality model is output, so that the user can participate in the display process of the augmented display model , by selecting different target audio data, the augmented reality model is driven to display according to the audio features of the target audio data, which improves the usability.
  • FIG. 2 is a schematic structural diagram of an image processing apparatus for augmented reality provided by Embodiment 2 of the present disclosure. This embodiment is applicable to a situation in which an augmented reality model is displayed.
  • the method may be performed by an electronic device that implements augmented reality.
  • the electronic device may For smart phones, tablet computers, etc., the device includes: a target image acquisition module 210 , an augmented reality model acquisition module 220 , a target audio acquisition module 230 , an audio feature determination module 240 or an output module 250 .
  • a target image acquisition module 210 configured to acquire a target image in response to an image acquisition instruction triggered by a user, and the target image includes a target object;
  • the augmented reality model acquisition module 220 is used to acquire the augmented reality model of the target object, and output the augmented reality model in combination with the target object;
  • the target audio acquisition module 230 is used to acquire the target audio data selected by the user;
  • An audio feature determining module 240 configured to determine audio features with timing according to the target audio data
  • the output module 250 is configured to drive the augmented reality model according to the playback progress and audio characteristics of the target audio data when outputting the target audio data.
  • the audio feature determination module 240 is used for:
  • the augmented reality model includes a plurality of model units; the output module 250 is used for:
  • the model unit in the augmented reality model is driven.
  • the output module 250 is used for:
  • the model unit in the augmented reality model is driven according to the target amplitude.
  • the output module 250 is used for:
  • the first deformation feature acquisition module is used for:
  • the augmented reality model is driven according to the first deformation feature.
  • the second deformation feature acquisition module is used for:
  • the augmented reality model is driven according to the second deformation feature.
  • the target image acquisition module 210 acquires a target image in response to an image acquisition instruction triggered by a user, and the target image contains a target object;
  • the augmented reality model acquisition module 220 acquires an augmented reality model of the target object , output the augmented reality model in combination with the target object;
  • the target audio acquisition module 230 acquires the target audio data selected by the user, and the audio feature determination module 240 determines the audio features with timing according to the target audio data; when the output module 250 outputs the target audio data, according to the target audio data
  • the playback progress of the audio data and the audio features drive the augmented reality model.
  • it lacks interactivity and has poor ease of use.
  • the augmented reality image processing apparatus disclosed in the embodiments of the present disclosure can drive the output of the augmented display model in combination with the audio features of the target audio data selected by the user when outputting the augmented reality model, so that the user can participate in the display process of the augmented display model , by selecting different target audio data, the augmented reality model is driven to display according to the audio features of the target audio data, which improves the usability.
  • the augmented reality image processing apparatus provided by the embodiment of the present invention can execute the augmented reality image processing method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
  • Terminal devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, Personal digital assistants (PDAs, personal digital assistants), Portable android devices (PADs, tablet computers), Portable media players ( Mobile terminals such as PMP, Portable Multimedia Player), in-vehicle terminals (eg, in-vehicle navigation terminals), etc., and stationary terminals such as digital TVs (Television), desktop computers, and the like.
  • PDAs Personal digital assistants
  • PDAs personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Personal digital assistants
  • PDAs Portable android devices
  • the electronic device 800 may include a processing device (eg, a central processing unit, a graphics processor, etc.) 801 , which may be stored in a read only memory (ROM) 802 or from a storage device 808 according to a program stored in a read only memory (ROM) 802
  • ROM read only memory
  • RAM random access memory
  • the processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804.
  • Input/output (I/O) interface 805 is also connected to bus 804 .
  • the following devices can be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD) Output device 807 , speaker, vibrator, etc.; storage device 808 including, eg, magnetic tape, hard disk, etc.; and communication device 809 .
  • Communication means 809 may allow electronic device 800 to communicate wirelessly or by wire with other devices to exchange data. While FIG. 3 shows electronic device 800 having various means, it should be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods illustrated in the flowcharts.
  • the computer program may be downloaded and installed from the network via the communication device 809, or from the storage device 808, or from the ROM 802.
  • the steps in the method of the embodiment of the present disclosure are executed to realize the above-mentioned functions defined by the computer program.
  • a computer program product includes: a computer program, the computer program is stored in a readable storage medium, and one or more processors of an electronic device The computer program is read from the readable storage medium, and the one or more processors execute the computer program, so that the electronic device executes the solution provided by any of the foregoing embodiments.
  • One or more embodiments of the present disclosure provide a computer program, where the computer program is stored in a readable storage medium, and one or more processors of an electronic device can read the computer from the readable storage medium A program, wherein the one or more processors execute the computer program, so that the electronic device executes the solution provided by any of the foregoing embodiments.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable of the above combination.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
  • the program code embodied on the computer readable medium may be transmitted by any suitable medium, including but not limited to: electric wire, optical fiber cable, radio frequency (RF), etc., or any suitable combination of the above.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or may exist alone without being assembled into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: acquires at least two Internet Protocol addresses; A node evaluation request for an Internet Protocol address, wherein the node evaluation device selects an Internet Protocol address from the at least two Internet Protocol addresses and returns it; receives the Internet Protocol address returned by the node evaluation device; wherein the obtained The Internet Protocol address indicates an edge node in the content distribution network.
  • the above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: receives a node evaluation request including at least two Internet Protocol addresses; From the at least two Internet Protocol addresses, the Internet Protocol address is selected; the selected Internet Protocol address is returned; wherein, the received Internet Protocol address indicates an edge node in the content distribution network.
  • Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional Procedural programming language - such as the "C" language or similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network—including a Local Area Network (LAN) or Wide Area Network (WAN)—or, can be connected to an external computer ( For example, using an Internet service provider to connect via the Internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure may be implemented in a software manner, and may also be implemented in a hardware manner.
  • the name of the unit does not constitute a limitation of the unit itself under certain circumstances, for example, the first obtaining unit may also be described as "a unit that obtains at least two Internet Protocol addresses".

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Architecture (AREA)
  • Social Psychology (AREA)
  • Child & Adolescent Psychology (AREA)
  • Hospice & Palliative Care (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种增强现实的图像处理方法、装置、电子设备及存储介质。包括：响应于用户触发的图像获取指令，获取目标图像，目标图像包含目标物体；获取目标物体的增强现实模型，结合目标物体输出增强现实模型；获取用户选择的目标音频数据，根据目标音频数据确定具有时序性的音频特征；输出目标音频数据时，根据目标音频数据的播放进度和音频特征驱动增强现实模型。能够在输出增强现实模型时，结合用户选择的目标音频数据的音频特征，驱动增强显示模型的输出，使得用户能够参与到增强显示模型的展示过程中，通过选择不同的目标音频数据，驱动增强现实模型根据目标音频数据的音频特征进行展示，提高易用性。

Description

增强现实的图像处理方法、装置、电子设备及存储介质
相关申请交叉引用
本申请要求于2020年7月10日提交中国专利局、申请号为202010662819.X、发明名称为“增强现实的图像处理方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用并入本文。
技术领域
本发明实施例涉及虚拟现实技术，尤其涉及一种增强现实的图像处理方法、装置、电子设备及存储介质。
背景技术
增强现实(Augmented Reality，简称AR)是一种现实信息与虚拟信息相叠加的技术。首先，计算机系统处理现实信息，根据现实信息生成匹配且包含虚拟物体、声音或文字等形式的虚拟信息；然后，将虚拟信息叠加到显示现实信息的人机交互界面中，从而增强用户对现实世界的感知。
目前，增强现实模型只能按照预先设置的固定方式展示，展示方式单一，缺乏互动，易用性差。
发明内容
本发明提供一种增强现实的图像处理方法、装置、电子设备及存储介质，以实现提高增强现实模型的互动性，提高易用性。
第一方面，本发明实施例提供了一种增强现实的图像处理方法，包括：
响应于用户触发的图像获取指令，获取目标图像，目标图像包含目标物体；
获取目标物体的增强现实模型，结合目标物体输出增强现实模型；
获取用户选择的目标音频数据，根据目标音频数据确定具有时序性的音频特征；
输出目标音频数据时，根据目标音频数据的播放进度和音频特征驱动增强现实模型。
第二方面，本发明实施例还提供了一种增强现实的图像处理装置，包括：
目标图像获取模块，用于响应于用户触发的图像获取指令，获取目标图像，目标图像包含目标物体；
增强现实模型获取模块，用于获取目标物体的增强现实模型，结合目标物体输出增强现实模型；
目标音频获取模块，用于获取用户选择的目标音频数据；
音频特征确定模块，用于根据目标音频数据确定具有时序性的音频特征；
输出模块，用于输出目标音频数据时，根据目标音频数据的播放进度和音频特征驱动增强现实模型。
第三方面，本发明实施例还提供了一种电子设备，电子设备包括：
一个或多个处理器；
存储装置，用于存储一个或多个程序，
当一个或多个程序被一个或多个处理器执行，使得一个或多个处理器实现如本公开实施例所示的增强现实的图像处理方法。
第四方面，本发明实施例还提供了一种包含计算机可执行指令的存储介质，计算机可执行指令在由计算机处理器执行时用于执行如本公开实施例所示的增强现实的图像处理方法。
第五方面，本发明实施例还提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序，所述计算机程序存储在可读存储介质中，电子设备的一个或多个处理器可以从所述可读存储介质读取所述计算机程序，所述一个或多个处理器执行所述计算机程序，使得所述电子设备执行如本公开实施例所示的增强现实的图像处理方法。
第六方面，本公开实施例还提供了一种计算机程序，所述计算机程序存储在可读存储介质中，设备的一个或多个处理器可以从所述可读存储介质中读取所述计算机程序，所述一个或多个处理器执行所述计算机程序，使得所述电子设备执行如本公开实施例所示的增强现实的图像处理方法。
本公开实施例公开的增强现实的图像处理方案，能够响应于用户触发的图像获取指令，获取目标图像，目标图像包含目标物体；获取目标物体的增强现实模型，结合目标物体输出增强现实模型；获取用户选择的目标音频数据，根据目标音频数据确定具有时序性的音频特征；输出目标音频数据时，根据目标音频数据的播放进度和音频特征驱动增强现实模型。相对于目前增强现实模型缺乏互动性，易用性差。本公开实施例公开的增强现实的图像处理方案，能够在输出增强现实模型时，结合用户选择的目标音频数据的音频特征，驱动增强显示模型的输出，使得用户能够参与到增强显示模型的展示过程中，通过选择不同的目标音频数据，驱动增强现实模型根据目标音频数据的音频特征进行展示，提高易用性。
附图说明
图1是本发明实施例一中的增强现实的图像处理方法的流程图；
图2是本发明实施例二中的增强现实的图像处理装置的结构示意图；
图3是本发明实施例三中的电子设备结构示意图。
具体实施方式
下面结合附图和实施例对本发明作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释本发明，而非对本发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本发明相关的部分而非全部结构。
实施例一
图1为本发明实施例一提供的增强现实的图像处理方法的流程图，本实施例可适用于展示增强现实模型的情况，该方法可以由实现增强现实的电子设备来执行，该电子设备可以为智能手机、平板电脑等，具体包括如下步骤：
步骤110、响应于用户触发的图像获取指令，获取目标图像，目标图像包含目标物体。
用户在预设应用中可以出发图像获取指令，当用户出发图像获取指令时，电子设备通过摄像头获取目标图像。用户可以在使用增强现实模型的意向的前提下，开启预设应用。使用电子设备拍摄目标物体的画面。目标物体可以为地标建筑等具备增强现实模型的物体。地标建筑物可以为某个区域中具有独特设计风格的建筑物。
预设应用可以为电子设备的拍照应用，或者具有增强现实功能的应用。电子设备的摄像头获取目标图像，电子设备将获取到的目标图像显示在预览页面中。预览页面可以为用户提供摄像头获取到的实时图像。
步骤120、获取目标物体的增强现实模型，结合目标物体输出增强现实模型。
若识别到当前图像中的拍摄物存在相应的增强现实模型，则将增强现实模型映射到目标图像中的目标物体上，实现结合目标物体输出增强现实模型。
进一步的，可以为用户提供人工调整增强现实模型大小的接口，若机器结合效果不佳，如增强现实模型与目标物体无法准确结合，则可以通过该接口调整增强现实模型的大小，进而实现由用户调节增强现实模型大小，提高易用性。
进一步的，在一种实现方式中，当目标物体为地标建筑物时，获取目标物体的增强现实模型，可以通过下述方式实施：
当监测到图标图像中包含建筑物时，根据电子设备当前位置信息和拍摄角度确定目标 物体标识，将目标物体标识表示的增强现实模型确定为目标物体的增强现实模型。
获取电子设备的全球定位系统(Global Positioning System，GPS)定位信息。通过陀螺仪获取电子设备的朝向，将该朝向作为拍摄角度。根据定位信息以及拍摄角度，确定在一定拍照范围内是否具有地标建筑物。若具有地标建筑物则将该地标建筑物的增强现实模型作为目标物体的增强现实模型。
进一步的，在另一种实现方式中，获取目标物体的增强现实模型，可以通过下述方式实施：
从目标图像中确定目标物体的基础图像，根据该基础图像确定目标物体的增强现实模型。基础图像可以为目标图像中目标物体的图像。可以通过网络侧服务器对基础图像进行图像分析，进而确定该基础图像中的目标物体具体为何物。目标物体可以为建筑物、车辆、服装、鞋帽等物体。
上述两种实现方式可以用于查找目标物体的增强现实模型，也可以用于检验目标物体的增强现实模型是否准确。
步骤130、获取用户选择的目标音频数据，根据目标音频数据确定具有时序性的音频特征。
目标音频数据可以为用户选择的歌曲或录音等音频数据。上述音频数据可以为本地存储的音频数据。也可以为用户在服务器提供的歌单中选择的音频数据。目标音频数据还可以为用户实时输入的音频数据。
可选的，对目标音频数据进行音频特征检测，得到目标音频数据的具有时序性的音频特征，音频特征包括重音、重拍或节拍中的一种或多种的组合。
音频特征检测用于解析目标音频数据中节拍规律、重音、重拍出现的位置或规律等。节拍为表示目标音频数据节奏的单位。目标音频数据中，具有一定强弱分别的一系列拍子在每隔一定时间重复出现。如2/4拍(四二拍)、4/4拍(四四拍)、3/4拍(四三拍)等。节拍随时间呈周期性变化。重音为目标音频中音量较高的音符。重拍指节拍中的强拍。
步骤140、输出目标音频数据时，根据目标音频数据的播放进度和音频特征驱动增强现实模型。
当用户选择目标音频数据后，通过电子设备的音频输出模组输出目标音频数据。在输出目标音频数据的同时，根据当前的播放进度以及目标音频特征驱动增强现实模型。
示例性的，随着重音、轻拍或重拍的出现，可以驱动增强现实模型整体的颜色进行改变。重拍、轻拍或重音对应的颜色变化不同。
可选的，增强现实模型包括多个模型单元。根据目标音频数据的播放进度和音频特征驱动增强现实模型，包括：根据目标音频数据的播放进度和音频特征，驱动增强现实模型中的模型单元。
增强现实模型可以由多个模型单元组成，每个模型单元可以为一个立方体单元。拼接多个立方体单元形成目标物体的增强现实模型。可以通过着色器并行处理多个立方体。根据目标音频数据的播放进度和音频特征，驱动增强现实模型中的模型单元。
可选的，根据目标音频数据的播放进度和音频特征，驱动增强现实模型中的模型单元，包括：
步骤一、根据具有时序性的音频特征确定模型单元进行形态变化的目标时间和目标幅度。
可以将节拍中的轻拍、重拍和重音所在的时间确定为目标时间，根据轻拍、重拍和重音的程度确定目标幅度。
步骤二、若目标音频数据的播放进度为目标时间，则根据目标幅度驱动增强现实模型中的模型单元。
模型单元的驱动方式可以包括凸起动作、颜色变化或透明度变化等。
示例性的，根据目标幅度驱动增强现实模型中的模型单元，包括：
驱动增强现实模型中的预设模型单元进行凸起动作；或者，驱动增强现实模型中多个模型单元执行颜色变化；或者，驱动增强现实模型中多个模型单元执行透明度变化。
驱动增强现实模型中的预设模型单元进行凸起动作时，预设模型可以为随机选择的模型单元。根据目标幅度确定凸起动作的凸起幅度。驱动增强现实模型中多个模型单元执行颜色变化时，可以从全部模型单元中随机选出多个模型单元执行颜色变化或对全部模型单元执行颜色变化。透明度指模型单元纹理图形的透明度。纹理图像可以为纯色也可以为目标物体实际纹理图案。
进一步的，在获取用户选择的目标音频数据之后，还包括：
获取目标音频数据的情绪特征；根据情绪特征确定增强现实模型的第一形变特征，第一形变特征用于驱动增强现实模型的形状与情绪特征表达的情绪一致；输出目标音频数据时，根据第一形变特征驱动增强现实模型。
可以根据目标音频数据的歌曲名称确定情绪特征。或则根据目标音频中用户录入的文字内容确定情绪特征。情绪特征用于表示目标音频数据的音频内容的情绪偏向，例如欢快、忧郁等。可预先配置不同情绪特征对应的形变特征。例如欢快的情绪特征对应的第一形变 特征中模型单元的凸起幅度较高，凸起频率较快。例如忧郁的情绪特征对应的第一形变特征中模型单元的凸起幅度较低，凸起频率较慢。示例性的，在拍摄电视塔后，如若情绪特征为伤心，则电视塔模型呈弯腰形态。若建筑物模型出现形变，则通过裁剪方式删除建筑物原始图像，使用模型覆盖原始图像区域。
进一步的，在获取用户选择的目标音频数据之后，还包括：
获取用户的肢体动作或表情；根据肢体动作或表情确定增强现实模型的第二形变特征，第二形变特征用于驱动增强现实模型的形状与肢体动作或表情一致；输出目标音频数据时，根据第二形变特征驱动增强现实模型。
通过肢体检测可获取用户肢体动作。通过人脸识别可获取用户的表情。根据肢体动作或表情生成第二形变特征。示例性的，在拍摄电视塔后，如若人脸表情为悲伤，则第二形变特征为弯腰形态，此时电视塔模型呈弯腰形态。若建筑物模型出现形变，则通过裁剪方式删除建筑物原始图像，使用模型覆盖原始图像区域。
本公开实施例公开的增强现实的图像处理方法，能够响应于用户触发的图像获取指令，获取目标图像，目标图像包含目标物体；获取目标物体的增强现实模型，结合目标物体输出增强现实模型；获取用户选择的目标音频数据，根据目标音频数据确定具有时序性的音频特征；输出目标音频数据时，根据目标音频数据的播放进度和音频特征驱动增强现实模型。相对于目前增强现实模型缺乏互动性，易用性差。本公开实施例公开的增强现实的图像处理方法，能够在输出增强现实模型时，结合用户选择的目标音频数据的音频特征，驱动增强显示模型的输出，使得用户能够参与到增强显示模型的展示过程中，通过选择不同的目标音频数据，驱动增强现实模型根据目标音频数据的音频特征进行展示，提高易用性。
实施例二
图2为本公开实施例二提供的增强现实的图像处理装置的结构示意图，本实施例可适用于展示增强现实模型的情况，该方法可以由实现增强现实的电子设备来执行，该电子设备可以为智能手机、平板电脑等，该装置包括：目标图像获取模块210、增强现实模型获取模块220、目标音频获取模块230、音频特征确定模块240或输出模块250。
目标图像获取模块210，用于响应于用户触发的图像获取指令，获取目标图像，目标图像包含目标物体；
增强现实模型获取模块220，用于获取目标物体的增强现实模型，结合目标物体输出增强现实模型；
目标音频获取模块230，用于获取用户选择的目标音频数据；
音频特征确定模块240，用于根据目标音频数据确定具有时序性的音频特征；
输出模块250，用于输出目标音频数据时，根据目标音频数据的播放进度和音频特征驱动增强现实模型。
进一步的，音频特征确定模块240用于：
对目标音频数据进行音频特征检测，得到目标音频数据的具有时序性的音频特征，音频特征包括重音、重拍或节拍中的一种或多种的组合。
进一步的，增强现实模型包括多个模型单元；输出模块250用于：
根据目标音频数据的播放进度和音频特征，驱动增强现实模型中的模型单元。
进一步的，输出模块250用于：
根据具有时序性的音频特征确定模型单元进行形态变化的目标时间和目标幅度；
若目标音频数据的播放进度为目标时间，则根据目标幅度驱动增强现实模型中的模型单元。
进一步的，输出模块250用于：
驱动增强现实模型中的预设模型单元进行凸起动作；或者，
驱动增强现实模型中多个模型单元执行颜色变化；或者，
驱动增强现实模型中多个模型单元执行透明度变化。
进一步的，还包括第一形变特征获取模块。第一形变特征获取模块用于：
获取目标音频数据的情绪特征；
根据情绪特征确定增强现实模型的第一形变特征，第一形变特征用于驱动增强现实模型的形状与情绪特征表达的情绪一致；
输出目标音频数据时，根据第一形变特征驱动增强现实模型。
进一步的，还包括第二形变特征获取模块。第二形变特征获取模块用于：
获取用户的肢体动作或表情；
根据肢体动作或表情确定增强现实模型的第二形变特征，第二形变特征用于驱动增强现实模型的形状与肢体动作或表情一致；
输出目标音频数据时，根据第二形变特征驱动增强现实模型。
本公开实施例公开的增强现实的图像处理装置，目标图像获取模块210响应于用户触发的图像获取指令，获取目标图像，目标图像包含目标物体；增强现实模型获取模块220获取目标物体的增强现实模型，结合目标物体输出增强现实模型；目标音频获取模块230获取用户选择的目标音频数据，音频特征确定模块240根据目标音频数据确定具有时序性 的音频特征；输出模块250输出目标音频数据时，根据目标音频数据的播放进度和音频特征驱动增强现实模型。相对于目前增强现实模型缺乏互动性，易用性差。本公开实施例公开的增强现实的图像处理装置，能够在输出增强现实模型时，结合用户选择的目标音频数据的音频特征，驱动增强显示模型的输出，使得用户能够参与到增强显示模型的展示过程中，通过选择不同的目标音频数据，驱动增强现实模型根据目标音频数据的音频特征进行展示，提高易用性。
本发明实施例所提供的增强现实的图像处理装置可执行本发明任意实施例所提供的增强现实的图像处理方法，具备执行方法相应的功能模块和有益效果。
实施例三
下面参考图3，其示出了适于用来实现本公开实施例三的电子设备800的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、Personal digital assistant(PDA，个人数字助理)、Portable android device(PAD，平板电脑)、Portable media player(PMP，便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV(Television)、台式计算机等等的固定终端。图3示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图3所示，电子设备800可以包括处理装置(例如中央处理器、图形处理器等)801，其可以根据存储在只读存储器(Read only memory,ROM)802中的程序或者从存储装置808加载到随机访问存储器(Random access memory,RAM)803中的程序而执行各种适当的动作和处理。在RAM 803中，还存储有电子设备800操作所需的各种程序和数据。处理装置801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(Input/output,I/O)接口805也连接至总线804。
通常，以下装置可以连接至I/O接口805：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置806；包括例如液晶显示器(Liquid crystal display,LCD)、扬声器、振动器等的输出装置807；包括例如磁带、硬盘等的存储装置808；以及通信装置809。通信装置809可以允许电子设备800与其他设备进行无线或有线通信以交换数据。虽然图3示出了具有各种装置的电子设备800，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上 的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置809从网络上被下载和安装，或者从存储装置808被安装，或者从ROM 802被安装。在该计算机程序被处理装置801执行时，执行本公开实施例的方法中的步骤，以实现其限定的上述功能。
根据本公开的一个或多个实施例提供的一种计算机程序产品，该计算机程序产品包括：计算机程序，该计算机程序存储在可读存储介质中，电子设备的一个或多个处理器可以从所述可读存储介质读取所述计算机程序，所述一个或多个处理器执行所述计算机程序，使得所述电子设备执行上述任一实施例提供的方案。
本公开的一个或多个实施例提供的一种计算机程序，该计算机程序存储在可读存储介质中，电子设备的一个或多个处理器可以从所述可读存储介质中读取所述计算机程序，所述一个或多个处理器执行所述计算机程序，使得所述电子设备执行上述任一实施例提供的方案。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Electrical programmable read only memory,EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(Compact disc read only memory,CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、射频(Radio Frequency,RF)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：获取至少两个网际协议地址；向节点评价设备发送包括所述至少两个网际协议地址的节点评价请求，其中，所述节点评价设备从所述至少两个网际协议地址中，选取网际协议地址并返回；接收所述节点评价设备返回的网际协议地址；其中，所获取的网际协议地址指示内容分发网络中的边缘节点。
或者，上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：接收包括至少两个网际协议地址的节点评价请求；从所述至少两个网际协议地址中，选取网际协议地址；返回选取出的网际协议地址；其中，接收到的网际协议地址指示内容分发网络中的边缘节点。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local area network,LAN)或广域网(Wide area network,WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应 当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (12)

  1. 一种增强现实的图像处理方法，其特征在于，包括：
    响应于用户触发的图像获取指令，获取目标图像，所述目标图像包含目标物体；
    获取所述目标物体的增强现实模型，结合所述目标物体输出所述增强现实模型；
    获取所述用户选择的目标音频数据，根据所述目标音频数据确定具有时序性的音频特征；
    输出所述目标音频数据时，根据所述目标音频数据的播放进度和所述音频特征驱动所述增强现实模型。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述目标音频数据确定具有时序性的音频特征，包括：
    对所述目标音频数据进行音频特征检测，得到所述目标音频数据的所述具有时序性的音频特征，所述音频特征包括重音、重拍或节拍中的一种或多种的组合。
  3. 根据权利要求1或2所述的方法，其特征在于，所述增强现实模型包括多个模型单元；
    所述根据所述目标音频数据的播放进度和所述音频特征驱动所述增强现实模型，包括：
    根据所述目标音频数据的播放进度和所述音频特征，驱动所述增强现实模型中的模型单元。
  4. 根据权利要求3所述的方法，其特征在于，根据所述目标音频数据的播放进度和所述音频特征，驱动所述增强现实模型中的模型单元，包括：
    根据所述具有时序性的音频特征确定所述模型单元进行形态变化的目标时间和目标幅度；
    若所述目标音频数据的播放进度为所述目标时间，则根据所述目标幅度驱动所述增强现实模型中的模型单元。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述目标幅度驱动所述增强现实模型中的模型单元，包括：
    驱动所述增强现实模型中的预设模型单元进行凸起动作；或者，
    驱动所述增强现实模型中多个模型单元执行颜色变化；或者，
    驱动所述增强现实模型中多个模型单元执行透明度变化。
  6. 根据权利要求1-5中任一所述的方法，其特征在于，在获取所述用户选择的目标音频数据之后，还包括：
    获取所述目标音频数据的情绪特征；
    根据所述情绪特征确定所述增强现实模型的第一形变特征，所述第一形变特征用 于驱动所述增强现实模型的形状与所述情绪特征表达的情绪一致；
    输出所述目标音频数据时，根据所述第一形变特征驱动所述增强现实模型。
  7. 根据权利要求1-5中任一所述的方法，其特征在于，在获取所述用户选择的目标音频数据之后，还包括：
    获取所述用户的肢体动作或表情；
    根据所述肢体动作或表情确定所述增强现实模型的第二形变特征，所述第二形变特征用于驱动所述增强现实模型的形状与所述肢体动作或表情一致；
    输出所述目标音频数据时，根据所述第二形变特征驱动所述增强现实模型。
  8. 一种增强现实的图像处理装置，其特征在于，包括：
    目标图像获取模块，用于响应于用户触发的图像获取指令，获取目标图像，所述目标图像包含目标物体；
    增强现实模型获取模块，用于获取所述目标物体的增强现实模型，结合所述目标物体输出所述增强现实模型；
    目标音频获取模块，用于获取所述用户选择的目标音频数据；
    音频特征确定模块，用于根据所述目标音频数据确定具有时序性的音频特征；
    输出模块，用于输出所述目标音频数据时，根据所述目标音频数据的播放进度和所述音频特征驱动所述增强现实模型。
  9. 一种电子设备，其特征在于，所述电子设备包括：
    一个或多个处理器；
    存储装置，用于存储一个或多个程序，
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-7中任一所述的增强现实的图像处理方法。
  10. 一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如权利要求1-7中任一所述的增强现实的图像处理方法。
  11. 一种计算机程序产品，其特征在于，包括计算机程序指令，所述计算机程序指令使得计算机执行如权利要求1-7中任一所述的增强现实的图像处理方法。
  12. 一种计算机程序，其特征在于，所述计算机程序使得计算机执行如权利要求1-7中任一所述的增强现实的图像处理方法。
PCT/CN2021/098456 2020-07-10 2021-06-04 增强现实的图像处理方法、装置、电子设备及存储介质 WO2022007565A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP21837648.1A EP4167192A4 (en) 2020-07-10 2021-06-04 IMAGE PROCESSING METHOD AND APPARATUS FOR AUGMENTED REALITY, ELECTRONIC DEVICE AND RECORDING MEDIUM
JP2023501106A JP2023533295A (ja) 2020-07-10 2021-06-04 拡張現実の画像処理方法、装置、電子機器及び記憶媒体
US18/053,476 US11756276B2 (en) 2020-07-10 2022-11-08 Image processing method and apparatus for augmented reality, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010662819.X 2020-07-10
CN202010662819.XA CN111833460B (zh) 2020-07-10 2020-07-10 增强现实的图像处理方法、装置、电子设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/053,476 Continuation US11756276B2 (en) 2020-07-10 2022-11-08 Image processing method and apparatus for augmented reality, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
WO2022007565A1 true WO2022007565A1 (zh) 2022-01-13

Family

ID=72899721

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/098456 WO2022007565A1 (zh) 2020-07-10 2021-06-04 增强现实的图像处理方法、装置、电子设备及存储介质

Country Status (5)

Country Link
US (1) US11756276B2 (zh)
EP (1) EP4167192A4 (zh)
JP (1) JP2023533295A (zh)
CN (1) CN111833460B (zh)
WO (1) WO2022007565A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111833460B (zh) 2020-07-10 2024-07-26 北京字节跳动网络技术有限公司 增强现实的图像处理方法、装置、电子设备及存储介质
CN112288877A (zh) * 2020-10-28 2021-01-29 北京字节跳动网络技术有限公司 视频播放方法、装置、电子设备及存储介质
CN112672185B (zh) * 2020-12-18 2023-07-07 脸萌有限公司 基于增强现实的显示方法、装置、设备及存储介质
CN113031781A (zh) * 2021-04-16 2021-06-25 深圳市慧鲤科技有限公司 增强现实资源显示方法及装置、电子设备和存储介质
CN116940967A (zh) * 2021-04-21 2023-10-24 深圳传音控股股份有限公司 图像控制方法、移动终端及存储介质
US11769289B2 (en) * 2021-06-21 2023-09-26 Lemon Inc. Rendering virtual articles of clothing based on audio characteristics

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105608745A (zh) * 2015-12-21 2016-05-25 大连新锐天地传媒有限公司 应用于图像或视频的ar显示系统
CN106506464A (zh) * 2016-10-17 2017-03-15 武汉秀宝软件有限公司 一种基于增强现实的玩具交互方法和系统
WO2019029100A1 (zh) * 2017-08-08 2019-02-14 山东科技大学 一种基于虚拟现实与增强现实的采矿操作多交互实现方法
CN109407918A (zh) * 2018-09-25 2019-03-01 苏州梦想人软件科技有限公司 增强现实内容多级交互方式的实现方法
CN111833460A (zh) * 2020-07-10 2020-10-27 北京字节跳动网络技术有限公司 增强现实的图像处理方法、装置、电子设备及存储介质

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101577114B (zh) * 2009-06-18 2012-01-25 无锡中星微电子有限公司 一种音频可视化实现方法及装置
US9361730B2 (en) * 2012-07-26 2016-06-07 Qualcomm Incorporated Interactions of tangible and augmented reality objects
CN103544724A (zh) * 2013-05-27 2014-01-29 华夏动漫集团有限公司 一种利用增强现实与卡片识别技术在移动智能终端实现虚拟动漫角色的系统及方法
US10445936B1 (en) * 2016-08-01 2019-10-15 Snap Inc. Audio responsive augmented reality
CN110214343B (zh) * 2017-01-27 2023-02-03 索尼公司 信息处理装置、信息处理方法及其程序
US11232645B1 (en) * 2017-11-21 2022-01-25 Amazon Technologies, Inc. Virtual spaces as a platform
CN108322802A (zh) * 2017-12-29 2018-07-24 广州市百果园信息技术有限公司 视频图像的贴图处理方法、计算机可读存储介质及终端
CN108769535B (zh) * 2018-07-04 2021-08-10 腾讯科技（深圳）有限公司 图像处理方法、装置、存储介质和计算机设备
US10679393B2 (en) * 2018-07-24 2020-06-09 Snap Inc. Conditional modification of augmented reality object
CN109144610B (zh) * 2018-08-31 2020-11-10 腾讯科技（深圳）有限公司 音频播放方法、装置、电子装置及计算机可读存储介质
CN110072047B (zh) * 2019-01-25 2020-10-09 北京字节跳动网络技术有限公司 图像形变的控制方法、装置和硬件装置
US10924875B2 (en) * 2019-05-24 2021-02-16 Zack Settel Augmented reality platform for navigable, immersive audio experience
JP2024507749A (ja) * 2021-02-08 2024-02-21 サイトフル コンピューターズ リミテッド エクステンデッドリアリティにおけるコンテンツ共有

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105608745A (zh) * 2015-12-21 2016-05-25 大连新锐天地传媒有限公司 应用于图像或视频的ar显示系统
CN106506464A (zh) * 2016-10-17 2017-03-15 武汉秀宝软件有限公司 一种基于增强现实的玩具交互方法和系统
WO2019029100A1 (zh) * 2017-08-08 2019-02-14 山东科技大学 一种基于虚拟现实与增强现实的采矿操作多交互实现方法
CN109407918A (zh) * 2018-09-25 2019-03-01 苏州梦想人软件科技有限公司 增强现实内容多级交互方式的实现方法
CN111833460A (zh) * 2020-07-10 2020-10-27 北京字节跳动网络技术有限公司 增强现实的图像处理方法、装置、电子设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4167192A4 *

Also Published As

Publication number Publication date
US20230061012A1 (en) 2023-03-02
CN111833460A (zh) 2020-10-27
EP4167192A4 (en) 2023-12-13
EP4167192A1 (en) 2023-04-19
CN111833460B (zh) 2024-07-26
US11756276B2 (en) 2023-09-12
JP2023533295A (ja) 2023-08-02

Similar Documents

Publication Publication Date Title
WO2022007565A1 (zh) 增强现实的图像处理方法、装置、电子设备及存储介质
US11158102B2 (en) Method and apparatus for processing information
JP7199527B2 (ja) 画像処理方法、装置、ハードウェア装置
CN109346111B (zh) 数据处理方法、装置、终端及存储介质
WO2020248900A1 (zh) 全景视频的处理方法、装置及存储介质
CN112672185B (zh) 基于增强现实的显示方法、装置、设备及存储介质
CN109600559B (zh) 一种视频特效添加方法、装置、终端设备及存储介质
WO2021057740A1 (zh) 视频生成方法、装置、电子设备和计算机可读介质
US20230307004A1 (en) Audio data processing method and apparatus, and device and storage medium
CN112965780B (zh) 图像显示方法、装置、设备及介质
US12019669B2 (en) Method, apparatus, device, readable storage medium and product for media content processing
WO2020155915A1 (zh) 用于播放音频的方法和装置
CN111652675A (zh) 展示方法、装置和电子设备
WO2020259152A1 (zh) 贴纸生成方法、装置、介质和电子设备
WO2022012349A1 (zh) 动画处理方法、装置、电子设备及存储介质
CN111312207A (zh) 文本转音频方法、装置、计算机设备及存储介质
US11810336B2 (en) Object display method and apparatus, electronic device, and computer readable storage medium
CN114677738A (zh) Mv录制方法、装置、电子设备及计算机可读存储介质
CN109472873B (zh) 三维模型的生成方法、装置、硬件装置
CN111292773A (zh) 音视频合成的方法、装置、电子设备及介质
CN114329001B (zh) 动态图片的显示方法、装置、电子设备及存储介质
CN115494953A (zh) 音频调节方法、装置、设备及计算机可读存储介质
WO2023182935A2 (zh) 图像处理方法、装置、电子设备及存储介质
CN112380380A (zh) 显示歌词的方法、装置、设备及计算机可读存储介质
CN117785085A (zh) 虚拟终端设备的信息提示方法、装置、设备、介质及产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21837648

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023501106

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021837648

Country of ref document: EP

Effective date: 20230112

NENP Non-entry into the national phase

Ref country code: DE