CN117745984A - AR picture generation and display method and device, electronic equipment and storage medium - Google Patents
AR picture generation and display method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117745984A CN117745984A CN202311579906.9A CN202311579906A CN117745984A CN 117745984 A CN117745984 A CN 117745984A CN 202311579906 A CN202311579906 A CN 202311579906A CN 117745984 A CN117745984 A CN 117745984A
- Authority
- CN
- China
- Prior art keywords
- initial
- image
- dimensional
- picture
- dimensional image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000009877 rendering Methods 0.000 claims abstract description 18
- 238000004891 communication Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 7
- 238000009792 diffusion process Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 14
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000003190 augmentative effect Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000001965 increasing effect Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The embodiment of the invention provides an AR picture generation and display method, an AR picture generation and display device, electronic equipment and a storage medium, wherein description information and constraint information of an image to be generated are acquired, the description information comprises information for describing a theme of the image to be generated, and the constraint information is used for representing the shape and/or edge of the image to be generated; generating an initial two-dimensional image based on the description information and the constraint information; determining a height map of the initial two-dimensional image, wherein the height map is used for representing stereoscopic information of the initial two-dimensional image; rendering is carried out based on the height map and the initial two-dimensional image, a three-dimensional model corresponding to the initial two-dimensional image is obtained, and the three-dimensional model is used for being combined with the AR scene image to generate an AR picture. The labor cost of the picture generation process can be reduced.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an AR picture generating and displaying method, apparatus, electronic device, and storage medium.
Background
With the continued development of augmented reality (Augmented Reality, AR) technology, AR pictures have become an effective way to propagate information and interactions, which can make content more stereoscopic and wonderful. For example, AR posters may be applied to show and propagate theme creatives, such as art shows, cultural knots, and the like. Currently, AR picture construction is mainly based on manual design, requiring specialized 3D (three-dimensional) modeling and designers, and having large labor and time costs.
Disclosure of Invention
The embodiment of the invention aims to provide an AR picture generation and display method, an AR picture generation and display device, electronic equipment and a storage medium, so that labor cost in a picture generation process is reduced. The specific technical scheme is as follows:
in a first aspect of the present invention, there is provided an AR picture generation method, including:
acquiring description information of an image to be generated and constraint information, wherein the description information comprises information for describing a theme of the image to be generated, and the constraint information is used for representing the shape and/or edges of the image to be generated;
generating an initial two-dimensional image based on the description information and the constraint information;
determining a height map of the initial two-dimensional image, the height map being used to represent stereo information of the initial two-dimensional image;
rendering is carried out based on the height map and the initial two-dimensional image, and a three-dimensional model corresponding to the initial two-dimensional image is obtained and is used for being combined with an AR scene image to generate an AR picture.
Optionally, the generating an initial two-dimensional image based on the description information and the constraint information includes:
and taking the description information and the constraint information as input of an artificial intelligent text-to-image model Stable diffusion, and processing the description information and the constraint information through the stabledifusion to obtain an initial two-dimensional image.
Optionally, the determining the height map of the initial two-dimensional image includes:
determining a depth map of the initial two-dimensional image;
determining the distance between each pixel point in the initial two-dimensional image and a space origin based on the depth map of the initial two-dimensional image, and taking the distance between each pixel point in the initial two-dimensional image and the space origin as the height of each pixel point;
and generating a height map corresponding to the initial two-dimensional image, wherein the height map comprises the heights of the pixel points.
Optionally, the rendering based on the height map and the initial two-dimensional image to obtain a three-dimensional model corresponding to the initial two-dimensional image includes:
applying the initial two-dimensional image as a texture map to an initial three-dimensional model to obtain a textured three-dimensional model;
and applying the height map to the textured three-dimensional model to obtain a three-dimensional model corresponding to the initial two-dimensional image.
In a second aspect of the present invention, there is provided an AR picture display method, applied to an AR device, including:
acquiring a pre-generated three-dimensional model, wherein the three-dimensional model is obtained based on the AR picture generation method according to any one of the first aspect;
combining the three-dimensional model with an AR scene image to generate an AR picture;
and when a picture viewing request is received, displaying the AR picture.
Optionally, the combining the three-dimensional model with the AR scene image generates an AR picture, including:
acquiring an AR scene image;
and selecting a target position point from the AR scene image, and embedding the three-dimensional model into the target position point.
In a third aspect of the present invention, there is provided an AR picture generation apparatus, including:
the device comprises an acquisition module, a generation module and a generation module, wherein the acquisition module is used for acquiring description information of an image to be generated and constraint information, the description information comprises information for describing a theme of the image to be generated, and the constraint information is used for representing the shape and/or the edge of the image to be generated;
the generation module is used for generating an initial two-dimensional image based on the description information and the constraint information;
the determining module is used for determining a height map of the initial two-dimensional image, wherein the height map is used for representing stereoscopic information of the initial two-dimensional image;
and the rendering module is used for rendering based on the height map and the initial two-dimensional image to obtain a three-dimensional model corresponding to the initial two-dimensional image, and the three-dimensional model is used for generating an AR picture by combining with the AR scene image.
Optionally, the generating module is specifically configured to use the description information and the constraint information as input of an artificial intelligent text-to-image model stablisafusion, and process the description information and the constraint information through stablisafusion to obtain an initial two-dimensional image.
Optionally, the determining module is specifically configured to determine a depth map of the initial two-dimensional image; determining the distance between each pixel point in the initial two-dimensional image and a space origin based on the depth map of the initial two-dimensional image, and taking the distance between each pixel point in the initial two-dimensional image and the space origin as the height of each pixel point; and generating a height map corresponding to the initial two-dimensional image, wherein the height map comprises the heights of the pixel points.
Optionally, the rendering module is specifically configured to apply the initial two-dimensional image as a texture map to an initial three-dimensional model to obtain a textured three-dimensional model; and applying the height map to the textured three-dimensional model to obtain a three-dimensional model corresponding to the initial two-dimensional image.
In a fourth aspect of the present invention, there is provided an AR picture display device, applied to an AR apparatus, including:
an acquisition module, configured to acquire a three-dimensional model generated in advance, where the three-dimensional model is obtained based on the AR picture generation method according to any one of the first aspect;
the generation module is used for combining the three-dimensional model with the AR scene image to generate an AR picture;
and the display module is used for displaying the AR picture when receiving a picture viewing request.
Optionally, the generating module is specifically configured to acquire an AR scene image; and selecting a target position point from the AR scene image, and embedding the three-dimensional model into the target position point.
In a fifth aspect of the present invention, there is also provided an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
a processor configured to implement the method steps of any one of the first or second aspects when executing a program stored on a memory.
In still another aspect of the present invention, there is also provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements any one of the above AR picture generation method or AR picture display method.
In yet another aspect of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the above-described AR picture generation method or AR picture display method.
According to the AR picture generation and display method, the AR picture generation and display device, the electronic equipment and the storage medium, the description information and the constraint information of the image to be generated are acquired, so that an initial two-dimensional image can be generated based on the description information and the constraint information, and the height map of the initial two-dimensional image is determined and used for representing the stereoscopic information of the initial two-dimensional image; and rendering can be performed based on the height map and the initial two-dimensional image to obtain a three-dimensional model corresponding to the initial two-dimensional image, so that the three-dimensional model can be embedded into the AR scene image to obtain an AR picture, automatic generation of the AR picture is realized, and labor cost in the picture generation process is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a flowchart of an AR picture generation method according to an embodiment of the present invention;
fig. 2 is a flowchart of an AR picture display method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an AR picture generating apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an AR picture display device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.
Fig. 1 is a flowchart of an AR picture generation method provided by an embodiment of the present invention, and referring to fig. 1, the method may include:
s101, acquiring description information and constraint information of an image to be generated, wherein the description information comprises information for describing a theme of the image to be generated, and the constraint information is used for representing the shape and/or the edge of the image to be generated;
s102, generating an initial two-dimensional image based on the description information and constraint information;
s103, determining a height map of the initial two-dimensional image, wherein the height map is used for representing stereoscopic information of the initial two-dimensional image;
and S104, rendering based on the height map and the initial two-dimensional image to obtain a three-dimensional model corresponding to the initial two-dimensional image, wherein the three-dimensional model is used for combining with the AR scene image to generate an AR picture.
In the embodiment of the invention, by acquiring the description information and the constraint information of the image to be generated, an initial two-dimensional image can be generated based on the description information and the constraint information, and the height map of the initial two-dimensional image is determined and used for representing the stereoscopic information of the initial two-dimensional image; and rendering can be performed based on the height map and the initial two-dimensional image to obtain a three-dimensional model corresponding to the initial two-dimensional image, so that the three-dimensional model can be embedded into the AR scene image to obtain an AR picture, automatic generation of the AR picture is realized, and labor cost in the picture generation process is reduced.
In addition, compared with the mode of manually generating images in the related art, the image generation result is not easy to control, and high-quality image content meeting the requirements is difficult to generate.
The AR picture generation method provided by the embodiment of the invention can be applied to electronic equipment, and specifically can be a server and the like, for example, a service server deployed with an artificial intelligent text-to-image model (stabiedifusion), and the like.
The description information of the image to be generated in S101 may include a subject of the image to be generated, for example, the AR picture to be generated is an AR poster for cultural festival propaganda, and the description information may include "cultural propaganda". Alternatively, the description information includes one or more description words, which may be used to describe a scene of an image to be generated, information of included objects, and the like.
Constraint information is used to represent the shape and/or edges of the image to be generated. Constraint information can be understood as Constraints that can be used to constrain the shape and/or edges of an image by invoking a controllnet plug-in during the generation of an initial two-dimensional image.
An image generation tool may be invoked in S102 to generate an initial two-dimensional image based on the descriptive information and constraint information.
In one implementation, description information and constraint information are input as an artificial intelligent text-to-image model stablistification, and the description information and the constraint information are processed through the stablistification to obtain an initial two-dimensional image. It is also understood that the description information and constraint information are input into an artificial intelligence text-to-image model (stabiedifusion), and an initial two-dimensional image is generated using the artificial intelligence text-to-image model.
The method can be beneficial to accurately controlling the poster image effect, and solves the problem that the quality of the AR poster image generated in the related technology is difficult to control to a certain extent.
Based on the description information and constraint information, multiple images can be generated simultaneously by using the artificial intelligence text-to-image model, the images can be displayed to a user side, and one or more images can be selected as target images by the user side to serve as initial two-dimensional images. The simultaneous generation of multiple images using stablistification provides the option of obtaining higher quality images, which in turn can enable the rapid generation of high quality image content.
The height map is used to represent stereoscopic information of the initial two-dimensional image in S103.
A height map is understood to be a texture image which represents the height information of a surface by means of grey values. In three-dimensional rendering, a height map is used to simulate the concave-convex details of the object surface, thereby increasing realism and detail.
In one implementation, the stereoscopic information is height information. The correspondence between the height map and the initial two-dimensional image can be understood as that the initial two-dimensional image is manipulated and changed to express the height information of the surface.
The operation of generating the height map may include calculating a height value for each pixel using depth information, normal map, gray map, or other algorithms. This may be accomplished by adjusting the contrast, brightness, filtering, and other image processing techniques of the image.
In one implementation, a depth map of an initial two-dimensional image is determined; determining the distance between each pixel point in the initial two-dimensional image and a space origin based on the depth map of the initial two-dimensional image, and taking the distance between each pixel point in the initial two-dimensional image and the space origin as the height of each pixel point; a height map including the heights of the pixels corresponding to the initial two-dimensional image is generated.
In one example, a deep map module of Blender (open source three-dimensional graphics image software) may be invoked to determine a height map of an initial two-dimensional image.
The initial two-dimensional image is imported into a Blender deep module, and the method can be operated according to the following steps:
a. the user opens the Blender and selects the model to apply the height map, and the selected model can then be determined.
b. In the node editor, a texture node is created and a texture node is selected, so that the texture node and the texture node can be determined.
c. Clicking texture nodes, and selecting an option of Image or Movie from the texture property panel, so that texture information can be determined.
d. Clicking the "Open" button, selecting the image file of the initial two-dimensional image to import into the Blender, so that the initial two-dimensional image can be imported into the Blender.
e. The imported image node (i.e., the initial two-dimensional image described above) is connected to the deepfump node.
f. The user can configure parameters of the deep bus node, such as height intensity, smoothness, etc., so that the parameters of the deep bus node configured by the user can be determined, and then the height map is determined based on the parameters of the deep bus node.
g. The output of the deep bump node is connected to the normal input of the texture node.
h. The results are previewed in the rendered view, with further adjustments and optimizations as needed.
The use of deep dump by Blender to generate a height map can add 3D feel to the image.
S104, the initial two-dimensional image can be used as a texture map to be applied to the initial three-dimensional model, and a textured three-dimensional model is obtained; and applying the height map to the textured three-dimensional model to obtain a three-dimensional model corresponding to the initial two-dimensional image.
A 3D model with details and realism is created. By utilizing image generation and 3D model technology, the AR poster with stereoscopic impression and interactive experience can be quickly constructed.
After the 3D model is obtained, an animation may also be added to the 3D model. Animation may include object motion, deformation, rotation, etc. in the real world. By setting key frames, path constraints, etc. on the 3D model, various dynamic effects can be created, making the model appear as a vivid animation in the AR scene.
The generated 3D model may be exported as a glTF file. glTF (GL Transmission Format) is an open, cross-platform three-dimensional model file format that supports use across multiple software and platforms. By exporting the model into the glTF file format, it can be ensured that the 3D model can be interacted and used on different software and platforms, and cross-platform compatibility is achieved.
The glTF file format supports embedding of textures and materials, texture image and material parameters can be stored together with the model in one file, and textures and materials can be reserved.
In one implementation manner, in order to generate an AR picture by using the generated 3D model under the IOS system, the glTF file format may be converted into a USDZ (universal scene description compression format) file, and the 3D model is saved by using the USDZ format file, so as to generate the AR picture by using the USDZ format file.
The 3D model is stored as a glTF file format and is converted into a USDZ file format when the IOS system is used, so that the information of textures and materials can be ensured to be reserved and correctly displayed in the AR application before the 3D model is converted into the USDZ file format.
USDZ is the file format used by the apple pushed AR Quick Look (AR Quick Look) function. By converting the 3D model from the glTF file format to the USDZ file format, it can be ensured that the 3D model can be directly viewed on the IOS device by using the AR Quick Look without any additional conversion or processing, and compatibility of the AR platform is achieved. And after being converted into the USDZ file format, the 3D model can benefit from the optimization and hardware acceleration of the ARKit framework of the apple, the USDZ file format can be loaded, rendered and processed more efficiently, rendering and performance optimization can be realized, and better performance and user experience are provided.
The embodiment of the invention can comprise image generation, altitude map generation, 3D model generation and AR scene construction, and is modularized and matched with each other, so that an efficient and controllable manner for automatically generating high-quality AR posters is provided.
An embodiment of the present invention provides an AR picture display method, as shown in fig. 2, may include:
s201, acquiring a pre-generated three-dimensional model, wherein the three-dimensional model is obtained based on the AR picture generation method in the embodiment;
s202, combining the three-dimensional model with an AR scene image to generate an AR picture;
s203, when a picture viewing request is received, displaying an AR picture.
In the embodiment of the invention, the pre-generated three-dimensional model can be directly obtained, the three-dimensional model can be embedded into the AR scene image, the AR picture is generated and displayed, so that the automatic generation of the AR picture can be realized, and the labor cost in the picture generation process is reduced. In addition, displaying the AR picture can facilitate user interaction.
The AR picture display method provided by the embodiment of the invention can be applied to AR equipment, and the AR equipment can comprise a mobile terminal such as a mobile phone and AR glasses.
AR applies virtual information to the real world, where real environment and virtual objects are superimposed in real time on the same picture or space and exist at the same time.
In S201, the three-dimensional model saved in the glTF file format may be obtained, and the three-dimensional model saved in the glTF file format is directly used to generate the AR picture, or if the AR picture is generated in the IOS system, after the three-dimensional model saved in the glTF file format is obtained, the three-dimensional model saved in the glTF file format is converted into the three-dimensional model saved in the USDZ file format, and then the three-dimensional model in the USDZ file format is used to generate the AR picture.
The AR scene image in S202 may be an initial AR poster, for example, a two-dimensional poster picture. Combining the three-dimensional model with the AR scene image can also be understood as embedding the three-dimensional model into the AR scene image, namely embedding the three-dimensional model into a two-dimensional poster picture, so that the poster has the function of augmented reality, and the AR poster with interaction is constructed. The generated 3D model may enhance the visual effect and user experience of the AR poster.
Specifically, embedding the three-dimensional model into the AR scene image, generating an AR picture may include: acquiring an AR scene image; and selecting a target position point from the AR scene image, and embedding the three-dimensional model into the target position point.
The target location point may be selected according to actual requirements. The size of the three-dimensional model can also be adjusted in the process of embedding the three-dimensional model into the target position point.
In one implementation, in the process of generating the 3D model, an animation can be added to the 3D model, so that the three-dimensional model is acquired, and meanwhile, the animation added to the 3D model can be acquired, and the 3D model and the animation added to the 3D model can be embedded into an AR scene image to generate an AR picture. For example, when an animation is further added to the 3D model after the 3D model is generated, the generated 3D model and animation may be embedded in the AR poster, so that the poster has an augmented reality function.
In the process of generating the 3D model, the height map is added, so that the stereoscopic impression and details of an AR scene are effectively increased, the AR effect is improved, and the problem that the 3D model of the AR poster lacks details in the related technology is solved to a certain extent. In the embodiment of the invention, the stereo information of the image is increased by utilizing the height map, and more detail information is added. According to the embodiment of the invention, the AR picture is generated by directly utilizing the three-dimensional model obtained by rendering the height map and the initial two-dimensional image, so that the AR picture can be automatically generated, more detail information is added in the 3D model generation process, and the AR picture such as an AR poster can be generated faster and better.
The embodiment of the invention can construct an interactive AR scene through an AR development tool under the AR development scene.
In S203, the picture viewing request may be a request triggered when the user scans through the AR device.
When a user scans an AR poster using an AR device, the 3D model and animation will be presented in a virtual manner on the user's device, interacting with the real world.
The 3D model may enhance the visual effect and user experience of the AR poster. By embedding the 3D model into the AR poster, a more lively, interactive and interesting experience can be brought to the user. The 3D model may provide more details, realistic lighting effects, and opportunities to interact with the user, thereby attracting the user's attention and enhancing the attractiveness of the poster.
The embodiment of the invention can be used for generating the AR poster of the AR art and cultural exhibition site. Enriches the interactive experience of the exhibition and attracts more audiences. The method is very useful for online popularization of cultural enterprises, and increases the viscosity of clients. The AR interaction function is utilized to attract the eyes more, so that the user participation degree is improved. Has great value for enterprise brand promotion and marketing activities, and can effectively promote brand impression and topic degree.
The embodiment of the invention also provides an AR picture generating device, which can comprise:
the acquiring module 301 is configured to acquire description information of an image to be generated and constraint information, where the description information includes information for describing a subject of the image to be generated, and the constraint information is used to represent a shape and/or an edge of the image to be generated;
a generation module 302, configured to generate an initial two-dimensional image based on the description information and constraint information;
a determining module 303, configured to determine a height map of the initial two-dimensional image, where the height map is used to represent stereo information of the initial two-dimensional image;
the rendering module 304 is configured to render based on the altitude map and the initial two-dimensional image, to obtain a three-dimensional model corresponding to the initial two-dimensional image, where the three-dimensional model is used to combine with the AR scene image to generate an AR picture.
Optionally, the generating module 302 is specifically configured to use the description information and the constraint information as input of the artificial intelligent text-to-image model stablisafusion, and process the description information and the constraint information through stablisafusion to obtain an initial two-dimensional image.
Optionally, the determining module 303 is specifically configured to determine a depth map of the initial two-dimensional image; determining the distance between each pixel point in the initial two-dimensional image and a space origin based on the depth map of the initial two-dimensional image, and taking the distance between each pixel point in the initial two-dimensional image and the space origin as the height of each pixel point; a height map including the heights of the pixels corresponding to the initial two-dimensional image is generated.
Optionally, the rendering module 304 is specifically configured to apply the initial two-dimensional image as a texture map to the initial three-dimensional model, so as to obtain a textured three-dimensional model; and applying the height map to the textured three-dimensional model to obtain a three-dimensional model corresponding to the initial two-dimensional image.
The embodiment of the invention also provides an AR picture display device, as shown in fig. 4, which is applied to an AR device and can include:
an obtaining module 401, configured to obtain a three-dimensional model that is generated in advance, where the three-dimensional model is obtained based on the AR picture generating method in the above embodiment;
a generating module 402, configured to combine the three-dimensional model with the AR scene image to generate an AR picture;
the display module 403 is configured to display an AR picture when a picture viewing request is received.
Optionally, the generating module 402 is specifically configured to acquire an AR scene image; and selecting a target position point from the AR scene image, and embedding the three-dimensional model into the target position point.
The embodiment of the invention also provides an electronic device, as shown in fig. 5, which comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 complete communication with each other through the communication bus 504,
a memory 503 for storing a computer program;
the processor 501 is configured to implement the above-described AR picture generation method or AR picture display method when executing the program stored in the memory 503.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In still another embodiment of the present invention, there is further provided a computer readable storage medium having a computer program stored therein, which when executed by a processor, implements the AR picture generation method or the AR picture display method described in any one of the above embodiments.
In yet another embodiment of the present invention, there is also provided a computer program product containing instructions, which when run on a computer, cause the computer to perform the AR picture generation method or the AR picture display method as described in any one of the above embodiments.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus, electronic device, computer readable storage medium, and computer program product embodiments, the description is relatively simple, as relevant to the method embodiments being referred to in the section of the description of the method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (10)
1. An AR picture generation method, comprising:
acquiring description information of an image to be generated and constraint information, wherein the description information comprises information for describing a theme of the image to be generated, and the constraint information is used for representing the shape and/or edges of the image to be generated;
generating an initial two-dimensional image based on the description information and the constraint information;
determining a height map of the initial two-dimensional image, the height map being used to represent stereo information of the initial two-dimensional image;
rendering is carried out based on the height map and the initial two-dimensional image, and a three-dimensional model corresponding to the initial two-dimensional image is obtained and is used for being combined with an AR scene image to generate an AR picture.
2. The method of claim 1, wherein the generating an initial two-dimensional image based on the description information and the constraint information comprises:
and taking the description information and the constraint information as input of an artificial intelligent text-to-image model Stable diffusion, and processing the description information and the constraint information through the stabledifusion to obtain an initial two-dimensional image.
3. The method of claim 1, wherein the determining a height map of the initial two-dimensional image comprises:
determining a depth map of the initial two-dimensional image;
determining the distance between each pixel point in the initial two-dimensional image and a space origin based on the depth map of the initial two-dimensional image, and taking the distance between each pixel point in the initial two-dimensional image and the space origin as the height of each pixel point;
and generating a height map corresponding to the initial two-dimensional image, wherein the height map comprises the heights of the pixel points.
4. The method according to claim 1, wherein the rendering based on the height map and the initial two-dimensional image to obtain a three-dimensional model corresponding to the initial two-dimensional image comprises:
applying the initial two-dimensional image as a texture map to an initial three-dimensional model to obtain a textured three-dimensional model;
and applying the height map to the textured three-dimensional model to obtain a three-dimensional model corresponding to the initial two-dimensional image.
5. An AR picture display method, which is applied to an AR device, includes:
acquiring a pre-generated three-dimensional model, the three-dimensional model being obtained based on the AR picture generation method according to any one of claims 1 to 4;
combining the three-dimensional model with an AR scene image to generate an AR picture;
and when a picture viewing request is received, displaying the AR picture.
6. The method of claim 5, wherein the combining the three-dimensional model with the AR scene image to generate an AR picture comprises:
acquiring an AR scene image;
and selecting a target position point from the AR scene image, and embedding the three-dimensional model into the target position point.
7. An AR picture generation apparatus, comprising:
the device comprises an acquisition module, a generation module and a generation module, wherein the acquisition module is used for acquiring description information of an image to be generated and constraint information, the description information comprises information for describing a theme of the image to be generated, and the constraint information is used for representing the shape and/or the edge of the image to be generated;
the generation module is used for generating an initial two-dimensional image based on the description information and the constraint information;
the determining module is used for determining a height map of the initial two-dimensional image, wherein the height map is used for representing stereoscopic information of the initial two-dimensional image;
and the rendering module is used for rendering based on the height map and the initial two-dimensional image to obtain a three-dimensional model corresponding to the initial two-dimensional image, and the three-dimensional model is used for generating an AR picture by combining with the AR scene image.
8. An AR picture display apparatus, characterized by being applied to an AR device, comprising:
an acquisition module for acquiring a pre-generated three-dimensional model, the three-dimensional model being obtained based on the AR picture generation method according to any one of claims 1 to 4;
the generation module is used for combining the three-dimensional model with the AR scene image to generate an AR picture;
and the display module is used for displaying the AR picture when receiving a picture viewing request.
9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor for carrying out the method steps of any one of claims 1-6 when executing a program stored on a memory.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311579906.9A CN117745984A (en) | 2023-11-24 | 2023-11-24 | AR picture generation and display method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311579906.9A CN117745984A (en) | 2023-11-24 | 2023-11-24 | AR picture generation and display method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117745984A true CN117745984A (en) | 2024-03-22 |
Family
ID=90253492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311579906.9A Pending CN117745984A (en) | 2023-11-24 | 2023-11-24 | AR picture generation and display method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117745984A (en) |
-
2023
- 2023-11-24 CN CN202311579906.9A patent/CN117745984A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109448099B (en) | Picture rendering method and device, storage medium and electronic device | |
| CN111476851B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN107358659B (en) | Multi-picture fusion display method based on 3D technology and storage device | |
| CN105183477A (en) | System and method for acquiring virtual item information of application program | |
| CN112991508A (en) | WebGL-based 3D rendering system and method | |
| CN109671147B (en) | Texture map generation method and device based on three-dimensional model | |
| Ebner et al. | Multi‐view reconstruction of dynamic real‐world objects and their integration in augmented and virtual reality applications | |
| CN117390322A (en) | Virtual space construction method and device, electronic equipment and nonvolatile storage medium | |
| CN110930492B (en) | Model rendering method, device, computer readable medium and electronic equipment | |
| CN113546410B (en) | Terrain model rendering method, apparatus, electronic device and storage medium | |
| CN112037341A (en) | Method and device for processing VR panorama interaction function based on Web front end | |
| CN115311395A (en) | Three-dimensional scene rendering method, device and equipment | |
| CN117745984A (en) | AR picture generation and display method and device, electronic equipment and storage medium | |
| CN117579885A (en) | Special effect display method and system for live broadcasting room | |
| CN110996087A (en) | Video display method and device | |
| CN108876925B (en) | Virtual reality scene processing method and device | |
| Stefan et al. | Prototyping 3D virtual learning environments with X3D-based content and visualization tools | |
| CN113509731B (en) | Fluid model processing method and device, electronic equipment and storage medium | |
| Yuan et al. | Research on VR virtual display technology of non-heritage cultural and creative products | |
| CN112423014A (en) | Remote review method and device | |
| CN117240831A (en) | Three-dimensional model transmission method and device, storage medium and program product thereof | |
| Al Hashimi | Building 360-degree VR video for AquaFlux and Epsilon research instruments | |
| KR101199643B1 (en) | System for virtual advertising space | |
| Nam et al. | Materialization of interactive stereoscopic artwork based on hand-painted images | |
| CN118096982B (en) | Construction method and system of fault inversion training platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |