KR20210001971U - Panoramic augmented reality system - Google Patents

Abstract

The present invention relates to a reality simulation panoramic system, and includes a panoramic photographing device as a main configuration. The panoramic photographing device is information-connected to the network access space, the network access space is for storing and retrieving the panoramic image captured by the panoramic photographing device, the network access space is connected to at least one electronic device, and the electronic device has a mobile app , an AR processing unit and at least one image control element are installed. Accordingly, when panoramic image recording is required, a panoramic photographing device can be installed in the shooting environment, and the captured panoramic image can be transmitted to a network access space through the panoramic photographing device, and the audience reads the panoramic image through a mobile app, In conjunction with the AR processing unit, it is possible to process and create a 3D stereoscopic space. The spectator can also generate a display screen in a corresponding 3D stereoscopic space by sensing the movement position and direction desired by the user using the image control element. Accordingly, it is possible to give the user a feeling of arbitrarily traversing in a 3D stereoscopic space, and to change the corresponding display screen continuously non-intermittently to achieve a realistic entertainment effect.

Description

Reality Simulation Panorama System {PANORAMIC AUGMENTED REALITY SYSTEM}

The present invention relates to a reality simulation panorama system, and in particular, to a reality simulation panorama system that is more convenient to view and use and can give a sense of presence.

Internet live broadcasting is a form of entertainment in which instant images are publicly played on the Internet according to the development of online AV platforms. Internet live broadcasting is still most commonly used today for relay broadcasting of entertainment-related content, and the main character who appears or hosts an Internet live broadcast is generally referred to as a "live host", "broadcaster", "anchor" or "live master". The biggest difference between live broadcasting and the conventional method of uploading their own video film is that in live broadcasting, the audience can have an immediate interaction with the anchor through a bullet screen message, and the anchor can immediately adjust the program content based on the audience's feedback or It can entertain the audience.

However, the general live broadcasting method is a method of interactively with the audience by shooting the screen of the live host or the computer using a single camera. However, with the progress of the times, shooting methods are becoming more and more diversified. For example, by shooting in a VR method, the audience can watch the corresponding 3D screen. However, since such VR technology is cumbersome to use, it is necessary to create a 3D screen corresponding to a screen of various angles by adding a plurality of photographing devices to a live broadcast stage and taking pictures alternately. However, this method is cumbersome to use and greatly increases the cost of live broadcasting.

In addition, since the general VR technology can achieve a complete viewing effect only when used with head-mounted reproduction glasses, the demand for viewing devices and viewing environments is relatively high. And if you use a 360-degree panoramic photo, you can only view it by reading it, and you have to rotate or zoom in and out of the photo in a slide way, and you cannot create a sense of presence effect for the audience.

The main purpose of the present invention is to create a 3D stereoscopic space by processing the panoramic image through the AR processing unit so that the user can form the corresponding display screen together with the image control element in the 3D stereoscopic space, which is more convenient and provides a sense of presence. to achieve an entertainment effect.

In order to achieve the above object, the main configuration of the present invention is at least one panorama photographing device, a network access space connected to the panorama photographing device, at least one electronic device connected to the network access space, a mobile app installed in the electronic device, and an electronic device. and an AR processing unit installed therein, and at least one image control element installed in the electronic device.

With the above configuration, it is possible to install a panoramic photographing apparatus in a photographing scene, take an image in the photographed scene as a panoramic image by using the panoramic photographing apparatus, and then transmit it back to the network access space. If the spectator wants to watch, it accesses the network access space through the mobile app of the electronic device, reads the related panoramic image, analyzes and calculates the panoramic image through the AR processing unit, and performs 3D stereoscopic space corresponding to the shooting scene. , and by detecting the position and direction of the spectator in the 3D stereoscopic space together with the image control element, the corresponding display screen may be displayed on the electronic device. In addition, when the spectator wants to move the display screen in the 3D stereoscopic space, the image control element detects the movement direction and the movement position desired by the user, and generates a corresponding display screen in the 3D stereoscopic space together with the AR processing unit. displayed on the device. Accordingly, as long as the spectator has the electronic device, he or she can view the corresponding display screen at an arbitrary location through the electronic device and move the display screen arbitrarily in the 3D stereoscopic space as if in a shooting scene. effect can be achieved.

According to the above technology, it is possible to have practical and inventive step with the above advantages by solving the cumbersome problem of using the conventional panoramic video.

1 is a perspective perspective view according to a preferred embodiment of the present invention.
2 is a schematic diagram of a building block according to a preferred embodiment of the present invention.
3 is a schematic diagram (1) of live broadcasting according to a preferred embodiment of the present invention.
4 is a schematic diagram (2) of live broadcasting according to a preferred embodiment of the present invention.
5 is a schematic diagram (3) of live broadcasting according to a preferred embodiment of the present invention.
6 is a schematic diagram (4) of live broadcasting according to a preferred embodiment of the present invention.
7 is a reading schematic diagram according to a preferred embodiment of the present invention.
8 is a schematic diagram (1) of live broadcasting according to another preferred embodiment of the present invention.
9 is a schematic diagram (2) of live broadcasting according to another preferred embodiment of the present invention.
10 is a schematic diagram (3) of live broadcasting according to another preferred embodiment of the present invention.
11 is a perspective perspective view according to another preferred embodiment of the present invention.
12 is a schematic diagram of a building block according to another preferred embodiment of the present invention.
13 is a schematic diagram of live broadcasting according to another preferred embodiment of the present invention.
14 is a schematic diagram of sponsorship according to another preferred embodiment of the present invention.
15 is a schematic diagram of a dialogue according to another preferred embodiment of the present invention.
16 is a perspective perspective view according to another preferred embodiment of the present invention.
17 is a schematic diagram (1) of live broadcasting according to another preferred embodiment of the present invention.
18 is a schematic diagram (2) of live broadcasting according to another preferred embodiment of the present invention.
19 is a schematic diagram (3) of live broadcasting according to another preferred embodiment of the present invention.

1 to 7 , these drawings are a perspective perspective view to a reading schematic diagram according to a preferred embodiment of the present invention. As can be seen clearly from the drawings, the present invention provides at least one panoramic photographing device 1, a network access space 2, at least one electronic device 3, an AR processing unit 32 installed in the electronic device 3, at least one image control element (33) installed in the electronic device (3),

In this embodiment, as the panoramic photographing device 1, a 360-degree camera is taken as an example,

In this embodiment, a network server is taken as an example of the network access space 2, and the panorama photographing device 1 is connected to information so that the panoramic image captured by the panoramic photographing device 1 is transferred to the network access space 2 ), and in this embodiment, the panoramic photographing apparatus 1 itself has a wireless connection element 11, and in this embodiment, an antenna having a network connection function is used as the wireless connection element 11 as an example. but not limited thereto, for transmitting the panoramic image to the network access space (2);

The electronic device 3 is any one of a mobile phone or a tablet, and the form is not limited, and a mobile app 31 is provided in the electronic device 3 , and the network access is performed through the mobile app 31 . It is connected to the space 2, and the mobile app 31 is an APP installed in the electronic device 3 as an example;

The AR processing unit 32 may process a panoramic image by AR technology (Augmented Reality), and the technical details are not limited, and may be either ARKit or ARcore;

The image control element 33 is not limited, and may be any one of a gyroscope or a GPS location tracking device, and its shape is not limited, and a gyroscope is exemplified in this embodiment.

Through the above description, it is possible to grasp the configuration of the present technology, and through the corresponding combination of the configuration, it is possible to have the advantage of more convenient use. A detailed description thereof will be provided below.

The panoramic photographing apparatus 1 may be applied to a photographing scene. As shown in FIG. 3 , in this embodiment, the application is taken as an example of an on-site live concert, and the number of the panoramic photographing apparatus 1 is exemplified, but the number is not limited, and the panoramic photographing apparatus 1 is used for the concert Panoramic images of the entire stage are captured by placing them at respective positions facing each other on the stage. Since the panoramic shooting device 1 is more complete than the angle that a general shooting device can shoot, all screens of the entire concert can be recorded without blind spots, and by using fewer shooting devices, the required angles and screens can be more It is possible to completely photograph and transmit the panoramic image to the network access space 2 using the wireless connection element 11 . In addition, the spectators can read the panoramic image by accessing the network access space 2 through the mobile app 31 of their electronic device 3 (a mobile phone is taken as an example in this embodiment). In addition, the AR processing unit 32 in the mobile phone generates a 3D stereoscopic space corresponding to the shooting site by performing an arithmetic processing operation on the panoramic image through AR technology. In this case, the user can detect the user's position in the 3D stereoscopic space in association with the image control element 33 , and detect the display screen at the position in association with the AR processing unit 32 and display it on the electronic device 3 . have. If it is assumed that the spectator A is in the stage position shown in FIG. 3, it is detected through the AR processing unit 32 and the image control device 33, and a corresponding screen is obtained and sent to the electronic device 3 as shown in FIG. can be displayed

If the spectator A moves in its own space, the image control element 33 detects the moving distance, position and direction of the spectator A, and in conjunction with the network access space 2, the relative display screen when moving may be updated frequently and continuously updated and displayed on the electronic device 3 non-intermittently. If the spectator A moves to the location shown in FIGS. 5 and 6 , the display screen of the location can be displayed on the electronic device 3 , thereby giving the audience A feeling as if they were directly participating in the concert site. You can move arbitrarily across the screen, and you can see the corresponding display screen.

The method used in the present invention is to directly process and operate the screen of the panoramic image through AR technology to create a 3D stereoscopic space so that the user can move arbitrarily in connection with the image control element 33 . Although AR technology generates 3D stereoscopic space by performing distance and orientation calculations on images or videos, conventionally sold in the market only transmits 3D stereoscopic pictures to mobile phones and uses the mobile phone's own photo viewing function. Then, the slide operation is performed to enlarge or reduce the item on the display screen as if the item on the screen is enlarged and moved in front of the user. However, the application method of the present invention is different from this, and the display screen on the electronic device 3 can be traversed in a 3D stereoscopic space similar to the field environment in connection with the AR processing unit 32 through the image control element 33 directly. , when the display screen moves, the display screen is continuously updated non-intermittently and displayed on the electronic device. This allows the user to feel as if he or she has walked directly in front of the article. In addition, the distance between each article in the corresponding 3D stereoscopic space on the display screen can be changed. Accordingly, the viewing angle that the user can observe is completely different, and if the standing position is different as shown in FIGS. 3 to 6 , the viewing angle is also completely different. In addition, compared to VR technology, the present invention does not require any head-mounted device to view, and only the electronic device 3 having the AR processing unit 32 and the image control element 33 is used to view the viewing. The effect can be achieved, and the effect of allowing the spectators to move randomly in the panoramic image as if they are in a real environment under fairly simple and easy conditions can be achieved, and a better entertainment function can be achieved.

In addition, referring to FIG. 7 in conjunction with FIG. 7 , the panoramic image is stored in the network access space 2 after the live broadcast ends, and the user can read it only when necessary, for example, [XXX concert VOD, XXX concert live broadcast site, XXX concert panorama photos]. Accordingly, even when the user cannot watch the live broadcast immediately, a sense of presence can be achieved likewise by selecting the effect of viewing the VOD or a static picture.

In addition, referring to FIGS. 8 to 10 simultaneously, these drawings are a live broadcast schematic diagram (1) to a live broadcast schematic diagram (3) according to another preferred embodiment of the present invention. As can be seen clearly from the drawings, the present embodiment is substantially the same as the above-described embodiment. In this embodiment, a cooking lecture is taken as an example, thereby indicating that the method of use of the present invention is not limited. In the present embodiment, a plurality of panoramic photographing devices 1a are installed next to the chef B, and the cooking screen of the chef B is photographed. In addition, the spectator A may view the electronic device 3a while arbitrarily moving in his/her own space. Thereby, when the spectator (A) moves next to the cook (B), it is possible to observe the operation of the cook (B) from various angles, and to learn the cooking point more completely.

In addition, looking in conjunction with FIGS. 11 to 15, these drawings are a perspective perspective view or a schematic diagram of a dialogue according to another preferred embodiment of the present invention. As can be clearly seen from the drawings, the present embodiment is substantially the same as the above embodiment. In this embodiment, the panoramic photographing device 1b is a 360-degree camera on the electronic product 4b as an example, and the electronic product 4b is a mobile phone as an example, and the captured panoramic image is transferred to the network access space through the electronic product 4b. Transmission in (2b) is an example. This indicates that the type of the panoramic photographing apparatus 1b is not limited.

In addition, in the present embodiment, the mobile app 31b includes a user support unit 311b and a message input unit 312b, and the audience can interactively proceed through the units during live broadcasting. For example, if the spectator wants a sponsorship, click the sponsorship icon 313b generated by the user sponsorship unit 311b, and proceed with a financial account transfer through a virtual transaction system such as a credit card or Internet banking, You can sponsor a live host. Alternatively, when a message corresponding to the chat window 314b formed by the message input unit 312b is input, the message is transmitted to the electronic product 4b of the live host, so that the live host can view the related message (eg, For example, asking the chef for cooking-related know-how), thereby greatly increasing the interactivity and enjoyment of live broadcasting through the above-described operation.

In addition, looking in conjunction with FIGS. 16 to 19, these drawings are a perspective perspective view or a live broadcast schematic (3) according to another preferred embodiment of the present invention. As can be clearly seen from the drawings, the present embodiment is substantially the same as the above embodiment. In the present embodiment, the number of the image control elements 33c is two, and one of them is a gyroscope and can sense the moving position and direction of the electronic device 3c. The other image control element 33c is an operation module (the operation module of this embodiment is not limited thereto, but is not limited to the cross keys 34c and the movable screen 35c displayed on the electronic device 3c). For example, a reception controller capable of receiving a control signal from a remote controller) is taken as an example. The electronic device 3c of the present embodiment is a mobile phone as an example, but may be a non-portable screen such as a TV or a computer screen. Accordingly, even when the user's moving space is insufficient, the user can move to the display screen in the corresponding 3D stereoscopic space through the remaining manipulation modules. Thereby, the user can change the display screen by moving the electronic device 3c, and when walking to a position that cannot be moved, the viewing angle can be changed by sliding the screen on the movable screen 35c as shown in FIGS. 18 to 19 . You can change and move forward by pressing the cross key 34c on the electronic device 3c. As a result, the convenience of use of the present invention is greatly increased, and a phenomenon in which the entertainment effect is reduced can be prevented from occurring because the movable place of the spectators themselves is too small.

1, 1a, 1b panoramic shooting device
11 wireless connection element
2, 2b network access space
3, 3a, 3c electronics
31, 31b Mobile App
311b User Sponsored Unit
312b message input unit
313b Sponsorship Icon
314b dialog
32 AR processing unit
33, 33c image control element
34c cross key
35c operation screen
4b electronics
A crowd
B cook

Claims (5)

In the reality simulation panorama system,
A reality simulation panorama system mainly comprising at least one panoramic photographing device, a network access space, a mobile app, an AR processing unit, and at least one image control element,
the network access space is information-connected to the panoramic photographing device, to receive the panoramic image captured by the panoramic photographing apparatus, and at least one electronic device accesses the network access space to read the panoramic image;
the mobile app is installed in the electronic device, and the electronic device reads the panoramic image through the mobile app;
the AR processing unit is installed in the electronic device, and generates a 3D stereoscopic space by processing the panoramic image captured by the panoramic photographing device through AR technology;
The image control element is installed in the electronic device, and controls a position corresponding to a display screen of the electronic device, thereby providing a display screen corresponding to a position in the 3D stereoscopic space to the electronic device in conjunction with the AR processing unit. and when the image control element detects that the electronic device has moved to the display screen in the corresponding 3D stereoscopic space, the AR processing unit is coupled with the image control element to display the display screen in the corresponding 3D stereoscopic space after the position is changed A real-time simulation panorama system that instantly refreshes and non-intermittently. According to claim 1,
The image control element is connected to a manipulation module to receive a control signal from the manipulation module. According to claim 1,
The panoramic image received by the network access space is any one of an instant live image captured by the panoramic photographing device, a pre-recorded image, or a static photo image. According to claim 1,
The panoramic photographing device is a reality simulation panorama system connected to an electronic product. According to claim 1,
The mobile app is a reality simulation panorama system comprising a user sponsorship unit provided for the spectators to proceed with sponsorship, and a message input unit provided for the spectators to input a message.

Images