KR102736563B1 - Hologram content generating apparatus, hologram content integration control sysetm having the same and operating method thereof - Google Patents

Abstract

According to the present invention, a hologram content integrated control system may include a sensor unit that detects at least one of a user's face, motion, and voice, a hologram content generation device that generates hologram content that can interact with the user using the detected information, a display device that displays the generated hologram content, and a rotation module that rotates the display device.

Description

Holographic content generation apparatus, holographic content integration control system including the same, and operating method thereof {HOLOGRAM CONTENT GENERATING APPARATUS, HOLOGRAM CONTENT INTEGRATION CONTROL SYSETM HAVING THE SAME AND OPERATING METHOD THEREOF}

The present invention relates to a holographic content generation device, a holographic content integration control system including the same, and an operating method thereof.

In general, holographic display technology is an ideal fully realistic 3D display technology that stereoscopically expresses objects in a 3D space. Specifically, holographic display technology is a technology that reproduces the wave-front generated by a given object as it is, thereby giving the same effect as the actual existence of the object to the human eye.

Registered patent: 10-1670328, Registration date: October 24, 2016, Title: Realistic image display device using multiple real-time image acquisition cameras and image control recognition method using the same Chinese Patent: CN 105929962, Registration Date: September 7, 2016, Title: A kind of 360 ° of holographic real-time interaction methods

The purpose of the present invention is to provide a holographic content generation device for displaying a full stereoscopic image capable of interaction between 3D data acquisition and user-reconstructed images, a holographic content integration control system including the same, and an operating method thereof.

According to an embodiment of the present invention, a hologram content integrated control system may include a sensor unit that detects at least one of a user's face, motion, and voice; a hologram content generation device that generates hologram content that can interact with the user using the detected information; a display device that displays the generated hologram content; and a rotation module that rotates the display device.

In an embodiment, the sensor unit may include a facial recognition sensor that recognizes the positions of the user's two eyes.

In an embodiment, the face recognition sensor is characterized by distinguishing a movement direction, a movement angle, and a movement speed of the user.

In an embodiment, the rotation module is characterized in that it is controlled based on data detected by the face recognition sensor.

In an embodiment, the rotation module is characterized in that it is controlled in rotation angle unit steps of 0.35° (360°/1024) or less.

In an embodiment, the sensor unit may include a voice sensor that recognizes the user's voice.

In an embodiment, the rotation direction, rotation angle, or rotation speed of the hologram content is determined based on data detected by the voice sensor.

In an embodiment, the sensor unit may include a motion sensor that recognizes a movement or gesture of the user.

In an embodiment, the rotation direction, rotation angle, or rotation speed of the hologram content is determined based on data detected by the motion sensor.

In an embodiment, the sensor unit is characterized by including a haptic sensor that detects the tactile sensation of the holographic content.

In an embodiment, the holographic content generating device is characterized by generating binocular holographic data providing a wide viewing angle.

In an embodiment, the holographic content generation device is characterized by acquiring 3D for an image or object selected from a left-eye sensor and a right-hand sensor.

In an embodiment, the holographic content generation device is characterized by acquiring 3D for a selected image or object in left-eye time and right-eye time.

A method of operating a hologram content generation device according to an embodiment of the present invention may include: a step of acquiring binocular content for an object; a step of generating left-eye CGH (Computer-Generated Hologram) data and right-eye CGH data corresponding to the binocular content; and a step of correcting the left-eye CGH data and the right-eye CGH data according to a user's face, motion, or voice.

In embodiments, the step of detecting the face, motion or voice of the user may be further included.

In an embodiment, the method may further include a step of transmitting the left-eye CGH data and the right-eye CGH data to a display terminal.

In an embodiment, the step of acquiring binocular content for the object may include the step of acquiring color information and depth information from a left-eye image for the object; and the step of acquiring color information and depth information from a right-eye image for the object.

In an embodiment, the step of acquiring binocular content for the object may include the step of acquiring color information and depth information at a left eye time for the object; and the step of acquiring color information and depth information at a right eye time for the object.

A device for generating holographic content according to an embodiment of the present invention includes at least one processor and a memory storing at least one instruction executed by the at least one processor, wherein the at least one instruction is executed by the at least one processor to: acquire binocular content for an object; generate left-eye CGH (Computer-Generated Hologram) data and right-eye CGH data corresponding to the binocular content; and correct the left-eye CGH data and the right-eye CGH data according to a user's face, motion, or voice.

In an embodiment, the at least one instruction is characterized in that it is executed by the at least one processor to read left-eye CGH data and right-eye CGH data from a memory storing an initial value, and add an option value corresponding to data corresponding to the user's face, motion or voice to the read left-eye CGH data and right-eye CGH data.

A hologram content creation device according to an embodiment of the present invention, a hologram content integration control system including the same, and an operation method thereof can overcome the limitations of conventional manual hologram content creation centered on computer hologram creation, and at the same time, can escape from a complex device and a long-time content calculation process for creating a conventional hologram.

A holographic content generating device according to an embodiment of the present invention, a holographic content integrated control system including the same, and an operating method thereof can provide wide viewing angle characteristics according to binocular stereoscopic holography to comfortably view restored images even when using an existing flat-panel holographic terminal.

According to an embodiment of the present invention, a holographic content creation device, a holographic content integrated control system including the same, and an operating method thereof can enable users to actively interact with real-time 3D content using only voice commands, facial positions, or user gesture commands of various types (realistic or CG) of 360-degree omnidirectional holographic content.

A holographic content generating device according to an embodiment of the present invention, a holographic content integrated control system including the same, and an operating method thereof enable viewers to experience a fully realistic image in a wider viewing space and more comfortably.

A holographic content generating device according to an embodiment of the present invention, a holographic content integrated control system including the same, and an operating method thereof are suitable for personal portable mobile applications because they enable weight reduction of the system and high-speed computational processing, while also providing viewers with an immersive and 3D realism-enhancing effect.

A holographic content generating device according to an embodiment of the present invention, a holographic content integrated control system including the same, and an operating method thereof can enable interaction between a user and a restored stereoscopic image in real time.

The drawings attached below are provided to help understand the present embodiment, and provide examples together with detailed descriptions. However, the technical features of the present embodiment are not limited to specific drawings, and the features disclosed in each drawing may be combined with each other to form a new embodiment.
FIG. 1 is a drawing exemplarily showing a hologram content integration control system (10) according to an embodiment of the present invention.
FIG. 2 is a drawing exemplarily showing a holographic content integration control system (10) for interaction between holographic content and a user.
FIG. 3 is a flowchart exemplarily showing an operation method of a hologram content creation device (100) according to an embodiment of the present invention.
FIG. 4a is a diagram showing an embodiment of a process for generating and encoding interactive holographic content between user restored images in a holographic content generating device (100) according to an embodiment of the present invention.
FIG. 4b is a diagram showing another embodiment of a process for generating and encoding interactive holographic content between user restored images in a holographic content generating device (100) according to an embodiment of the present invention.
FIG. 5a is a drawing showing an example of an integrated processing process (SW) of hologram content in an integrated control system (10) for implementing hologram content according to an embodiment of the present invention.
FIG. 5b is a drawing showing another embodiment of an integrated processing process (SW) of hologram content in an integrated control system (10) for implementing hologram content according to an embodiment of the present invention.
FIG. 6 is a drawing exemplarily showing a display device (150) and holographic content being synchronously updated along a user's face based on facial recognition of the user.
FIG. 7 is a drawing exemplarily showing a display device (150) and holographic content rotating synchronously according to a user voice command based on user voice recognition.
Figure 8 is a drawing exemplarily showing rotation of holographic content according to a gesture command based on recognition of a user's gesture.
FIG. 9 is a drawing exemplarily showing a hologram content generation device (1000) according to an embodiment of the present invention.

Below, the contents of the present invention will be described clearly and in detail using drawings to a degree that a person skilled in the art can easily practice the present invention.

The present invention can have various modifications and can take various forms, and specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to a specific disclosed form, but it should be understood that it includes all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention. The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms.

The above terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to the other component, but there may also be other components in between. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise.

In this application, it should be understood that the terms "include" or "have", etc., are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof implemented, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which this invention belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and shall not be interpreted in an idealistic or overly formal sense, unless expressly defined in this application.

A holographic content creation device according to an embodiment of the present invention, a holographic content integration control system including the same, and an operating method thereof can create content to acquire 3D data of a real-world-based object and display a full image as a full stereoscopic image processing technology.

Recently, with the development of 3D display technology, it is possible to express objects with perfect parallax and depth, thereby eliminating eye fatigue and dizziness caused by accommodation-convergence mismatch problems. Furthermore, holographic display technology can view natural images as if viewing the real world, depending on the movement of the viewpoint. However, there are several requirements for implementing this holographic display technology.

First, since 360-degree image content for holography based on various real objects or CG (Computer Graphic) has a very large amount of data, it is very difficult to process the image in real time and optically restore it. Second, since it is developed by restoring an existing image due to a large burden on the amount of computational processing, content is being produced in which the viewer observes in a passive position. Therefore, it is necessary to develop content that allows for an interactive experience between the user and the content. Third, even if the content for holography produced in this way is prepared, existing hologram devices (150) or spatial light modulators (SLMs) that can display the content have complex optical systems. Existing hologram devices (150) or spatial light modulators (SLMs) have complex optical systems in order to create light with uniform intensity and excellent coherence, so the system volume is large, and they are generally built on an optical table. Accordingly, there is an increasing need to develop a lightweight, wearable type mobile hologram device (150) that can be easily held by hand or worn on the user's face.

Meanwhile, in order to realize a wide-viewing-angle hologram display, a terminal with a pixel pitch of less than 1μm is required. In comparison, the pixel pitch of the panel using the current commercial LC (Liquid Crystal) is limited to several μm to several tens of μm. This size provides a viewing angle of less than 5 degrees. Therefore, holograms are reproduced through a system equipped with separate expensive devices that involve large-capacity and high-precision data processing technology such as eye-tracking technology.

According to an embodiment of the present invention, a holographic content generating device, a holographic content integrated control system including the same, and an operating method thereof provide a real-time interaction function between a user and a device (150) for restoring a holographic image based on 360-degree 3D (Dimension) image information, while at the same time easily producing a wide-viewing angle holographic content. In particular, holographic content capable of synesthetic (visual, tactile, auditory, gesture/motion)-based interaction between a user and a restored stereoscopic image can be implemented in order to provide immersive and 3D realism-enhanced effects to viewers while being suitable for a personally portable VR (Virtual Reality)/AR (Augmented Reality) terminal or a mobile/wearable application.

A holographic content generating device according to an embodiment of the present invention, a holographic content integrated control system including the same, and an operating method thereof can perform wide-angle display and interaction by linking holographic data with a user's voice recognition and gesture recognition information.

A hologram content generation device according to an embodiment of the present invention, a hologram content integration control system including the same, and an operation method thereof can perform an operation for high-speed hologram generation using an RGB-depth map as input information and a CGH (computer-generated hologram) calculation formula based on an FFT (fast-Fourier transformation) such as mathematical expression 1 below.

When a three-dimensional space including a 3D scene is sliced into several layers, and the hologram plane (H), the observing view-window plane (VW), and each layered layer are cut parallel to each other, point clouds that are almost continuously distributed can be assigned to the layer that is closest to them. Then, the Fourier transform can be performed as follows using the FFT algorithm to calculate complex fields in the hologram plane.

Here (u, v), (x _i , y _i ), U _i , f, , and d _i represent the viewer's observation plane, the i-th layer of the 3D scene, the object field of the i-th layer, the focal length of the field lens, the wavelength of the illumination light, and the viewing distance from the hologram plane, respectively. The depth-map-based CGH calculated in this way can reconstruct the original 3D scene in space. A viewer who places his eyes at the view-window position and observes the scene can see the optically reconstructed 3D scene.

FIG. 1 is a drawing exemplarily showing a hologram content integration control system (10) according to an embodiment of the present invention. Referring to FIG. 1, the hologram content implementation system may include a hologram content creation device (100), a display device (150), and a sensor unit (150).

A hologram content creation device (100) may include a 3D information acquisition unit (110), a hologram information creation unit (120), an encoding unit (130), and a content control unit (140).

The 3D information acquisition unit (110) may be implemented to acquire data capable of providing 3D information of a selected scene or object, or RGB color and depth map information. Here, the format of the data may include a format in which information is stored in the form of a point cloud, or a format in which information is stored in the form of RGB-Depth.

The hologram generation unit (120) can be implemented to calculate (generate) a computer-generated hologram (CGH) of complex values and store the generated data in a specific memory folder.

In an embodiment, the hologram generation unit (120) can generate a computer-generating hologram (CGH) by inputting a format in the form of RGB-Depth. In an embodiment, the hologram generation unit (120) can generate a computer-generating hologram (CGH) by inputting a format in the form of a point cloud.

The encoding unit (130) can be implemented to correct the CGH file calculated by the hologram generation unit (120) to suit the characteristics of the display terminal and to collate the corrected file into a form that is easy to transmit.

The content control unit (140) may be implemented to transmit data composed of one or at least two hologram pairs selected from among the hologram data generated/stored in the hologram information generation unit (120) and the encoding unit (130).

The content control unit (140) can receive the selected hologram data transmitted through the transmitter, and control the 3D information acquisition unit (110), the hologram information generation unit (120), and the encoding unit (130) to convert or additionally encode the selected hologram data to be suitable for the display device (150) to be used. In an embodiment, the two selected hologram pairs can be hologram data suitable for the left and right eyes of the viewer, respectively, considering the spatial position to be restored and the arrangement of the optical device.

In an embodiment, the image contents for the left and right eyes can be generated/encoded based on the depth value (z) and the binocular spacing (x-y) of the holographic image restored in three-dimensional space to provide appropriate parallax to the left and right eye positions. That is, the image contents for the left and right eyes selected by considering the spatial alignment can be generated or encoded.

The display device (150) can be implemented to upload hologram data selected from the hologram content creation device (100) and restore a hologram image in a three-dimensional space by illuminating it with appropriately prepared light in the uploaded state.

In an embodiment, the display device (150) can appropriately display (upload) and restore hologram content data created in the encoding unit (130).

In an embodiment, the display device (150) may be combined with a rotation module including a rotation function and a rotation angle/speed/direction control function appropriately in response to a command from the sensor unit (160).

In an embodiment, the display device (150) may be combined with a rotation module and a content control unit (140) that includes a function for updating content in response to a change in the degree of rotation angle/speed/direction according to a command from a sensor unit (160).

In an embodiment, the display device (150) may have an interface with the rotation module and an interface with the content control unit. In an embodiment, the display device (150) may be combined with the rotation module and the content control unit (140) to include a function of observing a holographic restoration image in the area of the user's eye position.

In an embodiment, an angular unit for a rotational step in the rotation module may be equal to or smaller than a minimum rotational angle used to acquire 360° digital 3D data in the 360° omnidirectional content generation step. For example, the unit step of the rotation module may be 15° or less, or more precisely, 0.35° (= 360°/1024) or less.

FIG. 2 is a drawing exemplarily showing a hologram content integration control system (10) for interaction between hologram content and a user. Referring to FIG. 2, the hologram content integration control system (10) may include a display device (150) installed on a rotation module (170). The display device (150) may include a face recognition sensor (161), a voice sensor (162), a motion sensor (163), or a haptic sensor (164).

In an embodiment, the display device (150) may include a spatial light modulator and a user's face tracking or eye tracking (161) for recognizing a user's face (face). The face recognition sensor (161) may include a function for recognizing the positions of the user's eyes when the user's face (or eyes) moves in the left or right direction while being at an appropriate distance from the sensor. In an embodiment, data recognized by the face recognition sensor (161) may distinguish not only the face position but also the rotation (movement) direction, the rotation (movement) angle, the rotation (movement) speed, etc.

In an embodiment, the display device (150) may include a spatial light modulator and a user's voice sensor (162) that recognizes the user's voice. The voice sensor (162) monitors the voice of the user using the display device (150) and can obtain/recognize information such as a rotation direction, a rotation angle, and a rotation speed according to the content indicated by the voice. In an embodiment, data recognized by the voice sensor (162) may distinguish a rotation direction, a rotation angle, a rotation speed, and the like.

In an embodiment, the display device (150) may include a spatial light modulator and a motion sensor (user's motion or gesture sensor; 163) that recognizes a user's motion/gesture. The motion sensor (163) may monitor a user's gesture using the display device (150) and obtain information such as a location of the motion, a direction of the motion (up/down/left/right), and a speed of the motion. At this time, the obtained information may be input into a feedback program to selectively update hologram data generated by the hologram content generation device (100). In an embodiment, hologram data matching the information obtained by the feedback program may be selected and read.

In an embodiment, the display device (150) may include a spatial light modulator and a sensor (user'haptic gesture sensor; 164) that provides a haptic function to a user. The display device (150) based on the haptic sensor (164) may provide various sensory information such as tactile and touch information to a user using the same. At this time, the acquired information may be input into a feedback program to selectively update hologram data generated by the hologram content generation device (100). In an embodiment, hologram data matching the information acquired by the feedback program may be selected and read.

In an embodiment, the hologram data may be in a stand-by state after pre-calculation of CGH is completed for the given 3D content (so that the CGH can be updated in real time). In an embodiment, the read data may be transmitted to the content control unit and then uploaded to the hologram display device (150) via the transmission unit in the content control unit. Thereafter, in the uploaded state, the hologram image may be restored in a three-dimensional space by illuminating the display device (150) with appropriately prepared light.

Meanwhile, the integrated control system (10) may include a rotational module (170) having a mount capable of connecting the display device (150) to rotate the display device (150) and a rotation control unit. In an embodiment, the rotational module control unit may be linked to voice recognition (or face recognition) data by a user to rotate the content and the display device (150) in a direction, rotation speed, and rotation angle corresponding to the voice (or face location) information under synchronization conditions.

In an embodiment, when the display device (150) rotates and the hologram displayed on the display device (150) is displayed in synchronization, as a result, the restored image of the hologram can be observed as an optimal hologram restored image in the user's binocular area.

In an embodiment, voice information input by the user to the voice sensor (162) can perform appropriate voice commands (left, right, fast, slow, more, etc.) corresponding to the direction/speed/angle at which the center of the user's face moves.

An integrated control system (10) for generating hologram content according to an embodiment of the present invention can overcome the limitations of manual hologram content creation centered on conventional computer hologram creation, and at the same time, can escape from complex devices and long-time content calculation processes for creating conventional holograms.

An integrated control system (10) for generating holographic content according to an embodiment of the present invention may include binocular stereoscopic holography that provides a wide viewing angle (e.g., an experimentally implemented angle of ±60 degrees or more) characteristic to comfortably view restored images even when using an existing flat-panel holographic terminal.

An integrated control system (10) for generating holographic content according to an embodiment of the present invention enables users to actively interact with real-time 3D content using only voice commands, facial positions, or user gesture commands, using various types of 360-degree omnidirectional holographic content (realistic or CG).

An integrated control system (10) for generating holographic content according to an embodiment of the present invention allows viewers to experience a fully realistic image in a wider viewing space and more comfortably.

An integrated control system (10) for generating holographic content according to an embodiment of the present invention is suitable for personal portable mobile applications because it enables lightweight system and high-speed computational processing, while providing viewers with an immersive and 3D reality-enhancing effect, thereby enabling real-time interaction between the user and the restored stereoscopic image.

FIG. 3 is a flowchart exemplarily showing an operation method of a hologram content generation device (100) according to an embodiment of the present invention. Referring to FIGS. 1 to 3, the hologram content generation device (100) can operate as follows.

Binocular content (left-eye content/right-eye content) for an object can be acquired (S110). Thereafter, a computer-generated hologram (CGH) corresponding to the acquired binocular content can be calculated (S120). The CGH can be separated into a left-eye CGH and a right-eye CGH (S130). Thereafter, the separated left-eye CGH can be uploaded to a spatial light modulator (SLM) for the left eye of a display device (150), and the separated right-eye CGH can be uploaded to a spatial light modulator (SLM) for the right eye of the display device (150).

FIG. 4a is a diagram showing an embodiment of a process for generating and encoding interactive holographic content between user restored images in a holographic content generating device (100) according to an embodiment of the present invention.

The 3D information acquisition unit (110) can acquire 3D information that can provide 3D information of an image (scene) or object selected from the left-eye sensor and the right-eye sensor. For example, left-eye RGB color information and left-eye depth information (depth map) can be acquired from the left-eye image. In addition, right-eye RGB color information and right-eye depth information (depth map) can be acquired from the right-eye image. Thereafter, the 3D information acquisition unit (110) can combine the left-eye RGB color information and the left-eye depth information, and can combine the right-eye RGB color information and the right-eye depth information. Thereafter, the hologram information generation unit (120) can receive the format of the combined RGB-Depth form and generate corresponding left/right-eye CGH.

Meanwhile, in Fig. 4a, the left-eye image and the right-eye image are acquired from separate image sensors. However, it should be understood that the 3D information acquisition unit of the present invention is not limited thereto. The present invention may acquire the left-eye image from one image sensor at different times and then acquire the right-eye image thereafter.

FIG. 4b is a diagram showing another embodiment of a process for generating and encoding holographic content that can interact between user restored images in a holographic content generating device (100) according to an embodiment of the present invention. Referring to FIG. 4b, a left-eye image may be acquired and a CGH may be generated during a left-eye time (T_Left) with a time difference from one image sensor, and a right-eye image may be acquired and a CGH may be generated during a right-eye time (T_Right).

Meanwhile, the process of operating in an integrated solution system capable of interacting between user information and restored images according to an embodiment of the present invention is as follows.

FIG. 5A is a diagram showing an embodiment of an integrated processing process (SW; software) of hologram content in an integrated control system (10) for implementing hologram content according to an embodiment of the present invention. Referring to FIG. 5A, the integrated processing process may operate in conjunction with a user recognition sensor, a motion sensor, and a user voice recognition sensor. In an embodiment, a rotation module may rotate in response to user face information recognized by a user recognition sensor. In an embodiment, hologram content may rotate in response to information recognized by a motion sensor. In an embodiment, hologram content may rotate in response to voice information recognized by a voice sensor.

Meanwhile, the integrated processing process can also add interactions that are triggered by haptic sensors.

FIG. 5b is a diagram showing another embodiment of an integrated processing process (SW) of hologram content in an integrated control system (10) for implementing hologram content according to an embodiment of the present invention. Referring to FIG. 5b, the integrated processing process can add an interaction linked by a haptic sensor compared to that of FIG. 5a. For example, in addition to image rotation in FIG. 5b, content update functions in the form of image size change, transfiguration, etc. can be added.

FIG. 6 is a drawing showing an example of a display device (150) and holographic content being synchronously updated along the user's face based on facial recognition of the user. Referring to FIG. 6, the display device (150) is rotated accordingly by tracking the user's pupil, and at the same time, the holographic content can be displayed in response to the rotation.

FIG. 7 is a drawing exemplarily showing that a display device (150) and holographic content are rotated in synchronization according to a user's voice command based on a user's voice recognition. Referring to FIG. 7, holographic content can be rotated according to a user's voice command.

Fig. 8 is a drawing exemplarily showing rotation of hologram content according to a gesture command based on recognition of a user's gesture. Referring to Fig. 8, hologram content can be moved according to a user's gesture command (e.g., moving the right hand to the left or moving the left hand to the right).

FIG. 9 is a drawing exemplarily showing a hologram content generation device (1000) according to an embodiment of the present invention. Referring to FIG. 9, the hologram content generation device (1000) may include at least one processor (1100), a network interface (1200), a memory (1300), a display (1400), and an input/output device (1500).

The processor (1100) may include at least one device as shown in FIGS. 1 to 8, or may be implemented with at least one method as described above as shown in FIGS. 1 to 8. The processor (1100) may execute instructions to acquire binocular content for an object, generate left-eye CGH (Computer-Generated Hologram) data and right-eye CGH data corresponding to the binocular content, and correct the left-eye CGH data and the right-eye CGH data according to the user's face, motion, or voice, as described above.

The processor (1100) can execute a program and control the hologram content generation device (1000). The hologram content generation device (1000) can be connected to an external device (e.g., a personal computer or a network) through an input/output device (1500) and exchange data.

The network interface (1200) can be implemented to perform communication with an external network through various wired/wireless methods.

The memory (1300) may include computer-readable instructions. The processor (1100) may perform the aforementioned operations as the instructions stored in the memory (1300) are executed by the processor (1100). The memory (1300) may be a volatile memory or a non-volatile memory.

The memory (1300) may include a storage device to store user data. The storage device may be an embedded multimedia card (eMMC), a solid state drive (SSD), a universal flash storage (UFS), etc. The storage device may include at least one nonvolatile memory device. The nonvolatile memory device may be a NAND Flash Memory, a Vertical NAND Flash Memory (VNAND), a NOR Flash Memory, a Resistive Random Access Memory (RRAM), a Phase-Change Memory (PRAM), a Magnetoresistive Random Access Memory (MRAM), a Ferroelectric Random Access Memory (FRAM), a Spin Transfer Torque Random Access Memory (STT-RAM), etc.

The embodiments described above can be implemented by hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments can be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding. The processing device can execute an operating system (OS) and one or more software applications running on the operating system.

Additionally, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone, but those skilled in the art will appreciate that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, the processing device may include multiple processors, or a processor and a controller. Additionally, other processing configurations, such as parallel processors, are also possible.

The software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and/or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal waves, for interpretation by the processing device or for providing instructions or data to the processing device. The software may be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

A hologram content generation device (1000) according to an embodiment of the present invention includes at least one processor (1100) and a memory (1300) storing at least one instruction executed by the at least one processor (1100), wherein the at least one instruction may be executed by the at least one processor to: acquire binocular content for an object; generate left-eye CGH (Computer-Generated Hologram) data and right-eye CGH data corresponding to the binocular content; and correct the left-eye CGH data and the right-eye CGH data according to a user's face, motion, or voice.

In an embodiment, the at least one instruction may be executed by the at least one processor to read left-eye CGH data and right-eye CGH data from a memory storing an initial value, and add an optional value corresponding to data corresponding to the user's face, motion, or voice to the read left-eye CGH data and right-eye CGH data.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The above-described hardware devices may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Meanwhile, the contents of the present invention described above are merely specific examples for carrying out the invention. The present invention will include not only specific and practically usable means itself, but also technical ideas, which are abstract and conceptual ideas that can be utilized as technology in the future.

100, 1000: Holographic content creation device
110: 3D information acquisition unit
120: Hologram information generation unit
130: Encoding section
140: Content Control
150: Display terminal
160: Sensor section
170: Rotation module
10: Holographic content integration control system

Claims (20)

In the holographic content integration control system,
A sensor unit that detects at least one of a user's face, motion, and voice;
A holographic content generation device that generates holographic content that can interact with the user using the above-detected information;
A display device for displaying the generated holographic content; and
Including a rotation module for rotating the above display device,
The above holographic content generation device generates binocular holographic data providing a wide viewing angle,
Acquire 3D images or objects selected from the left-eye sensor and the right-eye sensor, or
Acquire 3D for the selected image or object for each time for the left eye and the right eye, respectively.
A holographic content integration control system where the left-eye time and right-eye time have a predetermined time difference. In paragraph 1,
A holographic content integration control system in which the sensor unit includes a facial recognition sensor that recognizes the positions of the user's two eyes. In the second paragraph,
A holographic content integration control system, characterized in that the above facial recognition sensor distinguishes the user's movement direction, movement angle, and movement speed. In the second paragraph,
A holographic content integration control system, characterized in that the rotation module is controlled based on data detected by the facial recognition sensor. In paragraph 4,
A holographic content integration control system, characterized in that the above rotation module is controlled in rotation angle unit steps of 15° or less. In paragraph 1,
The above sensor unit is a holographic content integration control system including a voice sensor that recognizes the user's voice. In paragraph 6,
A hologram content integrated control system characterized in that the rotation direction, rotation angle, or rotation speed of the hologram content is determined based on data detected by the voice sensor. In paragraph 1,
The above sensor unit is a holographic content integrated control system including a motion sensor that recognizes the user's movement or gesture. In Article 8,
A hologram content integrated control system characterized in that the rotation direction, rotation angle, or rotation speed of the hologram content is determined based on data detected by the motion sensor. In paragraph 1,
A hologram content integrated control system, characterized in that the sensor unit includes a haptic sensor that detects the tactile sensation of the hologram content. delete delete delete In the operating method of a holographic content generation device,
A step of obtaining binocular content for an object;
A step of generating left-eye CGH (Computer-Generated Hologram) data and right-eye CGH data corresponding to the above binocular content; and
Including a step of correcting the left eye CGH data and the right eye CGH data according to the user's face, motion or voice,
The steps to create are:
Generate binocular holographic data providing a wide viewing angle,
Acquire 3D images or objects selected from the left-eye sensor and the right-eye sensor, or
Acquire 3D for the selected image or object for each time for the left eye and the right eye, respectively.
A method in which the time for the left eye and the time for the right eye have a predetermined time difference. In Article 14,
A method further comprising the step of detecting the face, motion or voice of the user. In Article 14,
A method further comprising the step of transmitting the left-eye CGH data and the right-eye CGH data to a display terminal. In Article 14,
The step of obtaining binocular content for the above object is:
A step of obtaining color information and depth information from a left-eye image for the above object; and
A method comprising the steps of obtaining color information and depth information from a right-eye image for the above object. In Article 14,
The step of obtaining binocular content for the above object is:
A step of acquiring color information and depth information for the left eye time for the above object; and
A method comprising the steps of obtaining color information and depth information at right-eye time for the above object. In a holographic content creation device,
Comprising at least one processor and a memory storing at least one instruction executed by the at least one processor,
At least one of the above instructions,
Obtain binocular content for an object;
Generating left-eye CGH (Computer-Generated Hologram) data and right-eye CGH data corresponding to the above binocular content; and
The at least one processor is executed to correct the left eye CGH data and the right eye CGH data according to the user's face, motion, haptics or voice.
Generate binocular holographic data providing a wide viewing angle,
Obtain color information and depth information for selected images or objects from the left-eye sensor and the right-eye sensor, or
Obtain color information and depth information for the selected image or object for each left-eye time and right-eye time, respectively.
A holographic content generation device characterized in that the time for the left eye and the time for the right eye have a predetermined time difference. In Article 19,
A hologram content generation device, characterized in that the at least one instruction is executed by the at least one processor to read left-eye CGH data and right-eye CGH data from a memory storing an initial value, and add an option value corresponding to data corresponding to the user's face, motion, haptic, or voice to the read left-eye CGH data and right-eye CGH data.

Images