KR102622094B1 - Indoor space 3D mapping support system for augmented reality - Google Patents

Abstract

The present invention attaches 3D mapping sensors to the walls of various indoor spaces, including homes, acquires accurate structural information of the interior, and supports 3D mapping of indoor spaces for augmented reality that provides realistic AR services through interactive actions with the spatial structure. It's about the system.

Description

Indoor space 3D mapping support system for augmented reality {Indoor space 3D mapping support system for augmented reality}

The present invention relates to a 3D mapping sensor. Specifically, a 3D mapping sensor is attached to the wall of various indoor spaces, including homes, to acquire accurate structural information of the room and provide realistic AR services through interaction with the spatial structure. This is about an indoor space 3D mapping support system for augmented reality.

Augmented reality technology is a technology that adds virtual objects or environments to existing objects or environments to make them appear as if they actually exist, and is attracting attention in various fields along with the rapid development of IT technology.

In particular, as outdoor activities are limited due to relatively insufficient physical activity compared to the past, weather conditions, environmental pollution, and fine dust, content that allows various experiences, including sports and play, has been developed to enable sufficient physical activity indoors. I'm losing.

In order to provide such augmented reality content smoothly, tracking technology that accurately recognizes the user's location in space as well as an accurate understanding of the indoor space structure is very important. Recently, cameras, GPS, and gyro sensors have been installed in smartphones, tablets, etc. , tracking and augmented reality using this can be smoothly provided.

However, in the case of augmented reality methods using GPS and gyro sensors, it is difficult to use indoors where GPS communication is not smooth, and the location of the virtual object is determined based on the image captured by the camera of the user terminal to output the augmented reality image. Therefore, when events such as camera view movement or program switching occur, the virtual object may not be expressed accurately in the same location.

In particular, as content is provided tailored to designated spaces to implement augmented reality, if you want to utilize various indoor spaces for user convenience, including home, you can only use mobile content that does not fully utilize indoor space, so you can enjoy content. It had to be very limited.

Republic of Korea Patent No. 10-2241024 (2021.04.12)

The present invention was created to solve the above problems, and the purpose of the present invention is to acquire accurate structural information of the room through a 3D mapping sensor that can be easily attached to the wall of various indoor spaces, including homes, and to measure the spatial structure of the acquired space. It provides an indoor space 3D mapping support system for augmented reality that can provide AR services with excellent realistic immersion by supporting optimized interactive actions.

For the above purpose, the present invention provides an indoor space 3D mapping support system, which measures the distance between an attachment part attached to an indoor wall on one side, a photographing part that photographs the indoor space on the other side, and an opposing wall. It has a partially exposed body, and consists of an information generation unit that generates collected information including the area and distance of the facing wall and indoor structures located in the space through the photographing unit and the distance measurement unit, and a short-range wireless communication module. A plurality of sensor modules with a built-in first communication unit that transmits collected information; An analysis module including a second communication unit that communicates with the first communication unit and a space implementation unit that generates spatial information that reconstructs the size and shape of the space through the received collected information; An interlocking unit that links with a terminal having an AR implementation function, a storage unit in which AR content is stored, an optimization unit that combines the AR content with the spatial information and sets an interaction point that reflects the spatial shape, and an interaction point. A linkage module including an output unit that outputs a set AR content image to a linked terminal; It is characterized by consisting of.

At this time, the body may be configured to have a magnetic body formed on one side and a metal body attached to the magnetic body of another sensor module on the other side, so that a plurality of sensor modules can be attached to each other.

In addition, the body further includes a detection unit that detects the attachment state of the metal body to the magnetic body of an adjacent sensor module and deactivates the power supply as the attachment is detected, a third communication unit that communicates with the sensor module, and each sensor module A management module including an operation management unit that generates and manages operation information through communication, and a loss alarm unit that outputs a loss alarm due to random deviation of the sensor module; It may further include.

In addition, the linkage module includes a motion analysis unit that analyzes the user's motion captured through the photographing unit and checks whether it corresponds to the set motion at the interaction point, and a reaction signal according to the motion correspondence judgment of the motion analysis unit. It may further include an interactive action unit that reflects and outputs the output AR content image.

In addition, the analysis module may further include a setting guide unit that analyzes spatial information to check the shadow area, calculates the attachment position of the sensor module that can eliminate the shadow area, generates location correction information for the sensor module, and provides guidance. You can.

In addition, the AR content may include content that reflects changes in plants, animals, and set meteorological and astronomical elements based on geographical locations and natural spaces collected based on street view services.

In addition, the analysis module includes an evaluation unit that receives evaluation information from users regarding the provided AR content, and an evaluation analysis unit that cumulatively stores and analyzes AR content and evaluation information for each user and classifies user groups according to the same AR content and evaluation information. It may further include a recommendation unit that recommends AR content that obtains high evaluation information for each user group to other users in the group.

Through the present invention, it is possible to obtain accurate indoor information by simply attaching it to the wall not only at home but also anywhere indoors, and to provide a realistic AR service optimized for space.

In addition, the highly portable exterior and built-in magnetic material-based detachable structure reduce the possibility of loss, and expand the range of AR content that was released mainly for mobile devices, allowing you to enjoy it more realistically in various indoor spaces.

Additionally, AR content tailored to individual tastes can be provided through data that reflects the user's tastes and lifestyle.

1 is a conceptual diagram of the present invention;
Figure 2 is a block diagram showing the configuration and connection relationship according to an embodiment of the present invention;
Figure 3 is a perspective view of a single sensor module according to an embodiment of the present invention;
4 is a side cross-sectional view of a sensor module according to an embodiment of the present invention;
Figure 5 is a side cross-sectional view of multiple sensor modules stacked according to an embodiment of the present invention.

Hereinafter, the configuration of the indoor space 3D mapping support system for augmented reality of the present invention will be described in detail with reference to the attached drawings.

1 is a conceptual diagram of the present invention. The present invention obtains indoor information by measuring the area of the wall and the distance between the walls by simply attaching a plurality of sensor modules (1) to the walls of various indoor spaces, including homes. , It supports output of AR content optimized for indoor space by linking with smartphones, screens, goggles, etc. that have AR implementation functions, and supports structure and interaction within the space to increase immersion and realism.

Figure 2 is a block diagram showing the configuration and connection relationship according to an embodiment of the present invention, for which the main components of the present invention are a sensor module (1), an analysis module (2), a linkage module (3), and a management module (4). ) is provided.

Figure 3 is a perspective view of a single sensor module according to an embodiment of the present invention, and Figure 4 is a side cross-sectional view of a sensor module according to an embodiment of the present invention. The sensor module 1 is configured with an attachment portion attached to an indoor wall on one side. It consists of a body 11 made of a flat, cylindrical synthetic resin material.

The attachment part is a component for temporarily fixing the body 11 to the wall and can be constructed by applying adhesive tape, a vacuum suction plate in the case of a smooth wall, a magnetic material in the case of a steel wall, and other attachments of the body 11. On the side, a photographing unit 12 for photographing the indoor space and a distance measuring unit 13 for measuring the distance to the opposing wall are exposed, and on the inside, an information generation unit 14, a first communication unit 15, and a detection unit are exposed. (16) is built-in. In addition, it is obvious that a power source such as a battery for power supply must be provided, and since this is a configuration commonly applied to all known electronic products, detailed description is omitted to prevent the purpose of the invention from being obscured.

The photographing unit 12 is configured to photograph the space by being exposed to the opposite side of the attachment portion. A single wide-angle camera may be applied, but in addition to the main wide-angle camera, a plurality of cameras such as an ultra-wide-angle camera and a telephoto camera with relatively low magnification may be used. You can obtain deeper and more three-dimensional images by shooting the space simultaneously through cameras, and if necessary, you can also configure an infrared camera, etc.

The distance measuring unit 13 is basically exposed in the same direction as the photographing unit 12 by applying various known distance measuring means such as laser or ultrasonic waves, and is connected to the wall opposite to the wall to which the sensor module 1 is attached. Measure the distance, but preferably install it on the top, bottom, left, and right sides so that the distance to the floor, ceiling, and adjacent walls on the left and right can be measured together, so that more accurate space information can be collected.

The information generation unit 14 generates collected information including the area and distance of the opposing wall and indoor structures located in the space through the photographing unit 12 and the distance measurement unit 13. In other words, as mentioned above along with the image taken of the front, the distance measuring units 13 are provided on the top, bottom, left, and front, respectively, so the distance measurement values between the wall facing the front and the walls adjacent to the top, bottom, left, and right sides, respectively, are collected information. Created with

The first communication unit 15 consists of a short-distance wireless communication module and transmits the collected information to the analysis module 2, which will be described later. In the present invention, as a plurality of sensor modules 1 are provided and used simultaneously, a unique ID is assigned to the first communication unit 15, so that each sensor module can be identified during communication.

In particular, in the present invention, by using these sensor modules (1) in plural rather than singly, accurate space reconstruction can be achieved, and as the indoor space in a typical building is made up of a square plane, four sensor modules are provided to provide four sensor modules. It is preferable to attach them individually on the wall.

Figure 5 is a side cross-sectional view of multiple sensor modules stacked according to an embodiment of the present invention. As a plurality of sensor modules are used organically, a magnetic material 112 is used to store them together and prevent loss. I do it.

That is, a magnetic body 112 is provided on one side of the body 11 and a metal body 113 attached to the magnetic body of another sensor module is provided on the other side of the body 11, so that a plurality of sensor modules are stacked in the same direction. can be attached to each other.

At this time, the detection unit 16 detects the state of attachment of the metal body 113 to the magnetic body of an adjacent sensor module, and deactivates the power supply as attachment is detected. It can be implemented in a variety of ways, and as a representative example, the detection unit 16 is also configured as a sensor that detects the magnetic material of another adjacent sensor module, or is configured to allow the metal body to move through a predetermined gap in the front and rear direction, but is not configured to bend the spring or metal body. Through its structure, it is maintained in a state away from the magnetic body of other adjacent sensor modules, and the detection unit can be configured in the form of a switch that can detect the movement of a metal body through the magnetic force of the magnetic body of other sensor modules.

The analysis module (2) generates specific spatial information through collected information generated through a plurality of sensor modules, and is configured to correspond to a computer terminal for setting the sensor module (1) and recommending AR content, and is connected to the second communication unit. It has a detailed configuration of (21), a space implementation unit (22), a setting guide unit (23), an evaluation unit (24), an evaluation analysis unit (25), and a recommendation unit (26).

The second communication unit 21 is composed of the same short-distance wireless communication module as the first communication unit 15, and enables mutual communication, and in particular, receives collected information transmitted from each sensor module.

The space implementation unit 22 generates spatial information that reconstructs the size and shape of the space through the collected information received through the second communication unit 21. Basically, the shape and size of the entire space centered on the wall can be obtained through the distance information to the wall located up, down, left, and in front measured through each sensor module (1), and through spatial images captured by each sensor module. The location and size of objects such as various structures and furniture located in space can be obtained, thereby reconstructing the space and creating spatial information.

The setting guide unit 23 analyzes the spatial information generated through the space implementation unit 22 to check the shaded area, calculates the attachment position of each sensor module that can eliminate the shaded area, and corrects the position of the corresponding sensor module. It is a structure that generates information and then provides guidance. If the space is made up of a rectangular parallelepiped like a typical building, there is no problem, or if there is a pillar in the center or the wall is distorted or protrudes irregularly, it is a shaded area that prevents accurate reconstruction of the space because measuring the distance to the opposite wall is not smooth. This can happen.

If such a sound range area is confirmed, it is necessary to move and install the attached sensor module to a location that avoids the shaded area, and provide voice guidance or a linked terminal (to move the sensor module to an attachment location that is not connected to the shaded area). Guidance is provided through the display of the computer.

The linkage module (3) is configured to improve the sense of immersion and realism by linking with the AR device and outputting AR content by optimizing the spatial information previously generated in the analysis module, and includes a linkage unit (31) and a storage unit. It has a detailed configuration of (32), an optimization unit (33), an output unit (34), a motion analysis unit (35), and a mutual action unit (36).

The linking unit 31 is configured to link with a terminal having an AR implementation function, and is connected wirelessly or wired to various terminals including AR goggles, smartphones, beam projectors, monitors, etc., so that content can be played. do.

The storage unit 32 is a memory where AR content is stored. A variety of AR content may be applied, and the present invention is not limited to this, but preferably the AR content is based on geographical location and natural space collected based on street view service, and changes in animals, plants, and set meteorological and astronomical elements. It may include content that reflects . In other words, street view images constructed around major cities or tourist destinations around the world are collected and set as the basic background, but trees, flowers, animals such as birds and insects, meteorological conditions such as the weather, and astronomical elements such as clouds, stars, and the moon are excluded. By reconstructing the basic background so that it can be changed, you can get the feeling of visiting the area at home even if you do not visit the site.

The optimization unit 33 combines the AR content with the spatial information and sets an interaction point that reflects the spatial shape. Basically, the size of the AR content image is enlarged, reduced, or cut so that the image is output in a size tailored to the space where the user is located. However, in the case of objects such as furniture or structures such as pillars or protrusions located in the space, it is possible to output the image through touch, etc. It is set as an interaction point that can accompany changes in the AR content video. For example, points can be designated so that changeable content elements such as the plants and animals mentioned above are displayed by touching furniture or structures located in the space.

The output unit 34 outputs the AR content image with the interaction point set to the linked terminal. When using AR goggles or a terminal, AR content images are displayed on the display, and set content elements are displayed when the user touches an interaction point, etc.

For this purpose, the motion analysis unit 35 analyzes the user's motion captured through the photographing unit and checks whether it corresponds to the set motion at the interaction point. The movement of the user located in the space is continuously captured through the camera of the sensor module attached to the wall, and the motion analysis unit 35 analyzes and monitors the user's movement.

The interactive action unit 36 is configured to reflect and output a reaction signal according to the motion response judgment of the motion analysis unit in the output AR content image. While analyzing and monitoring the user's movement, the interactive action unit 36 is configured to output a reaction signal, such as when the user touches a set interaction point. When operating, it is judged to correspond to the set operation and the set content element is displayed.

The management module (4) is configured to support individual confirmation of loss and malfunction due to the simultaneous use of multiple sensor modules, and includes a third communication unit (41), an operation management unit (42), and a loss alarm unit. It has the detailed structure of (43).

The third communication unit 41 is composed of the same short-distance wireless communication means as the first communication unit 15 and communicates with each sensor module.

The operation management unit 42 is configured to generate and manage operation information through communication with each sensor module. It determines whether the power supply is operating normally and whether the photographing unit 12 and the distance measuring unit 13 are operating normally. The overall operation status, such as normal communication status and remaining power, is generated as operation information.

The loss alarm unit 43 is configured to output a loss alarm due to random separation of the sensor module 1. In particular, when the sensor module is randomly separated from the stacked state as shown in the attached FIG. 5, it detects this through communication and An auditory or visual alarm is output for recognition, and an alarm is detected when the sensor module (1) is unintentionally separated from the wall and the value measured through the distance measuring unit (13) suddenly changes significantly. It can be configured to output and help ensure stable use.

Additionally, the analysis module 2 can increase satisfaction by recommending videos that suit the user's tastes among various AR contents.

The evaluation unit 24 provided for this purpose is configured to receive evaluation information from the user regarding the provided AR content, and can receive input of overall satisfaction with the experienced AR content using an intuitive score scale.

The evaluation analysis unit 25 is configured to cumulatively store and analyze AR content and evaluation information for each user and classify user groups according to the same AR content and evaluation information. In other words, the information from all users who received the service through the present invention is collected and analyzed. AR content and evaluation information played from all users are collected, classified by content and evaluation information, and then content with the same or similar degree is evaluated. User groups are formed according to information.

The recommendation unit 26 is configured to recommend AR content that receives high evaluation information for each user group to other users in the group. In other words, since content and evaluation reflect the tastes of the user, a user group based on content and evaluation information can be viewed as a group of users with substantially similar tastes. Therefore, high satisfaction can be expected by recommending content that receives high evaluation information for each user group to other users in the group.

The rights of the present invention are not limited to the embodiments described above but are defined by the claims, and those skilled in the art can make various changes and modifications within the scope of the claims. This is self-evident.

1: Sensor module 11: Body 112: Magnetic material
113: metal body 12: photographing unit
13: Distance measurement unit 14: Information generation unit
15: first communication unit 16: detection unit
2: Analysis module 21: Second communication department 22: Space implementation department
23: Setting guide section 24: Evaluation section
25: Evaluation and analysis department 26: Recommendation department
3: Link module 31: Link part 32: Storage part
33: optimization unit 34: output unit
35: Motion analysis unit 36: Mutual action unit
4: Management module 41: Third communication unit 42: Operation management unit
43: Loss Alert Department

Claims (7)

In the indoor space 3D mapping support system,
It has a body (11) with an exposed attachment portion attached to an indoor wall on one side, a photographing portion (12) for photographing the indoor space on the other side, and a distance measuring portion (13) for measuring the distance to the opposing wall, It consists of an information generation unit 14 that generates collected information including the area and distance of the facing wall and indoor structures located in the space through the photographing unit 12 and the distance measuring unit 13, and a short-range wireless communication module. A plurality of sensor modules (1) with a built-in first communication unit (15) that transmits the collected information;
An analysis module ( 2);
A linking unit 31 that links with a terminal having an AR implementation function, a storage unit 32 in which AR content is stored, and an optimization unit that combines the AR content with the spatial information and sets an interaction point that reflects the spatial shape. A linkage module (3) including (33) and an output unit (34) that outputs an AR content image with set interaction points to a linked terminal; It is made up of,
The body 11 is configured to have a magnetic body 112 formed on one side and a metal body 113 attached to the magnetic body of another sensor module on the other side, so that a plurality of sensor modules can be attached to each other. ) further includes a detection unit 16 that detects the state of attachment to the magnetic material of the adjacent sensor module and deactivates the power supply as the attachment is detected,
A third communication unit 41 that communicates with the sensor module 1, an operation management unit 42 that generates and manages operation information through communication with each sensor module, and a loss alarm due to random departure of the sensor module 1. A management module (4) including a loss alarm unit (43) that outputs; An indoor space 3D mapping support system for augmented reality, further comprising:
delete delete According to paragraph 1,
The linkage module (3) is,
A motion analysis unit 35 that analyzes the user's motion captured through the photographing unit 12 and checks whether it corresponds to the set motion at the interaction point, and a motion correspondence judgment of the motion analysis unit 35. An indoor space 3D mapping support system for augmented reality, further comprising an interactive action unit 36 that reflects the reaction signal according to the output AR content image and outputs it.
According to paragraph 1,
The analysis module (2) is,
It further comprises a setting guide unit 23 that analyzes spatial information to determine the shadow area, calculates the attachment position of the sensor module that can eliminate the shadow area, generates location correction information for the sensor module, and provides guidance. Indoor space 3D mapping support system for augmented reality.
According to paragraph 1,
The AR content is an indoor space for augmented reality, characterized in that it includes content that reflects changes in animals and plants and set meteorological and astronomical elements based on geographical locations and natural spaces collected based on street view services. 3D mapping support system.
In paragraph 6
The analysis module (2) is,
An evaluation unit 24 that receives evaluation information from users about the provided AR content, and an evaluation analysis unit 25 that cumulatively stores and analyzes AR content and evaluation information for each user and classifies user groups according to the same AR content and evaluation information. Wow, an indoor space 3D mapping support system for augmented reality, further comprising a recommendation unit 26 that recommends AR content that obtains high evaluation information for each user group to other users in the group.

Images