JP2018013562A - External control means in virtual reality system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem of a conventional general virtual reality system that a testee wearing a HMD operates a virtual space by motion of his own body and his sense of immersion into the virtual space thus heightens, so that the doer of operation is the testee himself, but in order to freely move around within the virtual space or move furniture arranged in the virtual space, it is often necessary to receive the explanation or training of an operation method in advance, for example, through the use of a gesture or controller, as will be experienced when proposing, using these systems, interior reform to visitors to the showroom of an interior supplier, in which case it will take too much time for preliminary explanation and the opportunity of proposing the main subject of interior coordination will be lost, or it will become impossible to make the proposal itself, if operation is difficult.SOLUTION: The present invention enables an external operator to control the position of a testee and the movement of furniture, for example, and thereby allows the testee to get himself immersed in a virtual space without receiving explanation or practicing training.SELECTED DRAWING: Figure 16

Description

The present invention relates to a virtual reality system, a virtual reality providing method, and a virtual reality providing apparatus.

The virtual space generated in the computer is displayed to the user using a device such as a head mounted display (HMD), and the virtual space (Virtual) is provided as if the user were in the space. There is a technology related to Reality (VR).

For example, in a study of home renovation, a device that enters into a house model generated in a virtual space using an HMD and changes the viewpoint position using a controller such as a mouse or the position of the user in real time. By moving around the virtual house freely by changing the viewpoint position in conjunction with, etc., and looking around by updating the display image in conjunction with the movement of the head, it is as if living in the house Providing a sense of being.

JP-A-9-81774 JP-A-8-161106

A 3D house model can be created in the computer, and the image seen from the viewpoint of the room can be displayed. The function of changing the material, color, and pattern of walls, floors, ceilings, etc., and curtains and furniture fixtures In addition, there is a system that has a function of arranging lighting and the like, and can perform interior coordination of a house by using these functions in combination. Even when looking at catalogs and material samples such as wall materials, floor materials, and ceiling materials when renovating homes, it is difficult to imagine the state after construction, and there are many cases where selection is wrong. Since this system can be used to display images with these materials coordinated, it is easy to imagine after construction and is useful for examination and evaluation.

Furthermore, in the virtual reality system that can enter a virtual house with VR, it is easy to grasp the size of the room and the size of the furniture, and the size of the pattern such as wall material and flooring can be confirmed in the same way as in reality, so it is more accurate. Can be examined and evaluated.

In a general virtual reality system, an experienced person wearing an HMD (hereinafter referred to as an “experience person”) manipulates the virtual space by the movement of his / her body, thereby causing the illusion that he / she is in the virtual space. In order to obtain an effect, the subject of the operation is the experience person himself. By attaching a sensor to the experience person or HMD and linking with a device that can measure the experience person's spatial position, when the experience person walks in the real space, the viewpoint moves in the virtual space and walks in the virtual space. If there is no such device, it is necessary to input a gesture indicating walking through a device that measures the movement of fingers and to move the viewpoint position by operating a controller such as a mouse. is there.

In addition, one input device with a function for inputting finger movement and a function for measuring the spatial position of the hand is provided for each hand or both hands. For example, a hand is placed at the position of the furniture in front of the eyes in the virtual space. A series of operations such as grasping furniture in the virtual space by moving the hand down and holding the hand, lifting the furniture by raising the hand as it is, moving the place to lower the hand, and opening the hand The furniture in the virtual space can be moved by operation. However, if there is no such input device, the furniture caught at the tip of the line of sight is grabbed by the operation such as pressing the mouse button, the furniture is lifted by raising the line of sight as it is, the mouse button is released and the furniture is lowered. It is necessary to provide a simple operation method, and since this is an operation different from the real world, it is necessary to learn the operation.

In particular, for the operation to select the interior material, color, and pattern in the virtual space and change the wall material and flooring, for example, a menu for selecting the interior material is displayed on the display screen of the HMD, and the line of sight and gestures are displayed. It is necessary to have a method for selecting menus and interior materials using the menu, and selecting the walls and floors to which the menu and interior materials are to be applied. However, it is necessary to explain how to use this function and practice it.

For example, when making interior renovation proposals using these systems to visitors such as showrooms of interior contractors, it takes time to give advance explanations and practice of operations, making it impossible to propose interior coordination that is the main purpose. End up. If the operation is difficult, this system is avoided and the proposal itself cannot be made.

The present invention has been made in view of the above problems, and an object of the present invention is to provide means capable of experiencing a virtual space while eliminating the need for explanation of operations and practice of operations.

In order to solve the above-mentioned problems, the present invention enables an external operator, not an experienced person, to control the position and direction in the virtual space of the experienced person wearing the HMD.

In addition, the operator can now perform operations such as selection / replacement of materials / colors / patterns of interior / exterior materials of virtual buildings in the virtual space.

In addition, the operator can now select, move, rotate, delete, and replace parts such as furniture and joinery placed in the virtual space.

As described above, the operation of the user wearing the HMD is limited to the intuitive operation of looking around with the movement of the head, and other operations are performed by an external operator, so that the user can operate the method. You can immediately experience the virtual space without learning or practicing operation.

For example, when making an interior renovation proposal for a visitor such as a showroom of an interior supplier, this system only requires the experience person to wear an HMD, and the operator controls the position of the experience person in the virtual space. , You can move to the location you want the experience to see. In addition, the operator performs operations such as changing the color and pattern of the wall material, floor material, ceiling material, etc. of the room that the user is viewing, replacing furniture, moving the position, etc. I can do things.

In other words, the experiencer does not need to learn difficult operations such as changing materials or replacing furniture, and can be immersed in the experience in the virtual space.

The figure which shows schematic structure of the virtual reality provision apparatus which concerns on embodiment of this invention. The block diagram which shows the external control means in the virtual reality provision apparatus shown in FIG. Example of floor plan creation work in virtual building definition means (room creation) Example of floor plan creation work in virtual building definition means (placement placement) Example of a virtual building created by the virtual building definition means (whole) Example of virtual building created by virtual building definition means (1st floor) Example of image displayed on HMD worn by experienced person Example of screen displayed by display means 21 such as a plan view (perspective view: overhead view display) Example of the screen displayed by the display means 21 such as a plan view (perspective view: display from the viewpoint of the user) Examples of figures representing the reference gaze and the experience gaze Display example of reference gaze and experience gaze in top view (1) Display example of reference gaze and experience gaze in top view (2) Example of moving the reference line of sight with the mouse Example of rotation operation of reference line of sight with mouse A state in which the reference line of sight in FIG. 12 is moved in another room and in another direction. An example in which a figure showing the reference line of sight and the user's line of sight is displayed on the top view. Display example when controlling the position of the reference line of sight by moving the doll model on the perspective view Example of changing the interior material in the interior / exterior material replacement method (while selecting materials) Example of operation to change interior material in interior / exterior material replacement method (after material change) Example of furniture placement operation in furniture fitting placement means (while furniture is selected) Example of furniture arrangement operation in furniture fitting arrangement means (after furniture arrangement) Example of furniture replacement operation in furniture fitting arrangement means

FIG. 1 is a diagram illustrating an example of the configuration of the embodiment. The computer 11 that performs processing is connected to a mouse 12 and a keyboard 13 for operation, a display device 14, an HMD 15 for displaying a virtual space, and a controller 16.
Further, an external storage device 17 and a printing device 18 for reading and saving data may be connected as necessary.

FIG. 2 is a diagram illustrating a processing flow in the embodiment.
The processing includes a virtual building definition means 21, an external control means 23, and a virtual space display means 24. The external control means 23 is a plan view display means 31, a viewpoint position line-of-sight direction control means 32, and an interior / exterior material replacement means 33. And furniture fitting arrangement means 34.

Embodiments of the present invention will be described below with reference to the drawings.

The virtual building definition means 21 is a means for defining a building such as a house in a virtual space. For example, as shown in FIG. 3, it is a means for inputting floor plan information using a mouse or the like, and has a function of defining the size, shape and position of a room by combining figures such as rectangles and polygons. Also, as shown in FIG. 4, it has a function of selecting a joinery symbol and arranging it on the wall position of the room.

Further, by adding height information to the floor plan information defined in this way, a three-dimensional house as shown in FIG. 5 can be created in the computer. FIG. 6 is an example in which the first floor portion of the house created in this way is displayed in an overhead view.

When reading virtual building data designed by another application such as CAD, a virtual building reading means 22 may be provided.

The virtual space display means 24 can display the virtual building in the virtual space on the virtual space display device such as an HMD on which the experience person wears, via the external control means 23. FIG. 7 is an example of an image displayed on the HMD worn by the experience person.

Furthermore, by interpreting information from the sensor that measures the movement of the head and controlling the line-of-sight direction displayed on the HMD, the user wearing the HMD rotates around the head in the virtual space by rotating the head up, down, left and right. Can look around. In addition, by interpreting information from the sensor that measures the position of the head and controlling the viewpoint position displayed on the HMD, it matched the movement, such as when the experienced person stood up or moved the head back and forth and left and right Can be displayed in view. In this respect, since the experience person can control the line of sight with his / her movement regardless of the control of the operator, the experience person can experience without losing the sense of immersion in the virtual space.

A display unit 31 such as a plan view in the external control unit 23 displays a virtual building in the virtual space on a display device using a plan view or a top view such as a three-view view (front view, top view, side view). can do. FIG. 8 shows a display example. The screen is divided into left and right, and a front view and a perspective view are displayed respectively. The size of the screen may be variable, or only the required drawing such as displaying only the top view may be selected and displayed, or the screen is divided into four parts, a front view, a top view, a side view, A perspective view may be displayed. Although the whole or part of the virtual structure is displayed as a bird's-eye view in the perspective view of FIG. 8, an image viewed by the user wearing the HMD may be displayed as shown in FIG. 9.

On the plan view of the display means 31 such as a plan view, the operator wearing the figure and the HMD that represents the viewpoint position / gaze direction (hereinafter referred to as “reference gaze”) as a reference for the operator to control the viewpoint of the experiencer. A graphic representing the position in the virtual space and the direction of the line of sight (hereinafter referred to as the “experience person's line of sight”) is displayed in an overlapping manner. FIG. 10A shows an example of a figure representing a reference line of sight, and FIG. The viewpoint position 42 and the line-of-sight direction 41 are shown, respectively. The angle of view 43 of the line of sight may be displayed.

11 and 12 are examples of display in a top view. In FIG. 11, the reference line-of-sight graphic and the experience person line-of-sight graphic are displayed so as to substantially overlap. That is, the experience person is in the vicinity of the reference position (origin) of the sensor that measures the position of the head and is almost facing the front. FIG. 12 shows a state where the experience person has moved his head to the front and turned to the right.

The position coordinates of the reference line of sight in the virtual space are (Ox, Oy, Oz), the rotation angle around the Z axis of the reference line of sight is Or, and the information from the sensor that measures the head position is the displacement from the sensor origin Experiential position coordinates (Hx, Hy, Hz), experience when the amount is (Dx, Dy, Dz) and the horizontal rotation angle of the head (Z axis rotation angle) is Dr (right front is 0) The rotation angle Hr of the person's line of sight can be obtained by the following calculation.
Hx = Ox + Dx × cos (Or)-Dy × sin (Or)
Hy = Oy + Dx × sin (Or) + Dy × cos (Or)
Hz = Oz + Dz
Hr = Or + Dr
It is assumed that the coordinate system of the virtual space and the coordinate system of the sensor for measuring the head position are the same, and the rotation angle 0 around the Z axis is the same as the Y axis. Note that the rotation angle that represents the viewer's line of sight is not only around the Z axis, but also the elevation angle (rotation angle around the X axis) and the tilt of the head (rotation angle around the Y axis). However, the virtual space display means 14 displays information from the sensor as it is.

The viewpoint position gaze direction control means 32 allows the operator to control the position and gaze direction of the experience person in the virtual space using an input device such as a mouse, a keyboard, or a controller. Specifically, the figure representing the reference line of sight is moved by dragging with the mouse as shown in FIG. 13, and the line of sight is rotated by dragging the tip of the arrow indicating the line of sight as shown in FIG. . In addition, even if it does not drag, it is good also as an operation method which moves a reference | standard gaze position to a mouse position by double click, or turns to the direction of a mouse | mouth by click.

In the above description, the top view is described, but the display of the reference line of sight and the user's line of sight is the same in the front view and the side view. However, the elevation angle and head tilt are not subject to the operator's operation, and on the front view and side view, the line of sight of the reference line of sight is always horizontal, and only the user's line of sight reflects the sensor information.

FIG. 15 shows a state in which the reference line of sight in FIG. 12 is moved in another room and in another direction. At this time, the experience person has not moved the position and direction of his / her head, and the relative positional relationship from the new reference line of sight is maintained. Regardless of the operation of the operator, the experiencer can freely move his head and look around.

FIG. 16 is an example in which a graphic 51 showing a reference line of sight and a graphic 52 showing an experience person's line of sight are displayed on the top view 50. In the perspective view 53, the same image as that displayed on the experience person's HMD is displayed. The perspective view may be displayed as an overhead image instead of the image of the viewer's line of sight.

As shown in FIG. 17, instead of the figure of FIG. 10 in the top view, the doll model 55 is displayed at the position of the user wearing the HMD on the perspective view, and this is moved and rotated with the mouse as a reference. The line of sight may be changed.

The interior / exterior material replacement means allows the operator to select / replace the interior material / exterior material / color / pattern of the virtual building in the virtual space using an input device such as a mouse, keyboard or controller. It can be performed. 18 and 19 are examples thereof. In FIG. 18, the perspective view is a screen viewed by the experience person via the HMD. The operator selects one material from the material palette and drags it to the floor portion of the perspective view. Then, as shown in FIG. 19, the floor material displayed on the perspective view of the display unit 31 such as a plan view changes. That is, the material of the floor on the virtual space display means 24 that the experience person sees through the HMD also changes.

The furniture / joint placement means allows the operator to select / move / rotate / delete / replace furniture / joint / parts placed in the virtual space using an input device such as a mouse, keyboard or controller. be able to. 20 and 21 are examples thereof. In FIG. 20, the perspective view is a screen viewed by the experience person via the HMD. The operator selects one piece of furniture from the furniture palette and drags it to the floor portion of the perspective view. Then, as shown in FIG. 21, furniture is arranged on the floor displayed on the perspective view of the display means 31 such as a plan view. That is, the furniture also appears on the floor on the virtual space display means 24 that the user sees through the HMD.

When the operator selects one piece of furniture on the perspective view, selects another piece of furniture from the furniture palette, and selects the “Replace furniture” menu, the furniture on the perspective view is the furniture selected on the furniture palette. Will be replaced. FIG. 22 is an example of a screen on which furniture replacement operation is performed. The furniture selected on the screen of FIG. 21 is replaced with another furniture.

In this way, the virtual space can be easily guided to the location that the operator wants to explain without having the user explain or practice the operation, and the user can stand up, look around, or look into the virtual space. You will be able to experience without losing your sense of immersion.

11: Computer 12: Mouse 13: Keyboard 14: Display device 15: HMD
16: controller 17: external storage device 18: printing device 21: virtual building definition means 22: virtual building data reading means 23: external control means 24: virtual space display means 31: plan view display means 32: viewpoint position line of sight Direction control means 33: interior / exterior material replacement means 34: furniture fitting arrangement means 36: top view 37: perspective view 41: line-of-sight direction 42: viewpoint position 43: angle of view 50: top view 51: figure showing reference line of sight 52: experience Figure 53 showing a person's line of sight 53: Perspective FIG. 55: A doll model 61 showing a reference line of sight position on the perspective view 61: A selection frame indicating that furniture is in a selected state

Claims (7)

The virtual space generated in the computer is displayed to the user using a device such as a head mounted display (HMD), and the virtual space (Virtual) is provided as if the user were in the space. In a system using Reality (VR), a virtual reality system is characterized in that an external operator can change the position of an experienced person. In a virtual reality system, a virtual reality system is characterized in that an interior operator or exterior material of a virtual building in a virtual space is changed by an external operator instead of an experienced person. In a virtual reality system, operations such as placing furniture and other parts in the virtual space, moving the placed furniture, and replacing the placed furniture with another furniture are not experienced. Virtual reality system characterized by being operated by an external operator The virtual reality system according to claim 1, wherein an interior operator and an exterior material of a virtual building in a virtual space are changed by an external operator instead of an experienced person. The virtual reality system according to claim 1, wherein an operation of arranging a part such as furniture in the virtual space, moving the arranged furniture, or replacing the arranged furniture with another furniture is performed. , A virtual reality system that is operated by an external operator rather than an experienced person The virtual reality system according to claim 1, wherein an interior operator and an exterior material of a virtual building in the virtual space are changed by an external operator instead of an experienced person, and the furniture is further provided in the virtual space. It is a feature that an external operator, not an experienced person, operates operations such as placing parts such as, moving the arranged furniture, and replacing the arranged furniture with another furniture Virtual reality system A recording medium storing a program for realizing the system according to any one of claims 1 to 6 by a computer