WO2022149361A1

WO2022149361A1 - Input device capable of providing tree-dimensional tactile information

Info

Publication number: WO2022149361A1
Application number: PCT/JP2021/042674
Authority: WO
Inventors: 俊輝西本; 彬池田; 誠平原
Original assignee: 菱洋エレクトロ株式会社
Priority date: 2021-01-05
Filing date: 2021-11-19
Publication date: 2022-07-14
Also published as: JP2022105939A

Abstract

Provided is an input device to which an operation of a user in a simulated virtual space can be input, the input device being capable of providing tactile feedback that can enhance a sense of presence. This input device to which an operation of a user in a simulated virtual space can be input comprises a plurality of actuators configured to provide tactile feedback to the input device, wherein the plurality of actuators are disposed so as to generate the tactile feedback in three-dimensional directions in cooperation with one another.

Description

Input device capable of providing 3D tactile information

The present invention relates to an input device capable of providing three-dimensional tactile information, and more particularly to an input device capable of providing three-dimensional tactile information equipped with a plurality of actuators that give tactile feedback to the human body.

In recent years, pressure-sensitive input devices such as touch panels have been adopted in a wide range of fields as interfaces for users to operate devices due to the advantage of being able to operate intuitively. In addition, the movement to replace touch panels is spreading to devices that have conventionally used physical buttons and keyboards.

An electronic device equipped with a touch panel may be equipped with a haptic actuator (hereinafter, also simply referred to as "actuator") that gives tactile feedback (haptic feedback) to the human body in combination with the touch panel. When a part of the user's body (eg, a finger) or an object operated by the user touches the input of an electronic device, or when the system generates a specific event, the haptic actuator is the housing or touch of the electronic device. Generate motion in components such as screens. The user can intuitively and easily operate the electronic device or receive information by perceiving the vibration at the part of the body that comes into contact with the electronic device or perceiving it as a sound.

Haptic actuators generally use electric power as a drive source, and can be roughly divided into impact drive type and vibration drive type according to the nature of motion. Typical examples of the impact drive type include SIA (Shape memory alloy Impact Actuator) using a shape memory alloy and a piezoelectric actuator using a piezo element. The impact-driven actuator includes a moving component capable of shocking movement, and gives a transient impact to the touch screen or housing of an electronic device by hitting or interlocking the moving component. Typical examples of vibration-driven actuators include ERM (Eccentric Rotating Mass) type actuators that use eccentric motors, and linear resonance type actuators (LRA: Linear Resnonant Actuator) that vibrate the mover by passing an alternating current through a coil in a magnetic field. Can be mentioned. The vibration-driven type gives a vibration of a constant amplitude to the vibrating body for the required time.

For example, in Patent Document 1 (International Publication No. 2012/023605), a horizontal (or planar direction) or vertical direction (front-back direction or Disclosed are shock-driven actuators that enable shocking movements (impact movements) in the thickness direction and rotational or straight-ahead movements based on repeated shocking movements. According to the impact-driven actuator, insulating heat conductors of various shapes are provided so as to be in contact with the wire-shaped shape memory alloy arranged in a predetermined wiring state as effectively as possible, and the insulating heat conduction is provided. Since the heat generated by the wire-shaped shape memory alloy during pulse energization is quickly dissipated and released by the body, the temperature of the wire-shaped shape memory alloy can be lowered quickly, and it can be repeated in a relatively short time. It is possible to realize a momentary operation that is possible, and it is possible to realize a highly practical impact-driven actuator.

In addition, a plurality of tactile feedback generation mechanisms may be installed so that a wide variety of tactile feedback patterns can be provided according to the operation on the user's touch panel and the display contents of the touch panel. For example, Patent Document 2 (Japanese Unexamined Patent Publication No. 2012-203895) discloses a touch panel device in which a piezoelectric vibration element is mounted on an upper portion and a lower portion of the smartphone under a glass substrate of the smartphone. Since the piezoelectric vibration elements are provided in two places, by controlling the amplitude and phase of the signal input to each, it is possible to induce a strong amplitude in a specific place in the plane of the protective panel 22, and a finger or the like can be used. It is possible to selectively vibrate the touched part.

Furthermore, in recent years, systems that allow users to operate specific characters and objects in a simulated virtual space have been proposed in multiple technical fields for the purposes of education and training, entertainment, and the like. For example, Patent Document 3 (Japanese Patent Laid-Open No. 2020-515891) discloses a system and a method for training and collaborative work in a virtual environment, whereby a plurality of people in remote locations are in the same virtual space. Above, collaborative work such as surgery can be performed.

In addition, some business-use game systems are designed so that games using VR (Virtual Reality) technology, so-called VR games, can be played. For example, Patent Document 4 (Japanese Unexamined Patent Publication No. 2017-143978) discloses a game system designed so that a gallery can appreciate the state of game play from the outside while considering safety.

International Publication No. 2012/023605 Japanese Unexamined Patent Publication No. 2012-203895 Special Table 2020-515891 Japanese Unexamined Patent Publication No. 2017-143978

In recent years, the use of simulated virtual spaces has been expanding more and more for the purpose of user convenience and entertainment, as well as advanced systems such as surgical systems and arcade game systems. For example, a two-dimensional display means such as a television, a smartphone, or a computer display displays a virtual space constructed in three dimensions, whereas a user uses an input device such as a remote control, a touch panel, a game controller, a mouse, or a keyboard. There is a technique for realizing movements such as movement of a viewpoint and movement of a character or an object in the virtual space by inputting an operation.

In a simulated virtual space, changes due to user input or some event are usually displayed to the user by some display means, and by looking at it, the user can grasp the changes in the virtual space. It is desired to provide information other than visual information to the user in order to more intuitively convey the event in the virtual space to the user and further enhance the sense of presence.

For example, when displaying a three-dimensional virtual space on a two-dimensional display device such as a computer display, it may be necessary to move the viewpoint from the display surface to the depth direction or the front direction, but the movement of the viewpoint is guided. Therefore, it is desirable to provide the user with tactile information as well as visual information in order to notify the user of the start / end of the movement of the viewpoint.

In addition, when a character or object operated by the user comes into contact with another character or object in the virtual space, the tactile sensation corresponding to the contact can be fed back to the user to enhance the sense of presence and to the user. It is preferable because it can also alert you to react quickly to it.

The present invention has been completed in view of the above points, and in one embodiment, it is possible to provide tactile feedback that can enhance the sense of presence in an input device capable of inputting a user's operation in a simulated virtual space. The subject is to provide an input device.

As a result of diligent studies by the present inventors, we focused on the fact that most of the simulated virtual space is three-dimensional, and found a configuration capable of giving tactile feedback corresponding to it, leading to the completion of the present invention. .. Hereinafter, some embodiments of the present invention will be illustrated.
[1]
An input device that can input user operations in a simulated virtual space.
It comprises a plurality of actuators configured to provide haptic feedback to the input device.
The plurality of actuators are input devices arranged so as to be able to jointly generate the tactile feedback in a three-dimensional direction.
[2]
The plurality of actuators consist of three actuators.
Each actuator can generate the tactile feedback by vibrating or displacing in one direction, and the directions in which each actuator vibrates or displaces are arranged so as to be orthogonal to each other in [1]. The input device described.
[3]
The input device according to [1] or [2], wherein the direction of the tactile feedback corresponds to the direction of the operation by the user or the operation with respect to the user in the virtual space.
[4]
The input device according to any one of [1] to [3], which is a touch panel device.
[5]
The input device according to any one of [1] to [3], which is a means for receiving game operations in a virtual space.
[6]
The input device according to any one of [1] to [3], which is an operation receiving means for driving a car simulated in a virtual space.
[7]
The input device according to any one of [1] to [3], which is an operation receiving means for a surgical operation simulated on a virtual space.

According to the present invention, it is possible to provide tactile feedback that can enhance the sense of presence in an input device capable of inputting a user's operation in a simulated virtual space.

It is a figure which shows the operation method of the touch panel in one Embodiment of this invention. It is a figure which shows the application example of the embodiment of FIG. 1A. It is a figure which shows another application example of the embodiment of FIG. 1A. It is a figure which shows an example of the VR game system which includes the VR controller in one Embodiment of this invention. It is a block diagram which shows the operation method of the VR controller in one Embodiment of this invention. It is a figure which shows the example which gives 3D tactile feedback corresponding to the operation in the game of a user in one Embodiment of this invention. It is a figure which shows the example which gives 3D tactile feedback corresponding to the operation in the game of a user in one Embodiment of this invention. It is a figure which shows the example which gives 3D tactile feedback corresponding to the operation in the game of a user in one Embodiment of this invention. It is a figure which shows an example of the automobile driving simulator in one Embodiment of this invention. It is a block diagram which shows the operation method of the handle in one Embodiment of this invention. It is a figure which shows the example which gives 3D tactile feedback corresponding to the simulated automobile driving motion in one Embodiment of this invention. It is a figure which shows the example which gives 3D tactile feedback corresponding to the simulated automobile driving motion in one Embodiment of this invention. It is a figure which shows an example of the surgical operation simulator in one Embodiment of this invention. It is a block diagram which shows the operation method of the surgical instrument in one Embodiment of this invention. It is a figure which shows the example which gives 3D tactile feedback corresponding to the motion of the simulated surgical operation in one Embodiment of this invention.

Next, an embodiment of the present invention will be described in detail with reference to the drawings. It is understood that the present invention is not limited to the following embodiments, and design changes, improvements, etc. may be appropriately made based on ordinary knowledge of those skilled in the art without departing from the spirit of the present invention. Should be.

The present invention is, in one embodiment, an input device capable of inputting user operations in a simulated virtual space.
It comprises a plurality of actuators configured to provide haptic feedback to the input device.
The plurality of actuators provide an input device that is arranged in cooperation with each other so as to generate the tactile feedback in a three-dimensional direction.

(1. Simulated virtual space)
As long as the simulated virtual space is three-dimensional, the means for realizing it is not particularly limited, but it can be realized by utilizing XR technology such as VR technology, AR technology, MR technology and SR technology. Is.

Here, VR is an abbreviation for "Virtual Reality". Translated as "virtual reality" in Japanese. It is a technology that allows you to create a virtual world that resembles reality on a computer and experience the feeling of being there.

Also, AR is an abbreviation for "Augmented Reality". Translated as "augmented reality" in Japanese. While VR (virtual reality) is "another virtual space", AR adds digital information created by CG (Computer Graphics) to the real world. In other words, it is a technology that reflects (extends) virtual reality in the real world. Unlike VR, which is mainly based on the world of "virtual space", AR is mainly based on the "real world".

Also, MR is an abbreviation for "Mixed Reality". Translated as "mixed reality" in Japanese. It is a technology that allows you to experience the virtual world by superimposing it on the real world. In the case of MR, contrary to the AR, the subject is a virtual world (digital space). Reflect real-world information in the virtual world through cameras and the like. Since the information in the real world can be fixed in the virtual world, multiple people in the same MR space can obtain the information and have the same experience at the same time.

SR is an abbreviation for "Substitutional Reality". Translated as "substitutional reality" in Japanese. By projecting the past images that were shot in advance on the images in the real world using a display such as a head-mounted display, the illusion that the events that occurred in the past are real events is created. Can be caused.

Also, XR is an abbreviation for "X Reality". It is a term used as a general term for the VR, AR, MR, and SR technologies described above, rather than the terms that refer to the individual technologies themselves, such as VR and AR described above.

The content of the user's operation in the simulated virtual space is not particularly limited, but for example, the movement of the viewpoint, the movement of a specific character or object, the trigger of a specific event, and the change of the attribute related to the virtual space. And so on.

(2. Input device)
The type of input device is not particularly limited as long as the user's operation can be input, but for example, in addition to conventional general-purpose input devices such as a mouse, keyboard, touch panel, and game controller, a VR controller, an AR controller, and the like. Can be mentioned. In the case of an input device that can be operated three-dimensionally, such as a VR controller, it is preferable that a tracking function (a function for quickly recognizing the position / rotation of the input device) for the input device is installed.

Each specific embodiment of the input device of the present invention will be described later.

(3. Actuator)
The input device of the present invention is equipped with a plurality of actuators configured to provide haptic feedback, and the plurality of actuators are arranged so as to be able to jointly generate haptic feedback in a three-dimensional direction.

The present invention can include any actuator that produces various tactile feedback effects. Both a shock-driven actuator and a vibration-driven actuator can be used for the present invention, but a shock-driven actuator that is relatively simple in operation and whose kinematic characteristics can be easily specified is preferable. The tactile feedback system described later will be described by taking an impact-driven actuator as an example, but it goes without saying that the tactile feedback system can also be applied to a vibration-driven actuator.

For example, the actuator in one embodiment of the present invention is made of a coil, a moving component (for example, a metal core material) and a spring. The coil is wound around the metal (where both the coil and the metal component may be referred to as "solenoids") and when the coil creates a magnetic field (eg, when a current flows between the coil terminals). ) The metal moves. The movement of the metal can have an impact on the electronic device. The spring may then be used to return the moving metal or other core material to a stationary position when the current is removed from the coil.

Alternatively, the impact-driven actuator in another embodiment has a wire-shaped shape memory alloy that contracts when energized and heated, and an insulating heat conduction that contacts the wire-shaped shape memory alloy and releases heat generated by the wire-shaped shape memory alloy. It is configured to include a body (moving component). When the wire-shaped shape memory alloy is momentarily energized by the drive circuit and the wire-shaped shape memory alloy is momentarily contracted, the insulating thermal conductor in contact with the wire-shaped shape memory alloy is momentarily pressed and displaced. .. Displacement of the insulating thermal conductor can cause a shock to the electronic device. Then, the heat generated in the wire-shaped shape memory alloy is rapidly dissipated by the heat conduction action of the insulating heat conductor, and as a result, the wire-shaped shape memory alloy immediately returns to the original length state (extended state). In this way, the wire-shaped shape memory alloy can be momentarily shrunk at relatively short time intervals. Details of such actuators are disclosed in International Publication No. 2012/023605.

Alternatively, the impact-driven actuator in another embodiment has a wire-shaped shape memory alloy that contracts when energized and heated, and an insulating heat conduction that contacts the wire-shaped shape memory alloy and releases heat generated by the wire-shaped shape memory alloy. It is configured to include a body stator (fixing component) and a mover (moving component). When the wire-shaped shape memory alloy is momentarily energized by the drive circuit and the wire-shaped shape memory alloy is momentarily contracted, the mover of the insulating thermal conductor in contact with the wire-shaped shape memory alloy is momentarily pressed. And displace. Displacement of the mover of the insulating thermal conductor can cause a shock to the electronic device. Then, the heat generated in the wire-shaped shape memory alloy is rapidly dissipated by the heat conduction action of the insulating heat conductor, and as a result, the wire-shaped shape memory alloy immediately returns to the original length state (extended state). In this way, the wire-shaped shape memory alloy can be momentarily shrunk at relatively short time intervals.

To prevent the position of the stator from changing, fix it to the parts of the electronic device with, for example, double-sided tape or adhesive, and attach the moving parts of the electronic device (housing, touch screen, etc.) to the mover to drive the impact. It is possible to bring an impact to an electronic device by sandwiching a mold actuator and holding down a part that operates with an elastic part such as a spring that returns them. The shapes of the stator and the mover are not particularly limited as long as the above functions can be realized, and can be appropriately selected from a disk shape, a wavy shape, a columnar shape, and the like.

In order for each actuator to generate tactile feedback in the three-dimensional direction, it is necessary to control the orientation of each actuator in the input device. For example, if tactile feedback can be generated by vibrating or displacing each actuator in one direction, by arranging each actuator so that the directions of vibration or displacement of each actuator are orthogonal to each other, 3 It is possible to generate tactile feedback in any direction in the dimension.

For example, by arranging one actuator that vibrates or displaces in the X-axis direction, the Y-axis direction, and the Z-axis direction in the (X, Y, Z) three-dimensional coordinate axes, the operation / stationary of each actuator can be determined. And by adjusting the strength of the motion, vibration or displacement in the direction of the vector sum of the vector representing the movement amount in the X-axis direction, the vector representing the movement amount in the Y-axis direction, and the vector representing the movement amount in the Z-axis direction. Can be generated, and the user can sense the tactile feedback from or in the direction.

Since the direction of the tactile feedback perceived by the user is the direction of the vector sum of vibrations or displacements by each actuator, it is not necessary to arrange the plurality of actuators in exactly orthogonal directions, and the tactile sensation is cooperatively performed in the three-dimensional direction. The feedback may be arranged in a direction in which it can be generated. Further, the plurality of actuators may be arranged in the vicinity of the input device, but it is not always necessary, and the plurality of actuators may be arranged at locations separated from each other depending on the shape and internal structure of the input device.

It is preferable that there are three or more actuators in order to generate tactile feedback in the three-dimensional direction. There is no particular upper limit on the number of actuators, but from the viewpoint of miniaturization of the input device and production cost, it can be, for example, 7 or less, for example, 6 or less, for example, 5 or less. It can be, for example, 4 or less.

To more intuitively convey events in the simulated virtual space to the user, or to enhance the immersive feeling, or to encourage the user's reaction, the direction of the tactile feedback is the user in the simulated virtual space. It is preferable that it corresponds to the direction of the operation by the user or the operation with respect to the user. By "corresponding", when an action of the user or an action for the user occurs in the real space, the tactile feedback is generated in the same direction as the tactile sense to be sensed by the user.

The movement by the user includes the movement of the character or object that the user controls in the virtual space, and the tactile feedback corresponding to the movement is typically generated when the character or object comes into contact with another character or object. Including impact or recoil. The motion for the user includes the motion received by the character or object that the user steers in the virtual space, and the tactile feedback corresponding to the motion is typically when the character or object is touched by another character or object. Includes impact or feedback generated in.

(4. First Embodiment)
In one embodiment of the invention, the input device is a touch panel device. The driving method of the touch panel 10 is not limited, and any known driving method such as a pressure-sensitive method, a capacitance method, and a resistance film method can be adopted. It is possible to place a display device that displays the simulated virtual space under the touch panel, but it is possible to place a display device that displays the simulated virtual space independently of the touch panel. Is. The driving method of the display device is not limited, and a display device using an organic EL or the like can be used in addition to the liquid crystal display device.

FIG. 1A is a diagram showing a method of operating a touch panel according to an embodiment of the present invention. A display device 12 is arranged below the touch panel 10, and a simulated virtual space is displayed. Further, in close contact with the touch panel, the three actuators 14 are arranged so that their vibration directions are in the X-axis direction, the Y-axis direction, and the Z-axis direction (the actuator 14 in the Z-axis direction is not shown). Arrows relating to the X-axis, Y-axis, and Z-axis at the bottom of FIG. 1A are time axes indicating changes in the operation of the respective actuators 14 over time. The upper point in FIG. 1A indicates the time point corresponding to the operation of each lower actuator 14, the dotted line indicates the movement of the viewpoint in the virtual space, and the solid line indicates the movement of the user's finger.

First, when the user's finger touches the touch panel 10, the X-axis direction actuator 14 and the Y-axis direction actuator 14 are activated to generate tactile feedback in the direction of the vector sum of the vector in the X-axis direction and the vector in the Y-axis direction. , Guide the user's finger to the upper right. In this example, by moving the user's finger to the upper right, the operation at the point in the virtual space to the right of the point currently being operated by the user (that is, the point to the right on the depth side in the virtual space). Is possible. When the user's finger moves in the upper right direction, the viewpoint in the virtual space can also move in the upper right direction in conjunction with it. This guidance is provided, for example, when the object to be operated by the user is on the depth side in the virtual space, or when the system determines that the user needs to operate on the depth side in the virtual space in order to progress the game or the like. It is possible to be executed. Therefore, in addition to the operation of the X-axis direction actuator 14 and the Y-axis direction actuator 14, it is also possible to add the operation of the actuator in the Z-axis direction to inform the user that the guidance is to the depth side. Further, when the actuator 14 reaches the depth side of the virtual space through the guidance in the X-axis direction, the Z-axis direction actuator 14 is strongly driven in the depth direction to notify the user that the user has reached the depth side of the virtual space. Further, when the destination point on the depth side of the virtual space is reached through the guidance in the X-axis direction and the lower right direction, the Z-axis direction actuator 14 is strongly driven in the depth direction to notify the user that the destination point has been reached.

For example, as shown in FIG. 1B, in a shooting game, when a user aims at a distant target, when the user aims at a target near the target (here, the left front), the target is on the right front. In order to show the user and fine-tune the aim, the user is guided to the upper right.

Alternatively, for example, as shown in FIG. 1C, in the balance operation game, the user operates the table to prevent the ball on the table from falling from the table. The table is rectangular, with one side of the long side facing the user's left front and the other side facing the user's right back. When the user puts his finger on the front side of the table, the table tilts toward the front side correspondingly and the ball tries to fall. Therefore, the game system guides the user's finger toward the upper right of the screen in order to adjust the orientation of the table, and when the user's finger moves toward the upper right of the screen accordingly, the table returns to the horizontal position or the front right side moves. Tilt and stop the movement of the ball.

In this way, the tactile feedback in the Z-axis direction (depth direction of the screen display) can more intuitively convey to the user the movement in the depth direction in the three-dimensional virtual space displayed on the two-dimensional display device 12. ..

Further, the Z-axis direction actuator 14 may be driven only when it reaches a specific point on the depth side of the virtual space, for example, so that the vibration becomes stronger (or weaker) as it advances to the depth side of the virtual space. It is also possible to provide continuously changing tactile feedback.

(5. Second Embodiment)
In another embodiment of the present invention, the input device is a game operation receiving means in a virtual space. Typically, the operation receiving means is a VR controller, which is held and used by the user. The VR controller can be equipped with, for example, a gyro sensor or an acceleration sensor, and can detect the movement of the user's hand holding the VR controller. The VR controller is equipped with a plurality of actuators arranged so as to be able to generate tactile feedback in the three-dimensional direction in cooperation with each other.

FIG. 2 is a diagram showing an example of a VR game system including a VR controller. In order to play a VR game, a user usually wears a VR headset or VR goggles and operates using a VR controller that can move in three dimensions with both hands or one hand. The VR headset detects the position of the user in the real space and sends the position information to the simulation system. The game server in the simulation system or the game server installed independently of the simulation system calculates the position coordinates in the virtual space based on the position coordinates in the user's real space, and outputs the corresponding video to the VR headset. do. While the game progresses, the position and rotation of the VR controller are tracked and reflected in the simulation system in real time. When the simulation system determines that tactile feedback to the user is necessary, such as when the user performs a predetermined operation using the VR controller, or when a specific event occurs in the game system, the simulation system uses the VR controller. Send instructions to generate tactile feedback in a particular direction.

More specifically, as shown in FIG. 3, when the data receiving unit of the VR controller receives an instruction from the simulation system, the arithmetic unit (not shown) mounted on the VR controller is based on the instruction. Calculate the direction and strength (load direction) at which tactile feedback should be generated. For example, when a character operated by a user touches a wall in a simulated virtual space, the direction and intensity of the impact to be received by the user are calculated based on the location of the contact and the speed at the time of contact. Further, based on the calculated direction and strength, it is calculated how strong each actuator should be vibrated (vibration force calculation). Each actuator is driven based on the calculation result of the vibration force. The number of actuators actually driven may be the minimum number that can realize the direction and strength of the feedback, or may be larger than that. The load direction calculation and the vibration force calculation may be performed by an arithmetic unit mounted in the VR controller, or may be calculated in the simulation system. When calculated in the simulation system, the drive signal of each actuator is sent directly to each actuator.

FIGS. 4A to 4C show an example of giving three-dimensional tactile feedback corresponding to the user's movement in the game. In the illustrated embodiment, the user controls a virtual character in the game and uses a VR controller to move the sword. The VR controller is equipped with three actuators, and the arrows relating to the X-axis, Y-axis, and Z-axis are time axes indicating changes in the operation of each actuator over time. When the user stabs the controller, the character in the game stabs the sword accordingly. When the tip of the sword hits the target, the direction and strength at which the repulsive force should be generated are calculated based on the direction and speed of the movement of the sword. The actuator is driven based on the calculation result. In the illustrated embodiment, the actuator in the X-axis direction is driven downward to feed back the downward repulsion to the user (FIG. 4A).

Next, when the tip of the sword of the virtual character pierces the target, the repulsive force becomes stronger, so the downward vibration of the actuator in the X-axis direction becomes stronger, and the user is informed that the repulsive force becomes stronger. Give feedback (Fig. 4B). Furthermore, when the movement of the virtual character stops with the sword stuck in the target, in order to express the instability after releasing the force, not only the actuator in the X-axis direction but also the Y-axis direction and the Z-axis direction The actuator of is also driven at the same time, and feeds back the state in which the hand trembles (Fig. 4C). The arrows shown on the left side of each of FIGS. 4A to 4C indicate the direction of the tactile feedback that the user senses from the VR controller.

(6. Third Embodiment)
In another embodiment of the present invention, the input device is an operation receiving means for driving a vehicle simulated in a virtual space. Typically, the operation receiving means is a steering wheel for driving simulation. The handle is equipped with a plurality of actuators configured to provide tactile feedback along its circumferential direction, and the plurality of actuators can cooperate to generate tactile feedback in a three-dimensional direction.

FIG. 5 is a diagram showing an example of a car driving simulator. The simulated virtual space is displayed on the monitor, and specifically, the inside and outside of the vehicle as seen from the driver's seat are displayed. When the user turns the steering wheel, the simulation system calculates how the vehicle moves depending on the direction of the steering wheel, and also calculates the direction and strength of the force that the steering wheel should receive due to the movement of the vehicle, and based on the calculated result. The actuator is driven. At the same time, the movement of the vehicle is reflected on the monitor.

More specifically, as shown in FIG. 6, the simulation system is based on the situation of the car operated by the player in the simulation (collision, riding, running on uneven road, etc.) and where in the car. , Sends information about how much magnitude / orientation force or impact is applied to the data receiver of the handle. Next, based on this information, the arithmetic unit (not shown) mounted on the handle calculates the direction and strength (load direction) at which tactile feedback should be generated. For example, when a user-operated vehicle comes into contact with another vehicle in a simulated virtual space, the direction and intensity of the impact that the handle should receive is calculated based on the location of the contact and the speed at the time of contact. Further, based on the calculated direction and strength, it is calculated how strong each actuator should be vibrated (vibration force calculation). Each actuator is driven based on the calculation result of the vibration force. The number of actuators actually driven may be the minimum number that can realize the direction and strength of the feedback, or may be larger than that. The load direction calculation and the vibration force calculation may be performed by an arithmetic unit mounted in the handle, or may be calculated in the simulation system. When calculated in the simulation system, the drive signal of each actuator is sent directly to each actuator.

7A and 7B show an example of providing three-dimensional tactile feedback in response to a simulated vehicle driving motion. The figure shows how the vehicle rides on the curb on the left side of the road. In the illustrated embodiment, in simulated vehicle driving, the user holds the steering wheel to steer a virtual vehicle. Three actuators are mounted, and the arrows relating to the X-axis, Y-axis, and Z-axis are time axes indicating changes in the operation of each actuator over time. When a virtual vehicle operated by the user rides on the curb from the left front wheel in the virtual space, the vehicle receives an impact from the left, and therefore, from the driver's point of view, the impact is directed to the right. In the illustrated embodiment, the actuator in the X-axis direction is driven to the right and feeds back the impact to the right to the user (FIG. 7A). In addition, there is also an impact that the vehicle is pushed up from below by riding on the curb, and in response to this, the actuator in the Z-axis direction is also driven upward to a certain extent (FIG. 7A).

Next, when the vehicle advances further and the left rear wheel also rides on the curb, the speed of the vehicle temporarily decreases due to the obstruction by the curb, so from the driver's point of view, it receives a backward impact. In the illustrated embodiment, the actuator in the Y-axis direction is driven backwards to feed back the backward impact to the user (FIG. 7B). In addition, there is also an impact that the vehicle is pushed up from below by riding on the curb, and in response to this, the actuator in the Z-axis direction is also driven upward to a certain extent (FIG. 7B).

Since an impact is generated when each wheel rides on the curb, four impacts are generated until the vehicle rides on the curb as a whole, and the actuator is driven four times accordingly. In the present embodiment, only the driving state when the left front wheel and the left rear wheel get on is shown, and the others are omitted.

(7. Fourth Embodiment)
In another embodiment of the invention, the input device is a surgical operation receiving means simulated in a virtual space. Typically, the operating receiving means is a controller for moving the surgical instrument. The shape of the controller can vary widely depending on the shape of the corresponding surgical instrument. For example, it can have a shape similar to a scalpel, a sessi, a pistol, nippers, and the like. The controller is equipped with a plurality of actuators configured to provide haptic feedback, and the plurality of actuators can cooperate to generate haptic feedback in a three-dimensional direction. Surgery simulated in virtual space can also be linked to actual surgery to realize remote surgery.

FIG. 8 is a diagram showing an example of a surgical operation simulator. The monitor displays a simulated virtual space, specifically what the doctor is doing during surgery. When the user moves the surgical instrument (controller), the simulation system calculates the direction and strength of the force that the surgical instrument should receive in the movement of the surgical instrument, and the actuator is driven based on the calculated result. At the same time, the movement of the surgical instrument is reflected on the monitor.

More specifically, as shown in FIG. 9, the simulation system transmits information about the force or impact applied to the human or animal undergoing surgery from the information in the simulation to the data receiving unit of the surgical instrument. Next, based on this information, the arithmetic unit (not shown) mounted on the surgical instrument calculates the direction and strength (load direction) at which tactile feedback should be generated. For example, when a user-operated surgical instrument comes into contact with a tissue of a human body in a simulated virtual space, the direction and intensity of the impact that the surgical instrument should receive are calculated based on the hardness of the tissue and the speed at the time of contact. .. Further, based on the calculated direction and strength, it is calculated how strong each actuator should be vibrated (vibration force calculation). Each actuator is driven based on the calculation result of the vibration force. The number of actuators actually driven may be the minimum number that can realize the direction and strength of the feedback, or may be larger than that. The load direction calculation and the vibration force calculation may be performed by an arithmetic unit mounted in the surgical instrument, or may be calculated in the simulation system. When calculated in the simulation system, the drive signal of each actuator is sent directly to each actuator.

FIG. 10 shows an example of providing three-dimensional tactile feedback in response to a simulated surgical movement. The illustration shows a state in which a surgical instrument having a shape similar to a nipper moves while sandwiching an object (tissue). In the illustrated embodiment, in a simulated surgical operation, the user moves a surgical instrument to perform a virtual operation. Three actuators are mounted on the surgical instrument, and the arrows relating to the X-axis, Y-axis, and Z-axis at the bottom of the drawing are time axes indicating changes in the operation of each actuator over time. When a surgical instrument operated by a user pulls upward while sandwiching an object in a virtual space, it receives downward resistance. In the illustrated embodiment, the actuator in the Z-axis direction is driven downward and feeds back the downward resistance to the user. Further, when the object is pulled upward and then sideways, the direction of the resistance becomes sideways, and the actuator in the Y-axis direction is driven sideways to feed back the tactile sensation of the resistance. Further, although not shown in the drawing, when the surgical instrument is moved toward the front side or the depth direction of the paper surface, the actuator in the X-axis direction is driven laterally correspondingly to feed back the tactile sensation of resistance.

The description of each of the above embodiments is merely a description of a preferred embodiment of the present invention, and the present invention can be applied to any situation in which user operation is required in a simulated virtual space. Is.

Claims

An input device that can input user operations in a simulated virtual space.
It comprises a plurality of actuators configured to provide haptic feedback to the input device.
The plurality of actuators are input devices arranged so as to be able to jointly generate the tactile feedback in a three-dimensional direction.
The plurality of actuators consist of three actuators.
According to claim 1, each actuator can generate the tactile feedback by vibrating or displacing in one direction, and the directions in which the respective actuators vibrate or displace are arranged so as to be orthogonal to each other. The input device described.
The input device according to claim 1 or 2, wherein the direction of the tactile feedback corresponds to the direction of the operation by the user or the operation with respect to the user in the virtual space.
The input device according to any one of claims 1 to 3, which is a touch panel device.
The input device according to any one of claims 1 to 3, which is a means for receiving game operations in a virtual space.
The input device according to any one of claims 1 to 3, which is an operation receiving means for driving a car simulated in a virtual space.
The input device according to any one of claims 1 to 3, which is a means for receiving an operation of a surgical operation simulated on a virtual space.