JP6693160B2 - Display device, display device control method, and program - Google Patents

Description

  The present invention relates to a display device, a display device control method, and a program.

  2. Description of the Related Art Conventionally, there is known an example in which a display device is operated by a device that a user (user) holds in his / her hand (for example, refer to Patent Document 1). The remote control device described in Patent Document 1 is provided with a ring-shaped housing or body and is operated by a user gripping it. The movement of the remote control device is reflected in the display.

Japanese Patent Publication No. 2009-500923

In the configuration described in Patent Document 1, the operation of the remote control device is reflected on the display. In such a configuration in which the display is controlled according to the movement, there is an advantage that the operability is excellent because the operation is simple, but the display changes according to the movement regardless of the intention of the user. For this reason, there is a possibility that the display may change unintentionally by the user.
The present invention has been made in view of the above circumstances, and a display device, a display device control method, and a program capable of easily changing the display in response to an operation and suppressing a display change not intended by the user. The purpose is to provide.

In order to achieve the above object, the present invention is a head-mounted display device having a display unit for allowing a user to visually recognize an image, and an operation device that can be moved independently of the display unit. An operation device having a motion detection unit that detects a motion of the operation device, and an operation reception unit that receives an operation different from the detection of the motion detection unit; and a state in which the operation reception unit receives the operation, And a control unit that controls the display of the display unit according to the movement detected by the movement detection unit.
According to the present invention, the display on the display unit can be easily controlled by the operation of moving the operation device. Further, it is possible to prevent the display from being changed unintentionally by the user depending on the operation state of the operation receiving unit.

Further, in the display device according to the present invention, the operation receiving unit receives at least one of a pressing operation and a contact operation.
According to the present invention, it is possible to enhance the operability when operating the operation device and operating the operation receiving unit. In addition, since the operability is excellent when operating the operation reception unit without visually recognizing it, the operation reception unit can be easily operated even when the user wearing the head-mounted display device has difficulty in visually recognizing the operation device. It can be operated reliably.

In the display device according to the present invention, the control unit changes a display magnification of an image displayed on the display unit when the motion detection unit detects a motion corresponding to an operation of rotating the device. What to do is characterized.
According to the present invention, the display magnification can be changed by an intuitive operation.

Further, in the display device according to the present invention, the control unit displays an image to be displayed on the display unit when the motion detection unit detects a motion corresponding to an operation of lowering or raising the device. It is characterized in that it is changed in a preset direction.
According to the present invention, the image displayed on the display unit can be changed by an intuitive operation.

Further, in the display device according to the present invention, the control unit displays the display when the operation is canceled after the display of the display unit is changed while the operation receiving unit is receiving the operation. It is characterized in that the change of the display of the part is confirmed.
According to the present invention, it is possible to perform an operation of confirming a change in display on the display unit. Therefore, the degree of freedom in changing the display can be increased, and the operability can be improved.

In the display device according to the present invention, the control unit is set with a predetermined function to be executed when the operation receiving unit receives an operation, and the predetermined function is provided when the operation receiving unit receives an operation. Switching between a normal mode for execution and a display operation mode for controlling the display of the display unit according to the motion detected by the motion detection unit while the operation reception unit is accepting an operation. And
According to the present invention, the operation reception unit can be used as an operation unit for instructing a predetermined function.

In the display device according to the present invention, the control unit may perform the display operation when the operation accepting unit accepts an operation within a set time after the motion detecting unit detects the start of motion. Running mode.
According to the present invention, it is possible to easily select whether to execute the normal mode or the display operation mode by operating the operation receiving unit.

Further, in the display device according to the present invention, when the control unit starts execution of the display operation mode, the movement detection unit detects a start of movement and the operation reception unit receives an operation. In the meantime, the display on the display unit is controlled by reflecting the detection history detected by the motion detection unit.
According to the present invention, the movement of the operation device until the display operation mode is started can be reflected in the display control.

Further, the present invention is characterized in that, in the display device, the operation device includes a position instruction operation unit that receives a position instruction operation.
According to the present invention, it is possible to perform a position pointing operation using the operation device.

In the display device according to the present invention, the control unit may receive the position pointing operation by the position pointing operation unit, and the operation receiving unit may receive the operation. The display on the display unit is changed with reference to the display position corresponding to the position instructed by the position instructing operation.
According to the present invention, the display of the display unit can be controlled with a high degree of freedom by utilizing the position pointing operation and the movement of the operation device. Moreover, complicated display control can be performed by a simple operation, and operability can be improved.

Further, in the display device according to the present invention, the control unit enlarges an image displayed on the display unit with a display position corresponding to a position designated by a position designation operation received by the position designation operation unit as a center. The feature is that the image is reduced and displayed.
According to the present invention, control for enlarging / reducing a display centered on a position desired by a user can be realized by a simple operation using an operation device.

Further, the present invention is characterized in that, in the display device, the position pointing operation unit functions as the operation receiving unit.
According to the present invention, since the number of objects to be operated is small, the operability when operating without operating the operating device is excellent, and thus it is difficult for a user wearing a head-mounted display device to view the operating device. Even in the state, it can be operated easily and surely. Further, it is advantageous when the operation device is downsized.

In the display device according to the present invention, the display unit includes a display unit movement detection unit that detects a movement of the display unit, and the control unit sets the position at the timing when the display unit movement detection unit detects a predetermined movement. It is characterized in that the pointing position of the operation accepted by the pointing operation unit is specified.
According to the present invention, it is possible to perform the operation of confirming the pointed position during the position pointing operation by the movement of the display unit. Therefore, it is possible to further improve the operability when the user wearing the head-mounted display device operates the operation device.

Further, in the display device according to the present invention, the control unit controls the display unit according to a type of operation corresponding to a motion detected by the motion detection unit in a state where the operation reception unit is receiving an operation. A first control for controlling a display, a second control for changing a display of the display unit based on a display position corresponding to a position designated by a position designation operation received by the position designation operation unit, and the motion detection The third control for changing the display of the display unit according to the direction of the movement detected by the unit can be switched and executed.
According to the present invention, it is possible to change the display of the display unit by three types of control according to the movement of the operation device.

Further, in the display device according to the present invention, the control unit moves an image displayed by the display unit in a first display operation direction and a second display operation direction forming a pair with the first display operation direction. Control is performed, the first display operation direction is associated with the first movement direction detected by the motion detection unit, and the second display operation direction is the second movement detected by the motion detection unit. It is characterized by being associated with a direction.
According to the present invention, the image displayed on the display unit can be moved by moving the operation device, and the direction of movement of the image can be indicated by the movement direction of the operation device.

Also, in the display device according to the present invention, the control unit starts the movement of the image in the first display operation direction in response to the movement in the first movement direction detected by the movement detection unit. After that, when the movement in the second movement direction is detected by the movement detection unit, the movement of the image in the first display movement direction is stopped.
According to the present invention, an image displayed on the display unit can be moved to a desired position by combining movements of the operation device in two directions.

Further, in the display device according to the present invention, the control unit starts the movement of the image in the first display operation direction or the second display operation direction in response to the movement detected by the movement detection unit. After that, when the operation accepted by the operation accepting unit is canceled, the movement of the image on the display unit is stopped.
According to the present invention, the image displayed on the display unit can be moved to a desired position by combining the movement of the operation device and the operation of the operation reception unit.

In the display device according to the present invention, the operation device is a wearable device worn on the body of the user.
According to the present invention, the display on the display unit can be easily controlled by the operation of moving the user's body. In addition, since it is possible to prevent the display from being changed unintentionally by the user depending on the operation state of the operation receiving unit, it is possible to more intuitively perform the operation for controlling the display.

In order to achieve the above object, the present invention is a display unit that allows a user to visually recognize an image, and an operation device that can be moved independently of the display unit, and detects movement of the operation device. A display device including a motion detection unit and an operation device having an operation reception unit that receives an operation is controlled, and in a state in which the operation reception unit is receiving an operation, the display device is operated in accordance with a motion detected by the motion detection unit. The display of the display unit is controlled.
According to the present invention, the display on the display unit can be easily controlled by the operation of moving the operation device. Further, it is possible to prevent the display from being changed unintentionally by the user depending on the operation state of the operation receiving unit.

In order to achieve the above object, the present invention is a display unit that allows a user to visually recognize an image, and an operation device that can be moved independently of the display unit, and detects movement of the operation device. A program executable by a computer that controls a display device including a motion detection unit and an operation device having an operation reception unit that receives an operation, wherein the motion detection unit is in a state in which the operation reception unit receives an operation. Is a program for controlling the display of the display unit in accordance with the movement detected by.
According to the present invention, the display on the display unit can be easily controlled by the operation of moving the operation device. Further, it is possible to prevent the display from being changed unintentionally by the user depending on the operation state of the operation receiving unit.

  The present invention can be implemented in various forms other than the above-described display device, display device control method, and program. For example, it can be realized in the form of a recording medium recording the above program, a server device distributing the program, a transmission medium transmitting the program, a data signal embodying the program in a carrier wave, and the like.

Explanatory drawing which shows the external appearance structure of HMD. The figure which shows the structure of the optical system of an image display part. Explanatory drawing which shows the correspondence of an image display part and an imaging range. The block diagram of each part which constitutes HMD. The block diagram of a control part and a memory | storage part. Explanatory drawing which shows the mode of lighting of an LED display part. Explanatory drawing which shows the example of operation using a control apparatus. Explanatory drawing which shows the change of the display corresponding to operation of HMD. Explanatory drawing which shows the change of the display corresponding to operation of HMD. Explanatory drawing which shows the change of the display corresponding to operation of HMD. The flowchart which shows operation | movement of HMD. The flowchart which shows operation | movement of HMD. FIG. The external view of a ring type device.

FIG. 1 is an explanatory diagram showing an external configuration of an HMD (Head Mounted Display) 100 according to an embodiment to which the present invention is applied.
The HMD 100 is a display device including an image display unit 20 (display unit) that allows a user to visually recognize a virtual image while being mounted on the head of the user (user), and a control device 10 that controls the image display unit 20. Is. As shown in FIG. 1, the control device 10 (operating device) includes a flat box-shaped case 10A (also referred to as a housing or a main body), and the case 10A includes each unit described later. On the surface of the case 10A, various buttons 11, switches, a track pad 14 and the like for receiving user's operations are provided. When the user operates these, the control device 10 functions as a controller of the HMD 100.

The image display unit 20 is a mounted body mounted on the head of the user, and has a spectacle shape in the present embodiment. The image display unit 20 includes a right display unit 22, a left display unit 24, a right light guide plate 26, and a left light guide plate 28 in a main body having a right holding unit 21, a left holding unit 23, and a front frame 27. ..
The right holding unit 21 and the left holding unit 23 respectively extend rearward from both ends of the front frame 27, and hold the image display unit 20 on the user's head like a temple (vine) of eyeglasses. Here, of both ends of the front frame 27, an end located on the right side of the user in the mounted state of the image display unit 20 is referred to as an end ER, and an end located on the left side of the user is referred to as an end EL. To do. The right holding portion 21 is provided so as to extend from the end portion ER of the front frame 27 to a position corresponding to the right side head of the user in the mounted state of the image display unit 20. The left holding unit 23 is provided so as to extend from the end EL to a position corresponding to the left side head of the user in the mounted state of the image display unit 20.

  The right light guide plate 26 and the left light guide plate 28 are provided on the front frame 27. The right light guide plate 26 is located in front of the right eye of the user when the image display unit 20 is attached, and allows the right eye to visually recognize the image. The left light guide plate 28 is located in front of the left eye of the user when the image display unit 20 is attached, and allows the left eye to visually recognize the image.

  The front frame 27 has a shape in which one end of the right light guide plate 26 and one end of the left light guide plate 28 are connected to each other, and at this connection position, when the user wears the image display unit 20, Correspond between the eyebrows. The front frame 27 may be provided with a nose pad that comes into contact with the user's nose when the image display unit 20 is attached, at the connecting position of the right light guide plate 26 and the left light guide plate 28. In this case, the image display unit 20 can be held on the head of the user by the nose pad, the right holding unit 21, and the left holding unit 23. Further, a belt (not shown) may be connected to the right holding portion 21 and the left holding portion 23 so as to contact the user's occipital region when the image display unit 20 is worn. Can be held on the person's head.

  The right display unit 22 is a unit for displaying an image by the right light guide plate 26, is provided in the right holding portion 21, and is located near the right side of the head of the user in the mounted state. The left display unit 24 is a unit related to the display of an image by the left light guide plate 28, is provided in the left holding portion 23, and is located near the left head of the user in the mounted state. The right display unit 22 and the left display unit 24 are collectively referred to simply as "display drive unit".

The right light guide plate 26 and the left light guide plate 28 of the present embodiment are optical parts formed of a light-transmissive resin or the like, and are, for example, prisms, and generate image light output from the right display unit 22 and the left display unit 24. , To the eyes of the user.
Further, a light control plate (not shown) may be provided on the surfaces of the right light guide plate 26 and the left light guide plate 28. The light control plate is an optical element on a thin plate having a different transmittance depending on the wavelength range of light, and functions as a so-called wavelength filter. The light control plate is arranged, for example, so as to cover the front side of the front frame 27, which is the side opposite to the eye side of the user. By appropriately selecting the optical characteristics of the light control plate, it is possible to adjust the transmittance of light in an arbitrary wavelength range such as visible light, infrared light, and ultraviolet light, and the right light guide plate 26 and the left light guide plate are externally supplied. The amount of external light that enters the light plate 28 and that passes through the right light guide plate 26 and the left light guide plate 28 can be adjusted.

  The image display unit 20 guides the image light generated by the right display unit 22 and the left display unit 24, respectively, to the right light guide plate 26 and the left light guide plate 28, and allows the user to visually recognize a virtual image by the image light, thereby forming an image. Is displayed. When external light enters the eyes of the user from the front of the user through the right light guide plate 26 and the left light guide plate 28, the image light and the outside light forming a virtual image enter the eyes of the user. Therefore, the visibility of the virtual image is affected by the intensity of outside light. Therefore, for example, by mounting a light control plate on the front frame 27 and appropriately selecting or adjusting the optical characteristics of the light control plate, the visibility of the virtual image can be adjusted. In a typical example, it is possible to use a light control plate having a light transmittance that allows a user wearing the HMD 100 to visually recognize at least the outside scenery. Further, when the light control plate is used, it is expected that the right light guide plate 26 and the left light guide plate 28 are protected, and that the right light guide plate 26 and the left light guide plate 28 are prevented from being damaged or attached with dirt. The light control plate may be attachable to and detachable from the front frame 27, or each of the right light guide plate 26 and the left light guide plate 28. Alternatively, a plurality of types of light control plates may be replaceable and attachable. May be omitted.

  The camera 61 is arranged on the front frame 27 of the image display unit 20. It is desirable that the camera 61 capture an image of the outside scene direction that the user visually recognizes with the image display unit 20 mounted, and in the front surface of the front frame 27, outside light that passes through the right light guide plate 26 and the left light guide plate 28 can be detected. It is provided in a position that does not block. In the example of FIG. 1, the camera 61 is arranged on the end portion ER side of the front frame 27. The camera 61 may be arranged on the end EL side, or may be arranged on the connecting portion between the right light guide plate 26 and the left light guide plate 28.

The camera 61 is a digital camera including an image pickup device such as a CCD or CMOS, an image pickup lens, and the like. The camera 61 of the present embodiment is a monocular camera, but may be a stereo camera. The camera 61 images the front side direction of the HMD 100, in other words, at least a part of the external view (real space) in the visual field direction of the user with the HMD 100 mounted. In other words, it can be said that the camera 61 images a range or a direction that overlaps the field of view of the user and an image of the direction in which the user gazes. The width of the angle of view of the camera 61 can be set as appropriate, but in the present embodiment, as will be described later, it includes the outside world visually recognized by the user through the right light guide plate 26 and the left light guide plate 28. More preferably, the imaging range of the camera 61 is set so that the entire visual field of the user that is visible through the right light guide plate 26 and the left light guide plate 28 can be imaged.
The camera 61 executes image capturing under the control of the image capturing control unit 149 included in the control unit 150 (FIG. 5) and outputs captured image data to the image capturing control unit 149.

  The HMD 100 may include a distance sensor (not shown) that detects a distance to a measurement target object located in a preset measurement direction. The distance sensor can be arranged, for example, at the connecting portion of the right light guide plate 26 and the left light guide plate 28 in the front frame 27. In this case, in the mounted state of the image display unit 20, the position of the distance sensor is approximately in the middle of both eyes of the user in the horizontal direction and above the both eyes of the user in the vertical direction. The measurement direction of the distance sensor can be, for example, the front side direction of the front frame 27, in other words, the direction that overlaps the imaging direction of the camera 61. The distance sensor can be configured to include, for example, a light source such as an LED or a laser diode, and a light receiving unit that receives the reflected light that the light emitted from the light source reflects on the measurement target. Under the control of the control unit 150, the distance sensor may execute a triangular distance measuring process or a distance measuring process based on a time difference. Further, the distance sensor may be configured to include a sound source that emits ultrasonic waves and a detection unit that receives the ultrasonic waves reflected by the measurement target. In this case, the distance sensor may execute the distance measurement process based on the time difference until the reflection of the ultrasonic wave under the control of the control unit 150.

FIG. 2 is a main part plan view showing a configuration of an optical system included in the image display unit 20. FIG. 2 illustrates the left eye LE and the right eye RE of the user for the sake of explanation.
As shown in FIG. 2, the right display unit 22 and the left display unit 24 are configured symmetrically. The right display unit 22 includes an OLED (Organic Light Emitting Diode) unit 221 that emits image light and a lens group that guides the image light L emitted by the OLED unit 221 as a configuration for allowing the user's right eye RE to visually recognize the image. And a right optical system 251. The image light L is guided to the right light guide plate 26 by the right optical system 251.

The OLED unit 221 has an OLED panel 223 and an OLED drive circuit 225 that drives the OLED panel 223. The OLED panel 223 is a self-luminous display in which light emitting elements that emit light by organic electroluminescence and emit R (red), G (green), and B (blue) color lights are arranged in a matrix. It is a panel. The OLED panel 223 includes a plurality of pixels with a unit including one element of R, G, and B as one pixel, and an image is formed by the pixels arranged in a matrix. The OLED drive circuit 225 causes the light emitting elements of the OLED panel 223 to emit light by selecting the light emitting elements included in the OLED panel 223 and energizing the light emitting elements under the control of the control unit 150 (FIG. 5). The OLED drive circuit 225 is fixed to the back surface of the OLED panel 223, that is, the back surface of the light emitting surface, by bonding or the like. The OLED drive circuit 225 is composed of, for example, a semiconductor device that drives the OLED panel 223, and may be mounted on a substrate (not shown) fixed to the back surface of the OLED panel 223. A temperature sensor 217 is mounted on this board.
The OLED panel 223 may have a structure in which light emitting elements that emit white light are arranged in a matrix and color filters corresponding to the colors R, G, and B are arranged in an overlapping manner. Further, an OLED panel 223 having a WRGB structure including a light emitting element that emits W (white) light may be used in addition to a light emitting element that emits R, G, and B color lights, respectively.

  The right optical system 251 has a collimating lens that turns the image light L emitted from the OLED panel 223 into a parallel light flux. The image light L converted into a parallel light flux by the collimator lens enters the right light guide plate 26. Inside the right light guide plate 26, a plurality of reflecting surfaces that reflect the image light L are formed in an optical path that guides the light. The image light L is guided to the right eye RE side through a plurality of reflections inside the right light guide plate 26. On the right light guide plate 26, a half mirror 261 (reflection surface) located in front of the right eye RE is formed. The image light L is reflected by the half mirror 261 and emitted from the right light guide plate 26 toward the right eye RE, and the image light L forms an image on the retina of the right eye RE to allow the user to visually recognize the image.

  The left display unit 24 includes a OLED unit 241 that emits image light, a left optical system that includes a lens group that guides the image light L emitted by the OLED unit 241, and the like so that the left eye LE of the user can visually recognize the image. And 252. The image light L is guided to the left light guide plate 28 by the left optical system 252.

  The OLED unit 241 has an OLED panel 243 and an OLED drive circuit 245 that drives the OLED panel 243. The OLED panel 243 is a self-luminous display panel configured similarly to the OLED panel 223. Under the control of the control unit 150 (FIG. 5), the OLED drive circuit 245 selects a light emitting element included in the OLED panel 243 and energizes the light emitting element to cause the light emitting element of the OLED panel 243 to emit light. The OLED drive circuit 245 is fixed to the back surface of the OLED panel 243, that is, the back surface of the light emitting surface, by bonding or the like. The OLED drive circuit 245 is composed of, for example, a semiconductor device that drives the OLED panel 243, and may be mounted on a substrate (not shown) fixed to the back surface of the OLED panel 243. A temperature sensor 239 is mounted on this board.

  The left optical system 252 has a collimating lens that converts the image light L emitted from the OLED panel 243 into a parallel light flux. The image light L converted into a parallel light flux by the collimator lens enters the left light guide plate 28. The left light guide plate 28 is an optical element having a plurality of reflecting surfaces that reflect the image light L, and is, for example, a prism. The image light L is guided to the left eye LE side through a plurality of reflections inside the left light guide plate 28. On the left light guide plate 28, a half mirror 281 (reflection surface) located in front of the left eye LE is formed. The image light L is reflected by the half mirror 281 and emitted from the left light guide plate 28 toward the left eye LE, and the image light L forms an image on the retina of the left eye LE to allow the user to visually recognize the image.

According to this configuration, the HMD 100 functions as a see-through type display device. That is, the image light L reflected by the half mirror 261 and the external light OL transmitted through the right light guide plate 26 are incident on the right eye RE of the user. Further, the image light L reflected by the half mirror 281 and the external light OL transmitted through the half mirror 281 are incident on the left eye LE. In this way, the HMD 100 allows the image light L of the internally processed image and the external light OL to be superimposed and incident on the user's eye, and for the user, the right light guide plate 26 and the left light guide plate 28 are transparent to the outside scene. Can be seen, and the image by the image light L is visually recognized so as to be superimposed on this outside scene.
The half mirrors 261 and 281 are image extraction units that reflect the image lights output by the right display unit 22 and the left display unit 24 to extract images, and can be referred to as display units.

  The left optical system 252 and the left light guide plate 28 are collectively referred to as “left light guide portion”, and the right optical system 251 and the right light guide plate 26 are collectively referred to as “right light guide portion”. The configurations of the right light guide section and the left light guide section are not limited to the above examples, and any method can be used as long as a virtual image is formed in front of the eyes of the user by using image light. It may be used or a semi-transmissive reflective film may be used.

  Returning to FIG. 1, the control device 10 and the image display unit 20 are connected by a connection cable 40. The connection cable 40 is detachably connected to a connector (not shown) provided in the lower portion of the case 10A, and is connected to various circuits provided inside the image display unit 20 from the tip of the left holding unit 23. The connection cable 40 includes a metal cable that transmits digital data or an optical fiber cable, and may have a metal cable that transmits an analog signal. A connector 46 is provided in the middle of the connection cable 40. The connector 46 is a jack for connecting a stereo mini plug, and the connector 46 and the control device 10 are connected by a line for transmitting an analog audio signal, for example. In the configuration example illustrated in FIG. 1, the headset 30 including the right earphone 32 and the left earphone 34 that configure the stereo headphones, and the microphone 63 is connected to the connector 46.

  For example, as shown in FIG. 1, the microphone 63 is arranged such that the sound collecting section of the microphone 63 faces the direction of the user's line of sight, collects sound, and outputs a sound signal to the sound interface 182 (FIG. 4). To do. The microphone 63 may be, for example, a monaural microphone or a stereo microphone, a directional microphone, or an omnidirectional microphone.

The control device 10 includes a button 11, an LED indicator 12, a track pad 14, an up / down key 15, a changeover switch 16, and a power switch 18 as an operated portion operated by a user. These operated parts are arranged on the surface of the case 10A.
The button 11 includes a menu key, a home key, a return key, and the like for operating the operating system 143 (FIG. 5) executed by the control device 10, and in particular, it is displaced by a pressing operation of these keys and switches. Including things. The LED indicator 12 lights up or blinks according to the operating state of the HMD 100. The up / down key 15 is used for inputting an instruction to increase / decrease the volume output from the right earphone 32 and the left earphone 34, and to input an instruction to increase / decrease the display brightness of the image display unit 20. The changeover switch 16 is a switch for changing over the input corresponding to the operation of the up / down key 15. The power switch 18 is a switch that switches on / off the power of the HMD 100, and is configured by, for example, a slide switch.

The track pad 14 (operation accepting unit, position instruction operating unit) has an operation surface for detecting a contact operation, and outputs an operation signal according to an operation on the operation surface. The detection method on the operation surface is not limited, and an electrostatic method, a pressure detection method, an optical method or the like can be adopted. The contact (touch operation) on the track pad 14 is detected by a touch sensor 13 (FIG. 4) described later.
Further, as indicated by a broken line in FIG. 1, an LED display unit 17 is installed on the track pad 14. The LED display unit 17 includes a plurality of LEDs, and the operation unit can be visually recognized on the LED display unit 17 by lighting each LED. In the example of FIG. 1, three symbols Δ (triangle), ◯ (circle), □ (square) appear when the LED of the LED display unit 17 is turned on. When the LED display unit 17 is off, these symbols are invisible. This configuration can be realized by, for example, configuring the track pad 14 with a transparent or colored transparent flat plate and arranging the LEDs directly under the flat plate.

  The LED display unit 17 functions as a software button. For example, when the LED display unit 17 is lit, the lighting position (display position) of the symbol functions as a button for giving an instruction corresponding to the symbol. In the example of FIG. 1, the symbol ◯ (circle) functions as a home button, and when an operation of touching the position of the symbol ◯ (circle) is performed, the control unit 150, which will be described later, Detects the operation of the home button. The symbol □ (square) functions as a history button. When the operation of touching the position of the symbol □ (quadrangle) is performed, the control unit 150 described later detects the operation of the history button based on the detection value of the touch sensor 13. The symbol Δ (triangle) functions as a back button. When the operation of touching the position of the symbol Δ (triangle) is performed, the control unit 150 described later detects the operation of the return button based on the detection value of the touch sensor 13.

  3A and 3B are diagrams showing a configuration of a main part of the image display unit 20, where FIG. 3A is a perspective view of the main part of the image display unit 20 viewed from the user's head side, and FIG. FIG. The connection cable 40 is not shown in FIG.

FIG. 3A is a side of the image display unit 20 in contact with the user's head, in other words, a side visible to the user's right eye RE and left eye LE. In other words, the back sides of the right light guide plate 26 and the left light guide plate 28 are visible.
In FIG. 3A, the half mirror 261 that irradiates the user's right eye RE with the image light and the half mirror 281 that irradiates the left eye LE with the image light are seen as substantially rectangular areas. Further, the entire right light guide plate 26 and left light guide plate 28 including the half mirrors 261 and 281 transmit outside light as described above. Therefore, the user sees the outside scene through the right light guide plate 26 and the left light guide plate 28 as a whole, and sees the rectangular display image at the positions of the half mirrors 261 and 281.

  The camera 61 is arranged at the right end of the image display unit 20 as described above, and images the direction in which both eyes of the user face, that is, the front of the user. FIG. 3B is a diagram schematically showing the position of the camera 61 in a plan view together with the right eye RE and the left eye LE of the user. The angle of view (imaging range) of the camera 61 is indicated by C. Although FIG. 3B shows the horizontal angle of view C, the actual angle of view of the camera 61 extends in the vertical direction as in a general digital camera.

  The optical axis of the camera 61 is a direction including the line-of-sight directions of the right eye RE and the left eye LE. The external view that the user can see with the HMD 100 mounted is not limited to infinity. For example, as shown in FIG. 3B, when the user gazes at the object OB with both eyes, the line of sight of the user is directed to the object OB as indicated by reference characters RD and LD in the figure. In this case, the distance from the user to the object OB is often about 30 cm to 10 m, and more often about 1 m to 4 m. Therefore, for the HMD 100, the upper limit and the lower limit of the distance from the user to the object OB during normal use may be set. This standard may be obtained by investigation or experiment, or may be set by the user. When the distance to the object OB during normal use corresponds to the set upper limit and when the distance to the object OB corresponds to the set lower limit, the object OB is It is preferably set so as to be included in the angle of view.

  Generally, a human viewing angle is about 200 degrees in the horizontal direction and about 125 degrees in the vertical direction, and the effective field of view having excellent information receiving ability is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction. Furthermore, the stable gazing point where a gazing point to which a human gazes quickly and stably is set to about 60 to 90 degrees in the horizontal direction and about 45 to 70 degrees in the vertical direction. In this case, when the gazing point is the object OB of FIG. 3 (B), the effective visual field is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction about the lines of sight RD and LD. Further, the stable fixation field is 60 to 90 degrees in the horizontal direction and 45 to 70 degrees in the vertical direction, and the viewing angle is approximately 200 degrees in the horizontal direction and 125 degrees in the vertical direction. Further, the actual visual field that the user sees through the image display unit 20 and the right light guide plate 26 and the left light guide plate 28 can be referred to as a real field of view (FOV). In the configuration of the present embodiment shown in FIGS. 1 and 2, the actual visual field corresponds to the actual visual field that the user sees through the right light guide plate 26 and the left light guide plate 28. The real field of view is narrower than the viewing angle and the stable fixation field, but wider than the effective field of view.

  The angle of view C of the camera 61 is preferably capable of capturing a wider range than the visual field of the user, and specifically, the angle of view C is preferably at least wider than the effective visual field of the user. Further, it is more preferable that the angle of view C is wider than the actual visual field of the user. More preferably, the angle of view C is wider than the stable fixation field of view of the user, and most preferably the angle of view C is wider than the viewing angles of both eyes of the user.

  The camera 61 may be provided with a so-called wide-angle lens as an imaging lens so that a wide field angle can be imaged. The wide-angle lens may include a lens called an ultra-wide-angle lens or a quasi-wide-angle lens, may be a single-focus lens or a zoom lens, and the camera 61 includes a lens group including a plurality of lenses. It may be.

FIG. 4 is a block diagram showing the configuration of each unit that constitutes the HMD 100.
The control device 10 includes a main processor 140 that executes a program to control the HMD 100. A memory 118 and a non-volatile storage unit 121 are connected to the main processor 140. Further, the track pad 14 and the operation unit 110 are connected to the main processor 140 as input devices. A 6-axis sensor 111, a magnetic sensor 113, and a GPS 115 are connected to the main processor 140 as sensors. Further, the main processor 140 is connected with the communication unit 117, the audio codec 180, the external connector 184, the external memory interface 186, the USB connector 188, the sensor hub 192, and the FPGA 194. These function as an interface with the outside.

  The main processor 140 is mounted on the controller board 120 included in the control device 10. In addition to the main processor 140, the memory 118, the non-volatile storage unit 121, and the like may be mounted on the controller board 120. In this embodiment, the 6-axis sensor 111, the magnetic sensor 113, the GPS 115, the communication unit 117, the memory 118, the non-volatile storage unit 121, the audio codec 180, etc. are mounted on the controller board 120. Further, the external connector 184, the external memory interface 186, the USB connector 188, the sensor hub 192, the FPGA 194, and the interface 196 may be mounted on the controller board 120.

  The memory 118 constitutes a work area for temporarily storing a program to be executed and data to be processed when the main processor 140 executes the program. The non-volatile storage unit 121 is composed of a flash memory or an eMMC (embedded Multi Media Card). The non-volatile storage unit 121 stores a program executed by the main processor 140 and various data processed by the main processor 140 by executing the program.

The main processor 140 detects a contact operation on the operation surface of the track pad 14 based on an operation signal input from the track pad 14 and acquires an operation position.
The operation unit 110 includes a button 11, a touch sensor 13, and an LED display unit 17. The touch sensor 13 detects a touch operation on the track pad 14 and specifies the operation position of the detected touch operation. When the button 11 is operated and when the touch sensor 13 detects a touch operation, an operation signal is output from the operation unit 110 to the main processor 140.
The LED display unit 17 includes an LED (not shown) arranged immediately below the track pad 14 (FIG. 1) and a drive circuit for lighting the LED. The LED display unit 17 turns on, blinks, and turns off the LED under the control of the main processor 140.

FIG. 6 shows the light emission pattern of the LEDs of the LED display unit 17.
6A and 6B are timing charts showing changes in the brightness of the LEDs of the LED display unit 17. In the light emission pattern shown in FIG. 6A, the LED display unit 17 periodically turns on the LEDs, and the rise and fall of the brightness of the LEDs are sharp. On the other hand, in the lighting pattern of FIG. 6B, the brightness of the LED changes gently. Under the control of the main processor 140, the LED display unit 17 causes the LEDs to emit light in the pattern shown in FIG. 6 (A) or (B). Further, the LED display unit 17 may always turn on the LED. In the pattern shown in FIG. 6A or 6B, the time T1 when the LED emits light and the time T2 when the LED turns off are arbitrary, and for example, the times T1 and T2 may be 100 milliseconds or less. In this case, even if the LED is repeatedly turned on and off, the user can visually recognize that the LED is continuously turned on. Further, the LED display unit 17 may perform PWM control on the brightness of the LED by controlling the ratio of the times T1 and T2.

Returning to FIG. 4, the 6-axis sensor 111 is a motion sensor (inertial sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 111 may employ an IMU (Inertial Measurement Unit) in which the above sensor is modularized.
The magnetic sensor 113 is, for example, a triaxial geomagnetic sensor.
The GPS (Global Positioning System) 115 includes a GPS antenna (not shown), receives a radio signal transmitted from a GPS satellite, and detects the coordinates of the current position of the control device 10.
The 6-axis sensor 111, the magnetic sensor 113, and the GPS 115 output the detected value to the main processor 140 according to a sampling cycle designated in advance. Alternatively, the 6-axis sensor 111, the magnetic sensor 113, and the GPS 115 output detection values to the main processor 140 at a timing designated by the main processor 140 in response to a request from the main processor 140.

The communication unit 117 executes wireless communication with an external device. The communication unit 117 is configured to include an antenna, an RF circuit, a baseband circuit, a communication control circuit, and the like, or a device in which these are integrated. The communication unit 117 performs wireless communication based on standards such as Bluetooth (registered trademark) and wireless LAN (including Wi-Fi (registered trademark)), for example.
The voice interface 182 is an interface that inputs and outputs a voice signal. In the present embodiment, the audio interface 182 includes the connector 46 (FIG. 1) provided on the connection cable 40. The audio codec 180 is connected to the audio interface 182, and encodes / decodes an audio signal input / output via the audio interface 182. The audio codec 180 may include an A / D converter that converts an analog audio signal into digital audio data, and a D / A converter that performs the reverse conversion. For example, in the HMD 100 of this embodiment, sound is output by the right earphone 32 and the left earphone 34, and is collected by the microphone 63. The audio codec 180 converts the digital audio data output by the main processor 140 into an analog audio signal and outputs the analog audio signal via the audio interface 182. The audio codec 180 also converts an analog audio signal input to the audio interface 182 into digital audio data and outputs the digital audio data to the main processor 140.

The external connector 184 is a connector for connecting an external device that communicates with the main processor 140. The external connector 184 is, for example, an interface that connects an external device to the main processor 140 when the program executed by the main processor 140 is debugged or the operation log of the HMD 100 is collected. Is.
The external memory interface 186 is an interface to which a portable memory device can be connected, and includes, for example, a memory card slot into which a card type recording medium can be attached and which can read data, and an interface circuit. The size, shape, and standard of the card-type recording medium in this case are not limited and can be changed as appropriate.
The USB (Universal Serial Bus) connector 188 includes a connector conforming to the USB standard and an interface circuit, and can connect a USB memory device, a smartphone, a computer, and the like. The size and shape of the USB connector 188 and the compatible version of the USB standard can be appropriately selected and changed.

  Further, the HMD 100 includes a vibrator 19. The vibrator 19 includes a motor (not shown), an eccentric rotor (not shown), etc., and generates vibration under the control of the main processor 140. The HMD 100 causes the vibrator 19 to vibrate in a predetermined vibration pattern, for example, when an operation on the operation unit 110 is detected or when the power of the HMD 100 is turned on / off.

  The image display unit 20 is connected to the sensor hub 192 and the FPGA 194 via an interface (I / F) 196. The sensor hub 192 acquires detection values of various sensors included in the image display unit 20 and outputs the detection values to the main processor 140. Further, the FPGA 194 executes processing of data transmitted and received between the main processor 140 and each unit of the image display unit 20, and transmission via the interface 196.

  The right display unit 22 and the left display unit 24 of the image display unit 20 are connected to the control device 10, respectively. As shown in FIG. 1, in the HMD 100, the connection cable 40 is connected to the left holding unit 23, the wiring connected to the connection cable 40 is laid inside the image display unit 20, and each of the right display unit 22 and the left display unit 24 is provided. It is connected to the control device 10.

The right display unit 22 has a display unit substrate 210. The display unit substrate 210 includes an interface (I / F) 211 connected to the interface 196, a reception unit (Rx) 213 that receives data input from the control device 10 via the interface 211, and an EEPROM 215 (storage unit). ) Is implemented.
The interface 211 connects the receiving unit 213, the EEPROM 215, the temperature sensor 217, the camera 61, the illuminance sensor 65, and the LED indicator 67 to the control device 10.

  An EEPROM (Electrically Erasable Programmable Read-Only Memory) 215 stores various data so that the main processor 140 can read them. The EEPROM 215 stores, for example, data regarding emission characteristics and display characteristics of the OLED units 221 and 241 included in the image display unit 20, data regarding characteristics of a sensor included in the right display unit 22 or the left display unit 24, and the like. Specifically, parameters for gamma correction of the OLED units 221, 241 and data for compensating the detection values of the temperature sensors 217, 239 are stored. These data are generated by the factory inspection of the HMD 100 and written in the EEPROM 215, and after the shipment, the main processor 140 can perform processing by using the data of the EEPROM 215.

The camera 61 performs imaging according to a signal input via the interface 211, and outputs captured image data or a signal indicating an imaging result to the control device 10.
As shown in FIG. 1, the illuminance sensor 65 is provided at the end ER of the front frame 27 and is arranged to receive external light from the front of the user wearing the image display unit 20. The illuminance sensor 65 outputs a detection value corresponding to the amount of received light (received light intensity).
As shown in FIG. 1, the LED indicator 67 is arranged near the camera 61 at the end ER of the front frame 27. The LED indicator 67 lights up while the image pickup by the camera 61 is being performed, and notifies that the image pickup is in progress.

  The temperature sensor 217 detects temperature and outputs a voltage value or a resistance value corresponding to the detected temperature as a detected value. The temperature sensor 217 is mounted on the back surface side of the OLED panel 223 (FIG. 3). The temperature sensor 217 may be mounted on the same substrate as the OLED drive circuit 225, for example. With this configuration, the temperature sensor 217 mainly detects the temperature of the OLED panel 223.

  The receiving unit 213 receives data transmitted by the main processor 140 via the interface 211. When receiving the image data of the image displayed by the OLED unit 221, the receiving unit 213 outputs the received image data to the OLED drive circuit 225 (FIG. 2).

The left display unit 24 has a display unit substrate 210. An interface (I / F) 231 connected to the interface 196, and a reception unit (Rx) 233 that receives data input from the control device 10 via the interface 231 are mounted on the display unit substrate 210. A 6-axis sensor 235 and a magnetic sensor 237 are mounted on the display unit substrate 210.
The interface 231 connects the receiving unit 233, the 6-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 to the control device 10.

The 6-axis sensor 235 (display section motion detection section) is a motion sensor (inertial sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 235 may employ an IMU (Inertial Measurement Unit) in which the above sensor is modularized.
The magnetic sensor 237 is, for example, a triaxial geomagnetic sensor.

The temperature sensor 239 detects the temperature and outputs the voltage value or the resistance value corresponding to the detected temperature as the detected value. The temperature sensor 239 is mounted on the back surface side of the OLED panel 243 (FIG. 3). The temperature sensor 239 may be mounted on the same substrate as the OLED drive circuit 245, for example. With this configuration, the temperature sensor 239 mainly detects the temperature of the OLED panel 243.
Further, the temperature sensor 239 may be built in the OLED panel 243 or the OLED drive circuit 245. Further, the substrate may be a semiconductor substrate. Specifically, when the OLED panel 243 is mounted as a Si-OLED together with the OLED drive circuit 245 and the like as an integrated circuit on an integrated semiconductor chip, the temperature sensor 239 may be mounted on this semiconductor chip.

The camera 61, the illuminance sensor 65, the temperature sensor 217 included in the right display unit 22, and the 6-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 included in the left display unit 24 are connected to the sensor hub 192. The sensor hub 192 sets and initializes the sampling cycle of each sensor under the control of the main processor 140. The sensor hub 192 executes energization to each sensor, transmission of control data, acquisition of a detection value, and the like in accordance with the sampling cycle of each sensor. Further, the sensor hub 192 outputs the detection value of each sensor included in the right display unit 22 and the left display unit 24 to the main processor 140 at a preset timing. The sensor hub 192 may have a function of temporarily holding the detection value of each sensor in synchronization with the timing of output to the main processor 140. Further, the sensor hub 192 may have a function of converting to a signal format or a data format of an output value of each sensor, converting the data into a unified data format, and outputting the data to the main processor 140.
Further, the sensor hub 192 starts and stops energization of the LED indicator 67 under the control of the main processor 140, and turns on or blinks the LED indicator 67 at the timing when the camera 61 starts and ends image capturing.

  The control device 10 includes a power supply unit 130 and operates by electric power supplied from the power supply unit 130. The power supply unit 130 includes a rechargeable battery 132 and a power supply control circuit 134 that detects the remaining capacity of the battery 132 and controls charging of the battery 132. The power supply control circuit 134 is connected to the main processor 140 and outputs the detected value of the remaining capacity of the battery 132 or the detected value of the voltage to the main processor 140. Further, the control device 10 may supply power to the image display unit 20 based on the power supplied by the power supply unit 130. Further, the main processor 140 may control the power supply state from the power supply unit 130 to each unit of the control device 10 and the image display unit 20.

  FIG. 5 is a functional block diagram of the storage unit 122 and the control unit 150 that configure the control system of the control device 10. The storage unit 122 shown in FIG. 5 is a logical storage unit configured by the nonvolatile storage unit 121 (FIG. 4), and may include the EEPROM 215. Further, the control unit 150 and various functional units included in the control unit 150 are formed by cooperation of software and hardware when the main processor 140 executes a program. The control unit 150 and each functional unit configuring the control unit 150 are configured by, for example, a main processor 140, a memory 118, and a nonvolatile storage unit 121.

The control unit 150 executes various processes by using the data stored in the storage unit 122 and controls the HMD 100.
The storage unit 122 stores various data processed by the control unit 150. The storage unit 122 stores setting data 123, content data 124, operation setting data 125, and display control data 126. The setting data 123 includes various setting values related to the operation of the HMD 100. Further, when the control unit 150 uses parameters, determinants, arithmetic expressions, LUTs (LookUp Table), etc. when controlling the HMD 100, these may be included in the setting data 123.

The content data 124 is data of content including images and videos displayed by the image display unit 20 under the control of the control unit 150, and includes image data or video data. The content data 124 may also include audio data. Further, the content data 124 may include image data of a plurality of images, and in this case, the plurality of images are not limited to the images displayed on the image display unit 20 at the same time.
The content data 124 is a bidirectional type in which the control device 10 accepts a user operation when the content is displayed by the image display unit 20, and the control unit 150 executes a process according to the accepted operation. It may be content. In this case, the content data 124 may include image data of a menu screen displayed when an operation is accepted, data defining processing and the like corresponding to items included in the menu screen, and the like.

  The operation setting data 125 is data used by the control unit 150 in a process of detecting a motion operation for moving the control device 10. The operation setting data 125 includes data for determining each motion for one or a plurality of motion patterns preset as the motion of the case 10A. Specifically, it includes the detected value of the motion detected by the 6-axis sensor 111 or the data of the change in the detected value for each type of motion. This data may include a plurality of detection values of the 6-axis sensor 111, or may be data of a feature amount obtained by calculation from the detection values. The operation setting data 125 may be any data that can determine whether or not the case 10A has moved corresponding to the set motion from the detection value of the 6-axis sensor 111. More preferably, it is data that can identify a motion corresponding to the movement of the case 10A detected by the 6-axis sensor 111 from the set plurality of motions.

  The display control data 126 is data used in the process of changing the display state of the image or the like displayed by the image display unit 20 in response to the motion of the case 10A. More specifically, it includes data indicating a mode of changing the display state in association with the set motion of the case 10A. When the control unit 150 detects the motion of the case 10A based on the operation setting data 125 and identifies the motion, the control unit 150 changes the display of the image display unit 20 in a manner set in the display control data 126 in association with the identified motion. Let

  The control unit 150 has functions of an operating system (OS) 143, an image processing unit 145, a display control unit 147, an imaging control unit 149, and an operation detection control unit 152. The function of the operating system 143 is the function of the control program stored in the storage unit 122, and the other units are the functions of the application program executed on the operating system 143.

The image processing unit 145 generates a signal to be transmitted to the right display unit 22 and the left display unit 24 based on the image data of the image or video displayed by the image display unit 20. The signal generated by the image processing unit 145 may be a vertical synchronizing signal, a horizontal synchronizing signal, a clock signal, an analog image signal, or the like.
Further, the image processing unit 145 may perform a resolution conversion process of converting the resolution of the image data into a resolution suitable for the right display unit 22 and the left display unit 24, if necessary. The image processing unit 145 also performs image adjustment processing for adjusting the brightness and saturation of the image data, creates 2D image data from 3D image data, or 2D / 3D conversion processing for generating 3D image data from 2D image data. May be executed. When these image processes are executed, the image processing unit 145 generates a signal for displaying an image based on the processed image data, and transmits the signal to the image display unit 20 via the connection cable 40.

  The image processing unit 145 may be configured not only by the main processor 140 executing a program but also by hardware different from the main processor 140 (for example, DSP (Digital Signal Processor)).

The display control unit 147 generates a control signal for controlling the right display unit 22 and the left display unit 24, and controls the generation and emission of image light by the right display unit 22 and the left display unit 24, respectively, by this control signal. .. Specifically, the display control unit 147 controls the OLED drive circuits 225 and 245 to cause the OLED panels 223 and 243 to display an image. The display control unit 147 controls the timing at which the OLED drive circuits 225 and 245 draw on the OLED panels 223 and 243 based on the signal output from the image processing unit 145, and controls the brightness of the OLED panels 223 and 243.
Further, when the operation detection control unit 152 detects the motion of the control device 10, the display control unit 147 changes the display mode of the image or the like displayed on the image display unit 20 according to the display control data 126.

  The image capturing control unit 149 controls the camera 61 to execute image capturing, generates captured image data, and temporarily stores it in the storage unit 122. When the camera 61 is configured as a camera unit including a circuit that generates captured image data, the imaging control unit 149 acquires captured image data from the camera 61 and temporarily stores it in the storage unit 122.

  The operation detection control unit 152 detects an operation on the track pad 14 and the operation unit 110, and outputs data corresponding to the operation. For example, when a button or the like of the operation unit 110 is operated, the operation detection control unit 152 generates operation data indicating the operation content and outputs the operation data to the display control unit 147. The display control unit 147 changes the display state according to the operation data input from the operation detection control unit 152.

The operation detection control unit 152 detects the operation of the software button while the LED display unit 17 is lit. The operation detection control unit 152 detects a touch operation on a position of a symbol displayed by the lighting of the LED display unit 17 while the LED display unit 17 is lighting, as an operation of a button (software operator) to which a function is assigned in advance. ..
For example, in the example of FIG. 1, the operation of touching the position of the symbol O (circle) is detected as the operation of the home button previously assigned to the position of the symbol O (circle). In this case, the operation detection control unit 152 shifts the display screen of the image display unit 20 and the operating state of the control device 10 to the basic state of the operating system 143.
Further, for example, in the example of FIG. 1, the operation of touching the position of the symbol □ (square) is detected as the operation of the history button previously assigned to the position of the symbol □ (square). In this case, the operation detection control unit 152 causes the image display unit 20 to display a display screen of the image display unit 20 or a screen showing past operation states of the control device 10.

The operation detection control unit 152 identifies the movement of the case 10A based on the detection value of the 6-axis sensor 111, and determines whether or not the movement of the case 10A corresponds to a preset motion. In addition, in the present embodiment, the operation detection control unit 152 performs a process of identifying a motion corresponding to the movement of the case 10A. In this process, the operation detection control unit 152 acquires the detection value of the 6-axis sensor 111. Further, a plurality of detection values detected by the 6-axis sensor 111 may be acquired. For example, when the 6-axis sensor 111 performs detection at a predetermined cycle, a plurality of detection values continuously detected by the 6-axis sensor 111 may be obtained. May be continuously acquired. The operation detection control unit 152 may store the detection value of the 6-axis sensor 111 in the memory 118 or the non-volatile storage unit 121 as needed. For example, a ring buffer may be formed in the memory 118 or the non-volatile storage unit 121, and the operation detection control unit 152 may constantly acquire the detection value of the 6-axis sensor 111 and store it in the ring buffer. In this case, the ring buffer stores the latest detected value in a number corresponding to the capacity of the ring buffer. The operation detection control unit 152 can perform the above processing based on the detected value stored in the ring buffer and the operation setting data 125.
Further, when the operation detection control unit 152 detects an operation on the track pad 14, the operation detection control unit 152 acquires the coordinates of the operation position in the operation detection area (detection area) of the track pad 14.

  The HMD 100 may include an interface (not shown) that connects various external devices serving as content supply sources. For example, the interface may be a wired interface such as a USB interface, a micro USB interface, or a memory card interface, or may be a wireless communication interface. The external device in this case is an image supply device that supplies an image to the HMD 100, and a personal computer (PC), a mobile phone terminal, a portable game machine, or the like is used. In this case, the HMD 100 can output images and sounds based on the content data input from these external devices.

  FIG. 7 is an explanatory diagram showing an operation example using the control device 10, and specifically shows an example of motion for moving the case 10A. 7A and 7B each show a state in which the user holds the case 10A in his / her hand, and FIG. 7A is a front view of the control device 10 seen from the user side. (B) is a side view. The connection cable 40 is not shown in FIGS. 7A and 7B.

In the present embodiment, six types of motions are defined in advance as modes of operations for moving the control device 10.
FIG. 7A shows four types of motions A, B, C, and D. FIG. 7B shows two types of motions E and F.
In the illustrated example, the user holds the case 10A with the right hand RH, but may hold it with the left hand. Alternatively, the user may jointly grasp the case 10A using a jig or the like.

  Motion A is an operation of rotating control device 10 clockwise (clockwise: CW) with reference to the left and right of the user's body as indicated by arrow A in the figure. The 6-axis sensor 111 detects acceleration and angular velocity indicating the clockwise movement of the case 10A. The motion B is an operation of rotating the control device 10 counterclockwise (counterclockwise: CCW) as shown by an arrow B in the figure. Motion A can be called a right rotation operation, and motion B can be called a left rotation operation.

  As shown in the figure, the controller 10 is preset with the left-right direction when the user holds and uses the hand. For example, as shown in FIGS. 7 (A) and 7 (B), when the user holds the case 10A in his / her hand, the upper end of the case 10A faces upward and the surface on which the track pad 14 is arranged faces the right side of the user. May be prescribed in advance. The upper end surface of the case 10A is the surface on which the LED indicator 12 and the power switch 18 are provided. Further, the right side and the left side when the user holds the control device 10 may be configured so that the user can set which surface of the case 10A by operating the HMD 100. In this case, the operation setting data 125 includes data that associates the detection direction of the 6-axis sensor 111 with the left and right of the user's body.

The motion C is an operation of moving the control device 10 so as to draw an arc on the right side as shown by an arrow C in the figure, and the motion D moves an arc on the left side as shown by an arrow D in the figure. This is an operation to move as you draw. Motion C can be referred to as a right move operation, and motion D can be referred to as a left move operation.
It should be noted that the operation of linearly moving the control device 10 to the right may be detected as the motion C, as indicated by an arrow C ′ in the figure. Similarly, as indicated by an arrow D ′ in the figure, the motion of linearly moving the control device 10 to the left may be detected as the motion D.

  The motion E is an operation of moving the control device 10 so as to draw an arc upward, as indicated by an arrow E in FIG. The motion F is an operation of moving the control device 10 so as to draw an arc downward, as indicated by F in FIG. 7 (B). Motion E can be called a swing-up motion, and motion D can be called a swing-down motion. As indicated by an arrow E ′ in the figure, the operation of linearly moving the control device 10 upward may be detected as the motion E. Further, as shown by an arrow F ′ in the figure, the motion of linearly moving the control device 10 downward may be detected as the motion E.

  Some of these six types of motion are associated with each other as a pair of motions. Motion A and motion B are set as paired motions. Similarly, motion C and motion D, and motion E and motion F are set as paired motions. Hereinafter, the paired motions are referred to as inverse motions. For example, motion B corresponds to the inverse motion of motion A and vice versa. The same applies to motions C, D, E, and F.

  One of these paired motions corresponds to a motion (motion, motion) in the first movement direction, and the other corresponds to a motion in the second movement direction. The first movement direction and the second movement direction are not limited to the opposite directions. It is only necessary that the 6-axis sensor 111 or the magnetic sensor 113 used in place of the 6-axis sensor 111 and / or the camera 61 can detect the movements in different directions. Although the motions A to F, C ′, D ′, E ′, and F ′ are set as the movements in the specific directions, even if the movements close to the specific direction are regarded as the movements in the specific direction. Good. For example, the motion C shown in FIG. 7A is a motion in the horizontal direction in the drawing, but a motion within an angle range of about 5 degrees with respect to the horizontal may be detected as the motion C. The range regarded as the motion in the same direction can be set arbitrarily, and may be, for example, ± 5 degrees or ± 10 degrees. The same applies to the first movement direction and the second movement direction.

  The operation detection control unit 152 detects the six movements of the case 10A of the motions A to F, and the display control unit 147 causes the image display unit 20 to display so as to correspond to the motion detected by the operation detection control unit 152. Control.

8, 9 and 10 are explanatory diagrams showing changes in the display corresponding to the operation of the HMD.
FIG. 8 shows an example of enlarging and reducing the display. 8A shows a display example before enlargement, FIG. 8B shows a display example after enlargement, FIG. 8C shows a display example before reduction, and FIG. 8D shows reduction. The following display example is shown. Reference numeral VR in the drawing indicates a visual field of the user, and reference numeral V indicates a display area in which an image can be displayed by the right display unit 22 and the left display unit 24. Reference numerals W1, W2, and W3 represent windows as an example of images displayed in the display area V.

  When the display is enlarged from the state shown in FIG. 8A corresponding to the motion of the case 10A as indicated by the arrow in the figure, the display control unit 147 determines the center position CE of the enlargement process in the display area V. To do. The center position CE may be designated by an operation on the track pad 14, or may be automatically determined based on the position of a pointer (not shown) when starting the enlargement process. In the example of FIG. 8B, the window W1 of the display area V shown in FIG. 8A is enlarged and displayed with the center position CE as the center. By comparing the display example of FIG. 8A and the display example of FIG. 8B, the enlargement ratio for enlarging the window W1 is determined by an operation on the control device 10, as described later with reference to the flowchart. Or may be set in advance.

  Further, when the display is reduced from the state shown in FIG. 8C as indicated by the arrow in the figure in response to the motion of the case 10A, the display control unit 147 also determines the center of the reduction process in the display area V. Determine the position CE. The center position CE may be designated by an operation on the track pad 14, or may be automatically determined based on the position of a pointer (not shown) when starting the enlargement process. In the example of FIG. 8D, the window W1 of the display area V shown in FIG. 8C is reduced and displayed with the center position CE as the center. By comparing the display example of FIG. 8C and the display example of FIG. 8D, the reduction ratio for reducing the window W1 is determined by an operation on the control device 10 as described later with reference to the flowchart. Or may be set in advance.

FIG. 9 shows an example of scrolling the display. 9A shows a display example before scrolling, FIG. 9B shows a display example scrolling upward, and FIG. 9C shows a display example scrolling downward.
From the state shown in FIG. 9A, the display control unit 147 can scroll the display up and down, as indicated by the arrow in the figure, corresponding to the motion of the case 10A. When scrolling upward, the display position of the window W1 moves upward as shown in FIG. 9 (B). Further, when scrolling downward, the display position of the window W1 moves downward as shown in FIG. 9 (C). The scroll amount for scrolling the window W1 as shown in the display examples of FIGS. 9A to 9C may be determined by an operation on the control device 10 or set in advance, as described later with reference to the flowchart. May be done.
Further, although not shown, the display control unit 147 may execute a horizontal scroll process of moving the image displayed in the display area V leftward or rightward.

  One of the upward scrolling in FIG. 9B and the downward scrolling in FIG. 9C corresponds to the first display operation direction, and the other corresponds to the second display operation direction. These are merely examples, and the first display operation direction and the second display operation direction may be display movements in different directions. For example, a horizontal scroll direction (either right or left) and a vertical scroll direction (either upper or lower) may be the first display operation direction and the second display operation direction. The display movement in the diagonal direction may be the first display operation direction and / or the second display operation direction. That is, the movement of the display in the two directions in which the user can identify the different directions can be the first display operation direction and the second display operation direction.

  In addition, in FIGS. 9B and 9C, scrolling of the display in the upward direction is performed corresponding to the operation of moving the case 10A in the upward direction, and corresponding to the operation of moving the case 10A in the downward direction. An example of executing scrolling of the display in the downward direction has been shown. These are merely examples, and the movement of the case 10A detected by the 6-axis sensor 111 and the display movement direction need not be the same direction relative to the user. The movement direction detected by the 6-axis sensor 111 and the display movement direction may be associated in advance, and may be opposite directions, for example.

FIG. 10 shows an example of switching the display. FIG. 10 (A) shows a display example before processing, FIG. 10 (B) shows a display example in which the display is changed to the previous screen, and FIG. 10 (C) is a table in which the display is changed to the next screen. An example is shown.
In the example shown in FIGS. 10A to 10C, the content data 124 displayed by the display control unit 147 includes a plurality of images in which the display order is set, and when the content data 124 is reproduced, the display control unit 147 is displayed. Corresponds to an example of switching and displaying images according to the display order. These images are not limited to images such as a single photograph, but may be a combination of images and texts that form one screen, for example. Specifically, in the configuration in which the control unit 150 executes the application program on the operating system 143, a web page displayed by a web browser can be mentioned. Further, in the configuration in which the presentation program is executed as the application program, a slide sheet for presentation having a plurality of slide sheets can be used as the content data 124 including a plurality of images.

FIG. 10A shows an example in which a slide sheet for presentation is displayed. For example, in FIG. 10A, the window W1 including the slide sheet is displayed, and when the display control unit 147 transitions the screen to the previous screen in this state, the window W2 including the previous slide sheet is displayed in the display area V. Is displayed. When the display control unit 147 transitions the screen to the next screen in FIG. 10A, the window W3 including the next slide sheet is displayed in the display area V.
That is, in the example of FIGS. 10A to 10C, in the operation of displaying a plurality of images in which the display order is set, the display change in which the images are switched in order and the display change in which the images are switched in the reverse order are shown.

  In the present embodiment, the display control unit 147 is set to enlarge the display in response to the motion A (right rotation), and the display control unit 147 is set to reduce the display in response to the motion B (rotation left). Further, the display control unit 147 scrolls the display upwards in response to the motion E (swing up), and the display control unit 147 scrolls the display downwards in response to the motion F (swing down). Further, the display control unit 147 causes the display to transition to the next screen in response to the motion C (moving to the right), and the display control unit 147 transitions the display to the next screen in response to the motion D (moving to the left).

11 and 12 are flowcharts showing the operation of the HMD 100.
When the motion operation for moving the control device 10 is started (step S11), the control unit 150 determines whether to use the position instruction (position input) by the track pad 14 (step S12). When using the position instruction, the center position CE is specified by operating the track pad 14 in the process of enlarging the display when the motion A is executed and the process of reducing the display when the motion B is executed. Whether or not to use the position instruction may be designated by the user by an input operation on the operation unit 110, or may be set in advance.

When using the position instruction (step S12; Yes), the control unit 150 proceeds to step S31 (FIG. 12). This case will be described later.
When the position instruction is not used (step S12; No), the controller 150 starts the acquisition of the detection value of the 6-axis sensor 111 (step S13). After that, the control unit 150 acquires the detection value of the 6-axis sensor 111 at a preset cycle.

  Subsequently, the control unit 150 determines whether or not there is an operation that becomes a trigger for controlling the display (step S14), and if there is no operation (step S14; No), waits for the operation. The operation that serves as a trigger includes, for example, an operation on various operators such as the button 11 provided on the case 10A. In this embodiment, a contact operation on the track pad 14 is set as a trigger.

  When the control unit 150 determines that the contact operation on the track pad 14 has been performed (step S14; Yes), the elapsed time from the start of the detection value acquisition in step S13 becomes the set time (set time). It is determined whether or not it has reached (step S15).

A function is assigned in advance to a manipulator corresponding to a manipulation that serves as a trigger for controlling display. For example, in the present embodiment, the contact operation of the track pad 14 is set to the trigger operation, but the track pad 14 is assigned with the position input operation and the function of the software button corresponding to the display position of the LED display unit 17. There is. The operation detection control unit 152 detects whether a touch operation on the track pad 14 is detected as a trigger operation, a position input operation on the track pad 14, or an operation of a software button at the display position of the LED display unit 17. Can be switched. When the elapsed time from the start of the acquisition of the detection value of the 6-axis sensor 111 to the operation of the track pad 14 reaches the set time (step S15; Yes), the operation detection control unit 152 performs the motion operation. Execute normal operation mode without accepting. The operation detection control unit 152 clears the detection value of the 6-axis sensor 111 that is already acquired and stored (step S16), and executes the function assigned as the normal operation for the trigger operation detected in step S14 (step S16). S17), and this process ends.
For example, when the contact at the position of the symbol ◯ (circle) on the track pad 14 is detected in step S14, the operation detection control unit 152 performs control corresponding to the operation of the home button in step S17.

  When the elapsed time has not reached the set time (step S15; No), the operation detection control unit 152 starts the display operation mode (step S18). The display operation mode is an operation mode in which a trigger operation is detected as a motion operation trigger. The operation detection control unit 152 analyzes the movement of the case 10A based on the detection value of the 6-axis sensor 111 (step S19). The detection value analyzed in step S19 includes the past detection value which is acquired in step S13 and stored in the memory 118 or the non-volatile storage unit 121. In particular, the detection value until the trigger operation is detected in step S14. Includes detection values obtained in between.

  For example, when the user holds the control device 10 in his / her hand and contacts the track pad 14 while the case 10A is moving, the control unit 150 causes the case 10A to move before the contact operation of the track pad 14. To analyze. For example, even if the operation of moving the control device 10 is deviated from the timing of the operation as the trigger contrary to the user's intention, the movement of the case 10A intended by the user is analyzed.

  The operation detection control unit 152 determines whether or not the motion corresponding to the movement of the case 10A analyzed in step S19 is included in preset motions (for example, motions A to F) (step S20). When there is no corresponding motion (step S20; No), the operation detection control unit 152 analyzes the movement of the case 10A again (step S21), and returns to step S20 to make the determination. In step S21, since the analysis is performed including the detection values of the 6-axis sensor 111 acquired during steps S19 to S21, there is a possibility that the motion of the new case 10A can be detected.

  When there is a motion corresponding to the movement of the case 10A (step S20; Yes), the operation detection control unit 152 determines whether or not to execute the combination operation (step S22). The combination operation is an operation method of controlling display by combining a plurality of paired motions. Whether to perform the combination operation may be determined by an operation on the control device 10 or may be set in advance.

  When the combination operation is not performed (step S23; No), the display control unit 147 determines the content of the display change corresponding to the motion determined by the operation detection control unit 152 in step S20 and the display change amount (step). S23). For example, the enlargement ratio / reduction ratio is determined for the motions A and B, and the scroll direction and the scroll amount are determined for the motions E and F. The amount of change in the display can be a preset amount for each type of motion.

  The display control unit 147 changes the display of the image display unit 20 according to the content of the display control determined in step S23 and the change amount of the display change (step S24), and proceeds to step S29 described later.

  When performing the combination operation (step S23; Yes), the display control unit 147 starts the display change corresponding to the motion determined by the operation detection control unit 152 in step S20 (step S25). After starting the display change, the operation detection control unit 152 analyzes the movement of the case 10A based on the detection value of the 6-axis sensor 111 (step S26). The operation detection control unit 152 determines whether or not the movement of the case 10A corresponding to the inverse motion of the motion determined in step S20 has been performed (step S27). While the reverse motion is not performed (step S27; No), the analysis of steps S26 to S27 is performed and the process waits.

When the operation detection control unit 152 determines that the reverse motion is performed (step S27; Yes), the display control unit 147 stops the display change (step S28), and proceeds to step S29.
For example, the control unit 150 starts scrolling in the upward direction corresponding to the motion E in step S25, and thereafter stops the scrolling in the upward direction when it is determined that the motion F is performed in step S27. In this way, the timing at which the display change corresponding to the motion is stopped can be specified by the reverse motion, so that the user can freely adjust the display change amount.

  In step S29, the operation detection control unit 152 determines whether the trigger operation has been released (step S29). If the trigger operation has not been released (step S29; No), the process returns to step S20. When the operation of the operator assigned to the trigger operation is released (step S29; Yes), this process ends.

  On the other hand, when using the position instruction (step S12; Yes), the operation detection control unit 152 determines whether or not the position instruction operation by the track pad 14 is performed (step S31), and when there is no operation (step S31). S31; No), wait until there is an operation. When the operation detection control unit 152 determines that the position pointing operation has been performed by the track pad 14 (step S31; Yes), the elapsed time waiting for the position pointing operation in step S31 reaches the set time (set time). It is determined whether or not (step S32).

  The operation detection control unit 152 executes the normal operation mode without accepting the motion operation when determining that the time for waiting the position instruction operation has reached the set time (step S32; Yes). The operation detection control unit 152 executes the operations of steps S16 to S17 described above, and ends this processing.

  When the elapsed time has not reached the set time (step S32; No), the operation detection control unit 152 starts the display operation mode (step S33). The operation detection control unit 152 analyzes the movement of the case 10A based on the detection value of the 6-axis sensor 111 (step S34). In step S33, the operation detection control unit 152 may perform analysis including the past detection values that have already been acquired and stored in the memory 118 or the non-volatile storage unit 121.

  The operation detection control unit 152 determines whether or not the motion corresponding to the movement of the case 10A analyzed in step S34 is included in preset motions (for example, motions A to F) (step S35). When there is no corresponding motion (step S35; No), the operation detection control unit 152 analyzes the movement of the case 10A again (step S36), and returns to step S35 to make the determination.

  When there is a motion corresponding to the movement of the case 10A (step S35; Yes), the operation detection control unit 152 acquires the coordinates of the pointed position of the position pointing operation detected in step S31 (step S37). Here, the operation detection control unit 152 may obtain the display position of the right display unit 22 and the left display unit 24 corresponding to the acquired designated position.

  The operation detection control unit 152 determines whether or not to execute a combination operation (step S38). The combination operation is as described above. When the combination operation is not performed (step S38; No), the display control unit 147, similar to step S23, the content of the display change corresponding to the motion determined by the operation detection control unit 152 in step S36 and the display change amount. Is determined (step S39).

  The display control unit 147 changes the display of the image display unit 20 according to the contents and the amount of change of the display determined in step S39 with the designated position acquired by the operation detection control unit 152 in step S37 as a reference (step S40). Then, the process proceeds to step S45 described later.

When performing the combination operation (step S38; Yes), the display control unit 147 starts the display change corresponding to the motion determined by the operation detection control unit 152 in step S36 (step S41). In step S41, the designated position acquired by the operation detection control unit 152 in step S37 is used as a reference for display change.
In steps S39 and S41, the display control unit 147 executes display enlargement / reduction with the designated position as the center position CE, for example, as shown in FIGS.

  After starting the display change, the operation detection control unit 152 analyzes the movement of the case 10A based on the detection value of the 6-axis sensor 111 (step S42). The operation detection control unit 152 determines whether or not the movement of the case 10A corresponding to the inverse motion of the motion determined in step S20 has been performed (step S43). While the reverse motion is not performed (step S43; No), the analysis of steps S42 to S43 is performed and the process waits.

When the operation detection control unit 152 determines that the reverse motion is performed (step S43; Yes), the display control unit 147 stops the display change (step S44), and proceeds to step S45.
In step S45, the operation detection control unit 152 determines whether the trigger operation has been released (step S45). If the trigger operation has not been released (step S45; No), the process returns to step S35. Further, when the operation of the operator assigned to the trigger operation is released (step S45; Yes), this processing ends.

If the trigger operation is canceled before it is determined that the reverse motion is performed in steps S26 to S27, the display control unit 147 stops the display change in step S28 as in the case where the reverse motion is performed. Then, this process may be terminated.
Similarly in steps S42 to S43, if the trigger operation is canceled before it is determined that the reverse motion is performed, the display control unit 147 changes the display in step S44 as in the case where the reverse motion is performed. This process may be terminated by stopping.
Furthermore, in addition to the inverse motion, a specific motion that stops the temporal change may be set in advance. For example, the movement of moving the case 10A up and down in small increments with a smaller motion amount than the motions E and F may be set as a stop motion that serves as a trigger to stop the change over time. In this case, in steps S27 and S43, the operation detection control unit 152 may determine that the reverse motion or the stop motion has been performed.

  Further, in step S31 described above, when the operation detection control unit 152 detects the first contact operation with respect to the track pad 14, it may be determined that the position pointing operation has been performed, but the pointing position is determined by the user's instruction. You may. For example, when the contact operation on the track pad 14 is continuously performed, it may be determined that the position pointing operation is performed at the timing designated by the user, and the designated position at the timing may be acquired in step S37. .. The method of instructing the timing by the user may be, for example, operation of an operator other than the track pad 14 included in the control device 10. Further, for example, the timing may be designated by the movement of the image display unit 20. Specifically, the motion of moving the image display unit 20 up and down in small increments may be set as the motion of the display unit that designates the timing of the position instruction. The operation detection control unit 152 can detect and determine the movement of the image display unit 20 based on the detection value of the 6-axis sensor 235. In this case, the user can specify the timing by moving the head while operating the track pad 14. Therefore, the user who wears the image display unit 20 on his / her head, especially in front of his / her eyes, can easily perform the operation of designating an arbitrary position without looking at the control device 10. Further, from the time when the user starts the contact operation of the track pad 14 until the timing is specified by the movement of the image display unit 20, an image or the like indicating the position designated by the track pad 14, that is, the contact position is displayed on the image display unit. It may be displayed on 20. In this case, the user can adjust the operation position of the track pad 14 while observing the display on the image display unit 20 and determine the arbitrary instructed position by the movement of the head, further improving the operability. be able to.

  As described above, the HMD 100 is a head-mounted display device having the image display unit 20 that allows the user to visually recognize the image. The HMD 100 has a control device 10 as an operation device that can be moved independently of the image display unit 20. The control device 10 includes a 6-axis sensor 111 that detects a movement of the control device 10, and a track pad 14 that receives an operation different from the detection of the 6-axis sensor 111. The HMD 100 includes a control unit 150 that controls the display of the image display unit 20 according to the movement detected by the 6-axis sensor 111 while the track pad 14 is accepting an operation. Accordingly, the user can easily control the display of the image display unit 20 by operating the control device 10. Further, it is possible to prevent the display from being changed unintentionally by the user depending on the operation state of the track pad 14.

  In addition, the operation detection control unit 152 detects an operation of the trackpad 14 which can be touched as a motion operation trigger. Further, the trigger operation may be set to the operation of a button that can be pressed. In either case, it is possible to enhance the operability when operating the track pad 14 while moving the case 10A of the control device 10. The contact operation and the pressing operation are operations that can be performed by the user wearing the image display unit 20 without looking at the track pad 14 or the buttons. Therefore, even when the user wearing the head-mounted HMD 100 has a difficulty in visually recognizing the control device 10, the operation of changing the display can be easily and reliably performed.

  Further, the control unit 150 changes the display magnification of the image displayed on the image display unit 20 when the movement corresponding to the operation of rotating the device by the 6-axis sensor 111, for example, the motions A and B are detected. This allows the display magnification to be changed by an intuitive operation.

  Further, the control unit 150 displays on the image display unit 20 when a motion corresponding to an operation of lowering or raising the device by the 6-axis sensor 111, for example, motions E, E ′, F, and F ′ are detected. The image to be changed is changed in a preset direction. Thereby, the image displayed on the image display unit 20 can be changed by an intuitive operation.

  Further, the control unit 150 changes the display of the image display unit 20 when the operation of the track pad 14 is canceled after changing the display of the image display unit 20 while the track pad 14 is accepting the operation. May be confirmed. Thereby, an operation for confirming a change in display by the image display unit 20 can be performed. Therefore, the degree of freedom in changing the display can be increased, and the operability can be improved.

  Further, the control unit 150 is set with a predetermined function to be executed when the track pad 14 receives an operation. For example, the function of position input and the function of the software button corresponding to the display position of the symbol during lighting of the LED display unit 17 are set. The operation detection control unit 152 switches between the normal mode and the display operation mode for execution. The normal mode is an operation mode in which a predetermined function is executed when the trackpad 14 receives an operation. The display operation mode is an operation mode in which the display of the image display unit 20 is controlled according to the movement detected by the 6-axis sensor 111 while the track pad 14 is accepting an operation. Thereby, the operator set as the trigger can be used as an operation unit for instructing a predetermined function. Therefore, it is not necessary to provide the control device 10 with a dedicated operator used as a trigger for motion operation, which is advantageous in terms of downsizing the control device 10.

  Further, the control unit 150 executes the display operation mode when the trackpad 14 receives an operation within a set time after the 6-axis sensor 111 detects the start of movement. Thereby, it is possible to easily select whether to execute the normal mode or the display operation mode by operating the track pad 14.

  When starting the display operation mode, the control unit 150 reflects the detection history of the 6-axis sensor 111 from when the 6-axis sensor 111 detects the start of movement to when the operation of the trackpad 14 is received, and the image is displayed. The display of the display unit 20 is controlled. Thus, the movement of the control device 10 until the display operation mode is started can be reflected in the display control.

  Further, the control device 10 includes the track pad 14 that receives a position pointing operation, and receives the operation of the track pad 14 as a position pointing operation.

  Further, the control unit 150 changes the display of the image display unit 20 with the display position corresponding to the position instructed by the position instructing operation received by the track pad 14 as a reference while the track pad 14 is receiving the operation. .. Thereby, the display of the image display unit 20 can be controlled with a high degree of freedom by utilizing the position pointing operation and the movement of the control device 10. Moreover, complicated display control can be performed by a simple operation, and operability can be improved.

  Further, the control unit 150 enlarges or reduces the image displayed by the image display unit 20 around the display position corresponding to the position instructed by the position instructing operation received by the track pad 14 and displays the image. Thereby, control for enlarging / reducing the display centered on the position desired by the user can be realized by a simple operation using the control device 10.

  In addition, in the present embodiment, the motion operation trigger functions as the track pad 14 that can perform a position pointing operation. With this, since the number of objects to be operated is small, the operability is excellent when operating the control device 10 without visually recognizing it. Therefore, it is difficult for the user wearing the head-mounted HMD 100 to visually recognize the control device 10. Even if there is, it can be operated easily and surely. Further, it is advantageous when the control device 10 is downsized.

  The HMD 100 also includes a 6-axis sensor 235 that detects the movement of the image display unit 20. The control unit 150 specifies the instruction position of the operation received by the track pad 14 at the timing when the 6-axis sensor 235 detects a predetermined movement. Accordingly, the operation of determining the designated position during the position designating operation can be performed by the movement of the image display unit 20. Therefore, the operability when the user wearing the head-mounted HMD 100 operates the control device 10 can be further improved.

Further, the control unit 150 performs the first control for controlling the display of the image display unit 20 in accordance with the type of operation corresponding to the movement detected by the 6-axis sensor 111 while the track pad 14 is accepting an operation. Run. The first control relates to control for changing the display according to the motions A to F, for example, and particularly corresponds to the control of steps S23 to S24 and S39 to S40 for changing the display without a position pointing operation. Further, the second control is executed to change the display of the image display unit 20 with the display position corresponding to the position designated by the position designation operation received by the track pad 14 as a reference. The second control corresponds to, for example, an operation of enlarging or reducing the display with the designated position of the track pad 14 as the center position CE. In addition, the third control that changes the display of the image display unit 20 according to the direction of movement detected by the 6-axis sensor 111 is executed. The third control corresponds to scrolling up, scrolling down, scrolling left, scrolling right, switching images based on the display order, and the like. More specifically, this corresponds to the control using the reverse motion in steps S25 to S28 and S41 to S44. The control unit 150 can switch and execute the first control, the second control, and the third control.
Accordingly, the display of the image display unit 20 can be changed by three types of control according to the movement of the control device 10.

  In addition, the control unit 150 controls to move the image displayed by the image display unit 20 in the first display operation direction and the second display operation direction forming a pair with the first display operation direction. For example, the up scroll and the down scroll illustrated in FIGS. 9A to 9C correspond to movements in the first display operation direction and the second display operation direction. The left and right scrolls correspond to movements in the first display operation direction and the second display operation direction. Further, regarding the switching of a plurality of images in which the display order is set as shown in FIGS. 10A to 10C, the display change for sequentially switching the images and the change for switching the images in the reverse order are the first display operation direction and the first display operation direction. It can be said that this corresponds to movement in two display operation directions.

  The first display operation direction is associated with the first movement direction detected by the 6-axis sensor 111, and the second display operation direction is associated with the second movement direction detected by the 6-axis sensor 111. The first motion direction and the second motion direction correspond to a pair of motions of the case 10A. For example, the motion E is associated with the upward scroll, and the motion F is associated with the downward scroll. Accordingly, the image displayed by the image display unit 20 can be moved by moving the control device 10, and the direction of the image movement can be instructed by the movement direction of the control device 10.

In addition, the control unit 150 starts the movement of the image in the first display movement direction in response to the movement in the first movement direction detected by the 6-axis sensor 111, and then moves in the second movement direction. When detected by the 6-axis sensor 111, the movement of the image in the first display operation direction is stopped. Thereby, the image displayed by the image display unit 20 can be moved to a desired position by combining the movements of the control device 10 in the two directions.
Here, the motion detection unit that detects the motion in the first motion direction and / or the motion in the second motion direction is not limited to the 6-axis sensor 111, but may be the magnetic sensor 113. It is also possible to detect the movement in the first movement direction and / or the movement in the second movement direction by using the captured image of the camera 61. In this case, the camera 61 corresponds to the movement detection unit. .. In addition, the motion detection unit may include a function of the control unit 150 that analyzes a captured image of the camera 61 to detect a motion.

  Further, the control unit 150 starts the movement of the image in the first display operation direction or the second display operation direction in response to the movement detected by the 6-axis sensor 111, and then releases the operation accepted by the track pad 14. If so, the movement of the image on the image display unit 20 is stopped. Thereby, the image displayed by the image display unit 20 can be moved to a desired position by combining the movement of the control device 10 and the operation of the track pad 14.

  Here, as a specific example of the operation different from the detection of the 6-axis sensor 111 as the motion detection unit, the operation of the track pad 14 has been illustrated in the above embodiment. That is, the configuration in which the operation detection unit corresponds to the track pad 14 is illustrated. The present invention is not limited to this, and the up / down key 15, the changeover switch 16, the power switch 18 and the like may be used as the operation detection unit to detect an operation different from the detection of the motion detection unit. Further, when the operation of moving the case 10A is detected by the 6-axis sensor 111, which is the motion detection unit, the operation of hitting the case 10A may be the above-mentioned “different operation”. In this case, the control device 10 may include a knock sensor and a pressure sensor, and the detection of these sensors may be an operation different from the detection of the 6-axis sensor 111 serving as the motion detection unit.

  In the above embodiment, the control device 10 having the case 10A is described as an example of the operating device of the present invention, but the present invention is not limited to this. For example, it may be a card-type operation device that does not have a housing and includes a sensor that corresponds to a motion detection unit and a functional unit that corresponds to an operation reception unit. Further, the present invention can be applied to an operation device or the like integrally formed with another device. The operation device can also be called, for example, a remote controller, a control device, a control device, a small device, a motion device, or the like.

  Further, in the above embodiment, the configuration in which the operation device is connected to the image display unit 20 by wire has been illustrated, but the present invention is not limited to this, and the operation device and the image display unit 20 are wirelessly connected. May be. The wireless communication method in this case may be the method exemplified as the communication method supported by the communication unit 117, or may be another communication method.

  Further, the operating device does not need to have the function of the control device 10. In addition to the control device 10, a device, a device, a unit, or an instrument used as an operation device may be provided.

  For example, instead of the control device 10, the present invention can be applied by using a wearable device that can be attached to the user's body, clothing, or clothing worn by the user as an operation device. In this case, the wearable device may be, for example, a watch type device, a ring type device, a laser pointer, a mouse, an air mouse, a game controller, a pen type device, or the like.

Here, another example of the operation device will be described.
FIG. 13 is an external view of the timepiece device 3. The clock device 3 can be used as another example of the operation device. The timepiece device 3 is used, for example, together with the control device 10. In this case, the control device 10 may have a configuration that does not function as an operation device.

  The watch-type device 3 has a band portion 300 having a shape similar to that of a wristwatch band. The band part 300 has a fixing part such as a buckle (not shown), and can be fixed, for example, by winding it around a user's forearm. In the band portion 300 of the timepiece device 3, a substantially disk-shaped flat surface portion 300A is formed at a position corresponding to the dial of the timepiece. A bezel 301, an LCD 303, a button 305, a crown type operator 307, and a plurality of buttons 309 are provided on the flat surface portion 300A.

  The bezel 301 is a ring-shaped operator and is arranged on the peripheral portion of the flat surface portion 300A. The bezel 301 is provided rotatably in the circumferential direction with respect to the band portion 300. The timepiece device 3 includes a mechanism for detecting the rotation direction and the rotation amount of the bezel 301, as described later. Further, the mechanism for instructing the band part 300 to rotate the bezel 301 may include a memory that generates a notch sound when the bezel 301 is rotated.

The LCD 303 is a liquid crystal display that displays characters and images.
The button 305 is a push button type switch arranged outside the bezel 301. The button 305 is located below the bezel 301 when viewed from the user when the timepiece device 3 is attached. The button 305 is larger than the crown-type operator 307 and the button 309, and can be operated by feeling.

  The crown-shaped operator 307 is an operator having a shape simulating a crown of a wristwatch and can be rotated as indicated by an arrow in the figure. The timepiece-type device 3 includes a mechanism that detects the rotation direction and the amount of rotation of the crown-type operating element 307 when the user rotates the crown-type operating element 307. Further, the mechanism for rotatably instructing the band-shaped manipulator 307 to the band portion 300 may include a memory for generating a notch sound when the crown-shaped manipulator 307 is rotated.

The button 309 is a push button type switch provided on the outer peripheral portion of the flat surface portion 300A. The number of buttons 309 is not particularly limited, and this embodiment shows an example in which four buttons 309 are provided.
Different functions can be assigned to the buttons 309, and the functions assigned to the respective buttons 309 can be displayed on the LCD 303.

In this timepiece device 3, the bezel 301, the button 305, the crown type operator 307, and the button 309 can be used as an operation receiving unit. These operation accepting units may, for example, perform a mute function for temporarily suspending the display of the image display unit 20 during normal operation, and / or a function of instructing to cancel the mute function, and a type of a pointer displayed by the image display unit 20. Assigned to the function that directs changes.
Further, the timepiece device 3 includes a communication unit (not shown) that functions similarly to the communication unit 117, and a motion sensor (not shown) similar to the 6-axis sensor 111. The timepiece device 3 detects an operation on each unit that functions as an operation receiving unit, and wirelessly transmits the detection result to the control device 10. Further, the timepiece device 3 wirelessly transmits the detection result of the motion sensor to the control device 10.

  With this configuration, the control device 10 can detect an operation of moving the timepiece device 3 to, for example, the above-described motions A to F, C ′, D ′, E ′, and F ′. Further, the control device 10 detects the operation of the bezel 301, the button 305, the crown-type operator 307, and the button 309 as an operation different from the detection of the motion sensor. In this case, the display on the image display unit 20 can be controlled by operating the bezel 301, the button 305, the crown-type operator 307, and the button 309 while operating the timepiece device 3. The movement direction of the timepiece device 3 and the display control of the image display unit 20 can be the same as those in the above-described embodiment, for example.

  FIG. 14 is an external view of the ring type device 4. The ring type device 4 has a shape similar to a ring, and is attached to, for example, a user's finger. On the surface of the ring type device 4, a track pad 401 capable of detecting a contact operation is installed like the track pad 14. The ring type device 4 detects contact with the track pad 401 by a detection circuit (not shown). In this structure, the track pad 401 functions as an operation detection unit. The track pad 401 has, for example, a function of instructing execution and / or cancellation of a mute function for temporarily stopping the display of the image display unit 20 during normal operation, and a change of the type of the pointer displayed by the image display unit 20. Assigned to instructing function.

  Further, the ring-type device 4 includes a communication unit (not shown) that functions similarly to the communication unit 117, and a motion sensor (not shown) similar to the 6-axis sensor 111. The ring-type device 4 detects an operation on each unit that functions as an operation receiving unit, and wirelessly transmits the detection result to the control device 10. Further, the timepiece device 3 wirelessly transmits the detection result of the motion sensor to the control device 10.

  In this configuration, the control device 10 can detect the operation of moving the ring-type device 4 to, for example, the above-described motions A to F, C ′, D ′, E ′, and F ′. Further, the control device 10 detects the operation of the track pad 401 as an operation different from the operation of detecting the motion sensor. In this case, the display on the image display unit 20 can be controlled by operating the ring-shaped device 4 while operating the track pad 401. The movement direction of the ring-type device 4 and the display control of the image display unit 20 can be the same as those in the above-described embodiment, for example.

  When a wearable device worn on the user's body such as the timepiece device 3 and the ring device 4 is used as the operation device, the display can be easily controlled by the user's operation of moving the body. Further, since the operation different from the detection by the motion sensor is detected and the display is controlled based on the detection value by the motion sensor, it is possible to prevent the display from being changed unintended by the user. Therefore, it is possible to realize an intuitive operation, prevent an erroneous operation, and control the display as intended by the user.

  The direction in which the clock-type device 3 and the ring-type device 4 are moved can be detected and specified based on, for example, the X-axis direction, the Y-axis direction, and the Z-axis direction shown in the drawing. The motion of moving the timepiece device 3 and the ring device 4 in a specific direction is the same as the above-described motions A to F, C ′, D ′, E ′, and F ′ in the first movement direction, and , In the second movement direction. Further, the motion in the first motion direction and the second motion direction are set as motions in a specific direction, but motions close to the specific direction may be regarded as motions in the specific direction. .. The range regarded as the motion in the same direction can be set arbitrarily, and may be, for example, ± 5 degrees or ± 10 degrees. The same applies to the first movement direction and the second movement direction. The reference of the movement direction of the timepiece device 3 and the ring device 4 is preset in association with the detection direction of the motion sensor incorporated in each of the timepiece device 3 and the ring device 4. Just do it. Therefore, it is possible to set any direction as a reference, and it is not limited to the X-axis direction, the Y-axis direction, and the Z-axis direction shown in the drawing.

  Further, the timepiece device 3 and the ring device 4 are not limited to the configuration used together with the control device 10. It suffices that the HMD 100 has a configuration that has a function equivalent to that of the control device 10 and that can control the display of the image display unit 20. Therefore, if the image display unit 20 is configured to be capable of accommodating each unit including the main processor 140, for example, the image display unit 20 and the clock type device 3 or the image display unit 20 and the ring type device 4 configure the HMD 100. it can.

The present invention is not limited to the configuration of the above embodiment, and can be implemented in various modes without departing from the scope of the invention.
For example, in the above-described embodiment, the configuration in which the user sees the outside scene through the display unit is not limited to the configuration in which the right light guide plate 26 and the left light guide plate 28 transmit the outside light. For example, it can be applied to a display device that displays an image in a state where the outside scene cannot be visually recognized. Specifically, the present invention is applied to a display device that displays a captured image of the camera 61, an image or CG generated based on this captured image, video data stored in advance, video based on video data input from the outside, or the like. Can be applied. This type of display device may include a so-called closed type display device in which the outside scene cannot be visually recognized. Further, as described in the above embodiment, an AR display for displaying an image in a real space by superimposing it, an MR (Mixed Reality) display for combining a captured real space image and a virtual image, or a VR (for displaying a virtual image). It can also be applied to a display device that does not perform processing such as virtual reality display. For example, a display device that displays video data or an analog video signal input from the outside is naturally included as an application target of the present invention.

  Further, in the above-described embodiment, enlargement, reduction, scrolling, and image switching based on the display order are illustrated as examples of changing the display mode of the image displayed on the image display unit 20, but the present invention is not limited to this. For example, control for rotating the image, control for changing the display brightness of the image, control for enlarging or reducing the display size of a part of the image or character being displayed may be performed in response to the motion. In this case, the rotation direction of the image, the change direction of the display brightness of the image (luminance increase or decrease), and the size enlargement or reduction may be associated with different motions. Further, the operation of the trigger for starting the operation by motion is not limited to the operation pad such as the trackpad 14 and the buttons of the control device 10, and may be any operation that can be detected by a method different from the movement of the case 10A. Specifically, it is sufficient that the detection value is different from the detection value of the 6-axis sensor 111, and thus the movement of the image display unit 20 detected by the 6-axis sensor 235 may be used as the trigger operation. Further, for example, a touch sensor (not shown) may be provided in the image display unit 20, and an operation on this touch sensor may be used as a trigger operation.

  Further, for example, instead of the image display unit 20, an image display unit of another system such as an image display unit worn like a hat may be adopted, and an image is displayed corresponding to the left eye of the user. It suffices to include a display unit that displays the image and a display unit that displays an image corresponding to the right eye of the user. Further, the display device of the present invention may be configured as, for example, a head mounted display mounted on a vehicle such as an automobile or an airplane. Further, for example, it may be configured as a head mounted display incorporated in a body protection tool such as a helmet. In this case, the part that positions the position of the user with respect to the body and the part that is positioned with respect to the part can be the mounting part.

Furthermore, in the above-described embodiment, the image display unit 20 and the control device 10 are separated and connected via the connection cable 40, but the control device 10 and the image display unit 20 are integrated. It is also possible to adopt a configuration in which it is attached to the head of the user.
A laptop computer, a tablet computer, or a desktop computer may be used as the control device 10. Further, as the control device 10, a portable electronic device including a game machine, a mobile phone, a smartphone, a mobile media player, or other dedicated device may be used. Further, the control device 10 may be configured separately from the image display unit 20, and various signals may be transmitted and received between the control device 10 and the image display unit 20 by wireless communication.

  In the above embodiment, the image display unit 20 and the control device 10 are separated and connected via the connection cable 40, but the control device 10 and the image display unit 20 are wireless. It may be configured to be connected by a communication line.

Further, as an optical system that guides image light to the eyes of the user, a configuration in which a virtual image is formed by the half mirrors 261 and 281 on a part of the right light guide plate 26 and the left light guide plate 28 has been illustrated. The present invention is not limited to this, and an image may be displayed in a display region having an area that occupies the entire surface or most of the right light guide plate 26 and the left light guide plate 28. In this case, the operation of changing the display position of the image may include a process of reducing the image.
Furthermore, the optical element of the present invention is not limited to the right light guide plate 26 and the left light guide plate 28 having the half mirrors 261, 281 and may be any optical component that allows image light to enter the eyes of the user. May use a diffraction grating, a prism, or a holographic display unit.

  Further, at least some of the functional blocks shown in FIGS. 4 and 5 may be realized by hardware, or may be realized by the cooperation of hardware and software. It is not limited to a configuration in which independent hardware resources are arranged as described above. The program executed by the control unit 150 may be stored in the non-volatile storage unit 121 or another storage device (not shown) in the control device 10. Alternatively, the program stored in an external device may be acquired via the communication unit 117 or the external connector 184 and executed. In addition, among the components formed in the control device 10, the operation unit 110 may be formed as a user interface (UI). Moreover, the configuration formed in the control device 10 may be duplicated in the image display unit 20. For example, a processor similar to the main processor 140 may be arranged in the image display unit 20, or the main processor 140 included in the control device 10 and the processor of the image display unit 20 may be configured to execute separate functions. Good.

  3 ... Clock type device (operating device), 4 ... Ring type device (operating device), 10 ... Control device (operating device), 14 ... Track pad (operation accepting unit, position instruction operating unit), 20 ... Image display unit ( Display unit), 22 ... Right display unit, 24 ... Left display unit, 26 ... Right light guide plate, 28 ... Left light guide plate, 30 ... Headset, 40 ... Connection cable, 61 ... Camera (motion detection unit), 63 ... Microphone , 65 ... Illuminance sensor, 67 ... LED indicator, 100 ... HMD (display device), 110 ... Operation part, 111 ... Six-axis sensor (motion detection part), 113 ... Magnetic sensor (motion detection part), 115 ... GPS, 117 ... communication section, 118 ... memory, 119 ... vibrator, 120 ... controller board, 121 ... nonvolatile storage section, 122 ... storage section, 123 ... setting data , 124 ... Content data, 125 ... Operation setting data, 126 ... Display control data, 130 ... Power supply section, 132 ... Battery, 134 ... Power supply control circuit, 140 ... Main processor, 143 ... Operating system, 145 ... Image processing section, 147 ... Display control unit, 149 ... Imaging control unit, 150 ... Control unit, 152 ... Operation detection control unit, 180 ... Audio codec, 182 ... Audio interface, 184 ... External connector, 186 ... External memory interface, 188 ... USB connector, 192 ... Sensor hub, 194 ... FPGA, 196 ... Interface, 211, 231 ... Interface, 213, 233 ... Receiver, 215 ... EEPROM, 217, 239 ... Temperature sensor, 221, 241 ... OLED unit, 235 ... 6-axis sensor -(Display unit motion detection unit), 237 ... Magnetic sensor, 300 ... Band unit, 300A ... Planar unit, 301 ... Bezel (operation receiving unit), 303 ... LCD, 305 ... Button (operation receiving unit), 307 ... Crown type Operator (operation accepting section), 309 ... Button (operation accepting section), 401 ... Track pad (operation accepting section).

Claims (18)

A head-mounted display device having a display unit for allowing a user to visually recognize an image,
An operation that is an operation device that can be moved independently of the display unit, and has a motion detection unit that detects a motion of the operation device, and an operation reception unit that receives an operation that is different from the operation detected by the motion detection unit. Device and
A control unit that controls the display of the display unit in accordance with an operation of moving the operation device, which is detected using the motion detection unit while the operation reception unit is receiving an operation ,
The control unit changes a display magnification of an image displayed on the display unit when a motion corresponding to an operation of rotating the operation device is detected by the motion detection unit;
Display device according to claim. A head-mounted display device having a display unit for allowing a user to visually recognize an image,
An operation that is an operation device that can be moved independently of the display unit, and has a motion detection unit that detects a motion of the operation device, and an operation reception unit that receives an operation that is different from the operation detected by the motion detection unit. Device and
A control unit that controls the display of the display unit in accordance with an operation of moving the operation device, which is detected using the motion detection unit while the operation reception unit is receiving an operation,
The control unit changes the image displayed on the display unit in a preset direction when the motion detection unit detects a motion corresponding to an operation of lowering or raising the operation device.
Viewing apparatus according to claim. A head-mounted display device having a display unit for allowing a user to visually recognize an image,
An operation that is an operation device that can be moved independently of the display unit, and has a motion detection unit that detects a motion of the operation device, and an operation reception unit that receives an operation that is different from the operation detected by the motion detection unit. Device and
A control unit that controls the display of the display unit in accordance with an operation of moving the operation device, which is detected using the motion detection unit while the operation reception unit is receiving an operation,
The control unit is set with a predetermined function to be executed when the operation receiving unit receives an operation,
The display of the display unit is controlled according to the normal mode in which the predetermined function is executed when the operation receiving unit receives the operation, and the motion detected by the motion detecting unit while the operation receiving unit receives the operation. To switch between the display operation mode and
Viewing apparatus according to claim. The control unit executes the display operation mode when the operation reception unit receives an operation within a set time after the motion detection unit detects the start of movement,
The display device according to claim 3, wherein: When starting the execution of the display operation mode, the control unit detects the motion detected by the motion detection unit from when the motion detection unit detects the start of motion until the operation reception unit receives an operation. Controlling the display of the display unit by reflecting the detection history,
The display device according to claim 4, wherein: A head-mounted display device having a display unit for allowing a user to visually recognize an image,
An operation that is an operation device that can be moved independently of the display unit, and has a motion detection unit that detects a motion of the operation device, and an operation reception unit that receives an operation that is different from the operation detected by the motion detection unit. Device and
A control unit that controls the display of the display unit in accordance with an operation of moving the operation device, which is detected using the motion detection unit while the operation reception unit is receiving an operation,
The operation device includes a position indicating operation unit that receives a position indicating operation ,
The control unit receives the position instruction operation by the position instruction operation unit, and in a state where the operation reception unit receives the operation, at the position instructed by the position instruction operation received by the position instruction operation unit. Changing the display of the display unit based on the corresponding display position,
Viewing apparatus according to claim. The control unit enlarges or reduces an image displayed by the display unit around a display position corresponding to a position designated by the position designation operation received by the position designation operation unit, and displays the image.
The display device according to claim 6 , characterized in that: The position instruction operation unit functions as the operation reception unit,
The display device according to claim 6 or 7 , characterized in that. A display unit for detecting movement of the display unit;
The control unit specifies a pointing position of an operation received by the position pointing operation unit at a timing at which the display unit movement detection unit detects a predetermined movement,
Display device according to claim 6, wherein 8. The control unit is
A first control for controlling the display of the display unit according to the type of operation corresponding to the motion detected by the motion detection unit in a state where the operation reception unit is receiving an operation;
Second control for changing the display of the display unit based on the display position corresponding to the position designated by the position designation operation received by the position designation operation unit;
And a third control for changing the display of the display unit in accordance with the direction of the movement detected by the movement detection unit, which is executable.
Display device according to claim 6, wherein 9 of the. The operation receiving unit receives at least one of a pressing operation and a contact operation,
The control unit controls display of the display unit according to movement of the operation device, including at least rotation or movement of the operation device;
The display device according to claim 1 , wherein the display device is a display device. The control unit, after changing the display of the display unit in a state in which the operation receiving unit is accepting an operation, when the operation is released, to confirm the change in the display of the display unit,
Display device according to any one of claims 1 to 11, characterized in. The control unit performs control to move an image displayed by the display unit in a first display operation direction and a second display operation direction that forms a pair with the first display operation direction,
The first display motion direction is associated with the first motion direction detected by the motion detection unit, and the second display motion direction is associated with the second motion direction detected by the motion detection unit. To be done,
Display device according to claim 1, wherein 12 of the. The control unit starts the movement of the image in the first display operation direction in response to the movement in the first movement direction detected by the movement detection unit, and then moves in the second movement direction. Is detected by the motion detector, the motion of the image in the first display motion direction is stopped.
The display device according to claim 13, wherein: The control unit starts the motion of the image in the first display motion direction or the second display motion direction in response to the motion detected by the motion detection unit, and then the operation accepted by the operation accepting unit. When is released, to stop the movement of the image of the display unit,
The display device according to claim 14, wherein: The operation device is a wearable device worn on the body of the user,
Display device according to any one of 15 claims 1, wherein the. Operation having a display unit that allows a user to visually recognize an image, an operation device that can be moved independently of the display unit, and has a motion detection unit that detects a motion of the operation device and an operation reception unit that receives an operation And a head-mounted display device including a device,
In a state where the operation accepting unit is accepting an operation, the display of the display unit is controlled according to the operation of moving the operation device, which is detected using the motion detecting unit ,
Changing a display magnification of an image displayed on the display unit when a motion corresponding to an operation of rotating the operation device is detected by the motion detection unit ,
And a method for controlling a display device. Operation having a display unit that allows a user to visually recognize an image, an operation device that can be moved independently of the display unit, and has a motion detection unit that detects a motion of the operation device and an operation reception unit that receives an operation A device and a program executable by a computer that controls a head-mounted display device including:
In a state where the operation accepting unit is accepting an operation, the display of the display unit is controlled according to the operation of moving the operation device, which is detected using the motion detecting unit ,
A program for changing a display magnification of an image displayed on the display unit when a motion corresponding to an operation of rotating the operation device is detected by the motion detection unit .

Images