JP2017078756A - Head-mounted display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a head-mounted display device with good visibility.SOLUTION: The present head-mounted display device is a head-mounted display device of a retina projection type, and comprises: a frame that is arranged on the head in front of the eyes of a user; a light source that emits a laser beam; a movable mirror that adjusts the direction of reflection of the laser beam emitted from the light source; a mirror that reflects the laser beam reflected on the movable mirror to the pupils of the user; a detection part that is attached to the frame, detects the positions of the pupils of the user, and outputs a position signal showing the positions; and a control part that, on the basis of displacement of the positions shown by the position signal with respect to reference positions, adjusts the angle of the movable mirror so that the laser beam reflected on the mirror is made incident on the pupils of the user.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a head mounted display device.

  Conventionally, a light source that emits modulated light, a scanner that scans the modulated light emitted from the light source, a control unit that controls the light source and the scanner, and the modulated light scanned by the scanner are projected onto the eye. There is an image display device including an eyepiece optical system and an image pickup unit provided so as to be able to take an image around at least a pupil. The image display device further includes a pupil position specifying unit that specifies a pupil position from an image around the pupil imaged by the imaging unit, and an optical axis that is a position of the optical axis of the modulated light based on the image around the pupil An optical axis position specifying means for specifying the position is provided. The image display device further uses the optical axis position of the modulated light as the pupil position based on the relative position between the pupil position specified by the pupil position specifying means and the optical axis position specified by the optical axis position specifying means. A correction means for moving the eyepiece optical system so as to fit is provided. Such a conventional image display device is a retinal scanning head-mounted display device (see, for example, Patent Document 1).

JP 2012-159559 A

  The correction means for moving the eyepiece optical system of the conventional retinal scanning head-mounted display device moves the eyepiece optical system so that the optical axis position of the modulated light matches the pupil position. Such movement is to move the eyepiece optical system in the horizontal direction or vertical direction of the user.

  By the way, when the pupil of the user wearing the retinal scanning type head-mounted display device is facing the front, the pupil moves so as to make an angle in the left-right direction or the up-down direction. The incident angle of light incident on the lens changes, and the position where the image is formed on the retina changes.

  For this reason, before the pupil moves, the image formed at the approximate center of the retina is changed to a state where the image is formed at the end of the retina after the pupil rotates, and the image formed on the retina is distorted. There is a problem that the visibility of an image is lowered. Due to such distortion of the image, for example, the user may get drunk on the video and may feel unwell.

  Further, if the pupil further moves and light does not pass through the pupil, there is a possibility that the user cannot see the image even if the eyepiece optical system is moved.

  As described above, the conventional head mounted display device has poor image visibility for the user.

  Accordingly, it is an object to provide a head mounted display device with good visibility.

  A head-mounted display device according to an embodiment of the present invention is a retinal projection-type head-mounted display device, and includes a frame disposed in front of a user's head, a light source that emits laser light, and the light source. A movable mirror that adjusts the reflection direction of the emitted laser light, a mirror that reflects the laser light reflected by the movable mirror toward the user's pupil, and a position of the user's pupil that is attached to the frame And a detection unit that outputs a position signal representing the position, and the laser light reflected by the mirror is incident on a user's pupil based on a displacement of the position represented by the position signal with respect to a reference position. And a controller for adjusting the angle of the movable mirror.

  A head-mounted display device with good visibility can be provided.

1 is a perspective view showing a head mounted display device 100 according to a first embodiment. 1 is a diagram illustrating a configuration of a head mounted display device 100 according to a first embodiment. 1 is a diagram illustrating a configuration of a head mounted display device 100 according to a first embodiment. 2 is a diagram showing an XY stage 120. FIG. It is a figure which shows the structure of the control part. It is a figure which shows the position of the pupil detected by the control part. It is a figure which shows the method of angle adjustment of the MEMS mirror 140 based on the displacement of the position of a pupil. It is a flowchart which shows the process which the control part 170 performs. It is a figure in which the pupil which can perform retina drawing shows a movement range. It is a figure which shows 100 A of head mounted display apparatuses by the modification of Embodiment 1. FIG. 6 is a diagram showing a head mounted display device 200 according to a second embodiment. FIG. It is a figure which shows the structure of the control part. FIG. 10 is a diagram illustrating a relationship between a range where raster scanning is performed and a region where laser light is output in the second embodiment.

  Embodiments to which the head mounted display device of the present invention is applied will be described below.

<Embodiment 1>
FIG. 1 is a perspective view showing a head mounted display device 100 according to the first embodiment. 2 and 3 are diagrams showing the configuration of the head-mounted display device 100 according to the first embodiment.

  FIG. 1 shows an appearance of the head mounted display device 100 as viewed obliquely from the front, and FIG. 2 shows components of the head mounted display device 100 in a plan view. FIG. 3 shows components of the head mounted display device 100 in a rear view. In FIG. 2 and FIG. 3, in order to make the configuration easy to understand, some components are shown enlarged.

  2 and 3, an XYZ coordinate system, which is an example of an orthogonal coordinate system, is defined as illustrated. In addition, since the head mounted display device 100 is a device worn by the user on the head, the following description will be made using the left, right, top, and bottom viewed from the user in addition to the XYZ coordinate system.

  The X axis is an axis extending in the left-right direction for the user wearing the head mounted display device 100. The X axis is positive in the direction from left to right. The Y axis is an axis extending in the vertical direction for a user wearing the head mounted display device 100. The Y axis is positive in the direction from top to bottom. The Z-axis is an axis that extends in the front-rear direction for a user wearing the head-mounted display device 100. The Z axis is positive in the direction from the rear to the front.

  The head mounted display device 100 includes a frame 110, lenses 115R and 115L, an XY stage 120, a light source 130, a MEMS (Micro Electro Mechanical Systems) mirror 140, a half mirror 150, a camera module 160, and a control unit 170.

  In the following description, FIG. 4 is added to FIG. 1 to FIG. FIG. 4 is a diagram showing the XY stage 120. Here, for convenience of explanation, transmission lines for transmitting control signals between the control unit 170, the XY stage 120, the light source 130, the MEMS mirror 140, and the camera module 160 are shown outside the frame 110. The transmission line is inserted into the frame 110.

  The frame 110 has the same shape as the frame of the glasses, and includes a front frame 111, a side frame 112R (Right: right), 112L (Left: left), and a nose pad 113R (Right: right), 113L (Left: Left). The side frames 112R and 112L are attached to the left and right ends of the front frame 111, respectively. Lenses 115R and 115L are attached to the front frame 111.

  A recess 111A is provided on the left side of the front frame 111 in the negative Z-axis direction. In addition, a recess 112LA is provided on the X-axis positive direction side on the Z-axis positive direction side in the longitudinal direction (Z-axis direction) of the side frame 112L. The recess 112LA is provided over approximately half of the side frame 112L in the longitudinal direction.

  Here, the side frames 112R and 112L are fixed to the front frame 111, but the side frames 112R and 112L may be attached to the front frame 111 so as to fold the glasses.

  The lenses 115R (Right: right) and 115L (Left: left) are respectively attached to the left and right of the front frame 111, and are positioned in front of both eyes when the user wears the head mounted display device 100. Are arranged as follows. The lenses 115R and 115L are made of transparent glass or plastic.

  The XY stage 120 includes a base 121, a movable part 122, a stage 123, and motors 124X and 124Y (see FIG. 4). The XY stage 120 is driven by a drive signal input from the control unit 170.

  The base 121 is a rectangular parallelepiped member that is fitted in a recess 111A provided on the left end of the front frame 111 on the Z-axis negative direction (see FIGS. 2 and 3). The base 121 is a portion that is fixed to the front frame 111 and does not move.

  The movable portion 122 is a rectangular parallelepiped member that is provided on the Z axis negative direction side of the base 121 via the rotation shaft 124YA of the motor 124Y and is movable in the Y axis direction with respect to the base 121. The movable part 122 moves in the Y-axis direction with respect to the base part 121 by driving the motor 124Y.

  The moving amount of the movable part 122 in the Y-axis direction with respect to the base part 121 is, for example, 10 mm, and can be moved at a speed of 10 mm / second as an example.

  The stage 123 is provided on the Z axis negative direction side of the movable portion 122 via the rotation shaft 124XA of the motor 124X. The stage 123 can move in the Y-axis direction with respect to the base 121 together with the movable portion 122, and can move in the X-axis direction with respect to the movable portion 122. By driving the motor 124X, the motor 124X moves in the X-axis direction with respect to the movable portion 122.

  The amount of movement of the stage 123 in the X-axis direction with respect to the movable portion 122 is, for example, 10 mm, and can be moved at a speed of 10 mm / second as an example.

  The light source 130 and the half mirror 150 are fixed to the stage 123 via the stay 125. A MEMS mirror 140 is fixed to the light source 130 via a stay 126. Therefore, the light source 130, the MEMS mirror 140, and the half mirror 150 move in the X axis direction and the Y axis direction together with the stage 123.

  The motor 124X is provided for moving the stage 123 in the X-axis direction with respect to the movable portion 122. The stage 123 is provided on the Z axis negative direction side of the movable portion 122 via the rotation shaft 124XA of the motor 124X. The drive control of the motor 124X is performed by the control unit 170.

  The motor 124Y is provided to move the movable portion 122 in the Y-axis direction with respect to the base portion 121. The movable part 122 is provided on the Z axis negative direction side of the base part 121 via the rotation shaft 124YA of the motor 124Y. The drive control of the motor 124Y is performed by the control unit 170.

  Here, a mode in which the light source 130 and the half mirror 150 are fixed to the stage 123 via the stay 125 and the MEMS mirror 140 is fixed to the light source 130 via the stay 126 will be described. Any fixing tool may be used.

  The MEMS mirror 140 may be fixed to the stage 123 by a fixture (not shown) instead of being fixed to the light source 130 by the stay 126.

  The light source 130 is fixed to the stage 123 via the stay 125. For this reason, the light source 130 moves in the X-axis direction and the Y-axis direction together with the stage 123. The light source 130 is provided so as to be accommodated in the recess 112LA of the side frame 112L in the XZ plan view and to protrude below the side frame 112L in the XY plan view.

  The light source 130 includes laser arrays 131R, 131G, and 131B. Each of the laser arrays 131R, 131G, and 131B is a laser device that emits laser light of three primary colors (red (R), green (G), and blue (B)) of light.

  The light source 130 adjusts the output of the laser arrays 131R, 131G, and 131B based on the image signal input from the control unit 170, and outputs laser light of the color and brightness represented by the image signal. The laser light output from the light source 130 is scanned pixel by pixel by a raster scan method when reflected by the MEMS mirror 140, and further reflected by the half mirror 150 and incident on the pupil of the user.

  The MEMS mirror 140 is fixed to the light source 130 via a stay 126. For this reason, the MEMS mirror 140 moves in the X axis direction and the Y axis direction together with the stage 123 and the light source 130. The MEMS mirror 140 is an example of a movable mirror.

  The MEMS mirror 140 is provided on the negative side of the Z-axis from the light source 130 so as to fit in the recess 112LA of the side frame 112L in the XZ plan view and to protrude below the side frame 112L in the XY plan view. ing.

  The MEMS mirror 140 is a mirror that can adjust the angle in the biaxial direction, and reflects the laser beam output from the light source 130 while being changed by being driven by the control unit 170. The laser beam reflected by the MEMS mirror 140 is further reflected by the half mirror 150 and enters the pupil of the user. An angle adjustment signal for adjusting the angle is input to the MEMS mirror 140 from the controller 170.

  When the MEMS mirror 140 is driven by the control unit 170, the laser light incident on the pupil from the half mirror 150 forms an image on the retina by a raster scan method.

  The angle of the MEMS mirror 140 is adjusted by the control unit 170 in accordance with the movement amount of the pupil of the user in addition to the driving for the raster scan as described above. The position of the pupil is detected by the control unit 170 based on the image captured by the camera module 160. The camera module 160 outputs an image signal representing the captured image to the control unit 170.

  Based on the image captured by the camera module 160, the movement amount of the pupil is detected as a displacement of the pupil position with respect to a predetermined reference position. The MEMS mirror 140 is driven by the laser beam reflected from the half mirror 150 toward the pupil in accordance with the change in the position of the pupil when the user changes the direction of the eye and changes the direction of the pupil. This is done to adjust the angle. The details of driving the MEMS mirror 140 will be described later.

  The half mirror 150 is a curved plate-like half mirror that is curved like an elliptical arc having two focal points in the XZ plan view and extends along the Y-axis direction. The half mirror 150 is an example of a fixed mirror.

  The two focal points of the half mirror 150 in the XZ plan view are located at the center of the MEMS mirror 140 and the center 181A of the user's pupil 181 shown in FIG. The position of the user's pupil 181 shown in FIG. 2 is a position that is given as the position of the ideal pupil 181 when the user's eyeball 182 is placed at an ideal position in the design of the head mounted display device 100. is there.

  Half mirror 150 is fixed to stage 123 via stay 125. For this reason, the half mirror 150 moves in the X axis direction and the Y axis direction together with the stage 123. The half mirror 150 is provided on the Z axis negative direction side of the lens 115L, and overlaps a part of the lens 115L in the XY plan view. The position of the half mirror 150 is a position that is placed in front of the user's left eye when the user wears the head-mounted display device 100, and is closer to the positive direction side than the MEMS mirror 140 in the Z-axis direction. Be placed.

  The half mirror 150 reflects the laser beam reflected by the MEMS mirror 140 and incident on the surface on the negative side of the Z axis, and reflects the light incident on the surface on the positive side of the Z axis without reflecting the light incident on the surface on the positive side of the Z axis. It is an optical element that passes through. The half mirror 150 is realized by, for example, a beam splitter.

  The beam splitter used as the half mirror 150 reflects the laser beam incident from the MEMS mirror 140 (the intensity of the reflected light), enters the half mirror 150 from the Z axis positive direction side, and exits from the Z axis negative direction side. The light intensity (transmitted light intensity) is approximately 1: 1. However, instead of such a half mirror 150, a beam splitter having different reflected light intensity and transmitted light intensity may be used.

  The half mirror 150 has a curved shape in the XZ plan view, and the surface on the Z-axis positive direction side is a convex surface and the surface on the Z-axis negative direction side is a concave surface. Such a half mirror 150 is fixed to the stay 125 in a state in which the concave surface is inclined to the X axis negative direction side with respect to the Z axis. This is because the concave surface of the half mirror 150 is directed toward the MEMS mirror 140.

  The camera module 160 is disposed at a position below the lens 115L and below the nose pad 113L on the negative side of the front frame 111 in the Z-axis direction. This is to take an image of the pupil of the user's left eye.

  The camera module 160 includes a camera 161 and a lighting device 162. The camera 161 and the illumination device 162 are attached to the front frame 111 in a state in which the orientation is set so as to face the center of the left eye of the user wearing the head mounted display device 100.

  Here, as an example, the camera 161 is an infrared camera, and the illumination device 162 is an infrared irradiation device. Imaging by the camera 161 and on / off switching of the lighting device 162 are performed by the control unit 170. Data representing an image captured by the camera 161 is input to the control unit 170.

  The controller 170 is attached to the side frame 112L on the X axis negative direction side. The control unit 170 controls each unit of the head mounted display device 100. The control unit 170 is realized by, for example, a microcomputer chip. The control unit 170 includes a memory.

  The control unit 170 detects the position of the pupil of the left eye of the user based on the image captured by the camera module 160, controls the position of the XY stage 120, controls the output of the laser arrays 131R, 131G, and 131B of the light source 130, and MEMS. The angle control of the mirror 140 is performed.

  The control unit 170 is connected to a computer (not shown) via a cable 170A. The computer outputs an image signal representing an image drawn on the retina of the user by the head mounted display device 100 to the control unit 170 of the head mounted display device 100 via the cable 170A.

  Here, the controller 170 will be described with reference to FIG. FIG. 5 is a diagram illustrating a configuration of the control unit 170. FIG. 6 is a diagram illustrating the position of the pupil detected by the control unit 170.

  The control unit 170 includes a position detection unit 171, a position control unit 172, an output control unit 173, and an angle control unit 174.

  The position detection unit 171 detects the position of the pupil by driving the camera module 160 and performing image processing on an image captured by the camera module 160. The position detector 171 detects the displacement of the pupil position with respect to a predetermined reference position in the image.

  Here, the position of the pupil detected by the position detection unit 171 will be described with reference to FIG. FIG. 6 shows an image 163 captured by the camera module 160. The image 163 is acquired as an image in the XY plane.

  As shown in FIG. 6, a reference position 164 is defined in the image 163. Here, as an example, the reference position 164 is the center of the image 163. Further, the position detection unit 171 obtains the center position 165 of the pupil in the image 163 and obtains the displacement of the center position 165 with respect to the reference position 164.

  In FIG. 6, for the convenience of explanation, a thick circle 165A is shown as the position of the pupil. The center of the thick circle 165A is the center position 165 of the pupil. FIG. 6 shows an ellipse 166 as a rough position of the eye contour. The thick circle 165A and the ellipse 166 are shown in FIG. 6 for convenience of explanation, and are not obtained by the position detection unit 171 and are not obtained as data.

  In FIG. 6, the center position 160C of the pupil is displaced from the reference position 164 by + X1 in the X-axis direction and + Y1 in the Y-axis direction.

  In addition to the above-described methods, there are various known methods for detecting the position of the pupil by performing pattern matching or the like on the image captured by the camera module 160. The position detection unit 171 may detect the center position 165 of the pupil by performing image processing on an image captured by the camera module 160 using any of these methods. Note that since the type of camera used as the camera module 160 differs depending on the method of detecting the pupil center position 165, a camera suitable for detecting the pupil center position 165 may be used as the camera module 160.

  Further, the position detection unit 171 obtains the angle adjustment amount (correction amount) of the MEMS mirror 140 based on the displacement (X, Y) of the pupil position obtained as described above. Further, the displacement (X, Y) of the pupil position obtained as described above is used as the movement amount of the XY stage 120.

  The position control unit 172 controls the position of the XY stage 120 based on the displacement of the pupil center position 165 detected by the position detection unit 171. The position control unit 172 drives the motor 124X to move the stage 123 to a movable unit when the displacement in the X-axis direction and the Y-axis direction of the center position 165 of the pupil detected by the position detection unit 171 is X1 and Y1, respectively. While moving X1 in the X-axis direction relative to 122, the motor 124Y is driven to move the movable part 122 relative to the base 121 in the Y-axis direction.

  The output control unit 173 adjusts the outputs of the laser arrays 131R, 131G, and 131B based on an image signal input to the control unit 170 via the cable 170A. As a result, the color and brightness of the laser light emitted from the light source 130 are set according to the image signal.

  Since the head-mounted display device 100 draws one pixel at a time on the user's retina using the raster scan method, the color and luminance of the image signal are switched for each pixel.

  The angle control unit 174 controls the angle of the MEMS mirror 140. There are two types of angle control performed by the angle control unit 174.

  As the first angle control, the angle control unit 174 drives the MEMS mirror 140 so that the laser light incident on the pupil from the half mirror 150 forms an image on the retina by the raster scan method. Since the raster scan is a scan method for drawing one pixel at a time, the angle control unit 174 adjusts the angle of the MEMS mirror 140 every time one pixel is drawn so that the laser light is guided to the next pixel. To do.

  The angle control unit 174 controls the angle of the MEMS mirror 140 based on the displacement of the center position 165 of the pupil detected by the position detection unit 171 as the second angle control. The angle control unit 174 adjusts the angle of the MEMS mirror 140 as follows when the displacement in the X-axis direction and the Y-axis direction of the pupil center position 165 detected by the position detection unit 171 is X1 and Y1, respectively. (to correct.

  FIG. 7 is a diagram illustrating a method of adjusting the angle of the MEMS mirror 140 based on the displacement of the center position of the pupil. Here, the light source 130, the MEMS mirror 140, and the half mirror 150 are referred to as an optical system, and are surrounded by a broken line. The optical system is mounted on an XY stage 120 (see FIGS. 2 to 4) and is movable in the X-axis direction and the Y-axis direction.

  In FIG. 7A, a SYT coordinate system is defined as an orthogonal coordinate system used when adjusting (correcting) the angle of the MEMS mirror 140. Of the STY coordinate system, the Y axis is equal to the Y axis of the XYZ coordinate system. The S axis and the T axis are obtained by rotating the X axis and the Z axis by α degrees clockwise in the XZ plane, respectively. The S axis is included in the reflective surface 140A of the MEMS mirror 140, and the T axis is parallel to the normal line of the reflective surface 140A. The reflective surface 140A is a plane parallel to the SY plane.

  In FIG. 7A, the center axis (optical axis) of the laser light that enters the pupil 181 from the MEMS mirror 140 through the half mirror 150 and reaches the retina 183 at the back of the eyeball 182 is indicated by a one-dot chain line. The light range is indicated by a solid line and a thin broken line.

An optical axis L ₀ shown in FIG. 7A is an optical axis of reflected light reflected by the half mirror 150 when the angle of the MEMS mirror 140 is an initial value, and is parallel to the Z axis. The initial value of the angle of the MEMS mirror 140 is an angle of the MEMS mirror 140 when the optical axis of the reflected light reflected by the half mirror 150 is parallel to the Z axis, and the reflecting surface 140A is parallel to the SY plane. The angle of the MEMS mirror 140 in the case.

Further, to the center 181A of the pupil 181, an axis passing through the center 182A of the eye 182 shown by a circle with the center axis _{L 1} of the pupil 181.

In FIG. 7A, the pupil 181 of the user is facing front (facing), and the position of the XY stage 120 is the optical axis L ₀ of the reflected light of the half mirror 150 and the central axis L of the pupil 181. ₁ is adjusted to match.

In FIG. 7A, since the optical axis L ₀ of the reflected light of the half mirror 150 and the central axis L ₁ of the pupil 181 coincide, the position of the optical system shown in FIG. This is the ideal position when facing the front of the pupil 181 of the user wearing the display device 100.

  Next, a case where the pupil 181 of the user is rotated will be described. Here, the angle of rotation about the X axis is defined as θx, and the angle of rotation about the Y axis as the rotation axis is defined as θy. The angles θx and θy are respectively positive rotations when viewed from the negative direction side of the X-axis and Y-axis as viewed in the positive direction, and from the negative direction side of the X-axis and Y-axis to the positive direction side. Rotation in the counterclockwise direction when looking at is regarded as rotation in the negative direction.

Here, if the radius of the eyeball is r and the displacement of the position of the pupil 181 detected by the position detector 171 in the Y-axis direction is Y, the angle θx is expressed by the following equation (1).
θx = arcsin (Y / r) (1)
Further, assuming that the radius of the eyeball is r and the displacement of the pupil position detected by the position detector 171 in the X-axis direction is X, the angle θy is expressed by the following equation (2).
θy = arcsin (X / r) (2)
When the displacement in the X-axis direction and the Y-axis direction of the position of the pupil 181 detected by the position detection unit 171 is X1 and Y1, respectively, the angle control unit 174 sets the angle θx and the angle θy to the following formula (3), Set as in (4).
θx = arcsin (Y1 / r) (3)
θy = arcsin (X1 / r) (4)
For example, it is assumed that the pupil 181 is rotated about the Y axis (clockwise direction) by θy in the XZ plan view. At this time, it is assumed that the displacement in the X-axis direction of the position of the pupil 181 detected by the position detection unit 171 is X1.

  In such a case, the angle control unit 174 obtains the angle θy using the displacement X1, and rotates the MEMS mirror 140 by the angle θy about the Y axis in the XZ plane. As a result, the MEMS mirror 140 is rotated clockwise from the state shown in FIG. 7A by the angle θy in the XZ plan view as shown in FIG. 7B, and the reflected light of the half mirror 150 is reflected from the pupil 181 to the eyeball. Incident on 182.

  Here, a case will be described in which the pupil 181 rotates around the Y axis (clockwise direction) by θy in the XZ plan view, but the pupil 181 rotates by θx around the X axis in the YZ plan view. In this case, you can do as follows.

  When the pupil 181 rotates by θx around the X axis in YZ plan view, the reflected light of the half mirror 150 enters the eyeball 182 from the pupil 181 by rotating the MEMS mirror 140 by θx around the S axis. You just have to do it. When the MEMS mirror 140 is rotated by θx around the S-axis, the angle is adjusted (corrected) so that the normal line of the reflective surface 140A has an elevation angle (angle θx) with respect to the T-axis.

  As described above, according to the displacement in the X-axis direction and the Y-axis direction of the position of the pupil 181 detected by the position detection unit 171, the MEMS mirror 140 has the angles θx and θy obtained by the equations (1) and (2). If the angle is adjusted (corrected), the reflected light of the half mirror 150 can enter the eyeball 182 from the pupil 181.

As shown in FIG. 7B, the pupil 181 is rotated clockwise by an angle θy in the XZ plan view, and the reflected light of the half mirror 150 enters the eyeball 182 from the pupil 181. At this time, although the optical axis L ₂ of the reflected light of the half mirror 150 is offset from the central axis L ₁ of the pupil 181, the reflected light is incident on the eyeball 182 through the pupil 181. Imaging is obtained.

  FIG. 7B shows a form in which the reflected light of the half mirror 150 enters the eyeball 182 through the outside of the center 181A of the pupil 181. However, when the angle of the MEMS mirror 140 is adjusted (corrected) from the state of FIG. 7A by changing the curved shape of the half mirror 150, the reflected light of the half mirror 150 is reflected at the center 181A or the center 181A of the pupil 181. You may make it pass the vicinity of.

  Further, since the head mounted display device 100 includes the XY stage 120, in addition to the angle adjustment (angle correction) of the MEMS mirror 140 as described above, when the position control unit 172 moves the optical system with the XY stage 120, As shown in FIG. In FIG. 7C, as an example, compared with FIG. 7B, the optical system surrounded by a broken line has moved to the X axis positive direction side.

  The position control unit 172 includes the motor 124X when the displacements in the X-axis direction and the Y-axis direction of the center position 165 in the image 163 (see FIG. 6) of the pupil 181 detected by the position detection unit 171 are X1 and Y1, respectively. Is driven to move the stage 123 in the X-axis direction relative to the movable part 122, and the motor 124Y is driven to move the movable part 122 in the Y-axis direction relative to the base 121.

When the XY stage 120 is moved in this way, the entire optical system moves. For this reason, as shown in FIG. 7B before the movement, the optical axis L ₂ of the reflected light of the half mirror 150 is offset from the central axis L ₁ of the pupil 181, as shown in FIG. as shown, the center axis _{L 1} of the pupil 181, and the optical axis _{L 2} of the reflected light of the half mirror 150 matches. That is, the position of the optical system is corrected by the XY stage 120 according to the displacements X1 and Y1 in the X-axis direction and the Y-axis direction.

  As described above, the head mounted display device 100 adjusts (corrects) the angle of the MEMS mirror 140 according to the displacement of the pupil 181 detected by the position detection unit 171, and adjusts the position of the optical system by the XY stage 120. Adjust (correct).

  Next, processing executed by the control unit 170 will be described with reference to FIG.

  FIG. 8 is a flowchart illustrating processing executed by the control unit 170.

  The controller 170 starts the process when the head-mounted display device 100 is turned on and a drawing mode on the retina is selected (start).

  The controller 170 starts imaging with the camera 161 of the camera module 160 and turns on the illumination device 162 (step S1). The process of step S2 is executed by the position detection unit 171.

  The control unit 170 adjusts the output of the laser arrays 131R, 131G, and 131B based on the image signal input via the cable 170A, and outputs laser light of the color and brightness represented by the image signal (step S2).

  The laser light output from the light source 130 is scanned pixel by pixel by a raster scan method when reflected by the MEMS mirror 140, and further reflected by the half mirror 150 and incident on the pupil of the user. Note that the process of step S2 is executed by the output control unit 173.

  The control unit 170 detects the position of the pupil by performing image processing on the image captured by the camera module 160 (step S3). The process of step S3 is executed by the position detection unit 171.

  The controller 170 determines whether or not the positional deviation of the pupil position detected in step S3 with respect to the reference position is larger than a predetermined threshold (step S4). The reference position is the reference position 164 in the image 163 shown in FIG. The predetermined threshold is 1 mm, for example. When the pupil is moving in an oblique direction with respect to the X axis and the Y axis, it may be determined whether or not the moving amount in the oblique direction is larger than 1 mm.

  Step S <b> 4 is a process for determining that the angle adjustment of the MEMS mirror 140 and the position adjustment of the XY stage 120 are not performed when the moving amount of the pupil is equal to or less than a predetermined threshold. The process of step S4 is provided in order to balance the visibility of the image for the user and the suppression of complication of control of the head mounted display device 100, for example, when the pupil movement frequency is high. Yes.

  If the controller 170 determines that the positional deviation is larger than the predetermined threshold (S4: YES), the angle adjustment values θx and θy of the MEMS mirror 140 and the XY stage based on the position of the pupil detected in step S3. The movement amounts X and Y of 120 are obtained (step S5). The process of step S5 is executed by the position detection unit 171.

  The controller 170 adjusts the angle of the MEMS mirror 140 and drives the XY stage 120 using the adjustment values θx and θy of the angle of the MEMS mirror 140 obtained in step S5 and the movement amounts X and Y of the XY stage 120. (Step S6). Of the processing in step S6, the angle control unit 174 adjusts the angle of the MEMS mirror 140, and the position control unit 172 drives the XY stage 120.

  The controller 170 determines whether or not drawing of the image has been completed (step S7). The process of step S7 is executed by the output control unit 173. Whether or not drawing of the image has ended may be determined by whether or not an image signal is input via the cable 170A. If the image signal includes data indicating the end of drawing, it may be determined based on whether or not the data indicating the end of drawing has been read.

  If the controller 170 determines that the drawing of the image has ended (S7: YES), the controller 170 ends the imaging with the camera 161 of the camera module 160 and turns off the illumination device 162 (step S8). The process of step S8 is executed by the position detection unit 171.

  If controller 170 determines in step S4 that the positional deviation is not greater than the predetermined threshold (S4: NO), the flow returns to step S3. This is to detect the current pupil position without correcting the angle of the MEMS mirror 140 based on the positional deviation and correcting the position of the optical system by the XY stage 120.

  If it is determined in step S8 that image drawing has not ended (S7: NO), the flow returns to step S3. This is to detect the position of the pupil at the current time.

  Thus, a series of processing ends.

  FIG. 9 is a diagram showing a movement range of a pupil capable of retina drawing.

  FIG. 9A shows a pupil capable of retina drawing when the optical axis of the reflected light of the half mirror 150 is adjusted by adjusting the angle of the MEMS mirror 140 and the position of the optical system is adjusted by the XY stage 120. Indicates the range of the position (effective range).

  FIG. 9B shows a range of pupil positions where retina can be drawn when the optical axis adjustment of the reflected light of the half mirror 150 by the MEMS mirror 140 and the optical system position adjustment by the XY stage 120 are not performed. Indicates.

  Here, it is assumed that the pupil diameter is 5 mm, and the movable amounts of the XY stage 120 in the X-axis direction and the Y-axis direction are 10 mm, respectively.

  As shown in FIG. 9A, when the MEMS mirror 140 and the XY stage 120 are used, an image is displayed on the user's retina as long as at least part of the pupil is in a 10 mm diameter circle shown as the effective range. Can be drawn.

  On the other hand, as shown in FIG. 9B, when the MEMS mirror 140 and the XY stage 120 are not used, the user is in a case where at least a part of the pupil is in a circle with a diameter of 5 mm shown as an effective range. An image can be drawn on the retina.

  As can be seen by comparing FIG. 9A and FIG. 9B, the use of the MEMS mirror 140 and the XY stage 120 increases the effective range compared to the case where the MEMS mirror 140 and the XY stage 120 are not used. It can be seen that the magnification is about 4 times.

  Therefore, according to the first embodiment, even if the user changes the direction of the eye and the pupil position changes, laser light is incident on the pupil within the effective range shown in FIG. Thus, it is possible to provide the head mounted display device 100 with good visibility that can be drawn on the retina.

  In the above, in addition to the optical axis adjustment of the reflected light of the half mirror 150 by the MEMS mirror 140, the form of adjusting the position of the optical system by the XY stage 120 has been described. However, the position adjustment of the optical system by the XY stage 120 is performed. It does not have to be done.

  That is, only the optical axis adjustment of the reflected light of the half mirror 150 by the MEMS mirror 140 may be performed in accordance with the movement of the pupil. When only the optical axis adjustment of the reflected light of the half mirror 150 by the MEMS mirror 140 is performed, the effective range is narrower than when the position of the optical system is adjusted by the XY stage 120, but the angle of the laser light incident on the pupil Can be adjusted. For this reason, the distortion of the image drawn on a retina can be suppressed and the head mounted display apparatus 100 with favorable visibility can be provided.

  In the above description, the head mounted display device is connected to the computer via the cable 170A and the image signal input from the computer to the control unit 170 of the head mounted display device 100 via the cable 170A is drawn on the retina. did. However, the head mounted display device 100 and the computer may be connected by wireless communication, and an image signal may be transmitted from the computer to the head mounted display device 100 by wireless communication.

  In the above description, the form in which the position of the user's pupil is detected using the camera module 160 has been described. However, instead of using the camera module 160, the position of the pupil may be detected by measuring the electrooculogram of the user.

  FIG. 10 is a diagram showing a head mounted display device 100A according to a modification of the first embodiment.

  The head mounted display device 100A has a configuration in which the camera module 160 is removed from the head mounted display device 100 shown in FIG. 3, an electrode 116 is added, and the nose pads 113R and 113L shown in FIG. 3 are replaced with the nose pads 113RA and 113LA. .

  The nose pads 113RA and 113LA also function as electrodes and are examples of first electrodes. The electrode 116 is an example of a second electrode.

  The nose pads 113RA and 113LA and the electrode 116 are connected to the control unit 170. The control unit 170 uses the potential difference between the potential detected by the nose pads 113RA and 113LA and the potential detected by the electrode 116 as an ocular potential. To detect. The eyeball is negatively charged on the retinal side and positively charged on the cornea side where the pupil is located. When the nose pads 113RA and 113LA and the electrode 116 are used, it is possible to detect an electrooculogram due to the above-described charge distribution.

  The control unit 170 detects the electrooculogram when the user is moving the pupil with reference to the electrooculogram when the user's pupil is facing the front (facing directly), so that the position of the pupil is detected. Ask for. Others are the same as those of the head mounted display device 100 of the first embodiment.

<Embodiment 2>
FIG. 11 is a diagram illustrating the head mounted display device 200 according to the second embodiment.

  The head mounted display device 200 according to the second embodiment has a configuration in which the control unit 170 of the head mounted display device 100 according to the first embodiment is replaced with a control unit 270. Other configurations are the same as those of the head mounted display device 100 of the first embodiment.

  FIG. 12 is a diagram illustrating a configuration of the control unit 270.

  The control unit 270 includes a position detection unit 171, a position control unit 172, an output control unit 273, and an angle control unit 274. The control unit 270 has a configuration in which the output control unit 173 and the angle control unit 174 of the control unit 170 (see FIG. 5) of Embodiment 1 are replaced with the output control unit 273 and the angle control unit 274, respectively.

When scanning the MEMS mirror 140 for rendering by raster scan, the angle control unit 274 is configured so that the optical path from the half mirror 150 toward the user's eyeball is scanned in a wider range than the user's pupil. The angle of the mirror 140 is adjusted. The optical path directed from the half mirror 150 to the eye of a user, FIG. 7 (B), the an optical path along the optical axis L ₁ in FIG. 7 (C).

  Such scanning of the MEMS mirror 140 is realized, for example, by performing raster scanning at the maximum adjustable angle of the MEMS mirror 140.

  Then, the output control unit 273 causes the light source 130 to emit laser light only when the optical path from the half mirror 150 toward the user's eyeball enters the user's pupil. The output control unit 273 does not cause the light source 130 to emit laser light when the optical path from the half mirror 150 toward the user's eyeball does not enter the user's pupil.

  Further, when the displacement (X, Y) of the pupil position is detected by the position detection unit 171, the output control unit 273 outputs an area in which laser light is output by the amount of the displacement (X, Y) of the pupil. Move in the axial direction and the Y-axis direction.

  When the output control unit 273 and the angle control unit 274 perform such control, the laser beam reflected by the half mirror 150 enters the user's pupil as in the first embodiment.

  FIG. 13 is a diagram illustrating a relationship between a raster scan range and a laser beam output region in the second embodiment.

  As shown in FIG. 13A, raster scanning is performed in a region 220 wider than a region (output region) 210 for outputting laser light.

  The output area 210 is an area in which raster scanning is performed when the user is facing the front (facing) in the first embodiment. The coordinates representing the output area 210 may be set in advance based on data such as experiments and / or statistics as an area to output laser light when the user is facing the front (facing). That's fine.

  When the user's pupil moves, the coordinates of the output area 210 move in the X-axis direction and the Y-axis direction by the displacement (X, Y) of the pupil position detected by the position detector 171.

  The region 220 is a region representing a range in which the MEMS mirror 140 is driven for raster scanning in the second embodiment. Since the size of the region 220 only needs to be larger than that of the output region 210, it is set based on specifications such as the size of the MEMS mirror 140. Since the coordinates of the region 220 are fixed, the output region 210 moves inside the region 220 according to the displacement (X, Y) of the pupil position by the position detection unit 171.

  As shown in FIG. 13A, the angle controller 274 scans from the upper left corner point (1) to the upper right corner point (2) of the region 220, and then from the point (2) to the point ( 1) Move to point (3) one line below, and scan from point (3) to right end point (4). By scanning each row by repeating such an operation, scanning is performed up to the point (5) in the lower right corner of the region 220.

  FIG. 13A shows only the trajectory of the raster scan by the angle control unit 274. When the angle control unit 274 performs the raster scan as shown in FIG. This is performed by the output control unit 273 as shown in FIG.

  When the angle control unit 274 drives the MEMS mirror 140 to perform a raster scan, the output control unit 273 is when the position of the optical path from the half mirror 150 toward the user's eyeball is within the output region 210. Then, the light source 130 is driven to output laser light.

  For this reason, when scanning is performed from the upper left corner point (1) to the upper right corner point (2) of the region 220, the laser is output in the region 211 of the output region 210. An area 211 is an area included in the top row of the output area.

  Further, when scanning is performed from the point (3) to the rightmost point (4), the laser is output in the region 212 of the output region 210. The area 212 is an area included in the second row from the top of the output area.

  By outputting the laser light in this way, the laser light can be output in the entire output region 210 as shown in FIG.

  And since the output control part 273 changes the position which outputs a laser beam according to the displacement (X, Y) of the position of a pupil by the position detection part 171, according to the displacement (X, Y) of the position of a pupil, The position of the output area 210 moves within the area 220.

  As described above, when the raster scan is performed by scanning the MEMS mirror 140, the optical path from the half mirror 150 toward the user's eyeball is scanned in a wider range than the user's pupil. Adjust the angle.

  Then, in accordance with the displacement (X, Y) of the pupil position, laser light is emitted to the light source 130 only when the optical path from the half mirror 150 toward the user's eye enters the user's pupil (through the pupil). By doing so, the laser beam can be made incident on the pupil of the user as in the first embodiment.

  Therefore, according to the second embodiment, even if the user changes the direction of the eyes and the position of the pupil changes, the laser beam can be incident on the pupil and drawn on the retina. A mount display device 200 can be provided.

  Further, as in the second embodiment, the raster scan using the maximum adjustable angle of the MEMS mirror 140 is particularly effective when using a resonance drive type MEMS mirror.

The head mounted display device according to the exemplary embodiment of the present invention has been described above, but the present invention is not limited to the specifically disclosed embodiment, and departs from the scope of the claims. Various modifications and changes are possible.
Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
A retinal projection type head-mounted display device,
A frame placed in front of the user's head;
A light source that emits laser light;
A movable mirror that adjusts the reflection direction of the laser light emitted from the light source;
A fixed mirror that reflects the laser light reflected by the movable mirror toward the pupil of the user;
A detection unit that is attached to the frame, detects a position of a pupil of a user, and outputs a position signal representing the position;
A control unit that adjusts the angle of the movable mirror so that the laser light reflected by the fixed mirror is incident on a user's pupil based on a displacement of the position represented by the position signal with respect to a reference position. Head mounted display device.
(Appendix 2)
Of the displacements, x is the displacement component in the horizontal direction in the user's field of view, y is the displacement component in the vertical direction in the user's field of view, and r is the radius of the user's eyeball.
The head-mounted display device according to claim 1, wherein the control unit adjusts the angle of the movable mirror by θx = arcsin (y / r) in the left-right direction and θy = arcsin (x / r) in the up-down direction. .
(Appendix 3)
A biaxial stage attached to the frame and movable relative to the frame in a biaxial direction in a vertical direction and a horizontal direction in a user's field of view;
The light source, the movable mirror, and the fixed mirror are mounted on the two-axis stage,
The control unit further moves the biaxial stage so that the laser beam reflected by the fixed mirror is incident on the pupil of the user based on a displacement of the position represented by the position signal with respect to a reference position. The head mounted display device according to appendix 1 or 2, wherein the amount is adjusted.
(Appendix 4)
The detection unit is attached to the frame and includes an imaging unit that images a user's pupil, and detects the position of the user's pupil based on an image of a pupil captured by the imaging unit. The head mounted display device according to any one of claims 1 to 3.
(Appendix 5)
The detection unit is attached to the frame, and has a first electrode and a second electrode that measure an electrooculogram in contact with a user's head, and is obtained as a potential difference between the first electrode and the second electrode. The head-mounted display device according to any one of appendices 1 to 3, wherein the position of the user's pupil is detected based on the electrooculogram to be obtained.
(Appendix 6)
The control unit scans the movable mirror in a range wider than the user's pupil, and an optical path from the light source to the user's eyeball through the movable mirror and the fixed mirror determines the position of the user's pupil. The laser beam is emitted from the light source when entering a region to be represented, and the region representing the position of a user's pupil is corrected based on a displacement of the position represented by the position signal with respect to a reference position. The head mounted display apparatus as described in any one of thru | or 3.
(Appendix 7)
The fixed mirror is curved in the shape of an elliptical arc in plan view, and the fixed mirror has two focal positions of the ellipse, the center position of the movable mirror and the center of the user's pupil. The head mounted display device according to any one of appendices 1 to 6, which is arranged so as to coincide with a position.

DESCRIPTION OF SYMBOLS 100 Head mounted display apparatus 110 Frame 112R, 112L Side frame 113R, 113L Nose pad 115R, 115L Lens 120 XY stage 121 Base 122 Movable part 123 Stage 124X, 124Y Motor 130 Light source 131R, 131G, 131B Laser array 140 MEMS mirror 150 Half mirror 160 Camera Module 161 Camera 162 Illumination Device 170 Control Unit 171 Position Detection Unit 172 Position Control Unit 173 Output Control Unit 174 Angle Control Unit 200 Head Mount Display Device 270 Control Unit 273 Output Control Unit 274 Angle Control Unit

Claims (6)

A retinal projection type head-mounted display device,
A frame placed in front of the user's head;
A light source that emits laser light;
A movable mirror that adjusts the reflection direction of the laser light emitted from the light source;
A fixed mirror that reflects the laser light reflected by the movable mirror toward the pupil of the user;
A detection unit that is attached to the frame, detects a position of a pupil of a user, and outputs a position signal representing the position;
A control unit that adjusts the angle of the movable mirror so that the laser light reflected by the fixed mirror is incident on a user's pupil based on a displacement of the position represented by the position signal with respect to a reference position. Head mounted display device. Of the displacements, x is the displacement component in the horizontal direction in the user's field of view, y is the displacement component in the vertical direction in the user's field of view, and r is the radius of the user's eyeball.
2. The head mounted display according to claim 1, wherein the adjustment amount by which the control unit adjusts the angle of the movable mirror is θx = arcsin (y / r) in the left-right direction and θy = arcsin (x / r) in the up-down direction. apparatus. A biaxial stage attached to the frame and movable relative to the frame in a biaxial direction in a vertical direction and a horizontal direction in a user's field of view;
The light source, the movable mirror, and the fixed mirror are mounted on the two-axis stage,
The control unit further moves the biaxial stage so that the laser beam reflected by the fixed mirror is incident on the pupil of the user based on a displacement of the position represented by the position signal with respect to a reference position. The head mounted display apparatus of Claim 1 or 2 which adjusts quantity.   The detection unit is attached to the frame and includes an imaging unit that images a user's pupil, and detects the position of the user's pupil based on an image of a pupil captured by the imaging unit. Item 4. The head mounted display device according to any one of Items 1 to 3.   The detection unit is attached to the frame, and has a first electrode and a second electrode that measure an electrooculogram in contact with a user's head, and is obtained as a potential difference between the first electrode and the second electrode. The head mounted display device according to any one of claims 1 to 3, wherein the position of the pupil of the user is detected based on an electrooculogram to be obtained. The control unit scans the movable mirror in a range wider than the user's pupil, and an optical path from the light source to the user's eyeball through the movable mirror and the fixed mirror determines the position of the user's pupil. The laser beam is emitted from the light source when entering a region to be represented, and the region representing the position of a user's pupil is corrected based on a displacement of the position represented by the position signal with respect to a reference position. The head mounted display apparatus as described in any one of thru | or 3.