JP6039525B2 - Head mounted display, control method therefor, computer program for controlling head mounted display, and recording medium storing computer program - Google Patents

Description

  The present invention relates to a head-mounted display that is mounted on a user's head and displays various information together with the user's field of view, and a control method thereof.

In recent years, augmented reality technology (AR technology) has been known as a technology for superimposing and displaying additional information (hereinafter also referred to as annotation) created by a computer on a real image through a user's field of view. Yes.
As a technique related to the above, Patent Document 1 discloses a technique for superimposing and displaying an annotation on an image captured by a mobile terminal using augmented reality technology. Specifically, the map used for the augmented reality service is preliminarily separated by a grid size suitable for the moving speed, and the annotation in the cell surrounded by the grid is adjusted, so that the annotation is given to the portable terminal according to the moving speed of the portable terminal. A technology capable of varying the amount of information is disclosed.
On the other hand, an apparatus for determining a mobile terminal user's moving means is disclosed in Patent Document 2.
See also Patent Document 3 or 4 disclosing the prior art related to the present invention.

JP 2011-123807 A JP 2010-014592 A JP 2005-303842 A Special table 2013-008232 gazette

If the AR technology is used, an annotation can be superimposed on an object appearing in the user's field of view. However, if the object is not in the field of view, such as when the user's field of view is at hand, the object corresponds to the object. Annotations are not displayed. However, when a user is reading a book or talking in the car, the station name or scenic spot may be missed.
In view of such a situation, the present inventor displays an annotation related to an object not only when the user is looking at the outside scene but also when looking at the hand, thereby creating a trigger for the user's awareness. We have intensively studied to make it convenient for users. As a result, when a user wearing a head-mounted display (hereinafter also referred to as an HMD) is moving, an image obtained by an imaging unit provided in the HMD is processed to select a mode related to the user's field of view. The inventors have conceived that the problem can be solved by controlling the annotation to be displayed on the display screen of the HMD according to the selected mode.

The present invention has been made in view of the above-mentioned problems, and the first aspect is defined as follows. That is,
A head mounted display,
A movement determination unit that determines whether or not the user of the head mounted display is moving;
An imaging unit for imaging an area in the user's line-of-sight direction;
When the movement determining unit determines that the user is moving, by processing an image captured by the imaging unit, a vehicle window mode indicating that the user's field of view is in a vehicle window landscape, or A user mode selection unit that selects any one of the in-vehicle modes indicating that the user's field of view is in the vehicle;
In accordance with the selected user mode, a display control unit that controls display on the display screen of the head mounted display;
A head-mounted display.

According to the head-mounted display of the first aspect defined as described above, when it is determined that the user wearing the HMD is moving, the image obtained by the imaging unit provided in the HMD is processed. It is selected whether the user mode related to the visibility is the vehicle window mode or the vehicle interior mode, and the annotation is controlled to be displayed on the display screen of the HMD according to the selected mode.
Here, when the vehicle window mode is selected as the user mode, the display control unit displays an annotation on the display screen in a normal display, and when the vehicle interior mode is selected, based on the position of the user. It is possible to control the annotation selected according to the predetermined rule to be displayed in a list display (second aspect).
According to the head-mounted display of the second aspect defined in this way, even when there is no object in the user's field of view, annotations selected according to a predetermined rule are displayed in a list as appropriate, which is a convenience for the user to notice. It is.
The annotation is an annotation for an object such as a facility, a building, or a scenic spot, and can include, for example, an object name, an advertisement, a feature, a user posting, and the like.
The list display can be performed by displaying the annotation on an outer edge portion of the display screen (seventh aspect). If the list is displayed in this way, it is difficult to disturb the user's field of view.

The third aspect of the present invention is defined as follows. That is,
In the head mounted display defined in the second aspect, a moving speed specifying unit that specifies the moving speed of the user,
A first selection unit that selects a first annotation to be displayed based on the identified moving speed,
The display control unit controls to display the first annotation.
According to the head mounted display of the 3rd situation prescribed | regulated in this way, the annotation which should be displayed can be selected based on the moving speed of a HMD user. Considering the type of annotation and the number of annotations suitable for the moving speed, it is more convenient for the user. For example, if the number of annotations within a certain range is the same, more annotations are displayed as the moving speed increases, which may be troublesome for the user. Further, the difference in moving speed is an indicator of the user's moving purpose. For example, the purpose of commuting or shopping can be inferred when the moving speed is slow, and the purpose of sightseeing can be inferred when the moving speed is fast. For example, the type of annotation to be displayed can be selected based on the difference in the purpose of movement. It is because it is cumbersome for the user to display the advertisement of the peripheral supermarket for the purpose of travel.

The fourth aspect of the present invention is defined as follows:
In the head mounted display defined in the second aspect, a moving means specifying unit for specifying the moving means of the user,
A second selection unit that selects a second annotation to be displayed based on the identified moving means,
The display control unit controls to display the second annotation.
According to the head mounted display of the 4th situation prescribed | regulated in this way, the annotation which should be displayed can be selected based on a moving means of a HMD user. Considering the types of annotations and the number of annotations suitable for the moving means will be more convenient for the user. As in the case of the moving speed, it is presumed that the moving purpose is different if the moving means is different. Moreover, even if the moving speed is the same, there are cases where the information desired to be obtained differs between the user in the train and the user in the car. For example, it is convenient for a user on a train to display an annotation about a passing station.

The second selection unit selects the second annotation in consideration of information on the position of the user, the traveling direction, and the position of the track when the identified moving means is a vehicle traveling on the track. (Fifth aspect).
If the vehicle is traveling on a track, the travel path is defined and predicted based on the user's position, traveling direction, and track position information, and is therefore more suitable for users traveling with the vehicle, such as station names and scenic spots. Annotations can be selected.
The second selection unit selects the second annotation in consideration of information on the position of the user, the traveling direction, and the position of the annotation when the identified moving means is a vehicle traveling on a road. (Sixth aspect).
In the case of a vehicle traveling on a road, it is possible to select an annotation suitable for the user moving with the vehicle based on the user's position, traveling direction, and annotation position information.

The eighth aspect of the present invention is defined as follows. That is,
A method for controlling a head mounted display,
A movement determining step for determining whether or not the user of the head mounted display is moving; and
An imaging step in which an imaging unit images an area in the user's line-of-sight direction;
When the user mode selection unit determines that the user is moving in the movement determination step, the user mode selection unit processes the image captured in the imaging step to indicate that the user's field of view is in a car window landscape A user mode selection step of selecting either a vehicle window mode or a user mode of the vehicle interior mode indicating that the user's field of view is in the vehicle;
A display control step for controlling display on the display screen of the head mounted display in accordance with the selected user mode;
A control method comprising:
According to the invention of the eighth aspect defined as described above, the same effect as that of the first aspect can be obtained.

The ninth aspect of the present invention is defined as follows. That is,
In the method defined in the eighth aspect, in the display control step, when the vehicle window mode is selected as the user mode, the annotation is displayed in a normal display on the display screen, and when the vehicle interior mode is selected, Control is performed so that annotations selected according to a predetermined rule based on the position of the user are displayed in a list display.
According to the ninth aspect of the invention thus defined, the same effects as those of the second aspect can be achieved.

The tenth aspect of the present invention is defined as follows. That is,
In the method defined in the ninth aspect, a moving speed specifying unit that specifies a moving speed of the user,
A first selection unit comprising: a first selection step of selecting a first annotation to be displayed based on the identified moving speed;
In the display control step, control is performed to display the first annotation.
According to the tenth aspect of the invention thus defined, the same effect as the third aspect can be obtained.

The eleventh aspect of the present invention is defined as follows. That is,
In the method defined in the ninth aspect, a moving means specifying unit that specifies the moving means of the user,
A second selection unit comprising: a second selection step of selecting a second annotation to be displayed based on the identified moving means;
The display control step controls to display the second annotation.
According to the eleventh aspect of the invention thus defined, the same effects as in the fourth aspect can be achieved.

The twelfth aspect of the present invention is defined as follows. That is,
In the method defined in the eleventh aspect, in the second selection step, when the identified moving means is a vehicle traveling on a track, information on the user's position, traveling direction, and track position is considered. Then, the second annotation is selected.
According to the twelfth aspect of the invention thus defined, the same effects as in the fifth aspect can be achieved.

The thirteenth aspect of the present invention is defined as follows. That is,
In the method defined in the eleventh aspect, in the second selection step, when the identified moving means is a vehicle traveling on a road, information on the position of the user, the traveling direction, and the position of the annotation is considered. Then, the second annotation is selected.
According to the thirteenth aspect of the invention thus defined, the same effects as in the sixth aspect are achieved.

The fourteenth aspect of the present invention is defined as follows. That is,
In the method defined in the ninth aspect, the list display is performed by displaying the annotation on an outer edge portion of the display screen.
According to the fourteenth aspect of the invention thus defined, the same effects as in the seventh aspect can be achieved.

Furthermore, the fifteenth aspect of the present invention is defined as follows. That is,
In a computer program for controlling a head mounted display, a computer,
Movement determination means for determining whether or not the user of the head mounted display is moving;
Imaging means for imaging an area in the user's line-of-sight direction;
When the movement determining means determines that the user is moving, by processing the image captured by the imaging means, a vehicle window mode indicating that the user's field of view is in a vehicle window landscape, or User mode selection means for selecting any user mode of the in-vehicle mode indicating that the user's field of view is in the vehicle;
Display control means for controlling display on the display screen of the head-mounted display according to the selected user mode;
As a computer program.
According to the fifteenth aspect of the invention thus defined, the same effects as those of the first aspect can be achieved.

The sixteenth aspect of the present invention is defined as follows. That is,
In the computer program defined in the fifteenth aspect, when the vehicle window mode is selected as the user mode, the display control means displays an annotation in a normal display on the display screen, and the vehicle interior mode is selected. The annotation selected according to a predetermined rule based on the position of the user is controlled to be displayed in a list display.
According to the sixteenth aspect of the invention thus defined, the same effects as those of the second aspect can be achieved.

The seventeenth aspect of the present invention is defined as follows. That is,
In the computer program defined in the sixteenth aspect, the computer is further
A moving speed specifying means for specifying the moving speed of the user;
Functioning as a first selection means for selecting a first annotation to be displayed based on the identified moving speed;
The display control means controls to display the first annotation.
According to the seventeenth aspect of the invention thus defined, the same effects as in the third aspect can be achieved.

The eighteenth aspect of the present invention is defined as follows. That is,
In the computer program defined in the sixteenth aspect, the computer is further
A moving means specifying means for specifying the moving means of the user;
Based on the identified moving means, functioning as second selecting means for selecting a second annotation to be displayed;
The display control means controls to display the second annotation.
According to the eighteenth aspect of the invention thus defined, the same effects as in the fourth aspect can be achieved.

The nineteenth aspect of the present invention is defined as follows. That is,
In the computer program stipulated in the eighteenth aspect, when the specified moving means is a vehicle traveling on a track, the second selecting means provides information on the user's position, traveling direction, and track position. In consideration, the second annotation is selected.
According to the nineteenth aspect of the invention thus defined, the same effects as in the fifth aspect are achieved.

The twentieth aspect of the present invention is defined as follows. That is,
In the computer program stipulated in the eighteenth aspect, when the identified moving means is a vehicle traveling on a road, the second selecting means provides information on the position of the user, the traveling direction, and the position of the annotation. In consideration, the second annotation is selected.
According to the twentieth aspect of the invention thus defined, the same effects as in the sixth aspect are achieved.

The 21st aspect of this invention is prescribed | regulated as follows. That is,
In the computer program defined in the sixteenth aspect, the list display is performed by displaying the annotation on an outer edge portion of the display screen.
According to the twenty-first aspect of the invention thus defined, the same effects as in the seventh aspect are achieved.

  A recording medium for recording the computer program defined in any one of the fifteenth to twenty-first aspects is defined as the twenty-second aspect.

FIG. 1 is a block diagram showing the configuration of the HMD 1 according to the embodiment of the present invention. 2A is an example of a list display on the display screen in the in-vehicle mode, FIG. 2B is another example of the list display on the display screen in the in-vehicle mode, and FIG. 2C is a normal display on the display screen in the vehicle window mode. It is a schematic diagram which shows an example. FIG. 3 is a flowchart illustrating an example of the operation of the HMD 1 according to the embodiment of this invention. FIG. 4 is a block diagram showing the configuration of the HMD 21 according to another embodiment of the present invention. FIG. 5 is a block diagram showing the configuration of the HMD 31 according to another embodiment of the present invention. FIG. 6 is a block diagram illustrating a configuration of the HMD 41 according to the embodiment of this invention.

A head mounted display according to an embodiment of the present invention will be described.
FIG. 1 shows a schematic configuration of the HMD 1.
As shown in FIG. 1, the HMD 1 includes a map database 3, a movement determination unit 5, an imaging unit 7, a user mode selection unit 9, a user position specifying unit 11, and a display control unit 13.

  The map database 3 stores various map data related to objects to be drawn on the display screen of the HMD 1. Examples of the object include facilities, stores, parks, stations, buildings such as castles and towers, scenic spots such as mountains, and events such as fireworks displays. The map data stored in the map database 3 includes the name and position information of the object. In addition, the genre and other information related to the object may be stored in association with the object. Examples of the genre related to the object include restaurants, stations, and scenic spots. In addition, the various information regarding the object includes an advertisement or a coupon when the object is a store “ABC super”, and an elevation or a world cultural heritage when the object is a mountain “Mt. Fuji”. . An annotation is displayed on the display screen of the HMD 1 based on the information stored in the map database 3.

  The movement determination unit 5 determines whether the user wearing the HMD 1 is moving by transportation. Although the determination method is not particularly limited, it can be determined whether or not the user is moving based on position information specified by a user position specifying unit described later. For example, the movement determination unit 5 compares the user position specified by the user position specifying unit with a predetermined time period and the user position specified last time, and when the interval between both positions is equal to or greater than a predetermined threshold, the user is moving It is determined that In addition, referring to a movement speed specified by a movement speed specifying unit described later, it may be determined that the user is moving when the movement speed is equal to or greater than a predetermined threshold. Examples of the transportation include trains, buses, and automobiles.

  The imaging unit 7 is provided in the HMD 1 so as to be able to capture the line-of-sight direction of the user wearing the HMD 1. The imaging unit 7 thus provided captures an outside scene that spreads in the direction of the user's line of sight, and acquires the captured outside scene image. The imaging unit 7 is preferably capable of imaging at an interval of 10 sheets / second or more, for example. This is because the accuracy when selecting whether the user's field of view is in the car window landscape or in the vehicle can be improved in the user mode selection unit described later. An example of the imaging unit 7 is a camera equipped with HMD1.

When the user mode selection unit 9 determines that the user is moving based on the determination of the movement determination unit 5, the user mode selection unit 9 processes the image captured by the imaging unit 7 so that the field of view of the user is Either the vehicle window mode indicating that the vehicle is in a landscape or the vehicle mode indicating that the user's field of view is in the vehicle is selected. As an example of the user mode selection method using the image processing, the following method can be cited. First, a first frame image and a second frame image are extracted from the frame images captured by the imaging unit 7. Both frame images may be continuous in time, or there may be a time interval between both frame images. Three or more frame images may be extracted. Next, an arbitrary feature point is identified in the first frame image, and the position of the identified feature point in the first frame image is compared with the position of the identified feature point in the second frame image . As a result of the comparison, when the position of the feature point moves a predetermined distance in a certain direction, it is determined that the user's field of view is in the car window landscape, and the car window mode is selected. On the other hand, when the movement of the position of the feature point is less than the predetermined distance, it is determined that the user's field of view is in the vehicle, and the in-vehicle mode is selected. The user mode can be selected with higher accuracy as the number of identified feature points increases.
The user position specifying unit 11 specifies the current position of the HMD 1. The user position specifying unit 11 can specify the current position information by using, for example, a GPS device. The user position specifying unit 11 may also specify the current time.

  The display control unit 13 controls display when the map data in the map database 3 is displayed as an annotation on the HMD1 display screen based on the user mode selected by the user mode selection unit 9. When the in-vehicle mode is selected by the user mode selection unit 9, the display control unit 13, for example, in the predetermined display target area from the map database 3 based on the user position specified by the user position specifying unit 11. An object having position information is selected, and map data such as the name of the object is displayed as a list on the display screen as an annotation. The display target area can be a constant area such as a 5 km area including the user position. The user may specify in advance the type of annotation to be displayed on the display screen. For example, an annotation related to a specific genre associated with the object can be displayed. As a specific example, when the user designates in advance to display an annotation relating to the genre “scenic spot”, the display control unit 13 displays an annotation relating to a scenic spot in the display target area. In this case, the display target area is not limited to a certain area, and the area can be sequentially enlarged so that the annotation of the genre “scenic spot” can be displayed. These annotations can be displayed on the outer edge of the display screen so as to effectively call attention so as not to disturb the user's field of view (see FIG. 2A). Further, when a list of annotations is displayed on the outer edge of the display screen, the annotations can be displayed at a position corresponding to the position of the object. For example, an annotation corresponding to the object can be displayed on the left outer edge of the display screen when the object is positioned on the left side of the user, and on the right outer edge when the object is positioned on the right side of the user (FIG. 2 ( B)). On the other hand, when the vehicle window mode is selected as the user mode, the display control unit 13 can normally display the annotation by a normal method. As the normal display method, for example, an annotation corresponding to an object shown in the display screen can be superimposed and displayed in a format such as a balloon on an actual field of view visible through a transparent (transparent) display screen provided in the HMD 1. (See FIG. 2C).

An example of the operation of the HMD 1 shown in FIG. 1 will be described with reference to FIG.
In step 1, the user position specifying unit 11 specifies the current position of the user wearing the HMD 1 based on a GPS device or the like provided in the HMD 1.
In step 2, the movement determination unit 5 determines whether or not the HMD user in step 1 is moving. This determination is made based on whether or not the current position specified at a predetermined time period in step 1 is sequentially compared and the interval between both positions is equal to or greater than a predetermined threshold. That is, when the distance between the two positions is equal to or greater than the predetermined threshold, it is determined that the movement is in progress (step 3: Yes), and the process proceeds to step 5.
In step 5, the imaging unit 7 captures the visual line direction of the HMD user.

In step 7, the user mode selection unit 9 performs image processing on the captured image captured in step 5 by the above-described method, and is in the vehicle window mode indicating that the field of view of the HMD user is in the vehicle window landscape or in the vehicle. To select one of the in-vehicle modes. When the in-vehicle mode is selected (step 9: Yes), the process proceeds to step 11.
In step 11, the display control unit 13 forms a predetermined display target area based on the current position specified in step 1. For example, an area formed by a circle with a radius of 1 km around the current position can be formed as a display target area.
In step 13, the display control unit 13 selects an annotation having position information in the display target area formed in step 11 from the map database 3.
In step 15, the display control unit 13 displays a list of the annotations selected in step 13 on the outer edge of the display screen of the HMD 1.

On the other hand, if No in step 9, the process proceeds to step 17.
In step 17, a user orientation specifying unit (not shown) specifies the orientation in which the line of sight of the HMD user in step 1 is facing.
In step 19, the display control unit 13 selects, from the map database 3, an annotation related to the object that appears in the field of view of the HMD user, based on the current position specified in step 1 and the orientation specified in step 17.
In step 20, the display control unit 13 superimposes and displays the annotation selected in step 19 on an object appearing in the actual field of view of the HMD user.

FIG. 4 shows an HMD 21 according to another embodiment. 4, the same elements as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is partially omitted.
FIG. 4 shows an HMD 21 capable of controlling display of an annotation suitable for the moving speed in consideration of the moving speed of the user. That is, the HMD 21 further includes a movement speed specifying unit 23 and a first selection unit 25 in the HMD 1 illustrated in FIG. 1, and includes a display control unit 27 instead of the display control unit 13.
The moving speed specifying unit 23 specifies the moving speed of the user. The movement speed can be specified based on, for example, acceleration information from an acceleration sensor provided in the HMD 1. The moving speed specifying unit 23 may specify a value of the moving speed of the user, or may specify a speed stage such as five stages allocated according to the value of the moving speed.

The first selection unit 25 selects a first annotation to be displayed on the display screen from the map database 3 based on the identified position and moving speed of the user. As the selection method, for example, the display target area based on the user position is determined in advance corresponding to the moving speed, and position information is stored in the determined display target area according to the specified moving speed of the user. An annotation having “” can be selected. As another example, a genre of an object to be displayed may be determined in advance corresponding to the moving speed, and an annotation related to the determined genre may be selected according to the specified moving speed of the user. .
As an example of the former, the relationship between the moving speed and the display target area is previously set to “less than 50 km / hour—radius 200 m area centered on the user position”, “50-100 km / hour—radius 200 m-1 km area”, “100-200 km. / Hour-radius 1 km to 2 km area "," 200 km / hour or more-radius 2 km or more area ", when the specified moving speed is 220 km / hour, An annotation having position information in an area excluding an area having a radius of less than 2 km from the position is selected as the first annotation.
As an example of the latter, the relationship between the moving speed and the genre of the display target object is previously set to “less than 50 km / hour—facility, store, park”, “50 to 200 km / hour—station, building, scenic spot”, “200 km / If it is determined that the above-mentioned moving speed is 150 km / hour when it is defined as “over time-station, scenic spot”, the object has position information in a predetermined display target area, and the genre of the object is An annotation “station, building, scenic spot” is selected as the first annotation. The first selection unit 25 may select the first annotation by combining the above two examples.

In addition to the above, “less than 50 km / hour—first layer”, “50 to 200 km / hour—second layer”, “200 km / hour or more—third layer”, etc. Another layer can be provided. In this case, annotations relating to facilities, stores, etc. are associated with layers corresponding to slow movement speeds, and annotations relating to stations, buildings, scenic spots, etc. are associated with layers corresponding to high movement speeds. And the 1st selection part 25 selects the annotation located in the display object area | region based on a user position as a 1st annotation from the annotation linked | related with the layer corresponding to the said moving speed specified above. And can.
The display control unit 27 refers to the user mode selection unit 9 and the first selection unit 25 and controls the display of the annotation on the display screen. Specifically, when the in-vehicle mode is selected as the user mode, the display control unit 27 displays a list of the first annotation selected by the first selection unit 25 on the display screen.

FIG. 5 shows an HMD 31 according to another embodiment. In FIG. 5, the same elements as those in FIGS. 1 and 4 are denoted by the same reference numerals, and description thereof is partially omitted.
FIG. 5 shows an HMD 31 capable of controlling display of an annotation suitable for the moving means in consideration of the moving means of the user. That is, the HMD 31 further includes a traveling direction specifying unit 35, a moving means specifying unit 37, and a second selecting unit 39 in the HMD 1 shown in FIG. 1, and instead of the map database 3 and the display control unit 13, respectively. 33 and a display control unit 40.

In addition to the map data stored in the map database 3, the map database 33 stores information on roads and tracks. Examples of the information related to the road include the type of road and intersection, the type of road such as a highway, a general road, and the like. Examples of the information related to the track include the position of the track, the name and position of the station, and the like. Further, information on scenic spots that are easy to view may be stored in association with the position on the track or the position of the station.
The traveling direction specifying unit 35 specifies the traveling direction of the user wearing the HMD 31. Although the said specific method is not specifically limited, Based on the positional information which the said user position specific | specification part 11 specified, it is possible to specify the advancing direction of this user. For example, the advancing direction specifying unit 35 can specify the advancing direction by comparing the user position specified by the user position specifying unit 11 with the user position specified last time in a predetermined time period.

  The moving means specifying unit 37 specifies the moving means of the user. Although it does not specifically limit as the said specific method, It can be specified based on at least one of movement speed, the position of a road, the position of a track, and other information. For example, when the user position corresponds to a road position, the moving means is specified as a vehicle traveling on a road such as a bus or an automobile, and when the user position corresponds to a track position, Identified as a vehicle traveling above. Further, based on the moving speed, it is possible to specify whether the bus is traveling on a highway or a bus traveling on a general road according to the average speed in a steady state during movement. The above train can also be identified as a Shinkansen or a conventional line. Other information includes a movement and stationary pattern, a stationary position, and the like.

The second selection unit 39 selects a second annotation to be displayed on the display screen from the map database 33 based on the identified user position, traveling direction, and moving means. As the annotation selection method based on the specified user position and moving means, the selection method of the first selection unit 25 can be referred to. Specifically, the display target area based on the user position is determined in advance corresponding to the moving means, and an annotation having position information is added to the determined display target area according to the specified user moving means. You can choose. As another example, the genre of the object to be displayed may be determined in advance corresponding to the moving means, and the annotation related to the determined genre may be selected according to the specified moving means of the user. . In addition, various layers are provided in advance corresponding to the moving means, and annotations located in the display target area based on the user position are selected from the annotations associated with the layer corresponding to the identified user moving means. You can choose. The second selection unit 39 further selects, as the second annotation, an annotation that is located on the side of the traveling direction from the user position among the selected annotations based on the identified traveling direction. Moreover, in the said movement means specific | specification part 37, when the movement means is specified as the vehicle which drive | works on a track, with reference to the position of the track in the map database 33, the name of the station to pass, It is possible to select an annotation that is more suitable for the user moving with the vehicle, such as the position of the track, the scenic spot associated with the station name, and the like.
The display control unit 40 refers to the user mode selection unit 9 and the second selection unit 39 to control the display of annotations on the display screen. Specifically, when the in-vehicle mode is selected as the user mode, the display control unit 40 displays a list of the second annotation selected by the second selection unit 39 on the display screen.

FIG. 6 shows an HMD 41 of the embodiment. In FIG. 6, the same elements as those in FIGS. 1, 4, and 5 are denoted by the same reference numerals, and description thereof is partially omitted.
FIG. 6 shows an HMD 41 that can reduce the processing load related to the annotation display by providing the layers in advance according to the moving speed. That is, the HMD 41 further includes a control unit 141, a memory unit 142, an input unit 143, an output unit 144, an interface unit 145, a user orientation specifying unit 146, and a speed-specific layer storage unit 148 in the HMD 21 shown in FIG.

The control unit 141 is a computer device including a CPU, a buffer memory, and other devices, and controls other elements constituting the HMD 41. The control unit 141 includes a display control unit 27.
The display control unit 27 controls to display the first annotation selected by the first selection unit 25 among the map data stored in the map database 3 on the display screen.
A computer program is stored in the memory unit 142, and this computer program is read into the control unit 141, which is a computer device, and causes it to function. This computer program can be stored in a general-purpose medium such as a built-in hard disk or built-in memory as a storage device of the HMD, or a storage medium replaceable with the HMD.
The input unit 143 is used for inputting a user command, for example. Specifically, it is used for inputting the genre of the annotation desired to be displayed on the display screen. As the input unit 143, a voice input device such as a button, a touch sensor, or a microphone provided in the HMD main body can be used.

The output unit 144 includes a display screen, and displays an input screen, annotations based on the control of the display control unit 27, and other information. The output unit 144 can also include a voice transmission unit.
The interface unit 145 connects the HMD 41 to a wireless network or the like.
The user azimuth specifying unit 146 is provided in the HMD 41, and specifies azimuth information (azimuth information) in which the user's line of sight is directed by detecting geomagnetism.
The speed-specific layer storage unit 148 stores the speed-specific layer used in the first selection unit 25 described above. That is, “less than 50 km / hour—first layer”, “50 to 200 km / hour—second layer”, “200 km / hour or more—third layer”, etc. are provided in advance corresponding to the moving speed. , Annotations are arbitrarily associated with each layer. For example, the first layer corresponding to a slow moving speed is a facility, a store, etc., the second layer is a station, a building, a scenic spot, etc. The third layer corresponding to a fast moving speed is a station, a scenic spot, etc. Etc. are linked together with position information.
It is good also as a display control system which makes a part of component of the said HMD41 executable by a server apparatus. The configuration executable by the server device is not particularly limited. For example, the map database 3, the speed-specific layer storage unit 148, and the first selection unit 25 can be used.

  Although the embodiments and examples of the present invention have been described above, two or more of the embodiments (examples) may be combined and implemented. Alternatively, one of these embodiments (examples) may be partially implemented. Furthermore, among these, two or more embodiments (examples) may be partially combined.

  The present invention is not limited to the description of the embodiments and examples of the invention described above. Various modifications may be included in the present invention as long as those skilled in the art can easily conceive without departing from the description of the scope of claims.

1 21 31 41 Head Mount Display (HMD)
3 33 Map database (Map DB)
DESCRIPTION OF SYMBOLS 5 Movement determination part 7 Imaging part 9 User mode selection part 11 User position specific | specification part 13 27 40 Display control part 23 Movement speed specific | specification part 25 1st selection part 35 Travel direction specific | specification part 37 Movement means specific | specification part 39 2nd selection part

Claims (22)

A head mounted display,
A movement determination unit that determines whether or not the user of the head mounted display is moving;
An imaging unit for imaging an area in the user's line-of-sight direction;
When the movement determining unit determines that the user is moving, by processing an image captured by the imaging unit, a vehicle window mode indicating that the user's field of view is in a vehicle window landscape, or A user mode selection unit that selects any one of the in-vehicle modes indicating that the user's field of view is in the vehicle;
In accordance with the selected user mode, a display control unit that controls display on the display screen of the head mounted display;
A head-mounted display. When the vehicle window mode is selected as the user mode, the display control unit displays the annotation in a normal display on the display screen, and when the vehicle interior mode is selected, a predetermined rule is based on the position of the user. Control to display the annotation selected according to
The head mounted display according to claim 1. A moving speed specifying unit for specifying the moving speed of the user;
A first selection unit that selects a first annotation to be displayed based on the identified moving speed,
The display control unit controls to display the first annotation;
The head mounted display according to claim 2. A moving means specifying unit for specifying the moving means of the user;
A second selection unit that selects a second annotation to be displayed based on the identified moving means,
The display control unit controls to display the second annotation;
The head mounted display according to claim 2. The second selection unit selects the second annotation in consideration of information on the position of the user, the traveling direction, and the position of the track when the identified moving means is a vehicle traveling on the track. To
The head mounted display according to claim 4. The second selection unit selects the second annotation in consideration of information on the position of the user, the traveling direction, and the position of the annotation when the identified moving means is a vehicle traveling on a road. To
The head mounted display according to claim 4. The list display is performed by displaying the annotation on an outer edge of the display screen.
The head mounted display according to claim 2. A method for controlling a head mounted display,
A movement determining step for determining whether or not the user of the head mounted display is moving; and
An imaging step in which an imaging unit images an area in the user's line-of-sight direction;
When the user mode selection unit determines that the user is moving in the movement determination step, the user mode selection unit processes the image captured in the imaging step to indicate that the user's field of view is in a car window landscape A user mode selection step of selecting either a vehicle window mode or a user mode of the vehicle interior mode indicating that the user's field of view is in the vehicle;
A display control step for controlling display on the display screen of the head mounted display in accordance with the selected user mode;
A control method comprising: In the display control step, when the vehicle window mode is selected as the user mode, the annotation is displayed in a normal display on the display screen, and when the in-vehicle mode is selected, a predetermined rule based on the position of the user is displayed. Control to display the annotation selected according to
The control method according to claim 8. A moving speed specifying unit for specifying a moving speed of the user;
A first selection unit comprising: a first selection step of selecting a first annotation to be displayed based on the identified moving speed;
In the display control step, control is performed to display the first annotation.
The control method according to claim 9. A moving means specifying step in which a moving means specifying unit specifies the moving means of the user;
A second selection unit comprising: a second selection step of selecting a second annotation to be displayed based on the identified moving means;
The display control step controls to display the second annotation;
The control method according to claim 9. In the second selection step, when the identified moving means is a vehicle traveling on a track, the second annotation is selected in consideration of information on the user's position, traveling direction, and track position. To
The control method according to claim 11. In the second selection step, when the identified moving means is a vehicle traveling on a road, the second annotation is selected in consideration of information on the position of the user, the traveling direction, and the position of the annotation. To
The control method according to claim 11. The list display is performed by displaying the annotation on an outer edge of the display screen.
The control method according to claim 9. In a computer program for controlling a head mounted display, a computer,
Movement determination means for determining whether or not the user of the head mounted display is moving;
Imaging means for imaging an area in the user's line-of-sight direction;
When the movement determining means determines that the user is moving, by processing the image captured by the imaging means, a vehicle window mode indicating that the user's field of view is in a vehicle window landscape, or User mode selection means for selecting any user mode of the in-vehicle mode indicating that the user's field of view is in the vehicle;
Display control means for controlling display on the display screen of the head-mounted display according to the selected user mode;
As a computer program. When the vehicle window mode is selected as the user mode, the display control means displays an annotation in a normal display on the display screen, and when the vehicle interior mode is selected, a predetermined rule based on the position of the user is displayed. Control to display the annotation selected according to
The computer program according to claim 15. Said computer further
A moving speed specifying means for specifying the moving speed of the user;
Functioning as a first selection means for selecting a first annotation to be displayed based on the identified moving speed;
The display control means controls to display the first annotation;
The computer program according to claim 16. Said computer further
A moving means specifying means for specifying the moving means of the user;
Based on the identified moving means, functioning as second selecting means for selecting a second annotation to be displayed;
The display control means controls to display the second annotation;
The computer program according to claim 16. The second selection means selects the second annotation in consideration of information on the position of the user, the traveling direction, and the position of the track when the identified moving means is a vehicle traveling on the track. To
The computer program according to claim 18. The second selection means selects the second annotation in consideration of information on the position of the user, the traveling direction, and the position of the annotation when the identified moving means is a vehicle traveling on a road. To
The computer program according to claim 18. The list display is performed by displaying the annotation on an outer edge of the display screen.
The computer program according to claim 16.   The recording medium which records the computer program as described in any one of Claims 15-21.

Images