JP5971466B2 - Flight path display system, method and program - Google Patents

Description

  The present invention relates to the display and creation of flight paths.

  Recently, in the event of a disaster, the importance of smoothing the subsequent response has increased by dispatching airplanes etc. to the area where the disaster has occurred and monitoring or reconnaissance of the current state of the area from the sky. Yes. When a large-scale disaster occurs, it is often difficult to approach the target area from the ground. In such cases, the target area is monitored and reconnaissance from the sky using an airplane, especially an unmanned aircraft. Is effective.

  Generally, when a disaster or the like occurs in a certain area, a front command station is provided in or near that area, and this command station works in cooperation with a base or support station provided behind. Conventionally, when operating an aircraft, the information collected by the command post is notified to the base, and the base creates a flight path based on the notified information.

  The command post can obtain more fresh information about the area of interest. Further, based on the obtained information, the command post may determine that the area or point to be monitored should be added or changed suddenly. For this reason, it is desirable to create a flight path at the command post rather than at the base.

  Conventionally, in the display of the flight path performed behind the base or the like, the flight path when viewed from a predetermined viewpoint and line-of-sight direction is displayed using a display device such as an LCD that is leaned on a desk or the like. It was. At that time, prior to the display of the flight path, the user inputs a desired viewpoint and line-of-sight direction via an input device such as a keyboard or a mouse.

  In order to create a flight path, it is necessary to consider the flight path from various viewpoints and line-of-sight directions. For this purpose, it is necessary to frequently input the viewpoint and the line-of-sight direction. However, if such an input operation is performed with a keyboard or the like, a complicated operation is required for the user, and the entire flight path is diversified. It was an obstacle to understand intuitively.

  As a technique related to the present invention, Patent Document 1 describes a flight profile display system.

JP 2002-293299 A

  The present invention has been made in view of such a situation, and a problem to be solved by the present invention is to display a flight path that allows the user to visually recognize the entire flight path from a desired viewpoint and line-of-sight direction through an intuitive operation. To provide a system, method and program.

  In order to solve the above-described problems, the present invention has, as one aspect thereof, a head-mounted display device (HMD) that a user wears on the head and a target area that is a space including terrain is determined in advance. Reduced area presenting means for presenting to the user a reduced area that is a space that has been reduced at a reduced scale and reproduced at a predetermined installation location, and an HMD that measures the position and orientation of the HMD relative to the reduced area Positioning means, reduced flight path generation means for obtaining a reduced flight path that is a space curve in the reduced area corresponding to the flight path of the flying object in the target area, the position of the HMD obtained by the HMD positioning means, and Based on the direction and the reduced flight path obtained by the reduced flight path generation means, when the reduced area is viewed from the position and orientation It generates an image indicating the field, to provide a flight path display system characterized in that it comprises an image generating means for outputting to the HMD.

  As another aspect of the present invention, the user wears a head-mounted display device (HMD) on the head, and reduces the target area, which is a space including the terrain, at a predetermined scale rate. A reduced area presenting stage for presenting a reduced area, which is a space reproduced at a predetermined installation location, to the user, an HMD positioning stage for measuring the position and orientation of the HMD relative to the reduced area, and the target area A reduced flight path generation step for obtaining a reduced flight path, which is a space curve in the reduced region, corresponding to the flight path of the flying object in FIG. 5, the position and orientation of the HMD obtained in the HMD positioning step, and the reduced flight Based on the reduced flight path obtained in the path generation stage, generate an image showing the field of view when viewing the reduced area from the position and orientation, Providing flight path display method characterized by comprising an image generating step of outputting the MD.

  As another aspect of the present invention, the user wears a head-mounted display device (HMD) on the head, and reduces the target area, which is a space including the terrain, at a predetermined scale rate. HMD positioning means for calculating the position and orientation of the HMD with respect to the reduced area, which is a space reproduced at a predetermined installation location, from the measured values, and the inside of the reduced area corresponding to the flight path of the flying object in the target area Reduced flight path generation means for obtaining a reduced flight path that is a spatial curve of the HMD, the position and orientation of the HMD obtained by the HMD positioning means, and the reduced flight path obtained by the reduced flight path generation means As an image generation means for generating an image showing a field of view when the reduced area is viewed from the position and orientation, and outputting the image to the HMD, a computer It provides a program for causing a.

  According to the present invention, a user can view a flight path as an image of a field of view that can be seen in his / her viewpoint and line-of-sight direction in a reduced space that is obtained by reducing an actual space. For this reason, in order to view the flight path from the desired viewpoint and line-of-sight direction, the user inputs a desired viewpoint and line-of-sight direction with a mouse or keyboard and then views the image of the corresponding field of view. Without passing through, it is possible to visually recognize the flight path when viewed from a desired field of view simply by viewing a desired angle from a desired position. For this reason, according to the present invention, the user can visually recognize the entire flight path from a desired viewpoint and line-of-sight direction by a more intuitive operation.

1 is a block diagram of a flight path display system 1 according to an embodiment of the present invention. 1 is a block diagram of a flight path creation system 10 that is an embodiment of the present invention. 3 is a flowchart for explaining a flight path input method by the flight path creation system 10; 4 is a flowchart for explaining a flight path display method by the flight path creation system 10. It is a figure for demonstrating the field of view of the flight pass creator who mounted HMD12. 3 is a diagram for explaining data transmitted in the flight path creation system 10. FIG. It is a block diagram of the flight path creation system 40 which is one Example of this invention. 4 is a flowchart for explaining a method of displaying a virtual object in the field of view of a flight path creator by the flight path creation system 40; 5 is a flowchart for explaining a flight path input method by the flight path creation system 40; 4 is a flowchart for explaining a flight path display method by the flight path creation system 10. It is a block diagram of the flight path creation system 50 which is one Example of this invention. It is a figure for demonstrating the image which shows a visual field displayed on HMD12b in the viewpoint and visual line direction different from HMD12a at the same time when the image which shows a visual field like FIG. 5 is displayed on HMD12a.

  A flight path display system 1 according to an embodiment of the present invention will be described. The flight path display system 1 is a system for displaying a space curve consisting of waypoints indicating the trajectories of airplanes such as airplanes and helicopters and other flying objects, that is, flight paths. Referring to FIG. 1, the flight path display system 1 includes a reduced area presentation unit 2, a head mounted display (hereinafter referred to as “HMD”) 3, an HMD positioning unit 4, and an image processing device 5.

  The reduced area presenting unit 2 presents to the user a space obtained by reducing the area through which the flight pass passes at a predetermined scale, specifically, a model of the terrain of the area, so-called A diorama and a space that presents a reduced area as a space on the diorama can be considered. Alternatively, based on the terrain data of the area, the image processing device 5 generates an image in which the terrain viewed from the position and direction of the HMD 3, that is, the user's viewpoint and line-of-sight direction, as a virtual object is drawn at the corresponding position What is displayed with HMD3 is also considered. In any case, the reduced area to be presented is fixedly set or set at a predetermined position. When presenting a reduced area using a diorama, the diorama is placed at a predetermined position. When a reduced area is presented by computer graphics, the terrain is set at a predetermined position as a virtual object by computer graphics and is presented to the user in the same manner as installing a diorama.

  The HMD 3 is used by a user such as a flight pass creator wearing it on his / her head, and displays an image output from the image processing device 5 to the user. When the terrain of the reduced area is presented to the user as a terrain model, the HMD is a so-called transmissive HMD that includes a transmissive display that allows the user to simultaneously view an image projected on the screen and a scene outside the screen. Is preferred. The shape may be eyeglass type or hat type. The projection system may display an image using a half mirror, or may be a retinal scanning type that forms an image directly on the retina.

  The HMD positioning unit 4 measures the position (spatial coordinates) and orientation of the HMD 3 with respect to the reduced area presented by the reduced area presentation unit 2. The HMD positioning unit 4 may be provided separately from the HMD 3, may be provided integrally with the HMD 3, or both may be provided.

  As the HMD positioning unit 4 provided separately from the HMD 3, a plurality of digital video cameras are installed around the reduced area so as to fit the entire reduced area within the angle of view. It is conceivable to obtain the position and orientation of the HMD 3 by performing real-time image processing on an image acquired from each camera in real time with an arithmetic processing unit after calibration. By this image processing, the position including the altitude of the HMD 3 and its direction can be obtained, but the direction of the HMD 3 can be obtained more accurately by calculating the outputs of an acceleration sensor and a gyro sensor as will be described later. Can do.

  As the HMD positioning unit 4 provided integrally with the HMD 3, there is an HMD 3 provided with an acceleration sensor and a gyro sensor and calculated from these outputs by an arithmetic processing unit in real time. For this operation, the three-dimensional angular velocity and acceleration are obtained in some mobile phone terminals equipped with an aircraft inertial motion measuring instrument or an acceleration sensor and a gyro sensor configured as a MEMS (Micro Electro Mechanical System), The direction of HMD3 is calculated | required with an arithmetic processing unit with respect to these. Furthermore, it is good also as calculating | requiring the position of HMD3 by performing the process which calculates a movement distance by integral of an inertia in an arithmetic processing unit. Furthermore, it is good also as calculating | requiring the position of HMD3 from the output of a GPS receiver as providing a GPS receiver in HMD3.

  The image processing device 5 generates an image of the flight path viewed from the current position and direction of the HMD 3 based on the input flight path and the position and direction of the HMD 3 output by the HMD positioning unit 4 and outputs the image to the HMD 3 To do.

  The flight path display system 1 operates as follows to display the flight path to the user.

  First, the reduced area presentation unit 2 presents a reduced area to the user via the HMD 3. The reduced area is a space that reproduces an actual space (target area) through which a flight path to be displayed passes at a predetermined scale ratio, and presents the reduced area in the vicinity of the HMD 3.

  Next, the HMD positioning unit 4 measures the current viewpoint coordinates and line-of-sight direction of the HMD 3.

  Next, the image processing device 5 obtains the viewpoint and line-of-sight direction of the HMD 3 in the coordinate system of the reduced area based on the measurement result in the HMD positioning unit 4.

  Further, the image processing device 5 generates an image when the reduced flight path in the reduced area is viewed from the obtained viewpoint and line-of-sight direction. The reduced flight path passes through a point in the reduced area corresponding to a point in the target area through which the flight path passes.

  Finally, the HMD 3 displays the image generated by the image processing device 5 to the user wearing the HMD 3.

  A flight path creation system 10 according to an embodiment of the present invention will be described with reference to FIG. The flight path creation system 10 displays an imaginary flight path on a diorama to the flight path creator using augmented reality technology. The flight path creation system 10 includes a diorama 11 corresponding to the reduced area presentation unit 2, an HMD 12 corresponding to the HMD 3, an HMD positioning unit 13 corresponding to the HMD positioning unit 4, and a display calculation unit 14 corresponding to the image processing device 5. Further, as a means for inputting a flight path, an aircraft model / diorama position recognition device 15, cameras 16, 17 and an aircraft model 18 are provided. Furthermore, the communication interface device 19, the data communication network 20, and various databases 21-25 are provided. Among these, in particular, with respect to the diorama 11, the HMD 12, the HMD positioning unit 13, the display calculation unit 14, the aircraft model / diorama position recognition device 15, the cameras 16, 17, and the aircraft model 18, the command installed in the target area or in the vicinity thereof. It is preferable to be disposed at a place. As a result, the flight path created by the rear unit can be changed based on the latest information obtained from the front by the command post, and the flight path can be determined in accordance with the actual condition of the front.

  The diorama 11 is a terrain model obtained by reducing the terrain of a region where a flight path is to be created at a predetermined scale. Hereinafter, in this embodiment, this scale is simply referred to as a scale. For example, a mountain in the region has a height that is reduced in scale. When there are buildings such as buildings and towers, it is preferable that these are also reproduced at a reduced height.

  The HMD 12 is a head-mounted display device having a transmissive display, and is used by a flight pass creator wearing it on the head.

  The HMD positioning unit 13 is an acceleration sensor, a gyro sensor, and a GPS receiver provided integrally with the HMD 12 in order to measure the position and direction of the HMD 12.

  The display calculation unit 14 is a calculation processing device, and generates an image to be displayed on the HMD 12 based on the outputs of the HMD positioning unit 13 and the aircraft model / diorama position recognition device 15.

  The aircraft model / diorama position recognition device 15 acquires the position of the aircraft model 18 from the images generated by the cameras 16 and 17 and notifies the display calculation unit 14 of the position. Prior to the display and input of the flight path, the installation position data of the diorama 11 and the installation positions data of the cameras 16 and 17 are registered in advance in the storage device of the aircraft model / diorama position recognition device 15. The aircraft model / diorama position recognition device 15 performs processing for recognizing the aircraft model 18 on the two images simultaneously generated by the cameras 16 and 17 to identify the position of the aircraft model 18 in each image, and further Based on the positions in these two images, the diorama installation position data, and the camera installation position data, the three-dimensional coordinates of the aircraft model 18 in the reduced space are obtained.

  The cameras 16 and 17 are arranged around the diorama so that the optical axes are orthogonal to each other. The cameras 16 and 17 capture the entire scene of the diorama 11 and a scene including a space in which a spatial curve indicating the flight path is drawn in synchronization with each other. The positions of the cameras 16 and 17 are measured in advance and set in the aircraft model / diorama position recognition device 15.

  The aircraft model 18 is used as a pointer when drawing a flight path in space. The aircraft model 18 is, for example, a model imitating an aircraft to be created, but this is not necessarily the case.

  The communication interface device 19 connects the display calculation unit 14 and the aircraft model / diorama position recognition device 15 to transmit data to each other. In addition, the display calculation unit 14 is connected to the data communication network 20 via the communication interface device 19 and accesses various resources on the network.

  Each of the databases (DB) 21-25 is a database, and stores information related to a region where a flight path is to be created, in association with the position information. The DB 21 stores entry prohibited area information. The entry prohibition area information is information indicating an area where passage of a general aircraft is prohibited. The DB 22 stores monitoring target position information. The monitoring target position information stores the position information of the target, particularly when the aircraft that is trying to create a flight path is an aircraft that performs monitoring or reconnaissance. The DB 23 stores altitude data. This altitude data associates the altitude of each point in the area with its position. The DB 24 stores weather data. That is, the current weather, such as clear / cloudy / rain / snow, temperature, humidity, wind speed, wind direction, rainfall, snow cover, etc. are stored in the region where the flight path is to be created. The DB 25 stores sunshine data.

  Next, the operation of the flight path creation system 10 will be described. First, the operation of the aircraft model / diorama position recognition apparatus 15 when inputting a flight path will be described with reference to FIG. The flight path creator holds the aircraft model 18 and moves the aircraft model 18 on the diorama 11 so as to draw a desired flight path in the air. This situation is photographed in synchronization with the cameras 16 and 17, and image data is generated in real time (step S1). The cameras 16 and 17 simultaneously generate images of the aircraft model 18 and the diorama 11 taken from different angles. The aircraft model / diorama position recognition device 15 performs image recognition processing on the image data, and identifies the aircraft model 18 in the image data. By performing image recognition processing based on the positional relationship between the diorama 11 and the cameras 16 and 17 for each of the image data of the cameras 16 and 17 at a certain time, a plane perpendicular to the optical axis of the camera in the image data of each camera. The two-dimensional coordinates of the aircraft model 18 are obtained (step S2), and the three-dimensional coordinates of the aircraft model 18 are obtained from these two two-dimensional coordinates (step S3). The coordinates thus obtained are transmitted as the three-dimensional coordinates of the aircraft model 18 at that time together with the time to the display calculation unit 14 via the communication interface device 19 (step S4). By repeating this at regular intervals (step S5), the coordinates of the points on the space curve and the time of each point drawn in the air on the diorama 11 by the aircraft model 18 that the flight path creator has in hand are displayed. The calculation unit 14 is notified.

  Next, with reference to FIG. 4, the operation of the display calculation unit 14 when the flight path being input is displayed on the HMD 12 will be described. The display calculation unit 14 stores the photographing time and three-dimensional coordinates notified so far in the storage device. The display calculation unit 14 acquires the direction and position in which the HMD 12 is currently facing from the HMD positioning unit 13 (step S10). Based on this direction and position and the position of the diorama 11 registered in advance, the display calculation unit 14 calculates the position and direction of the HMD 12 with reference to the diorama 11 (step S11). Further, the three-dimensional coordinates of the aircraft model 18 already stored in the storage device when viewed from the calculated position and direction of the HMD 12, that is, the viewpoint and line-of-sight direction of the flight path creator, are the view of the flight path creator. The position is calculated (step S12), a point is drawn at each calculated position, and an image in which a straight line or a curve connecting the points is drawn is generated as necessary (step S13). Is displayed (step S14). It is assumed that the angle of view (viewing angle) of the flight pass creator's eyes is predetermined and stored in the storage device of the display calculation unit 14. As a result, the HMD 12 displays a state when the space curve drawn by the flight path creator in the air on the diorama 11 is viewed from the current viewpoint and line-of-sight direction of the flight path creator. Since the HMD 12 has a transmissive display, the flight path creator can simultaneously view the image of the spatial curve generated by the display calculation unit 14 and displayed on the HMD 12 and the sight of the diorama 11 transmitted through the transmissive display. Will do.

  In this manner, steps S1 to S5 are repeated, and the flight path creator obtains the coordinates of the space curve drawn above the diorama 11 at regular intervals, in parallel with steps S10 to S14. An image showing how the space curve looks in the current view point and the line-of-sight direction of the path creator is generated and displayed on the HMD 12. Thus, the real image of the diorama 11 that is viewed through the transmissive display and the virtual image of the flight path corresponding to the position and direction of the diorama 11 are simultaneously provided to the flight path creator.

  In the flight path creation system 10, a point on the input flight path is moved as follows. The aircraft model / diorama position recognizing device 15 analyzes the images of the cameras 16 and 17 in real time, and when a specific operation is performed on the diorama 11, the operation is recognized as an editing command associated in advance. For example, in the air on the diorama 11, when it is recognized that a human hand has grasped and a point indicating a flight path has already been input at the grasped position, the aircraft model / diorama position recognition device 15 Is recognized as a command to be selected for editing. Further, after this selection command, when it is recognized that a human hand has opened in the air on the diorama 11, the aircraft model / diorama position recognition device 15 recognizes it as a command to move the point, and the selection command Move the point selected in step to the position where the hand is open.

  As described above, the flight path creation system 10 uses the transmission type HMD 12 to provide the flight path creator with a field of view including the real image of the diorama 11 and the virtual image of the flight path, but further creates a flight path. An image is generated that displays various information to be referred to at the same time as the flight path at a corresponding position in the field of view.

  One such additional display is a footprint display. Here, the footprint is a three-dimensional shape that indicates an area that can be photographed or measured by a ground shooting camera provided on an aircraft that defines a flight path or various sensors that measure the ground. A virtual cone having a rotation axis that is a straight line indicating the optical axis of the camera or the direction of the sensor. The closed curve formed by the intersection of the cone and the curved surface indicating the terrain indicates, for example, an area that can be photographed by the camera. In the present embodiment, only the position of the aircraft model 18 is acquired and the attitude is not acquired. Therefore, the position of the aircraft model 18 is a vertex, and a cone having a rotation axis as a straight line extending vertically downward from the vertex is a footprint. Based on the position of the aircraft model 18 obtained in step S2 and the viewpoint and line-of-sight direction of the flight path creator obtained in step S11, the display calculation unit 14 uses one point at the bottom of the aircraft model 18 as a vertex. A conical image having a rotation axis as a straight line drawn vertically downward from the apex is imaged together with the flight path obtained as described above. When considering the degree of opening of the cone, that is, the cross section of the cone including the rotation axis, the angle formed by the two sides sandwiching the apex angle is determined in advance according to the specifications of the camera and sensor mounted on the aircraft. When there is no limitation on the distance that can be taken by the camera and the sensor, the display calculation unit 14 draws a conical image from the lower vertex of the aircraft model 18 to the lower end of the generated image. On the other hand, when there is a limit to the distance that can be photographed and measured, the height of the cone may be determined according to the upper limit distance and the scale of the diorama 11, and no cone image may be drawn below that. In the image output to the HMD 12, the footprint is displayed as a colored cone, for example. By displaying such a cone, the flight path creator creates a flight path while visually judging in real time whether the point to be photographed and measured is within the effective range of the airborne camera and sensor. can do.

  Further, the flight path creation system 10 refers to the data in the database 21-25, and images various information at corresponding positions on the diorama 11 and corresponding positions in the space above the diorama 11. It is good. Values stored in these databases may be updated based on data provided by an external device (not shown) connected via the data communication network 20. In this way, the flight path creation system 10 can create a flight path based on the latest data.

  As this type of external device, for example, there is a ground station that receives observation data transmitted by an observation satellite or an observation airplane by wireless communication and provides the observation data to another device via a data communication network. At this time, the DB 23 is updated based on the latest altitude data provided by the ground station, and the display calculation unit 14 uses a numerical value indicating the updated altitude data as a virtual object at a corresponding point on the diorama 11 or in the vicinity thereof. It is conceivable that a drawn image is generated and output to the HDM 12.

  Alternatively, the monitoring target 33 is not a fixed object such as a building but a moving body such as a vehicle or an animal, and the observation satellite or the observation airplane may observe the position of the monitoring target 33. In this case, the ground station notifies the latest monitoring target position information to the DB 22, and the display calculation unit 14 generates an image in which the monitoring target 33 is drawn at a position based on the latest monitoring target position information and outputs the image to the HDM 12.

  The display calculation unit 14 refers to the data in the DB 21 and displays an image indicating that the space entering the field of view when viewed from the position and direction of the HMD 12 obtained in step S11 described above enters the entry prohibition region. It is good also as producing | generating. In order to indicate the entry prohibition region, for example, it is conceivable to generate an image in which the corresponding airspace is colored or to draw a straight line or a curve for dividing the airspace from another airspace in the image. By displaying the no entry area as a virtual object, the flight path creator can enter the flight path while visually avoiding areas that cannot be entered for regulatory reasons, not for terrain reasons. .

  Moreover, the display calculation part 14 is good also as producing | generating the image which has arrange | positioned the image which shows the monitoring target in the location of the monitoring target with reference to the data of DB22. When the monitoring target is in a position that falls within the view of the flight path creator, an image as if the monitoring target is placed at a corresponding point in the diorama 11 is generated and displayed on the HMD 12. It is easy to create a flight path while displaying. This is because the flight path creator wearing the HMD 12 can create the flight path while visually recognizing both the monitoring target and the footprint.

  In addition, the display calculation unit 14 refers to the data in the DB 23, and a numerical value indicating the altitude of the point is in the vicinity of a main part on the topography such as a summit or a valley in an area that enters the view of the flight pass creator. An image may be generated so as to be displayed and displayed on the HMD 12 together with a flight path or the like.

  Further, the display calculation unit 14 refers to the weather data and the sunshine data in the DBs 24 and 25, and for the region entering the flight path creator's field of view, the weather state indicated by the weather data and the image corresponding to the sunshine state indicated by the sunshine data. May be generated and displayed on the HMD 12 together with the flight path and the like.

  When an image in which the flight path 31, the footprint 32, the monitoring target 33, and the entry prohibition area 34 are drawn on the HMD 12 with the diorama 11 as a background is viewed on the HMD 12, an image seen from the flight path creator is, for example, as shown in FIG. A flight path 31, a space curve following the tail of the aircraft model 18, a cone 32 indicating a footprint, a monitoring target 33 through which the footprint 32 should pass, and an area 34 surrounded by an ellipse indicating a no entry area, An image showing the state when viewed from the current viewpoint and the line-of-sight direction of the flight path creator is displayed on the HMD 12. The flight path creator makes sure that the inside of the closed curve formed by the intersection of the footprint 32 and the diorama 11 passes through the monitoring target 33 and that the aircraft model 18 does not touch either the diorama 11 or the entry prohibition area 34. Then, the aircraft model 18 is moved to input the flight path. In FIG. 5, the double-headed arrow extending between the diorama 11 and the entry prohibition area 33 and the characters “flyable space altitude” are described for easy understanding of the entry prohibition area 33. These may be generated as virtual objects and may or may not be displayed on the screen.

  As described above, the flight path creator inputs the flight path so that the flight path 31 does not collide with either the diorama 11 or the entry prohibition area 34. Further, the flight path creator inputs the flight path so that the footprint 32 and the monitoring target 33 collide at least once. These collision determinations may be made visually by the flight pass creator, or may be made by calculation by the display calculation unit 14.

  In other words, the display calculation unit 14 performs a collision determination between the object drawn in the virtual space such as the entry prohibition area 34 and the aircraft model 18 and notifies the flight path creator of such a collision when they collide with each other. For this reason, the flight path creator may be prompted to change the flight path. As the notification means, a text message for notifying that effect or an image indicating the occurrence of a collision may be displayed on the screen of the HMD 12, and a display unit 14 is provided with a speaker (not shown) to detect the collision. It is good also as making a beep.

  Further, when the flight path creator performs an operation indicating that the creation of the flight path has been completed, the display calculation unit 14 performs a collision determination between the footprint 32 and the monitoring target 33, and the created flight path. If the footprint 32 that passes along the line never touches the monitoring target 33, the flight path creator is requested to change the flight path by outputting to notify the flight path creator of that fact. It may be encouraged.

  As described above, the flight path creation system 10 indicates the cone indicating the footprint, the entry prohibition area, the monitoring target, the altitude of each place, the weather condition, the sunshine condition, and the like at the corresponding positions of the diorama 11 and the aircraft model 18. By generating an image and displaying it on the transmissive HMD 12, the flight path creator draws a spatial curve drawn on the diorama 11 while providing the flight path creator with a field of view in which the diorama 11 and the image are superimposed. Record. The coordinates of the points constituting the spatial curve thus recorded are determined based on the diorama 11 and are different from the coordinates through which the actual flight path passes. Therefore, when outputting the created flight path, the display calculation unit 14 uses the coordinates of the points on the spatial curve drawn on the diorama 11 as the terrain in the space on the actual terrain that is the model of the diorama 11. Perform processing to convert to. The flight path data converted into the actual scale is stored in, for example, an external storage device (not shown) included in the display calculation unit 14, or is output to the external device via the communication interface device 10 and the data communication network 20. . As such an external device, for example, there is a device installed at a base provided at the back or a support station in cooperation with a command post where the diorama 11 or the like is installed.

  The contents of data communication performed between the functional blocks of the flight path creation system 10 are shown in FIG.

  A flight path creation system 40 according to another embodiment will be described with reference to FIG. In the above-described flight path creation system 10, the diorama is used as a means for presenting the reduced area to the flight path creator, and the input flight path, footprint 32, monitoring target 33, entry prohibition area 34, etc. Is displayed so as to overlap the diorama 11 as augmented reality. That is, in the first embodiment, the aircraft model 18 and the diorama 11 are used as the substantial models, and the objects such as the flight path, the footprint 32, the monitoring target 33, and the entry prohibition area 34 are reproduced in augmented reality. In the flight path creation system 40 of the present embodiment, the augmented reality is applied up to the reproduction of the terrain and the virtual space 41 is used. The flight path creation system 40 includes an aircraft model / virtual space position recognition device 43 instead of the aircraft model / diorama position recognition device 15, and further stores detailed terrain data instead of the DB 23 for storing altitude data. DB 44 is provided.

  The augmented reality display marker 41 a is provided with markings that can be optically individually recognized and identified through images taken by the cameras 16 and 17. The four markers 41a are installed at appropriate places so that each of the four markers 41a is a vertex of a square or a rectangle. Hereinafter, a rectangular area surrounded by the marker 41a is referred to as a virtual space installation position. The coordinates of the virtual space installation position, that is, the coordinates where each marker 41 a is arranged are registered in advance in the storage device of the display calculation unit 42.

  The virtual space 41 is a virtual space that is generated as a computer graphic by the display calculation unit 42 using the installed marker 41a as a reference. In advance, the flight path creator sets the coordinates of the actual area in which the flight path is to be created in the display calculation unit 42. Here, an actual area where the virtual space is generated at the virtual space installation position is referred to as a target area. Based on the coordinates of the marker 41a, the display calculation unit 42 obtains a scale ratio that allows the target area to be within the virtual space installation position. The display calculation unit 42 generates an image showing the view of the flight path creator when the target area reduced at the scale ratio thus obtained is viewed from the viewpoint and line-of-sight direction separately obtained by the HMD positioning unit 13, Output to HMD12.

  In the storage device of the aircraft model / virtual space position recognition device 43, in addition to the positions of the cameras 16 and 17, the positions where the markers 41a are installed are stored in advance. In each of the two images generated by simultaneously photographing with the cameras 16 and 17, an image recognition process for specifying the aircraft model 18 and the marker 41 a is performed, and the aircraft in each image is based on the position of the marker 41 a registered in advance. The two-dimensional coordinates of the model 18 are obtained, and the three-dimensional coordinates of the aircraft model 18 are obtained based on these two two-dimensional coordinates.

  With reference to FIG. 8, a method of drawing a virtual space when viewed from the viewpoint and the line-of-sight direction of the flight path creator will be described. First, the flight path creator determines a target area through which the flight path to be created passes and sets the coordinates in the display calculation unit 42 (step S21). When the flight path creator wears the HMD 12 and activates the HMD positioning unit 13, the HMD positioning unit 13 obtains measurement values regarding the current position and direction, that is, the viewpoint and line-of-sight direction of the flight path creator, and displays them in the display calculation unit 42. Notice. Based on the measured values, the display calculation unit 42 obtains the positional relationship between the marker 41a registered in advance and the viewpoint and the line-of-sight direction (step S22), and enters the flight path creator's view seen from the viewpoint and the line-of-sight direction. Then, the range of the terrain within the virtual space installation position is obtained (step S23). Further, the display calculation unit 42 acquires the terrain data of a point that falls within the field of view from the DB 44 (step S24), and generates an image showing the field of view based on the acquired terrain data (step S25). By repeating such processing at a sufficiently short time interval (step S26), a scene of the virtual space corresponding to the current viewpoint and line-of-sight direction of the flight path creator is displayed on the HMD 12.

  With reference to FIG. 9, the operation of the aircraft model / virtual landform position recognition device 43 when inputting a flight path will be described. The flight path creator holds the aircraft model 18 in his hand and moves the aircraft model 18 within the virtual space installation position so as to draw a desired flight path in the air. This state is photographed in synchronism with the cameras 16 and 17, and image data is generated in real time (step S31). The cameras 16 and 17 simultaneously generate images of the aircraft model 18 and the virtual space installation position taken from different angles. The aircraft model / virtual landform position recognizing device 43 performs image recognition processing on the image data, and specifies the aircraft model 18 in the image data. By performing image recognition processing based on the positional relationship between the diorama 11 and the cameras 16 and 17 for each of the image data of the cameras 16 and 17 at a certain time, a plane perpendicular to the optical axis of the camera in the image data of each camera. The two-dimensional coordinates of the aircraft model 18 are obtained (step S32), and the three-dimensional coordinates of the aircraft model 18 are obtained from these two two-dimensional coordinates (step S33). The coordinates thus obtained are transmitted as the three-dimensional coordinates of the aircraft model 18 at the time to the display calculation unit 42 through the communication interface device 19 together with the time (step S34). By repeating this at regular intervals (step S35), the coordinates of the points on the space curve drawn at the virtual space installation position by the aircraft model 18 held by the flight path creator in the hand and the time of each point are displayed and calculated. Notify the unit 42. By performing the flight path input process of FIG. 9 in parallel with the virtual space drawing process described in FIG. 8, the flight path creator is viewing the scene of the virtual space viewed from his / her viewpoint and line-of-sight direction. The flight model can be input by moving the aircraft model 18 within the virtual space installation position.

  Next, with reference to FIG. 10, the operation of the display calculation unit 42 when displaying the flight path being input on the HMD 12 will be described. The display calculation unit 42 stores the photographing time and three-dimensional coordinates notified so far in the storage device. The display calculation unit 42 acquires the direction and position in which the HMD 12 is currently facing from the HMD positioning unit 13 (step S41). Based on this direction and position and the position of the marker 41a registered in advance, the display calculation unit 42 calculates the position and direction of the HMD 12 with reference to the marker 41a (step S42). Furthermore, when the calculated position and direction of the HMD 12 are viewed from the viewpoint and line-of-sight direction of the flight path creator, the three-dimensional coordinates of the points indicating the input flight path already stored in the storage device are as follows: Calculate where in the flight path creator's field of view (step S43), draw a point at each calculated position, and generate an image that draws a straight line or curve connecting the points as necessary ( Step S44) and display on the HMD 12 (Step S45). It is assumed that the angle of view (viewing angle) of the flight pass creator's eyes is predetermined and stored in the storage device of the display calculation unit 14. As a result, the HMD 12 displays a state when the space curve drawn by the flight path creator in the air on the diorama 11 is viewed from the current viewpoint and line-of-sight direction of the flight path creator. Since the HMD 12 has a transmissive display, the flight path creator can simultaneously view the image of the spatial curve generated by the display calculation unit 14 and displayed on the HMD 12 and the sight of the diorama 11 transmitted through the transmissive display. Will do. In parallel with the drawing process of the virtual space of FIG. 8 and the flight path input process of FIG. 9, the flight path creator viewed from his own viewpoint and line-of-sight direction by performing the input flight path display process of FIG. When a flight path is input by moving the aircraft model 18 within the virtual space installation position while viewing the scene of the virtual space, the input can be performed while viewing the flight path input so far.

  The flight path creation system 40 draws the terrain in the reduced area as a virtual object and presents it to the flight path creator. For this reason, in this system, the scale ratio of the reduction | restoration area | region shown with respect to a flight path creator can be changed by changing the scale ratio of the virtual object at the time of drawing.

  In the flight path creation system 40, the target area for creating the flight path can be changed by changing the setting in step S21.

  Further, using the generation of the terrain as a virtual object, the display calculation unit 14 performs a collision determination between the flight path and the virtual object, and when a collision is detected, a message for warning the occurrence of the collision, An image representing a collision, a moving image, or the like may be displayed at or near the coordinates where the collision is detected. Alternatively, a speaker (not shown) may be connected via the communication interface device 19, and a predetermined warning sound may be generated from the speaker. In this way, not only the flight creator's own eyes but also the determination by the display calculation unit 14 confirms that the flight path does not collide with virtual objects such as terrain, entry prohibited areas, and monitoring targets. Can create a flight path. Further, the presence or absence of a collision can be used as an index when judging the validity of the completed flight path.

  A flight path input system 50 according to a third embodiment will be described with reference to FIG. The flight path input system 50 includes two sets of elements corresponding to the HMD 12 and the HMD positioning unit 13 of the flight path creation system 10, that is, HMDs 12 a and 12 b and HMD positioning units 13 a and 13 b, and display calculation instead of the display calculation unit 14. A unit 51 is provided.

  In this system, two flight path creators wear HMD and HMD positioning units, respectively. It is assumed that the flight path creator A wears the HMD 12a and the HMD positioning unit 13a, and the flight path creator B wears and uses the HMD 12b and the HMD positioning unit 13b. Therefore, the HMD 12a displays an image for the flight path creator A, and the HMD positioning unit 13a is a device for measuring the flight path creator A's viewpoint and line-of-sight direction. Similarly, the HMD 12b displays an image for the flight path creator B, and the HMD positioning unit 13b is a device for measuring the viewpoint and line-of-sight direction of the flight path creator B.

  The display calculation unit 51 is a calculation processing device that generates an image to be displayed on the HMD 12a based on the outputs of the HMD positioning unit 13a and the aircraft model / diorama position recognition device 15, and also includes the HMD positioning unit 13b and the aircraft model. Based on the output of the diorama position recognition device 15, an image to be displayed on the HMD 12b is generated. The image generation process performed for each HMD is the same as in the first embodiment.

  Each of the flight path creators A and B wearing the HMD and the HMD positioning unit is watching the diorama 11 and the flight path, the entry prohibition area, and the augmented reality displayed on the diorama 11 simultaneously from different viewpoints. Enter and correct the flight path.

  For example, it is assumed that a scene as shown in FIG. 5 is seen in the field of view of the flight path creator A who is inputting the flight path with the aircraft model 18 in the right hand (not shown). At this time, the flight path creator A is standing in the vicinity of the side far from the mountain of the diorama 11 and close to the position of the monitoring target 33 and looking at the viewpoint and the line-of-sight direction overlooking the diorama 11 and the aircraft model 18. The display calculation unit 51 generates an image in which the flight path 31, the footprint 32, the monitoring target 33, and the entry prohibition area 34, which are virtual objects, are drawn as shown in the figure, and outputs the image to the HMD 12 a. At the same time, it is assumed that the flight path creator B stands in the vicinity of the opposite side across the flight path creator A and the diorama 11 and is looking at the viewpoint and the line of sight overlooking the diorama 11 and the aircraft model 18. At this time, the display calculation unit 51 generates an image in which the virtual object is drawn with the positional relationship as shown in FIG. 12 based on the viewpoint position and the line-of-sight direction of the flight path creator B, and outputs the image to the HMD 12b. 12 is a diagram for explaining that the display calculation unit 51 simultaneously generates a plurality of images based on a plurality of sets including the viewpoint and the line-of-sight direction, and outputs the images to the corresponding HMD. The positional relationship between the virtual objects in FIG. 12 does not necessarily correspond exactly.

  As described above, the flight path creation system 50 generates an image showing the same flight path as an image showing a view from different viewpoints and line-of-sight directions for each user at the same time. Display. Thus, a plurality of users can simultaneously view the same flight path and virtual object from different viewpoints and line-of-sight directions and create a flight path while discussing.

  Observation data may be collected from a flying object such as an airplane that flies in accordance with the flight path created by the above-described flight path creation system and displayed on the flight path creation system. In this case, the flying object includes an observation device having an observation range represented by a footprint 32, a positioning device for measuring the current position of the flying object, an observation result measured by the observation device, and the observation result. And a storage device that stores the position and time measured by the positioning device in association with each other. For example, in the case of the flight path creation system 10, the observation result, position, and time stored in the storage device are transmitted to the display calculation unit 14 via the data communication network 20. The display calculation unit 14 generates an image in which the virtual object indicating the observation result is drawn at a corresponding position in the field of view in the viewpoint and line-of-sight direction obtained by the HDM positioning unit 13 and outputs the generated image to the HDM 12. In this way, it is possible to verify the observation result drawn at the position observed as a virtual object based on the viewpoint and line-of-sight direction of the flying object flying along the flight path. .

  As mentioned above, although this invention was demonstrated according to embodiment and an Example, this invention is not limited to this, A various deformation | transformation is possible.

  In the above-described embodiment, only the position of the aircraft model 18 is acquired and the posture is not acquired. However, the posture may be acquired. In the first embodiment, a plurality of images are generated by simultaneously capturing images with the cameras 16 and 17 in step S1, and based on these images, image recognition processing is performed in the aircraft model / diorama position recognition device 15 in steps S2 and S3. As a result, the position of the aircraft model 18 with respect to the diorama 11 was calculated. At the same time, the attitude of the aircraft model 18 was calculated by image recognition processing, and the attitude information together with the three-dimensional coordinates of the aircraft model 18 and time information. In step S4, the information may be transmitted to the display calculation unit 14 via the communication interface device 19, and this may be repeated at regular intervals.

  In the above-described embodiment, since the attitude of the aircraft model 18 was not taken into consideration, regardless of the attitude of the aircraft model 18, the footprint was assumed as a cone whose vertical axis extends from the aircraft model 18 vertically downward. However, when the attitude of the aircraft model 18 is taken into consideration, the rotation axis may be determined according to the attitude. In this case, the orientation of the cone serving as the footprint is determined according to the posture, and further, the closed curved surface formed by the intersection of the cone and the terrain curved surface is determined. The closed curved surface moves together with the aircraft model 18, and a region through which the closed curved surface becomes a subject to be photographed by a camera and a subject to be measured by a sensor.

  There are two types of HDM projection methods, one that projects an image on the retina and the other that projects a virtual image on a half mirror, either of which can be used as a transmissive display. When the terrain is drawn as a virtual object as in the second embodiment, the HDM may not be a transmissive type. In the above description, as an image to be displayed on one HDM at a time, a projection view representing a field of view viewed from one viewpoint is generated as one image. However, when the HDM can display different images for the left and right eyes, for example, when the HDM has a display device for each of the left and right eyes, the HDM is viewed from the viewpoints of the left and right eyes. An image representing a field of view, that is, a right-eye image and a left-eye image that have a parallax with each other, are displayed on the right-eye display device and the left-eye display device, thereby providing stereoscopic viewing to the user. Also good.

  Further, in the above description, the flight path creator has been described as inputting the flight path by holding the aircraft model in hand, but instead of holding the aircraft model 18 directly in the hand, for example, it may be attached to the tip of a rod and moved. Alternatively, it may be moved by being attached to the tip of the robot arm and operating the robot arm.

  In the above-described embodiment, the position of the aircraft model is measured using the two cameras 16 and 17, but the number of cameras is not limited to two and may be three or more. .

  Part or all of the above embodiments and examples can be described as the following supplementary notes, but are not limited thereto.

(Appendix 1)
A head-mounted display device (HMD) that a user wears on the head, and
A reduced area presentation means for reducing a target area, which is a space including terrain, at a predetermined scale ratio and presenting a reduced area, which is a space reproduced at a predetermined installation location, to the user;
HMD positioning means for measuring the position and orientation of the HMD relative to the reduced area;
Reduced flight path generation means for obtaining a reduced flight path corresponding to a flight path of the flying object in the target area, which is a space curve in the reduced area;
Based on the position and orientation of the HMD obtained by the HMD positioning means and the reduced flight path obtained by the reduced flight path generating means, a view when the reduced area is viewed from the position and orientation is shown. A flight path display system comprising image generation means for generating an image and outputting the image to the HMD.

(Appendix 2)
It is held directly by a user's hand or indirectly via a holder or the like, and indicates the position of the flying object in the reduced area. A pointer to be moved,
The flight path display system according to claim 1, further comprising means for measuring a position of a pointer moving in the reduced area as a position of a point on the reduced flight path.

(Appendix 3)
The HMD includes a transmissive display, and a user can view an image output from the image generation unit and a scene outside the HMD at the same time. Flight path display system.

(Appendix 4)
The reduced area presentation means includes a terrain model obtained by reducing the terrain of the target area at the scale, and transmits the transmissive display, the terrain model installed at the installation location and a view of the space above it. The flight path display system according to appendix 3, wherein the flight path is presented to a user as the reduced area.

(Appendix 5)
The image generation means receives the terrain data from a storage device preliminarily storing terrain data indicating the terrain in the target area, and the image generation means when viewed from a viewpoint and a line-of-sight direction determined based on the position and orientation of the HMD. The flight path display system according to any one of appendix 1 and appendix 2, wherein an image in which the terrain in the target area is drawn as a virtual object is generated.

(Appendix 6)
The image generation means obtains an intersection between a cone having a point on the reduced flight path as a vertex and the terrain in the reduced area, and outputs an image indicating the intersection to the HMD as a footprint of the flight path. The flight path display system according to any one of appendix 1 to appendix 5, characterized by:

(Appendix 7)
The image generation means receives the attribute data from a storage device that stores in advance attribute data indicating the point, area, and space attributes in the target area, and the viewpoint and line-of-sight direction determined based on the position and orientation of the HMD The flight path display system according to any one of appendix 1 to appendix 6, wherein an image in which a virtual object related to the attribute data is drawn is generated at or near a corresponding position in the field of view when viewed from above .

(Appendix 8)
A plurality of the HMDs;
The image generation means generates an image showing a field of view of the reduced space from a viewpoint and a line-of-sight direction determined based on the position and orientation of the HMD for each HMD, and outputs the generated image to the corresponding HMD. The flight path display system according to any one of appendix 1 to appendix 7.

(Appendix 9)
The user wears a head-mounted display device (HMD) on his head, reduces the target area, which is a space including terrain, at a predetermined scale, and reproduces it at a predetermined installation location. A reduced area presentation stage for presenting the reduced area to the user;
An HMD positioning step for measuring the position and orientation of the HMD relative to the reduced area;
A reduced flight path generation step for obtaining a reduced flight path that is a space curve in the reduced area corresponding to a flight path of the flying object in the target area;
Based on the position and orientation of the HMD obtained in the HMD positioning stage and the reduced flight path obtained in the reduced flight path generation stage, a view when the reduced area is viewed from the position and orientation is shown. A flight path display method comprising: an image generation step of generating an image and outputting the image to the HMD.

(Appendix 10)
It is held directly by a user's hand or indirectly via a holder or the like, and indicates the position of the flying object in the reduced area. A pointer to be moved,
The flight path display method according to claim 9, further comprising a step of measuring a position of a pointer moving in the reduced area as a position of a point on the reduced flight path.

(Appendix 11)
11. The appendix 9 or appendix 10, wherein the HMD includes a transmissive display, and a user can simultaneously view an image output by the image generation stage and a scene outside the HMD. Flight path display method.

(Appendix 12)
The reduced area presentation stage includes a terrain model obtained by reducing the terrain of the target area at the scale, and transmits the transmissive display, the terrain model installed at the installation location and a view of the space above it, The flight path display method according to appendix 11, wherein the reduced area is presented to a user.

(Appendix 13)
The image generation step receives the terrain data from a storage device preliminarily storing the terrain data indicating the terrain in the target region, and the image generation stage when viewed from a viewpoint and a line-of-sight direction determined based on the position and orientation of the HMD. The flight path display method according to any one of Supplementary Note 9 and Supplementary Note 10, wherein an image in which the terrain in the target region is drawn as a virtual object is generated.

(Appendix 14)
The image generation step obtains an intersection between a cone having a point on the reduced flight path as a vertex and the topography in the reduced area, and outputs an image indicating the intersection to the HMD as a footprint of the flight path. 14. The flight path display method according to any one of appendix 9 to appendix 13, characterized by:

(Appendix 15)
The image generation step receives the attribute data from a storage device preliminarily storing attribute data indicating point, area, and space attributes in the target area, and determines the viewpoint and line-of-sight direction based on the position and orientation of the HMD. The flight path display method according to any one of supplementary notes 9 to 14, wherein an image in which a virtual object related to the attribute data is drawn is generated at or near a corresponding position in the field of view when viewed from .

(Appendix 16)
A plurality of the HMDs;
The image generation step generates an image showing a field of view of the reduced space from the viewpoint and the line-of-sight direction determined based on the position and orientation of the HMD for each HMD, and outputs the generated image to the corresponding HMD. The flight path display method according to any one of appendix 9 to appendix 15.

(Appendix 17)
The user wears a head-mounted display device (HMD) on his head, reduces the target area, which is a space including terrain, at a predetermined scale, and reproduces it at a predetermined installation location. HMD positioning means for calculating the position and orientation of the HMD relative to the reduced area from the measured values;
Reduced flight path generation means for obtaining a reduced flight path corresponding to a flight path of the flying object in the target area, which is a space curve in the reduced area;
Based on the position and orientation of the HMD obtained by the HMD positioning means and the reduced flight path obtained by the reduced flight path generating means, a view when the reduced area is viewed from the position and orientation is shown. A program for causing a computer to function as image generation means for generating an image and outputting it to the HMD.

(Appendix 18)
It is held directly by a user's hand or indirectly via a holder or the like, and indicates the position of the flying object in the reduced area. 18. The program according to appendix 17, for causing a computer to function as means for calculating a position of a pointer to be moved from a measurement value as a position of a point on the reduced flight path.

(Appendix 19)
19. The appendix 17 or appendix 18, wherein the HMD includes a transmissive display, and a user can simultaneously view an image output from the image generation unit and a scene outside the HMD. Program.

(Appendix 20)
The reduced area presentation means includes a terrain model obtained by reducing the terrain of the target area at the scale, and transmits the transmissive display, the terrain model installed at the installation location and a view of the space above it. The program according to appendix 19, wherein the program is presented to the user as the reduced area.

(Appendix 21)
The image generation means receives the terrain data from a storage device preliminarily storing terrain data indicating the terrain in the target area, and the image generation means when viewed from a viewpoint and a line-of-sight direction determined based on the position and orientation of the HMD. 19. The program according to any one of appendix 17 and appendix 18, wherein an image in which the terrain in the target area is drawn as a virtual object is generated.

(Appendix 22)
The image generation means obtains an intersection between a cone having a point on the reduced flight path as a vertex and the terrain in the reduced area, and outputs an image indicating the intersection to the HMD as a footprint of the flight path. The program according to any one of appendix 17 to appendix 21, characterized by:

(Appendix 23)
The image generation means receives the attribute data from a storage device that stores in advance attribute data indicating the point, area, and space attributes in the target area, and the viewpoint and line-of-sight direction determined based on the position and orientation of the HMD The program according to any one of appendix 17 to appendix 22, wherein an image in which a virtual object related to the attribute data is drawn is generated at or near a corresponding position in the field of view when viewed from above.

(Appendix 24)
A plurality of the HMDs;
The image generation means generates an image showing a field of view of the reduced space from a viewpoint and a line-of-sight direction determined based on the position and orientation of the HMD for each HMD, and outputs the generated image to the corresponding HMD. The program according to any one of supplementary notes 17 to 23.

DESCRIPTION OF SYMBOLS 1 Flight path display system 2 Reduction | restoration area presentation part 3, 12 Head-mounted display apparatus (Head Mounted Display, HMD)
4, 13 HMD positioning unit 5 Image processing device 10, 40 Flight path creation system 11 Diorama 14 Display operation unit 15 Aircraft model / diorama position recognition device 16, 17 Camera 18 Aircraft model 19 Communication interface device 20 Data communication network 21-25, 44 Database 43 Aircraft model / virtual space position recognition device 41a Augmented reality display marker

Claims (9)

A head-mounted display device (HMD) that a user wears on the head, and
A reduced area presentation means for reducing a target area, which is a space including terrain, at a predetermined scale ratio and presenting a reduced area, which is a space reproduced at a predetermined installation location, to the user;
HMD positioning means for measuring the position and orientation of the HMD relative to the reduced area;
Reduced flight path generation means for obtaining a reduced flight path corresponding to a flight path of the flying object in the target area, which is a space curve in the reduced area;
Based on the position and orientation of the HMD obtained by the HMD positioning means and the reduced flight path obtained by the reduced flight path generating means, a view when the reduced area is viewed from the position and orientation is shown. Image generating means for generating an image and outputting it to the HMD ;
It is held directly by a user's hand or indirectly via a holder or the like, and indicates the position of the flying object in the reduced area. A pointer to be moved,
Means for measuring a position of a pointer moving in the reduced area as a position of a point on the reduced flight path .   2. The flight path display system according to claim 1, wherein the HMD includes a transmissive display, and a user can simultaneously view an image output from the image generation unit and a scene outside the HMD. . The reduced area presentation means includes a terrain model obtained by reducing the terrain of the target area at the scale, and transmits the transmissive display, the terrain model installed at the installation location and a view of the space above it. The flight path display system according to claim 2 , wherein the reduced area is presented to a user.   The image generation means receives the terrain data from a storage device preliminarily storing terrain data indicating the terrain in the target area, and the image generation means when viewed from a viewpoint and a line-of-sight direction determined based on the position and orientation of the HMD. The flight path display system according to claim 1, wherein an image in which the terrain in the target area is drawn as a virtual object is generated. The image generation means obtains an intersection between a cone having a point on the reduced flight path as a vertex and the terrain in the reduced area, and outputs an image indicating the intersection to the HMD as a footprint of the flight path. Flight path display system according to any one of claims 1 to 4, characterized in. The image generation means receives the attribute data from a storage device that stores in advance attribute data indicating the point, area, and space attributes in the target area, and the viewpoint and line-of-sight direction determined based on the position and orientation of the HMD in the appropriate location or the vicinity thereof in the field of view when viewed from the flight path according to any one of claims 1 to 5, characterized in that to generate the image drawn virtual objects related to the attribute data Display system. A plurality of the HMDs;
The image generation means generates an image showing a field of view of the reduced area from a viewpoint and a line-of-sight direction determined based on the position and orientation of the HMD for each HMD, and outputs the generated image to the corresponding HMD. The flight path display system according to any one of claims 1 to 6 . The user wears a head-mounted display device (HMD) on his head, reduces the target area, which is a space including terrain, at a predetermined scale, and reproduces it at a predetermined installation location. A reduced area presentation stage for presenting the reduced area to the user;
An HMD positioning step for measuring the position and orientation of the HMD relative to the reduced area;
A reduced flight path generation step for obtaining a reduced flight path that is a space curve in the reduced area corresponding to a flight path of the flying object in the target area;
Based on the position and orientation of the HMD obtained in the HMD positioning stage and the reduced flight path obtained in the reduced flight path generation stage, a view when the reduced area is viewed from the position and orientation is shown. Generating an image and outputting it to the HMD ; and
A pointer indicating the position of the flying object in the reduced area is held in the reduced area in accordance with the user's operation by being held directly by the user's hand or indirectly through a holder or the like. Moving the stage,
Measuring a position of a pointer moving in the reduced area as a position of a point on the reduced flight path . The user wears a head-mounted display device (HMD) on his head, reduces the target area, which is a space including terrain, at a predetermined scale, and reproduces it at a predetermined installation location. HMD positioning means for calculating the position and orientation of the HMD relative to the reduced area from the measured values;
Reduced flight path generation means for obtaining a reduced flight path corresponding to a flight path of the flying object in the target area, which is a space curve in the reduced area;
Based on the position and orientation of the HMD obtained by the HMD positioning means and the reduced flight path obtained by the reduced flight path generating means, a view when the reduced area is viewed from the position and orientation is shown. Image generating means for generating an image and outputting it to the HMD ;
The position of the pointer that is held directly by the user's hand or indirectly through a holder or the like and indicates the position of the flying object in the reduced area is set as the position of the point on the reduced flight path. A program for causing a computer to function as a means for measuring .

Images