JP2022187788A - Virtual environment providing system and virtual environment providing method - Google Patents

Abstract

To provide a virtual environment providing system and a virtual environment providing method capable of setting virtual environment that is capable of giving a passenger of a mobile object a high sense of immersion.SOLUTION: A virtual environment providing system 10 that displays a virtual environment video representing virtual environment different from real environment in which a mobile object 1 is moving on a display apparatus 70 used by a passenger of the mobile object 1, comprises: a mobile object behavior recognition unit 24 that recognizes behavior of the mobile object 1; a virtual environment video generation unit 23 that generates a virtual environment video based on the behavior of the mobile object 1; and a video reproduction control unit 25 that reproduces the virtual environment video and displaying it on the display apparatus 70.SELECTED DRAWING: Figure 2

Description

The present invention relates to a virtual environment providing system and a virtual environment providing method.

Conventionally, when a vehicle occupant selects a route that matches a virtual theme, and the vehicle travels along the selected route, the virtual route is adjusted according to the actual road conditions (curves, uphills and downhills). A system has been proposed that displays an image of a vehicle traveling in an environment on virtual reality glasses worn by a passenger (see, for example, Patent Document 1).
In addition, content items selected based on the autonomous vehicle passenger profile and vehicle information (speed and direction of travel, location and route information, maps and points of interest, real-time traffic information) are augmented into images, and A system for displaying images on a display device has been proposed (see Patent Document 2, for example).

Japanese Patent Publication No. 2020-537258 JP 2017-204261 A

As in the above-mentioned conventional system, by displaying images of a virtual environment different from the environment in which the vehicle is actually moving in a moving object such as a vehicle, passengers can enjoy a simulated sensation while moving. can give. And it is assumed that by increasing the sense of immersion of the passenger in the virtual environment, the enjoyment felt by the passenger will be increased.
The present invention has been made in view of such a background, and provides a virtual environment providing system and a virtual environment providing method that can set a virtual environment that can give a high sense of immersion to a passenger of a mobile object. intended to provide

As a first aspect for achieving the above object, a virtual environment for displaying a virtual environment image representing a virtual environment different from the real environment in which the moving object is moving on a display device used by a passenger of the moving object. A provision system, comprising: a moving object behavior recognition unit that recognizes behavior of the moving object; a virtual environment image generation unit that generates the virtual environment image based on the behavior of the moving object; and reproduces the virtual environment image. and a video playback control unit for causing the display device to display the virtual environment providing system.

The virtual environment providing system includes a planned route recognition unit that recognizes a planned route from the starting position of the moving object to the destination position, and the virtual environment image generating unit generates a virtual environment based on the behavior of the moving object and the planned route , the virtual environment image may be generated.

In the virtual environment providing system, a moving body behavior estimating unit for estimating a position where a predetermined behavior assumed to be exhibited by the moving body when the moving body moves along the planned route and the occurrence of the predetermined behavior. and a moving object position recognizing unit that recognizes the position of the moving object, and the virtual environment image generating unit includes a predetermined behavior image according to the aspect of the predetermined behavior estimated by the moving object behavior estimating unit. , the virtual environment image is generated in advance, and the image reproduction control unit detects the position of the moving object recognized by the moving object position recognition unit and the occurrence position of the predetermined behavior estimated by the moving object behavior estimation unit. The predetermined behavior video may be played back and displayed on the display device when the distance from the player becomes equal to or less than a predetermined distance.

In the above virtual environment providing system, the video playback control unit suspends playback of the virtual environment video when the moving body stops moving while the moving body is playing back the virtual environment video. Further, when the moving object starts to move before a predetermined time elapses after the moving object stops, the virtual environment image may be reproduced from the interrupted point.

In the virtual environment providing system described above, the virtual environment image generation unit generates a plurality of game-compatible images based on a stage or a play mode included in predetermined game content. The virtual environment image may be generated by switching according to the above.

In the virtual environment providing system, the moving body behavior recognition unit may recognize lateral acceleration or vertical acceleration of the moving body as the behavior of the moving body.

As a second aspect for achieving the above object, a virtual environment image expressing a virtual environment different from the real environment in which the mobile body is moving is displayed on a display device used by a passenger of the mobile body. a computer-executed virtual environment providing method, comprising: a moving object behavior recognition step of recognizing the behavior of the moving object; and a virtual environment image of generating the virtual environment image based on the behavior of the moving object. and a video display control step of reproducing the virtual environment video and displaying it on the display device.

According to the above-described virtual environment providing system, by generating a virtual environment image based on the behavior of the mobile object, it is possible to set a virtual environment that can give a high sense of immersion to the passengers of the mobile object. .

FIG. 1 is an explanatory diagram of the configuration of a vehicle equipped with a virtual environment providing system. FIG. 2 is a configuration diagram of the virtual environment providing system. FIG. 3 is an explanatory diagram showing the correspondence between the areas of the planned route of the vehicle and the stages of the game content. FIG. 4 is a flowchart of processing for generating a virtual environment video. FIG. 5 is a flowchart of processing for reproducing a virtual environment video.

[1. Viewing Mode of Virtual Environment Video]
A manner in which passengers U1, U2, and U3 of vehicle 1 view virtual environment images will be described with reference to FIG. The vehicle 1 can travel autonomously, and passengers U1, U2, and U3 are wearing head-mounted displays 70a, 70b, and 70c with headphones, respectively. Passenger U1 in the driver's seat removes head-mounted display 70a when operating vehicle 1. FIG.

The vehicle 1 includes a front camera 59a, a right side camera 59b, a left side camera 59c, and a rear camera 59d as cameras for capturing images of the surroundings of the vehicle 1. The vehicle 1 also includes a communication unit 50 , a display 51 , speakers 52 a and 52 b , a microphone 53 , and a radar 60 that detects objects existing around the vehicle 1 . Hereinafter, the head mounted displays 70a, 70b, and 70c are also collectively referred to as the head mounted display 70, and the front camera 59a, the right side camera 59b, the left side camera 59c, and the rear camera 59d are also collectively referred to as the camera 59. Passengers U1, U2, and U3 are also collectively referred to as passenger U.

The communication unit 50 wirelessly communicates with the head mounted displays 70a, 70b and 70c. Also, the communication unit 50 communicates with the information providing server 210 via the communication network 200 . The information providing server 210 is a computer system configured by a processor, memory, etc. (not shown). Furthermore, the communication unit 50 communicates directly or via the communication network 200 with the communication terminal 100 used by any of the passengers U1, U2, U3.

The virtual environment providing system 10 is based on images captured by the cameras 59a to 59d, the positions of objects detected by the radar 60, the behavior of the vehicle 1 recognized based on detection information from sensors described later, and the like. generates a virtual environment video that expresses a virtual environment different from the real environment in which the is moving. Then, the virtual environment providing system 10 reproduces the virtual environment video and transmits the reproduction data of the virtual environment video to the head mounted displays 70a, 70b, and 70c, thereby providing the virtual environment video to the head mounted displays 70a, 70b, and 70c. display.

This allows the passengers U1, U2, and U3 to enjoy traveling to their destination while viewing the virtual environment video. In this embodiment, the virtual environment providing system 10 uses car racing game content to generate and reproduce a virtual environment video 90 that expresses the world view of car racing. Passengers U1, U2, and U3 can be immersed in a feeling as if they were participating in a car race.

The virtual environment images displayed on the head-mounted displays 70a, 70b, and 70c may be the same, or may be individually different. Passengers U1, U2, and U3 may be allowed to participate in the game by voice operation, switch operation, or the like, and the virtual environment image may be changed according to the operation.

Also, instead of the head-mounted displays 70a, 70b, 70c, the display 51 of the vehicle 1 may reproduce the virtual environment image, and the reproduced sound may be output from the speakers 52a, 52b. Alternatively, for example, a HUD (Head-Up Display) may be used to display a virtual environment image on a windshield (not shown). Also, the virtual environment image may be displayed on the entire vehicle interior by displaying the virtual environment image on the windshield and a plurality of other windows (side window, rear window, sunroof, etc.).

[2. Configuration of virtual environment provision system]
The configuration of the virtual environment providing system 10 will be described with reference to FIGS. 2 and 3. FIG. The virtual environment providing system 10 is a control unit composed of a processor 20, a memory 30, an interface circuit (not shown), and the like.

The virtual environment providing system 10 includes a communication unit 50, a display 51, speakers 52a and 52b, a microphone 53, a speed sensor 54, a vertical communication unit 50, a display 51, speakers 52a and 52b, a microphone 53, and a vertical communication bus 5 according to communication specifications such as CAN (Controller Area Network). Acceleration sensor 55, lateral acceleration sensor 56, yaw rate sensor 57, raindrop sensor 58, cameras 59 (front camera 59a, right side camera 59b, left side camera 59c, rear camera 59d), radar 60, and car navigation system 40 are connected. . A car navigation system 40 includes a GNSS (Global Navigation Satellite System) sensor 41 for detecting the current position of the vehicle 1 and map data 42, and provides route guidance from a departure point (departure position) of the vehicle 1 to a destination point (destination position). to run.

The processor 20 of the virtual environment providing system 10 reads and executes the control program 31 stored in the memory 30 to obtain a planned route recognition unit 21, a moving body behavior estimation unit 22, a virtual environment video generation unit 23, a moving body behavior It functions as a recognition unit 24 and a video reproduction control unit 25 .

The processing executed by the virtual environment image generation unit 23 corresponds to the virtual environment image generation step in the virtual environment provision method of the present disclosure, and the processing executed by the moving object behavior recognition unit 24 corresponds to the virtual environment provision method of the present disclosure. corresponds to the mobile body behavior recognition step in . The processing executed by the video reproduction control unit 25 corresponds to the video reproduction control step in the virtual environment providing method of the present disclosure.

The scheduled route recognition unit 21 recognizes the scheduled route from the departure point to the destination point set by the car navigation system 40 . Note that when a scheduled route from a departure point to a destination point is set by a car navigation application installed in the communication terminal 100, the scheduled route may be recognized through communication with the communication terminal 100.

Here, 500 in FIG. 3 is a real environment map showing an example of a planned route from the current position (departure point) Ps of the vehicle 1 to the destination point Pd. The planned route in the map 500 of the real environment is a route in which the vehicle travels through a first unpaved area 501, a second mountain road area 502, a third highway area 503, and a fourth urban area 504 in this order.

Based on the map data 42, the moving object behavior estimation unit 22 estimates the occurrence point (occurrence position) and mode of the predetermined behavior assumed to be exhibited by the vehicle 1 when the vehicle 1 travels along the planned route. For example, when the vehicle 1 travels in the second area 502, the moving object behavior estimation unit 22 determines that the vehicle 1 has a high level of lateral acceleration (lateral G) and longitudinal acceleration when the vehicle 1 travels through successive corners. (Vertical G). Further, the moving body behavior estimation unit 22 estimates that when the vehicle 1 travels in the third area 503, it exhibits a high level of longitudinal acceleration at the point of entry into and exit from the expressway.

The virtual environment image generation unit 23 associates the planned route with the planned route in the real environment of the vehicle 1 recognized by the planned route recognition unit 21 and the behavior of the vehicle 1 estimated by the moving body behavior estimation unit 22. generate a virtual environment image. Here, 510 in FIG. 3 indicates an example of generating a virtual environment image corresponding to the scheduled route from the departure point Ps to the destination point Pd of the vehicle 1 on the map 500 of the real environment.

A virtual environment image generation unit 23 generates a virtual environment image by setting a race course corresponding to a planned route in a real environment map 500 using a virtual environment map 510 reflecting car racing game content. . Specifically, the virtual environment image generation unit 23 assigns the first stage 511 between uneven areas to the first unpaved area, and assigns the second stage 512 of the mountain section to the second area 502 of the mountain road. Assignment, assigning the third stage 513 of the high-speed section to the third area 503 of the highway, and assigning the fourth stage 514 between urban districts to the fourth area 504 of the urban area.

Then, the virtual environment image generation unit 23 creates a race course from START to GOAL on the virtual environment map 510 corresponding to the scheduled route from the starting point Ps to the destination point Pd on the real environment map 500. A running image is generated as a virtual environment image. The virtual environment image generator 23 stores the generated virtual environment image data 32 in the memory 30 .

In addition, the virtual environment image generation unit 23 generates a virtual environment image so that the predetermined behavior image corresponding to the mode of the predetermined behavior is displayed at the point where the predetermined behavior estimated by the moving object behavior estimation unit 22 occurs. . For example, for the second stage 512 of the mountainous section, the virtual environment image generation unit 23 causes the virtual environment image generation unit 23 to display a predetermined behavior image in which the vehicle 1 approaches the side wall at a point where the vehicle 1 is estimated to exhibit lateral acceleration. Generate an image video.

In addition, the virtual environment image generation unit 23 changes the virtual environment image according to the behavior of the vehicle 1 recognized by the mobile object behavior recognition unit 24 while the vehicle 1 is traveling, traffic conditions on the planned route, weather conditions, and the like. A variation pattern video for the virtual image is generated, and virtual image video data 32 including the variation pattern video is stored in the memory 30 .

The mobile object behavior recognition unit 24 recognizes the behavior of the vehicle 1 while traveling based on information detected by the speed sensor 54, longitudinal acceleration sensor 55, lateral acceleration sensor 56, and yaw rate sensor 57, an image captured by the camera 59, and an object detected by the radar 60. Recognize based on the detection status of the object. When the vehicle 1 travels along the planned route, the image reproduction control unit 25 reproduces the virtual environment image of the portion corresponding to the travel point and causes the head mounted display 70 to display the virtual environment image. For example, referring to FIG. 3 , when the vehicle 1 is traveling in the second area 502 , the image reproduction control section 25 reproduces the virtual environment image of the second stage 512 .

Further, the image reproduction control unit 25 reproduces the predetermined behavior image at the timing when the distance between the current position of the vehicle 1 and the point where the predetermined behavior occurs estimated by the moving object behavior estimation unit 22 becomes equal to or less than the predetermined distance. . Thereby, the change in the behavior of the vehicle 1 experienced by the passenger U can be synchronized with the change in the virtual environment image.

[3. Generation processing of virtual environment video]
The execution procedure of the virtual environment video generation process will be described according to the flowchart shown in FIG. When the destination point is set by the car navigation system 40 or the communication terminal 100 in step S1 of FIG. 4, the virtual environment image generator 23 advances the process to step S2 to start the virtual environment image generation process.

In step S<b>2 , the planned route recognition unit 21 recognizes the planned route from the departure point to the destination point set by the car navigation application installed in the car navigation system 40 or the communication terminal 100 . In the next step S3, the virtual environment image generator 23 extracts game content that matches the planned route. The virtual environment image generation unit 23 determines game content to be extracted according to the environment of the travel area along the planned route, the preference of the passenger, the selection operation of the passenger, and the like. The virtual environment video generation unit 23 downloads and acquires game content data from the information providing server 210, the communication terminal 100, and the like.

In subsequent step S4, the virtual environment image generator 23 determines the stages of the game content to be associated with each area of the planned route, as described above with reference to FIG. Generate a virtual environment video that includes behavior video. In the next step S5, the virtual environment image generation unit 23 detects the behavior of the vehicle 1 recognized by the moving body behavior recognition unit 24 and the behavior of the camera 59 when the vehicle 1 travels along the scheduled route for each area of the scheduled route. A change pattern image for changing the virtual environment image is generated according to the captured image or surrounding objects recognized by the radar 60, traffic information and weather information provided from the information providing server 210, and the like.

In the next step S6, the virtual environment image generator 23 generates a virtual environment image including the change pattern image generated in step S5, and stores the data 32 of the virtual environment image in the memory 30. FIG.

[4. Playback processing of virtual environment video]
The execution procedure of the virtual environment video reproduction process will be described according to the flowcharts shown in FIGS. 4 and 5. FIG. In step S7 of FIG. 4, when the video reproduction control unit 25 recognizes that the vehicle 1 has started traveling due to a change in the current position of the vehicle 1 detected by the GNSS sensor 41, the process proceeds to step S8 of FIG. The process proceeds to start playing back the virtual environment video.

At step S<b>8 in FIG. 5 , the video reproduction control unit 25 recognizes the current position of the vehicle 1 based on the detection signal from the GNSS sensor 41 . In subsequent step S9, the video reproduction control unit 25 reproduces the virtual environment video representing the stage of the game content associated with the area to which the current position of the vehicle 1 belongs. For example, in the example of FIG. 3, when the current position of the vehicle 1 is the highway in the third area 503, the video reproduction control unit 25 reproduces the virtual environment video representing the high-speed section of the third stage of the game content. . As a result, a virtual environment image that matches the sense of speed experienced by the passenger U is displayed on the head-mounted display 70, and it is expected that the passenger U will feel more immersed in the virtual world of the car race. .

Further, in step S9, the video reproduction control unit 25 determines that the distance between the current position of the vehicle 1 and the point where the predetermined behavior occurs estimated by the moving object behavior estimation unit 22 is equal to or less than the predetermined distance. Sometimes, a predetermined behavior video corresponding to the predetermined behavior is reproduced.

In the next step S10, the video reproduction control unit 25 recognizes from the behavior of the vehicle 1 recognized by the moving body behavior recognition unit 24, the objects around the vehicle 1 detected by the radar 60, and the images captured by the surrounding camera 59. types of objects around the vehicle 1, road traffic information (congestion information, accident information, traffic regulation information, etc.) provided by the information providing server 210, weather information provided by the information providing server 210, raindrop sensor 58 Recognize rainfall information and the like detected by

In the following step S11, the video reproduction control unit 25 generates a change pattern video of the virtual environment image to be reproduced based on the behavior of the vehicle 1, objects and their types around the vehicle 1, road traffic information, weather information, and rainfall information. to:
(1) Change according to the behavior of the vehicle 1 .
For example, when the vehicle 1 exhibits a lateral acceleration of a predetermined level or more, the image reproduction control unit 25 reproduces a change pattern image in which the screen sharply shifts in the horizontal direction. Further, the image reproduction control unit 25 reproduces a change pattern image in which the racing vehicle C descends rapidly like a roller coaster when the vehicle 1 exhibits a forward longitudinal acceleration of a predetermined level or more.

(2) Changes according to surrounding objects and their types.
For example, when other vehicles traveling around the vehicle 1 are recognized, the image reproduction control unit 25 reproduces a change pattern image displaying images of other vehicles participating in the car race. Further, the image reproduction control unit 25 reproduces a change pattern image in which, when the vehicle 1 is stopped at a signal, the race vehicle C enters the pit and performs pit work such as refueling and tire replacement.

(3) Changes according to road traffic information.
For example, when recognizing a traffic jam, the image reproduction control unit 25 reproduces a change pattern image in which the vehicle is running slowly led by the safety car due to an accident during the race. Further, the video reproduction control unit 25 reproduces a change pattern video that displays the road construction site as a pseudo image of an obstacle when the vehicle 1 travels around the road construction site.

(4) Change of virtual image video according to weather information.
For example, when a strong wind is blowing, the image reproduction control unit 25 reproduces a change pattern image in which the racing car C swings left and right. In addition, when it is raining, the image reproduction control unit 25 reproduces a change pattern image in which the racing vehicle C runs while slipping in a puddle.

In step S<b>12 , the video reproduction control unit 25 determines whether the vehicle 1 has stopped based on the change in the current position of the vehicle 1 detected by the GNSS sensor 41 . Then, when the vehicle 1 has stopped, the video reproduction control unit 25 advances the process to step S13. On the other hand, when the vehicle 1 is not stopped, the video reproduction control section 25 advances the process to step S8 and executes the processes of steps S8 to S11 again.

In step S13, the video reproduction control unit 25 suspends reproduction of the virtual environment video. In the next step S14, the video reproduction control unit 25 determines whether or not the vehicle 1 has arrived at the destination point Pd from the current position of the vehicle 1 detected by the GNSS sensor 41. Then, when the vehicle 1 has arrived at the destination point Pd, the video reproduction control unit 25 advances the processing to step S15 and ends the reproduction of the virtual image video.

On the other hand, when the vehicle 1 has not arrived at the destination point Pd, the video reproduction control unit 25 advances the process to step S20 and starts the stop time timer. The set time of the stop time timer is set to several hours, for example. Through the loop processing of subsequent steps S21 and S30, the video reproduction control unit 25 controls the vehicle 1 according to the change in the current position of the vehicle 1 detected by the GNSS sensor 41 in step S21 until the stop time timer expires in step S30. 1 determines whether or not it has resumed running.

Then, when the vehicle 1 resumes running, the image reproduction control unit 25 advances the process from step S21 to step S22, resumes reproduction of the virtual environment image from the interrupted point, and advances the process to step S8. As a result, for example, when the vehicle 1 is parked in the parking area for about an hour and the passenger U takes a break, the virtual environment video is reproduced from the interrupted point, and the passenger U can enjoy the game content based on the virtual environment video. You can continue to enjoy the world of

On the other hand, when the stop time timer times out in step S30, the video reproduction control unit 25 advances the process to step S15 to end the reproduction of the virtual environment video. As a result, for example, when the passenger U stays for one night at an accommodation facility on the way to the destination point Pd, the vehicle 1 starts running on the next day, and the virtual environment image of the new course is displayed. is started, and the passenger U can enjoy the virtual world of the car race on the new course.

[5. Other embodiments]
In the above-described embodiment, the vehicle 1 traveling on the ground surface is illustrated as the mobile object of the present disclosure, but the mobile object of the present disclosure may be another type of mobile object such as an aircraft or a ship.

In the above embodiment, the virtual environment image generation unit 23 generates a virtual environment image that expresses the virtual environment of car racing game content. An image may be generated. Further, in the above-described embodiment, the areas of the real environment and the stages of the game are associated, but the real environment and the play mode of the game may be associated with each other. For example, when there are no other vehicles in the vicinity of the vehicle 1, the play mode may be for solo driving, and when there are other vehicles in the vicinity of the vehicle 1, the play mode may be for racing. In addition to game content, a virtual environment video representing a virtual environment corresponding to image content such as movies and TV dramas may be generated.

Also, a virtual environment image may be generated based on an actual image of another area different from the actual environment in which the vehicle 1 travels. For example, while the vehicle 1 is traveling on a mountain road, a virtual environment image may be generated using an image of a scenic mountainous area existing in another area. Also, an animation image or the like may be superimposed on an image of the real environment in which the vehicle 1 is traveling to generate a virtual environment image.

In the above-described embodiment, the virtual environment image generator 23 generated the virtual image in advance in correspondence with the scheduled route from the departure point Ps of the vehicle 1 to the destination point Pd. As another embodiment, the virtual video image may be generated in real time according to the behavior of the vehicle 1 recognized by the mobile body behavior recognition unit 24, regardless of the planned route. For example, when the vehicle 1 shows a high level of longitudinal acceleration and the speed suddenly increases, the virtual environment image generation unit 23 switches to a game compatible image based on a high-speed running stage included in car racing game content. , may generate a virtual environment image.

In the above-described embodiment, the moving object behavior estimation unit 22 is provided to generate a virtual environment image based on the predicted behavior of the vehicle 1 when traveling on the planned route. good.

In the above embodiment, the configuration in which the virtual environment providing system 10 is installed in the vehicle 1 is shown, but part or all of the configuration of the virtual environment providing system 10 is configured by a system outside the vehicle 1 such as the information providing server 210. You may For example, the information providing server 210 is provided with the virtual environment image generation unit 23, and the virtual environment image data generated by the virtual environment image generation unit 23 is transmitted from the information providing server 210 to the vehicle 1. The image reproduction control unit 25 may be configured to reproduce the virtual environment image.

1 and 2 are schematic diagrams showing the configurations of the vehicle 1 and the virtual environment providing system 10, divided according to main processing contents, in order to facilitate understanding of the present invention. The virtual environment providing system 10 may be configured with other divisions. Also, the processing of each component may be executed by one hardware unit or may be executed by a plurality of hardware units. Also, the processing by each component shown in FIGS. 4 and 5 may be executed by one program or may be executed by a plurality of programs.

[6. Configuration supported by the above embodiment]
The above embodiment is a specific example of the following configuration.

(Configuration 1) A virtual environment providing system for displaying, on a display device used by a passenger of a mobile object, a virtual environment image representing a virtual environment different from the real environment in which the mobile object is moving, A moving object behavior recognition unit that recognizes behavior of a body, a virtual environment image generation unit that generates the virtual environment image based on the behavior of the moving object, and reproduces the virtual environment image and displays it on the display device. A virtual environment providing system comprising a video playback control unit.
According to the virtual environment providing system of configuration 1, by generating a virtual environment image based on the behavior of the moving object, a virtual environment that can give a high sense of immersion to the passengers of the moving object is set. can be done.

(Arrangement 2) A planned route recognition unit for recognizing a planned route from the starting position of the moving body to the destination position is provided, and the virtual environment image generating unit generates the virtual environment based on the behavior of the moving body and the planned route. The virtual environment providing system according to Configuration 1, which generates an environment image.
According to the virtual environment providing system of Configuration 2, by generating a virtual environment image based on the route actually traveled by the moving body, a virtual environment image corresponding to the situation of the real environment assumed to be felt by the passenger during movement is generated. Environmental images can be generated.

(Configuration 3) A moving body behavior estimating unit for estimating a position of occurrence of a predetermined behavior assumed to be exhibited by the moving body when the moving body moves along the planned route and a mode of the predetermined behavior; a moving object position recognition unit that recognizes a position of a body, wherein the virtual environment image generating unit includes a predetermined behavior image according to the mode of the predetermined behavior estimated by the moving object behavior estimation unit; An environment image is generated in advance, and the image reproduction control unit determines the distance between the position of the moving object recognized by the moving object position recognition unit and the position of occurrence of the predetermined behavior estimated by the moving object behavior estimation unit. The virtual environment providing system according to configuration 2, in which the predetermined behavior image is played back and displayed on the display device when the distance becomes equal to or less than the predetermined distance.
According to the virtual environment providing system of configuration 3, the predetermined behavior video generated based on the aspect of the predetermined behavior assumed to be exhibited by the mobile body when moving along the planned route is displayed at the point where the mobile body exhibits the predetermined behavior. Plays when approached. This makes it possible for the passenger to feel a virtual environment that reflects the behavior of the moving body, thereby enhancing the sense of immersion in the virtual environment.

(Configuration 4) The video playback control unit interrupts playback of the virtual environment video when the moving body stops moving while the moving body is playing back the virtual environment video, and any one configuration from configuration 1 to configuration 3, wherein when the moving object starts moving before a predetermined time elapses after the moving object stops, the virtual environment image is reproduced from the interrupted point. The described virtual environment providing system.
According to the virtual environment providing system of Configuration 4, when the moving object is stopped only for a short time, the virtual environment video is reproduced from the interrupted point, thereby continuing the sense of movement in the virtual environment to the passenger. can be generated.

(Arrangement 5) The virtual environment image generation unit generates a plurality of game-compatible images based on a stage or a play mode included in predetermined game content, according to the behavior of the moving object recognized by the moving object behavior recognition unit. The virtual environment providing system according to any one of configurations 1 to 4, wherein the virtual environment video is generated by switching.
According to the virtual environment providing system of configuration 5, it is possible to make the passenger feel as if he or she is moving in the virtual world of the game content, thereby increasing the passenger's sense of immersion in the virtual environment.

(Configuration 6) The virtual environment providing system according to any one of Configurations 1 to 5, wherein the mobile body behavior recognition unit recognizes lateral acceleration or vertical acceleration of the mobile body as the behavior of the mobile body. .
According to the virtual environment providing system of configuration 6, the passenger can feel the lateral acceleration or the longitudinal acceleration of the mobile body, which is assumed to be easily felt by the passenger, among the behavior of the mobile body. It is possible to generate a virtual environment image that matches the behavior of the moving object.

(Configuration 7) Executed by a computer to display a virtual environment image representing a virtual environment different from the real environment in which the mobile body is moving on a display device used by a passenger of the mobile body A virtual environment providing method, comprising a moving object behavior recognition step for recognizing behavior of the moving object, a virtual environment image generating step for generating the virtual environment image based on the behavior of the moving object, and the virtual environment image. and a video display control step of reproducing and displaying on the display device.
By executing the virtual environment providing method of configuration 7 by a computer, the same effects as those of the virtual environment providing system of configuration 1 can be obtained.

REFERENCE SIGNS LIST 1 vehicle (moving object), 10 virtual environment providing system, 20 processor, 21 planned route recognition unit, 22 moving object behavior estimating unit, 23 virtual environment image generating unit, 24 moving object behavior recognizing unit, 25 Video playback control unit 30 Memory 31 Control program 40 Car navigation system 50 Communication unit 51 Display 52a, 52b Speaker 53 Microphone 54 Speed sensor 55 Longitudinal acceleration sensor 56... Lateral acceleration sensor, 57... Yaw rate sensor, 58... Raindrop sensor, 59... Surrounding camera, 60... Radar, 70... Head mounted display, 100... Communication terminal, 200... Communication network, 210... Information providing server.

Claims (7)

A virtual environment providing system for displaying, on a display device used by a passenger of a mobile object, a virtual environment image representing a virtual environment different from the real environment in which the mobile object is moving,
a moving object behavior recognition unit that recognizes the behavior of the moving object;
a virtual environment image generation unit that generates the virtual environment image based on the behavior of the moving object;
a video reproduction control unit that reproduces the virtual environment video and displays it on the display device;
A virtual environment providing system. a planned route recognition unit for recognizing a planned route from the starting position of the moving body to the destination position;
2. The virtual environment providing system according to claim 1, wherein the virtual environment image generation unit generates the virtual environment image based on the behavior of the moving object and the planned route. a moving body behavior estimation unit that estimates a position where a predetermined behavior assumed to be exhibited by the moving body when the moving body moves along the planned route and a mode of the predetermined behavior;
a moving body position recognition unit that recognizes the position of the moving body;
with
The virtual environment image generation unit generates in advance the virtual environment image including the predetermined behavior image corresponding to the mode of the predetermined behavior estimated by the moving object behavior estimation unit,
The video reproduction control unit is configured such that the distance between the position of the moving object recognized by the moving object position recognition unit and the position of occurrence of the predetermined behavior estimated by the moving object behavior estimation unit is equal to or less than a predetermined distance. 3. The virtual environment providing system according to claim 2, wherein the predetermined behavior video is reproduced and displayed on the display device when the virtual environment is activated. When the moving object stops moving while the moving object is reproducing the virtual environment image, the video reproduction control unit interrupts the reproduction of the virtual environment image, and the moving object stops. 4. The virtual environment image according to any one of claims 1 to 3, wherein when the moving object starts to move before a predetermined time elapses from the point in time when the virtual environment video image was Environment provision system. The virtual environment image generation unit switches a plurality of game-compatible images based on the stage or play mode included in predetermined game content according to the behavior of the moving object recognized by the moving object behavior recognition unit, 5. The virtual environment providing system according to any one of claims 1 to 4, wherein said virtual environment image is generated. 6. The virtual environment providing system according to any one of claims 1 to 5, wherein the mobile body behavior recognition unit recognizes lateral acceleration or vertical acceleration of the mobile body as the behavior of the mobile body. A virtual environment providing method executed by a computer for displaying, on a display device used by a passenger of a mobile object, a virtual environment image expressing a virtual environment different from the real environment in which the mobile object is moving and
a moving object behavior recognition step for recognizing the behavior of the moving object;
a virtual environment image generation step of generating the virtual environment image based on the behavior of the moving object;
a video display control step of reproducing the virtual environment video and displaying it on the display device;
virtual environment provision method including

Images