JP2020020987A - In-car system - Google Patents

Abstract

To provide an in-car system capable of acquiring video and audio privately even when a passenger is present.SOLUTION: An in-car system 10 includes: a monitor camera 20 and a body sensor 25 for detecting the body information emitted by a specific occupant in the cabin; a pair of smart glasses 26; and a speaker 24 that outputs audio for a specific occupant. In the in-car system 10, an information generation center 14 generates a piece of useful information suitable for the specific occupant on the basis of the body information. A control ECU 30 is configured to control a speaker 24 to output sound related to the useful information, and the pair of smart glasses 26 to display a display related to the useful information superimposing on the landscape inside and outside the vehicle cabin.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an in-vehicle system capable of providing occupants with information by voice and video.

  Patent Literature 1 discloses an operation unit in which operation switches are arranged, a display unit capable of stereoscopic viewing with naked eyes, an imaging unit that captures a user's hand operating the operation unit from a plurality of directions, and a parallax obtained from the imaging unit. There is disclosed an operating device including: a stereoscopic image generation unit configured to generate a stereoscopic image of a hand based on an image. This operation device is characterized in that an operation menu image indicating the positions of the operation switches and the functions of the operation switches in the operation unit and a stereoscopic image of the hand are combined and displayed on the display unit.

JP 2008-210359 A

  On the other hand, when video and audio are provided to an occupant using a stereoscopic display unit disclosed in Patent Document 1 and a general speaker provided in a vehicle cabin, in order to provide video and audio to a specific occupant, additional There is room for improvement.

  The present invention has been made in view of the above-described facts, and has as its object to provide an in-vehicle system capable of privately acquiring video and audio even when a passenger is present.

  The in-vehicle system according to claim 1, wherein a detection unit that detects physical information emitted by a specific occupant in the vehicle interior, an AR display worn by the specific occupant, and a voice for the specific occupant are output. A speaker, a generation unit that generates useful information suitable for the specific occupant based on the physical information detected by the detection unit, and a voice related to the useful information is output from the speaker, and And a control unit that controls the display so that the display is superimposed on the landscape inside and outside the vehicle interior and displayed on the AR display.

  Here, the “physical information emitted by the occupant” includes biological information such as pulse, brain waves, blood pressure, and heart rate, and occupant images such as facial expressions of the occupant. The “useful information suitable for the occupant” includes information for comfortably traveling inside the vehicle, information for safely driving the vehicle, and drop-in information around the vehicle.

  According to the first aspect of the present invention, the specific occupant can transmit his / her own feelings and intentions to the generation unit without being recognized by the passenger by the detection unit. Further, the generation unit generates useful information suitable for a specific occupant, and the specific occupant can obtain the useful information through an AR (Augmented Reality) display and a speaker without being realized by other passengers. it can. That is, according to the first aspect of the present invention, it is possible to privately obtain video and audio even when a passenger is present.

  ADVANTAGE OF THE INVENTION According to this invention, a video and audio | voice can be acquired privately even if a fellow passenger exists.

It is a figure showing the outline of the in-vehicle system concerning this embodiment. It is a block diagram showing composition of an in-vehicle system concerning this embodiment. It is an image figure of the vehicle interior in the vehicle interior system concerning this embodiment. It is an example of the image displayed on the smart glass in the in-vehicle system according to the present embodiment. It is an example of the image displayed on the smart glass in the in-vehicle system according to the present embodiment. It is an example of the image displayed on the smart glass in the in-vehicle system according to the present embodiment. It is a flowchart which shows an example of the flow of a process performed by the vehicle equipment of the in-vehicle system which concerns on this embodiment. It is a flowchart which shows an example of the flow of a process performed in the information generation center of the in-vehicle system which concerns on this embodiment. It is a flowchart which shows an example of the flow of a process performed in the information DB center of the in-vehicle system concerning this embodiment.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram schematically illustrating an in-vehicle system according to the present embodiment. FIG. 2 is a block diagram illustrating the configuration of the in-vehicle system according to the present embodiment.

  As shown in FIG. 1, an in-vehicle system 10 according to the present embodiment includes an on-vehicle device 12, a network 16, an information generation center 14 as a generation unit, and an information DB (database) center 15 mounted on a vehicle 50. . In the in-vehicle system 10 of the present embodiment, the vehicle-mounted device 12 acquires physical information of a specific occupant, and the information generation center 14 recognizes the occupant's emotions and intentions based on the physical information. Then, the information generation center 14 generates useful information suitable for the occupant, and the useful information is provided to a specific occupant through the speaker 24 and the smart glasses 26 connected to the vehicle-mounted device 12.

  In the in-vehicle system 10 according to the present embodiment, each of the on-vehicle device 12, the information generation center 14, and the information DB center 15 is connected via a network 16 including a mobile phone line or the like. The in-vehicle system 10 is a device connected to the network 16 and may include a vehicle-mounted device 52A installed in another vehicle 52 and a communication device 54A installed in the home 54.

  As shown in FIG. 2, the vehicle-mounted device 12 includes a vehicle periphery monitoring unit 18, a monitor camera 20 as a detection unit, a microphone 22, a speaker 24, a biological sensor 25 as a detection unit, a smart glass 26 as an AR display, A high-speed mobile communication device 28, a GPS device 31, and a vehicle-mounted device 32 are provided, and each is connected to a control ECU (Electronic Control Unit) 30 as a control unit.

  The vehicle periphery monitoring unit 18 monitors a situation around the vehicle 50. Specifically, the vehicle periphery monitoring unit 18 includes at least one of a camera, a radar (Radar), and a lidar (LIDER: Laser Imaging Detection and Ranging). The camera is provided, for example, on the indoor side above the windshield of the vehicle 50, and obtains an image outside the vehicle by capturing an external situation of the vehicle 50. The camera may be a monocular camera or a stereo camera. In the case of a stereo camera, it has two imaging units arranged to reproduce binocular parallax. The image outside the vehicle of the stereo camera also includes information on the depth direction. The radar detects an obstacle by transmitting a radio wave (for example, a millimeter wave) around the vehicle 50 and receiving a radio wave reflected by the obstacle. The rider transmits the light around the vehicle 50, receives the light reflected by the obstacle, measures the distance to the reflection point, and detects the obstacle. The camera, the rider, and the radar do not necessarily have to be provided in duplicate.

  The monitor camera 20 is provided in the vehicle interior, photographs the driver D and other occupants in the vehicle interior, and outputs the captured occupant image to the control ECU 30.

  The microphone 22 is provided in the vehicle interior, converts audio in the vehicle interior such as an occupant's voice into an electric signal, and outputs the signal to the control ECU 30 as voice information.

  The loudspeaker 24 is provided in the vehicle interior, and generates sound such as voice by converting voice information or the like transmitted from the control ECU 30 into physical vibration. The speaker 24 according to the present embodiment is a speaker that outputs sound to a specific occupant seated in a specific seat, and is, for example, a speaker having high directivity such as an ultrasonic speaker or embedded in each headrest 51A. A speaker is employed (see FIG. 3).

  The biological sensor 25 detects biological information such as a pulse, an electroencephalogram, a blood pressure, and a heart rate in order to detect the state of the occupant. The biological sensor 25 outputs the detected biological information to the control ECU 30. The sensor that measures the pulse and the brain wave of the biological sensor 25 may be formed integrally with the smart glass 26. Further, among the biological sensors 25, the sensors for measuring the pulse and the heart rate may be formed integrally with the sheet 51 (see FIG. 3).

  The smart glass 26 uses an AR (Augmented Reality) technology to display a virtual passenger V (see FIG. 3), which is a virtual occupant, and an image related to useful information in a superimposed manner on a scene inside and outside the vehicle interior. This is a wearable device of the head mounted display type. Specifically, the technology described in Japanese Patent No. 6033866 or HoloLens (https://www.microsoft.com/en-us/hololens) manufactured by Microsoft Corporation can be applied.

  The high-speed mobile communication device 28 is connected to a network 16 of a mobile phone line network or a public line network, and exchanges information with the information generation center 14 and the information DB center 15 connected to the network 16. The high-speed mobile communication device 28 transmits the occupant image captured by the monitor camera 20 and the biological information detected by the biological sensor 25 to the information generation center 14 and the information DB center 15 via the network 16 as physical information. In addition, the high-speed mobile communication device 28 uses the outside image captured by the camera or the like of the vehicle periphery monitoring unit 18 and position information acquired by a GPS device 31 to be described later as environmental information via the network 16 as the information generation center 14 or The information is transmitted to the information DB center 15. Further, the high-speed mobile communication device 28 receives information from the information generation center 14 and the information DB center 15 via the network 16.

  The GPS device 31 is a device that acquires position information of the vehicle 50, and may be an integrated device with the car navigation device or an independent device.

  The in-vehicle device 32 is a device mounted on the vehicle 50, and for example, various in-vehicle devices such as an air conditioner and an audio device can be applied.

  Further, the control ECU 30 performs various controls for communication based on communication with the information generation center 14 and the information DB center 15 connected to the network 16, and presents useful information for a specific occupant, The device 32 is controlled. The control ECU 30 is a microcomputer including a CPU, a ROM, and a RAM. When presenting information or controlling the in-vehicle device 32, the control ECU 30 controls display of a stereoscopic image and generation of sound as if the virtual passenger V was speaking, operating the in-vehicle device 32, and the like. .

  On the other hand, the information generation center 14 includes a voice recognition system 34, a facial expression recognition system 35, an emotion recognition system 36, an information control unit 37, and a communication device 38. The information generation center 14 is realized by, for example, a computer including a CPU, a ROM, a RAM, and the like.

  The voice recognition system 34 performs voice recognition by analyzing voice information included in the physical information received from the vehicle-mounted device 12 and recognizes the intention and emotion of the occupant by a known voice recognition technology.

  The facial expression recognition system 35 analyzes the occupant image included in the physical information received from the vehicle-mounted device 12 and recognizes the occupant's intention and emotion by using a well-known facial expression recognition technology.

  The emotion recognition system 36 analyzes the occupant's biological information included in the physical information received from the vehicle-mounted device 12 and recognizes the occupant's emotion.

  The information control unit 37 is suitable for a specific occupant based on the recognition result of intention and emotion in the voice recognition system 34, the recognition result of intention and emotion in the facial expression recognition system 35, and the recognition result of emotion in the emotion recognition system 36. Generate useful information. The information control unit 37 generates useful information by reflecting the occupant's preference and the current location of the vehicle obtained from the information DB center 15.

  The communication device 38 is connected to the network 16 of a mobile phone line network or a public line network, and can communicate with the vehicle-mounted device 12 and the information DB center 15 connected to the network 16.

  The information DB center 15 includes a personal information DB 40, a map information DB 41, a search unit 42, and a communication device 44. The information DB center 15 is realized by a computer having a CPU, a ROM, a RAM, and the like, for example, like the information generation center 14.

  The personal information DB 40 stores various types of information regarding the occupant as personal information. For example, a history of online settlement, credit card usage information, positional information associated with a smartphone carried by an occupant, topic information collected from a network such as the Internet, and the like are stored. More specifically, the search unit 42 collects information such as the category and the location of the restaurant when the occupant went to the restaurant by car from the vehicle-mounted device 12 or the occupant's mobile phone and stores it in the personal information DB 40. .

  The map information DB 41 stores information on roads on which the vehicle 50 can travel, store information on restaurants, shopping centers, and the like, along with map information. Further, the map information DB 41 may store traffic congestion information and the like obtained from a road manager or the like.

  The search unit 42 performs an information search on the occupant's preference and the current location of the vehicle based on the physical information and environmental information acquired by the vehicle-mounted device 12 and the recognition result of the intention and emotion in the information generation center 14, and The information is transmitted to the information generation center 14. Here, the physical information and the environment information may be obtained directly from the vehicle-mounted device 12 or may be obtained via the information generation center 14.

  The search unit 42 presents various types of information, such as, for example, various information such as air-conditioning temperature settings and audio volume settings, including artificial information using a neural network or the like, including the proposed timing of the information. I do. The information search on the preference in the search unit 42 can be performed using an artificial intelligence technology such as AI (Artificial Intelligence).

  The communication device 44 is connected to the network 16 of a mobile phone line network or a public line network, and can communicate with the vehicle-mounted device 12 and the information generation center 14 connected to the network 16.

  An example of information provision in the in-vehicle system 10 according to the present embodiment configured as described above will be described. In the in-vehicle system 10 according to the present embodiment, the virtual passenger V is displayed on the smart glass 26, and the useful information is provided from the vehicle 50 as if the virtual passenger V is talking to the virtual passenger V.

Here, an example of information provision when the driver D is a specific occupant will be described.
For example, when the driver D is hungry, the vehicle-mounted device 12 can provide the driver D with recommended restaurant information as useful information. Specifically, the information generation center 14 recognizes the emotion of the driver D from the occupant image captured by the monitor camera 20 and the biological information detected by the biological sensor 25, and determines that the driver D is hungry.

  Further, the information DB center 15 searches for shop information to be provided to the driver D. For example, the information DB center 15 acquires environmental information including an image outside the vehicle 50 and position information from the vehicle-mounted device 12 and searches for a store around the vehicle 50 from the map information DB 41. At this time, the information DB center 15 analyzes the preference of the driver D based on the number of stores in the past that have dropped in by category and searches for a recommended store.

  Then, the information DB center 15 returns a search result to the information generation center 14, so that the information generation center 14 generates a recommended shop as useful information. In the vehicle-mounted device 12 that has received the useful information, the virtual fellow passenger V proposes a store to the driver D based on the useful information. For example, as shown in FIG. 4, the vehicle-mounted device 12 controls the smart glasses 26 to display the virtual passenger V who speaks to the driver D, and displays the recommended menu image 62. Also, the vehicle-mounted device 12 controls the speaker 24 to emit a message such as “The ramen of XX seems to be popular.”

  On the other hand, when the information generation center 14 determines that the driver D nods and displays a will or has a favorable emotion, based on the location information of the shop included in the useful information, The navigation device of the in-vehicle device 32 sets the destination. At this time, an image is displayed as if the virtual passenger V is setting the destination of the navigation device.

  Further, for example, when the driver D who is driving the vehicle 50 has overlooked the sign on the traveling road, the vehicle-mounted device 12 can provide the information of the sign to the driver D as useful information. Specifically, as shown in FIG. 5, when the sign 70 exists in the outside image captured by the vehicle periphery monitoring unit 18 and the driver D's line of sight extracted from the monitor camera 20 is not on the sign 70, information generation is performed. The center 14 determines that the driver D has overlooked the sign. In this determination, the information generation center 14 may utilize the biological information detected by the biological sensor 25. For example, when the pulse and the brain wave are more stable than usual, the information generation center 14 can recognize that the driver D is not in a nervous state, and can recognize that there is no intention to be alert when driving.

  Further, the information DB center 15 acquires the environment information of the vehicle 50, and searches for information to be careful around the traveling road from the position information included in the environment information. Then, the information DB center 15 returns the search result to the information generation center 14, so that the information generation center 14 generates information to be noted as useful information. At the vehicle-mounted device 12 that has received the useful information, the virtual passenger V alerts the driver D based on the useful information. For example, the vehicle-mounted device 12 controls the smart glasses 26 to display the virtual passenger V who speaks to the driver D, and displays an image 64 of an animal to be careful on the traveling road. Further, the vehicle-mounted device 12 controls the speaker 24 to generate a message such as "Be careful of animals until passing XX" (see FIG. 5).

  Note that the above alert is performed when the driver D has overlooked the sign on the traveling road, but is not limited thereto, and may be executed each time the vehicle-mounted device 12 detects the sign. . Further, the information generation center 14 may provide the fixed information that does not depend on the position information of the vehicle 50 without performing the search for the information DB center 15.

  For example, as shown in FIG. 6, when the bicycle 72 exists in the blind spot of the building, the vehicle-mounted device 12 can provide the driver D with information on the bicycle 72 existing in the blind spot as useful information. First, the vehicle-mounted device 12 can acquire information on the bicycle 72 existing in the blind spot from the vehicle-mounted device 52A of the other vehicle 52 by inter-vehicle communication. When the vehicle-mounted device 12 acquires information on the bicycle 72 existing in the blind spot, and the driver D's line of sight extracted from the monitor camera 20 is not in the direction of the bicycle 72, the information generation center 14 determines that the driver D It is determined that there is a possibility that 72 will be overlooked. In this determination, the information generation center 14 may utilize the biological information detected by the biological sensor 25. For example, when the pulse is faster than usual and the brain wave is disturbed, the information generation center 14 recognizes that the driver D is in an anger or an excitement feeling, and recognizes that the driver D has little intention to consider surroundings when driving. be able to.

  Then, the information generation center 14 generates information on the blind spot bicycle 72 as useful information. At the vehicle-mounted device 12 that has received the useful information, the virtual passenger V alerts the driver D based on the useful information. For example, the vehicle-mounted device 12 controls the smart glasses 26 to display the virtual passenger V talking to the driver D, and displays the image image 66 of the bicycle 72 lurking in the blind spot. Also, the vehicle-mounted device 12 controls the speaker 24 to generate a message such as "Be careful of a bicycle coming out from the right" (see FIG. 6).

  Note that the above alert is performed when there is a possibility that the driver D may overlook the bicycle 72. However, the present invention is not limited to this. It may be executed each time information on the bicycle 72 existing in the blind spot is acquired. In this example, the case where the bicycle 72 exists in the blind spot of the building has been described as an example. However, a target to be alerted when the bicycle 72 exists in the blind spot includes not only bicycles but also pedestrians and vehicles.

  Subsequently, specific processing contents performed in each unit of the in-vehicle system 10 according to the present embodiment will be described.

  First, a specific process performed by the vehicle-mounted device 12 will be described. FIG. 7 is a flowchart illustrating an example of a flow of a process performed by the vehicle-mounted device 12 of the in-vehicle system 10 according to the present embodiment.

  In step 100, the control ECU 30 determines whether the display of the virtual passenger V is permitted. When the control ECU 30 determines that the display of the virtual passenger V is permitted, the process proceeds to step 102. On the other hand, when the control ECU 30 determines that the display of the virtual passenger V is not permitted, the process proceeds to step 104.

  In step 102, the control ECU 30 controls the smart glasses 26 to display the virtual passenger V. Then, the process proceeds to the next step 104.

  In step 104, the control ECU 30 acquires the outside image as the environment information from the vehicle periphery monitoring unit 18 and acquires the position information as the environment information from the GPS device 31. Then, the process proceeds to the next Step 106.

  In step 106, the control ECU 30 acquires the occupant image, which is the physical information, from the monitor camera 20 for the specific occupant, and acquires the biological information, which is the physical information, from the biological sensor 25. In addition, the control ECU 30 may acquire audio information from the microphone 22 as physical information. Then, control goes to the next step 108.

  In step 108, the control ECU 30 transmits the acquired physical information of the occupant and the environment information of the vehicle 50 to the information generation center 14. Then, the process proceeds to the next step 110.

  In step 110, the control ECU 30 determines whether useful information for the occupant has been received from the information generation center 14. When the control ECU 30 determines that the useful information has not been received, the process returns to step 108 to repeat the above-described processing, and when it determines that the useful information has been received, the process proceeds to step 112.

  In step 112, the control ECU 30 controls the smart glasses 26 and the speaker 24 to execute the control of the AR content including the virtual passenger V (hereinafter, referred to as “AR control”). Then, control goes to the next step 114. In the AR control processing, for example, when useful information indicating that the occupant is hot and the vehicle interior temperature is to be reduced is generated, the virtual passenger V is controlled in order to reproduce a state in which the temperature setting of the air conditioner is reduced. The ECU 30 controls the smart glass 26 and the speaker 24. The useful information is generated by the information generation center 14 based on the information search result of the information DB center 15 as described later.

  In step 114, the control ECU 30 determines whether to end the AR control. The determination may be, for example, whether or not an instruction to end the display of the virtual passenger V has been given by voice, or whether or not a switch or the like for instructing to end the display of the virtual passenger V has been operated. May be. When the control ECU 30 determines that the AR control is not to be ended, the process returns to step 104 to repeat the above-described processing, and when it is determined that the AR control is to be ended, a series of processing ends.

  Next, specific processing contents performed in the information generation center 14 will be described. FIG. 8 is a flowchart illustrating an example of a flow of a process performed in the information generation center 14 of the in-vehicle system 10 according to the present embodiment. The process in FIG. 8 is started, for example, when the transmission of the physical information and the environmental information is performed from the vehicle-mounted device 12 in step 108.

  In step 200, the information control unit 37 determines whether the physical information and the environmental information transmitted from the vehicle-mounted device 12 have been received. The process waits until the determination is affirmed, and proceeds to step 202.

  In step 202, the information generation center 14 recognizes the occupant's intention and emotion. Specifically, the facial expression recognition system 35 recognizes the occupant's intention and emotion based on the occupant image included in the physical information received from the vehicle-mounted device 12. The emotion recognition system 36 recognizes the occupant's emotion based on the occupant's biological information included in the physical information received from the vehicle-mounted device 12. When the information generation center 14 recognizes the intention and emotion of the occupant, the process proceeds to the next step 204.

  In step 204, the information control unit 37 instructs the information DB center 15 to search for information based on the physical information and environmental information received from the vehicle-mounted device 12 and the recognition result of the occupant's intention and emotion. Then, control goes to the next step 206.

  In step 206, the information control unit 37 determines whether or not an information search result has been received from the information DB center 15. The process waits until the determination is affirmed, and proceeds to step 208.

  In step 208, the information control unit 37 generates useful information reflecting the information search result. For example, when the occupant intends to eat, useful information including recommended shop information is generated according to the occupant's preference included in the information search result. In addition, for example, when the occupant has overlooked the sign, useful information including caution information around the vehicle 50 included in the information search result is generated. The generated useful information includes character and image information to be displayed on the smart glass 26 and audio information to be output from the speaker 24. When the information control unit 37 generates the useful information, the process proceeds to the next step 210.

  In step 210, the information control unit 37 transmits the generated useful information to the vehicle-mounted device 12, and ends a series of processing.

  Next, a specific process performed by the information DB center 15 will be described. FIG. 9 is a flowchart illustrating an example of a flow of a process performed by the search unit 42 of the information DB center 15 of the in-vehicle system 10 according to the present embodiment.

  In step 300, the search unit 42 determines whether or not the body information and the environment information and the recognition result of the occupant's intention and emotion have been received from the information generation center 14. After the reception is completed, the process waits until the determination is affirmed, and proceeds to step 302.

  In step 302, the search unit 42 performs an information search. Specifically, the search unit 42 performs an information search on the preference based on the physical information and the recognition result of the occupant's intention and emotion. The information search is executed using the occupant's personal information stored in the personal information DB 40. For example, based on the intention of the occupant and the information stored in the personal information DB 40, a preference analysis is performed to search for information on a store that the occupant likes. When the search unit 42 ends the information search, the process proceeds to the next step 304.

  In step 304, the search unit 42 returns an information search result to the information generation center 14, and ends a series of processing.

(Summary)
By performing the processing in each unit as described above, the in-vehicle system 10 according to the present embodiment can receive useful information from the vehicle without the occupant uttering a word. The display related to the useful information is provided from the smart glasses 26 individually worn by the occupant, and the sound related to the useful information is provided from the speaker 24 output for each occupant. That is, according to the in-vehicle system 10 of the present embodiment, it is possible to privately acquire video and audio even when a passenger is present.

  When there are a plurality of occupants in the vehicle cabin, each occupant can acquire information privately by wearing the smart glasses 26 and providing a speaker 24 for each occupant. Further, at some times, different information may be provided to each occupant, and at other times, the same information may be provided to each occupant.

  Further, in the present embodiment, the provision of the useful information is performed via the virtual passenger V. Therefore, the occupant feels that the virtual fellow passenger V takes his intention and takes the action he desires only by thinking or feeling. Therefore, according to the present embodiment, bidirectional real-time communication with the virtual passenger V can be achieved. Then, by smoothly communicating with the private partner called the virtual passenger V, it is possible to enjoy traveling and reduce the burden of driving.

  In particular, in the present embodiment, in a situation where there is a driving load other than the automatic driving, the driver can think and feel and give an instruction to the virtual passenger V to use each function without delay. . For example, operations such as adjusting the temperature of the air conditioner, adjusting the temperature of the audio device, and setting the destination of the car navigation device can be performed through the virtual passenger V.

  In the present embodiment, the smart glasses 26 can display the virtual passenger V in the vehicle interior, and can display an image relating to useful information outside the vehicle interior. For example, the control ECU 30 controls the smart glass 26 to display the recommended product image superimposed on the real image of the recommended store (see FIG. 4), or to display the animal image on the real image of the traveling road with a caution sign. Images can be superimposed and displayed (see FIG. 5). Further, for example, an image of a bicycle, a pedestrian, or the like recognized by another vehicle 52 can be displayed on the smart glass 26 (see FIG. 6). Similarly, an image of a bicycle, a pedestrian, or the like in the blind spot of the vehicle 50 may be displayed on the smart glass 26. By displaying the blind spot bicycle, pedestrian, and the like on the smart glass 26 and alerting the virtual passenger V to speak to the driver, it is possible to contribute to safer driving.

  In the present embodiment, the occupant can receive useful information through the virtual fellow passenger V without uttering a word, but is not limited to this. For example, the information generation center 14 may generate useful information based on a voice uttered by the occupant toward the microphone 22 and receive the useful information through the virtual fellow passenger V. That is, the occupant can communicate with the virtual passenger V in an interactive manner. Then, it is possible to enjoy driving as if a normal passenger were present.

  In the in-vehicle system 10 of the present embodiment, it is possible to communicate with the virtual fellow passenger V displayed so as to be present in the passenger compartment. However, the present invention is not limited to this. Can communicate with a child or the like who is away from home at home 54. For example, when trying to communicate with a child who is at home 54, the communication device 54 </ b> A at home 54 transmits an image captured by a camera (not shown) to the vehicle-mounted device 12 in real time. Then, in the vehicle-mounted device 12, the control ECU 30 controls the smart glass 26 to display the image Vc of the child in the vehicle interior (see FIG. 3), thereby sharing the moving space and sharing the user experience. .

  The in-vehicle system 10 of the present embodiment enables the display of the virtual fellow passenger V regardless of the driving load of the driver D. However, the present invention is not limited to this. The virtual passenger V may be displayed only when there is no driving load. Specifically, the control ECU 30 detects whether the in-vehicle system 10 can be used safely as a driving load based on the result of monitoring the periphery of the vehicle 50 by the vehicle periphery monitoring unit 18. For example, the control ECU 30 performs control to shift to the automatic driving mode based on the monitoring result of the vehicle periphery monitoring unit 18 and safely performs the operation when the driver D has no driving load such as when the vehicle 50 enters the automatic driving mode. It is determined that it can be used. Then, the control ECU 30 displays the virtual passenger V. The determination of the transition to the automatic driving mode can be made by using a known automatic driving technique based on the monitoring result of the vehicle periphery monitoring unit 18.

  Further, even during the communication with the virtual passenger V, if the in-vehicle system 10 cannot be used safely from the monitoring result of the vehicle periphery monitoring unit 18, the control ECU 30 transmits the virtual passenger V May be interrupted. This allows the driver D to communicate with the virtual passenger V while ensuring safety.

(Supplement)
In the above-described embodiment, a description has been given of a mode in which each unit of the in-vehicle device 12, the information generation center 14, and the information DB center 15 performs the processing, but the present invention is not limited to this. For example, an in-vehicle system in which the configuration of the information DB center 15 is mounted on the vehicle-mounted device 12 may be employed. Alternatively, not only the information DB center 15 but also the configuration of the information generation center 14 may be mounted on the vehicle-mounted device 12 so that the vehicle-mounted device 12 can perform processing.

  The processing in each unit of the on-vehicle device 12, the information generation center 14, and the information DB center 15 is such that the CPU of each device reads the execution program from the ROM and expands it in the RAM, and the CPU loads and executes the execution program. This is achieved by: Note that the execution program may be stored in a non-temporary recording medium such as a DVD (Digital Versatile Disc), read through a recording medium reading device, and expanded in a RAM.

  The in-vehicle system 10 according to the present embodiment may further include a device that generates an ultrasonic wave in the on-vehicle device 12 so that the occupant can be given a tactile sensation such as warmth of human skin.

  The processing performed by each unit of the in-vehicle system 10 in the above embodiment may be a software processing performed by executing a program or a processing performed by hardware. Alternatively, the processing may be a combination of both software and hardware.

  Furthermore, the present invention is not limited to the above, and it goes without saying that, other than the above, various modifications can be made without departing from the spirit of the present invention.

10 In-vehicle system 14 Information generation center (generation unit)
20 monitor camera (detection unit)
24 speaker 25 biological sensor (detection unit)
26 Smart Glasses (AR Display)
30 control ECU (control unit)

Claims (1)

A detection unit that detects physical information emitted by a specific occupant in the vehicle interior;
An AR indicator worn by the specific occupant;
A speaker from which sound for the specific occupant is output,
A generation unit that generates useful information suitable for the specific occupant based on the physical information detected by the detection unit;
A control unit that controls so that a sound related to the useful information is output from the speaker, and a display related to the useful information is displayed on an AR display in a manner superimposed on scenes inside and outside the vehicle cabin;
In-car system with.