JP6819086B2 - Display control device for vehicles - Google Patents

Description

The present invention relates to a display control device for a vehicle that controls a display screen of a display mounted on a vehicle.

In recent years, the practical application of a technology for automatically performing a vehicle driving operation (that is, an automatic driving technology) has been aimed at. For convenience, the function of automatically driving the vehicle by the automatic driving technology is referred to as an automatic driving function, and the electronic control device that provides the automatic driving function is referred to as a vehicle control device. The vehicle control device performs steering, acceleration, braking, and the like so that the vehicle travels along a travel path set by the user.

A vehicle equipped with such a vehicle control device includes, as a driving mode, an automatic driving mode in which the vehicle is automatically driven by the vehicle control device and a manual driving mode in which an occupant in the driver's seat performs a driving operation. Shifting from the manual driving mode to the automatic driving mode means that the occupant in the driver's seat transfers the authority to control the driving of the vehicle (hereinafter referred to as the driving authority) to the vehicle side control device. Further, shifting from the automatic driving mode to the manual driving mode means that the vehicle-side control device transfers the driving authority to the driver's seat occupant.

The transfer of authority from the vehicle control device to the driver's seat occupant is carried out after the system mounted on the vehicle (hereinafter referred to as the in-vehicle system) confirms that the driver's seat occupant is in a state where the driving operation can be performed. There is a need. If the driver's seat occupant is in a state where the driving operation cannot be performed (for example, during a nap) and the vehicle control device passes the driving authority to the driver's seat occupant, the vehicle may be in an uncontrollable state. Because.

The state of the driver's seat occupant can be specified by, for example, the method disclosed in Patent Document 1. In Patent Document 1, a far-infrared image showing the temperature distribution of the face surface of the driver's seat occupant is taken by using a far-infrared camera or the like, and an abnormal state of the driver's seat occupant is detected based on the captured image. For convenience, a camera that captures an image for sensing the state of the driver's seat occupant is called a state monitoring camera. The condition monitoring camera is arranged on the meter panel, the steering column cover, or the like so as to capture the face of the driver's seat occupant who is in a posture for performing the driving operation from substantially the front.

Japanese Unexamined Patent Publication No. 2016-57944

In order to detect the condition of the driver's seat occupant using the condition monitoring camera, the driver's seat occupant needs to face in the direction in which the camera is provided (in other words, the front direction of the vehicle). However, when the vehicle is driving in the automatic driving mode, the driver's seat occupant operates the information terminal brought into the vehicle to perform subtasks such as e-mail, games, web browsing, watching videos, and browsing e-books. It is conceivable to do. The above-mentioned information terminal is, for example, a smartphone or a tablet terminal.

Since these information terminals can be arranged and operated at any place of the driver's seat occupant, for example, the driver's seat occupant may operate the information terminal by placing it on his / her lap or in the passenger seat. When the driver's seat occupant is operating with the information terminal placed on the knee or the like, the face of the driver's seat occupant seated in the driver's seat is downward relative to the front direction of the vehicle. And tends to face sideways. If the face of the driver's seat occupant is facing in a direction other than the front direction of the vehicle, the amount of imaging of the driver's seat occupant's face is reduced, and the detection accuracy of the driver's seat occupant's state is lowered or undetectable. It may become.

Further, in order for the driver's seat occupant to acquire the driving authority from the vehicle control device and execute the driving operation, the driver's seat occupant needs to be aware of the traffic situation (particularly in front) around the own vehicle in advance. Therefore, even when the vehicle is traveling in the automatic driving mode, it is preferable that the driver's seat occupant is in a posture that includes the front region outside the vehicle interior in the field of view.

The present invention has been made based on this circumstance, and an object of the present invention is to maintain the convenience of the driver's seat occupant and to prevent the driver's seat occupant from recognizing the traffic condition in front of the vehicle. The purpose is to provide an information providing device capable of suppressing the turning of the face.

The first invention for achieving the object is used in a vehicle equipped with a vehicle control device that provides an automatic driving function that automatically executes acceleration, braking, and steering of the vehicle, and communicates with the vehicle control device. By carrying out the operation, the operation mode determination unit determines whether the operation mode of the vehicle is an automatic operation mode in which the vehicle control device automatically drives the vehicle or a manual operation mode in which the occupant in the driver's seat performs an operation operation. (F3), a display processing unit (F1) that controls the display screen of the driver's seat display installed at a position included in the view of the driver's seat occupant facing the front of the vehicle, and a display processing unit (F1) existing in the vehicle interior. Communication between the vehicle interior communication processing unit (F7), which communicates with the information terminal brought into the vehicle interior by the occupant, and the vehicle interior communication processing unit via the vehicle interior communication module for communicating with the information terminal. The image captured by the terminal control unit (F8) that controls the operation of the information terminal and the camera provided in the area facing the driver's seat in the vehicle interior so as to image the face of the occupant sitting in the driver's seat. Based on this, the face orientation acquisition unit (F4) that sequentially acquires information indicating the orientation of the occupant's face, and the face orientation acquisition unit acquires when the operation mode determination unit determines that the operation mode is the automatic operation mode. To wirelessly access the wide area communication network and the task determination unit (F5) that determines whether or not the driver's seat occupant is executing a subtask using the tool brought into the vehicle interior based on the orientation of the occupant's face. When there is no wide area communication processing unit (F6) that transmits / receives data to / from an external device connected to the wide area communication network and an information terminal that the vehicle interior communication processing unit is communicating with via the wide area communication module of In the subtask information acquisition unit (S124, S222), when the task determination unit determines that the driver's seat occupant is executing the subtask, the subtask information acquisition unit (S124, S222) acquires the subtask information indicating the content of the subtask based on the operation of the occupant on the input device. ), And the content data acquisition unit (F9) that acquires the digital content data corresponding to the subtask information from a predetermined database connected to the wide area communication network using the subtask information acquired by the subtask information acquisition unit as a search word. The terminal control unit acquires an application image which is an image corresponding to the application software running on the information terminal, and based on the operation of the occupant on the input device mounted on the vehicle, the occupant's operation on the information terminal Accepts the instruction operation and executes the processing corresponding to the instruction operation on the information terminal. The display processing unit is characterized in that the image corresponding to the digital content acquired by the content data acquisition unit is displayed on the driver's seat display as a subtask screen.
The second invention is used in a vehicle equipped with a vehicle control device that provides an automatic driving function that automatically executes acceleration, braking, and steering of the vehicle, and by performing mutual communication with the vehicle control device, the vehicle The driving mode determination unit (F3) for determining whether the driving mode is the automatic driving mode in which the vehicle control device automatically drives the vehicle or the manual driving mode in which the occupant in the driver's seat performs the driving operation, and the vehicle Communicates with the display processing unit (F1) that controls the display screen of the driver's seat display installed at a position included in the view of the driver's seat occupant facing the front, and the information terminal existing in the vehicle interior. The operation of the vehicle interior communication processing unit (F7) that communicates with the information terminal brought into the vehicle interior by the occupant and the information terminal communicating with the vehicle interior communication processing unit via the vehicle interior communication module for It includes a terminal control unit (F8) to control, and the terminal control unit acquires an application image, which is an image corresponding to the application software running on the information terminal, and operates the occupant on the input device mounted on the vehicle. Based on the above, the information terminal receives the instruction operation of the occupant to the information terminal and causes the information terminal to execute the process corresponding to the instruction operation, so that the face of the occupant sitting in the driver's seat is imaged in the vehicle interior. The operation mode is automatically operated by the face orientation acquisition unit (F4) that sequentially acquires information indicating the orientation of the occupant's face based on the image captured by the camera provided in the area facing the driver's seat, and the operation mode determination unit. Based on the orientation of the occupant's face acquired by the face orientation acquisition unit when it is determined to be in the mode, it is determined whether or not the driver's seat occupant is executing a subtask using the tool brought into the vehicle interior. The task determination unit (F5), the wide area communication processing unit (F6) that transmits and receives data to and from an external device connected to the wide area communication network via the wide area communication module for wireless access to the wide area communication network, and the vehicle interior. When the communication processing unit does not have an information terminal that is executing communication, and the task determination unit determines that the driver's seat occupant is executing the subtask, the subtask is based on the operation of the occupant with respect to the input device. The subtask information acquisition unit (S124, S222) that acquires the subtask information indicating the content and the subtask information acquired by the subtask information acquisition unit are used as search words to support the subtask information from a predetermined database connected to the wide area communication network. Content data acquisition unit (F9) that acquires digital content data, Further prepared, the display processing unit displays an application image on the driver's seat display when the driving mode changes from the manual driving mode to the automatic driving mode, and when the driving mode becomes the automatic driving mode, the vehicle interior communication processing unit If the information terminal cannot be detected, the driver's seat display will guide you to turn on the short-range communication function of the information terminal in order to make a communication connection between the information terminal and the vehicle interior communication module. When the task judgment unit determines that the occupant is executing the subtask, the application image is displayed as the subtask screen on the driver's seat display, and the image corresponding to the digital content acquired by the content data acquisition unit is displayed. It is characterized in that it is displayed as a subtask screen on the driver's seat display .

According to the above configuration, when the driving mode of the vehicle is the automatic driving mode, an image (that is, an application image) corresponding to the application software running on the information terminal brought by the occupant is displayed on the driver's seat display. Will be done. The driver's seat display here is a display installed at a position included in the field of view of the driver's seat occupant facing the front direction of the vehicle. Therefore, even when the driver's seat occupant is looking at the driver's seat display, it is possible to maintain a state in which the traffic situation in front of the vehicle is captured in the field of view such as the effective field of view and the stable gaze field of view.

Further, the driver's seat occupant can refer to the information provided in the information terminal by looking at the driver's seat display instead of the display provided in the information terminal. That is, according to the above configuration, it is possible to maintain the convenience of the driver's seat occupant and prevent the driver's seat occupant from turning his / her face in a direction in which the traffic condition in front of the vehicle cannot be recognized.

The reference numerals in parentheses described in the claims indicate, as one embodiment, the correspondence with the specific means described in the embodiments described later, and limit the technical scope of the present invention. is not it.

It is a block diagram which shows an example of the schematic structure of the in-vehicle system 1. It is a block diagram which shows an example of the arrangement place of the display 15. It is a block diagram which shows the schematic structure of HCU10. It is a flowchart corresponding to the display process at the time of mode transition carried out by HCU10. It is a flowchart corresponding to the subtask screen provision processing carried out by HCU10. It is a block diagram which shows the schematic structure of the vehicle-mounted system 1 in the modification 2. It is a block diagram which shows an example of a task information input screen.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an example of a schematic configuration of an in-vehicle system 1 having a function as a display control device for a vehicle according to the present invention. The in-vehicle system 1 is mounted on a vehicle and includes an HCU 10, a wide area communication unit 11, a vehicle control ECU 12, a DSM 13, an input device 14, a display 15, an indicator 16, and a short-range communication unit 17, as shown in FIG. HCU is an abbreviation for HMI (Human Machine Interface) Control Unit, and DSM is an abbreviation for Driver Status Monitor.

The wide area communication unit 11, the vehicle control ECU 12, the DSM 13, the input device 14, the display 15, the indicator 16, and the short-range communication unit 17 each pass through a communication network (hereinafter, LAN: Local Area Network) constructed in the vehicle. It is connected to the HCU 10 so that it can communicate with each other. Hereinafter, for convenience, the vehicle on which the in-vehicle system 1 is mounted will also be referred to as the own vehicle.

The HCU 10 is configured as a computer including a CPU, RAM, ROM, flash memory, I / O, and a bus line connecting these configurations. The CPU executes various arithmetic processes, and is realized by using, for example, a microprocessor or the like. RAM is a volatile memory and ROM is a non-volatile memory. The I / O is an interface for the HCU 10 to input / output data to / from the wide area communication unit 11 and the like. I / O may be realized by using an IC, a digital circuit element, an analog circuit element, or the like.

The ROM included in the HCU 10 stores a program (hereinafter, a display control program) for causing a normal computer to function as the HCU 10. The above-mentioned display control program may be stored in a non-transitionary tangible storage medium. Executing the display control program by the CPU corresponds to executing the method corresponding to the display control program.

The HCU 10 provides various functions described later by the CPU executing a display control program. In addition, by linking these functions, the mode transition display process and the subtask screen provision process, which will be described later, are executed.

The wide area communication unit 11 is a communication module for wirelessly accessing the wide area communication network and for the in-vehicle system 1 to communicate with another communication device (for example, a search server 2) connected to the wide area communication network. For example, the wide area communication unit 11 modulates the data input from the HCU 10 and transmits it to the search server 2. Further, the data transmitted from the search server 2 is received and provided to the HCU 10. The wide area communication unit 11 corresponds to the wide area communication module described in the claims.

The wide area communication network here refers to a public communication network provided by a telecommunications carrier, such as a mobile phone network or the Internet. The search server 2 is a server that searches for information on the Internet. The search server 2 executes a search process based on the request from the in-vehicle system 1 and returns the search result.

The vehicle control ECU 12 is an electronic control device (that is, an automatic driving function) that automatically executes steering, acceleration, braking, etc. so that the own vehicle travels along a planned traveling route set by an occupant. ECU: Electronic Control Unit). When the automatic driving function is set to operate based on the operation of the driver's seat occupant, the vehicle control ECU 12 automatically steers, accelerates, and brakes so that the own vehicle travels along the planned travel route. Execute. Control of steering, acceleration / deceleration, etc. may be realized by outputting a control signal indicating a control target value toward the actuator or motor to be controlled. The vehicle control ECU 12 corresponds to the vehicle control device according to claim.

For convenience, the operation mode in which the own vehicle automatically travels by the vehicle control ECU 12 is described as an automatic operation mode, and the operation mode in which the own vehicle travels by manual operation is also described as a manual operation mode. That is, the own vehicle has an automatic driving mode and a manual driving mode as the driving modes.

The manual operation here is not limited to a complete manual operation. It also includes a mode in which the vehicle control ECU 12 executes a part of the driving operation (for example, speed control) on behalf of the driver's seat occupant. The vehicle control ECU 12 provides the HCU 10 with mode information indicating the current driving mode.

The conditions under which the vehicle is driven by the automatic driving function (that is, automatic driving) may be appropriately designed. For example, if the predetermined automatic driving conditions are satisfied and the occupant instructs the start of automatic driving (in other words, the driving authority is transferred from the driver's seat occupant to the vehicle control ECU 12). Good.

The specific contents of the above-mentioned automatic driving conditions may be appropriately designed. For example, if the current position of the own vehicle can be positioned based on the navigation message transmitted from the positioning satellite, and the peripheral monitoring device for recognizing the traffic condition around the own vehicle is operating normally. Just do it. The peripheral monitoring device is, for example, a distance measuring sensor such as LIDAR, a camera installed to photograph the outside of the vehicle interior, an inter-vehicle communication device that performs mutual communication with other vehicles, and the like.

The DSM 13 is realized by using a near-infrared light source, a near-infrared camera, a control unit for controlling them, and the like. The DSM 13 captures the face of the driver's seat occupant irradiated with near-infrared light by the near-infrared light source with the near-infrared camera. The DSM 13 sequentially detects the posture of the occupant in the driver's seat, the orientation of the face of the occupant in the driver's seat, the degree of opening of the eyelids, and the like by performing a well-known image recognition process on the image captured by the near-infrared camera.

If the DSM 13 is arranged at an appropriately designed position such as a steering column cover or a part of the instrument panel facing the driver's seat so as to photograph the face area of the occupant seated in the driver's seat. Good. Here, as an example, it is assumed that the DSM 13 is attached to a portion of the instrument panel located on the upper side of the meter display 15B in a posture that includes the eye lip portion of the driver's seat occupant in the imaging range. Of course, the DSM 13 may be attached to the front surface or the inside of the meter display 15B instead of the portion located above the meter display 15B.

The iris portion is an elliptical region in which the positions of the eyes of a person sitting in the driver's seat are distributed. The irips section is determined as a population of data indicating the positions of the eyes when people of various physiques are in a normal driving posture. The iris section is determined by actual tests and simulations for each vehicle model.

By the way, when the driver's seat occupant faces the face in a direction other than the front direction of the vehicle, such as the downward direction such as the feet or the lateral direction where the passenger seat is located, the amount of the image of the face portion by the near infrared camera is increased. It may be reduced and it may not be possible to accurately extract feature points such as eyes, nose, and mouth from the captured image. In addition, if the state where the feature points such as eyes, nose, and mouth cannot be accurately extracted continues, as a result, it may not be possible to obtain the information necessary for estimating the state of the driver's seat occupant. .. The information necessary for estimating the state of the driver's seat occupant is, for example, the orientation of the driver's seat occupant's face and the degree of eyelid opening. The front direction of the vehicle is a direction from the rear end to the front end of the vehicle.

Therefore, in order to estimate the state of the driver's seat occupant using the DSM 13, the angle formed by the direction in which the driver's seat occupant's face is facing and the vehicle front direction is a predetermined angle determined according to the installation position of the DSM 13. Must be within. Such an angle range of face orientation is referred to as a detectable angle range. The detectable angle range is determined according to the hardware configuration of the DSM 13, the image recognition algorithm, and the installation position.

The DSM 13 sequentially outputs information (hereinafter, occupant state information) indicating the posture of the driver's seat occupant identified from the captured image, the orientation of the face, the degree of opening of the eyelids, and the like to the HCU 10. Further, when the DSM 13 cannot detect a predetermined state quantity such as the orientation of the face and the degree of opening of the eyelids, the DSM 13 shall output error information indicating that fact as the occupant state information.

The input device 14 is a device for receiving an instruction operation of the driver's seat occupant with respect to the in-vehicle system 1. Further, as will be described later, when the mobile terminal 3 brought into the vehicle interior by the driver's seat occupant and the in-vehicle system 1 are connected by communication, the input device 14 performs an instruction operation of the driver's seat occupant to the mobile terminal 3. It also functions as a device for accepting.

In the present embodiment, as an example, the input device 14 is a mechanical switch (that is, a steering switch) provided on the steering wheel. The input device 14 outputs a control signal corresponding to the operation performed by the driver's seat occupant to the input device 14 to the HCU 10. The HCU 10 specifies the operation content of the driver's seat occupant based on the control signal input from the input device 14.

In the present embodiment, the input device 14 is configured to employ a steering switch, but the present invention is not limited to this. The input device 14 may be a voice input device realized by using a well-known voice recognition technique, or may be a mechanical switch arranged at a position on the instrument panel that is easy for the driver to operate. It may be. Further, a touch panel laminated on the center display 15C may be used. Further, it may be a haptic device arranged around the driver's seat, a mouse, a keyboard, or the like. Further, the in-vehicle system 1 may include the above-mentioned plurality of types of devices as the input device 14.

The display 15 is a device that displays an image input from the HCU 10. The display 15 is within the detectable angle range of the face orientation even when the driver's seat occupant is looking at the display 15 in the area included in the field of view of the driver's seat occupant facing the front direction of the vehicle. It is placed in a position that is maintained at.

As shown in FIG. 2, the vehicle-mounted system 1 in the present embodiment includes a head-up display 15A, a meter display 15B, and a center display 15C as the display 15. Although FIG. 2 discloses a mode in which the driver's seat is arranged on the right side of the vehicle, the driver's seat may be provided on the left side of the vehicle. Various displays 15 correspond to the driver's seat display according to claim.

The head-up display 15A is a display 15 that projects a virtual image on a part of the windshield in front of the driver's seat. The above-mentioned virtual image is formed by projecting a display light representing predetermined information onto the windshield from a floodlight device arranged in the instrument panel.

The meter display 15B is a display 15 arranged on the instrument panel above an area located in front of the driver's seat (hereinafter referred to as a driver's seat facing area). Further, the center display 15C is a display provided at the uppermost portion of the instrument panel in the center portion in the vehicle width direction (hereinafter, the central region). The central region corresponds to an region of the instrument panel that faces the passenger seat (hereinafter, the passenger seat facing region) and the driver seat facing region.

The position of the center display 15C is not limited to the uppermost part of the central region. It may be installed below the top. However, it is preferable that the steering wheel is arranged in a region above the center of rotation (hereinafter, a region above the center).

The meter display 15B and the center display 15C are capable of full-color display, for example, and can be realized by using a liquid crystal display, an organic EL display, a plasma display, or the like.

The indicator 16 is a device that notifies the driver's seat occupant of information regarding the whereabouts of driving authority by causing a plurality of light emitting elements (for example, light emitting diodes) to emit light in a predetermined light emitting pattern. For example, the indicator 16 can indicate whether the driving authority is on the in-vehicle system 1 or the driver's seat occupant, that the vehicle control ECU 12 is trying to transfer the driving authority to the driver's seat occupant, and that the driving authority can be transferred to the vehicle control ECU 12. The driver's seat occupant is notified that the vehicle is in a state by causing the light emitting element to emit light in a light emitting pattern corresponding to each state.

The light emission pattern is composed of a combination of light color, blinking cycle, and the like. The indicator 16 may be provided at an appropriately designed place such as a steering wheel or an upper end portion of an instrument panel.

The short-range communication unit 17 is a communication module for carrying out wireless communication (hereinafter, short-range communication) conforming to a predetermined short-range wireless communication standard with a mobile terminal 3 which is an information terminal carried by a driver's seat occupant. .. The short-range wireless communication standard adopted here is a wireless communication standard in which the maximum communicable range is about several tens of meters. As a standard for short-range communication, for example, Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark) and the like can be adopted.

The mobile terminal 3 has the above-mentioned short-range communication function (hereinafter, short-range communication function), and is application software for operating a normal computer as the mobile terminal 3 in the present embodiment (hereinafter, vehicle cooperation). Any electronic device that has a computer on which the application) is installed will do. For example, a smartphone, a mobile phone, a tablet terminal, a wearable device, a portable music player, a portable game machine, or the like can be used as the mobile terminal 3. Application software will be abbreviated as application below.

When the short-range communication function is turned on, the mobile terminal 3 periodically transmits an advertisement signal including a unique terminal number (hereinafter referred to as a terminal ID) assigned to itself. By receiving this advertisement signal, the in-vehicle system 1 recognizes that the mobile terminal 3 is present in the vehicle interior. As another aspect, the mobile terminal 3 may transmit an advertisement signal based on a request from the in-vehicle system 1.

The short-range communication unit 17 included in the in-vehicle system 1 is provided to carry out short-range communication with the mobile terminal 3 existing in the vehicle interior. Therefore, the transmission strength and reception sensitivity of the radio wave output by the short-range communication unit 17 can communicate with the mobile terminal 3 existing in the vehicle interior and communicate with the mobile terminal 3 existing at a position away from the vehicle. It is preferable that the level is appropriately set so as not to occur. The short-range communication unit 17 corresponds to the vehicle interior communication module according to the claim.

<About the general configuration of HCU10>
The HCU 10 provides functions corresponding to the various functional blocks shown in FIG. 3 by the CPU executing the above-mentioned seating determination program. That is, as a functional block, the HCU 10 includes a display processing unit F1, an operation reception unit F2, an operation mode determination unit F3, a face orientation acquisition unit F4, a task determination unit F5, a wide area communication processing unit F6, a short-range communication processing unit F7, and a terminal. A control unit F8 is provided.

A part or all of the functional blocks included in the HCU 10 may be realized by using one or a plurality of ICs (in other words, as hardware). Further, a part or all of the functional blocks included in the HCU 10 may be realized by executing software by the CPU and combining hardware members.

The display processing unit F1 is a functional block that controls display screens on various displays 15. The specific operation of the display processing unit F1 will be described later. The operation reception unit F2 receives the instruction operation of the driver's seat occupant to the in-vehicle system 1 based on the operation of the driver's seat occupant executed on the input device 14. Further, when there is a mobile terminal 3 for which a communication connection with the in-vehicle system 1 is established, the operation reception unit F2 has a driver's seat for the mobile terminal 3 based on a control signal input from the input device 14. It also accepts instructions from occupants.

The driving mode determination unit F3 determines whether the current driving mode is the automatic driving mode or the manual driving mode based on the mode information provided by the vehicle control ECU 12. The determination result of the operation mode determination unit F3 is sequentially provided to the task determination unit F5 and the like. The face orientation acquisition unit F4 acquires information indicating the posture of the driver's seat occupant, the orientation of the face, and the like from the DSM25. The acquisition result of the face orientation acquisition unit F4 is provided to the task determination unit F5.

The task determination unit F5 is a tool brought in from outside the vehicle by the driver occupant based on the determination result of the driving mode determination unit F3 and the face orientation of the driver's seat occupant acquired by the face orientation acquisition unit F4 (hereinafter, brought in). Determine if the subtask using the tool) is being executed. As the carry-on tool, a mobile terminal 3, an electronic device not provided with a short-range communication function, a book on a paper medium, a material printed on paper, or the like is assumed.

The subtasks here are watching videos such as movies and TV programs, watching music, checking news, playing games, browsing photos and image data, using SNS (social networking service), referring to maps, books. Refers to things such as reading. Books shall also include magazines and comics. Further, the operation itself of the mobile terminal 3 is also included in the subtask.

In the task determination unit F5, the operation mode determination unit F3 determines that the driving mode of the vehicle is the automatic driving mode, and the driver's seat occupant faces a predetermined subtask direction for a certain period of time. If it continues (for example, 10 seconds), the driver's seat occupant determines that the subtask is being executed. The subtask direction corresponds to the direction of the face in a state where the driver and occupant are executing a subtask using a tool brought in from outside the vehicle (hereinafter referred to as a carry-on tool), such as a downward direction or a passenger seat direction.

For example, in the task determination unit F5, when the direction of the driver's seat occupant's face is downward by a predetermined angle (for example, 40 degrees) or more or the passenger seat direction with respect to the vehicle front direction, the driver's seat occupant is in the subtask direction. Judge that you are watching. Further, the task determination unit F5 may determine that the driver's seat occupant is facing the subtask direction even when the error information is provided by the DSM 13.

As another aspect, the task determination unit F5 determines the driver's seat when the direction of the driver's seat occupant's face is not in front of the vehicle, in the direction in which the side mirrors are provided, or in the direction in which the display 15 is arranged. It may be determined that the occupant is facing the subtask direction. The orientation of the driver's seat occupant's face when looking at the front of the vehicle, the orientation of the driver's seat occupant's face when looking at the direction in which the side mirrors are provided, and the driver's seat occupant when looking at the display 15. The orientation of the face may be defined in advance.

Further, in the task determination unit F5, the operation mode determination unit F3 determines that the driving mode of the vehicle is the automatic driving mode, and the mobile terminal 3 is not detected by the mobile terminal detection unit F71 described later. Further, when the driver's seat occupant turns his face toward the subtask for a certain period of time, it is determined that the driver's seat occupant is executing the independent subtask.

Independent subtasks include executing subtasks using an electronic device that does not have a short-range communication function or a mobile terminal 3 that has the short-range communication function turned off, reading a paper-based book, and materials printed on paper. To read, take notes with a writing instrument, etc.

The wide area communication processing unit F6 acquires the data received by the wide area communication unit 11. Further, the wide area communication processing unit F6 generates a predetermined communication packet based on a request from another functional block, and performs a process of transmitting the communication packet to the wide area communication unit 11.

The short-range communication processing unit F7 acquires the data received by the short-range communication unit 17. Further, the short-range communication processing unit F7 generates data addressed to the mobile terminal 3 and causes the short-range communication unit 17 to transmit the data. That is, the short-range communication processing unit F7 is a functional block that executes data transmission / reception with the mobile terminal 3 in the HCU 10. The short-range communication processing unit F7 corresponds to the vehicle interior communication processing unit according to the claim. The short-range communication processing unit F7 includes a mobile terminal detection unit F71 and a connection processing unit F72 as more detailed functions.

The mobile terminal detection unit F71 is a functional block that detects the mobile terminal 3 existing in the vehicle interior. The mobile terminal detection unit F71 detects that the mobile terminal 3 is present in the vehicle interior based on the reception of the advertisement signal. The fact that the advertisement signal is received means that the mobile terminal 3 is present in the vehicle interior. Further, the terminal ID included in the received advertisement signal is stored in a RAM or the like (not shown).

The connection processing unit F72 is a functional block for establishing a communication connection with the mobile terminal 3 detected by the mobile terminal detection unit F71. In the present embodiment, as an example, the connection processing unit F72 automatically establishes a communication connection with the mobile terminal 3 when the mobile terminal 3 to which the in-vehicle system 1 has been connected in the past is detected by the mobile terminal detection unit F71. To establish. The history of terminal IDs that have been connected by communication may be stored in a rewritable non-volatile storage medium (for example, flash memory). For convenience, the mobile terminal 3 that exists in the vehicle interior and has a communication connection established by the connection processing unit F72 will also be referred to as a connected terminal.

The terminal control unit F8 is a functional block that controls the operation of the connected terminal, and is mainly for the driver's seat occupant to use the function (in other words, the application) provided in the mobile terminal 3 via the input device 14 and the display 15. Execute the processing of.

Specifically, the terminal control unit F8 acquires image data for displaying the screen corresponding to the application active on the connected terminal on the display from the connected terminal and provides it to the display processing unit F1. As a result, the display 15 displays the screen of the application that is active on the connected terminal.

Further, the terminal control unit F8 receives an instruction operation of the driver's seat occupant to the mobile terminal 3 in cooperation with the operation reception unit F2, and transfers an operation content signal indicating the content of the instruction operation to the connected terminal. The connected terminal executes processing according to the transferred operation content signal, and outputs image data to be newly displayed on the display 15 as the processing result. The application information acquisition unit F81 described above may be a sub-function provided in the terminal control unit F8.

With such a configuration, the driver's seat occupant directly operates the mobile terminal 3 with the functions provided by various applications installed in the mobile terminal 3, such as a call application, a mail application, and a web browser application. It can be used without any problems. That is, the subtask using the mobile terminal 3 can be performed with the face facing the display 15.

The call application here is an application that provides functions related to making and receiving calls, and the mail function is a function related to transmission / reception processing such as composing an e-mail and reading aloud the received e-mail. It is an application that provides. A web browser application refers to an application that provides a function of browsing a website on the Internet.

Further, the terminal control unit F8 includes an application information acquisition unit F81 as a sub-function for executing the various processes described above. The application information acquisition unit F81 cooperates with the short-range communication processing unit F7 to acquire information on the application (hereinafter referred to as application information) being executed on the connected terminal. Specifically, the application information acquisition unit F81 transmits a communication packet requesting the provision of application information to the connected terminal. The generation of the communication packet and the transmission to the connected terminal are realized by the cooperation of the short-range communication processing unit F7 and the short-range communication unit 17. Then, the application information acquisition unit F81 acquires the application information returned from the connected terminal as a response to the communication packet via the short-range communication unit 17 and the short-range communication processing unit F7. The vehicle cooperation application described above corresponds to application software that executes a process corresponding to a request from the HCU 10 and returns the process result to the HCU 10.

In the present embodiment, as an example, the application information is information including data about all the applications running on the connected terminal. The applications running on the mobile terminal 3 include applications running in the background.

The data for an app shall include at least the app name. Further, in the present embodiment, as a more preferable embodiment, it is assumed that the image data for displaying the screen corresponding to the application (hereinafter referred to as the application screen) on the display 15 is included. The application screen is a screen that displays the execution state of the application and displays information for inputting instructions for the application. The image displayed as the application screen corresponds to the application image described in the claims.

In the present embodiment, as an example, the application information includes data about all the running applications, but is not limited to this. The application information may include only the data of the application that was last operated by the user (hereinafter, the latest operation application) among the various applications that are running on the mobile terminal 3. The user for the mobile terminal 3 is the owner of the mobile terminal 3, and here, the driver's seat occupant corresponds to the user of the mobile terminal 3.

<Display processing at the time of mode transition>
Next, the mode transition display process performed by the HCU 10 will be described with reference to the flowchart shown in FIG. In the mode transition display process, when the operation mode transitions from the manual operation mode to the automatic operation mode, the display 15 displays a screen corresponding to / related to the subtask that the user is expected to execute from now on (hereinafter, the subtask screen). It is a process to do.

The flowchart shown in FIG. 4 may be started when the operation mode is switched from the manual operation mode to the automatic operation mode. The switching of the driving mode is detected by the driving mode determination unit F3 based on the mode information provided by the vehicle control ECU 12.

First, in step S100, the connection processing unit F72 determines whether or not the mobile terminal 3 is present in the vehicle interior based on the detection result of the mobile terminal detection unit F71. If the mobile terminal 3 is detected by the mobile terminal detection unit F71, step S100 is positively determined and the process proceeds to step S102. On the other hand, if the mobile terminal 3 is not detected by the mobile terminal detection unit F71, step S100 proceeds to step S110 in which a negative determination is made.

In step S102, the connection processing unit F72 establishes a communication connection with the detected mobile terminal 3 and proceeds to step S104. The mobile terminal 3 for establishing the communication connection in step S102 is preferably the mobile terminal 3 that has made the communication connection in the past. When only the mobile terminal 3 which has not been connected in the past is detected, the user is allowed to select whether or not to establish a communication connection. Of course, as another aspect, the connection processing unit F72 may establish a communication connection with the mobile terminal 3 existing in the vehicle interior regardless of the presence or absence of the past connection history. Further, the connection process with the mobile terminal 3 may be executed before the start of this flow, such as when the process of transferring the driving authority from the driver's seat occupant to the vehicle control ECU 12 is being executed.

In step S104, the application information acquisition unit F81 acquires application information from the mobile terminal 3 (that is, the connected terminal) for which the communication connection is established in step S102, and proceeds to step S106. In step S106, the display processing unit F1 displays a screen corresponding to the latest operation application on the display 15 based on the application information acquired in step S104, and ends this flow.

The latest operation application is an application in which a user operation is performed last among various applications running on the mobile terminal 3. In other words, it corresponds to an application whose operation is interrupted when the user starts manual operation. Therefore, by displaying the screen corresponding to the latest operation application on the display 15, the driver's seat occupant can continue the interrupted operation via the display 15 and the input device 14.

Further, the display 15 is arranged at a position where the orientation of the face is maintained within the detectable angle range even when the user is looking at the display 15. Therefore, at the timing when the operation mode is switched from the manual operation mode to the automatic operation mode, the screen of the latest operation application is displayed on the display 15 to maintain the state in which the direction of the user's face is directed to the detectable angle range. Can be done.

The display 15 on which the display processing unit F1 displays the application screen may be any of the head-up display 15A, the meter display 15B, and the center display 15C. This is because all the displays 15 are arranged in an area within the detectable angle range.

Here, as an example, the display processing unit F1 displays the application screen on the meter display 15B, but the display 15 as the output destination of the application screen may be appropriately designed. Further, the display 15 as the output destination of the application screen may be switched by the user. It is preferable that a message or the like indicating that a similar screen is displayed on the display 15 is output on the screen of the connected terminal.

In step S110, the display processing unit F1 displays a menu screen on the display 15 (for example, the center display 15C) and proceeds to step S112. The menu screen is a screen for selecting a function that the user wants to use from a plurality of functions that can be used by the user in the vehicle interior. Functions that can be used by the user in the vehicle interior include, for example, an air conditioning function that controls the operation of an air conditioner, a navigation function that is used to set a destination and confirm the current position, and music that performs music data playback processing. It is a viewing function. Note that the functions available to the user in the vehicle interior may include functions corresponding to various applications installed on the connected terminal. The menu screen is a screen showing a list of a plurality of functions available to the user, and may be, for example, a screen in which icon images corresponding to each of the plurality of functions available to the user are arranged.

In step S112, it is determined whether or not the operation reception unit F2 has accepted an operation for selecting one of the plurality of functions available to the user in the vehicle interior (hereinafter, function selection operation). When the function selection operation is accepted, step S112 is determined to be affirmative, the process proceeds to step S114, the screen corresponding to the selected function is displayed on the display 15, and this flow ends. On the other hand, if the function selection operation is not accepted even after a certain period of time has passed since the menu screen is displayed, the process proceeds to step S120.

In step S120, the task determination unit F5 determines whether or not the subtask is being executed based on the orientation of the user's face. That is, when the driver's seat occupant turns his face toward the subtask for a certain period of time, it is determined that the user is executing the subtask. In other cases, it may be determined that the subtask is not executed. If it is determined that the user is executing the subtask, the process proceeds to step S122. On the other hand, if it is determined that the subtask is not executed, this flow ends.

In step S122, the display processing unit F1 displays a subtask information input screen requesting input of subtask information on the display 15. The subtask information is information representing the content of the subtask executed by the driver's seat occupant, and is information that functions as a keyword (that is, a search word) used by the search server 2 when searching for information on the Internet. If the subtask executed by the driver's occupant is reading, the title of the book or the like corresponds to the subtask information. The subtask information is input by the driver's seat occupant operating the input device 14.

In step S124, the operation receiving unit F2 acquires subtask information based on the control signal input from the input device 14, and proceeds to step S126. The operation reception unit F2 that executes this step S124 and step S222 described later corresponds to the subtask information acquisition unit described in the claim. In addition, when the input device 14 is not operated even after the predetermined input waiting time elapses after the transition to the state of accepting the input of the subtask information (in other words, after displaying the subtask information input screen). Determines a timeout and ends this flow.

In step S126, the wide area communication processing unit F6 transmits a communication packet (hereinafter, a search request message) requesting the search server 2 to execute a search process using the subtask information as a search word, and acquires the search result. To do. When the process in step S126 is completed, the process proceeds to step S128.

In step S128, the display processing unit F1 displays a screen showing the search result on the display 15 and ends this flow. If, as a search result, digital content showing the same content as the subtask executed by the driver's seat occupant or digital content related to the subtask is found as the search result, the driver's seat occupant uses the display 15 and the input device 14. The digital content can be used through.

The digital contents here are movies, TV programs, video works, electronic books, songs, images, texts, games, and other applications expressed in digital data. Further, the digital content shall include not only the data itself but also a service on the Internet (for example, a schedule management service) provided by a predetermined service provider.

<Subtask screen provision processing>
Next, the subtask screen providing process performed by the HCU 10 will be described with reference to the flowchart shown in FIG. In the subtask screen providing process, when the task determination unit F5 determines that the driver's seat occupant is executing the subtask in the situation where the operation mode is the automatic operation mode, the screen corresponding to / related to the subtask is displayed. This is a process for displaying on 15.

The flowchart shown in FIG. 5 may be executed sequentially (for example, every second) when the traveling power source (for example, the ignition power source) of the vehicle is turned on. First, in step S200, the operation mode determination unit F3 determines whether or not the current operation mode is the automatic operation mode based on the mode information provided by the vehicle control ECU 12. When the current operation mode is the automatic operation mode, step S200 is positively determined and the process proceeds to step S202. On the other hand, when the current operation mode is the manual operation mode, step S200 is negatively determined and this flow ends.

In step S202, the task determination unit F5 determines whether or not the driver's seat occupant is executing the subtask based on the detection result of the DSM 13 acquired by the face orientation acquisition unit F4. If it is determined that the subtask is being executed, step S202 is determined to be affirmative and the process proceeds to step S204. On the other hand, if it is determined that the subtask is not executed, step S202 is determined to be negative and the present flow is terminated.

In step S204, the connection processing unit F72 determines whether or not the mobile terminal 3 is present in the vehicle interior, as in step S100 described above. If the mobile terminal 3 is detected by the mobile terminal detection unit F71, the process proceeds to step S206. On the other hand, if the mobile terminal 3 is not detected by the mobile terminal detection unit F71, the process proceeds to step S210.

In step S206, the terminal control unit F8 acquires the application information from the mobile terminal 3, displays the application screen corresponding to the latest operation application on the display 15, and ends this flow. If the communication connection with the mobile terminal 3 has not been established at the time of moving to step S206, the communication connection may be established at this point.

In step S210, the display processing unit F1 displays the menu screen on the predetermined display 15 and moves to step S212. In step S212, it is determined whether or not the operation reception unit F2 has accepted the function selection operation. When the function selection operation is accepted, step S212 is determined to be affirmative, the process proceeds to step S214, the screen corresponding to the selected function is displayed on the display 15, and this flow ends. On the other hand, if the function selection operation is not accepted even after a certain period of time has passed since the menu screen is displayed, the process proceeds to step S220.

In step S220, the display processing unit F1 displays the subtask information input screen on the display 15 and proceeds to step S222. In step S222, the operation reception unit F2 acquires subtask information based on the operation of the driver's seat occupant with respect to the input device 14, and proceeds to step S224. If the input device 14 is not operated even after a predetermined input waiting time has elapsed since the subtask information input screen is displayed, a timeout is determined and this flow is terminated.

In step S224, the wide area communication processing unit F6 transmits a search request message to the search server 2 and acquires the search result. When the process in step S224 is completed, the process proceeds to step S226. In step S226, the display processing unit F1 displays a screen showing the search result on the display 15 and ends this flow.

<Summary of Embodiment>
In the above configuration, the information related to the subtasks that can be executed by the driver's seat occupant is displayed on the display 15 at the timing of switching from the manual operation mode to the automatic operation mode. Specifically, when the mobile terminal 3 is brought into the vehicle interior by the driver's seat occupant, the screen of the latest operation application on the mobile terminal 3 is displayed on the display 15.

Further, when it is determined in step S120 that the driver's seat occupant is executing the subtask, the digital content related to the subtask information acquired based on the operation of the occupant is acquired from the Internet and displayed on the display 15.

As described above, the various displays 15 such as the head-up display 15A, the meter display 15B, and the center display 15C all have a view of the driver's seat occupant in a posture in which the driver's seat occupant faces the front of the vehicle. It is placed in a position to enter. Therefore, even when the driver's seat occupant faces any of these displays 15, the driver's seat occupant's face faces the front of the vehicle to some extent, and the traffic condition in front of the vehicle can be effectively viewed. It is possible to maintain the state captured with a stable gaze.

The effective field of view is a range in which things can be recognized, and is generally said to be a range of ± 35 degrees in the horizontal direction and ± 30 degrees in the vertical direction with the gazing point as the center. The stable gazing field is a range in which the focal point is quickly stabilized even when the gazing point is moved, and is in an elliptical range of ± 80 degrees in the horizontal direction and ± 60 degrees or less in the vertical direction. is there.

In particular, the various displays 15 such as the head-up display 15A, the meter display 15B, and the center display 15C in the present embodiment all have the orientation of the face of the driver's seat occupant looking at the display contents of the display 15 within the detectable angle range. It is placed in a position to be maintained. Therefore, even after the operation mode is switched from the manual operation mode to the automatic operation mode, the driver's seat occupant can maintain his / her face facing the detectable angle range.

Further, even during the automatic driving mode, the task determination unit F5 determines whether or not the driver's seat occupant is executing the subtask (step S202). Then, when the task determination unit F5 determines that the driver's seat occupant is executing the subtask, the screen of the application being executed on the mobile terminal 3 brought into the vehicle interior by the driver's seat occupant or , Information related to the subtask being executed by the driver's seat occupant is displayed on the display 15.

That is, not only at the timing of switching from the manual operation mode to the automatic operation mode, but also at the timing when a while has passed since the transition to the automatic operation mode, the display content of the display 15 corresponds to the subtask performed by the driver's seat occupant. Execute the process to display the screen. Therefore, it can be expected that the driver's seat occupant keeps his face facing the detectable angle range even while traveling in the automatic driving mode.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications described below are also included in the technical scope of the present invention, and other than the following. Can be changed and implemented within the range that does not deviate from the gist.

The members having the same functions as the members described in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted. Further, when only a part of the configuration is referred to, the configuration of the embodiment described above can be applied to the other parts.

[Modification 1]
The various displays 15 are preferably displays arranged at positions included in the effective field of view, among the regions included in the field of view of the driver's seat occupant who is in a posture of facing the front of the vehicle. The area included in the effective field of view of the driver's seat occupant who faces the front direction of the own vehicle is, for example, a predetermined angle (for example, 35 degrees) with respect to a line segment extending in the front direction of the vehicle from the center of the headrest of the driver's seat. ) Within the area. The predetermined angle here may be determined based on the angle range that forms the average effective visual field of a human being.

As a matter of course, the visual field characteristics are different for each individual. The installation position of the display 15 may be determined based on the average visual field identification. It should be noted that all of the various displays 15 disclosed in FIG. 2 represent a mode in which the displays 15 are arranged at positions included in the effective field of view of the driver's seat occupant who is in a posture of facing the front of the vehicle.

In such an aspect, the display 15 on which the subtask screen is displayed is arranged at a position within the field of view of the driver's seat occupant who is in a posture of facing the front of the vehicle. Therefore, even when the driver's seat occupant faces any of these displays 15, the state in which the vehicle exterior area in front of the vehicle is captured in the effective field of view or the peripheral field of view is maintained. In addition, since the face is not completely turned downward or in the passenger seat direction, the DSM 13 maintains a state in which the face of the driver's seat occupant can be imaged by a certain amount or more.

Further, the driver's seat occupant can visually recognize the front of the vehicle by moving the line of sight less from the state of looking at the display 15. That is, the amount of movement of the line of sight for checking the front of the vehicle during automatic driving is reduced, the focus adjustment load can be reduced, and safety can be improved.

As another aspect, a display arranged at a position included in the stable gaze of the driver's seat occupant who is in a posture of facing the front of the vehicle may be adopted as the display 15.

[Modification 2]
In the above-described embodiment, the search result of the search server 2 is displayed as the information related to the subtask performed by the driver's seat occupant, but the present invention is not limited to this. As shown in FIG. 6, when the content management server 4 which is a server for managing the digital contents owned by the driver's seat occupants exists among various external devices connected to the wide area communication network, the HCU 10 is the content management server. The digital content data corresponding to the subtask information input by the driver's seat occupant may be acquired from No. 4 and displayed on the display 15 (such a mode is referred to as a modification 2).

The content management server 4 stores information about digital contents owned by the driver's occupant in association with the user ID given to the driver's occupant. The user ID is a character string (including a number string) for identifying each of a plurality of persons who use the content management server 4.

The content management server 4 is connected to a database that stores digital contents owned by the driver's seat occupants, and transmits digital content data (hereinafter, content data) requested by the in-vehicle system 1 to the in-vehicle system 1. .. The distribution of the content data to the in-vehicle system 1 may be performed by a service provider that provides the data distribution service of the digital content. That is, the content management server 4 may be an entity that centrally manages services and the like used by the driver's seat occupants.

The HCU 10 in the second modification includes a content data acquisition unit F9 as shown in FIG. 6 as a functional block for acquiring digital content data corresponding to the subtask information input by the driver's seat occupant from the content management server 4. When the operation reception unit F2 acquires the subtask information in steps S124 and S222, the content data acquisition unit F9 informs the content management server 4 whether or not the driver's seat occupant possesses the digital content corresponding to the subtask information. Inquire. Then, when the digital content corresponding to the input subtask information exists as a result of the inquiry, the data of the content is acquired from the content management server 4 and displayed on the display 15.

Further, as a result of the inquiry, when the digital content corresponding to the input subtask information does not exist, the HCU 10 displays the search result screen on the search server 2 as in the above-described embodiment.

Whether the data corresponding to the subtask information is acquired from the search server 2 or the content management server 4 may be specified by the driver's seat occupant when the subtask information is input. For example, the display processing unit F1 has a button B1 for receiving a search execution command by the content management server 4 and a button B2 for receiving a search execution command by the search server 2 as a subtask information input screen, as shown in FIG. Display a screen that includes and. Then, the wide area communication processing unit F6 generates and transmits a communication packet targeting the server corresponding to the button selected by the driver's seat occupant.

According to such an aspect, the data stored in the content management server 4 (for example, the data recording a personal schedule) can be accessed via the display 15 or the input device 14. Therefore, it is possible to further increase the possibility that the driver's seat occupant will perform the subtask using the display 15 during the automatic driving. Along with this, the possibility that the in-vehicle system 1 exerts the effects described in the above-described embodiment and modification 1 can be further increased.

[Other various modifications]
If the mobile terminal detection unit F71 cannot detect the mobile terminal 3 at the time when the operation mode shifts from the manual operation mode to the automatic operation mode, the display processing unit F1 turns on the short-range communication function of the mobile terminal 3. The connection guide screen for guiding the communication connection with the in-vehicle system 1 (specifically, the short-range communication unit 17) may be displayed on the display 15. According to such an aspect, it is possible to reduce the possibility that the application screen cannot be displayed on the display 15 due to the short-range communication function of the mobile terminal 3 being set to off.

Further, in the above, the mode in which the mobile terminal 3 and the in-vehicle system 1 are wirelessly connected has been disclosed, but the present invention is not limited to this. The mobile terminal 3 and the in-vehicle system 1 may be connected by wire. The communication module for wired connection also corresponds to the vehicle interior communication module according to claim.

Further, in steps S106 and S206 described above, the mode of displaying the application screen on the predetermined display 15 without prior notice is disclosed, but the present invention is not limited to this. When the display processing unit F1 can acquire the data of the application screen, the display processing unit F1 displays a display confirmation screen prompting the selection of whether or not to display the application screen once. Then, after the operation reception unit F2 accepts the selection operation to display the application screen by the driver's seat occupant, the mode of actually displaying the application screen on the display 15 may be adopted.

1 In-vehicle system, 2 Search server, 3 Mobile terminal, 4 Content management server, 10 HCU, 11 Wide area communication unit, 12 Vehicle control ECU, 13 DSM, 14 Input device, 15 display, 15A head-up display, 15B meter display, 15C Center display, 16 indicator, 17 short-distance communication unit, F1 display processing unit, F2 operation reception unit, F3 operation mode judgment unit, F4 house orientation identification unit, F5 task judgment unit, F6 wide area communication processing unit, F7 short-range communication processing Unit, F71 mobile terminal detection unit, F72 connection processing unit, F8 terminal control unit, F81 application information acquisition unit, F9 content data acquisition unit

Claims (7)

Used in vehicles equipped with vehicle controls that provide automatic driving capabilities to automatically accelerate, brake, and steer the vehicle.
By performing mutual communication with the vehicle control device, the driving mode of the vehicle is either an automatic driving mode in which the vehicle control device automatically drives the vehicle, or a manual driving in which an occupant in the driver's seat performs a driving operation. The operation mode determination unit (F3) that determines whether the mode is set, and
A display processing unit (F1) that controls the display screen of the driver's seat display installed at a position included in the field of view of the driver's seat occupant facing the front direction of the vehicle, and
The vehicle interior communication processing unit (F7) that communicates with the information terminal brought into the vehicle interior by the occupant via the vehicle interior communication module for communicating with the information terminal existing in the vehicle interior.
A terminal control unit (F8) that controls the operation of the information terminal communicating with the vehicle interior communication processing unit, and
A face that sequentially acquires information indicating the orientation of the occupant's face based on an image captured by a camera provided in an area facing the driver's seat in the vehicle interior so as to image the face of the occupant seated in the driver's seat. Orientation acquisition unit (F4) and
A tool brought into the vehicle interior by the driver's seat occupant based on the face orientation of the occupant acquired by the face orientation acquisition unit when the operation mode is determined to be the automatic driving mode by the operation mode determination unit. Task determination unit (F5) that determines whether or not a subtask using
A wide area communication processing unit (F6) that transmits / receives data to / from an external device connected to the wide area communication network via a wide area communication module for wirelessly accessing the wide area communication network.
When the information terminal with which the vehicle interior communication processing unit is executing communication does not exist and the task determination unit determines that the driver's seat occupant is executing the subtask, the vehicle is mounted on the vehicle. Subtask information acquisition units (S124, S222) that acquire subtask information indicating the contents of the subtask based on the operation of the occupant with respect to the input device.
A content data acquisition unit (F9) that acquires digital content data corresponding to the subtask information from a predetermined database connected to the wide area communication network using the subtask information acquired by the subtask information acquisition unit as a search word. And with
The terminal control unit
In addition to acquiring an application image that is an image corresponding to the application software running on the information terminal,
Based on the operation of the occupant with respect to the input device, the information terminal receives the instruction operation of the occupant with respect to the information terminal and causes the information terminal to execute a process corresponding to the instruction operation.
The display processing unit is a display control device for a vehicle, characterized in that an image corresponding to the digital content acquired by the content data acquisition unit is displayed on the driver's seat display as a subtask screen. In claim 1,
When the occupant has faced the occupant for a certain period of time in the downward direction or in the direction in which the passenger seat is arranged at a predetermined angle or more with respect to the front direction of the vehicle, the task determination unit causes the occupant to perform the subtask. A display control device for a vehicle, characterized in that it is determined that the vehicle is executing. The display control for a vehicle according to claim 1 or 2, wherein the display processing unit displays the application image on the driver's seat display when the driving mode changes from the manual driving mode to the automatic driving mode. apparatus. In any one of claims 1 to 3,
If the vehicle interior communication processing unit cannot detect the information terminal at the time when the operation mode becomes the automatic driving mode, the display processing unit displays a list of functions available to the occupant in the vehicle interior. A display control device for a vehicle, characterized in that the displayed menu screen is displayed on the driver's seat display. Used in vehicles equipped with vehicle controls that provide automatic driving capabilities to automatically accelerate, brake, and steer the vehicle.
By performing mutual communication with the vehicle control device, the driving mode of the vehicle is either an automatic driving mode in which the vehicle control device automatically drives the vehicle, or a manual driving in which an occupant in the driver's seat performs a driving operation. The operation mode determination unit (F3) that determines whether the mode is set, and
A display processing unit (F1) that controls the display screen of the driver's seat display installed at a position included in the field of view of the driver's seat occupant facing the front direction of the vehicle, and
The vehicle interior communication processing unit (F7) that communicates with the information terminal brought into the vehicle interior by the occupant via the vehicle interior communication module for communicating with the information terminal existing in the vehicle interior.
A terminal control unit (F8) that controls the operation of the information terminal communicating with the vehicle interior communication processing unit is provided.
The terminal control unit
In addition to acquiring an application image that is an image corresponding to the application software running on the information terminal,
Based on the operation of the occupant with respect to the input device mounted on the vehicle, the information terminal receives the instruction operation of the occupant with respect to the information terminal and causes the information terminal to execute a process corresponding to the instruction operation.
A face that sequentially acquires information indicating the orientation of the occupant's face based on an image captured by a camera provided in an area facing the driver's seat in the vehicle interior so as to image the face of the occupant seated in the driver's seat. Orientation acquisition unit (F4) and
A tool brought into the vehicle interior by the driver's seat occupant based on the face orientation of the occupant acquired by the face orientation acquisition unit when the operation mode is determined to be the automatic driving mode by the operation mode determination unit. Task determination unit (F5) that determines whether or not a subtask using
A wide area communication processing unit (F6) that transmits / receives data to / from an external device connected to the wide area communication network via a wide area communication module for wirelessly accessing the wide area communication network.
In the absence of the information terminal with which the vehicle interior communication processing unit is executing communication, when the task determination unit determines that the driver's seat occupant is executing the subtask, the said to the input device. Subtask information acquisition units (S124, S222) that acquire subtask information indicating the contents of the subtask based on the operation of the occupant, and
A content data acquisition unit (F9) that acquires digital content data corresponding to the subtask information from a predetermined database connected to the wide area communication network using the subtask information acquired by the subtask information acquisition unit as a search word. And, with more
The display processing unit displays the application image on the driver's seat display when the operation mode shifts from the manual operation mode to the automatic operation mode.
If the vehicle interior communication processing unit cannot detect the information terminal at the time when the operation mode becomes the automatic operation mode, the information terminal and the vehicle interior communication module are connected in order to make a communication connection. A connection guide screen for guiding the setting of the short-range communication function of the information terminal is displayed on the driver's seat display .
When the task determination unit determines that the occupant is executing the subtask, the application image is displayed as a subtask screen on the driver's seat display.
A display control device for a vehicle, characterized in that an image corresponding to the digital content acquired by the content data acquisition unit is displayed on the driver's seat display as the subtask screen . In claim 5 ,
When the occupant has faced the occupant for a certain period of time in the downward direction or in the direction in which the passenger seat is arranged at a predetermined angle or more with respect to the front direction of the vehicle, the task determination unit causes the occupant to perform the subtask. A display control device for a vehicle, characterized in that it is determined that the vehicle is executing. In any one of claims 1 to 6 ,
The driver's seat display is a meter display provided in the driver's seat facing area, which is an area facing the driver's seat in the instrument panel, and the instrument panel has a center display and a window provided in the center in the width direction. A display control device for a vehicle, which is one of a head-up display that projects a virtual image on a shield in an area facing the driver's seat.

Images