WO2018127962A1

WO2018127962A1 - Reporting control device and reporting control method

Info

Publication number: WO2018127962A1
Application number: PCT/JP2017/000193
Authority: WO
Inventors: 真規塚田; 下谷　光生
Original assignee: 三菱電機株式会社
Priority date: 2017-01-06
Filing date: 2017-01-06
Publication date: 2018-07-12
Also published as: JPWO2018127962A1; JP6661796B2

Abstract

The purpose of the present invention is to provide a reporting control device and a reporting control method that make it possible to perform proper reporting to a driver. A reporting control device according to the present invention is provided with: a peripheral information obtaining unit that obtains peripheral information indicating at least the peripheral situation of a local vehicle; a primary-event recognition unit that recognizes, from the peripheral information obtained by the peripheral information obtaining unit, a primary event, which is a situation regarding an object or a person present around the local vehicle; a secondary-event estimation unit that estimates, in accordance with a predetermined rule, a secondary event, which is a situation that can occur after the primary event recognized by the primary-event recognition unit and that has a causal relationship with the situation of the primary event; and a reporting control unit that performs control to report the secondary event estimated by the secondary-event estimation unit.

Description

Notification control device and notification control method

The present invention relates to a notification control apparatus and a notification control method for notifying a driver of a situation around a host vehicle.

2. Description of the Related Art Conventionally, a technique for alerting a driver of an own vehicle by detecting an object or person existing around the own vehicle and highlighting the detected object or person has been disclosed (for example, Patent Documents). 1, 2, 3).

Japanese Patent Laid-Open No. 11-115660 JP 2008-62762 A JP 2008-310376 A

For example, when a ball is present in front of the traveling vehicle, it is conceivable that the child will jump out in front of the traveling vehicle to take the ball. If a situation that may occur after such an actual situation is notified to the driver in addition to the actual situation, the driver can take an appropriate action in consideration of the situation that may occur in the future.

Patent Documents

1, 2, and 3 merely notify the driver of the actual situation, and do not notify the driver of a situation that may occur after the actual situation. Therefore, in

Patent Documents

1, 2, and 3, it cannot be said that proper notification is given to the driver.

The present invention has been made to solve such a problem, and an object of the present invention is to provide a notification control device and a notification control method capable of performing appropriate notification to a driver.

In order to solve the above-described problem, a notification control apparatus according to the present invention includes a surrounding information acquisition unit that acquires at least surrounding information indicating a situation around the own vehicle, and the surrounding information acquired by the surrounding information acquisition unit. A first event recognition unit for recognizing a first event which is a situation of an object or a person existing in the vicinity of the first event, and a situation that can occur after the first event recognized by the first event recognition unit, A secondary event estimator that estimates a secondary event, which has a causal relationship with the situation of the primary event, according to a predetermined rule, and a secondary event estimated by the secondary event estimator is reported. A notification control unit that performs control.

In addition, the notification control method according to the present invention acquires at least peripheral information indicating a situation around the host vehicle, and obtains a first event that is a situation of an object or person existing around the host vehicle from the acquired peripheral information. Recognizing and recognizing a secondary event that can occur after the recognized primary event and has a causal relationship with the situation of the primary event according to a predetermined rule Control to notify the secondary event is performed.

According to the present invention, the notification control device includes at least a peripheral information acquisition unit that acquires peripheral information indicating a situation around the host vehicle, and an object existing around the host vehicle from the peripheral information acquired by the peripheral information acquisition unit. A first event recognition unit that recognizes a first event that is a person's situation, and a situation that can occur after the first event recognized by the first event recognition unit, the situation and the causality of the first event A secondary event estimator that estimates a secondary event that is a related situation according to a predetermined rule, and a notification controller that performs control to notify the secondary event estimated by the secondary event estimator Therefore, it is possible to appropriately notify the driver.

In addition, the notification control method acquires at least peripheral information indicating a situation around the host vehicle, and recognizes a primary event that is a situation of an object or a person existing around the host vehicle from the acquired surrounding information, A secondary event that is a situation that can occur after the recognized primary event and has a causal relationship with the situation of the primary event is estimated according to a predetermined rule. Therefore, it is possible to appropriately notify the driver.

The objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description and the accompanying drawings.

It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a figure which shows an example of the 1st event and the 2nd event by Embodiment 1 of this invention. It is a block diagram which shows an example of the hardware constitutions of the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a figure which shows an example of the display in the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a figure which shows an example of the display in the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 2 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 2 of this invention. It is a figure which shows an example of the display in the alerting | reporting control apparatus by Embodiment 2 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 3 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 3 of this invention. It is a figure for demonstrating the recognition range in the alerting control apparatus by Embodiment 3 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 4 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 4 of this invention. It is a flowchart which shows an example of operation | movement of the external server by Embodiment 4 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 5 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 5 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control system by embodiment of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control system by embodiment of this invention.

Embodiments of the present invention will be described below based on the drawings.

<Embodiment 1>
<Configuration>
First, the configuration of the notification control apparatus according to Embodiment 1 of the present invention will be described.

FIG. 1 is a block diagram showing an example of the configuration of the notification control apparatus 1 according to the first embodiment. Note that FIG. 1 shows the minimum necessary components constituting the notification control apparatus according to the present embodiment.

As shown in FIG. 1, the notification control device 1 includes a peripheral information acquisition unit 2, a primary event recognition unit 3, a secondary event estimation unit 4, and a notification control unit 5. The peripheral information acquisition unit 2 acquires peripheral information indicating at least the situation around the host vehicle. The primary event recognition unit 3 recognizes a primary event that is a situation of an object or person existing around the host vehicle from the peripheral information acquired by the peripheral information acquisition unit 2. The primary event recognized by the primary event recognition unit may be a situation where the object or person is stationary, or a situation where the object or person is moving.

The secondary event estimation unit 4 is a situation that can occur after the primary event recognized by the primary event recognition unit 3 and has a causal relationship with the situation of the primary event. Is estimated according to a predetermined rule. The notification control unit 5 performs control to notify the second event estimated by the second event estimation unit 4.

Next, another configuration of the notification control device including the notification control device 1 shown in FIG. 1 will be described.

FIG. 2 is a block diagram showing an example of the configuration of the notification control device 6 according to another configuration.

As shown in FIG. 2, the notification control device 6 includes a surrounding information acquisition unit 2, a primary event recognition unit 3, a secondary event estimation unit 4, a notification control unit 5, and a host vehicle information acquisition unit 7. A map information acquisition unit 8 and an alarm image generation unit 9. The peripheral information acquisition unit 2 is connected to the peripheral recognition device 10. The notification control unit 5 is connected to a HUD (Head Up Display) 11.

The periphery recognition device 10 recognizes at least the situation around the host vehicle and outputs the recognized situation around the host vehicle to the periphery information acquisition unit 2 as the periphery information. The periphery recognition device 10 is installed in the host vehicle, and is configured by, for example, a camera, a millimeter wave radar, or a combination thereof. The periphery recognition device 10 is not limited to the camera and the millimeter wave radar, and may be any device that recognizes the surrounding situation of the host vehicle. The peripheral information acquisition unit 2 acquires peripheral information output from the peripheral recognition device 10.

The own vehicle information acquisition unit 7 acquires own vehicle information, which is various information about the own vehicle, via an in-vehicle network provided in the own vehicle. For example, the host vehicle information acquisition unit 7 acquires the current position information of the host vehicle from the GPS (Global Positioning System) as host vehicle information.

The map information acquisition unit 8 is constituted by a storage device such as a hard disk (HDD: Hard Disk Drive) or a semiconductor memory, for example, and acquires and stores map information. The map information acquisition unit 8 may acquire map information from the outside. For example, the map information acquisition unit 8 may be acquired by downloading from an external server or the like via a communication network, or may be acquired by reading from a storage medium such as a memory.

For example, when the peripheral recognition device 10 is a camera, the primary event recognition unit 3 recognizes a primary event that is a situation of an object or a person from an image that is peripheral information acquired by the peripheral information acquisition unit 2. The first event recognition unit 3 may recognize the first event based on the current position information of the host vehicle acquired by the host vehicle information acquisition unit 7 and the map information acquired by the map information acquisition unit 8. Good. Note that the primary event recognition unit 3 may recognize the primary event by arbitrarily combining the peripheral information, the host vehicle information, and the map information.

The secondary event estimation unit 4 estimates the secondary event from the primary event recognized by the primary event recognition unit 3 according to a predetermined rule. The predetermined rule is data that is converted into data and held by the secondary event estimation unit 4, and as an example, the primary event and the secondary event as shown in FIG. 3 are associated with each other.

For example, when the primary event recognition unit 3 recognizes “ball jumping out” as the primary event, the secondary event estimation unit 4 estimates “child jumping out” as the secondary event. That is, the secondary event estimation unit 4 estimates that when a ball pops out, a child pops out to take the ball. In this case, the primary event recognition unit 3 can recognize the ball from an image captured by a camera that is the periphery recognition device 10, for example. The primary event recognition unit 3 is not limited to recognizing the movement of the ball as the primary event, but may recognize that the ball is stationary as the primary event.

When the primary event recognizing unit 3 recognizes “stopped vehicle” parked on the side of the road as the first event, the secondary event estimating unit 4 performs “second jump of a person or motorcycle from behind the vehicle”. Estimated as the next event. That is, the secondary event estimation unit 4 estimates that a person or a motorcycle jumps out from behind the vehicle when there is a stopped vehicle. In this case, the primary event recognition unit 3 can recognize a stopped vehicle from an image captured by a camera that is the periphery recognition device 10, for example.

When the primary event recognition unit 3 recognizes “crossing” as the primary event, the secondary event estimation unit 4 estimates “passing the train” as the secondary event. That is, the secondary event estimation unit 4 estimates that the train passes when there is a railroad crossing. In this case, the primary event recognition unit 3 can recognize a railroad crossing from, for example, an image captured by a camera that is the periphery recognition device 10. Alternatively, the primary event recognition unit 3 can recognize a railroad crossing based on the current position information of the host vehicle acquired by the host vehicle information acquisition unit 7 and the map information acquired by the map information acquisition unit 8.

When the primary event recognition unit 3 recognizes “vehicle lane reduction” as the primary event, the secondary event estimation unit 4 estimates “interrupt vehicle” as the secondary event. That is, when the vehicle lane on the road on which the host vehicle travels decreases, the secondary event estimation unit 4 estimates that the other vehicle will enter the vehicle lane on which the host vehicle is traveling. In this case, the primary event recognition unit 3 can recognize that the vehicle lane of the road on which the host vehicle is traveling decreases from an image captured by a camera that is the periphery recognition device 10, for example. Alternatively, the primary event recognition unit 3 is based on the current position information of the host vehicle acquired by the host vehicle information acquisition unit 7 and the map information acquired by the map information acquisition unit 8. It can be recognized that the lane decreases. In addition, as a case where the vehicle lane of the road where the own vehicle travels decreases, for example, the case where the vehicle lane decreases from two lanes to one lane can be cited. In addition, the first event when the second event is “interrupted vehicle” is not limited to the case where the vehicle lane decreases, but may be a vehicle lane in which the host vehicle is traveling, such as a junction of an expressway. A situation in which another vehicle is supposed to interrupt may be used as the first event.

When the primary event recognizing unit 3 recognizes “a person holding an object facing this on an intersecting overhead road” as the primary event, the secondary event estimating unit 4 determines that “the object has fallen”. Is estimated as a secondary event. Examples of the overhead road that intersects include a pedestrian bridge. That is, for example, when there is a person holding an object on a pedestrian bridge that intersects the road on which the host vehicle is traveling, the secondary event estimation unit 4 determines that the object that the person has is the road on which the host vehicle is traveling. Estimated to fall. In this case, the primary event recognizing unit 3 recognizes that there is a person who is facing the object on the intersecting overhead road from the image taken by the camera that is the periphery recognition device 10, for example. Can do.

The alarm image generation unit 9 generates a primary event alarm image indicating the primary event recognized by the primary event recognition unit 3. Further, the warning image generation unit 9 generates a secondary event warning image indicating the secondary event estimated by the secondary event estimation unit 4.

The notification control unit 5 performs control to display the first event alarm image or the second event alarm image generated by the alarm image generation unit 9 on the HUD 11. A primary event warning image or a secondary event warning image is displayed on the HUD 11 using AR (Augmented Reality) technology. The driver can see the first event warning image or the second event warning image superimposed on the scenery seen through the windshield of the host vehicle. Note that the notification control unit 5 may perform control to simultaneously display the first event warning image and the second event warning image on the HUD 11.

FIG. 4 is a block diagram illustrating an example of a hardware configuration of the notification control device 6. The same applies to the notification control apparatus 1.

Peripheral information acquisition unit 2, primary event recognition unit 3, secondary event estimation unit 4, notification control unit 5, own vehicle information acquisition unit 7, map information acquisition unit 8, and alarm image generation unit in notification control device 6 Each function of 9 is realized by a processing circuit. That is, the notification control device 6 acquires peripheral information, recognizes a first event, estimates a second event, controls notification, acquires own vehicle information, acquires map information, and acquires first information. A processing circuit is provided for generating a next event warning image and a second event warning image. The processing circuit is a processor 12 (also called a central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, DSP (Digital Signal Processor)) that executes a program stored in the memory 13.

Peripheral information acquisition unit 2, primary event recognition unit 3, secondary event estimation unit 4, notification control unit 5, own vehicle information acquisition unit 7, map information acquisition unit 8, and alarm image generation unit in notification control device 6 Each function of 9 is realized by software, firmware, or a combination of software and firmware. Software or firmware is described as a program and stored in the memory 13. The processing circuit reads out and executes the program stored in the memory 13, thereby realizing the function of each unit. That is, the notification control device 6 obtains peripheral information, a step of recognizing a first event, a step of estimating a second event, a step of controlling notification, a step of acquiring own vehicle information, and map information. A memory 13 is provided for storing a program in which the step of obtaining, the step of generating the primary event warning image and the secondary event warning image will be executed as a result. Moreover, these programs are the surrounding information acquisition part 2, the 1st event recognition part 3, the 2nd event estimation part 4, the alerting | reporting control part 5, the own vehicle information acquisition part 7, the map information acquisition part 8, and an alarm image. It can also be said that the computer executes the procedure or method of the generation unit 9. Here, the memory is non-volatile or volatile such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), etc. May be any other semiconductor memory, magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), or any other storage medium used in the future.

<Operation>
Next, the operation of the notification control device 6 will be described.

FIG. 5 is a flowchart showing an example of the operation of the notification control device 6. In the following, it is assumed that the periphery recognition device 10 is a camera and photographs the front of the host vehicle.

In step S11, the peripheral information acquisition unit 2 acquires an image captured by the peripheral recognition device 10 as peripheral information. In step S <b> 12, the primary event recognition unit 3 recognizes the primary event from the image acquired by the peripheral information acquisition unit 2. Here, the primary event recognition unit 3 recognizes the ball existing in front of the host vehicle as the primary event.

In step S13, the alarm image generation unit 9 generates a primary event alarm image indicating the primary event recognized by the primary event recognition unit 3. The notification control unit 5 performs control to display the first event warning image generated by the warning image generation unit 9 on the HUD 11.

FIG. 6 is a diagram illustrating an example when the first event warning image is displayed on the HUD 11. In FIG. 6, the primary event warning image 14 is an image that emphasizes the ball itself. The first event warning image 15 is an image that alerts the presence of an object that obstructs traveling ahead of the host vehicle, and an image that alerts the presence of a ball in the example of FIG. 6. It is. The primary event warning image 14 and the primary event warning image 15 are not limited to being displayed together as shown in FIG. 6, and either one may be displayed.

In step S14, the secondary event estimation unit 4 estimates the secondary event from the primary event recognized by the primary event recognition unit 3. Specifically, the secondary event estimation unit 4 estimates the secondary event based on data as shown in FIG. Here, the secondary event estimation unit 4 estimates “child's jumping out” as the secondary event.

In step S15, the warning image generation unit 9 generates a secondary event warning image indicating the secondary event estimated by the secondary event estimation unit 4. The notification control unit 5 performs control to display the second event warning image generated by the warning image generation unit 9 on the HUD 11.

FIG. 7 is a diagram showing an example when the second event warning image is displayed on the HUD 11. In FIG. 7, the secondary event warning image 16 is an image in which a child jumps out in front of the host vehicle. The secondary event warning image 16 is displayed at a predetermined position. The position at which the second event warning image 16 is displayed may be arbitrarily set by the user.

From the above, according to the first embodiment, the notification control devices 1 and 6 not only provide the primary event that is the actual situation, but also the secondary event that can occur after the primary event. Since the notification is made, it is possible to appropriately notify the driver. Therefore, the driver can take an appropriate response in consideration of the secondary event.

<Embodiment 2>
<Configuration>
First, the configuration of the notification control apparatus according to Embodiment 2 of the present invention will be described.

FIG. 8 is a block diagram showing an example of the configuration of the notification control device 17 according to the second embodiment.

As shown in FIG. 8, the notification control device 17 is characterized by including a second event position estimation unit 18. Since other configurations and operations are the same as those in the first embodiment, detailed description thereof is omitted here.

The secondary event position estimation unit 18 estimates the occurrence position of the secondary event estimated by the secondary event estimation unit 4 based on the position where the primary event recognition unit 3 has recognized the primary event. . The alarm image generation unit 9 generates a secondary event alarm image corresponding to the occurrence position estimated by the secondary event position estimation unit 18.

The function of the secondary event position estimation unit 18 in the notification control device 17 is realized by a processing circuit. That is, the secondary event position estimation unit 18 includes a processing circuit for estimating the occurrence position of the secondary event. The processing circuit is, for example, a processor 12 that executes a program stored in the memory 13 as shown in FIG.

The function of the secondary event position estimation unit 18 in the notification control device 17 is realized by software, firmware, or a combination of software and firmware. Software or firmware is described as a program and stored in the memory 13. The processing circuit implements the function of the secondary event position estimating unit 18 by reading and executing the program stored in the memory 13. That is, the notification control device 17 includes a memory 13 for storing a program in which the step of estimating the occurrence position of the second event is executed as a result. It can also be said that this program causes the computer to execute the procedure or method of the secondary event position estimation unit 18.

<Operation>
Next, the operation of the notification control device 17 will be described.

FIG. 9 is a flowchart showing an example of the operation of the notification control device 17. Note that steps S21 to S24 in FIG. 9 correspond to steps S11 to S14 in FIG. Below, step S25 and step S26 are demonstrated. The periphery recognition device 10 is a camera and shoots ahead of the host vehicle.

In step S25, the secondary event position estimation unit 18 determines a predetermined position near the position where the primary event recognition unit 3 has recognized the primary event in the image acquired by the peripheral information acquisition unit 2. Presumed as the occurrence position of the secondary event.

In step S26, the alarm image generation unit 9 generates a secondary event alarm image corresponding to the occurrence position of the secondary event estimated by the secondary event position estimation unit 18. The notification control unit 5 performs control to display the second event warning image generated by the warning image generation unit 9 on the HUD 11.

FIG. 10 is a diagram illustrating an example when the second event warning image is displayed on the HUD 11. In FIG. 10, the secondary event warning image 16 is displayed at the occurrence position estimated by the secondary event position estimation unit 18. In the example of FIG. 10, a state in which a child follows the ball and jumps out ahead of the host vehicle is expressed.

From the above, according to the second embodiment, the notification control device 17 displays the second event at a position where the second event is highly likely to occur, so the driver performs the second event. It is possible to grasp more accurately than the first form.

<Embodiment 3>
<Configuration>
First, the configuration of the notification control apparatus according to Embodiment 3 of the present invention will be described.

FIG. 11 is a block diagram illustrating an example of the configuration of the notification control device 19 according to the third embodiment.

As shown in FIG. 11, the notification control device 19 includes a recognition range restriction unit 20. Since other configurations and operations are the same as those in the second embodiment, detailed description thereof is omitted here.

The recognition range restriction unit 20 restricts the recognition range by the first event recognition unit 3 to a predetermined range including the occurrence position of the second event estimated by the second event position estimation unit 18. The primary event recognition unit 3 recognizes the primary event within the range restricted by the recognition range restriction unit 20. For example, when the primary event recognition unit 3 recognizes the primary event from the image, the primary event recognition unit 3 recognizes the primary event within the range of the image restricted by the recognition range restriction unit 20. I do.

The function of the recognition range restriction unit 20 in the notification control device 19 is realized by a processing circuit. That is, the recognition range restriction unit 20 restricts the recognition range by the primary event recognition unit 3 to a predetermined range including the occurrence position of the secondary event estimated by the secondary event position estimation unit 18. The processing circuit is provided. The processing circuit is, for example, a processor 12 that executes a program stored in the memory 13 as shown in FIG.

The function of the recognition range restriction unit 20 in the notification control device 19 is realized by software, firmware, or a combination of software and firmware. Software or firmware is described as a program and stored in the memory 13. The processing circuit implements the function of the recognition range restriction unit 20 by reading and executing the program stored in the memory 13. That is, the notification control device 19 includes a step of limiting the recognition range by the primary event recognition unit 3 to a predetermined range including the occurrence position of the secondary event estimated by the secondary event position estimation unit 18. A memory 13 is provided for storing a program to be executed as a result. It can also be said that this program causes a computer to execute the procedure or method of the recognition range restriction unit 20.

<Operation>
Next, the operation of the notification control device 19 will be described.

FIG. 12 is a flowchart showing an example of the operation of the notification control device 19. Note that steps S31 to S36 in FIG. 12 correspond to steps S21 to S26 in FIG. Below, step S37 and step S38 are demonstrated. The periphery recognition device 10 is a camera and shoots ahead of the host vehicle.

In step S37, the notification control device 19 determines whether or not to limit the recognition range by the primary event recognition unit 3. The determination of whether or not to limit the recognition range by the primary event recognition unit 3 may be performed according to the resolution of the image acquired by the peripheral information acquisition unit 2. Specifically, when the resolution of the image is high, the processing load on the primary event recognition unit 3 increases, and therefore it may be determined that the recognition range by the primary event recognition unit 3 is limited. When the resolution of the image is low, it may be determined that the recognition range by the primary event recognition unit 3 is not limited because the processing load on the primary event recognition unit 3 is small. The determination as to whether or not to limit the recognition range by the primary event recognition unit 3 may be arbitrarily set by the user. When limiting the recognition range by the primary event recognition part 3, it transfers to step S38. On the other hand, when the recognition range by the primary event recognition unit 3 is not limited, the process proceeds to step S31.

In step S38, the recognition range restriction unit 20 restricts the recognition range by the primary event recognition unit 3 to a predetermined range including the occurrence position of the secondary event estimated by the secondary event position estimation unit 18. To do. Specifically, the recognition range restriction unit 20 restricts the recognition range by the primary event recognition unit 3 to, for example, a recognition range 21 as shown in FIG. Note that the recognition range 21 shown in FIG. 13 is shown for explanation, and is not actually displayed on the HUD 11.

From the above, according to the third embodiment, since the recognition range by the primary event recognition unit 3 is limited, the efficiency of recognizing the primary event by the primary event recognition unit 3 can be improved. The secondary event can be displayed quickly. Moreover, even if the resources for recognizing the primary event by the primary event recognition unit 3 are limited, the primary event recognition unit 3 can smoothly recognize the primary event. it can.

In the example of FIG. 12, the primary event recognition unit 3 recognizes the primary event within the range limited by the recognition range limitation unit 20 for all images acquired by the peripheral information acquisition unit 2. However, the present invention is not limited to this. For example, the primary event recognition unit 3 recognizes the primary event for all images within the range restricted by the recognition range restriction unit 20 and is outside the range restricted by the recognition range restriction unit 20. May recognize the primary event for the image acquired by the peripheral information acquisition unit 2 at predetermined intervals.

<Embodiment 4>
<Configuration>
First, the configuration of a notification control apparatus according to Embodiment 4 of the present invention will be described.

FIG. 14 is a block diagram showing an example of the configuration of the notification control device 22 according to the fourth embodiment.

As shown in FIG. 14, the notification control device 22 is characterized by including a communication unit 23. Other configurations and operations are the same as those in the third embodiment, and thus detailed description thereof is omitted here.

The communication unit 23 is communicably connected to each of the image recognition learning server 24 and the second event estimation data server 25 provided outside the notification control device 22, and the image recognition learning server 24 and the second event. Parameters and data to be described later are acquired from each of the estimated data servers 25.

The image recognition learning server 24 improves the recognition accuracy of the primary event by the primary event recognition unit 3 for parameters used when the primary event recognition unit 3 recognizes the primary event from the image. Machine learning. The secondary event estimation data server 25 accumulates data used when the secondary event estimation unit 4 estimates the secondary event. The image recognition learning server 24 and the second event estimation data server 25 may be provided separately or may be provided integrally.

The function of the communication unit 23 in the notification control device 22 is realized by a processing circuit. That is, the communication unit 23 includes a processing circuit for communicating with each of the image recognition learning server 24 and the second event estimation data server 25. The processing circuit is, for example, a processor 12 that executes a program stored in the memory 13 as shown in FIG.

The function of the communication unit 23 in the notification control device 22 is realized by software, firmware, or a combination of software and firmware. Software or firmware is described as a program and stored in the memory 13. The processing circuit implements the function of the communication unit 23 by reading and executing the program stored in the memory 13. That is, the notification control device 22 includes a memory 13 for storing a program in which a step of communicating with each of the image recognition learning server 24 and the second event estimation data server 25 is executed as a result. It can also be said that this program causes a computer to execute the procedure or method of the communication unit 23.

<Operation>
Next, the operation of the notification control device 22 will be described.

FIG. 15 is a flowchart showing an example of the operation of the notification control device 22. Note that steps S41 to S48 in FIG. 15 correspond to steps S31 to S38 in FIG. In step S47 of FIG. 15, if the recognition range by the primary event recognition unit 3 is not limited, the process proceeds to step S49. Hereinafter, step S49 will be described. The periphery recognition device 10 is a camera and shoots ahead of the host vehicle.

In step S49, the communication unit 23 acquires, from the image recognition learning server 24, parameters used when the primary event recognition unit 3 recognizes the primary event from the image. The parameter used by the primary event recognition unit 3 is updated to the parameter acquired by the communication unit 23. Further, the communication unit 23 acquires data used when the secondary event estimation unit 4 estimates the secondary event from the secondary event estimation data server 25. The data used by the second event estimation unit 4 is updated to the data acquired by the communication unit 23. Note that the process of step S49 is not limited to being performed after step S47 or step S48, and may be performed at an arbitrary timing.

Next, operations of the image recognition learning server 24 and the secondary event estimation data server 25 will be described.

FIG. 16 is a flowchart showing an example of operations of the image recognition learning server 24 and the secondary event estimation data server 25.

In step S51, the image recognition learning server 24 recognizes the primary event recognition accuracy by the primary event recognition unit 3 for parameters used when the primary event recognition unit 3 recognizes the primary event from the image. Do machine learning that will improve. Machine-learned parameters are updated.

In step S52, the secondary event estimation data server 25 accumulates data used when the secondary event estimation unit 4 estimates the secondary event. Specifically, the secondary event estimation data server 25 accumulates data associating the primary event and the secondary event as shown in FIG. 3, for example, and updates it as necessary.

In step S <b> 53, the image recognition learning server 24 transmits parameters used when the primary event recognition unit 3 recognizes the primary event from the image to the communication unit 23 of the notification control device 22. Further, the secondary event estimation data server 25 transmits data used when the secondary event estimation unit 4 estimates the secondary event to the communication unit 23 of the notification control device 22. Note that the image recognition learning server 24 and the second event estimation data server 25 may transmit parameters and data simultaneously or separately. Further, the timing at which the image recognition learning server 24 and the second event estimation data server 25 transmit parameters and data may be a predetermined timing or an arbitrary timing set by the user.

From the above, according to the fourth embodiment, the primary event recognition unit 3 updates the parameters used when the primary event recognition unit 3 recognizes the primary event from the image, so that the primary event recognition unit 3 It becomes possible to improve the recognition accuracy of the primary event. Moreover, the accuracy which the secondary event estimation part 4 estimates a secondary event can be improved by updating the data used when the secondary event estimation part 4 estimates a secondary event. .

In addition, although FIG. 14 demonstrated the case where the notification control apparatus 19 shown in FIG. 11 was further equipped with the communication part 23, it is not restricted to this. For example, the notification control device 6 shown in FIG. 2 and the notification control device 17 shown in FIG.

<Embodiment 5>
<Configuration>
First, the configuration of the notification control apparatus according to Embodiment 5 of the present invention will be described.

FIG. 17 is a block diagram showing an example of the configuration of the notification control device 26 according to the fifth embodiment.

As shown in FIG. 17, the notification control device 26 includes a vehicle control unit 27. Since other configurations and operations are the same as those in the first embodiment, detailed description thereof is omitted here.

The vehicle control unit 27 controls the driving operation of the host vehicle based on the secondary event estimation result by the secondary event estimation unit 4. Specifically, the vehicle control unit 27 controls each operation of the accelerator, the brake, and the handle of the host vehicle.

The function of the vehicle control unit 27 in the notification control device 26 is realized by a processing circuit. That is, the vehicle control unit 27 includes a processing circuit for controlling the driving of the host vehicle based on the secondary event estimation result by the secondary event estimation unit 4. The processing circuit is, for example, a processor 12 that executes a program stored in the memory 13 as shown in FIG.

The function of the vehicle control unit 27 in the notification control device 26 is realized by software, firmware, or a combination of software and firmware. Software or firmware is described as a program and stored in the memory 13. The processing circuit implements the function of the vehicle control unit 27 by reading and executing the program stored in the memory 13. That is, the vehicle control unit 27 stores a program in which the step of controlling the driving of the host vehicle is executed as a result based on the estimation result of the secondary event by the secondary event estimation unit 4. A memory 13 is provided. It can also be said that this program causes a computer to execute the procedure or method of the vehicle control unit 27.

<Operation>
Next, the operation of the notification control device 26 will be described.

FIG. 18 is a flowchart showing an example of the operation of the notification control device 26. Note that steps S61 to S65 in FIG. 18 correspond to steps S11 to S15 in FIG. Hereinafter, step S66 will be described. The periphery recognition device 10 is a camera and shoots ahead of the host vehicle.

In step S66, the vehicle control unit 27 controls the driving operation of the host vehicle based on the secondary event estimation result by the secondary event estimation unit 4. Specifically, the vehicle control unit 27 reduces the reaction force of the steering wheel in the direction of avoiding the secondary event, operates the steering wheel in the direction of avoiding the secondary event, applies the brake early, etc. Control the driving operation of the vehicle. For example, referring to FIG. 3, when the second event is “interrupt vehicle”, vehicle control unit 27 performs control to decelerate the host vehicle.

The vehicle control unit 27 may control the driving operation of the host vehicle based on the recognition result by the primary event recognition unit 3. For example, referring to FIG. 3, when the first event is “ball jumping out”, the vehicle control unit 27 may perform control so that the steering wheel operation avoids the ball.

From the above, according to the fifth embodiment, not only the secondary event is displayed, but also the driving operation of the host vehicle is controlled so as to avoid the secondary event. Therefore, the notification control device 26 can not only notify the second event but also support the driving operation of the host vehicle.

In addition, although FIG. 17 demonstrated the case where the notification control apparatus 6 shown in FIG. 2 was further provided with the vehicle control part 27, it is not restricted to this. For example, the vehicle control unit 27 may be provided in each of the notification control device 17 shown in FIG. 8, the notification control device 19 shown in FIG. 11, and the notification control device 22 shown in FIG.

In Embodiments 1 to 5, the case where the primary event recognition unit 3 recognizes the primary event has been described, but the present invention is not limited to this. For example, the first event recognized by another vehicle may be acquired from the other vehicle by inter-vehicle communication. Moreover, you may acquire the 1st event recognized by infrastructure facilities, such as a camera installed in the road, from the said infrastructure facility by road-to-vehicle communication. In this case, the host vehicle can estimate the second event using the first event regarding the place that cannot be captured by the periphery recognition device 10 provided in the host vehicle.

In Embodiments 1 to 5, the secondary event estimated by the secondary event estimation unit 4 may be transmitted to another vehicle.

In Embodiments 1 to 5, the case where the camera that is the periphery recognition device 10 captures the front of the host vehicle has been described, but the present invention is not limited to this. The camera that is the periphery recognition device 10 may capture all directions around the host vehicle or any direction.

The notification control device described above is not only a vehicle navigation device, that is, a car navigation device, but also a navigation device or navigation constructed as a system by appropriately combining a PND (Portable Navigation Device) and a server that can be mounted on a vehicle. The present invention can be applied to apparatuses other than the apparatus. In this case, each function or each component of the notification control device is distributed and arranged in each function that constructs the system.

Specifically, as an example, the function of the notification control device can be arranged in the server. For example, as shown in FIG. 19, the vehicle side includes a peripheral recognition device 10 and a HUD 11, and the server 28 includes a peripheral information acquisition unit 2, a primary event recognition unit 3, a secondary event estimation unit 4, and a notification control unit 5. By providing the own vehicle information acquisition unit 7, the map information acquisition unit 8, and the alarm image generation unit 9, a notification control system can be constructed. The same applies to the notification control device 17 shown in FIG. 8, the notification control device 19 shown in FIG. 11, the notification control device 22 shown in FIG. 14, and the notification control device 26 shown in FIG.

Further, the function of the notification control device can be arranged in the server and the mobile communication terminal 30. For example, as shown in FIG. 20, the vehicle is provided with the periphery recognition device 10 and the HUD 11, and the server 29 includes the periphery information acquisition unit 2, the primary event recognition unit 3, the secondary event estimation unit 4, and the own vehicle information acquisition. By providing the unit 7, the map information acquisition unit 8, and the alarm image generation unit 9, and the notification control unit 5 in the mobile communication terminal 30, a notification control system can be constructed. The same applies to the notification control device 17 shown in FIG. 8, the notification control device 19 shown in FIG. 11, the notification control device 22 shown in FIG. 14, and the notification control device 26 shown in FIG.

Even in the case of the above configuration, the same effect as the above embodiment can be obtained.

Further, software (notification control method) for executing the operation in the above embodiment may be incorporated in, for example, a server or a mobile communication terminal.

Specifically, as an example, the above-described notification control method acquires at least peripheral information indicating a situation around the host vehicle, and is a situation of an object or a person existing around the host vehicle from the acquired peripheral information. Recognizing a primary event and estimating a secondary event that can occur after the recognized primary event and has a causal relationship with the situation of the primary event according to a predetermined rule Then, control for notifying the estimated secondary event is performed.

From the above, the same effect as in the above embodiment can be obtained by operating the software that executes the operation in the above embodiment in a server or a portable communication terminal.

It should be noted that the present invention can be freely combined with each other within the scope of the invention, and each embodiment can be appropriately modified or omitted.

Although the present invention has been described in detail, the above description is illustrative in all aspects, and the present invention is not limited thereto. It is understood that countless variations that are not illustrated can be envisaged without departing from the scope of the present invention.

1 notification control device, 2 surrounding information acquisition unit, 3 primary event recognition unit, 4 secondary event estimation unit, 5 notification control unit, 6 notification control device, 7 own vehicle information acquisition unit, 8 map information acquisition unit, 9 alarm image generation unit, 10 peripheral recognition device, 11 HUD, 12 processor, 13 memory, 14, 15 primary event warning image, 16 secondary event warning image, 17 notification control device, 18 secondary event position estimation Unit, 19 notification control device, 20 recognition range restriction unit, 21 recognition range, 22 notification control device, 23 communication unit, 24 image recognition learning server, 25 secondary event estimation data server, 26 notification control device, 27

vehicle control unit

28, 29 servers, 30 mobile communication terminals.

Claims

A peripheral information acquisition unit that acquires peripheral information indicating at least the situation around the host vehicle;
A primary event recognition unit for recognizing a primary event that is a situation of an object or person existing around the host vehicle from the peripheral information acquired by the peripheral information acquisition unit;
A secondary event that is a situation that can occur after the primary event recognized by the primary event recognition unit and has a causal relationship with the situation of the primary event is predetermined. A secondary event estimator that estimates by rules;
A notification control unit that performs control to notify the second event estimated by the second event estimation unit;
A notification control device comprising:
An alarm image generation unit that generates a secondary event alarm image indicating the secondary event estimated by the secondary event estimation unit;
The notification control apparatus according to claim 1, wherein the notification control unit performs control to notify the second event alarm image generated by the alarm image generation unit.
A second event position estimating unit for estimating an occurrence position of the second event based on a position at which the first event recognizing unit has recognized the first event;
The notification control device according to claim 2, wherein the warning image generation unit generates the second event warning image corresponding to the generation position estimated by the second event position estimation unit.
The system further comprises a recognition range limiting unit that limits a recognition range by the first event recognition unit to a predetermined range including the occurrence position estimated by the second event position estimation unit. Item 6. The notification control device according to item 3.
The notification control device according to claim 1, wherein the parameter used when the first event recognition unit recognizes the first event can be updated.
The notification control apparatus according to claim 1, wherein the data used when the secondary event estimation unit estimates the secondary event can be updated.
The notification control device according to claim 1, further comprising a vehicle control unit that controls a driving operation of the host vehicle based on an estimation result of the secondary event by the secondary event estimation unit. .
Get at least peripheral information that shows the situation around your vehicle,
Recognizing a first event which is a situation of an object or person existing around the vehicle from the acquired surrounding information,
Estimating a secondary event that may occur after the recognized primary event and has a causal relationship with the situation of the primary event according to a predetermined rule;
A notification control method for performing control to notify the estimated second event.