JP2014048978A - Moving body warning device, and moving body warning method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To give a warning about the existence of a moving body even when a number of moving bodies exists without imposing a burden on a driver.SOLUTION: A moving body is detected from among forward images and a warning is given to a driver about the existence of the moving body. At this time, the moving body (already recognized moving body) continuously existing in the visual range of the driver more than a reference time is detected, and with respect to the already recognized moving body, the driver is warned based on a threshold value different from that of a moving body (unrecognized moving body) which is not the recognized body. By so doing, even when a plurality of moving bodies exists in the forward images, the already recognized body and the unrecognized body can be discriminated based on a mode of warning, and therefore, a burden of the driver can be reduced. With respect to the unrecognized moving body, since the warning is issued in a mode different from that of the already recognized moving body, the driver can also surely recognize the existence of the unrecognized moving body.

Description

  The present invention relates to a technique for detecting a moving body present in front of a vehicle and warning a driver.

  A technique has been proposed in which a moving body such as a pedestrian or a bicycle is detected from an image taken in front of a vehicle and a driver is warned that the moving body exists. This technology not only prevents the driver from overlooking pedestrians and the like, but also prevents the occurrence of accidents, and can reduce the burden on the driver. It has been. However, when there are many moving bodies, there is a possibility that the driver will be burdened if the warning of the existence of all these moving bodies is issued.

  Therefore, it has been attempted to warn according to the degree of danger of the moving body, instead of warning all the moving bodies in the same manner. As one of such attempts, a technique has been proposed in which a driver's line of sight is detected and a moving body located at a position deviated from the line of sight is emphasized to display a warning to the driver (Patent Document 1).

JP 2009-40107 A

  However, the proposed technique described above has a problem that it is still difficult to reduce the burden on the driver when there are many moving bodies. This is because the driver's line of sight is constantly moving, and the moving body that is highlighted and displayed in the proposed method changes one after another, which makes the driver feel annoying. Also, if a moving object is highlighted and displayed at a position outside the driver's line of sight, the driver's line of sight is naturally guided there, and attention to the part that the driver should really pay attention to is displayed. There is also a risk of scraping.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a technique capable of warning a driver without placing a burden even when a large number of moving objects exist.

  In order to solve the above-described problem, the moving body warning device or moving body warning method of the present invention detects a moving body from the front image and warns the driver of the presence of the moving body. At this time, a moving body (recognized moving body) that is continuously present in the driver's visual recognition range for a reference time or longer is detected, and the recognized moving body is not a recognized moving body (unrecognized moving body). The driver is warned at a threshold different from that of the body.

  In this way, even when there are a plurality of moving bodies in the front image, the recognized mobile body and the unrecognized mobile body can be distinguished from the warning mode, so the burden on the driver can be reduced. . Further, since the warning is given to the unrecognized moving body in a mode different from that of the already-recognized moving body, it is possible to make the driver surely recognize the presence of the unrecognized moving body.

  Moreover, in the moving body warning device of the present invention described above, an image of the driver taken from the front of the driver may be taken, and the viewing range may be detected based on the image.

  If a driver's image is taken from the front, the visual recognition range can be easily detected from the driver's face direction, pupil position, and the like.

  In the above-described moving body warning device of the present invention, the already recognized moving body may be warned as follows. First, the number of times that the moving body is determined to be a recognized moving body is counted for each moving body. Then, the driver may be warned of the presence of the recognized mobile object in a manner corresponding to the number of determinations.

  It is considered that the degree of recognition of the driver is different between a moving body with a large number of times determined as a recognized mobile body and a moving body with a small number of times. Therefore, for a recognized mobile object, if the driver is warned in a manner corresponding to the number of times of determination, it is possible to reliably warn of the existence of the mobile object while placing an excessive burden on the driver. Become.

  In addition, as described above, in the moving body warning device of the present invention that warns in a manner according to the number of determinations determined as a recognized mobile body, the operation of the recognized mobile body that has reached the predetermined number of times The warning to the person may be stopped.

  A moving body that has been determined many times as an already recognized moving body is considered to be well recognized by the driver. Therefore, if the warning to the driver is stopped for the already-recognized moving body that has reached the predetermined number of times, the burden on the driver is reduced without causing the driver to overlook the presence of the moving body. be able to.

  Moreover, in the mobile body warning device of the present invention that warns in a manner corresponding to the number of determinations determined as a recognized mobile body, the number of determinations may be reduced in the following cases. That is, when it is not determined again as a recognized mobile body after a predetermined threshold time elapses after it is determined as a recognized mobile body, the number of determinations for the mobile body may be reduced.

  For a moving body in which the state that is not recognized by the driver for a long time continues, it may be possible to review the warning mode even if it is an already recognized moving body. Therefore, in the moving body warning device of the present invention that warns in a mode according to the number of determinations, it is possible to warn the driver in a more appropriate mode by reducing the number of determinations as described above.

  Moreover, in the moving body warning device of the present invention described above, the following may be performed. First, a moving body in which the state of the unrecognized moving body continues for a predetermined oversight time is detected and determined as an overlooked moving body. And about an oversight moving body, it is good also as alerting a driver of existence of a moving body in a mode different from an unrecognized moving body which is not an oversight moving body, and an already recognized moving body.

  Since the state of the unrecognized moving body continues over a predetermined oversight time, it is desirable that the overlooking moving body warns the driver more clearly. Therefore, the warning can be clearly given to the driver if the warning is given in a different manner from the unrecognized moving object that is not the overlooking moving object and the recognized moving object.

It is explanatory drawing of the vehicle carrying the moving body warning apparatus of a present Example. It is a block diagram of the mobile body warning device of a present Example. It is a flowchart of the mobile body warning process for warning a mobile body. It is explanatory drawing which illustrated a mode that the moving body was detected from the front image. It is explanatory drawing which showed the method of determining whether a moving body exists in the visual recognition range. It is the list figure which showed the outline | summary of the attribute provided to a moving body. It is a flowchart of the attribute provision process which provides an attribute to a moving body. It is the list which illustrated various warning modes of a mobile. It is explanatory drawing which illustrated a mode that the warning aspect with respect to a several mobile body changes. It is a flowchart of the attribute provision process of a modification.

Hereinafter, examples will be described in order to clarify the contents of the present invention described above.
A. Device configuration :
FIG. 1 shows a general configuration of a vehicle 1 on which a moving body warning device 10 of the present embodiment is mounted. As shown in the figure, the moving body warning device 10 of the present embodiment is mounted on a front camera 12 that captures the situation in front of the vehicle 1 and a dashboard (or steering column) of the vehicle 1 so that the driver can view the vehicle from the front. A driver camera 14 that captures an image of the face, a display 16 that displays the situation in front of the vehicle on the screen and displays a warning to the driver of the presence of a moving object (pedestrian, bicycle, car, etc.); A control unit 18 for controlling the operation is provided. Note that "warning display" is a warning that is displayed by highlighting the marker image in some way, such as overlaying the marker image, displaying the contour with emphasis, or displaying the relevant part brighter than the surroundings. This is the display form.
Further, in this embodiment, an image of the front situation (front image) taken by the front camera 12 is displayed on the display 16 and a warning is displayed on the screen of the display 16 indicating the presence of the moving object. However, the presence of the moving body may be displayed as a warning by projecting a warning image on the windshield over the driver's field of view from the dashboard of the vehicle 1 or the like. Alternatively, an audible warning may be given by generating a warning sound or sound that informs the driver of the presence of the moving body.

  FIG. 2 shows an internal configuration of the control unit 18 of the mobile object warning device 10. As illustrated, the control unit 18 includes a moving body detection unit 18a, a determination unit 18b, a visual range detection unit 18c, and a warning unit 18d. The moving body detection unit 18a, the determination unit 18b, the visual recognition range detection unit 18c, and the warning unit 18d are conceptually classified from the inside of the control unit 18 focusing on functions, and each is physically separated. Need not exist. Also, these can be realized in hardware by an integrated circuit called a so-called LSI, or can be realized in software by a computer program. Of course, a part may be realized by hardware and the remaining part may be realized by software.

When the moving body detection unit 18a receives the front image captured by the front camera 12, the moving body detection unit 18a detects a moving body such as a pedestrian, a bicycle, or a car. The viewing range detection unit 18c detects a range (viewing range) that the driver is viewing from the driver's face image captured by the driver camera 14. The determination unit 18b determines whether or not the driver recognizes the moving object for each of the moving objects detected by the moving object detection unit 18a based on the visual field range detected by the visual range detection unit 18c. . Then, the warning unit 18d displays a warning on the moving body on the screen of the display 16 based on the determination result in the determination unit 18b. At this time, by displaying a warning by the method described below, even if a plurality of moving bodies are detected from the front image, it is possible to warn the moving bodies without burdening the driver. Become.
In the present embodiment, the front camera 12 that captures the front image corresponds to the “photographing means” in the present invention, and the mobile body detection unit 18a that detects the mobile body based on the front image is “moving” in the present invention. Corresponds to “body detection means”. Further, the viewing range detection unit 18c of the present embodiment corresponds to the “viewing range detection unit” in the present invention, and the determination unit 18b corresponds to the “determination unit” in the present invention. In this embodiment, the warning unit 18d and the display 16 correspond to “warning means” in the present invention.

B. Moving object warning processing:
FIG. 3 shows a flowchart of processing (moving body warning processing) in which the moving body warning device 10 according to the present embodiment detects a moving body existing in front of the vehicle 1 and warns the driver. This process is executed by the control unit 18 of the mobile object warning device 10.

  When the moving body warning process is started, first, a front image captured by the front camera 12 is acquired (S100). The front camera 12 has a so-called moving image shooting function, and takes a front image at a rate of 30 frames per second and outputs it to the control unit 18. In S100, image data for one frame output from the front camera 12 is acquired and stored in a memory (not shown) in the control unit 18.

Next, a moving body is detected from the front image (S102). The moving object can be detected by a method for obtaining a difference between images or a method for obtaining a correlation between images. That is, since the front camera 12 captures the front image at a rate of 30 frames per second, the image of a certain frame is almost the same as the image of the previous frame, but only the part where the moving object is shown Will be a different image. Therefore, if a difference is taken between two images, a moving object shown in the image can be detected. In addition, the moving body is shown in a certain size in the front image, and one moving body moves in substantially the same direction. Therefore, strong correlation will appear if image correlation is taken in the part where the moving object exists. By using this property, information such as the position, size, moving direction, moving speed, etc. of the moving object can be obtained. it can. In S102, the moving object existing in the front image is detected by using any one of the image difference and the correlation, or by combining these.
FIG. 4 illustrates a state in which a moving object is detected from the front image. In the illustrated example, three moving bodies A to C are detected.

Subsequently, the driver's visible range is detected (S104). Here, the “viewing range” refers to a range within the field of view of the driver of the vehicle 1 and in which the driver can correctly recognize the object. That is, a human has a field of view close to 180 degrees, but can only recognize such things as “something” or “something moved” at the edge of the field of view. An object can be correctly recognized only within a predetermined solid angle range with a so-called line of sight as a central axis. The viewing range refers to a range of a predetermined solid angle with this line of sight as the central axis.
In the present embodiment, it is assumed that the left and right angle (Yaw angle) of the driver's face is detected from the driver's image taken from the front using the driver camera 14, and the face direction is the direction of the line of sight. To detect the viewing range.

FIG. 5 conceptually shows how the driver's visible range is detected. In the illustrated example, the driver is directed rightward by a Yaw angle with respect to the front front indicated by a one-dot chain line in the drawing. Accordingly, assuming that the face direction is the direction of the line of sight, the range of the angle α on each side is set as the viewing range. Accordingly, in the illustrated example, only the moving body C among the moving bodies A to C exists within the viewing range.
In this embodiment, for the sake of simplicity, description will be made assuming that only the face direction in the left-right direction (Yaw angle direction) is detected, but the face direction in the up-down direction (Pitch angle direction) may also be detected. . In this case, assuming that the direction defined by the Yaw angle and the Pitch angle is the direction of the line of sight, within the range of the angle α vertically or horizontally, or within the range of the solid angle 2α with the direction of the line of sight as the central axis. It is good also as a visual recognition range.
In this embodiment, it is assumed that the face direction represents the direction of the line of sight. However, the direction of the line of sight may be detected more directly by detecting the position of the driver's pupil.

  As described above, when the moving object in the front image and the driver's visible range are detected, an attribute assigning process for assigning an attribute to the detected moving object is started (S200). Although details of the attribute assignment process will be described later, in this process, various attributes are assigned to each detected moving object. For example, in the example shown in FIG. 4, three moving bodies A to C are detected, but various attributes are assigned to each moving body.

  Thereafter, a warning display mode is determined for each moving body in accordance with the attached attribute, and the result is output to the display 16 to display the warning on the moving body for the driver (S106).

  Then, it is determined whether or not the driving of the vehicle 1 is finished (S108). If the driving is not finished (S108: no), the process returns to S100 to acquire a new front image, and then the series of processes described above are performed. repeat. As described above, since the front camera 12 acquires a front image at a rate of 30 frames per second, the moving object warning process of FIG. 3 also executes the above-described series of processes at a rate of 30 times per second. As a result, the attribute of the moving object in the front image is updated every frame, and the manner in which the moving object is displayed as a warning on the display 16 also changes every moment.

C. Attribute assignment processing:
In the following, the details of the attribute assigning process (S200) for assigning attributes to a mobile object will be described. As preparation, an outline of attributes provided to the mobile object by the control unit 18 of this embodiment will be described in advance.

  FIG. 6 shows a list of attributes assigned to the moving object. Of these seven attributes, attribute 1 and attribute 2 are attributes that are given when a moving object is detected from the front image, and the remaining five attributes (attributes 3 to 7) are attribute attributes. This attribute is given by the grant process.

  Attribute 1 is an attribute related to the position and size of the moving object on the front image. In the present embodiment, the detected moving body is surrounded by a rectangular frame, and the coordinate values of two points existing on the diagonal of the rectangular frame are used as the attribute 1. For example, for the moving object A shown in FIG. 4, the coordinate values on the images of the points a1 and a2 are the attribute 1.

The attribute 2 is an attribute that represents the moving direction of the moving body. In the present embodiment, the attribute 2 can take a value of 0 to 7, and “0” of the attribute 2 indicates that it moves rightward on the image. Further, “1” indicates a movement in the lower right direction, and “2” indicates a movement in the lower direction. Further, “3” indicates that the movement is in the lower left direction, “4” indicates that the movement is in the left direction, “5” indicates that the movement is in the upper left direction, “6” indicates that the movement is in the upward direction, and “7” indicates that the movement is in the upper right direction.
In the present embodiment, the result of roughly classifying the moving direction of the moving body into eight as described above is used as attribute 2. However, the present invention is not limited to this, and a vector representing the moving direction and moving speed of the moving body is used. It can also be used as attribute 2.

The attribute 3 is a flag (viewing range flag) indicating whether or not the moving body is within the driver's viewing range (see FIG. 5). The viewing range flag ON indicates that the moving body is within the viewing range.
The attribute 4 is an attribute that indicates how many frames the moving body within the viewing range is continuously within the viewing range. If a certain moving body is continuously within the viewing range over a certain number of frames, it can be considered that the driver has recognized.
Attribute 5 is an attribute indicating the number of times recognized by the driver. This attribute 5 counts the number of times of recognition in response to the fact that the driver may recognize the moving body once recognized many times.
The attribute 6 is an attribute indicating the number of frames that the moving body in the front image has passed without being recognized even once by the driver. If the predetermined number of frames has passed without being recognized by the driver even once, the moving body is considered to be overlooked by the driver.
The attribute 7 is a flag (missing flag) indicating that the moving body is overlooked by the driver. The oversight flag ON indicates that the moving body is overlooked.

  At the stage when the moving object is detected for the first time, the attribute 3 (viewing range flag) and the attribute 7 (missing flag) are set to OFF, and the attributes 4 to 6 are set to “0”. However, thereafter, the image of the frame is supplied from the front camera 12 and is updated each time the attribute assigning process described below is executed. Note that attribute 1 and attribute 2 are updated not by the attribute assignment process but by the process of detecting a moving object in the moving object warning process (S102 in FIG. 3).

FIG. 7 shows a flowchart of attribute assignment processing. Prior to this process, in the moving body warning process of FIG. 3, the moving body of the front image and the driver's visual recognition range have already been detected.
In the attribute assignment process (S200), first, one arbitrary moving body is selected from the detected moving bodies (S202). Then, it is determined whether or not the selected moving object is within the driver's visual recognition range (S204).
Whether or not the moving body is within the visible range can be determined based on the attribute 1 of the moving body and the driver's previously detected visual range. That is, from the attribute 1 of the moving body, it is possible to determine how much the moving body is located at the Yaw angle from the front in front of the driver. Then, if the calculated Yaw angle of the moving body is within the range of α on both sides of the Yaw angle of the driver's line of sight, it is determined that the moving body is within the driver's viewing range. (See FIG. 5).

As a result, when it is determined that the moving body is within the viewing range (S204: yes), the viewing range flag of attribute 3 is turned ON (S206), and "1" is added to the value of attribute 4 (S208). . As described above with reference to FIG. 6, the attribute 4 indicates the number of frames in which the moving object is continuously present in the viewing range. Here, in the currently processed frame, in response to the determination that the moving body is within the viewing range (S204: yes), the number of frames that are continuously within the viewing range (that is, attribute 4) is set. Add “1”.
On the other hand, when it is determined that the moving body does not exist within the viewing range (S204: no), the viewing range flag of attribute 3 is turned off (S210), and the value of attribute 4 is initially set to “0”. (S212). As a result, the attribute 4 is not the cumulative value of the number of frames in which the moving object is present in the viewing range, but the number of frames that are continuously present in the viewing range.

When “1” is added to attribute 4 (S208), it is determined whether or not the value of attribute 4 has reached a predetermined first threshold value TH1 (S214). Here, the first threshold value TH1 is a threshold value used for determining whether or not the driver has recognized the moving body. In other words, the driver cannot recognize that the moving body is present in the viewable range, but the moving body continues to stay in the viewable range for a certain period of time. If so, it may be considered that the driver has recognized the moving body. This is because, if the moving object exists only within the viewing range, it may be possible that the line of sight happens to pass over the moving object, whereas the moving object remains within the viewing range for a certain period of time. The fact that the vehicle is staying is because it is considered that the driver intentionally directs his / her line of sight toward the moving body.
Therefore, when the value of attribute 4 (the number of frames in which the moving body is continuously present within the driver's visual recognition range) reaches a certain value (first threshold value TH1), the driver intends the moving body. Can be determined (that is, recognized by the driver).
As described above, in the present embodiment, 30 frames can be converted to time at a rate of 1 second, so the number of frames continuously present in the viewing range is continuously present in the viewing range. Synonymous with time. Therefore, in the present embodiment, the first threshold value TH1 corresponds to the “reference time” in the present invention.

As a result, if the value of attribute 4 has reached the first threshold value TH1 (S214: yes), “1” is added to the value of attribute 5 (S216). As described above with reference to FIG. 6, the attribute 5 represents the number of times that the moving body is recognized by the driver. In addition, since the value of attribute 5 of the present embodiment represents the number of times recognized by the driver, it corresponds to the “number of determinations” in the present invention.
After the value of attribute 4 is initialized to “0” (S218), the oversight flag of attribute 7 is set to OFF (S220). As described above with reference to FIG. 6, the oversight flag is a flag indicating that the moving body is overlooked by the driver. Therefore, if “1” is added to the value of attribute 5 (the number of times the moving body has been recognized by the driver) in S216, it will be unreasonable if the oversight flag is set to ON. Therefore, the oversight flag of attribute 7 is set to OFF (S220).

On the other hand, when the value of attribute 4 (the number of frames in which the moving body is continuously present in the viewing range) does not reach the first threshold value TH1 (S214: no), or the moving body is within the viewing range in the first place. If the attribute 3 viewing range flag and the attribute 4 value are initialized (S210, S212), the attribute 5 value (number of times recognized by the driver) is “0”. Is determined (S222).
As a result, if the value of attribute 5 is “0” (S222: yes), the moving body has not been recognized by the driver yet, so the value of attribute 6 is “1”. Are added (S224). As described above with reference to FIG. 6, the value of the attribute 6 indicates the number of frames that have passed without being recognized by the driver even once after being detected as a moving body.

Then, it is determined whether or not the value of attribute 6 has reached a predetermined second threshold value TH2 (S226). The second threshold TH2 is set to a value larger than the first threshold TH1 described above. As a result, if the attribute 6 has reached the second threshold value TH2 (S226: yes), it is considered that the mobile body has passed a long time without being recognized even once by the driver. Is set to ON (S228). On the other hand, when the attribute 6 has not reached the second threshold value TH2 (S226: no), the oversight flag of the attribute 7 is kept OFF. In addition, since the attribute 6 and the attribute 7 are attributes related to a state in which the moving body has not been recognized by the driver even once, when the value of the attribute 5 is not “0” (S222: no), The processing relating to attribute 6 and attribute 7 (S224 to S228) is not necessary.
Note that the number of frames that have passed without the mobile body being recognized even once by the driver is synonymous with the time that has passed without the driver being recognized even once. Therefore, in this embodiment, the second threshold value TH2 corresponds to the “missing time” in the present invention. In addition, the moving body in which the oversight flag is set to ON corresponds to the “overlooked moving body” in the present invention.

  When the attributes 3 to 7 of the selected moving object are set as described above, it is determined whether or not all the moving objects detected from the front image have been selected (S230). As a result, when there is still a moving body that has not been selected (S230: no), the process returns to S202 to select one new moving body and then repeat the series of processes described above. On the other hand, if it is determined that all the moving objects have been selected (S230: yes), the attribute assignment process shown in FIG. 7 is terminated and the process returns to the moving object warning process of FIG.

D. Moving object warning display:
As shown in FIG. 3, after returning from the attribute assignment process (S200) in the moving object warning process, the display mode is determined after the warning display mode is determined for each moving object according to the attribute given to the moving object. A warning is displayed on the screen (S106 in FIG. 3).

  FIG. 8 shows a list of warning display modes set according to the attributes of the moving object. As shown in FIG. 8A, in this embodiment, the warning display mode is set based on the value of attribute 5 (number of times recognized by the driver) and the setting of the oversight flag of attribute 7. Yes. FIG. 8B conceptually shows how the warning display changes according to the attribute setting.

  In all the moving objects, the value of attribute 5 immediately after detection is set to “0”, and the oversight flag of attribute 7 is set to OFF. In this state (attribute 5 = 0), since it is considered that the driver has not yet recognized, the mobile object is surrounded by a red rectangular frame (red frame) and a warning is displayed. In FIG. 8B, the red frame is displayed as a thick solid line frame.

  After that, when the number of frames (first threshold TH1) or more continuously remains within the driver's visual recognition range, “1” is added to the value of attribute 5 (see S216 in FIG. 7). Accordingly, since it can be considered that the mobile object having the attribute 5 value of “1” is recognized once by the driver, it is surrounded by a thin blue line rectangular frame (blue frame) and displayed as a warning. Compared with the red frame in the case of the attribute 5 value “0”, the color of the frame line has been changed from red to blue, and the thickness of the frame line has become thinner. The blue frame is a warning display that is less noticeable than the red frame.

  After that, “1” is added to the value of attribute 5 again, and the mobile object having the value of attribute 5 of “2” is recognized twice by the driver. Therefore, such a moving body is displayed with a warning by being surrounded by a thin dashed blue frame (dashed blue frame). Compared with the blue frame, the broken blue frame is a less noticeable warning as the frame line is changed to a broken line.

Furthermore, the moving object in which “1” is added to the value of attribute 5 and the value of attribute 5 is “3” is recognized three times by the driver. Since such a moving body is considered to be sufficiently recognized by the driver, there is no need to display a warning. Therefore, no frame display (warning display) is performed for a moving object having an attribute 5 value of “3” or more.
As described above, in this embodiment, since the moving object having the attribute 5 value of “1” or more is considered to be a moving object already recognized by the driver (recognized moving object), the blue frame (dashed blue frame) The warning display mode is not so conspicuous as well as the warning display is not performed.

  On the other hand, when the number of frames (second threshold value TH2) or more passes without being recognized by the driver even once (that is, the value of attribute 5 remains “0”), the oversight flag of attribute 7 is turned on. (See S228 of FIG. 7). For such a moving body, it is considered that the warning display of the extent surrounded by the red frame is insufficient, so the red frame is blinked to make the warning display more conspicuous.

As described above, in this embodiment, the moving object having the attribute 5 value of “0”, including the moving object with the oversight flag ON, is not yet recognized by the driver (unrecognized). Therefore, the warning is displayed in a more conspicuous manner than the recognized mobile object.
And since the mode of warning display is different between the recognized mobile body and the unrecognized mobile body, even if there are a large number of mobile bodies, there is no burden on the driver, but the existence of the mobile body Can be reliably warned. This point will be described below.

  FIG. 9 shows how the warning display mode changes for a plurality of moving bodies detected from the front image. FIG. 9A illustrates an image displayed on the screen of the display 16 when a moving object is detected for the first time from the front image. In the illustrated example, ten moving bodies from the moving bodies a to j are detected. In addition, since these mobile objects are detected for the first time (attribute 5 = 0), all of them are surrounded by a red frame and displayed as a warning.

  FIG. 9B shows an image after a predetermined time (for example, 1 to 2 seconds) has elapsed from the image of FIG. 9A. During this predetermined time, several moving bodies (here, moving bodies b, c, d, g, h) are recognized by the driver, and the value of attribute 5 is set to “1”. The display mode is changed from a red frame to a blue frame.

  FIG. 9C shows an image after a predetermined time has elapsed from the image of FIG. 9B. In the image of FIG. 9 (c), several more moving bodies (here, moving bodies e and i) are recognized by the driver and the warning display mode is changed from the red frame to the blue frame. Among the mobile bodies that have already been recognized, there are mobile bodies (here, mobile bodies b, c, g, and h) that have been recognized again and whose warning display mode has been changed from a blue frame to a dashed blue frame. ing. On the other hand, there are also moving bodies (here, moving bodies a and f) that are not yet recognized by the driver at this stage.

  FIG. 9D shows an image after a predetermined time has elapsed from the image of FIG. 9C. In the image of FIG. 9D, the moving body that has not yet been recognized (here, the moving body f) is recognized by the driver, and the warning display mode is changed from the red frame to the blue frame. In addition, a mobile body (here, mobile bodies d, e, i, j) in which a warning display mode is changed from a blue frame to a broken blue frame is recognized again among mobile bodies that have already been recognized. There are also mobile objects (in this case, mobile objects b, c, g, and h) that are no longer displaying warnings. However, there is still a moving body (moving body a in this case) that is not yet recognized by the driver at this stage, and the warning flag of attribute 7 is set to ON for such a moving body. The display mode is changed from a red frame to a flashing red frame.

Thus, in the moving body warning device 10 of the present embodiment, even when there are a plurality of moving bodies in the front image (see FIG. 9A), the moving body recognized by the driver (recognized moving body). With respect to, the burden on the driver can be reduced by changing the manner of warning display. On the other hand, since the moving body that is not recognized by the driver (unrecognized moving body) is more noticeable than the already recognized moving body, the driver can be surely warned of the presence of the moving body. It becomes possible.
Further, as illustrated in FIGS. 8 and 9, the burden on the driver can be further reduced by changing the warning display mode for the already-recognized moving body according to the value of the attribute 5. Similarly, for an unrecognized moving body, if the mode of warning display is changed according to the setting of the oversight flag of attribute 7, it is possible to more reliably warn the driver of the presence of the moving body.

E. Modified example:
In the above-described embodiment, the number of times the moving body has been recognized by the driver (value of attribute 5) has been described as increasing but not decreasing. However, the value of attribute 5 may be subtracted if the state that is not recognized by the driver continues for a long time after being recognized by the driver.

FIG. 10 shows a flowchart of such a modification attribute assignment process. The attribute assignment process (S300) of the modification is a process executed instead of the attribute assignment process (S200) of the present embodiment described above with reference to FIG.
The processing content of the attribute assignment process (S300) of the modification is substantially the same as the attribute assignment process (S200) of the present embodiment, but the following points are greatly different. That is, in the attribute assignment process (S200) of the present embodiment, the value of attribute 6 is counted and the oversight flag of attribute 7 is set only while the moving object is not recognized once (during attribute 5 = 0). Had gone. On the other hand, in the attribute assignment process (S300) of the modified example, the value of attribute 6 is counted even when the value of attribute 5 is “1” or more, and the value of attribute 6 reaches the second threshold value TH2. Subtracts the value of attribute 5. Hereinafter, the attribute assignment process (S300) of the modification will be briefly described with the difference as a center.

Also in the attribute assignment process (S300) of the modified example, first, one arbitrary moving body is selected from the detected moving bodies (S302), and the selected moving body is within the driver's viewing range. It is determined whether or not there is (S304).
As a result, when it is determined that the moving body is within the viewing range (S304: yes), the viewing range flag of attribute 3 is turned ON (S306), and the value of attribute 4 (continuously exists within the viewing range). “1” is added to the number of frames) (S308). On the other hand, when it is determined that the moving object does not exist within the viewing range (S304: no), the viewing range flag of attribute 3 is turned off (S310), and the value of attribute 4 is initialized to “0” (S312). ).

  When “1” is added to attribute 4 (S308), it is determined whether or not the value of attribute 4 has reached a predetermined first threshold value TH1 (S314), and if it has reached the first threshold value TH1. (S314: yes) Since it can be determined that the vehicle has been recognized by the driver, "1" is added to the value of attribute 5 (the number of times the driver has recognized) (S316). Thereafter, after the value of attribute 4 is initialized to “0” (S318), the oversight flag of attribute 7 is set to OFF (S320).

  On the other hand, when the value of attribute 4 (the number of frames continuously present in the viewing range) does not reach the first threshold value TH1 (S314: no), or the moving object is not in the viewing range. When the viewing range flag of attribute 3 and the value of attribute 4 are initialized (S310, S312), the following process is executed in the attribute assignment process of the modified example.

  First, “1” is added to the value of attribute 6 (S322). The attribute 6 in the modified example represents the number of frames that have passed without the moving body being recognized by the driver. Therefore, in the modification, even after the moving body is recognized by the driver, the number of frames is counted by the attribute 6 if the state in which the moving body is not recognized thereafter continues.

  Then, it is determined whether or not the value of attribute 6 has reached the second threshold value TH2 (S324). As a result, if the value of attribute 6 has reached the second threshold value TH2 (S324: yes), the value of attribute 6 is initialized to “0” (S326), and then the value of attribute 5 (recognized by the driver). It is determined whether or not the number of times is “0” (S328). As a result, if the value of attribute 5 is “0” (S328: yes), the oversight flag of attribute 7 is set to ON as in the attribute assignment process of the above-described embodiment (S330).

On the other hand, when the value of attribute 5 is not “0” (S328: no), “1” is subtracted from the value of attribute 5 (S332). Therefore, in the attribute assignment process of the modified example, even if the moving object having the attribute 5 value of “1” is continued to be recognized by the driver for the number of frames corresponding to the second threshold value TH2, the value of the attribute 5 Returns to “0”. Similarly, if the value of attribute 5 is “2”, it returns to “1”, and if the value of attribute 5 is “3”, it returns to “2”.
If the value of attribute 6 has not reached the second threshold value TH2 (S324: no), the above-described processing for subtracting the value of attribute 5 or setting the oversight flag of attribute 7 (S326) ~ S330) are not required.

In the modification described above, the second threshold value for determining whether to subtract the value of attribute 5 is the same as the threshold value for determining whether to set the oversight flag for attribute 7 to ON. It has been described that the threshold value TH2 is used. Of course, a threshold different from the second threshold TH2 may be used as a threshold for determining whether to subtract the value of attribute 5.
The second threshold value TH2 used to determine whether or not to subtract the value of attribute 5 corresponds to the “threshold time” in the present invention.

  When the attributes 3 to 7 are set as described above, it is determined whether or not all the moving objects detected from the front image have been selected (S334). As a result, if there is still a moving body that has not been selected (S334: no), the process returns to S302 to select one new moving body and then repeat the series of processes described above. On the other hand, if it is determined that all the moving objects have been selected (S334: yes), the attribute assignment process of the modification shown in FIG. 10 is terminated and the process returns to the moving object warning process of FIG.

  In the attribute assignment process of the modified example described above, even if the moving body is recognized by the driver and the mode of the warning display is changed to a mode that is not conspicuous, if the state that is not recognized thereafter continues, the warning display of the mode that is easy to stand out Returned. For this reason, when there is a mobile body that has not been recognized for a long time among the already recognized mobile bodies, it is possible to warn the presence of such a mobile body and call the driver's attention.

  As mentioned above, although the present Example and the modification were demonstrated, this invention is not restricted to said Example and a modification, It can implement in a various aspect in the range which does not deviate from the summary.

DESCRIPTION OF SYMBOLS 1 ... Vehicle, 10 ... Moving body warning device, 12 ... Front camera,
14 ... Driver camera, 16 ... Display, 18 ... Control part,
18a ... moving body detection unit, 18b ... determination unit, 18c ... viewing range detection unit,
18d ... Warning part

Claims (7)

A moving body warning device that warns a driver of a vehicle that a moving body exists in front of the vehicle,
Photographing means for photographing a front image which is an image obtained by photographing a situation in front of the vehicle;
Moving body detecting means for detecting the moving body from the front image;
Visual range detection means for detecting a visual range that is visible to the driver in a range corresponding to the front image in the front situation;
A determination unit that detects the moving body that is continuously present in the visual recognition range for a predetermined reference time and determines the moving body as an already recognized moving body;
A moving body warning device comprising: a warning unit that warns the driver of the presence of the moving body in a different manner between the unrecognized moving body that is the moving body that is not the recognized moving body and the recognized moving body. The moving body warning device according to claim 1,
The said visual range detection means is a moving body warning device which is a means to detect the said visual range based on the driver | operator's image image | photographed from the front of the said driver | operator. The moving body warning device according to claim 1 or 2,
A counting means for counting the number of times the moving body is determined to be the recognized moving body for each moving body;
The warning means is a moving body warning device which is a means for warning the driver of the presence of the recognized moving body in a manner corresponding to the number of times of determination. The moving body warning device according to claim 3,
The warning means is a moving body warning device that is a means for stopping a warning to the driver for the already-recognized moving body that has reached the predetermined number of times of determination. The moving body warning device according to claim 3 or 4,
The counting means determines the number of times of determination for the moving object for the moving object that is not determined to be the recognized recognized moving object again after a predetermined threshold time elapses after it is determined to be the already recognized moving object. A moving body warning device which is a means for subtracting the number. The moving body warning device according to any one of claims 1 to 5,
The determination means is means for detecting the moving body in which the state of the unrecognized moving body continues for a predetermined oversight time and determining the moving body as an overlooking moving body,
The warning means is a moving body warning device that is a means for warning the driver of the presence of the moving body in a mode different from the unrecognized moving body and the recognized moving body with respect to the overlooked moving body. A moving object warning method for warning a driver of a vehicle that a moving object exists in front of the vehicle,
Photographing a front image that is an image obtained by photographing a situation in front of the vehicle;
Detecting the moving body from the front image;
Detecting a viewing range that is a range that the driver is viewing in a range corresponding to the front image in the front situation; and
Detecting the moving body that is continuously present in the visual recognition range for a predetermined reference time, and determining the moving body as an already recognized moving body;
A moving body warning method comprising: warning the driver of the presence of the moving body in a different manner between the unrecognized moving body that is the moving body that is not the recognized mobile body and the recognized mobile body.

Images