JP7434882B2 - Dangerous driving determination device, dangerous driving determination method, and dangerous driving determination program - Google Patents

Description

The present invention relates to a dangerous driving determination device, a dangerous driving determining method, and a dangerous driving determining program that detect dangerous driving of nearby vehicles.

There is a known technology that calculates the following distance that the vehicle behind should maintain based on the type of license plate of the vehicle behind it and the vehicle type identified from the shape of the vehicle body, and projects an image showing the calculated distance onto the road (for example, , see Patent Document 1).

JP 2019-151169 Publication

In recent years, the frequent occurrence of dangerous driving, including so-called "distracted driving," has become a social problem. In this regard, it is difficult to accurately distinguish between dangerous driving and normal driving by simply calculating the inter-vehicle distance as in the technique described above. Therefore, there is a need to establish technology that can accurately detect dangerous driving.

The present invention has been made in view of these circumstances, and an object thereof is to provide a dangerous driving determination device that can accurately detect dangerous driving for each vehicle type.

In order to solve the above problems, a dangerous driving determination device according to an aspect of the present invention includes an image acquisition unit that acquires an image of the surroundings of a vehicle, and a vehicle that detects surrounding vehicles from the image acquired by the image acquisition unit. a detection unit, a vehicle type recognition unit that recognizes the vehicle type of a nearby vehicle detected by the vehicle detection unit, a size measurement unit that measures the size of the nearby vehicle detected by the vehicle detection unit on the image, and a size measurement unit that is determined in advance for each vehicle type. The present invention includes a reference storage section that stores a size reference for determining dangerous driving, and a danger detection section that detects dangerous driving of surrounding vehicles based on the size measured by the size measurement section and the reference.

Another aspect of the present invention is a dangerous driving determination method. This method involves the process of acquiring an image of the surroundings of a vehicle, the process of detecting surrounding vehicles from the image acquired during the image acquisition process, and the vehicle type of the surrounding vehicle detected in the process of detecting surrounding vehicles. a process of measuring the size of the surrounding vehicle detected in the image of the surrounding vehicle in the process of detecting the surrounding vehicle, and a process of reading out the size standard for determining dangerous driving determined in advance for each vehicle type. , and a step of detecting dangerous driving of surrounding vehicles based on the size measured in the step of measuring the size and a standard.

Note that arbitrary combinations of the above-mentioned constituent elements and mutual substitution of constituent elements and expressions of the present invention among methods, devices, systems, programs, etc. are also effective as aspects of the present invention.

According to the present invention, it is possible to provide a dangerous driving determination device that can accurately detect dangerous driving for each vehicle type.

FIG. 2 is a block diagram schematically showing the functional configuration of a drive recorder. It is a figure which typically illustrates the identification state of the surrounding vehicle in the image of a rear camera. FIG. 3 is a diagram schematically illustrating a recognition state of a dangerously driven vehicle in an image taken by a rear camera. It is a flow chart showing a process of detecting dangerous driving.

Embodiments of the present invention will be described below with reference to the drawings. The specific numerical values and the like shown in these embodiments are merely illustrative to facilitate understanding of the invention, and do not limit the invention unless otherwise specified. In this specification and the drawings, elements with substantially the same functions and configurations are given the same reference numerals to omit redundant explanation, and elements not directly related to the present invention are omitted from illustration. do. In the embodiment described below, a drive recorder and a navigation device are mainly exemplified as the dangerous driving determination device.

FIG. 1 is a block diagram schematically showing the functional configuration of the drive recorder 10. As shown in FIG. Each of the functional blocks shown in the figure can be realized in terms of hardware by elements and mechanical devices such as a computer's CPU and memory, and in terms of software by computer programs, etc.; It is depicted as a functional block to be realized. Therefore, those skilled in the art will understand that these functional blocks can be realized in various ways by combining hardware and software.

The drive recorder 10 includes an image acquisition section 12, an audio acquisition section 14, a vehicle information acquisition section 16, an event detection section 18, a recording control section 20, a temporary storage section 22, a communication section 24, an operation reception section 26, an image processing section 50, It includes a reference storage section 58, a danger detection section 59, a display control section 28, and an audio output section 29. The image acquisition unit 12 may be configured as a part of the vehicle information acquisition unit 16, or the image acquisition unit 12 may be included in the vehicle information acquisition unit 16. Drive recorder 10 is mounted on a vehicle. The communication unit 24 transmits and receives information to and from an external device via a wireless communication connection. The operation reception unit 26 receives operation input from the user. The display control unit 28 controls the display of the photographed image on the display device 49. The display device 49 is a monitor included in the navigation device 11, but as a modification, the drive recorder 10 may have the display device 49 built-in, or an infotainment system, electronic mirror, etc. (not shown) may be used. The information may be displayed in a superimposed manner or in an interrupt manner on a display device installed in the vehicle, such as a head-up display. The audio output unit 29 controls audio output to a speaker 47 that outputs audio such as recorded audio and warning sounds. The speaker 47 is a speaker included in the navigation device 11 or the vehicle, but as a modification, the drive recorder 10 may have the speaker 47 built-in. The connection between the drive recorder 10 and the navigation device 11 may be a wireless communication connection via the communication unit 24 or a wired communication connection such as UART (Universal Asynchronous Receiver/Transmitter) or USB (Universal Serial Bus). Good too.

The image acquisition unit 12 acquires video data of moving images captured by a front camera 40 and a rear camera 42 provided in the vehicle. The front camera 40 is configured to photograph the surroundings of the vehicle, particularly the front of the own vehicle. The rear camera 42 is configured to photograph the surroundings of the vehicle, particularly the rear of the own vehicle. The front camera 40 may be configured to photograph both the front of the own vehicle and the interior of the vehicle, or may be configured to photograph the entire front, side, and interior of the own vehicle, such as a single camera that can photograph the surrounding 360 degrees. It may be configured with a circumferential camera, or it may be configured with a plurality of cameras that take images of the front, rear, and sides. The front camera 40 and the rear camera 42 may be built into the drive recorder 10 or may be separate from the drive recorder 10. The rear camera 42 is a camera that also sends rear images to the navigation device 11, and is used as a camera for both the drive recorder 10 and the navigation device 11.

The image acquisition unit 12 may acquire images captured by the front camera 40 and the rear camera 42 as vehicle information regarding the running of the vehicle, and the image acquisition unit 12 may be considered as part of the vehicle information acquisition unit 16. The image acquisition unit 12 acquires an image from the front camera 40, and also acquires an image from the rear camera 42, regardless of whether vehicle information indicating that the gear is in reverse is acquired. . The display control unit 28 causes the display device 49 to display an image from the front camera 40 while vehicle information indicating that the gear is set to a gear other than reverse, such as drive or parking, is acquired. An image from the rear camera 42 is displayed on the display device 49 while vehicle information indicating that the gear is in reverse is acquired. However, as will be described later, if at least a nearby vehicle that may be driving dangerously is detected, the image from the rear camera 42 is displayed on the display device 49. As a modified example, the display device 49 displays not only the image from the front camera 40 but also the image from the rear camera 42, regardless of whether vehicle information indicating that the gear is in reverse is acquired. may be displayed all the time.

The audio acquisition unit 14 acquires audio data acquired by a microphone 44 provided in the vehicle. The microphone 44 is configured to collect sounds from inside and outside the vehicle. The microphone 44 may be built into the drive recorder 10, may be separate from the drive recorder 10, or may be integrated with the front camera 40 or the rear camera 42. Alternatively, it may be built into the navigation device 11. When built into the navigation device 11, the microphone 44 is used for voice recognition operations of the navigation device 11. The audio acquisition unit 14 may acquire audio using the microphone 44 as vehicle information regarding the running of the vehicle, or may be considered as part of the vehicle information acquisition unit 16.

The vehicle information acquisition unit 16 acquires vehicle information regarding the running of the vehicle from an on-vehicle device 46 provided in the vehicle. Specific examples of the in-vehicle device 46 include a vehicle speed sensor, a steering angle sensor, an accelerator operation amount sensor, a brake operation amount sensor, an acceleration sensor, a gyro sensor, a radar sensor, a lidar (Light Detection and Ranging), and a position information sensor (for example, Examples include, but are not limited to, GNSS (Global Navigation Satellite System) and occupant seating sensors. The vehicle information acquisition section 16 may acquire vehicle information via various control ECUs (Electronic Control Units), or may acquire vehicle information via the navigation device 11. Alternatively, the position information may be acquired from a position information sensor built into the navigation device 11. The vehicle information acquisition section 16 may acquire, via the communication section 24, information detected by sensors such as an acceleration sensor, a gyro sensor, and a position information sensor built into a terminal such as a user's mobile phone. The vehicle information acquisition unit 16 may acquire information regarding the running of the vehicle from a sensor included in the drive recorder 10. For example, the drive recorder 10 may include a built-in sensor such as an acceleration sensor or a position information sensor. Note that the front camera 40, the rear camera 42, and the microphone 44 may be regarded as one of the in-vehicle devices, and the vehicle information acquisition section 16 may be configured to include the image acquisition section 12 and the audio acquisition section 14.

The vehicle information acquisition unit 16 collects, for example, information related to the state of the vehicle, information related to operations on the vehicle, information related to the speed of the vehicle, information related to the position of the vehicle, information related to obstacles around the vehicle, and information related to the operating state of the driving support function of the vehicle. Information etc. are acquired from the in-vehicle device 46. The various information acquired by the vehicle information acquisition unit 16 is detection information from a sensor other than a camera, unlike detection of objects around the vehicle through image recognition by an image processing unit 50, which will be described later. The vehicle information acquisition unit 16 may acquire, for example, information indicating the open/closed states of doors and windows as information regarding the state of the vehicle. The vehicle information acquisition unit 16 may acquire, as information regarding operations on the vehicle, information indicating that an operation instructing opening/closing of a door or window has been performed, for example. The vehicle information acquisition unit 16 collects information regarding obstacles around the vehicle, such as whether there are other vehicles within a certain range around the vehicle, pedestrians, bicycles, fallen objects, or buildings in the lane in which the vehicle is traveling. Information indicating whether or not an obstacle such as an object exists may be acquired. The vehicle information acquisition unit 16 may acquire information indicating whether or not an occupant is present in the vehicle. The vehicle information acquisition unit 16 acquires information regarding on/off of an automatic driving function and a remote control function, and information regarding whether a specific driving support function is in operation, as the operating state of the driving support function of the vehicle. It's okay. Examples of driving support functions include, but are not limited to, adaptive cruise control (ACC) and lane keeping assist system (LKAS).

The event detection unit 18 detects various predetermined events (hereinafter referred to as “ Detects the occurrence of a trigger event (referred to as a trigger event).

A trigger event is a storage trigger event that is assumed to be a serious event such as a vehicle accident or trouble, and is a storage trigger event that, when it occurs or is detected, causes video data to be stored as overwrite-prohibited data. The event detection unit 18 detects, as a trigger event, the occurrence of an accident such as a vehicle collision, contact, rollover, or fall, or a trouble such as rough behavior or dangerous driving by a driver of a nearby vehicle. In addition, in this embodiment, the main feature is the method of detecting dangerous driving of surrounding vehicles, and when dangerous driving of surrounding vehicles is detected, it is assumed that a trigger event has also occurred, but the details will be described later. do. The event detection unit 18 detects the occurrence of an accident or trouble by image recognition or voice recognition based on a predetermined recognition pattern, for example from video data acquired by the front camera 40 and rear camera 42 and audio data acquired by the microphone 44. You may. The event detection unit 18 detects a trigger event by recognizing a sudden change in vehicle behavior due to a sudden start, sudden stop, sudden turn, etc., from information on the vehicle's running speed and acceleration, and operation information such as accelerator, brake, and steering. may also be detected. The event detection unit 18 uses information such as video data from the front camera 40 and rear camera 42 and the radar sensor of the vehicle to detect the approach of objects around the vehicle, deviation from the lane in which the vehicle is running, etc. based on a predetermined recognition pattern. The occurrence of a trigger event may be detected by recognition. The event detection unit 18 may also detect the occurrence of a trigger event when the operation reception unit 26 receives a press of a save start button provided on the drive recorder 10 or a voice instruction indicating the start of save.

The image processing section 50 includes an object detection section 51 , a vehicle type data storage section 54 , a vehicle type recognition section 55 , a size measurement section 56 , and an image composition section 57 . The object detection section 51 includes an object detection engine 52 and a vehicle detection section 53. The object detection engine 52 stores in advance image recognition patterns for detecting various objects from images, and detects objects appearing in images using a predetermined image recognition algorithm based on the image recognition patterns. The image recognition pattern may be defined by machine learning such as deep learning, and may be based on objects that can be photographed by a drive recorder camera, such as roads and surrounding areas such as four-wheeled vehicles, two-wheeled vehicles, pedestrians, and road installations. It may be an image recognition pattern specialized for objects that may exist. The object detection engine 52 further recognizes white lines and lanes on the road. The vehicle detection unit 53 detects a vehicle such as a four-wheeled vehicle or a two-wheeled vehicle as a surrounding vehicle from among the objects detected by the object detection engine 52 from the image taken by the rear camera 42 . The vehicle detection unit 53 may be integrally configured as a part of the object detection engine 52.

The vehicle type data storage unit 54 holds vehicle type data, which is data related to image characteristics such as the shape and color of each vehicle type, extracted as a result of machine learning such as deep learning from a large number of previously collected vehicle images. Car types may be divided into categories such as passenger cars, trucks, buses, and motorcycles, and within each category may be further divided into large cars, medium-sized cars, and small cars, or car manufacturer names and car names. , model, year, and other detailed vehicle types. The data on image characteristics for each vehicle type mainly includes information regarding the image characteristics when viewed from the front for each vehicle type, such as the shape, arrangement of parts, and color when viewed from the front. be. In addition, image feature data when viewed from the side and image feature data when viewed from the rear are similarly stored for each vehicle type. As a modified example, an image of the vehicle type itself may be stored as image feature data for each vehicle type, and the vehicle type may be recognized based on the degree of similarity to the image. In that case, a large number of images for each vehicle type may be stored.

The vehicle type recognition section 55 recognizes the vehicle type of the surrounding vehicle detected from the image by the vehicle detection section 53 based on the vehicle type data held in the vehicle type data storage section 54 . The surrounding vehicles detected from the image by the vehicle detection section 53 are highlighted on the image by the image composition section 57. For example, the image synthesis unit 57 superimposes a rectangular emphasis frame line on the image around the surrounding vehicle detected from the image. In addition, as a modification, the size of the license plate detected from the image may be used as one of the standards to recognize the vehicle type of the surrounding vehicle.

The size measurement unit 56 measures the size and position of the surrounding vehicle detected by the vehicle detection unit 53 on the image. The size of the surrounding vehicle on the image is the number of pixels indicating at least one of the length and width of the image of the vehicle detected on the image. For example, the vertical size and horizontal size of the emphasis frame synthesized around the image of the surrounding vehicle by the image synthesis unit 57 may be treated as the size on the image of the surrounding vehicle. The size measuring unit 56 measures both the vertical size and the horizontal size of surrounding vehicles. As a modified example, only one of the vertical size and the horizontal size may be measured, but in that case, the vertical size, which is less prone to errors, may be measured. This is because if the surrounding vehicle to be measured is hidden behind another vehicle, the horizontal size is more likely to be impossible to measure because the entire width cannot be seen than the vertical size. Alternatively, if the surrounding vehicle to be measured is viewed from an angle, the sides of the vehicle can be seen, making it difficult to determine the width. Further, as a modification, the area of the image of the vehicle detected on the image may be set as the size on the image. The size measurement unit 56 measures the size of the surrounding vehicle on the image for each frame image included in the video acquired by the image acquisition unit 12, thereby detecting a change in size as described later.

The size measurement unit 56 further detects the position of the surrounding vehicle detected by the vehicle detection unit 53 on the image. The position of the surrounding vehicle on the image is, for example, a relative position in the horizontal direction, and information indicating how much it is shifted to the left or right with respect to the virtual vertical center line that divides the image into left and right. It may be. Further, based on the position of the white line or lane on the road detected by the object detection engine 52, the degree of deviation of the target surrounding vehicle from the lane may be detected as information on the position of the vehicle on the image. Based on this position information, it is possible to determine whether a nearby vehicle at the rear is running directly behind the host vehicle.

The reference storage unit 58 stores a size threshold on an image for determining dangerous driving, which is predetermined for each vehicle type, as a risk determination standard. The reference storage unit 58 stores not only the size threshold on the image, but also the detected position of the vehicle on the image, the manner in which it changes, and the manner in which the size changes as danger determination standards. The size of the vehicle image varies depending on the characteristics such as distortion of the lenses mounted on the front camera 40 and the rear camera 42 and the size of the image sensor, and is therefore determined by measurement or calculation in advance for each lens and image sensor. A specific example of the risk determination criteria will be described later.

The danger detection section 59 detects dangerous driving of the surrounding vehicle based on the size of the surrounding vehicle on the image measured by the size measuring section 56 and the danger determination criteria held in the reference storage section 58. If we were to try to detect the approach of a vehicle based only on the size on the image without taking into account differences in vehicle types, the larger the vehicle, the larger the size on the image, and the closer the vehicle would be than the actual distance between vehicles. There is a risk of misjudgment. According to this embodiment, since the size on the image at the time of approach is determined in advance as a reference according to the actual size of the vehicle for each vehicle type, it is possible to accurately determine whether the vehicle is approaching a dangerous range. In addition, without estimating or measuring the distance between surrounding vehicles and your own vehicle, dangerous approaches of nearby vehicles can be efficiently detected just by checking whether the size on the image exceeds the standard for each vehicle type. can do. The danger detection unit 59 detects dangerous driving of nearby vehicles not only from the video acquired by the image acquisition unit 12 but also from the audio acquired by the audio acquisition unit 14. For example, when a horn sound is detected from surrounding sounds, and the horn sound continues for a predetermined period of time or more, or the number of detections continues for a predetermined number of times or more, dangerous driving of nearby vehicles is detected. When the danger detection section 59 detects dangerous driving, the event detection section 18 detects this as the occurrence of a trigger event.

The image synthesis unit 57 synthesizes an identification vehicle frame, which is an emphasis frame drawn with a broken line to surround the detected nearby vehicle and emphasize its presence, to the display video so as to surround the detected nearby vehicle. When dangerous driving is detected, the image synthesis unit 57 displays a caution vehicle frame, which is an emphasis frame drawn with a solid line, around the vehicle driving the dangerous vehicle. Note that the process of compositing the identification vehicle frame and the caution vehicle frame with the display video may be performed by a function equivalent to the image compositing unit 57 included in the navigation device 11, in which case the object image to be superimposed and Information on the position coordinates at which the image will be displayed may be transmitted to the navigation device 11.

The recording control unit 20 records video data that is constantly photographed by the front camera 40 or rear camera 42 and acquired by the image acquisition unit 12, audio data that is constantly collected by the microphone 44 and acquired by the audio acquisition unit 14, and vehicle information acquisition unit. 16 is temporarily stored in the temporary storage section 22. The recording control unit 20 records the video data and audio data stored in the temporary storage unit 22 onto the recording medium 48 using a ring buffer method.

The temporary storage unit 22 may be a buffer memory configured with a nonvolatile memory such as a flash memory or a solid state drive (SSD), or a volatile memory such as a DRAM. The recording medium 48 is, for example, a memory card such as an SD card (registered trademark), is used by being inserted into a slot provided in the drive recorder 10, and is configured to be removable from the drive recorder 10. The recording medium 48 may be composed of an auxiliary storage device such as a solid state drive or a hard disk drive, or an auxiliary storage device built into the navigation device 11 may be used as such an auxiliary storage device. Note that the audio data stored in the temporary storage unit 22 and the audio data recorded on the recording medium 48 are stored or recorded in the form of moving image data combined with video data. However, depending on the mode selected by the user, the moving image data may be stored or recorded as silent video data that does not include audio data. Therefore, although not specifically explained, the concept of "video data" described below may or may not include audio data.

The video data to be stored in the temporary storage unit 22 may be stored as moving image data in a streaming format, such as MPEG2-TS, separated by a predetermined time period. The video data recorded on the recording medium 48 may be in the same format as the video data stored in the temporary storage unit 22, or may be in a different format, for example, a format with a higher compression rate such as the MP4 format.

Since the recording control unit 20 records video data in the temporary storage unit 22 and the recording medium 48 using a ring buffer method, when the data recorded in the temporary storage unit 22 and the recording medium 48 reach their capacity, the oldest data is overwritten. be done. The recording control unit 20 temporarily records the video data in the temporary storage unit 22 and the recording medium 48 with an overwritable attribute.

When the occurrence of a trigger event is detected by the event detection unit 18, the recording control unit 20 controls the video data recorded on the recording medium 48 to prevent overwriting of video data for a period that includes at least the detection timing of the trigger event. Set the save format. This “occurrence of a trigger event” includes a case where dangerous driving is detected by the danger detection unit 59. The overwritable and non-overwritable attributes may be set, for example, by writing in a predetermined flag included in the video data, or by writing the overwritable or non-overwritable attribute in a management file such as an index that is independent of the video data. You may. When writing an attribute indicating whether or not overwriting is possible in the original video data itself, for example, information indicating the start position and end position of a storage target period in which overwriting is not possible may be written in the original video data. In this case, when you later overwrite the part other than the part to be saved, that is, the part that can be overwritten, with other data, the other part is cut out and left as a result from the start position to the end position of the part to be saved. Overwrite. Alternatively, a specification may be adopted in which overwritable and non-overwritable parts are distinguished by cutting out the part to be saved from the original video data and saving it separately in the form of an independent file. If the portion to be saved spans multiple pieces of video data, a file may be saved that is cut out from each piece of video data and concatenated. When saving separately in the form of an independent file, it may be possible to distinguish between overwritable and non-overwritable data by recording it in a specific area, for example a specific folder, that is different from the area in which overwritable data is recorded. . Data set with an attribute that cannot be overwritten is maintained in an overwrite-inhibited state until it is deleted or changed to an overwrite-enabled attribute by an explicit operation by the user. When the recording control unit 20 stores video data using the detection of dangerous driving as a trigger event, the recording control unit 20 stores the video data until the detection of dangerous driving by the danger detection unit 59 is completed, or until the surrounding vehicles surrounded by the caution vehicle frame are stored in the image acquisition unit 12. Save the video data until it is no longer detected from the video acquired with. By saving video data when dangerous driving of nearby vehicles is detected, it is possible to save video data from the previous stage even if a serious event such as a collision or accident has not occurred, increasing the reliability of storage. can be increased.

The display control unit 28 displays the identification vehicle frame and the caution vehicle frame synthesized by the image synthesis unit 57 on the screen of the navigation device 11, thereby informing the driver and passengers that there is a vehicle driving dangerously nearby. Visually notify. When it is determined that the following vehicle is driving dangerously, the display control unit 28 lights up or flashes a red LED lamp provided on the rear camera 42 to indicate that recording is in progress, thereby indicating that the vehicle following the vehicle is driving dangerously. The driver of the vehicle may be warned. When dangerous driving is detected by the danger detection section 59, the audio output section 29 audibly notifies the driver and fellow passengers that a vehicle driving dangerously is present in the surrounding area. Although visual notification and auditory notification are illustrated as examples of notification modes in this embodiment, in a modified example, tactile notification, for example, a notification mode in which vibration is applied to the steering wheel to alert the driver, can be realized. It's okay.

Note that, when dangerous driving is detected, the danger detection section 59 may store feedback information in the reference storage section 58 regarding the degree of danger for each vehicle attribute, for example, for each vehicle type or color. For example, the higher the risk attribute of a vehicle, the smaller the size threshold on the image so that it can be determined to be dangerous driving at an earlier timing, and conversely, the lower the risk attribute of a vehicle , the size threshold on the image may be increased to make it difficult to easily determine that dangerous driving is occurring. The danger detection unit 59 transmits feedback information regarding the degree of danger for each attribute of the vehicle to a predetermined server via the communication unit 24, and uses the feedback information as learning data for recognizing the tendency of the degree of danger for each attribute of the vehicle by deep learning. It may be accumulated. Such data can be used as statistical information on areas where dangerous driving is likely to occur, or as a basis for calculating car insurance premiums.

FIG. 2 schematically illustrates the identification state of surrounding vehicles in the image of the rear camera 42. In the video example shown in this figure, four surrounding vehicles, namely a first surrounding vehicle 60a, a second surrounding vehicle 60b, a third surrounding vehicle 60c, and a fourth surrounding vehicle 60d, are reflected in the rear image 66, which is the image of the rear camera 42. ing. The vehicle detection unit 53 detects surrounding vehicles 60 from the rear image 66, and assigns unique identification information to each vehicle to distinguish the vehicles. Although not shown, the vehicle detection unit 53 similarly detects surrounding vehicles 60 from the image of the front camera 40 and assigns unique identification information to each vehicle. However, if a nearby vehicle 60 detected from the rear image 66 passes the own vehicle and appears in the image of the front camera 40, the same identification information is given so that the same vehicle takes over. Similarly, when the own vehicle passes a nearby vehicle 60 detected from the image of the front camera 40 and appears in the rear image 66, the same identification information is assigned so that the same vehicle takes over.

The vehicle type recognition unit 55 recognizes the vehicle type of the surrounding vehicle 60 detected by the vehicle detection unit 53. An identification vehicle frame 64 is displayed around these surrounding vehicles 60 so as to surround each vehicle. The identification vehicle frame 64 is a rectangle drawn with broken lines. The size of the identification vehicle frame 64 is set so that the outline of the surrounding vehicle 60 is inscribed in the identification vehicle frame 64. When the size of the surrounding vehicle 60 on the image changes due to movement, the size of the identification vehicle frame 64 is also changed to follow the change. The length and width of the rectangle of the identification vehicle frame 64 may be measured by the size measurement unit 56 as the size of the surrounding vehicle 60 on the image. In a modified example, the identification vehicle frame 64 may be displayed with different line types, thicknesses, and colors to distinguish each vehicle type, or may be displayed with different line types, thicknesses, and colors so that the larger the size on the image, the more emphasized it is. It may be displayed in color. Alternatively, even if the surrounding vehicle 60 is detected and the vehicle type is identified, the specification may be such that the identification state such as the identification vehicle frame 64 is not displayed in the video.

FIG. 3 schematically illustrates the recognition state of a dangerous driving vehicle in the image of the rear camera 42. Among the detected surrounding vehicles 60, a vehicle in which dangerous driving has been detected is displayed so as to be surrounded by a caution vehicle frame 62 instead of an identification vehicle frame 64. In the example of FIG. 3, the first surrounding vehicle 60a approaches the host vehicle and the size of the vehicle on the rear image 66 exceeds a threshold value, so it is determined that dangerous driving is occurring. For example, the vertical size of the caution vehicle frame 62 is 1.2 times the vertical size of the identification vehicle frame 64 surrounding the first surrounding vehicle 60a in FIG. The vehicle has been determined to be a dangerously driven vehicle. The rectangular line of the caution vehicle frame 62 is a line type that is thicker and more conspicuous than the rectangular line of the identification vehicle frame 64, and is drawn as a solid line so that its presence is emphasized more than that of the identification vehicle frame 64. The size of the caution vehicle frame 62 is set so that the outline of the surrounding vehicle 60 is inscribed in the caution vehicle frame 62. When the surrounding vehicle 60 on the image moves and changes in size, the position and size of the caution vehicle frame 62 are also changed to follow the change. The line type, line thickness, and color of the caution vehicle frame 62 may be different depending on the level of danger. For example, the more dangerous a vehicle is, the more emphasized the color, such as red or yellow, is displayed. Based on the feedback information held by the reference storage unit 58, the identification vehicle frame 64 and the caution vehicle frame 62 may be color-coded depending on the level of risk for each attribute of the vehicle.

Most types of dangerous driving are commonly referred to as ``distracted driving.'' Although there are various forms of ``distracted driving,'' most of them have one thing in common: it begins with an act of threatening behavior by extremely closing the distance between vehicles from behind. Therefore, in this embodiment, in order to detect a dangerously driven vehicle, a nearby vehicle that causes the inter-vehicle distance to approach the vehicle excessively is identified from the rear image 66. At that point, it is once determined that the vehicle is driving dangerously, but it may also be incorrectly determined that a vehicle that is not driving dangerously but has only accidentally approached the vehicle is driving dangerously. Therefore, surrounding vehicles that have been determined to be driving dangerously are displayed surrounded by a caution vehicle frame 62, and by following subsequent changes in size and detection position on the image, each vehicle that has been determined to be driving in a dangerous manner is displayed. It is determined whether the risk determination criteria corresponding to the aspect are satisfied. Dangerous driving includes not only (1) excessively close following distance, but also (2) meandering driving, (3) swerving, (4) overtaking or cutting in by forcefully changing course, and (5) driving in front of the vehicle. This includes sudden deceleration and sudden stopping. Also includes (6) intimidation by honking, high beams, and passing. These risk criteria for determining dangerous driving include one based on the size of the vehicle on the image, one based on the position of the vehicle on the image, and the manner in which the vehicle size changes and the position of the vehicle. There are judgments based on The danger detection unit 59 determines dangerous driving of surrounding vehicles based on the danger determination criteria held in the reference storage unit 58. Below, each risk judgment standard and judgment method will be explained.

(1) Danger judgment criteria regarding excessively close following distance When the distance between the vehicle and the surrounding vehicle located behind becomes a predetermined dangerous distance (for example, 3 meters), the size of the vehicle on the image Threshold values are defined in advance as danger determination criteria for each vehicle type. The danger detection unit 59 detects that the size of the nearby vehicle detected by the vehicle detection unit 53 on the image is equal to or larger than the threshold of the size on the image defined as the danger determination criterion for the vehicle type, as shown in FIG. In some cases, the system determines that the vehicle has come within a dangerous range, that is, an excessively close range, from the own vehicle. The danger detection unit 59 may determine that driving is dangerous the moment the size of the vehicle on the image exceeds a predetermined threshold, or may determine that driving is dangerous when the size of the vehicle on the image exceeds the threshold for a predetermined period of time. It may be determined that the user is driving. Alternatively, the moment the size of the vehicle on the image exceeds a predetermined threshold, it is displayed surrounded by an identification vehicle frame 64, and when the size of the vehicle exceeds the threshold for a predetermined period of time, it is determined that the degree of danger has increased. The display color of the identification vehicle frame 64 may be changed to an emphasis color to increase the degree of emphasis.

As the risk determination criterion, not only the threshold value of the size of the vehicle on the image, but also the manner in which the size changes may be further defined. For example, the manner in which the size changes when surrounding vehicles alternately approach, leave, and approach is defined. In this case, the size on the image alternately becomes larger than the threshold value and becomes smaller than the threshold value. Further, for example, a change mode in which the size changes rapidly, such as when a nearby vehicle approaches rapidly, is defined as a danger determination criterion. The danger detection unit 59 determines that driving is dangerous when the changes in the size and position of the surrounding vehicle 60 in the rear image 66 match the danger determination criteria. Note that the danger detection unit 59 may exclude situations such as when the vehicle is stopped or in traffic jams, where the distance between vehicles tends to become narrow even if the driving is not dangerous. For example, based on the speed information acquired from the vehicle information acquisition unit 16 or the speed information of the own vehicle measured by image recognition from the image acquired by the image acquisition unit 12, the speed of the own vehicle is below a predetermined low speed. In some cases, it may not be determined that dangerous driving is occurring, and conditions related to speed information that exclude these situations may be set in the danger determination criteria.

(2) Danger Judgment Criteria for Meandering Driving For the surrounding vehicles 60 whose dangerous driving has been detected according to the criteria in (1), the detection position of the vehicle on the rear image 66 repeatedly moves alternately left and right with the center line as the border. These changes are determined as the criteria for determining the danger of meandering driving. As for the number of times that the left and right movements are repeated, for example, repeating the left and right movements two or more times may be defined as a danger determination criterion. Also, during meandering driving where the detected position of the vehicle on the image repeatedly moves left and right, it is not necessary to be in an excessively close state, and therefore, during that time, the size of the vehicle on the image does not exceed the criterion (1). It's okay. This makes it possible to continuously detect dangerous driving that threatens to occur by repeatedly meandering after getting too close to the rear of the own vehicle.

(3) Danger Judgment Criteria for Side Pulling As a risk judgment standard for detecting an act in which the surrounding vehicle 60, whose dangerous driving has been detected according to the criteria in (1), moves to the side of the own vehicle and runs parallel to the own vehicle, and then pulls to the side. , a threshold value of the size of the vehicle on the image when the surrounding vehicle 60 is in a dangerously close position to the own vehicle in the side image of the front camera 40 is defined for each vehicle type. The danger detection unit 59 determines that the state in which the size of the vehicle on the side image of the front camera 40 is equal to or larger than a predetermined threshold is dangerous driving. However, since the side image taken by the front camera 40 is the image through the side window, in a situation where the surrounding vehicle is too close to the side of the own vehicle, the entire surrounding vehicle may not be reflected through the window. Not exclusively. Therefore, as the size of the vehicle on the side image, a threshold value of the size of a part of the vehicle, such as a window, may be defined instead of the size of the entire vehicle. As a result, it is possible to continuously detect dangerous driving in which the driver approaches the rear of the host vehicle too much and then approaches the side of the host vehicle to threaten the driver.

(4) Danger Judgment Criteria for Overtaking or Interrupting Due to Forcible Course Change As a risk judgment criterion for overtaking due to forcible course change, detection on the rear image 66 of surrounding vehicles 60 whose dangerous driving was detected according to the criteria in (1) A change mode in which the position deviates from the center line and deviates from the lane at a predetermined speed or higher, and a change mode in which the detected position on the side image of the front camera 40 moves forward at a predetermined speed or higher are defined. In addition, as a risk judgment criterion regarding an interruption due to a forcible course change, regarding the surrounding vehicle 60 whose dangerous driving was detected according to the criterion (1), the detection position on the side image of the front camera 40 moves forward at a predetermined speed or more. and a change mode in which the detected position on the side image of the front camera 40 moves toward the front of the own vehicle at a predetermined speed or higher. This makes it possible to continuously detect dangerous driving in which the driver approaches the rear of the own vehicle excessively, then suddenly accelerates to overtake the own vehicle and forcibly cuts in front of the own vehicle.

(5) Danger Judgment Criteria for Sudden Deceleration or Sudden Stop that Blocks the Front Side When a nearby vehicle 60 whose dangerous driving has been detected according to the criteria in (1) overtakes the host vehicle and cuts in front of the own vehicle, then suddenly decelerates or suddenly stops that blocks the front area. As a risk judgment criterion for detecting the act of stopping, a threshold value of the size on the image taken by the front camera 40 in which the surrounding vehicle 60 approaches excessively in front is defined for each vehicle type. In addition, when the surrounding vehicle 60 suddenly decelerates or suddenly stops in front of the vehicle, the distance between the host vehicle and the own vehicle rapidly decreases, so a threshold value for the speed of change in the size of the surrounding vehicle 60 on the image of the front camera 40 is further defined. If the size increases at a rate of change greater than or equal to the threshold value, it may be determined that there has been a sudden deceleration or sudden stop. Alternatively, it may be determined that there has been a sudden deceleration or sudden stop by detecting the lighting of the brake lamps of the surrounding vehicles 60 on the image of the front camera 40 through image processing. This makes it possible to continuously detect dangerous driving in which the driver suddenly decelerates or suddenly stops in such a way as to overtake the vehicle and block the front of the vehicle after coming too close to the rear of the vehicle.

(6) Danger Judgment Criteria for Intimidating Acts Using High Beams, Passing, Horns, etc. For surrounding vehicles 60 whose dangerous driving has been detected according to the criteria in (1), the headlights may be switched to high beam lighting on the rear image 66 or instantaneously An image recognition pattern for detecting repeated high beam lighting (so-called passing) by image recognition is defined as a risk judgment standard for threatening behavior. In addition, a voice recognition pattern for detecting an act of excessively sounding a horn such as a horn sound from surrounding sounds is defined as a risk judgment standard regarding an intimidation act. Alternatively, an image recognition pattern can be used to detect a situation in which the driver or passenger of a nearby vehicle 60 is threatening by leaning out of the window or getting off the vehicle, or a situation in which the driver or passenger is loudly threatening. A speech recognition pattern for detection may further be defined.

In addition, the degree of danger is divided according to the length of time that has elapsed in which one or more of the danger judgment criteria of (1) to (6) are repeated, and the caution vehicle frame 62 is color-coded according to the degree of danger. It's okay.

FIG. 4 is a flowchart showing the process of detecting dangerous driving. When the image acquisition unit 12 acquires an image (S10) and the vehicle detection unit 53 detects the surrounding vehicle 60 from the rear image 66 (Y in S12), the vehicle type recognition unit 55 identifies the vehicle type of the detected surrounding vehicle 60. Recognize (S14). The danger detection unit 59 reads out the danger determination standard for each vehicle type (S16), and when the size of the surrounding vehicle 60 on the image is equal to or larger than the danger determination standard for each vehicle type (Y in S18), the danger detection unit 59 reads out the danger determination standard for each vehicle type (S16). However, it is determined that the vehicle is driving dangerously (S20), and the recording control unit 20 saves the video data, and the display control unit 28 notifies the driver of the dangerous driving vehicle (S22). If the vehicle detection unit 53 does not detect the surrounding vehicle 60 from the rear image 66 in S12, the process from S14 onward is skipped (N in S12). In S18, if the size of the surrounding vehicle 60 on the image is not larger than the danger determination standard for each vehicle type, the process from S20 onward is skipped (N in S18).

As described above in the embodiment, the size on the image at the time of approach is determined in advance as a reference according to the actual size of the vehicle for each vehicle type, so it is possible to accurately determine whether the vehicle is approaching a dangerous area. Can be done. In addition, without estimating or measuring the distance between surrounding vehicles and your own vehicle, dangerous approaches of nearby vehicles can be efficiently detected just by checking whether the size on the image exceeds the standard for each vehicle type. can do. Furthermore, by saving video data when dangerous driving of nearby vehicles is detected, it is possible to save video data from the previous stage even if a serious event such as a collision or accident has not occurred. Can increase certainty.

The present invention has been described above based on the embodiments. Those skilled in the art will understand that this embodiment is merely an example, and that various modifications can be made to the combinations of the constituent elements and processing processes, and that such modifications are also within the scope of the present invention. be.

10 drive recorder, 11 navigation device, 12 image acquisition unit, 20 recording control unit, 40 front camera, 42 rear camera, 53 vehicle detection unit, 54 vehicle type data storage unit, 55 vehicle type recognition unit, 56 size measurement unit, 58 reference storage Department, 59 Hazard Detection Department, 60 Surrounding Vehicles.

Claims (5)

an image acquisition unit that acquires an image of the surroundings of the vehicle;
a vehicle detection unit that detects surrounding vehicles from the image acquired by the image acquisition unit;
a vehicle type recognition unit that recognizes the vehicle type of a nearby vehicle detected by the vehicle detection unit;
a size measuring unit that measures the size of the surrounding vehicle detected by the vehicle detecting unit on the image;
a reference storage unit that stores a danger determination reference size that is a size on an image of a vehicle for determining dangerous driving that is predetermined for each vehicle type;
a danger detection unit that determines dangerous driving of surrounding vehicles based on a comparison between the size measured by the size measurement unit and the danger determination reference size ;
A dangerous driving determination device comprising: The risk determination standard size includes a vertical size on an image of a vehicle for determining dangerous driving that is predetermined for each vehicle type,
The danger detection unit determines dangerous driving of surrounding vehicles based on a comparison between only the vertical size among the sizes measured by the size measurement unit and the vertical size included in the danger determination reference size. The dangerous driving determination device according to claim 1. The image acquired by the image acquisition unit is a rear image taken of the rear of the vehicle,
The surrounding vehicle detected by the vehicle detection unit is a vehicle detected from the rear image,
2. The risk determination reference size stored in the reference storage unit is a size on an image of a vehicle detected from an image taken of the rear of the vehicle to determine that the vehicle is driving dangerously. 2. The dangerous driving determination device according to 2. a process of acquiring an image of the surroundings of the vehicle;
a step of detecting surrounding vehicles from an image acquired in the process of acquiring the image;
a step of recognizing the vehicle type of the surrounding vehicle detected in the step of detecting the surrounding vehicle;
a step of measuring the size of the detected surrounding vehicle on the image in the process of detecting the surrounding vehicle;
a process of reading out a danger determination standard size, which is a size on an image of a vehicle for determining dangerous driving, which is predetermined for each vehicle type;
a step of determining dangerous driving of surrounding vehicles based on a comparison between the size measured in the step of measuring the size and the dangerous determination reference size ;
A method for determining dangerous driving, comprising : a process realized by a computer . A function to acquire images of the surroundings of the vehicle,
a function of detecting surrounding vehicles from an image acquired by the image acquisition function;
a function of recognizing the model of a nearby vehicle detected by the function of detecting a nearby vehicle;
a function of measuring the size of the surrounding vehicle detected by the function of detecting the surrounding vehicle on the image;
A function that stores a dangerous judgment standard size, which is the size of the vehicle image on which dangerous driving is determined in advance for each vehicle type;
a function that determines dangerous driving of surrounding vehicles based on a comparison between the size measured by the size measuring function and the dangerous determination reference size ;
A dangerous driving determination program characterized by causing a computer to realize the following.

Images