JP7492453B2 - Vehicle recognition system and method - Google Patents

Description

The present invention relates to a vehicle recognition system and a recognition method.

In recent years, research has been conducted into automatically controlling vehicles. In automatic vehicle control, a LIDAR (Light Detection and Ranging) attached to the vehicle is used to detect the distance from the vehicle to objects around the vehicle. LIDAR irradiates light (or electromagnetic waves with a wavelength close to that of light), measures the scattered light, and measures the distance to the target based on the time from light emission to light reception. Therefore, if dirt adheres to the light irradiating part or the scattered light receiving part, the measurement performance may be reduced. Patent Document 1 describes a technology that estimates the adhesion of dirt to the measurement part based on the distance measurement results before and after a change in the distance measurement range. Patent Document 2 describes a technology that estimates the adhesion of dirt to a target part based on the amount of reflection of light irradiated to the target part. Patent Document 3 describes a technology that estimates the adhesion of dirt to a lens system based on the level of a light receiving signal when distance measurement is successful.

JP 2016-70796 A Japanese Patent Application Laid-Open No. 9-211108 Japanese Patent Application Laid-Open No. 1-169386

However, with conventional technology, it was sometimes not possible to appropriately control the operation of the vehicle depending on the degree of contamination of the measuring device. For example, even if the degree of contamination of the LIDAR was only slight, there was a possibility that its function would be restricted more than necessary.

The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a vehicle recognition system and recognition method that can prevent the vehicle's control functions from being unnecessarily restricted by the adhesion of dirt to the LIDAR.

The vehicle recognition system and the recognition method according to the present invention employ the following configuration.
(1): A vehicle recognition system according to one embodiment of the present invention is an on-board LIDAR device that detects objects using light or electromagnetic waves similar to light, the on-board LIDAR device having a dirt detection unit that calculates a dirt value indicating the degree of dirt of the device based on the object detection results, for each section into which the object detection range is divided, and a dirt determination device having a dirt determination unit that determines whether or not dirt that should be removed is attached to the on-board LIDAR device, based on the dirt values calculated for each of the sections.

(2): In the above aspect (1), the dirt detection unit calculates the dirt value based on the detection distance and detection point of the object by the vehicle-mounted LIDAR device.

(3): In the above aspect (1) or (2), the sections are obtained by dividing the detection range into a plurality of regions in the horizontal or vertical direction with the vehicle-mounted LIDAR device at the center, and the dirt determination unit determines whether or not dirt that should be removed is attached to the vehicle-mounted LIDAR device by comparing the dirt value measured for each section with a threshold value set for each section.

(4): In any one of the above aspects (1) to (3), a lower threshold is set for the section corresponding to the front or rear of the vehicle among the multiple sections, and a higher threshold is set for the section corresponding to the lateral direction of the vehicle, compared to the other areas.

(5): In any one of the above aspects (1) to (4), an automatic driving control device is further provided that controls driving assistance or automatic driving of the vehicle based on the object detection result, and the automatic driving control device is provided with a notification unit that notifies the vehicle-mounted LIDAR device that dirt that should be removed is attached based on the judgment result of the dirt judgment unit.

(6): In the above aspect (5), the automatic driving control device further includes a control unit that controls the driving assistance or the automatic driving, and the control unit determines whether or not the driving assistance or the automatic driving is possible based on the determination result of the contamination determination unit.

(7): A recognition method according to one aspect of the present invention is a recognition method in which an in-vehicle LIDAR device that detects an object using light or electromagnetic waves similar to light calculates a dirt value indicating the degree of dirt on the device based on the detection result of the object for each section obtained by dividing the detection range of the object into multiple sections, and a dirt determination device determines whether or not dirt that should be removed is attached to the in-vehicle LIDAR device based on the dirt value calculated for each of the multiple sections.

According to (1) to (7), an in-vehicle LIDAR device that detects objects using light or electromagnetic waves similar to light calculates a dirt value indicating the degree of dirt of the device for each section obtained by dividing the detection range of the object into a plurality of sections based on the detection result of the object, and a dirt determination device determines whether or not dirt that should be removed is attached to the in-vehicle LIDAR device based on the dirt value calculated for each of the plurality of sections, thereby preventing the control function of the vehicle from being unnecessarily suppressed due to the attachment of dirt to the LIDAR.

1 is a configuration diagram of a vehicle system using a vehicle control device according to a first embodiment. FIG. 2 is a functional configuration diagram of a first control unit and a second control unit according to the first embodiment. FIG. 2 is a diagram showing an example of a detection range of a LIDAR mounted on a host vehicle in the vehicle system of the first embodiment. FIG. 2 is a diagram showing an example of a detection range of a LIDAR mounted on a host vehicle in the vehicle system of the first embodiment. FIG. 4 is a diagram illustrating an example of threshold information in the first embodiment. FIG. 11 is a configuration diagram of a vehicle system using a vehicle control device according to a second embodiment. FIG. 11 is a diagram illustrating an example of condition information according to the second embodiment. 13 is a flowchart illustrating an example of a method by which a dirt determination unit manages a dirt flag in an object recognition device according to a second embodiment.

Below, an embodiment of the vehicle recognition system and recognition method of the present invention will be described with reference to the drawings.

First Embodiment
[overall structure]
1 is a configuration diagram of a vehicle system 1A using a vehicle control device according to a first embodiment. The vehicle on which the vehicle system 1A is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or discharged electric power from a secondary battery or a fuel cell.

The vehicle system 1A includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14A, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by multiple communication lines such as a CAN (Controller Area Network) communication line, serial communication lines, wireless communication networks, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added.

The camera 10 is, for example, a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of the vehicle (hereinafter, the host vehicle) in which the vehicle system 1A is mounted. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, or the like. The camera 10, for example, periodically and repeatedly captures images of the surroundings of the host vehicle. The camera 10 may be a stereo camera.

The radar device 12 emits radio waves such as millimeter waves around the vehicle and detects radio waves reflected by objects (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 is attached to any location on the vehicle. The radar device 12 may detect the position and speed of an object using the FM-CW (Frequency Modulated Continuous Wave) method.

The LIDAR 14A irradiates light (or electromagnetic waves with a wavelength close to that of light) around the vehicle and measures the scattered light. For example, the LIDAR 14A irradiates infrared light. The LIDAR 14A detects the distance to the target based on the time between emitting and receiving the light. The irradiated light is, for example, a pulsed laser light. The LIDAR 14A can be attached to any location on the vehicle.

In addition to the above basic functions (hereinafter referred to as "LIDAR functions"), the LIDAR 14A in this embodiment has a function to detect dirt attached to the light-emitting unit or the light-receiving unit (hereinafter referred to as "LIDAR dirt"), and a function to determine the degree of dirt for each LIDAR 14A and output a dirt flag. The dirt flag is a flag that indicates whether or not each LIDAR 14A is in a state that requires removal of LIDAR dirt (hereinafter referred to as "dirty state"). To realize these functions, the LIDAR 14A in this embodiment is equipped with a dirt detection unit 151 and a dirt determination unit 152A, and stores threshold information 153.

The dirt detection unit 151 and the dirt determination unit 152A are realized, for example, by a hardware processor such as a CPU executing a program (software). In addition, part or all of the dirt detection unit 151 and the dirt determination unit 152A may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (storage device having a non-transient storage medium) such as an HDD or flash memory of the LIDAR 14A, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the LIDAR 14A by mounting the storage medium (non-transient storage medium) in a drive device.

The dirt detection unit 151 detects LIDAR dirt based on the distance (reflection point) detection results using the LIDAR function and the intensity of reflected light observed during distance detection (hereinafter referred to as "reflection intensity"). Note that the reflection point is an example of a "detection point."

The dirt detection unit 151 is realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). The dirt detection unit 151 may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by collaboration between software and hardware. The program may be stored in advance in a storage device (storage device having a non-transient storage medium) such as an HDD or flash memory of the LIDAR 14A, or may be stored in a removable storage medium such as a DVD or CD-ROM, and may be installed in the HDD or flash memory of the LIDAR 14A by mounting the storage medium (non-transient storage medium) in a drive device. The LIDAR 14A is an example of an "on-vehicle LIDAR device".

Specifically, the dirt detection unit 151 calculates an index value (hereinafter referred to as a "dirt value") indicating the degree of LIDAR dirt based on the detection result by the LIDAR function and the reflection intensity. The dirt value may be any value indicating the degree of LIDAR dirt, and may be calculated by any method. For example, the dirt detection unit 151 calculates the dirt value based on the number of reflection points near the LIDAR 14A, the distance to the reflection points, and the reflection intensity. The dirt detection unit 151 calculates the dirt value for each region obtained by dividing the horizontal detection range of the LIDAR 14A into a predetermined number. In the following, each region obtained by dividing the detection range into a predetermined number is called a section, and in this embodiment, a case will be described in which the detection range of each LIDAR 14A is divided into 10 sections. The dirt detection unit 151 notifies the dirt determination unit 152A of the calculated dirt value.

The dirt determination unit 152A compares the dirt value notified from the dirt detection unit 151 with the threshold information 153 to determine whether the LIDAR 14A is in a dirty state. Here, the threshold information 153 is information used when determining whether the LIDAR 14A is in a dirty state, and is information indicating the threshold value of the dirt value. Hereinafter, this determination is referred to as "dirt determination." The dirt determination unit 152A updates the dirt flag according to the result of the dirt determination, and notifies the object recognition device 16 of the updated dirt flag value. For example, the dirt determination unit 152A sets the dirt flag when it determines that the LIDAR 14A is in a dirty state (sets the dirt flag value to 1), and lowers the dirt flag when it determines that the LIDAR 14A is not in a dirty state (sets the dirt flag value to 0).

The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14A to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition result to the autonomous driving control device 100. The object recognition device 16 also notifies the autonomous driving control device 100 of the value of the dirt flag notified by the LIDAR 14A.

The communication device 20 communicates with other vehicles in the vicinity of the vehicle using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), or DSRC (Dedicated Short Range Communication), or communicates with various server devices via a wireless base station.

The HMI 30 presents various information to the vehicle occupants and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys, etc. For example, the HMI 30 performs an operation to notify the driver that the LIDAR 14A is dirty (hereinafter referred to as a "dirt notification operation") in response to an instruction from the automatic driving control device 100. For example, the HMI 30 performs an operation such as displaying a message or outputting a voice to encourage the driver to remove LIDAR dirt as the dirt notification operation.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and a direction sensor that detects the direction of the vehicle.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 holds first map information 54 in a storage device such as a HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle based on a signal received from a GNSS satellite. The position of the vehicle may be identified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, a key, and the like. The navigation HMI 52 may be partially or entirely shared with the above-mentioned HMI 30. The route determination unit 53 determines, for example, a route (hereinafter, a route on a map) from the position of the vehicle identified by the GNSS receiver 51 (or any input position) to a destination input by the occupant using the navigation HMI 52, by referring to the first map information 54. The first map information 54 is, for example, information in which road shapes are expressed by links indicating roads and nodes connected by the links. The first map information 54 may also include road curvature and POI (Point Of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may perform route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by the functions of a terminal device such as a smartphone or tablet terminal owned by the occupant. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a number of blocks (for example, every 100 m in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. When there is a branch point on the route on the map, the recommended lane determination unit 61 determines the recommended lane so that the vehicle can travel a reasonable route to proceed to the branch point.

The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of lanes or information on lane boundaries. The second map information 62 may also include road information, traffic regulation information, address information (address and zip code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time by the communication device 20 communicating with other devices.

The driving operators 80 include, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, an irregular steering wheel, a joystick, and other operators. The driving operators 80 are fitted with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100, or some or all of the driving force output device 200, the brake device 210, and the steering device 220.

The automatic driving control device 100 is a device that controls automatic driving (including driving assistance) of the vehicle. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, and a notification control unit 180. The first control unit 120, the second control unit 160, and the notification control unit 180 are each realized by, for example, a hardware processor such as a CPU executing a program (software). In addition, some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (storage device having a non-transient storage medium) such as an HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and may be installed in the HDD or flash memory of the automatic driving control device 100 by attaching the storage medium (non-transient storage medium) to a drive device. The combination of the HMI 30 and the notification control unit 180 is an example of an "notification unit."

FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120 realizes, for example, a function based on AI (Artificial Intelligence) and a function based on a pre-given model in parallel. For example, the function of "recognizing an intersection" may be realized by executing recognition of an intersection by deep learning or the like and recognition based on pre-given conditions (such as traffic lights and road markings that can be pattern matched) in parallel, and scoring both and evaluating them comprehensively. This ensures the reliability of autonomous driving.

The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the vehicle based on information input from the camera 10, the radar device 12, and the LIDAR 14A via the object recognition device 16. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of an object may include the acceleration or jerk of the object, or the "behavioral state" (for example, whether or not the object is changing lanes or is about to change lanes).

The recognition unit 130 also recognizes, for example, the lane in which the vehicle is traveling (driving lane). For example, the recognition unit 130 recognizes the driving lane by comparing the pattern of road dividing lines (for example, an arrangement of solid lines and dashed lines) obtained from the second map information 62 with the pattern of road dividing lines around the vehicle recognized from the image captured by the camera 10. Note that the recognition unit 130 may recognize the driving lane by recognizing road boundaries (road boundaries) including road dividing lines, shoulders, curbs, medians, guardrails, etc., in addition to road dividing lines. In this recognition, the position of the vehicle obtained from the navigation device 50 and the processing results by the INS may be taken into account. The recognition unit 130 also recognizes stop lines, obstacles, red lights, toll booths, and other road phenomena.

When recognizing the driving lane, the recognition unit 130 recognizes the position and attitude of the host vehicle relative to the driving lane. For example, the recognition unit 130 may recognize the deviation of the host vehicle's reference point from the center of the lane and the angle with respect to a line connecting the centers of the lanes in the host vehicle's traveling direction as the relative position and attitude of the host vehicle relative to the driving lane. Alternatively, the recognition unit 130 may recognize the position of the host vehicle's reference point relative to either side end of the driving lane (a road dividing line or a road boundary) as the relative position of the host vehicle relative to the driving lane.

In principle, the action plan generation unit 140 drives the recommended lane determined by the recommended lane determination unit 61, and further generates a target trajectory along which the vehicle will automatically (without the driver's operation) drive in the future so that the vehicle can respond to the surrounding conditions of the vehicle. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) to be reached by the vehicle. The trajectory points are points to be reached by the vehicle at each predetermined driving distance (for example, about several meters) along the road, and separately, the target speed and target acceleration for each predetermined sampling time (for example, about a few tenths of a second) are generated as part of the target trajectory. The trajectory points may also be the positions to be reached by the vehicle at each sampling time for each predetermined sampling time. In this case, the information on the target speed and target acceleration is expressed as the interval between the trajectory points.

The behavior plan generation unit 140 may set an autonomous driving event when generating the target trajectory. Autonomous driving events include a constant speed driving event, a low speed following driving event, a lane change event, a branching event, a merging event, and a takeover event. The behavior plan generation unit 140 generates a target trajectory according to the activated event.

The second control unit 160 controls the driving force output device 200, the brake device 210, and the steering device 220 so that the vehicle passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

The first control unit 120 or the second control unit 160 may be configured to determine whether driving assistance or automatic driving is possible based on the determination result of the dirt determination unit 152A, and to determine whether to execute, temporarily suspend, stop, resume, end, etc., processing related to driving assistance or automatic driving based on the determination result.

Returning to FIG. 2, the second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the vehicle and feedback control based on the deviation from the target trajectory.

Returning to FIG. 1, the notification control unit 180 has a function of instructing the HMI 30 to perform a dirt notification operation according to the value of the dirt flag notified by the LIDAR 14A via the object recognition device 16. For example, when the notification control unit 180 is notified of a dirt flag value of 1, it instructs the HMI 30 to perform a dirt notification operation, and when the notification control unit 180 is notified of a dirt flag value of 0, it does not instruct the HMI 30 to perform a dirt notification operation.

The driving force output device 200 outputs a driving force (torque) to the drive wheels for the vehicle to travel. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls these. The ECU controls the above configuration according to information input from the second control unit 160 or information input from the driving operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the driving operation unit 80, so that a brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include a backup mechanism that transmits hydraulic pressure generated by operating the brake pedal included in the driving operation unit 80 to the cylinder via a master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by, for example, applying a force to a rack and pinion mechanism. The steering ECU drives the electric motor according to information input from the second control unit 160 or information input from the driving operator 80, to change the direction of the steered wheels.

3A and 3B are diagrams showing an example of the detection range of the LIDAR 14A mounted on the vehicle M in the vehicle system 1A of the first embodiment. FIG. 3A and FIG. 3B show the detection ranges of a total of five LIDARs 14A, one each provided in the front right, front left, rear center, rear right, and rear left of the vehicle M. In FIG. 3A, LIDAR 14A-FR represents the LIDAR 14A in the front right portion, and LIDAR 14A-FL represents the LIDAR in the front left portion. Also, in FIG. 3B, LIDAR 14A-RR represents the LIDAR in the rear right portion, LIDAR 14A-RL represents the LIDAR 14A in the rear left portion, and LIDAR 14A-RC represents the LIDAR 14A in the rear center portion. In the following, when there is no distinction between FIG. 3A and FIG. 3B, both may be collectively referred to as FIG. 3.

In Figure 3, the detection ranges DR-FR, DR-FL, DR-RR, DR-RL, and DR-RC represent the detection ranges of LIDAR14A-FR, LIDAR14A-FL, LIDAR14A-RR, LIDAR14A-RL, and LIDAR14A-RC in the front right, front left, rear center, rear right, and rear left, respectively.

As described above, in the vehicle system 1A of this embodiment, the contamination value is calculated for each of the 10 sections obtained by dividing the detection range of each LIDAR 14A. Specifically, the detection range DR-FR is divided into sections FR01 to FR10, the detection range DR-FL is divided into sections FL01 to FL10, the detection range DR-RR is divided into sections RR01 to RR10, the detection range DR-RL is divided into sections RL01 to RL10, and the detection range DR-RC is divided into sections RC01 to RC10.

In the case of autonomous vehicle driving control, distance detection directly in front of the vehicle M, directly behind the vehicle M, and diagonally to the left and right is generally important, and relatively higher detection accuracy is required for distance detection in these directions than in other areas. For this reason, it is desirable to eliminate LIDAR dirt from the LIDAR 14A used for detection in these directions while the degree of dirt is still relatively light. Therefore, in the vehicle system 1A of this embodiment, a dirt judgment threshold is set for each section of each detection range. This makes it possible to detect localized LIDAR dirt and to detect LIDAR dirt in high-priority sections at a relatively early stage.

The orientation of the detection range of each LIDAR 14A illustrated in FIG. 3 and the number of sections constituting each detection range are merely examples, and may be changed to a different form from FIG. 3 as necessary. The number and installation positions of the LIDAR 14A installed on the vehicle M may also be changed to a different form from FIG. 3. For example, the detection range may be divided into 11 or more sections, or into 9 or fewer sections. The detection range may be divided not only horizontally but also vertically, or may be divided only vertically. For example, the multiple LIDARs 14A may be provided so that their respective detection ranges are symmetrical left-right or up-down with respect to a predetermined axis (for example, the central axis or traveling direction of the vehicle), or may be provided so that they are asymmetrical.

FIG. 4 is a diagram showing an example of threshold information 153 in the first embodiment. For example, as shown in FIG. 4, threshold information 153 is stored in LIDAR 14A as a table showing the correspondence between the section ID of each section and the dirt value threshold in the dirt determination of each section. FIG. 4 shows threshold information 153-FL as an example of threshold information for LIDAR 14A-FL installed in the front left part of the vehicle M, and shows threshold information 153-FR as an example of threshold information for LIDAR 14A-FR installed in the front right part of the vehicle M. FIG. 4 also shows threshold information 153-RL as an example of threshold information for LIDAR 14A-RL installed in the rear left part of the vehicle M, shows threshold information 153-RC as an example of threshold information for LIDAR 14A-RC installed in the rear center part of the vehicle M, and shows threshold information 153-RR as an example of threshold information for LIDAR 14A-RR installed in the rear right part of the vehicle M. In the example of Figure 4, the section ID values correspond to the section IDs shown in Figure 3.

As described above, in the autonomous driving control of the host vehicle M, it is important to detect distances in the front, rear, and diagonally rearward directions of the host vehicle M. Therefore, in the example of threshold information 153-FL, the thresholds for sections FL03 to FL06 corresponding to the front of the host vehicle M are set to 50%, which is relatively low compared to the other sections. Similarly, in the example of threshold information 153-FR, the thresholds for sections FR05 to FR08 corresponding to the front of the host vehicle M are set to 50%, which is relatively low compared to the other sections.

On the other hand, in the example of threshold information 153-RL, the thresholds for sections RL06 to RL08 corresponding to the left diagonal rear of the host vehicle M are set to 60%, which is relatively low compared to the other sections. Similarly, in the example of threshold information 153-RC, the thresholds for sections RC04 to RC07 corresponding to the front rear of the host vehicle M are set to 50%, which is relatively low compared to the other sections. Similarly, in the example of threshold information 153-RR, the thresholds for sections RR03 to RR05 corresponding to the right diagonal rear of the host vehicle M are set to 60%, which is relatively low compared to the other sections.

In this way, of the multiple sections, a lower threshold is set for the sections that correspond to the front or rear of the vehicle M (e.g., sections FL03-FL06, FR05-FR08, RC04-RC07, RL06-RL08, RR03-RR05) than for other areas within the same detection range, and a higher threshold is set for the sections to the side of the vehicle (e.g., sections other than sections FL03-FL06, FR05-FR08, RC04-RC07, RL06-RL08, RR03-RR05) than for other areas.

In this case, in the automatic driving control device 100, the dirt determination unit 152A compares the dirt value (hereinafter referred to as "measurement value") notified for each section of each LIDAR 14A with the threshold value set for each section. If there is a section where the measurement value is equal to or greater than the threshold value, the dirt determination unit 152A sets a dirt flag for the LIDAR 14A that has that section within its detection range and notifies the automatic driving control device 100. In the automatic driving control device 100 that receives this notification, the notification control unit 180 instructs the HMI 30 to perform a dirt notification operation for that LIDAR 14A.

For example, in the example of FIG. 4, if a dirt value of 55% is measured for section FL05, the dirt determination unit 152A will set a dirt flag for LIDAR14A-FL, which has section FL05 in its detection range, and notify the automatic driving control device 100, since the measured value is equal to or greater than the threshold value of 50% for section FL05. In the automatic driving control device 100 that receives this notification, the notification control unit 180 instructs the HMI 30 to perform a dirt notification operation for LIDAR14A-FL.

In the vehicle system 1A of the first embodiment configured in this manner, each LIDAR 14A mounted on the vehicle calculates a dirt value for each section obtained by dividing the horizontal detection range into a predetermined number of sections, and the automatic driving control device 100 determines whether or not to perform a dirt notification operation for each LIDAR 14A based on the dirt value calculated for each section and the dirt value threshold value set for the corresponding section. With this configuration, the vehicle system 1A of the embodiment can detect localized LIDAR dirt and can detect LIDAR dirt at a timing according to the priority of each section. In other words, the vehicle system 1A of the embodiment can prevent the control function of the vehicle from being unnecessarily suppressed due to the adhesion of dirt to the LIDAR.

Second Embodiment
FIG. 5 is a configuration diagram of a vehicle system 1B using a vehicle control device according to the second embodiment. The vehicle system 1B differs from the vehicle system 1A of the first embodiment in that the vehicle system 1B includes a LIDAR 14B instead of the LIDAR 14A. The LIDAR 14B also differs from the LIDAR 14A of the first embodiment in that the LIDAR 14B includes a dirt determination unit 152B instead of the dirt determination unit 152A, and that the condition information 154 is stored instead of the threshold information 153. The other configurations of the vehicle system 1B are the same as those of the vehicle system 1A. Therefore, in FIG. 5, the same reference numerals as those in FIG. 1 are used, and the description of the similar configurations will be omitted.

The dirt determination unit 152B differs from the dirt determination unit 152A in that, whereas the dirt determination unit 152A of the first embodiment managed the dirt flag based on the comparison result between the dirt value notified from the dirt detection unit 151 and a threshold value, the dirt determination unit 152B manages the dirt flag based on the fluctuation status of the dirt value in addition to the comparison result with the threshold value. Specifically, the dirt determination unit 152B manages the dirt flag based on the result of the dirt determination according to multiple determination conditions defined by the condition information 154.

Figure 6 is a diagram showing an example of condition information 154 in the second embodiment. For example, the condition information 154 registers the names (condition names) of multiple judgment conditions used in the dirt judgment, the contents (condition contents) of the judgment conditions, and the types (condition types) of the judgment conditions in association with each other. For example, in the example of condition information 154 in Figure 6, a dirt threshold condition and a dirt fluctuation condition are registered as conditions for setting a dirt flag (flag ON condition), and a recovery condition is set as a condition for lowering the dirt flag (flag OFF condition). Details of each flag ON condition and flag OFF condition are as follows:

[Contamination threshold condition]
The dirt threshold condition is a condition for determining whether or not the amount of dirt adhering to the LIDAR 14B exceeds a certain amount. In general, when a vehicle is traveling at high speed in a low-temperature environment, water droplets on the surface of the LIDAR 14B may adhere for a short time like dew. Therefore, in such a situation, the dirt value may temporarily increase, and this state may be erroneously detected as a dirty state. In addition, in a low-speed traveling state immediately after the vehicle starts, the possibility that water droplets due to rainfall will remain on the surface of the LIDAR 14B is low, and the dirt value basically does not exceed the threshold. Therefore, in this embodiment, a condition for determining that the judgment result is true when a predetermined time (e.g., 300 seconds) has passed since the dirt value exceeded the threshold without falling below the threshold is set as the dirt value condition. Hereinafter, the above-mentioned predetermined time when determining the dirt threshold condition is referred to as the judgment time.

[Conditions for fluctuation of dirt value]
The dirt value fluctuation condition is a condition for determining whether the LIDAR dirt is dirt that needs to be wiped off. Specifically, the dirt value fluctuation condition is a condition for determining that the judgment result is true when the magnitude of the dirt value fluctuation in the past judgment time (e.g., 300 seconds) that has elapsed in a state in which the dirt value has exceeded the threshold value is less than a predetermined ratio (e.g., the difference between the maximum value and the minimum value is less than 10%). This is due to the fact that, as an example of dirt that needs to be wiped off, when a mixture of oil and sand is attached to the surface of the LIDAR 14B, the magnitude of the dirt value fluctuation may exceed 10% before the judgment time has elapsed, and if the judgment time is too long, it may become impossible to detect dirt that needs to be wiped off. In addition, since it usually takes several minutes to stop the vehicle in order to wipe off the LIDAR dirt, the judgment time may be, for example, about 300 seconds.

[Recovery conditions]
As described above, the restoration condition is a condition for determining whether or not the dirt flag that is raised when the flag ON condition is satisfied may be lowered. Therefore, the restoration condition needs to be a condition that can determine that the vehicle is not in a dirty state. Therefore, in this embodiment, the restoration condition is a condition that determines that the determination result is true when the dirt value is equal to or less than a threshold value (e.g., 30%) in all sections. In this case, the threshold value may be designed in consideration of being able to lower the dirt flag even if scratches or dirt that do not affect the detection performance remain on the surface of the LIDAR 14B, and being able to prevent the dirt flag from being frequently turned ON and OFF.

Figure 7 is a flowchart showing an example of a method in which the dirt determination unit 152B manages dirt flags in the LIDAR 14B of the second embodiment. Here, it is assumed that dirt flags are managed for each LIDAR 14B, and the process flow for managing the dirt flag of one LIDAR 14B will be described. First, the dirt determination unit 152B determines whether the dirt values of all sections satisfy the dirt value condition (step S101). If it is determined that the dirt values of all sections satisfy the dirt value condition, the dirt determination unit 152B determines whether the dirt values of all sections satisfy the dirt value fluctuation condition (step S102). If it is determined that the dirt values of all sections satisfy the dirt value fluctuation condition, the dirt determination unit 152B sets a dirt flag (step S103).

On the other hand, if it is determined in step S101 that the dirt value of any section does not satisfy the dirt value condition, or if it is determined in step S102 that the dirt value of any section does not satisfy the dirt value fluctuation condition, the dirt determination unit 152B determines whether the restoration condition is satisfied (step S104). If it is determined that the restoration condition is satisfied, the dirt determination unit 152B lowers the dirt flag (step S105) and returns the process to step S101. On the other hand, if it is determined in step S104 that the restoration condition is not satisfied, the dirt determination unit 152B returns the process to step S101 without lowering the dirt flag.

The vehicle system 1B of the second embodiment configured in this manner includes a dirt determination unit 152B that determines whether each LIDAR 14B is in a dirty state by combining multiple conditions including conditions related to the dirt value. By being configured in this manner, the vehicle system 1B of the embodiment can more accurately determine whether the LIDAR 14B is in a dirty state. In other words, the vehicle system 1B of the embodiment can prevent the control function of the host vehicle from being unnecessarily suppressed due to LIDAR dirt.

<Modification>
In the first embodiment, the dirt determination unit 152A and the threshold information 153 may be provided in the object recognition device 16 or the automatic driving control device 100. Similarly, the dirt determination unit 152B and the condition information 154 may be provided in the object recognition device 16 or the automatic driving control device 100.

The object recognition device 16 may output the detection results (including the dirt value) of the camera 10, the radar device 12, and the LIDAR 14A (or 14B, the same below) directly to the autonomous driving control device 100. Also, the object recognition device 16 may be omitted from the vehicle system 1A (or 1B, the same below).

In the first control unit 120 and the second control unit 160, each functional unit such as the recognition unit 130, the action plan generation unit 140, the acquisition unit 162, the speed control unit 164, or the steering control unit 166 may be configured to branch or change the processing based on the value of the dirt flag notified from the LIDAR 14A via the object recognition device 16. Similarly, the object recognition device 16 may also be configured to branch or change the processing based on the value of the dirt flag notified from the LIDAR 14A. For example, the first control unit 120 and the second control unit 160 may be configured to degenerate the operation related to the automatic driving of the host vehicle when it is recognized that any of the LIDARs 14A is in a dirty state based on the notified dirt flag. In addition, the automatic driving control device 100 may notify other devices such as the navigation device 50 and the MUP 60 of the notified dirt flag. In this case, the other devices may be configured to branch or change the processing of their own devices based on the value of the dirt flag.

The notification control unit 180 may cause a device other than the HMI 30 to perform the dirt notification operation, or may perform the dirt notification operation itself. For example, the notification control unit 180 may send an email to a user who is to be notified of LIDAR dirt, or may send a message notification to a terminal device used by that user.

The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

1A, 1B...vehicle system, 10...camera, 12...radar device, 14A, 14B...LIDAR (Light Detection and Ranging), 151...dirt detection unit, 152A, 152B...dirt determination unit, 153...threshold information, 154...condition information, 16...object recognition device, 20...communication device, 30...HMI (Human Machine Interface), 40...vehicle sensor, 50...navigation device, 51...GNSS (Global Navigation Satellite System) receiver, 52...navigation HMI, 53...route determination unit, 54...first map information, 60...MPU (Map Positioning Unit), 61...recommended lane determination unit, 62...second map information, 80...driving operator, 100...automatic driving control device, 100-1...communication controller, 100-2...CPU (Central Processing Unit), 100-3...RAM (Random Access Memory), 100-4...ROM (Read Only Memory), 100-5...storage device, 100-5a...program, 100-6...drive device, 120...first control unit, 130...recognition unit, 140...action plan generation unit, 160...second control unit, 164...speed control unit, 166...steering control unit, 180...notification control unit, 200...driving force output device, 210...brake device, 220...steering device

Claims (5)

A plurality of vehicle-mounted LIDAR devices are installed at five locations on the front left and right sides, the rear left and right sides, and the rear front of a vehicle, and detect objects using light or electromagnetic waves similar to light, the plurality of vehicle-mounted LIDAR devices each having a dirt detection unit that calculates a dirt value indicating a degree of dirt of the vehicle-mounted LIDAR device for each section obtained by dividing a detection range of the object into a plurality of sections based on the detection result of the object;
A dirt determination device including a dirt determination unit that determines whether or not dirt to be removed is attached to the on-board LIDAR device based on the dirt values calculated for each of the multiple sections;
Equipped with
The sections are obtained by dividing the detection range into a plurality of regions in at least the horizontal direction of the horizontal and vertical directions with the vehicle-mounted LIDAR device at the center, and among the plurality of sections, a first threshold value is set in a section corresponding to the front or rear of the vehicle, a second threshold value larger than the first threshold value is set in a section corresponding to the lateral direction of the vehicle , and a value lower than the second threshold value of the vehicle-mounted LIDAR devices on the left and right front and the front rear is set for the second threshold value of the vehicle-mounted LIDAR devices on the left and right rear;
The dirt determination unit determines whether or not dirt that should be removed is attached to the on-board LIDAR device by comparing the dirt value measured for each section with a threshold value set for each section.
Vehicle recognition system. The dirt detection unit calculates the dirt value based on a distance from the in-vehicle LIDAR device to a detection point at which the object is detected by the in-vehicle LIDAR device and an intensity of the light or the electromagnetic wave reflected by the object at the detection point in relation to the detection of the distance.
The vehicle recognition system according to claim 1 . The vehicle further includes an automatic driving control device that controls driving assistance or automatic driving of the vehicle based on the detection result of the object,
The automatic driving control device includes:
and a notification unit that notifies the vehicle-mounted LIDAR device that dirt to be removed is attached to the vehicle-mounted LIDAR device based on a determination result of the dirt determination unit.
A recognition system for a vehicle according to claim 1 or 2. The automatic driving control device further includes a control unit that controls the driving assistance or the automatic driving,
The control unit determines whether or not the driving assistance or the automatic driving is possible based on a determination result of the dirt determination unit.
The vehicle recognition system according to claim 3. A plurality of vehicle-mounted LIDAR devices that detect objects using light or electromagnetic waves close to light are installed at five locations on the front left and right, rear left and right, and rear front of the vehicle .
Calculating a dirt value indicating a degree of dirt on the device itself based on the object detection result for each of a plurality of sections obtained by dividing the object detection range;
The dirt detection device
determining whether or not dirt that should be removed is attached to the vehicle-mounted LIDAR device based on the dirt value calculated for each of the multiple sections;
A method of recognition comprising:
The sections are obtained by dividing the detection range into a plurality of regions in at least the horizontal direction of the horizontal and vertical directions with the vehicle-mounted LIDAR device at the center, and among the plurality of sections, a first threshold value is set in a section corresponding to the front or rear of the vehicle, a second threshold value larger than the first threshold value is set in a section corresponding to the lateral direction of the vehicle , and a value lower than the second threshold value of the vehicle-mounted LIDAR devices on the left and right front and the front rear is set for the second threshold value of the vehicle-mounted LIDAR devices on the left and right rear;
The vehicle-mounted LIDAR device compares the dirt value measured for each section with a threshold value set for each section to determine whether or not dirt that should be removed is attached to the vehicle-mounted LIDAR device.
How to recognize.

Images