WO2024161481A1

WO2024161481A1 - Vehicle control device

Info

Publication number: WO2024161481A1
Application number: PCT/JP2023/002942
Authority: WO
Inventors: 僚太稲垣; 聡柏村
Original assignee: 日立Astemo株式会社
Priority date: 2023-01-30
Filing date: 2023-01-30
Publication date: 2024-08-08

Abstract

Provided is a vehicle control device capable of highly accurate sensing, by reducing incorrect grouping of millimeter-wave radars by using a camera x millimeter-wave radar algorithm, in driving assistance technologies. This vehicle control device comprises: a camera that recognizes a three-dimensional object around a host vehicle; and a millimeter-wave radar that recognizes peripheral targets of the host vehicle, sets the three-dimensional object as an independent target on the basis of the information on the three-dimensional object recognized by the camera, and when the millimeter-wave radar recognizes the three-dimensional object and another target as peripheral targets, recognizes the three-dimensional object as a target different from the other target.

Description

Vehicle control device

The present invention relates to a vehicle control device.

In automotive driving assistance control, systems that use forward sensors consisting of cameras and millimeter-wave radar to detect obstacles ahead and automatically apply the brakes to avoid a collision or mitigate damage if there is a risk of collision are becoming widespread.

In recent years, in order to improve safety, there has been a need to properly detect obstacles and apply the brakes even when an obstacle (pedestrian, bicycle, etc.) suddenly appears in front of the vehicle from outside the detection range of the forward sensor at intersections and crosswalks. Also, automobile assessments (such as EURO NCAP) have introduced protocols such as AEB pedestrian forward intersection protocols, which avoid collisions with pedestrians crossing the sidewalk in the same direction when the vehicle turns at an intersection.

These systems need to be able to detect obstacles well before a collision occurs. However, detection using forward sensors consisting of cameras and millimeter-wave radar is insufficient for obstacles that suddenly appear in front of the vehicle from outside the forward sensor's detection range.

Currently, due to cost considerations, cameras are not installed on the sides, and in most cases sensing is done using millimeter wave radar alone. However, while millimeter wave radar is generally cheaper than cameras, it has low detection accuracy, so there is a high risk of it not working if the vehicle is controlled based only on the sensing results of the millimeter wave radar. For example, when the vehicle turns at an intersection, the millimeter wave radar may mistakenly group a pedestrian crossing the sidewalk in the same direction with a (stationary) three-dimensional object in the vicinity, which could result in the pedestrian not being detected correctly and a collision with the pedestrian.

As a countermeasure, attention has been focused on camera x millimeter wave radar fusion technology, which uses millimeter wave radar and a camera to improve detection accuracy. For example, the following Patent Document 1 is disclosed.

Patent Document 1 discloses a technology that aims to improve the obstacle detection rate by determining whether a target is moving on the road surface based on target information obtained by radar and vehicle driving information.

JP 2012-103858 A

Patent Document 1 is based on the assumption that the detection results of the target information obtained by the radar are correct, and does not take into account erroneous grouping by the radar. Therefore, if the radar erroneously groups moving targets such as pedestrians with stationary targets such as posts, there is a risk that the moving speed of the moving target will be detected as lower than it actually is.

The object of the present invention is to provide a vehicle control device capable of highly accurate sensing in driving assistance technology by suppressing erroneous grouping of millimeter wave radar using a camera x millimeter wave radar algorithm.

In order to solve the above problems, the vehicle control device according to the present invention has a camera that recognizes three-dimensional objects around the vehicle, and a millimeter wave radar that recognizes targets in the vicinity of the vehicle, and is characterized in that the three-dimensional object is set as an independent target based on information about the three-dimensional object recognized by the camera, and when the millimeter wave radar recognizes the three-dimensional object and other targets as the surrounding targets, it recognizes the three-dimensional object as a target different from the other targets.

The present invention provides a vehicle control device that can suppress erroneous grouping of millimeter wave radar.

　Problems, configurations, and advantages other than those mentioned above will become clear from the description of the embodiments below.

FIG. 1 is a configuration explanatory diagram showing an example of a hardware configuration of a system including a controller (vehicle control device) according to a first embodiment of the present invention. 4 is a flowchart showing a process flow of a fusion program executed by a controller (vehicle control device) in the first embodiment of the present invention. 2 is an overhead view of a host vehicle and its surrounding objects at a certain time _t0 in the first embodiment of the present invention. 2 is an overhead view of a host vehicle and its surrounding objects at a certain time _t1 after a certain time has elapsed from a time _t0 in the first embodiment of the present invention. FIG. 5 is an explanatory diagram of a radar detection point cloud viewed on a coordinate system fixed at the time (time t ₁ ) in FIG. 4 in the first embodiment of the present invention. _{2 is an overhead view of a host vehicle and its surrounding objects at a certain time t2} after a certain time has elapsed from time _t1 in the first embodiment of the present invention. 10 is a flowchart showing a process flow of a fusion program executed by a controller (vehicle control device) according to a second embodiment of the present invention. 13 is an overhead view of the host vehicle and its surrounding objects at a certain time _t3 in the second embodiment of the present invention. FIG. 11 is an overhead view of the host vehicle and its surrounding objects at time _t3 in the second embodiment of the present invention, in which a dummy target is placed in a camera occlusion region of a stationary three-dimensional object O _OBJ . 13 is an overhead view of the host vehicle and its surrounding objects at a certain time _t4 after a certain time has elapsed from the time _t3 in the second embodiment of the present invention. FIG. 11 is an explanatory diagram of a radar detection point cloud viewed on a fixed coordinate system at the time (time t ₄ ) in FIG. 10 in the second embodiment of the present invention. FIG. 11 is an overhead view of the host vehicle and its surrounding objects at a certain time _t5 after a certain time has elapsed from the time _t4 in the second embodiment of the present invention. 10 is a flowchart showing a process flow of a fusion program executed by a controller (vehicle control device) according to a third embodiment of the present invention. 13 is an overhead view of the host vehicle and its surrounding objects at a certain time _t6 in the third embodiment of the present invention. FIG. 13 is an overhead view of the host vehicle and its surrounding objects at time _t6 in the third embodiment of the present invention, in which a dummy target is placed in the camera occlusion area of a pedestrian _OPED . 13 is an overhead view of the host vehicle and its surrounding objects at a certain time _t7 after a certain time has elapsed from the time _t6 in the third embodiment of the present invention. FIG. 17 is an explanatory diagram of a radar detection point cloud viewed on a fixed coordinate system at the time (time t ₇ ) in FIG. 16 in the third embodiment of the present invention. FIG. 11 is an overhead view of the host vehicle and its surrounding objects at a certain time _t8 after a certain time has elapsed from the time _t7 in the third embodiment of the present invention.

The following describes an embodiment of the present invention with reference to the drawings.

[Example 1]
First, a first embodiment of the present invention will be described with reference to FIGS.

In this embodiment, the vehicle coordinate system is set with the vehicle's longitudinal (length) direction as the X-axis, the vehicle's lateral (width) direction as the Y-axis, the center point of the front end of the vehicle body as the origin, and angles set at 0 degrees in the forward direction of the vehicle and positive in the counterclockwise direction.

FIG. 1 is a configuration diagram illustrating an example of the hardware configuration of a system including a controller (vehicle control device) in the first embodiment of the present invention.

This system includes an external information acquisition unit H1, which is composed of a radar device (hereinafter, sometimes simply referred to as a radar or millimeter wave radar) H11 that detects the left and right front or sides, a camera device (hereinafter, sometimes simply referred to as a camera) H12 that detects the front, and the like.
a vehicle information acquisition unit H2 including a speed sensor H21, a steering angle sensor H22, a yaw rate sensor H23, etc.;
a controller H3 that calculates fusion target information that integrates the detection results of the radar and the camera based on the external environment information from the external environment information acquisition unit H1 and the host vehicle information acquired by the host vehicle information acquisition unit H2, and outputs, as necessary, a warning command to the driver to avoid a collision with the fusion target, a brake command to stop the host vehicle, a steering command to turn the host vehicle, etc.;
The vehicle control unit H4 is composed of an alarm device H41 which performs alarm control based on an alarm command calculated by the controller H3, a brake system H42 which performs brake control based on a brake command calculated by the controller H3, and a steering system H43 which performs steering control based on a steering command calculated by the controller H3.

Figure 2 shows the processing flow of the fusion program executed by the controller H3.

First, in step S101, the camera detection results (X position, Y position, width, target attributes, etc.) are obtained from the external information acquisition unit H1.

Next, in process S102, radar detection point information (X position, Y position, radar reflection intensity, etc.) obtained from the external information acquisition unit H1 is acquired.

Next, in process S103, the time axes of the radar detection point information and camera detection results obtained in processes S101 and S102 are unified, and the radar detection point information and camera detection results at the same time are calculated.

Next, in process S104, the position coordinates of the radar detection point information and camera detection results obtained at the same time in process S103 are unified to the center of the front end of the vehicle body.

Next, in process S105, it is determined whether or not a stationary three-dimensional object is present in the camera detection results obtained in process S104.

If it is determined that a stationary three-dimensional object is present based on the camera's detection results (Process S105: Yes), the coordinate origin is fixed in Process S106, and the X position, Y position, and width of the stationary three-dimensional object detected by the camera are stored.

Next, in process S108, the X and Y positions of the vehicle from the fixed coordinate origin and the yaw angle when the yaw angle at the time the coordinates were fixed are estimated as 0 degrees. Furthermore, the coordinate origin of the camera and radar detection results obtained in process S104 is moved to the fixed coordinate origin and rotated by the yaw angle.

On the other hand, if it is determined in step S105 that the camera's detection results indicate that no stationary three-dimensional object exists (step S105: No), and then in step S107 it is determined that the coordinate origin is fixed (step S107: Yes), step S108 is executed.

Next, in process S109, if it is determined that the vehicle has not passed the stationary three-dimensional object stored in process S106 (stationary three-dimensional object detected by the camera) (process S109: No), in process S110, the radar detection points near the stationary three-dimensional object stored in process S106 (stationary three-dimensional object detected by the camera) are excluded from grouping.

On the other hand, if it is determined in step S109 that the vehicle has passed the stationary three-dimensional object stored in step S106 (step S109: Yes), the coordinate fix is released in step S111.

Next, in step S112, grouping processing is performed on radar detection points other than those that are not subject to grouping. This is a process in which radar detection points that are close to each other are regarded as the same target and are treated as a single detection representative point.

On the other hand, if it is determined in process S105 that the camera detection results indicate that no stationary three-dimensional object exists (process S105: No), and then in process S107 it is determined that the coordinate origin is not fixed (process S107: No), process S112 is executed.

The process is now complete.

A scene example for more specifically explaining the processing of the first embodiment is shown in the figure. As an example, a scene in which a stationary solid object exists at the corner of the intersection before the vehicle enters the intersection, and the stationary solid object is detected by the radar device H11 and the camera device H12, and a pedestrian is walking on the sidewalk on the left side of the vehicle in the same traveling direction as the vehicle, and the pedestrian is near the stationary solid object when the vehicle turns the intersection is described in chronological order. FIG. 3 is an overhead view of the vehicle and its surrounding objects at a certain time _t0 . In FIG. 3, a stationary solid object O _OBJ exists in the area where the detection area A _L of the radar device H11 and the detection area A _CAM of the camera device H12 overlap, and a pedestrian O _PED exists in the detection area A _L of the radar device H11. In FIG. 3, if the stationary solid object O _OBJ can be identified as a stationary solid object by the camera, the coordinate origin is fixed in processing S106, and the X position, Y position, and width of the stationary solid object are recorded. The above-mentioned processing is performed when the camera can detect and identify a stationary three-dimensional object, and in Figure 3, the coordinates of the stationary three-dimensional object at the time just before the camera can no longer detect and identify it are stored as ( _X'OBJ ( _t0 ), _Y'OBJ ( _t0 )) and the width is stored as _WOBJ .

Fig. 4 is an overhead view of the host vehicle and its surrounding objects at a certain time _t1 after a certain time has elapsed from time _t0 . In Fig. 4, a stationary three-dimensional object _OOBJ and a pedestrian _OPED are present in a detection area _AAL of a radar device H11 outside a detection area _ACAM of a camera device H12.

Fig. 5 shows the radar detection point cloud as viewed on the coordinates fixed in Fig. 4. In Fig. 5, in process S110, all radar detection points within a circle of diameter W _OBJ centered on the position (X' _OBJ (t ₀ ), Y' _OBJ (t ₀ )) of the stationary three-dimensional object recorded in process S106 are assigned an attribute of not being grouped. Next, in process S112, grouping process is performed on the radar detection points other than those with the non-grouping attribute.

FIG. 6 is an overhead view of the vehicle and its surrounding objects at a certain time _t2 after a certain time has elapsed since time _t1 . In FIG. 6, a stationary three-dimensional object O _OBJ and a pedestrian O _PED are present in a detection area A _L of the radar device H11 outside the detection area A _CAM of the camera device H12. FIG. 6 shows an example of a scene in which it is determined in process S109 that the vehicle has passed the stationary three-dimensional object recorded in process S106. The coordinates (X' _OBJ (t ₀ ), Y' _OBJ (t ₀ )) of the stationary three-dimensional object recorded in process S106 are converted into coordinates with the X axis in the longitudinal direction of the vehicle, the Y axis in the lateral direction of the vehicle, the center point of the front end of the vehicle body as the origin, and the angle being positive counterclockwise with the vehicle front direction as 0 degrees, to be (X'' _OBJ (t ₀ ), Y'' _OBJ (t ₀ ). In this case, when the overall length of the vehicle is L _OVERALL , if formula (1) is established, it is determined that the host vehicle has passed the stationary three-dimensional object.

In the above explanation of Example 1, there is only one stationary three-dimensional object to avoid cluttering the diagram, but if there are multiple stationary three-dimensional objects, the processing of Example 1 is performed on all stationary three-dimensional objects detected by the camera.

In this embodiment, by assigning the attribute of non-grouping to stationary three-dimensional objects detected by the camera, it is possible to detect each separately (as different targets) without mistakenly grouping stationary three-dimensional objects with moving objects such as pedestrians nearby in the radar detection area outside the camera detection area. Also, although only one radar device is described in this embodiment, it is also applicable to multiple radar devices.

[Example 2]
Next, a second embodiment of the present invention will be described with reference to Figures 7 to 12. In the first embodiment, the scene is one in which the pedestrian moves from a state in which the pedestrian and the three-dimensional object are located apart to a state in which the pedestrian moves close to the three-dimensional object, but in the second embodiment, the processing will be described using an example of a scene in which the three-dimensional object and the pedestrian are already close to each other at the time of detection by the camera device and the radar device. Note that the same parts and processing in the second embodiment and the first embodiment are given the same reference numerals, and the description thereof will not be repeated.

FIG. 7 shows a flowchart of the fusion program processing executed by the controller H3 in Example 2.

The initial steps S101 to S108 are the same as in Example 1, so a detailed explanation will be omitted.

Next, in process S201, it is determined whether the area, height, or width of the stationary three-dimensional object detected by the camera in process S106 is equal to or greater than a threshold value.

If it is determined that the area, height, or width of the stationary three-dimensional object detected by the camera is equal to or greater than the threshold (Process S201: Yes), the azimuth angle of the stationary three-dimensional object detected by the camera is calculated in Process S202. The azimuth angle is determined based on the camera installation position.

Next, in process S203, a dummy target is placed a certain distance away from the position of the stationary three-dimensional object detected by the camera in the azimuth direction obtained in process S202. This is equivalent to assuming that the camera is unable to detect a target in the blind spot of the stationary three-dimensional object, in other words, that the target may be in the camera occlusion area of the stationary three-dimensional object.

Next, in process S204, the dummy target set in process S203 is retained as a grouping object for the radar.

Next, step S109 is performed, and if the determination in step S109 is No, step S110 is performed. However, since steps S109 and S110 are the same as in Example 1, the explanation will be omitted.

On the other hand, if it is determined that the vehicle has passed a stationary three-dimensional object (process S109: Yes), process S111 is carried out, and then in process S205, the dummy target is deleted from the radar's grouping objects. Process S111 is the same as in Example 1, so a description thereof will be omitted.

Next, step S112 is carried out, but as this is the same as in Example 1, a detailed explanation will be omitted.

Next, in process S206, it is determined whether the grouping object obtained in process S112 is present near the dummy target.

If it is determined that the grouping object obtained in step S112 is present near the dummy target (step S206: Yes), the dummy target is removed from the radar's grouping objects in step S205, and the process ends.

On the other hand, if it is determined that the grouping object obtained in process S112 does not exist near the dummy target (process S206: No), the process ends.

A scene example for more specifically explaining the process of the second embodiment is shown in the figure. As an example, a scene in which a stationary solid object and a pedestrian are present at a nearby position at the corner of the intersection before the vehicle enters the intersection, and the pedestrian approaches the vehicle when the vehicle turns the intersection is explained in time series. FIG. 8 is an overhead view of the vehicle and its surrounding objects at a certain time _t3 . In FIG. 8, a stationary solid object O _OBJ and a pedestrian O _PED are present in an area where the detection area A _L of the radar device H11 and the detection area A _CAM of the camera device H12 overlap. However, since the pedestrian O _PED is present in the camera occlusion area of the stationary solid object O _OBJ , it is assumed that the camera device H12 cannot detect the pedestrian O _PED . In FIG. 8, if the stationary solid object O _OBJ can be identified as a stationary solid object by the camera, the coordinate origin is fixed and the X position, Y position, and width of the stationary solid object are recorded in process S106. The above-mentioned processing is performed when the camera can detect and identify a stationary three-dimensional object. In FIG. 8, the coordinates of the stationary three-dimensional object at the time just before the camera can no longer detect and identify it are stored as ( _X'OBJ ( _t3 ), _Y'OBJ ( _t3 )), its width as _WOBJ , and its height as _HOBJ .

9 is an overhead view of the vehicle and its surrounding objects at time _t3 , and in order to prevent the diagram from becoming too complicated, the pedestrian O _PED that has not been detected by the camera is removed, and a dummy target is placed in the camera occlusion area of the stationary three-dimensional object O _OBJ . The dummy target placement process will be described. First, in process S201, it is determined whether the area, height, or width of the stationary three-dimensional object is equal to or greater than a threshold value based on equations (2), (3), and (4). The threshold values for area, height, and width are S _TH , H _TH , and W _TH .

Next, if any of the formulas (2), (3), and (4) is satisfied, in step S202, the azimuth angle θ _OBJ (t ₃ ) of the stationary three-dimensional object O _OBJ with the camera installation position as the origin is calculated based on the formula (5).

Here, L _FCAM is the distance between the front end of the vehicle body and the camera.

Next, in process S203, a dummy target is placed at a position ( _X'DUM ( _t3 ), _Y'DUM ( _t3 )) a distance _L0 away from the position ( _X'OBJ ( _t3 ), _Y'OBJ ( _t3 )) of the stationary three-dimensional object in the direction of azimuth angle _θOBJ ( _t3 ). As mentioned above, radar grouping processing is a process in which nearby radar detection points are regarded as a single object, and the grouping range is generally determined as a circle. Therefore, _L0 is calculated from the width _WOBJ of the stationary three-dimensional object detected by the camera and the diameter _RDUM of the radar grouping range using equation (6).

Moreover, X' _DUM (t ₃ ) and Y' _DUM (t ₃ ) are calculated using equations (7) and (8), respectively.

Next, in process S204, it is assumed that the dummy target has been detected by the radar, and is treated as a grouping object for the radar.

Fig. 10 is an overhead view of the host vehicle and its surrounding objects at a certain time _t4 after a certain time has elapsed from time _t3 . In Fig. 10, a stationary three-dimensional object _OOBJ , a pedestrian _OPED , and a dummy target _ODUM are present in a detection area _AAL of the radar device H11 outside a detection area _ACAM of the camera device H12.

Fig. 11 shows the radar detection point cloud as viewed on the coordinates fixed in Fig. 10. In Fig. 11, in process S110, all radar detection points within a circle of diameter W _OBJ centered on the position (X' _OBJ (t ₃ ), Y' _OBJ (t ₃ )) of the stationary three-dimensional object recorded in process S106 are assigned an attribute of non-grouping target. Next, in process S112, grouping process is performed on radar detection points other than those with the non-grouping target attribute. Next, in process S206, if the grouping object obtained in process S112 exists within a circle of diameter R _DUM centered on the position (X' _DUM (t ₃ ), Y' _DUM (t ₃ )) of the dummy object obtained in process S203, it is determined that the dummy object actually exists, and in process S205, the dummy object is erased.

Fig. 12 is an overhead view of the vehicle and its surrounding objects at a certain time _t5 , a certain time after the time _t4 . In Fig. 12, a stationary three-dimensional object _OOBJ and a pedestrian _OPED are present in a detection area A- _L of the radar device H11 outside a detection area A- _CAM of the camera device H12. Fig. 12 shows an example of a scene in which it is determined in step S109 that the vehicle has passed the stationary three-dimensional object recorded in step S106, as in Fig. 6, and therefore a detailed description of the process will be omitted.

In the above explanation of Example 2, there is only one stationary three-dimensional object to avoid cluttering the diagram, but if there are multiple stationary three-dimensional objects, the processing of Example 2 is performed on all stationary three-dimensional objects detected by the camera.

In this embodiment, by assigning an attribute to stationary three-dimensional objects detected by the camera that indicates that they are not subject to grouping, and by assuming that the target exists in the camera occlusion area (the area near the object that makes it difficult for the millimeter wave radar to detect it) if the stationary three-dimensional object is large, it is possible to detect each separately (as different targets) in a radar detection area outside the camera detection area, without mistakenly grouping stationary three-dimensional objects and moving objects such as pedestrians nearby. Also, although only one radar device is described in this embodiment, multiple radar devices can also be used.

[Example 3]
Next, a third embodiment of the present invention will be described with reference to Figures 13 to 18. As a scene different from those of the first and second embodiments, a scene in which a moving object such as a pedestrian exists between a stationary three-dimensional object and the vehicle itself, and the stationary three-dimensional object cannot be detected by the camera, will be taken as an example to describe the processing. Note that the same parts and processing in the third embodiment and those in the first and second embodiments will be given the same reference numerals, and the description thereof will not be repeated.

FIG. 13 shows a flowchart of the fusion program processing executed by the controller H3 in Example 3.

The initial steps S101 to S104 are the same as in Example 1, so a detailed explanation will be omitted.

Next, in process S301, it is determined whether or not a pedestrian (or two-wheeled vehicle) is present in the camera detection results obtained in process S104.

If it is determined that a pedestrian (or two-wheeled vehicle) is present in the camera detection results (Process S301: Yes), the azimuth angle of the pedestrian (or two-wheeled vehicle) detected by the camera is calculated in Process S302. The azimuth angle is set to originate from the camera installation position.

Next, in process S303, a dummy target is placed a certain distance away from the position of the pedestrian (or motorcycle) detected by the camera in the azimuth direction obtained in process S302. This is equivalent to assuming that the camera is unable to detect a target in the blind spot of the pedestrian (or motorcycle), in other words, that the target may be in the camera occlusion area of the pedestrian (or motorcycle).

Next, in process S304, the dummy target placed in process S303 is retained as a grouping object for the radar.

Next, in S305, the coordinate origin is fixed and the position of the dummy target is stored.

Next, step S108 is performed, but since step S108 is the same as in Example 1, a detailed explanation will be omitted.

Next, in process S306, it is determined whether the vehicle has passed the dummy target obtained in process S305.

If the vehicle has not passed the dummy target (step S306: No), the radar detection points near the dummy target are excluded from grouping in step S307.

On the other hand, if it is determined that the vehicle has passed the dummy target (process S306: Yes), process S111 is carried out, and then process S205 is carried out. Process S111 is the same as in Example 1, and process S205 is the same as in Example 2, so a description thereof will be omitted.

On the other hand, if it is determined in step S301 that no pedestrian (or two-wheeled vehicle) is present in the camera detection results (step S301: No), step S107 is performed. Step S107 is the same as in Example 1, so a description thereof will be omitted.

The subsequent steps S206 and S205 are the same as those in Example 2, so the explanation will be omitted.

This completes the process.

A scene example for more specifically explaining the processing of the third embodiment is shown in the figure. As an example, a scene in which a pedestrian moves in the same direction as the traveling direction of the vehicle on the sidewalk in front of the vehicle on the left before the vehicle enters an intersection, and a solid object at the corner of the intersection always exists in the camera occlusion area of the pedestrian until it leaves the detection area of the camera is explained in chronological order. FIG. 14 is an overhead view of the vehicle and its surrounding objects at a certain time _t6 . In FIG. 14, a stationary solid object O _OBJ and a pedestrian O _PED exist in the area where the detection area A _L of the radar device H11 and the detection area A _CAM of the camera device H12 overlap. However, since the stationary solid object O _OBJ exists in the camera occlusion area of the pedestrian O _PED , the stationary solid object O _OBJ cannot be detected by the camera device H12.

Fig. 15 is an overhead view of the vehicle and its surrounding objects at time _t6 , and in order to prevent the diagram from becoming too complicated, stationary three-dimensional objects O _OBJ that cannot be detected by the camera are removed, and dummy targets are placed in the camera occlusion area of the pedestrian O _PED . The dummy target placement process will now be described. In Fig. 15, if the pedestrian O _PED can be identified as a pedestrian by the camera, in process S302, the azimuth angle θ _PED ( _t6 ) of the pedestrian O _PED with the camera installation position as the origin is calculated based on equation (9).

Next, in process S303, a dummy target is placed at a position ( _X'DUM ( _t6 ), _Y'DUM ( _t6 )) a distance L _P away in the azimuth angle θ _PED ( _t6 ) direction from the pedestrian position ( _X'PED ( _t6 ), _Y'PED ( _t6 )). L _P is calculated from the width W _PED of the pedestrian detected by the camera and the diameter R _DUM of the radar grouping range using equation (10), as in process S203 of the second embodiment.

Moreover, X' _DUM (t ₆ ) and Y' _DUM (t ₆ ) are calculated by the equations (11) and (12), respectively.

Next, in step S304, it is assumed that the dummy target has been detected by the radar, and is treated as a grouping object for the radar.

Next, in process S305, the coordinate origin is fixed, and the X and Y positions of the dummy object are recorded. The above process is performed when the camera is able to detect and identify pedestrians, and in Fig. 15, the coordinates of the dummy object just before the camera is no longer able to detect and identify pedestrians are stored as ( _X'DUM ( _t6 ), _Y'DUM ( _t6 )).

Fig. 16 is an overhead view of the host vehicle and its surrounding objects at a certain time _t7 , which is a certain time after the time _t6 . In Fig. 16, a stationary three-dimensional object _OOBJ , a pedestrian _OPED , and a dummy target _ODUM are present in a detection area _AAL of the radar device H11 outside a detection area _ACAM of the camera device H12.

Fig. 17 shows the radar detection point cloud as viewed on the coordinates fixed in Fig. 16. In Fig. 17, in process S307, all radar detection points within a circle of diameter R _DUM centered on the position (X' _DUM (t ₆ ), Y' _DUM (t ₆ )) of the dummy target recorded in process S305 are assigned an attribute of non-grouping. Next, in process S112, grouping process is performed on radar detection points other than those with the non-grouping attribute. Next, in process S206, if the grouping object obtained in process S112 exists within the circle of diameter R _DUM centered on the position (X' _DUM (t ₆ ), Y' _DUM (t ₆ )) of the dummy target obtained in process S305, it is determined that the dummy target actually exists, and the dummy target is erased in process S205.

Fig. 18 is an overhead view of the host vehicle and its surrounding objects at a certain time _t8 after a certain time has elapsed from time _t7 . In Fig. 18, a stationary three-dimensional object _OOBJ and a pedestrian _OPED are present in a detection area _AAL of the radar device H11 outside a detection area _ACAM of the camera device H12. Fig. 18 shows an example of a scene in which it is determined in step S306 that the host vehicle has passed the dummy object recorded in step S305, as in Fig. 6, and therefore a detailed description of the process will be omitted.

In the above explanation of Example 3, there is only one stationary object to avoid cluttering the diagram, but if there are multiple stationary objects, the processing of Example 3 is performed on all stationary objects detected by the camera.

In this embodiment, if a moving object such as a pedestrian is present between the vehicle and a stationary object, and the stationary object cannot be detected by the camera due to camera occlusion by the pedestrian, etc., a dummy target is placed in the camera occlusion area of the pedestrian, etc. (area near the moving object that is blocked by the moving object), and this is treated as a stationary object and an attribute is given to the stationary object that it is not subject to grouping. This makes it possible to detect the stationary object separately (as different targets) in the radar detection area outside the camera detection area, without erroneously grouping the stationary object with a moving object such as a pedestrian nearby. Also, although only one radar device is described in this embodiment, multiple radar devices can also be used.

As described above, the controller (vehicle control device) H3 of this embodiment has a camera that recognizes three-dimensional objects around the vehicle, and a millimeter wave radar that recognizes targets in the vicinity of the vehicle, and sets the three-dimensional object as an independent target based on the information of the three-dimensional object recognized by the camera (by giving the three-dimensional object an attribute that is not subject to grouping (by the millimeter wave radar)), and when the millimeter wave radar recognizes the three-dimensional object and other targets as the surrounding targets, it recognizes the three-dimensional object as a target different from the other targets.

In addition, a first virtual target (dummy target) is set in an area near the three-dimensional object where the millimeter wave radar has difficulty detecting the three-dimensional object, and (by giving the three-dimensional object an attribute that is not subject to grouping (by the millimeter wave radar)) when the first virtual target is detected by the millimeter wave radar due to the movement of the vehicle, the three-dimensional object and the first virtual target are recognized as different targets.

In addition, if at least one of the area, width, and height of the three-dimensional object is equal to or greater than a predetermined value (threshold), the first virtual target is placed, and if it is less than the predetermined value (threshold), the first virtual target is not placed.

In addition, when a moving target is recognized by the camera, a second virtual target (dummy target) is placed in the vicinity of the moving target that is blocked by the moving target, and (by giving the second virtual target an attribute that is not subject to grouping (by the millimeter wave radar)) when the second virtual target is detected by the millimeter wave radar due to the movement of the host vehicle, the moving target and the second virtual target are recognized as different targets.

In other words, this embodiment targets a vehicle equipped with an external information acquisition device that is composed of a millimeter wave radar and a camera and acquires information on the vehicle's traveling position and traveling environment, and in a scene where surrounding obstacles present in front of and to the sides of the vehicle are detected, any target detected by the camera as an arbitrary three-dimensional object is excluded from grouping, and fusion is performed based on the grouping results of the radar detection points other than those that are not grouped (embodiment 1), and in addition to the above-mentioned processing, a dummy target is placed in the camera occlusion area of the arbitrary three-dimensional object according to the height, width, area, etc. of the target detected by the camera as an arbitrary three-dimensional object, and fusion is performed (embodiment 2), and a dummy target is placed in the camera occlusion area of a target detected by the camera as a pedestrian or two-wheeled vehicle, and the dummy target is excluded from grouping, and fusion is performed based on the grouping results of the radar detection points other than those that are not grouped (embodiment 3).By these means, a vehicle control device capable of highly accurate sensing is provided.

According to this embodiment, a vehicle control device can be provided that can suppress erroneous grouping of millimeter wave radar.

The present invention is not limited to the above-mentioned embodiment, but includes various modifications. For example, the above-mentioned embodiment is described in detail to explain the present invention in an easy-to-understand manner, and is not necessarily limited to having all of the configurations described.

Furthermore, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in hardware, in part or in whole, for example by designing them as integrated circuits. The above-mentioned configurations, functions, etc. may be realized in software, by a processor interpreting and executing a program that realizes each function. Information on the programs, tables, files, etc. that realize each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

H1 External information acquisition unit H11 Radar device (millimeter wave radar)
H12 Camera equipment (camera)
H2 Vehicle information acquisition unit H3 Controller (vehicle control device)
H4 Vehicle control unit

Claims

The vehicle has a camera that recognizes three-dimensional objects around the vehicle and a millimeter wave radar that recognizes objects around the vehicle,
A vehicle control device characterized in that the three-dimensional object is set as an independent target based on information of the three-dimensional object recognized by the camera, and when the millimeter wave radar recognizes the three-dimensional object and other targets as the surrounding targets, the vehicle control device recognizes the three-dimensional object as a target different from the other targets.
The vehicle control device according to claim 1,
a first virtual target is set in a region near the three-dimensional object where the three-dimensional object makes it difficult for the millimeter wave radar to detect the first virtual target;
a vehicle control device that recognizes the three-dimensional object and the first virtual object as different objects when the first virtual object is detected by the millimeter wave radar due to movement of the host vehicle.
The vehicle control device according to claim 1,
A vehicle control device characterized in that the first virtual target is placed when at least one of the area, width, and height of the three-dimensional object is equal to or greater than a predetermined value, and the first virtual target is not placed when the area, width, and height are smaller than the predetermined value.
The vehicle control device according to claim 1,
When the moving target is recognized by the camera, a second virtual target is disposed in a vicinity of the moving target that is blocked by the moving target;
When the second virtual target is detected by the millimeter wave radar due to movement of the host vehicle, the moving target and the second virtual target are recognized as different targets.
The vehicle control device according to claim 1,
A vehicle control device characterized in that the three-dimensional object is recognized as a target object different from the other targets by giving the three-dimensional object an attribute that is not subject to grouping.
The vehicle control device according to claim 2,
A vehicle control device comprising: a vehicle control system that recognizes the three-dimensional object and the first virtual target as different targets by assigning an attribute that is not subject to grouping to the three-dimensional object.
The vehicle control device according to claim 4,
A vehicle control device comprising: a vehicle control unit that recognizes the moving target and the second virtual target as different targets by assigning an attribute of being excluded from grouping to the second virtual target.