JP6962726B2 - Track recognition device - Google Patents

Description

The present disclosure relates to a technique for recognizing the track of a vehicle.

For automatic driving of a vehicle, it is required to recognize the running path of the own vehicle, for example, the traveling lane of the own vehicle if it is a road. The track recognition is performed based on, for example, detecting a white line which is a division line of the track in an captured image obtained by an imaging device such as a camera. When the track of the own vehicle is recognized, actions necessary for automatic driving such as detection of obstacles on the track, automatic operation of brakes, and estimation of lane deviation can be performed. Therefore, in order to realize more stable automatic driving, a technique for recognizing the shape of the track from the vicinity of the current position to a distant position with high accuracy is desired. In general, the recognition accuracy of a distant track shape is lower than that of a nearby track shape. Therefore, in the captured image, first, the white line in the nearby runway is detected to recognize the shape of the nearby runway, and then in the distant region (generally, the upper region of the captured image) in the captured image, the white line exists in the distance. A method has been proposed in which a method of narrowing down based on a nearby track shape that recognizes an expected region, detecting a white line in the narrowed-down area, and recognizing a distant track shape based on the detected white line (see Patent Document 1). ).

Japanese Unexamined Patent Publication No. 2013-196341

In the method of Patent Document 1, when the white line does not exist in the region near the own vehicle, such as when the own vehicle is traveling at an intersection, the shape of the running road in the vicinity cannot be recognized. Therefore, there is a problem that it is not possible to narrow down the area where the white line is expected to exist in the distant area, and it is not possible to accurately recognize the distant track shape. Therefore, there is a demand for a technique capable of accurately recognizing a distant track shape even when the track shape in the vicinity of the vehicle cannot be recognized.

The present disclosure has been made to solve at least a part of the above-mentioned problems, and can be realized in the following forms.

According to one embodiment of the present disclosure, a track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle is provided. This track recognition device; has a neighborhood shape calculation unit (101) that repeatedly calculates a nearby track shape, which is a track shape in the vicinity of the vehicle, based on an image captured repeatedly by the image pickup device (21) mounted on the vehicle. The processing areas (Er1 to Er4, El1 to El4) in each captured image are set based on the calculated neighborhood track shape, and the distant track shape, which is the distant track shape of the vehicle in the processing area, is repeatedly calculated. With the distance shape calculation unit (102); with the neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in time series; and with the neighborhood shape storage unit (103); stores the calculated distance track shape as a history in time series. The far shape storage unit (104) and; Is used to set the processing area.

According to the track recognition device of this form, when the near track shape cannot be calculated using the captured image, the processing area is determined by using at least one of the near track shape history and the far track shape history. Since it is set, it is possible to accurately recognize the distant track shape even when the track shape near the vehicle cannot be recognized.

The present disclosure can also be realized in various forms other than the track recognition device. For example, it can be realized in the form of a vehicle equipped with a track recognition device, a track recognition method, a computer program for recognizing a track, a storage medium that stores such a computer program, and the like.

The block diagram which shows the schematic structure of the track recognition apparatus as one Embodiment of the image processing apparatus of this disclosure. The flowchart which shows the procedure of the neighborhood track shape calculation processing in 1st Embodiment. Explanatory drawing which shows an example of the captured image. The flowchart which shows the procedure of the distant track shape calculation process in 1st Embodiment. Explanatory drawing which expands and shows an example of a distant track area. Explanatory drawing which shows an example of the captured image. The flowchart which shows the procedure of the target passing point setting process in 1st Embodiment. The flowchart which shows the procedure of the main part of the distant track shape calculation process in 2nd Embodiment. The flowchart which shows the procedure of the target passing point setting process in 3rd Embodiment. The block diagram which shows the schematic structure of the track recognition apparatus of 4th Embodiment. The flowchart which shows the procedure of the main part of the distant track shape calculation process in 4th Embodiment. The flowchart which shows the procedure of the target passing point setting process in 5th Embodiment. The flowchart which shows the procedure of the main part of the distant track shape calculation process in 6th Embodiment. The explanatory view which shows an example of the setting contents of the evaluation value table in 6th Embodiment. The explanatory view which shows an example of the setting contents of the integrated likelihood table in the 6th Embodiment. The block diagram which shows the schematic structure of the track recognition apparatus of 7th Embodiment. The flowchart which shows the procedure of execution determination processing in 7th Embodiment. The block diagram which shows the schematic structure of the track recognition apparatus of 8th Embodiment. The explanatory view which shows an example of the processing area set in 8th Embodiment. The block diagram which shows the schematic structure of the track recognition apparatus in 9th Embodiment. The block diagram which shows the schematic structure of the track recognition apparatus in 10th Embodiment. The block diagram which shows the schematic structure of the track recognition apparatus in 11th Embodiment. The flowchart which shows the procedure of the main part of the distant track shape calculation process in eleventh embodiment. The block diagram which shows the schematic structure of the track recognition apparatus in 12th Embodiment. The flowchart which shows the procedure of the main part of the distant track shape calculation process in 12th Embodiment. An explanatory view schematically showing a measurement distance to a pole when there is no change in the posture of the own vehicle in the twelfth embodiment. The explanatory view which shows typically the state of measuring the distance to a pole when a nose down occurs in a twelfth embodiment based on a captured image. An explanatory view schematically showing how the distance to the pole when a nose down occurs in the twelfth embodiment is measured based on the measurement result of the millimeter wave radar. The block diagram which shows the schematic structure of the track recognition apparatus in 13th Embodiment. The flowchart which shows the procedure of the pattern matching processing in 14th Embodiment. The block diagram which shows the schematic structure of the track recognition apparatus in 15th Embodiment. The flowchart which shows the procedure of the main part of the distant track shape calculation process in 15th Embodiment. Explanatory drawing which shows an example of the processing area set in 15th Embodiment. The block diagram which shows the schematic structure of the track recognition apparatus in 16th Embodiment. The flowchart which shows the procedure of the main part of the distant track shape calculation process in 16th Embodiment. Explanatory drawing which shows the white line candidate determined by step S805 and the score value thereof. The block diagram which shows the schematic structure of the track recognition apparatus in 17th Embodiment. The flowchart which shows the procedure of the main part of the distant track shape calculation process in 17th Embodiment. The explanatory view which shows the determination example of the normality of the white line candidate in 17th Embodiment. In the eighteenth embodiment, the flowchart which shows the procedure of the process which determines the white line candidate.

A. First Embodiment:
A1. Device configuration:
The track recognition device 100 of the present embodiment shown in FIG. 1 is mounted on a vehicle (not shown) and recognizes the track of such a vehicle. In the present embodiment, the vehicle equipped with the track recognition device 100 is also referred to as "own vehicle". The own vehicle is a vehicle that can be driven automatically. The own vehicle may be either a vehicle that switches between automatic driving and manual driving (driving by the driver) and a vehicle that executes only automatic driving of these two types of driving. The track recognition device 100 is composed of an ECU (Electronic Control Unit) 10 including a CPU and a memory (not shown). The track recognition device 100 is also electrically connected to an image pickup unit 21, a vehicle speed sensor 22, a yaw rate sensor 23, and a vehicle control unit 31 mounted on the vehicle.

The image pickup unit 21 is composed of two cameras (so-called stereo cameras) arranged in the horizontal direction. Each camera is provided with a condensing lens and a light receiving element, and images the front of the own vehicle to obtain an captured image. When the ignition of the own vehicle is turned on, the imaging unit 21 repeatedly acquires the captured image. The vehicle speed sensor 22 measures the moving speed of the own vehicle. The vehicle speed sensor 22 repeatedly detects the vehicle speed when the ignition of the own vehicle is turned on. The yaw rate sensor 23 detects the yaw rate (rotation angular velocity) of the own vehicle. The yaw rate sensor 23 repeatedly detects the yaw rate when the ignition of the own vehicle is turned on. The vehicle control unit 31 controls the movement of the vehicle. The vehicle control unit 31 includes a plurality of ECUs such as an ECU that controls the operation of a drive mechanism such as an internal combustion engine or an electric motor, an ECU that controls an electronically controlled brake system (ECB), and an ECU that controls the steering angle of wheels. Consists of.

By executing the control program stored in the memory by the CPU included in the track recognition device 100, the track recognition device 100 has a neighborhood shape calculation unit 101, a distance shape calculation unit 102, a neighborhood shape storage unit 103, and a distance shape storage unit. It functions as 104 and the target passing point setting unit 105.

The neighborhood shape calculation unit 101 executes the nearby track shape identification process described later, and based on the image captured repeatedly obtained by the imaging unit 21, the shape of the track on which the own vehicle travels and the shape of the track in the vicinity of the own vehicle. (Hereinafter referred to as "neighboring track shape") is repeatedly calculated. The “neighborhood of the own vehicle” means a region within a predetermined distance from the own vehicle in the region ahead of the imaging unit 21 in the traveling direction. In the present embodiment, the above-mentioned predetermined distance that defines the vicinity of the own vehicle is 7 m (meter). The distance is not limited to 7 m and may be any distance. In the present embodiment, the "runway shape" is defined by the width of the runway, the positions of both ends in the width direction of the runway, and the curvature (radius of curvature) of the runway. Therefore, "calculating the shape of the nearby track" means the width of the track on which the own vehicle travels and the track in the vicinity of the own vehicle (hereinafter referred to as "near track") and both ends in the width direction of the track. It means that the position of and the curvature (radius of curvature) of the track are calculated respectively.

The distant shape calculation unit 102 repeatedly calculates the shape of the distant track (hereinafter, referred to as "distant track shape"), which is the track scheduled to be traveled by the own vehicle, by executing the distant track shape specifying process described later. do. The detailed procedure will be described later, but to explain the outline, the distance shape calculation unit 102 is the target of processing for calculating the distance track shape in the captured image based on the neighborhood track shape calculated by the neighborhood shape calculation unit 101. A region (hereinafter, simply referred to as a “processing region”) is specified, and a distant track shape is calculated within the processing region. "Calculating the shape of a distant track" means the width of the distant track (hereinafter referred to as "distant track") that the own vehicle plans to travel, the positions of both ends in the width direction of the distant track, and the distance. It means to calculate the curvature (radius of curvature) of the track.

The neighborhood shape storage unit 103 stores the neighborhood track shape calculated by the neighborhood shape calculation unit 101 as a history in chronological order. Specifically, each parameter indicating the calculated neighborhood track shape is stored together with the calculated time. The distant shape storage unit 104 stores the distant track shape calculated by the distant shape calculation unit 102 as a history in chronological order. Specifically, each parameter indicating the calculated distance track shape is stored together with the calculated time.

The target passing point setting unit 105 sets the target passing point of the own vehicle by executing the target passing point setting process described later. This target passing point is used as a target passing point when the own vehicle automatically drives. Basically, the target passing point is set based on the shape of the nearby track, but when the calculation accuracy of the shape of the nearby track is low, it is exceptionally set based on the shape of the far track. This is because the calculation accuracy of the near track shape is basically higher than the calculation accuracy of the far track shape.

A2. Neighborhood track shape calculation process:
The neighborhood track shape calculation process shown in FIG. 2 is periodically and repeatedly executed when the ignition of the own vehicle is turned on. The neighborhood shape calculation unit 101 extracts the edge of the neighborhood track in the obtained captured image, more specifically, the edge of the lane marking that divides the neighborhood track (step S105).

The neighborhood shape calculation unit 101 performs a well-known Hough transform on the extracted edge to detect a straight line (step S110), and calculates a straight line (white line) candidate straight line from the detected straight lines (step). S115). Specifically, from the detected straight lines, the one with the largest number of votes for the Hough transform is calculated as a white line candidate.

As shown in FIG. 3, the captured image F1 shows the lane Ln1 and the oncoming lane Ln2, which are the lanes of the own vehicle. A region including a distant track located above the captured image F1 (hereinafter referred to as a "far track region") Af1 and a region including a nearby track located below the captured image F1 (hereinafter referred to as a "near track"). The edges of the lane markings Lr1 and Ll1 of the lane Ln1 are extracted from the An1), and the straight lines representing the respective lane markings Lr1 and Ll1 are calculated as white line candidates.

As shown in FIG. 2, the neighborhood shape calculation unit 101 narrows down the white line candidates (step S120). Specifically, for example, the ratio of the contrast of the white line candidate to the road surface around the white line candidate in the captured image is higher than a predetermined threshold value, or the difference between the brightness of the white line candidate portion and the brightness around the white line candidate portion is a predetermined threshold value. Limit the white line candidates to those that are larger than this. In addition, it may be narrowed down in consideration of various features such as line thickness and total extension distance. Then, one white line candidate closest to the center of the vehicle in the left-right direction is selected.

The neighborhood shape calculation unit 101 determines whether or not the white line candidates have been narrowed down in step S120 (step S125), and when it is determined that the white line candidates have not been narrowed down (step S125: NO), the above-mentioned Return to step S105. On the other hand, when it is determined that there is a white line candidate (step S125: YES), the neighborhood shape calculation unit 101 converts the edge points constituting the white line candidate narrowed down in S120 into plane coordinates (S130). ). Plane coordinates are coordinates when the road surface is assumed to be a plane. Here, the edge points on the captured image are coordinate-converted based on the position / orientation information (hereinafter referred to as “imaging parameter”) of the cameras constituting the imaging device obtained by calibration in advance. By using the edge points as the information on the plane coordinates in this way, it becomes possible to easily combine the edge points with the coordinate information of the edge points based on the past captured images.

The neighborhood shape calculation unit 101 calculates the neighborhood track shape from the plane coordinates calculated in step S130 (step S135). As described above, the shape of the nearby runway means the width of the runway, the positions of both ends in the width direction of the runway, and the curvature (radius of curvature) of the runway with respect to the nearby runway. As a method for calculating each of these parameters, a well-known method such as the method described in JP2013-196341A may be adopted.

The neighborhood shape calculation unit 101 outputs the neighborhood track shape calculated in step S135 to the distance shape calculation unit 102 and the target passing point setting unit 105, and stores the calculated time in the neighborhood shape storage unit 103 (step). S140).

A3. Distance track shape calculation process:
The neighborhood track shape calculation process shown in FIG. 4 is periodically and repeatedly executed when the ignition of the own vehicle is turned on. The distant shape calculation unit 102 determines whether or not there is an input of a nearby track shape at a predetermined time (step S205). As described above, when the step S140 of the neighborhood track shape calculation process is executed, it is determined that there is an input. On the other hand, if it is determined that there is no white line candidate as a result of narrowing down the white line candidates in step S125 of the neighborhood track shape calculation process, step S140 is not executed and the neighborhood track shape is calculated as a distant shape at a predetermined time. It is not input to the unit 102.

When it is determined that there is an input of the near-runway shape (step S205: YES), the far-field shape calculation unit 102 uses the input near-runway shape to extend the near-runner (lane marking) to form the far-field shape. Is estimated (step S210). The distant shape calculation unit 102 sets the processing area based on the estimated distant track shape (step S215). The processing area means an area of the distant track area that is narrowed down as a target area for extracting the edge point of the lane marking when calculating (specifying) the distant track shape. In other words, it means an area to be processed to extract the edge point of the lane marking and calculate the distant track shape.

In FIG. 5, the distant track region Af1 in the captured image F1 of FIG. 3 is enlarged and schematically shown. In FIG. 5, two processing regions Er1 and El1 are set in the far track region Af1 based on the distance track shape estimated in step S210. The processing areas Er1 and El1 are also shown in FIG. Hereinafter, a method of setting these two processing areas Er1 and El1 will be described. In the lane marking Lr1, the point Pr1 on the foremost side and the point Pr11 on the front side by a predetermined distance d1 from the vanishing point P0 are defined. The vanishing point P0 is obtained as a point where the two lane markings Lr1 and Ll1 intersect. Next, for the two defined points Pr1 and Pr11, two sides (short sides) having a width of a predetermined distance in the horizontal direction are obtained with the corresponding points as the centers, and the ends of the sides are demarcated. Two sides (long sides) are defined by connecting along Lr1. The area surrounded by the two short sides and the two long sides defined in this way is set as the processing area Er1. Similarly, on the lane marking Ll1, the point Pl1 on the foremost side and the point Pl11 on the front side by a predetermined distance d1 from the vanishing point P0 are defined. Next, for the two defined points Pl1 and Pl11, two sides (short sides) having a width of a predetermined distance in the horizontal direction are obtained with the corresponding points as the centers, and the ends of the sides are demarcated. Two sides (long sides) are defined by connecting along Ll1. The area surrounded by the two short sides and the two long sides defined in this way is set as the processing area El1. As shown in FIGS. 3 and 5, when the runway (lane Ln1) is straight and the lane markings Lr1 and Ll1 on both sides are both straight, the shapes of the processing regions Er1 and El1 are all captured images. It becomes a trapezoidal shape.

As shown in FIG. 4, after setting the processing area (after step S215), the distant shape calculation unit 102 extracts an edge that divides the distant track in the processing area (step S220). The Hough transform (step S225), white line candidate calculation (step S230), candidate narrowing (step S235), determination of the presence or absence of candidates (step S240), and edge point plane transform (step S245), which are executed thereafter, are all described above. Since it is the same as the steps S110 to S130 of the near-runway shape calculation process of the above, detailed description thereof will be omitted. If it is determined in step S240 that there are no white line candidates (step S240: NO), the distant track shape calculation process ends.

After executing step S245, the distant shape calculation unit 102 calculates the distant track shape from the plane coordinates calculated in step S240 (step S250). Since such a calculation method is the same as step S135 of the above-mentioned neighborhood track shape calculation process, detailed description thereof will be omitted.

The distant shape calculation unit 102 outputs the distant track shape calculated in step S250 to the target passing point setting unit 105, and stores the calculated time in the distant shape storage unit 104 (step S255).

In step S205 described above, when it is determined that there is no input of the nearby track shape at a predetermined time (step S205: NO), the distant track shape calculation unit 102 stores the distant track shape calculated at the previous timing as the distant shape. It is determined whether or not it is in the unit 104 (step S270).

The captured image F2 shown in FIG. 6 is an example of an image obtained when the own vehicle is stopped in front of the intersection CR. In the captured image F2, the lane markings Lr2 and Ll2 are slightly reflected at the lower end (front end) of the nearby track area An2, but before the pedestrian crossing Pd1 captured below the nearby track area An2. , The pedestrian crossing is not shown. Therefore, in this case, the white line candidates are not narrowed down, the near track shape is not input to the far shape calculation unit 102, and it is determined that the near track shape is not input at a predetermined time in step S205 described above. In the captured image F2, the pedestrian crossing Pd2 existing across the intersection CR and the continuation of the lane markings Lr2 and Ll2 are shown in the distant track region Af2.

As shown in FIG. 4, when it is determined that the distant track shape calculated at the previous timing is not in the distant shape storage unit 104 (step S270: NO), the distant track shape calculation process ends. On the other hand, when it is determined that there is a history of the distant track shape (step S270: YES), the distant shape calculation unit 102 includes the previous distant track shape stored in the distant shape storage unit 104 and the imaging parameters. , The processing area is set in the distant track area based on the information indicating the movement of the vehicle (step S275). The information indicating the movement of the vehicle means, for example, the moving speed of the own vehicle obtained from the vehicle speed sensor 22, the yaw rate obtained from the yaw rate sensor 23, the steering angle obtained from the vehicle control unit 31, and the like. In step S275, the distant track shape calculation unit 102 corrects the previously calculated distant track shape from the imaging parameters and the movement of the vehicle, and estimates the distant track shape this time (in the captured image this time). Then, the lane marking in the distant track region is specified from the estimated distant track shape, and the processing area is set based on the lane marking. Since the method of setting the processing area based on the lane marking is the same as the method described in step S215 described above, detailed description thereof will be omitted. In the example of FIG. 6, two processing areas Er2 and El2 are set in the far track area Af2. After the execution of step S275, the above-mentioned steps S220 to S255 are executed.

A4. Target passing point setting process:
The target passing point setting process shown in FIG. 7 is periodically and repeatedly executed when the ignition of the own vehicle is turned on. The target passing point setting unit 105 determines whether or not there is an input of a nearby track shape at a predetermined time (step S305). As described above, the neighborhood track shape calculation process is periodically executed, and when step S140 is executed, the neighborhood track shape is periodically input to the target passing point setting unit 105. On the other hand, when the nearby track shape is not calculated, the nearby track shape is not input to the target passing point setting unit 105 at a predetermined time.

When it is determined that there is an input of the nearby track shape (step S305: YES), the target passing point setting unit 105 sets the target passing point based on the nearby track shape (step S310). Specifically, the target passing point setting unit 105 sets the center point of the nearby running track in the nearby running track region as the target passing point. For example, in the example of FIG. 3, the horizontal center of the lane markings Lr1 and Ll1 at both ends of the lane Ln1 and the central point PT1 in the depth direction in the nearby track region An1 are set as the target passing points.

As shown in FIG. 7, when it is determined in step S305 described above that there is no input of the nearby track shape (step S305: NO), whether or not the target passing point setting unit 105 has input of the far track shape. Is determined (step S315). When step S255 of the above-mentioned far track shape calculation process is executed, the far track shape is input to the target passing point setting unit 105. On the other hand, when step S255 is not executed, that is, when it is determined that there is no input of the nearby track shape (step S205: NO) and there is no far track shape calculated at the previous timing (step S270). : NO), the distant track shape is not input to the target passing point setting unit 105.

When it is determined that there is an input of the distant track shape (step S315: YES), the target passing point setting unit 105 sets the target passing point based on the distant track shape (step S330). Specifically, the target passing point setting unit 105 sets the frontmost point as the target passing point, which is the center in the horizontal direction of the far running track in the far running track region. For example, in the example of FIG. 6, the foremost point PT2 in the far track region Af2, which is the center along the horizontal direction of the lane markings Lr2 and Ll2, is set as the target passing point.

After the execution of the above-mentioned step S310, after the execution of the step S330, or when it is determined in the step S315 that there is no input of the distant track shape (step S315: NO), the target passing point setting process ends.

According to the track recognition device 100 of the first embodiment described above, when the near track shape cannot be calculated using the captured image, the processing area is set using the previous far track shape, so that the vehicle Even when the track shape in the vicinity of is not recognizable, the track shape in the distance can be recognized accurately. In addition, when the nearby track shape is output, the target passing point is set based on the nearby track shape, so that the target passing point can be set accurately. Further, when the near track shape is not output, the target passing point is set based on the far track shape, so that it is possible to avoid not being able to set the target passing point even when the near track shape is not output. Even in such a case, the target passing point can be set based on the track shape recognized with high accuracy. Therefore, the target passing point can be set with high accuracy.

B. Second embodiment:
Since the device of the track recognition device of the second embodiment has the same device configuration as the track recognition device 100 of the first embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. .. As shown in FIG. 8, the track shape calculation process of the second embodiment is different from the track shape calculation process of the first embodiment shown in FIG. 4 in that step S209 is additionally executed, and the other procedures are the same. Is.

If it is determined in step S205 that there is no input of the nearby track shape (step S205: YES), the far shape calculation unit 102 checks whether the input of the nearby track shape is continuous beyond the predetermined threshold time. Whether or not it is determined (step S209). In the present embodiment, 1 second is set as a predetermined period (predetermined time). The time is not limited to 1 second, and may be set to an arbitrary time longer than the acquisition interval of the captured image. When it is determined that the input of the nearby track shape is not continuous beyond the predetermined threshold value (step S209: YES), the far track shape calculation process ends. On the other hand, when it is determined that the input of the nearby track shape is not continuous (that is, there is an input) beyond the predetermined threshold value (step S209: NO), the above-mentioned step S270 is performed. Will be executed. In this way, when the input of the nearby track shape is not continuous beyond the predetermined threshold time, the far track shape calculation process is terminated by the calculation of the far track shape using the history of the far track shape. This is to prevent the calculation accuracy of the distant track shape from being lowered due to the continuation for a long time. The threshold time in step S209 corresponds to the subordinate concept of the first period in the claims.

The track recognition device of the second embodiment described above has the same effect as the track recognition device 100 of the first embodiment. In addition, when the nearby track shape is not continuously calculated beyond a predetermined period, the calculation of the far track shape is stopped, so that it is possible to suppress a decrease in the calculation accuracy of the far track shape. Therefore, it is possible to prevent the long-distance track shape calculated with low accuracy from being stored as a history, and it is possible to suppress a decrease in the specific accuracy of the processing area.

C. Third Embodiment:
Since the device of the track recognition device of the third embodiment has the same device configuration as that of the track recognition device 100 of the first embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. .. As shown in FIG. 9, the target passing point setting process of the third embodiment is different from the target passing point setting process of the first embodiment shown in FIG. 7 in that steps S319 and S320 are additionally executed. The procedure is the same.

As shown in FIG. 9, when it is determined in step S315 that there is no input of the distant track shape (step S315: NO), the target passing point setting unit 105 sets a predetermined threshold time (predetermined threshold time). It is determined whether or not there is an input of the distant track shape beyond that (step S319). The above-mentioned predetermined threshold time (predetermined threshold time) is set to 2 seconds in the present embodiment. When it is determined that there is no input of the distant track shape beyond the predetermined threshold time (step S319: YES), the target passing point setting process ends. On the other hand, when it is determined that there is no input of the distant track shape beyond the predetermined threshold time (that is, there is an input within the threshold time) (step S319: NO), the target passing point. The setting unit 105 calculates the current distant track shape based on the history of the distant track shape stored in the distant shape storage unit 104 (step S320). In the present embodiment, this step S320 is executed by replacing the "previous far track shape" with the "latest far track shape" in step S275 of the far track shape calculation process in the first embodiment shown in FIG. It means a process of executing steps S215 to S255. It should be noted that, not limited to the "latest far track shape", using the far track shapes for a predetermined number of times counting from the newest one, for example, these far track shapes are averaged, the process of step S275 is performed, and then the step. S220 to S225 may be executed. That is, the current distant track shape may be calculated by an arbitrary method using the history of the distant track shape. Further, instead of the history of the distant track shape, or in addition to the history of the distant track shape, the history of the near track shape may be used to calculate the current distant track shape. For example, a lane marking line specified by the latest nearby track shape may be simply extended (extrapolated) to obtain a distant track shape, and such a distant track shape may be used as the current distant track shape. Alternatively, step S210 is executed by replacing the "(input) neighborhood track shape" in step S210 of the distance track shape calculation process shown in FIG. 4 with the "latest neighborhood track shape", and then steps S220 to S255 are executed. Then, the shape of the distant track this time may be calculated. Furthermore, the average value of the distant track shape obtained by simply extending (extrapolating) the lane marking specified by the latest nearby track shape and the past distant track shape is obtained, and the average value is shown in FIG. Step S275 may be executed in place of the "previous far track shape" in step S275 of the far track shape calculation process, and then steps S220 to S255 may be executed to calculate the current far track shape. That is, in general, any method of calculating the long-distance track shape this time using at least one of the history of the near-run track shape and the history of the far-run track shape may be executed in step S320. After the execution of step S320, the above-mentioned step S330 is executed. The threshold time in step S320 corresponds to the subordinate concept of the second period in the claims.

The track recognition device of the third embodiment described above has the same effect as the track recognition device 100 of the first embodiment. In addition, when the distant track shape is not input beyond the threshold time, that is, the distant track shape is not calculated beyond the threshold time, at least one of the near track shape history and the distant track shape history. Since the distant track shape of this time is calculated using the above, it is possible to prevent the distant track shape from being frequently calculated. In addition, it is possible to prevent the amount of history of the distant track shape from being reduced and the accuracy of specifying the processing area from being lowered. On the other hand, when the period during which the distant track shape is not calculated exceeds the threshold time, the calculation of the distant track shape is stopped, so that it is possible to prevent the distant track shape recognized with low accuracy from being stored as a history.

D. Fourth Embodiment:
The track recognition device 100a of the fourth embodiment shown in FIG. 10 is different from the track recognition device 100 of the first embodiment in that it functions as a vehicle position determination unit 106. Since the other configurations of the track recognition device 100a of the fourth embodiment are the same as those of the track recognition device 100 of the first embodiment, detailed description thereof will be omitted.

The vehicle position determination unit 106 determines whether or not the current position of the own vehicle is within the intersection region. The intersection area means an area including an intersection and an area in the vicinity of the intersection. The area near the intersection means an area within a predetermined distance from the center of the intersection, and in the present embodiment, means an area within 50 m from the center of the intersection. The distance is not limited to 50 m, and may mean an area within an arbitrary distance recognized as being near an intersection. In the present embodiment, the vehicle position determination unit 106 determines whether or not the current position is within the intersection region based on whether or not the traffic light is shown in the captured image and the size thereof is equal to or larger than a predetermined size. do. When the traffic light is shown in a size larger than a predetermined size, the position of the own vehicle is located in or near the intersection. The determination may be made based on the position information of the own vehicle obtained by a navigation system (not shown) mounted on the own vehicle.

As shown in FIG. 11, the distant track shape calculation process of the fourth embodiment is different from the track shape calculation process of the second embodiment in that steps S206, S207, and S208 are additionally executed. It is the same.

If it is determined in step S205 that there is no input of the nearby track shape (step S205: YES), the distant shape calculation unit 102 determines whether or not the current position of the own vehicle is within the intersection region (step S206). ).

When it is determined that the current position of the own vehicle is not within the intersection region (step S206: NO), the distant shape calculation unit 102 sets the threshold time in step S209 described above to a normal time (step S207). As a normal time, 1 second is set in this embodiment. The time is not limited to 1 second, and may be set to an arbitrary time longer than the acquisition interval of the captured image.

On the other hand, when it is determined that the current position of the own vehicle is within the intersection region (step S206: YES), the distant shape calculation unit 102 sets the predetermined threshold time in step S209 described above from the normal time. Is also set to a long time (step S208). As a longer time, 2 seconds is set in this embodiment. The time is not limited to 2 seconds, and may be set to any time longer than the normal time.

After executing step S207 or S208, step S209 described above is executed. That is, the distant shape calculation unit 102 determines whether or not the input of the nearby track shape is continuous beyond the predetermined threshold time. In the fourth embodiment, the threshold time in step S209 is set by step S207 or S208. In this way, when the current position of the own vehicle is within the intersection region, the threshold time in step S209, that is, the nearby track shape, which is a reference for determining whether to stop the calculation of the far track shape, is used. The reason why the time during which continuous input is not input is set to be long is as follows. That is, when the current position of the own vehicle is within the intersection region, it is clear that the calculation of the nearby track shape is restarted after a short period of time as the own vehicle travels, so that the calculation of the far track shape is not stopped. However, this is because the influence on the subsequent calculation of the distant track shape is small. Further, by doing so, it is possible to prevent the amount of history of the distant track shape from being unnecessarily reduced due to the stop of the calculation of the distant track shape. On the other hand, if the current position of the own vehicle is not within the intersection region and the nearby track shape is not continuously input, some problem such as a failure of the captured image or a failure of the track recognition device 100 may occur. It may have occurred. If the calculation of the distant track shape is continued in such a case, the automatic operation may be executed based on the distant track shape having low accuracy. Therefore, in the present embodiment, the threshold time of step S209 in such a case is set as the normal time. , I try not to set it for a long time.

The track recognition device of the fourth embodiment described above has the same effect as the track recognition device 100 of the first embodiment and the second embodiment. In addition, since the length of the threshold time in step S209 is set according to whether or not the current position of the own vehicle is within the intersection region, the reason why the nearby track shape is not calculated is that "the own vehicle is within the intersection region". If it is clear that the calculation of the near-track shape will be resumed after a short period of time due to the running of the own vehicle, the calculation of the distant track shape will be suppressed by suppressing the stop. It is possible to prevent the amount of history of the long-distance track shape from being inadvertently reduced. In addition, when the current position of the own vehicle is not within the intersection region, the threshold time is set to a shorter time than when the current position is within the intersection region, so that erroneous recognition of the distant track shape can be suppressed.

E. Fifth embodiment:
Since the device of the track recognition device of the fifth embodiment has the same device configuration as the track recognition device 100 of the first embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. .. As shown in FIG. 12, the target passing point setting process of the fifth embodiment is different from the target passing point setting process of the third embodiment in that steps S316, S317, and S318 are additionally executed. Are the same.

As shown in FIG. 12, when it is determined in step S315 that there is no input of the distant track shape (step S315: NO), the target passing point setting unit 105 determines whether or not the current position of the own vehicle is within the intersection region. (Step S316). Since step S316 is the same process as step S206 of the fourth embodiment described above, detailed description of the procedure will be omitted.

When it is determined that the current position of the own vehicle is not within the intersection region (step S316: NO), the target passing point setting unit 105 sets the threshold time in step S319 described above to a normal time (step S317). As a normal time, 1 second is set in this embodiment. The time is not limited to 1 second, and may be set to an arbitrary time longer than the acquisition interval of the captured image.

On the other hand, when it is determined that the current position of the own vehicle is within the intersection region (step S316: YES), the target passing point setting unit 105 sets the predetermined threshold time in step S319 described above to a normal time. It is set to a longer time than (step S318). As a longer time, 2 seconds is set in this embodiment. The time is not limited to 2 seconds, and may be set to any time longer than the normal time.

After the execution of step S317 or S318, the above-mentioned step S319 is executed. That is, the distant shape calculation unit 102 determines whether or not the input of the distant track shape is continuous beyond the predetermined threshold time. In the fifth embodiment, the threshold time in step S319 is set by step S317 or S318. In this way, when the current position of the own vehicle is within the intersection region, the threshold time in step S319, that is, the distance, which is a reference for determining whether to stop the calculation (estimation) of the distant track shape, is distant. The reason for setting the time during which the distant track shape as a result of the track shape calculation process is not input to be long is the same as the reason for setting the threshold time in step S209 to be long in the fourth embodiment described above.

The track recognition device according to the fifth embodiment described above has the same effect as the track recognition device 100 according to the first embodiment and the third embodiment. In addition, since the length of the threshold time in step S319 is set according to whether or not the current position of the own vehicle is within the intersection region, the reason why the far track shape is not calculated by the far track shape calculation process is "self". If it is clear that the vehicle is in the intersection area "and the calculation of the distant track shape is resumed after a short period of time as the own vehicle travels, the calculation of the distant track shape is stopped. This can be suppressed and the amount of history of the distant track shape can be suppressed from being inadvertently reduced. In addition, when the current position of the own vehicle is not within the intersection region, the threshold time is set to a shorter time than when the current position is within the intersection region, so that erroneous recognition of the distant track shape can be suppressed.

F. Sixth Embodiment:
Since the device of the track recognition device of the sixth embodiment has the same device configuration as the track recognition device 100 of the first embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. .. As shown in FIG. 13, the track shape calculation process of the sixth embodiment is different from the track shape calculation process of the first embodiment in that steps S405 and S410 are additionally executed, and the other procedures are the same. ..

As shown in FIG. 13, when it is determined in step S205 that the nearby track shape is input at a predetermined time (step S205: YES), the distance shape calculation unit 102 determines the accuracy of the input nearby track shape. The evaluation value of (step S405) is determined. Specifically, the distant shape calculation unit 102 specifies a distance along the traveling direction of the nearby track (hereinafter, referred to as “near track distance”) from the nearby track shape input in step S205, and the near track is such. The evaluation value is determined by referring to the evaluation value table based on the distance. The evaluation value table is stored in advance in a memory (not shown) included in the track recognition device 100.

As shown in FIG. 14, in the evaluation value table, the proximity track distance and the evaluation value are set in association with each other. In FIG. 14, the horizontal axis represents the nearby track distance, and the vertical axis represents the evaluation value. As shown in FIG. 14, in the evaluation value table, the evaluation value is set to be higher as the nearby track distance becomes longer. This is because the longer the distance of the nearby track, the higher the accuracy of calculating the shape of the nearby track. In the example shown in FIG. 14, the rate of change of the evaluation value with respect to the change of the proximity track distance from 0 to the distance La is the largest, and the evaluation value with respect to the change of the proximity track distance from the distance La to the distance Lb is the largest. The change between the nearby runway distance and the evaluation value is set in three stages so that the change rate of the evaluation value with respect to the change in the nearby runway distance when the rate of change is the second largest and is larger than the distance Lb is the lowest. There is. The number of steps is not limited to three, and may be set stepwise in any number of steps. Moreover, it may change continuously exponentially.

As shown in FIG. 13, the distant shape calculation unit 102 sets the nearby track shape calculated this time, that is, the nearby track shape input this time and the previously calculated distant track shape according to the determined evaluation value. , The processing area is set based on the imaging parameters and the information indicating the movement of the vehicle (step S410). Hereinafter, the specific procedure of step S410 will be described.

First, the distant shape calculation unit 102 determines the degree of likelihood (hereinafter referred to as “likelihood”) of the processing region estimated based on the nearby track shape input this time based on the evaluation value determined in step S405. , The likelihood of the processing area estimated based on the previous far track shape stored in the far shape storage unit 104 is specified with reference to the integrated likelihood table. In the present embodiment, the "processing area estimated based on the shape of the nearby track input this time" is also referred to as the "first estimation processing area". Further, the "processing area estimated based on the previous distance track shape" is also referred to as a "second estimation processing area".

The integrated likelihood table shown in FIG. 15 is stored in advance in a memory (not shown) included in the track recognition device 100. As shown in FIG. 15, in the integrated likelihood table, the likelihood of the first estimation processing area and the likelihood of the second estimation processing area are set in association with the evaluation value. In FIG. 15, the horizontal axis shows the evaluation value, and the vertical axis shows the likelihood of the first estimation processing area and the likelihood of the second estimation processing area. In FIG. 15, the region A1 below the straight line AL shows the likelihood of the first estimation processing region, and the region AL2 above the straight line AL (upper limit 100%) shows the likelihood of the second estimation processing region. In the present embodiment, the sum of the likelihood of the first estimation processing area and the likelihood of the second estimation processing area is normalized so as to always be 100%. Then, in the integrated likelihood table, the higher the evaluation value, that is, the higher the certainty of the input neighboring track shape, the higher the likelihood of the first estimation processing region and the lower the likelihood of the second estimation processing region. For example, as shown in FIG. 15, when the evaluation value is the value E1, the likelihood of the first estimation processing area is 70% (0.7), and the likelihood of the second estimation processing area is 30%. (0.3).

Next, the distant shape calculation unit 102 estimates the processing area based on the shape of the nearby track this time. That is, the first estimation processing area is set. In the present embodiment, such estimation is realized by executing steps S210 and S215 of the long-distance track shape calculation process in the first embodiment. Next, the distant shape calculation unit 102 estimates the processing area based on the previous distant track shape. That is, the second estimation processing area is set. In the present embodiment, such estimation is realized by executing step S275 of the long-distance track shape calculation process in the first embodiment.

Next, the far shape calculation unit 102 calculates the likelihood of the overlapping region between the first estimation processing region and the second estimation processing region. The likelihood of the overlapping region is derived based on Bayesian estimation using the likelihood of the first estimation processing region and the likelihood of the second estimation processing region. Specifically, it is calculated based on the following formula (1). In the following equation (1), X indicates the likelihood of the overlapping region. The variable A indicates the likelihood of the first estimation processing area. Further, the variable B indicates the likelihood of the second estimation processing area.
X = A × B / {A × B + (1-A) × (1-B)} ... (1)

For example, when the evaluation value is E1 shown in FIG. 15, 0.7 is substituted into the above equation (1) as the variable A for the overlapping area between the first estimation processing area and the second estimation processing area. 0.3 is substituted into the above equation (1) as the variable B. Therefore, for such an overlapping region, 0.5 is calculated as the integration likelihood X.

Next, the distant shape calculation unit 102 sets, among the regions (each pixel) in the image pickup apparatus, a region whose likelihood is equal to or higher than a predetermined threshold value as the processing region this time. For example, when the predetermined threshold value is set to 0.3 and the evaluation value is E1 shown in FIG. 15, the first estimation processing area (likelihood: 0.7) and the second estimation processing area (probability: 0.7) The area in which the likelihood: 0.3) and the overlapping area (likelihood: 0.5) of these two estimated processing areas are combined is set as the processing area this time. The likelihood is set to 0 for an area that does not correspond to either the first estimation processing area or the second estimation processing area, and is not set as the processing area this time.

As shown in FIG. 13, after the execution of such step S410, the above-mentioned steps S220 to S255 are executed.

The track recognition device of the sixth embodiment described above has the same effect as the track recognition device 100 of the first embodiment. In addition, the longer the nearby track distance is, the higher the evaluation value is set, and as a result, the likelihood of the nearby track shape is set to be higher. Therefore, the processing area estimated based on the nearby track shape is set to be higher this time. It can be easily set (easily used) as a processing area. Therefore, when the nearby track distance is long and the calculation accuracy of the nearby track shape is expected to be high, the processing area estimated based on the nearby track shape can be easily set as the current processing area, and the specific accuracy of the processing area ( It is possible to suppress a decrease in certainty as an area including a distant track. In the sixth embodiment, the far shape calculation unit 102 also corresponds to the evaluation unit in the claim.

G. Seventh Embodiment:
As shown in FIG. 16, the track recognition device 100b of the seventh embodiment is different from the track recognition device 100 of the first embodiment in that it functions as an object detection unit 107 and a stop determination unit 108. Since the other device configurations of the track recognition device 100b of the seventh embodiment are the same as those of the track recognition device 100 of the first embodiment, detailed description thereof will be omitted.

The object detection unit 107 detects the presence or absence of another vehicle in front of the own vehicle. In the present embodiment, the vehicle position determination unit 106 detects the presence or absence of another vehicle in front of the own vehicle by analyzing the captured image obtained by the image pickup unit 21. The stop determination unit 108 determines whether or not the own vehicle is stopped at a red light. In the present embodiment, the vehicle stop determination unit 108 determines whether or not there is a red light by analyzing the image captured by the image pickup unit 21, and stops based on the moving speed of the own vehicle obtained from the vehicle speed sensor 22. Determine if it is inside.

The track recognition device 100b of the seventh embodiment is different from the track recognition device 100 of the first embodiment in that the execution determination process shown in FIG. 17 is executed. The track recognition device 100b of the seventh embodiment executes the neighborhood track shape calculation process and the target passing point setting process in the same procedure as the track recognition device 100 of the first embodiment. Further, the track recognition device 100b executes the distant track shape calculation process in the same procedure as the track recognition device 100 of the sixth embodiment.

The execution determination process is a process for determining whether or not to execute the nearby track shape calculation process, and is executed when the ignition of the own vehicle is turned on. Therefore, in the seventh embodiment, the proximity track shape calculation process is not executed periodically. As shown in FIG. 17, the neighborhood shape calculation unit 101 determines whether or not the preceding vehicle does not exist in the traveling direction of the own vehicle and the own vehicle is stopped by the red light (step S505). Specifically, the neighborhood shape calculation unit 101 has no preceding vehicle in the traveling direction of the own vehicle and the own vehicle is red based on the detection result by the object detection unit 107 and the determination result by the stop determination unit 108. It is determined whether or not the vehicle is stopped by the signal.

When there is no preceding vehicle in the traveling direction of the own vehicle and it is determined that the own vehicle is stopped by the red light (step S505: YES), the periodic execution timing of the nearby track shape calculation process has arrived. Even so, the processing area for calculating the shape of the nearby track is not executed, and the processing area specified before the vehicle stops is specified as the current processing area (step S510). In the present embodiment, as the processing area specified before the vehicle stops, the processing area specified immediately before the vehicle stops is specified as the current processing area. Instead of the processing area specified immediately before the stop, the processing area specified a predetermined number of times before the stop may be specified as the current processing area.

When it is determined that "there is no preceding vehicle in the traveling direction of the own vehicle and the own vehicle is stopped due to a red light" (step S505: NO), the near shape calculation unit 101 and the far shape calculation unit 102 Executes the above-mentioned near track calculation process and far track shape calculation process (step S520).

When the above-mentioned step S510 is executed, steps S220 to S255 are executed as the distant track shape calculation process.

The track recognition device 100b of the seventh embodiment described above has the same effect as the track recognition device 100 of the first embodiment. In addition, if there is no preceding vehicle in the traveling direction of the own vehicle and the own vehicle is stopped due to a red light, the processing area specified immediately before the stop is specified as the current processing area. , It is possible to suppress a decrease in the specific accuracy of the processing area. When there is no other vehicle in front of the vehicle and the vehicle is stopped at a red light, the recognition accuracy of the nearby track shape is the same as before the stop. On the other hand, when there is a preceding vehicle and the vehicle is stopped at a red light, for example, it is assumed that the paint on the vehicle body of the preceding vehicle is erroneously recognized as a lane marking (neighboring track shape), and the characteristic accuracy of the processing area is lowered. There is a risk. Therefore, according to the track recognition device 100b of the present embodiment, it is possible to suppress a decrease in the accuracy of specifying the processing region.

H. Eighth embodiment:
As shown in FIG. 18, the track recognition device 100c of the eighth embodiment is electrically connected to the millimeter wave radar 24 mounted on the own vehicle, and functions as a three-dimensional structure detection unit 109. Is different from the track recognition device 100 of the first embodiment. Since the other configurations of the track recognition device 100c of the eighth embodiment are the same as the device configurations of the track recognition device 100 of the first embodiment, the same components are designated by the same reference numerals and detailed description thereof will be given. Is omitted.

The millimeter wave radar 24 detects an object in front of the own vehicle by using radio waves in the millimeter wave band. More precisely, the object is detected as a plurality of detection points (targets). The three-dimensional structure detection unit 109 is based on the detection result by the millimeter-wave radar 24 and the captured image obtained by the imaging unit 21, and the three-dimensional structure (preceding vehicle, oncoming vehicle, vehicle on an intersecting road, central separation zone, etc.) Repeat the type of pedestrian, guard rail, manhole, traffic light, etc.), the distance between the three-dimensional structure and the own vehicle, the position of the three-dimensional structure, the size of the three-dimensional structure, and the relative speed of the three-dimensional structure with respect to the own vehicle. To detect.

The track recognition device 100c of the eighth embodiment executes the near track shape calculation process, the far track shape calculation process, and the target passing point setting process in the same procedure as the track recognition device 100 of the first embodiment. However, the specific procedure of step S215 of the long-distance track shape calculation process shown in FIG. 4 is slightly different from that of the first embodiment.

In step S215 in the first embodiment, the near track (partition line) is extended to estimate the shape of the far track, and the processing area is set based on the shape of the far track. On the other hand, in the eighth embodiment, the far shape calculation unit 102 exists at the side edge of the road detected by the three-dimensional structure detection unit 109 in the processing area set in the same manner as in the first embodiment. The processing area is set inside the area partitioned by the three-dimensional structure (hereinafter referred to as "roadside object").

In the example of FIG. 19, the own vehicle 500 travels on the traveling lane Ln3 from the left side to the right side in FIG. 19, and is located in front of the intersection CR2. At this time, of the traveling lanes Ln3, two regions Er3 and El3 beyond the intersection CR2 are temporarily specified as processing regions based on the shape of the distant track. Then, of these two regions Er3 and El3, the region in the region LW is finally set as the processing region. The processing area LW is a roadside object PgR which is a three-dimensional structure detected along the traveling lane Ln3 in the vicinity of the traveling lane Ln3, and a three-dimensional structure detected along the traveling lane Ln3 in the vicinity of the adjacent traveling lane Ln4. It is specified as an area partitioned by the roadside object PgL. The roadside object PgR corresponds to, for example, a so-called median strip located at the boundary with the opposite lane (not shown). Further, the roadside object PgL corresponds to a guardrail located at the boundary between the traveling lane Ln4 and the sidewalk (not shown).

The track recognition device of the eighth embodiment described above has the same effect as the track recognition device 100 of the first embodiment. In addition, since the processing area is set inside the area partitioned by the inside of the roadside object detected by the three-dimensional structure detection unit, it is possible to suppress a decrease in the identification accuracy of the processing area. This is because the track is generally located inside the area partitioned by the roadside objects.

I. Ninth embodiment:
As shown in FIG. 20, the track recognition device 100d of the ninth embodiment is electrically connected to the connection interface unit (connection IF unit) 25, and serves as a travel lane identification unit 110 and a travel lane estimation unit 111. It differs from the track recognition device 100c of the eighth embodiment in that it functions. Since the other configurations of the track recognition device 100d of the ninth embodiment are the same as the device configurations of the track recognition device 100c of the eighth embodiment, the same components are designated by the same reference numerals and detailed description thereof will be given. Is omitted.

The connection IF unit 25 is electrically connected to the navigation device 600 mounted on the own vehicle, and acquires information indicating the position of the own vehicle and map information from the navigation device 600. Such map information includes information indicating the traveling lanes of each road. The traveling lane specifying unit 110 identifies the current traveling lane of the own vehicle based on the information about the current traveling lane input from the navigation device 600 via the connecting IF unit 25. When the own vehicle approaches the intersection, the traveling lane estimation unit 111 estimates the traveling lane after passing, which is the lane in which the own vehicle travels after passing through the intersection. Specifically, the traveling lane estimation unit 111 identifies a roadside object existing in front of the current traveling lane among the three-dimensional structures detected by the three-dimensional structure detecting unit 109, and identifies the identified roadside object and the current roadside object. Based on the travel lanes of, the travel lane after passing is estimated from the travel lanes existing in front of the current position of the own vehicle (after passing the intersection) obtained from the map information. For example, in the example of FIG. 19, although the own vehicle 500 is currently traveling in the traveling lane Ln3, since the intersection CR2 exists, the traveling lane scheduled to be traveled in the future cannot be specified by the analysis of the captured image. However, each information that the own vehicle 500 is currently traveling in the traveling lane Ln3, that the roadside object PgR exists in front, and that the traveling lane Ln3 and the traveling lane Ln4 exist ahead of the intersection CR2. Therefore, it is predicted that the own vehicle 500 will travel in the traveling lane Ln3 after passing through the intersection CR2, and the traveling lane Ln3 is estimated as the traveling lane after passing through.

The distant shape calculation unit 102 is the roadside detected by the three-dimensional structure detection unit 109 in the processing area (temporary processing area) set in the same manner as in the first embodiment in the same manner as in the eighth embodiment. The area inside the area partitioned by the object and further in the traveling lane after passing is set as the final processing area. Therefore, in the ninth embodiment, unlike the eighth embodiment, in the region El3 shown in FIG. 19, the region outside the traveling lane Ln3 is not set as the processing region.

The track recognition device 100d of the ninth embodiment described above has the same effect as the track recognition device 100c of the eighth embodiment. Further, since the traveling lane after passing is estimated and the processing area is further limited by using the traveling lane after passing, it is possible to further suppress a decrease in the accuracy of specifying the processing area. In particular, even in a situation where the calculation accuracy of the nearby track shape is low due to the existence of the intersection, it is possible to suppress a decrease in the specific accuracy of the processing area.

J. Tenth embodiment:
As shown in FIG. 21, the track recognition device 100e of the tenth embodiment has a point of being electrically connected to the acceleration sensor 26, a point of functioning as a posture change detection unit 112, and a correction amount map storage unit 120. It is different from the track recognition device 100 of the first embodiment in that it is provided. Since the other configurations of the track recognition device 100e of the tenth embodiment are the same as the device configuration of the track recognition device 100 of the first embodiment, the same components are designated by the same reference numerals and detailed description thereof will be given. Is omitted.

The acceleration sensor 26 detects the acceleration of the own vehicle. The posture change detection unit 112 detects a change in the posture of the own vehicle. In the present embodiment, the posture change detection unit 112 detects a change in the posture of the own vehicle based on the acceleration obtained by the acceleration sensor 26. Specifically, when the acceleration is a positive value in the direction of travel, that is, when the own vehicle is accelerating, the posture of the own vehicle is such that the front side faces the vertically upward side and the rear side faces the vertically downward side. When it changes like this, it is detected. Such a change in posture is so-called nose-up. Further, when the own vehicle is decelerating, it is detected that the posture of the own vehicle changes so that the front side faces the vertically downward side and the rear side faces the vertically upward side. Such a change in posture is so-called nose down.

The correction amount map storage unit 120 stores the correction amount map in advance. The correction amount map is a map in which the moving speed (vehicle speed) of the own vehicle, the acceleration of the own vehicle, and the correction amount are associated with each other. The correction amount is a correction amount for correcting the position of the processing area set in step S215 of the long-distance track calculation process, and is stored together with the correction direction in units of the number of pixels (pixels). When the nose-up described above occurs, the imaging direction of the imaging unit 21 changes, so that the imaging range captured in the captured image shifts upward from the imaging range set at the time of the initial calibration. Therefore, the processing area set in step S215 is detected so as to be shifted upward with respect to the actual area. On the contrary, when the nose down occurs, the imaging range captured in the captured image shifts downward from the imaging range set at the time of the initial calibration. Therefore, the processing area set in step S215 is detected so as to be shifted downward with respect to the actual area. Therefore, in step S215, the distant shape calculation unit 102 obtains a processing area according to the above procedure, and corrects the processing area so as to cancel the deviation amount due to the change in the posture of the own vehicle. Since the correction amount at this time correlates with the vehicle speed and the acceleration, the vehicle speed, the acceleration, and the correction amount are obtained in advance by an experiment, a correction amount map is created, and the correction amount map is stored in the correction amount map storage unit 120. Therefore, the distant shape calculation unit 102 specifies the correction amount with reference to the correction amount map based on the vehicle speed obtained from the vehicle speed sensor 22 and the acceleration obtained from the acceleration sensor 26.

The track recognition device 100e of the tenth embodiment described above has the same effect as the track recognition device 100 of the first embodiment. In addition, when the attitude change of the own vehicle is detected, the position of the processing area is corrected and specified according to the attitude change. Therefore, an area that is not appropriate due to the attitude change, for example, an area that does not include a distant track. Can be suppressed from being specified as a processing area. Further, since the correction amount when correcting the processing area is set with reference to the correction amount map stored in advance, the correction amount can be set easily and in a short time.

K. Eleventh embodiment:
As shown in FIG. 22, the track recognition device 100f of the eleventh embodiment has a point that functions as a template image identification unit 113 and a point that does not include a correction amount map storage unit 120. It is different from the device configuration of the device 100e. Since the other components of the track recognition device 100f of the eleventh embodiment are the same as those of the track recognition device 100e of the tenth embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. do. The template image specifying unit 113 specifies a template image to be used when setting the processing area. Such a template image will be described later.

The track recognition device 100f of the eleventh embodiment is different from the track recognition device 100e of the tenth embodiment in the procedure of the distant track shape calculation process. The track recognition device 100f of the eleventh embodiment executes the neighborhood track shape calculation process and the target passing point setting process in the same procedure as the track recognition device 100e of the tenth embodiment.

As shown in FIG. 23, when it is determined in step S205 that there is an input of a nearby track shape (step S205: YES), the attitude change detection unit 112 changes the attitude of the own vehicle, specifically, nose-up. Alternatively, it is determined whether or not nose down has occurred (step S440). When it is determined that nose-up or nose-down has occurred (step S440: YES), the template image specifying unit 113 sets a template image including a processing area set before the nose-up or nose-down occurs (step S440: YES). Step S445). Specifically, in the captured image used when the previous processing region was set (that is, the captured image obtained last time), the region including the processing region, for example, the processing region Er1 and the processing region El1 in FIG. An image in a rectangular area that includes both is cut out as a template image and set.

The distant shape calculation unit 102 executes pattern matching of the template image set in step S445 in the captured image this time (step S450). That is, while specifying a region having the same size as the template image set in step S445 while shifting the captured image obtained this time by one pixel, the similarity between the region and the template image is obtained.

As a result of step S450, the far shape calculation unit 102 sets a processing area in the most similar image area (step S455). Specifically, the distant shape calculation unit 102 sets a region at the same position as the relative position of the processing region in the template image as the processing region in the most similar image region. After the execution of step S445, the above-mentioned steps S220 to S255 are executed.

If it is determined in step S440 described above that nose-up or nose-down does not occur (step S440: NO), step S210 described above is executed.

The track recognition device 100f of the eleventh embodiment described above has the same effect as the track recognition device 100e of the tenth embodiment. In addition, when nose-up or nose-down occurs, an image of the area including the processing area is set as a template image in the captured image obtained before the occurrence, and pattern matching is performed on the captured image obtained this time with the template image. Since the processing area is set by performing the above, it is not necessary to correct the processing area when nose-up or nose-down occurs. Further, since it is not necessary to store the correction amount at the time of such correction in advance, the storage capacity required for such storage can be omitted.

L. 12th Embodiment:
As shown in FIG. 24, the track recognition device 100g of the twelfth embodiment is different from the track recognition device 100c of the eighth embodiment shown in FIG. 18 in that it functions as a correction amount specifying unit 114. Since the other components in the track recognition device 100g of the twelfth embodiment are the same as those of the track recognition device 100c of the eighth embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. do. The correction amount specifying unit 114 specifies the correction amount used when correcting the position of the processing area. In the track recognition device 100g, the position of the processing area is corrected according to the posture change of the own vehicle, similarly to the track recognition device 100e of the above-described 10 embodiments. In the track recognition device 100e of the tenth embodiment, the correction amount is specified by referring to the correction amount map based on the vehicle speed and the acceleration, but the track recognition device 100g of the twelfth embodiment is obtained by the image pickup unit 21. The correction amount is specified by using the captured image and the measurement result of the millimeter wave radar 24.

The track recognition device 100g of the twelfth embodiment is different from the track recognition device 100 of the first embodiment in the procedure of the distant track shape calculation process. The track recognition device 100g of the twelfth embodiment executes the neighborhood track shape calculation process and the target passing point setting process in the same procedure as the track recognition device 100 of the first embodiment.

As shown in FIG. 25, when it is determined in step S205 that there is an input of a nearby track shape (step S205: YES), the correction amount specifying unit 114 is from the own vehicle up to a specific point in the previous captured image. The difference between the distance and the distance from the own vehicle to the specific point in the captured image this time is calculated (step S460). The specific position can be set to any position as long as it is a position in the actual environment that is the same between the previous time and this time. For example, the position of a pole arranged on the roadside (for example, a pole for displaying a sign) may be set as a specific position.

Next, the correction amount specifying unit 114 determines the distance to the specific three-dimensional structure based on the measurement result of the previous millimeter-wave radar 24 and the distance to the specific three-dimensional structure based on the measurement result of the millimeter-wave radar 24 this time. The difference is calculated (step S465). The specific three-dimensional structure can be defined as any three-dimensional structure as long as it is a three-dimensional structure in the actual environment that is the same between the previous time and this time. Specific examples of steps S460 and S465 described above will be described with reference to FIGS. 26, 27 and 28.

In the example shown in FIG. 26, the posture of the own vehicle 500 is a normal posture, that is, a posture in which the longitudinal direction of the vehicle is parallel to the horizontal direction. At this time, the distance from the own vehicle 500 to the position P2 of the pole PL existing in front of the own vehicle 500 is estimated as the distance L1 based on the captured image. In this embodiment, such a distance is obtained from the relative position P2 of the pole PL in the captured image. Since the imaging range Ag is a predetermined range in the normal posture, that is, the posture when the imaging unit 21 is calibrated, the position P2 in the captured image is determined from the own vehicle 500. The distance to position P2 can be obtained (estimated). In the example of FIG. 26, the distance to the pole PL is estimated as the distance L2 based on the measurement result of the millimeter wave radar 24. Since the millimeter wave radar 24 irradiates the millimeter wave forward and receives the reflected wave from the pole PL, the millimeter wave radar 24 collides with the pole PL in the horizontal direction from the position of the millimeter wave radar 24 as the distance from the own vehicle 500 to the pole PL. The distance L2 to the position P1 is obtained.

As shown in FIG. 27, for example, when the own vehicle 500 suddenly brakes after the state of FIG. 26, nose down occurs. In this case, the imaging range Ag faces vertically downward as compared with the case of FIG. 26. Therefore, the relative position of the pole PL in the captured image changes vertically upward, and as shown by the broken line in FIG. 27, the pole PL appears as if it is located farther away. Therefore, in the example of FIG. 27, the position of the pole PL is specified as the position P2a farther from the own vehicle 500, unlike the position P2 shown in FIG. 26. At this time, the distance between the own vehicle 500 and the position P2a is estimated as the distance L11. Then, the difference between the distance L11 shown in FIG. 27 and the distance L1 shown in FIG. 26 is obtained in step S460 described above. This difference includes the mileage of the own vehicle 500 from the state of FIG. 26 to the state of FIG. 27 and the relative position deviation of the pole PL in the captured image due to the nose down described above. It has been.

On the other hand, as shown in FIG. 28, when the own vehicle 500 suddenly brakes and a nose down occurs after the state of FIG. 26 as in FIG. 27, according to the measurement result of the millimeter wave radar 24, the pole PL Therefore, the distance L12 from the own vehicle 500 to the pole PL measured at this time is also obtained as the distance from the own vehicle 500 to the position P1 of the pole PL, as in FIG. 26. Then, the difference between the distance L12 shown in FIG. 28 and the distance L2 shown in FIG. 26 is obtained in step S465. This difference is substantially equal to the mileage of the own vehicle 500 from the state of FIG. 26 to the state of FIG. 28.

As shown in FIG. 25, the correction amount specifying unit 114 specifies the correction amount so as to cancel the difference between the two types of the difference calculated in step S460 and the difference calculated in step S465 (step). S470). Specifically, the difference obtained in step S465 is subtracted from the difference obtained in step S460, and the correction amount is specified so as to cancel the remaining difference. For example, in the above-mentioned examples of FIGS. 26 to 28, the difference between the distance L12 and the distance L2 is subtracted from the difference between the distance L11 and the distance L1. Then, if the difference is positive, that is, if the pole PL is located farther, it is located closer, that is, it is located lower in the captured image. The correction amount is specified so as to be.

As shown in FIG. 25, the distant shape calculation unit 102 estimates the distant track shape by extending the nearby lane (parting line) by using the input near lane shape (step S475). This step S475 is the same as step S210 described above. The distant shape calculation unit 102 sets a temporary processing area based on the estimated distant track shape (step S480). The temporary processing area means a processing area before correction by the correction amount specified in step S470. This step S480 is the same as step S215 described above. The far shape calculation unit 102 sets the processing area by shifting the temporary processing area set in step S480 by the correction amount specified in step S470 (step S485). After the execution of step S485, the above-mentioned steps S220 to S255 are executed.

The track recognition device 100g of the twelfth embodiment described above has the same effect as the track recognition device 100g of the tenth embodiment. In addition, the amount of correction when correcting the position of the processing area when nose-up or nose-down occurs is calculated using the captured image obtained by the imaging unit 21 and the measurement result of the millimeter-wave radar 24. , The amount of correction can be obtained with high accuracy.

M. 13th Embodiment:
As shown in FIG. 29, the track recognition device 100h of the thirteenth embodiment is different from the track recognition device 100f of the eleventh embodiment in that it functions as the template image correction unit 115. Since the other components of the track recognition device 100h of the thirteenth embodiment are the same as those of the track recognition device 100f of the eleventh embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. do. The track recognition device 100h executes the near track shape calculation process, the far track shape calculation process, and the target passing point setting process in the same procedure as the track recognition device 100f of the eleventh embodiment.

The template image correction unit 115 corrects the template image in step S445 of the long-distance track shape calculation process shown in FIG. Specifically, the template is based on the mounting parameters that correlate with the mounting position of the imaging unit 21 or the posture of the imaging unit 21, the parameters that correlate with the shape of the traveling lane of the own vehicle, and the parameters that correlate with the behavior of the own vehicle. Correct the image. The mounting parameters include, for example, the yaw, pitch, and roll of the imaging unit 21 in the imaging direction, and the relative position (coordinates) of the imaging unit 21 from the position of the center of gravity of the own vehicle. As the parameters that correlate with the shape of the traveling lane, each parameter indicating the shape of the nearby running track and the like correspond. The parameters that correlate with the behavior of the own vehicle include, for example, the presence or absence of a lane change, the schedule for turning right or left, the vehicle speed of the own vehicle, acceleration / deceleration, and the like.

The track recognition device 100h of the thirteenth embodiment described above has the same effect as the track recognition device 100f of the eleventh embodiment. In addition, the template image is corrected based on the mounting parameters that correlate with the mounting position or orientation of the image pickup device, the parameters that correlate with the shape of the traveling lane of the vehicle, and the parameters that correlate with the behavior of the vehicle. An appropriate image can be used as a template image even when each parameter changes.

N. 14th Embodiment:
Since the device configuration of the track recognition device of the 14th embodiment is the same as the device configuration of the track recognition device 100f of the 11th embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. do.

The track recognition device 100f of the 14th embodiment performs the near track shape calculation process, the far track shape calculation process, and the target passing point setting process in the same procedure as the track recognition device 100f of the 11th embodiment. However, the far track shape calculation process of the 14th embodiment is different from the far track shape calculation process of the 11th embodiment in step S450 in the far track shape calculation process shown in FIG. 23, that is, the pattern matching execution procedure.

As shown in FIG. 30, in the pattern matching process in the 14th embodiment, the distant shape calculation unit 102 is a region to be compared with the template image in the captured image this time (hereinafter, referred to as a “comparison target region”). (Step S705). The distant shape calculation unit 102 specifies a broken line-containing region in the template image set in step S445 shown in FIG. 23 (step S710). The broken line-containing region is a region including a broken line, and means, for example, a region including such a lane marking when the lane marking is a broken line. As will be described later, the distant shape calculation unit 102 compares each pixel in the comparison target region with each pixel at the same position in the template image and gives a score according to whether or not they match. At this time, it is determined whether or not the pixel to be compared in the comparison target region (hereinafter referred to as “comparison pixel”) is a pixel corresponding to the pixel in the broken line-containing region in the template image (step S715).

When it is determined that the comparison pixel is a pixel corresponding to the pixel in the broken line-containing region (step S715: YES), the distant shape calculation unit 102 indicates that the comparison pixel and the corresponding pixel in the template image (pixels having the same relative position). Is compared with and it is determined whether or not they match (step S720). If it is determined that they match (step S720: YES), the far shape calculation unit 102 gives a score of "1" to the comparison pixel (step S725). On the other hand, if it is determined that they do not match (step S735: NO), a score of "0.5" is given to the comparison pixels (step S730).

In step S715 described above, when it is determined that the comparison pixel is not a pixel corresponding to the pixel in the broken line-containing region (step S715: NO), the comparison pixel and the corresponding pixel in the template image (pixels having the same relative position) are displayed. It is compared and it is determined whether or not they match (step S735). Such step S735 is the same as step S720 described above. If it is determined that they match (step S735: YES), the far shape calculation unit 102 gives a score of "1" to the comparison pixel (step S740), as in step S725 described above. On the other hand, when it is determined that they do not match (step S735: NO), a score "0" is given to the comparison pixel (step S745), unlike the above-mentioned step S730.

After any of the above steps S725, S730, S740, and S745 is executed, the far shape calculation unit 102 determines whether or not the comparison is completed for all the pixels in all the comparison target regions (step S750). If it is determined that the comparison has not been completed (step S750: NO), the process returns to step S705 described above. On the other hand, when it is determined that the comparison is completed (step S750: YES), the distant shape calculation unit 102 calculates and stores the total score for each comparison target area (step S755). The total score is a value obtained by adding up the scores given by executing any one of steps S725, S730, S740, and S745 for each pixel in each comparison target area. The higher the total score, the more similar it is to the template image. Then, after the execution of step S755, the pattern matching process ends, and step S455 shown in FIG. 23 is executed. At this time, the area having the highest total score is the most similar area, and the processing area is set in such an area.

Here, as described above, when the comparison pixel is a pixel corresponding to the pixel in the broken line containing region, the score given when the comparison pixel does not match the corresponding pixel is "0.5". , The score is higher than the score "0" when the comparison pixel is not a pixel corresponding to the pixel in the broken line containing region. By doing so, when the lane marking of the traveling lane of the own vehicle is a broken line, the influence of the result of the pattern matching processing on the total score can be reduced as compared with the case where the lane marking is a solid line. In general, when the lane marking is a broken line, even if the lane marking is the same, there is a high possibility that the mode shown in the previously captured image and the mode shown in the current captured image are different. Specifically, the position of the boundary between the painted part and the unpainted part of the broken line shifts as the vehicle runs, so even if it is the same lane marking, it appears in the previous captured image. There is a high possibility that the aspect is different from the aspect shown in the captured image this time. Therefore, in the case of the broken line, the same lane marking can be evaluated as the same by reducing the influence of the pattern matching result on the total score.

The 100f of the 14th embodiment described above has the same effect as the track recognition device 100f of the 11th embodiment. In addition, when the lane marking is a broken line, the influence of the pattern matching result on the score indicating the degree of matching is reduced as compared with the case where the lane marking is a solid line. can. When the lane marking line is a broken line, there is a high possibility that the shape will be different if the imaging timing is different even though the line is the same line. Therefore, in this case, since the influence on the score indicating the degree of coincidence (total score) is reduced, the possibility of recognizing the same lane marking as the same lane marking can be increased.

O. Fifteenth embodiment:
As shown in FIG. 31, the track recognition device 100i of the fifteenth embodiment is different from the track recognition device 100 of the first embodiment in that it functions as an intersection determination unit 116 and a direction identification unit 117. Since the other configurations of the track recognition device 100i of the fifteenth embodiment are the same as those of the track recognition device 100 of the first embodiment, detailed description thereof will be omitted.

The intersection determination unit 116 determines whether or not the own vehicle is traveling in the intersection area or stopped in the intersection area. The direction specifying unit 117 detects a plurality of paints forming a striped pattern (so-called zebra) of a pedestrian crossing on a traveling lane of the own vehicle, and identifies the longitudinal directions of the detected plurality of paints. Generally, the longitudinal direction of the plurality of paints is along the travel lane.

As shown in FIG. 32, the long-distance track shape calculation process of the fifteenth embodiment is the same as the long-distance track shape calculation process of the first embodiment shown in FIG. 4 in that steps S280 to S284 are executed instead of step S275. Different, the other steps are the same. The neighborhood track shape calculation process and the target passing point setting process of the fifteenth embodiment are the same as those of the first embodiment.

In step S270 described above, when it is determined that there is a history of the previous long-distance track shape (step S270: YES), the intersection determination unit 116 is traveling in the intersection area or stopped in the intersection area. It is determined whether or not there is (step S280). The definition of the intersection region is as described in the fourth embodiment. Further, it is possible to determine whether or not the own vehicle is traveling or stopped in the intersection based on the vehicle speed measured by the vehicle speed sensor 22 in addition to the same procedure as in step S206 of the fourth embodiment. When it is determined that the own vehicle is traveling in the intersection area or not stopped in the intersection area (step S280: NO), the distant track shape calculation process ends.

When it is determined that the own vehicle is traveling in the intersection area or stopped in the intersection area (step S280: YES), the distant shape calculation unit 102 crosses the traveling lane of the own vehicle in this captured image. It is determined whether or not there are a plurality of paints constituting the striped pattern of the sidewalk (step S281). When it is determined that there is no plurality of paints (step S281: NO), the distant track shape calculation process ends. On the other hand, when it is determined that there are a plurality of paints (step S281: YES), the direction specifying unit 117 among the edges along the longitudinal direction of the plurality of paints specified in step S281 of the own vehicle. The edge located at the right end and the edge located at the left end in the traveling lane are specified (step S282). Based on the two edges identified in step S282, the direction specifying unit 117 identifies the position of a point (so-called vanishing point) where the virtual lines intersect when the edge is extended in the captured image this time (step). S283). The far shape calculation unit 102 sets the processing area based on the position of the vanishing point specified in step S283 (step S284).

In the example of FIG. 33, it is determined that the traveling lane Ln5 of the own vehicle has a plurality of paints constituting the striped pattern of the pedestrian crossing Pd3. The traveling lane Ln5 is partitioned by the division lines Lr4 and Ll4 that are slightly reflected below the captured image F20. In step S282, of the plurality of edges, the paint on the right side, which is partially contained in the traveling lane Ln5, the edge Eg1 inside the Zr, and the paint on the left side, which is also partially contained in the traveling lane Ln5. The inner edge Eg2 of Zl is identified. In step S283, the point P10 where the two virtual lines Vl1 and Vl2, which are the extensions of the two edges Eg1 and Eg2 specified in step S282, intersect with each other is specified as a vanishing point.

The method of setting the processing area in step S284 will be specifically described with reference to the example of FIG. 33. First, the two first points on the two virtual lines Vl1 and Vl2, which are two first points in front of the point P10 (the intersection position of the two virtual lines Vl1 and Vl2) by the predetermined first distance L21. The points P1r and P1l are specified. Next, two points different from the two first points P1r and P1l on the two virtual lines Vl1 and Vl2, which are two points behind the pedestrian crossing Pd3 on the back side by a predetermined second distance L22. The side points are specified as the second points P2r and P2l. Then, a rectangle having a predetermined third distance L3 as the width in the direction intersecting the traveling lane Ln5 with the line segment connecting the first point P1r and the second point P2r as the center along the virtual line Vl1. Area is set as the processing area Er4. Similarly, along the virtual line Vl2, a rectangular region having the width of the third distance L3 in the direction intersecting the traveling lane Ln5 with the line segment connecting the first point P1l and the second point P2l as the center. , Set as the processing area El4.

As shown in FIG. 32, after the execution of step S284, the above-mentioned steps S220 to S255 are executed.

The track recognition device 100i of the fifteenth embodiment described above has the same effect as the track recognition device 100 of the first embodiment. In addition, when it is determined that the own vehicle is traveling or stopped in the intersection area, the processing area is set by using the longitudinal direction of the paint forming the striped pattern of the pedestrian crossing. It is possible to suppress a decrease in specific accuracy. In general, the longitudinal direction of a plurality of paints constituting the striped pattern of the pedestrian crossing on the traveling lane is parallel to the traveling lane. This is because the decrease in the amount can be suppressed.

P. 16th Embodiment:
As shown in FIG. 34, the track recognition device 100j of the 16th embodiment is different from the device configuration of the track recognition device 100 of the 1st embodiment in that it functions as a normality determination unit 118. Since the other components of the track recognition device 100j of the 16th embodiment are the same as those of the track recognition device 100 of the 1st embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. do. The normality determination unit 118 determines the normality of the white line candidate temporarily selected in the distant track shape calculation process in the 16th embodiment described later.

As shown in FIG. 35, the distance track shape calculation process of the 16th embodiment executes steps S805 to S810 instead of steps S225 to S240 as compared with the distance track calculation process of the first embodiment shown in FIG. Only different in that. The track recognition device 100j of the 16th embodiment executes the neighborhood track shape calculation process and the target passing point setting process in the same procedure as the track recognition device 100 of the 1st embodiment. In the first embodiment, the Hough transform is performed to identify the white line candidate in the processing area, but in the sixteenth embodiment, LMedS (Least Median Squares) is used. LMedS is a kind of so-called random sample method. RANSAC (RANdom SAmple Consensus) may be used instead of LMedS.

As shown in FIG. 35, after the execution of step S220, the distant shape calculation unit 102 determines a white line candidate using LMedS (step S805). Specifically, any two edges in the processing area extracted in step S220 are randomly sampled. Next, the square of the error between all the other edges (for example, N) and the straight line is calculated from the linear equation connecting the two edges. Next, the median of the obtained N squared errors is specified and recorded as the evaluation score of such a straight line. The above is executed for the combination of two of all edges. Then, the straight line with the lowest evaluation score among all the obtained evaluation scores is determined as a white line candidate. The white line candidate can be called a "temporary distant track shape" because it is calculated as a distant track shape when it is determined to be normal as described later.

The normality determination unit 118 determines whether or not the white line candidate determined in step S805 is normal (step S810). A specific method for determining the normality will be described with reference to FIG. In FIG. 36, the white line candidates m1 to m4, the error distribution, and the evaluation scores S1 to S4 of each white line candidate are shown for each of the four cases C1 to C4.

In case C1, the evaluation score S1 is a relatively small value, and is determined to be normal in this embodiment. On the other hand, in the other three cases C2 to C4, it is determined that it is not normal, that is, it is abnormal. Specifically, in case C2, since the evaluation score is relatively large, it is determined to be abnormal. Further, in case C3, the evaluation score S3 is relatively small. However, in case C3, an error range of values deviating from the median value (square error) is a continuous error range, and an error range having a large frequency and equal to each other exists relatively widely. As shown in case C3, such a situation is assumed to be a case where a similar straight line m30 may exist in addition to the straight line m3 determined as a white line candidate. Specifically, when there is a guardrail installed along the white line near the white line (partition line), the straight line consisting of the edges of the guardrail also deviates from the median in the error distribution. It appears in a manner in which the error is evenly distributed. Further, in case C4, although the evaluation score S4 is relatively small, the number of errors calculated is also small because the total number of edges is small, and as a result, it is assumed that the probability that the white line candidate m4 is a white line is low.

In the present embodiment, in order to recognize the above three cases C2 to C4 as abnormal, if any of the following conditions (i) to (iii) is satisfied, it is not normal (abnormal). Judgment is made, and if none of the conditions are satisfied, it is judged to be normal.
(I) The median evaluation score in the error distribution obtained by LMedS is larger than the predetermined threshold evaluation score.
(Ii) In the error distribution obtained by LMedS, the error range in which the error of the value deviating from the median is continuous and the frequency is larger than the predetermined threshold frequency and equal to each other is the predetermined range. Wider than.
(Iii) The number of errors obtained is less than the predetermined number of thresholds.

As shown in FIG. 35, when it is determined that the white line candidate is normal as a result of the above-mentioned step S810 (step S810: YES), the far shape calculation unit 102 executes the above-mentioned steps S245 to S255. On the other hand, when it is determined that the white line candidate is not normal (step S810: NO), the distant track shape calculation process ends. Therefore, in this case, the distant track shape is not calculated.

The track recognition device 100j of the 16th embodiment described above has the same effect as the track recognition device 100 of the 1st embodiment. In addition, since the distant track shape is not calculated when the white line candidate determined by LMedS is determined to be abnormal, it is possible to prevent the distant track shape that is not normal from being stored in the distant shape storage unit 104 as a history. Therefore, it is possible to suppress a decrease in accuracy when the processing area is set using the long-distance track shape.

Further, since the normality of the white line candidate is determined based on the error distribution, such normality can be accurately determined. Further, when any of the above conditions (i) to (iii) is satisfied, it is determined that the white line candidate is not normal, so that the normality of the white line candidate can be accurately determined.

Q. 17th Embodiment:
As shown in FIG. 37, the track recognition device 100k of the 17th embodiment is different from the device configuration of the track recognition device 100j of the 16th embodiment in that it functions as the track estimation unit 119. Since the other components of the track recognition device 100k of the 17th embodiment are the same as those of the track recognition device 100j of the 16th embodiment, the same components are designated by the same reference numerals and detailed description thereof will be omitted. do. The track estimation unit 119 estimates the current distant track shape based on the previous distant track shape.

As shown in FIG. 38, the distance track shape calculation process of the 17th embodiment is different from the distance track calculation process of the 16th embodiment shown in FIG. 35 only in that step S800 is additionally executed. The track recognition device 100k of the 17th embodiment executes the neighborhood track shape calculation process and the target passing point setting process in the same procedure as the track recognition device 100 of the first embodiment.

As shown in FIG. 38, after the execution of step S220, the track estimation unit 119 is based on the previous distance track shape stored in the distance shape storage unit 104, the imaging parameters, and the information indicating the movement of the vehicle. The shape of the distant track this time is estimated (step S800). Since this step S800 is the same process as the process of estimating the long-distance track shape this time in step S275 described above, detailed description thereof will be omitted. After step S800, steps S805 and S810 described above are executed. However, the specific procedure of step S810 in this embodiment is different from the procedure of the 16th embodiment.

Specifically, the normality determination unit 118 compares the current distance track shape estimated in step S800 with the white line candidate determined in step S805, and positions and angles of these two shapes (straight lines). Find the difference between. It should be noted that only the difference between the position and the angle may be obtained. Then, when the difference is equal to or more than a predetermined threshold value, it is determined to be abnormal (abnormal), and when it is smaller than the threshold value, it is determined to be normal.

In FIG. 39, the straight lines Lr4 and Ll4 show the shape of the nearby track. Further, the straight lines Lr5 and Ll5 indicate the current distant track shape estimated from the previous distant track shape. Further, the straight lines Lr6 and Ll6 indicate straight lines of white line candidates. As shown in FIG. 39, the straight line Lr5 and the straight line Lr6 intersect each other, and the angle difference is relatively large. Similarly, the straight line Ll5 and the straight line Ll6 intersect each other, and the angle difference is relatively large. Therefore, in this case, the straight lines Lr6 and Ll6 of the white line candidates are likely to be determined to be abnormal. The reason why the white line candidate is so different from the current distant track shape estimated from the previous distant track shape is, for example, the white line from the guardrail existing along the lane marking or the edge extracted from the planting. It is assumed that candidates are decided.

The track recognition device 100k of the 17th embodiment described above has the same effect as the track recognition device 100j of the 16th embodiment. In addition, the white line candidate determined by LMedS is compared with the current distance track shape estimated from the previous distance track shape, and when the difference between the angles and positions of the two straight lines is equal to or greater than the threshold value, the white line candidate is selected. Is not normal, so the normality of the white line candidate can be judged accurately.

R. 18th Embodiment:
Since the device configuration of the track recognition device in the 18th embodiment is the same as the device configuration of the track recognition device 100k in the 17th embodiment shown in FIG. 37, the same components are designated by the same reference numerals. The detailed description thereof will be omitted.

The long-distance track shape calculation process in the 18th embodiment is the same as the long-distance track shape calculation process in the 17th embodiment shown in FIG. 38. However, the specific process of step S805 is different from that of the 17th embodiment.

The track recognition device 100k of the eighteenth embodiment executes step S805 in the procedure shown in FIG. 40. Specifically, the normality determination unit 118 selects two edges and specifies a straight line passing through the two edges (step S905). The normality determination unit 118 obtains a tentative error distribution using LMedS for the straight line specified in step S905 (step S910). The reason for calling it a "temporary error distribution" is that such an error distribution may be corrected as described later. The normality determination unit 118 obtains the degree of coincidence between the current distance track shape estimated in step S800 and the straight line specified in step S905 (step S915). For example, as in the 17th embodiment, the difference between the positions and angles of these two shapes (straight lines) is obtained, and the degree of agreement is determined so that the smaller the difference, the larger the difference.

The normality determination unit 118 corrects the error distribution obtained in step S910 according to the degree of matching obtained in step S915, and obtains the final error distribution (step S920). Specifically, in the present embodiment, the median value, that is, the evaluation score is corrected to be low according to the degree of agreement. At this time, the higher the degree of agreement, the greater the amount of reduction in the evaluation score.

The normality determination unit 118 determines whether or not all edges have been selected (step S925), and if it is determined that all edges have not been selected (step S925: NO), the process returns to step S905 described above. .. On the other hand, when it is determined that all the edges have been selected (step S925: YES), the straight line having the lowest evaluation score is determined as a white line candidate (step S930). As described above, it can be said that the straight line having a higher degree of agreement with the estimated distance track shape this time is more likely to be determined as a white line candidate in step S930 because the evaluation score is greatly reduced.

The track recognition device 100k of the 18th embodiment described above has the same effect as the track recognition device 100k of the 17th embodiment. In addition, among the straight lines to be evaluated in LMedS, the higher the degree of coincidence with the current far track shape estimated based on the previously calculated far track shape, the greater the evaluation score will be reduced, so it can be used as a white line candidate. It can be easily determined. Therefore, the white line candidate can be determined with higher accuracy.

S. Other embodiments:
(S-1) In each embodiment, the process using the nearby track shape calculated this time may be replaced with the process using the history of the nearby track shape. For example, in step S210 of the far track shape calculation process in the first embodiment, the far track shape may be estimated using the history of the near track shape instead of the near track shape calculated this time. For example, the distant track shape may be estimated from the previous near track shape and the near track shape two times before, and these two distant track shapes may be combined (externalized) to estimate the current distant track shape. Further, for example, in step S275 of the far track shape calculation process of the first embodiment, the processing area may be set using the history of the far track shape instead of the previously calculated far track shape.

(S-2) The fourth embodiment and the fifth embodiment may be combined. That is, when it is determined that the current position of the own vehicle is within the intersection region (step S206: YES and step S316: YES), both the threshold time in step S209 and the threshold time in step S319 are set to be longer than the normal time. May be set to a long time.

(S-3) In each embodiment, there is no other vehicle in front of the own vehicle, the own vehicle is stopped at a red light, and there is a moving object that crosses the front of the own vehicle. In this case, the processing area set before the stop of the own vehicle may be set as the current processing area without newly setting the processing area. In this configuration, the track recognition device further includes an object detection unit that detects an object in front of the own vehicle and a stop determination unit that determines whether or not the own vehicle is stopped at a red light. The stop determination unit can determine whether or not the vehicle is stopped at a red light, for example, by analyzing the captured image. When there is a moving object that crosses the front of the own vehicle, the accuracy of the nearby track shape calculated based on the captured image is lowered, and the accuracy of the processing area set by using such a nearby track shape is also reduced. The accuracy of the distant track shape is also reduced. Therefore, with such a configuration, it is possible to suppress the calculation of the near track shape and the far track shape with low accuracy.

(S-4) In the fifteenth embodiment, the edge of the paint on the pedestrian crossing used for setting the processing area is the own vehicle among the edges along the longitudinal direction of the plurality of paints specified in step S281. The edge is located at the right end and the edge is located at the left end in the traveling lane, but the present disclosure is not limited to this. It may be two edges at arbitrary positions along the longitudinal direction of the plurality of paints identified in step S281. Further, the processing area may be specified by using not only two edges but also a plurality of edges. That is, the position of the vanishing point may be specified by using a plurality of edges, and the processing area may be set based on the position of the vanishing point.

(S-5) In the fifteenth embodiment, the specific method for setting the processing area can be changed as appropriate. For example, when the position of the vanishing point P10 is specified by extending the edge specified in step S282, a triangular region having a predetermined shape having the vanishing point P10 as an apex is converted into a virtual line in which two edges are extended. It may be arranged and set along the line.

(S-6) In the fourth embodiment, the length of the threshold time in step S319 of the target passing point setting process is changed depending on whether or not the current position of the own vehicle is within the intersection region. Disclosure is not limited to this. For example, in step S205 of the distant track shape calculation process, "a process of determining whether or not a nearby track shape is input beyond a predetermined time (predetermined threshold time) (hereinafter," step " It is called "S205a"). Then, before the step S205a, the above-mentioned steps S260 to S264 are executed. At this time, the threshold time set in step S262 or S264 may be used as the threshold time in step S205a.

(S-7) The template image may be corrected based on at least one of the three parameters of the thirteenth embodiment.

(S-8) In step S800 of the far track shape calculation process of the 17th embodiment, the current far track shape may be estimated by using the current far track shape instead of the previous far track shape. Such estimation may be performed, for example, by the same procedure as in step S210 of the long-distance track shape calculation process of the first embodiment.

(S-9) In each embodiment, the calculated near track shape and far track shape are used to set a target passing point, that is, a target passing point when the own vehicle automatically drives. However, the present disclosure is not limited to this. For example, an alarm may be generated when at least one of the near track shape and the far track shape is not calculated, and the near track shape and the far track shape may be used as a trigger for generating such an alarm. In such a configuration, the execution of the target passing point setting unit 105 and the target passing point setting process may be omitted.

(S-10) In each embodiment, a part of the configuration realized by the hardware may be replaced with software, and conversely, a part of the configuration realized by the software may be replaced with the hardware. You may do it. For example, in the first embodiment, at least one functional unit of the neighborhood shape calculation unit 101, the distance shape calculation unit 102, the neighborhood shape storage unit 103, the distance shape storage unit 104, and the target passing point setting unit 105 is integrated. It may be realized by a circuit, a discrete circuit, or a module that combines these circuits. Further, when a part or all of the functions of the present disclosure are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. "Computer readable recording medium" is not limited to portable recording media such as flexible disks and CD-ROMs, but is fixed to internal storage devices in computers such as various RAMs and ROMs, and computers such as hard disks. It also includes external storage devices that have been installed. That is, the term "computer-readable recording medium" has a broad meaning including any recording medium in which data packets can be fixed rather than temporarily.

The present disclosure is not limited to the above-described embodiment, and can be realized by various configurations within a range not deviating from the gist thereof. For example, the technical features in each embodiment are appropriately replaced or combined in order to solve some or all of the above-mentioned problems, or to achieve some or all of the above-mentioned effects. It is possible. Further, if the technical feature is not described as essential in the present specification, it can be deleted as appropriate.

100, 100a to 100k track recognition device, 21 imaging device, 101 neighborhood shape calculation unit, 102 distance shape calculation unit, 103 neighborhood shape storage unit, 104 distance shape storage unit, Er1 to Er4, El1 to El4 processing area

Claims (22)

A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
A target passing point setting unit (105) for setting a target passing point used for automatic driving of the vehicle is further provided.
When the near-runway shape is calculated, the target passage point setting unit sets the target passage point based on the near-runway shape, and when the near-runway shape is not calculated, the far-distance runway shape is used. Based on the above target passing point,
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
The distant track shape calculation unit does not calculate the distant track shape when the nearby track shape is not continuously calculated beyond a predetermined first period.
Track recognition device. In the track recognition device according to claim 2,
The far shape calculation unit uses at least one of the near track shape history and the far track shape history when the period during which the far track shape is not calculated does not exceed a predetermined second period. The long-distance track shape is calculated, and when the uncalculated period exceeds the second period, the long-distance track shape is not calculated.
Track recognition device. In the track recognition device according to claim 3,
Further, a vehicle position determination unit (106) for determining whether or not the current position of the vehicle is within the intersection region including the intersection and the region near the intersection is provided.
As a result of the determination by the vehicle position determination unit, the distant shape calculation unit indicates that the current position of the vehicle is within the intersection region, as compared with the case where the vehicle is not within the intersection region, the first period and the first period. Set at least one of the two periods to a longer time,
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
An evaluation unit for acquiring an evaluation value that correlates with the recognition accuracy for the calculated nearby track shape is further provided.
The evaluation unit acquires a higher evaluation value as the length in the depth direction of the calculated neighborhood track shape is longer.
The distant shape calculation unit sets the processing area using the calculated neighborhood track shape with a utilization rate according to the calculated evaluation value.
Track recognition device. In the track recognition device according to claim 5,
An object detection unit (107) that detects the presence or absence of another vehicle in front of the vehicle, and
A stop determination unit (108) that determines whether or not the vehicle is stopped at a red light,
With more
When the other vehicle does not exist in front of the vehicle and the vehicle is stopped at a red light, the distant shape calculation unit obtains the processing area set before the vehicle stops. Set as the processing area this time,
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
An object detection unit (107) that detects an object in front of the vehicle, and
Using the captured image, a stop determination unit (108) for determining whether or not the vehicle is stopped at a red light, and a stop determination unit (108).
With more
The distant shape calculation unit is used when there is no other vehicle in front of the vehicle, the vehicle is stopped at a red light, and there is a moving object crossing the front of the vehicle. Does not set a new processing area, but sets the processing area specified before the vehicle stops as the current processing area.
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
Further equipped with a three-dimensional structure detection unit (109) that detects a three-dimensional structure using the reflected wave of the irradiated electromagnetic wave.
The distant shape calculation unit identifies a roadside object existing on the side edge of the road among the detected three-dimensional structures, and sets the processing area inside the area partitioned by the specified roadside object. ,
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
A three-dimensional structure detection unit that detects a three-dimensional structure using the reflected wave of the irradiated electromagnetic wave,
A traveling lane specifying unit (110) that identifies the current traveling lane of the vehicle, and
Among the detected three-dimensional structures, a roadside object existing on the side edge of the road is identified, and the vehicle passes through the intersection by using the specified roadside object and the specified current traveling lane. A traveling lane estimation unit (111) that estimates a traveling lane after passing, which is a lane to be traveled later,
With more
The distant shape calculation unit sets the processing area by using the information indicating the estimated travel lane after passing.
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
A posture change detection unit (112) for detecting a posture change of the vehicle, which is nose-up or nose-down, is further provided.
When the posture change is detected, the far shape calculation unit corrects and sets the position of the processing region according to the posture change.
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
A three-dimensional structure detection unit (109) that repeatedly detects a three-dimensional structure using the reflected wave of the irradiated electromagnetic wave, and
A correction amount specifying unit (114) that specifies a correction amount when correcting a positional deviation of the processing area in the captured image due to a change in the posture of the vehicle that is nose-up or nose-down.
With more
The far shape calculation unit corrects the position of the processing area with the correction amount and sets it.
The correction amount specifying unit is
The difference between the distance from the vehicle to the specific point in the previous captured image and the distance to the vehicle to the specific point in the current captured image.
The difference between the distance from the vehicle to the specific three-dimensional structure detected this time by the three-dimensional structure detection unit and the distance from the vehicle to the specific three-dimensional structure detected before the previous time. Based on this, the correction amount is specified.
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
A posture change detection unit (112) that detects a posture change of the vehicle that is nose-up or nose-down, and a posture change detection unit (112).
A template image specifying unit (113) that specifies an image of a predetermined size area including the processing area specified before the occurrence of the posture change as a template image, and
With
When the posture change is detected, the distant shape calculation unit executes pattern matching on the captured image using the template image to set the processing area.
Track recognition device. In the track recognition device according to claim 12,
The template image is created based on at least one of a mounting parameter that correlates with the mounting position or attitude of the imaging device, a parameter that correlates with the shape of the traveling lane of the vehicle, and a parameter that correlates with the behavior of the vehicle. Further equipped with a template image correction unit for correction,
Track recognition device. In the track recognition device according to claim 12 or 13.
When the lane marking that divides the traveling lane of the vehicle is a broken line, the distant shape calculation unit gives the result of the pattern matching to the score indicating the degree of matching as compared with the case where the lane marking is a solid line. Reduce the impact,
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
An intersection determination unit (116) for determining whether or not the vehicle is traveling or stopped in an intersection region including an intersection and an area in the vicinity of the intersection.
In the captured image, a direction specifying portion (117) that detects a plurality of paints constituting a striped pattern of a pedestrian crossing on a traveling lane of the vehicle and specifies the longitudinal direction of the detected plurality of paints.
With more
The far shape calculation unit
When it is determined that the vehicle is not running or stopped in the intersection area, the processing area is set based on the calculated shape of the nearby track.
When it is determined that the vehicle is running or stopped in the intersection area, the processing area is set by using the longitudinal directions of the plurality of paints specified by the direction specifying portion.
Track recognition device. In the track recognition device according to claim 15,
When it is determined that the vehicle is traveling or stopped in the intersection region, the direction specifying portion is located in the traveling lane among the edges along the longitudinal direction of the plurality of paints. The intersection position (P10) between the virtual lines (Vl1, Vl2) extending the edges (Eg1, Eg2) is specified, and the intersection position (P10) is specified.
The far shape calculation unit sets the processing area based on the intersection position.
Track recognition device. In the track recognition device according to claim 16,
The two edges are located on the farthest ends of the traveling lane, respectively.
Track recognition device. In the track recognition device according to claim 16 or 17.
The distant shape calculation unit has two first points (P1r, P1l) on two virtual lines extending two edges located at both ends in the traveling lane among the edges along the longitudinal direction of the plurality of paints. The two first points on the front side by the predetermined first distance (L21) from the intersection position and the two second points different from the two first points on the two virtual lines ( P2r, P2l) centered on a line segment connecting two second points on the back side by a predetermined second distance (L22) from the end on the back side of the pedestrian crossing (Pd3). Two rectangular regions (Er4, El4) having a predetermined third distance (L3) as the width in the direction intersecting with the traveling lane (Ln5) are specified as the processing regions.
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
A normality determination unit (118) for determining the normality of the distant track shape to be calculated is further provided.
The distant shape calculation unit does not calculate the distant track shape when it is determined that the calculated distant track shape is not normal.
The far shape calculation unit has the lowest median error frequency in the error distribution based on the error distribution obtained by LMedS based on the error between the straight line passing through any two edges in the captured image and the other edge. A candidate straight line, which is a straight line, is calculated as a temporary distant track shape, and
The normality determination unit
The condition that the evaluation score, which is the frequency of the median error in the error distribution for the candidate straight line, is larger than the predetermined threshold score,
The condition that the continuous range of errors deviating from the median error in the error distribution of the candidate straight line is wider than the predetermined range.
The condition that the obtained error number is less than the predetermined threshold number and
If at least one of the conditions is satisfied, it is determined that the temporary long-distance track shape is not normal, and if none of the conditions are satisfied, it is determined that the temporary long-distance track shape is normal. The provisional long-distance track shape is calculated as the long-distance track shape.
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
A normality determination unit (118) for determining the normality of the distant track shape to be calculated is further provided.
The distant shape calculation unit does not calculate the distant track shape when it is determined that the calculated distant track shape is not normal.
A track estimation unit (119) for estimating the estimated distant track shape, which is the distant track shape this time, is further provided based on the history of the distant track shape.
The far shape calculation unit has the lowest median error frequency in the error distribution based on the error distribution obtained by LMedS based on the error between the straight line passing through any two edges in the captured image and the other edge. A candidate straight line, which is a straight line, is calculated as a temporary distant track shape, and
The normality determination unit compares the estimated long-distance track shape with the temporary long-distance track shape, and considers the angle and position between the estimated long-distance track shape and the temporary long-distance track shape. When at least one of the differences is greater than or equal to the predetermined difference, it is determined that the temporary long-distance track shape is not normal, and when it is smaller than the predetermined difference, the temporary long-distance track shape is determined to be normal. Judgment is made, and the provisional long-distance track shape is calculated as the long-distance runway shape.
Track recognition device. A track recognition device (100, 100a to 100k) mounted on a vehicle and recognizing the track of the vehicle.
A neighborhood shape calculation unit (101) that repeatedly calculates a neighborhood track shape, which is a track shape in the vicinity of the vehicle, based on images repeatedly obtained by the image pickup device (21) mounted on the vehicle.
A processing area (Er1 to Er4, El1 to El4) in each captured image is set based on the calculated nearby track shape, and a distant track shape which is a distant track shape of the vehicle in the processing area is repeatedly calculated. Shape calculation unit (102) and
A neighborhood shape storage unit (103) that stores the calculated neighborhood track shape as a history in chronological order,
A distant shape storage unit (104) that stores the calculated distant track shape as a history in chronological order, and a distant shape storage unit (104).
With
When the near track shape is not calculated, the far shape calculation unit sets the processing area using at least one of the history of the near track shape and the history of the far track shape.
A normality determination unit (118) for determining the normality of the distant track shape to be calculated is further provided.
The distant shape calculation unit does not calculate the distant track shape when it is determined that the calculated distant track shape is not normal.
The far shape calculation unit has the lowest median error frequency in the error distribution based on the error distribution obtained by LMedS based on the error between the straight line passing through any two edges in the captured image and the other edge. A candidate straight line, which is a straight line, is calculated as a temporary distant track shape, and
The normality determination unit compares the near track shape calculated this time with the temporary far track shape, and determines the angle and position between the near track shape and the temporary far track shape. When the difference of at least one of them is equal to or greater than the predetermined difference, it is determined that the temporary long-distance track shape is not normal, and when it is smaller than the predetermined difference, the temporary long-distance track shape is normal. It is determined that there is, and the provisional long-distance track shape is calculated as the long-distance track shape.
Track recognition device. In the track recognition device according to claim 19,
A track estimation unit (119) for estimating the estimated distant track shape, which is the distant track shape this time, is further provided based on the history of the distant track shape.
The normality determination unit compares a straight line passing through any two edges in the captured image with the estimated far track shape, and determines the angle and position between the straight line and the estimated far track shape. The smaller the difference between at least one of the two, the smaller the evaluation score, which is the frequency of the median error in the error distribution for the straight line.
Track recognition device.

Images