KR20180075985A - Apparatus for autonomous steering prediction considering driving environment and method thereof - Google Patents

Abstract

The present invention relates to a driving environment-sensitive autonomous steering prediction apparatus and a method thereof. The driving environment-sensitive autonomous steering prediction apparatus includes a basic steering value predicting unit for extracting a feature from an image photographed by a camera mounted on a driving vehicle and predicting a steering value by using the extracted feature; a situation determination unit for determining the driving environment specificity of the driving vehicle by analyzing the driving information on the driving vehicle at a previous time point and the driving environment recognition result at a current time point; a steering value correcting unit for correcting the predicted steering value in consideration of an environmental factor corresponding to the driving environment specificity; and a control unit for extracting a reliability value of the predicted steering value and comparing the reliability value with a predetermined threshold value, wherein the control unit outputs the predicted steering value when the reliability value is greater than the threshold value, and outputs a corrected steering value when the reliability value is smaller than the threshold value. According to the present invention, it is possible to predict and provide a steering value suitable for an actual driving environment and capable of flexibly coping with an unexpected situation by correcting the steering value in consideration of the state information on the driving vehicle and the external driving environment.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an autonomous steering system,

The present invention relates to an apparatus and method for predicting a traveling environment-sensitive autonomous steering, and more particularly, to a traveling environment-sensitive autonomous-steering prediction apparatus and method for predicting a steering value in consideration of traveling information and a traveling environment of a traveling vehicle .

A self-driving car is a car that automatically recognizes the condition of a road without the driver's operation. Autonomous vehicles have a number of advantages, including saving drivers time, reducing accidents behind the road, reducing fuel use and reducing pollutant emissions.

In order for such an autonomous vehicle to be put to practical use, it should be possible to make a quick judgment for the next driving stage in various driving situations in terms of stability and to derive the best driving result. Therefore, the technology to autonomously control the steering wheel steering, which is classified as basic autonomous driving function in the highway and the urban environment, requires environmental awareness and flexibility of situational coping.

Current autonomous steering control technology detects and recognizes lanes using cameras or optical sensors such as LiDAR (LiDAR) using near-infrared rays, and calculates the preferred steering angle of the traveling vehicle based on the detection. For this, a lateral control model is used by using information (steering angle, speed, etc.) of the current driving vehicle, and a radar (RADAR) stronger to the outside environment than the camera is used in order to detect the preceding vehicle and speed up and down.

This method is specialized for lane-following autonomous steering control and provides stable driving performance in a typical driving environment.

However, since the actual driving environment is exposed to various environments such as weather, road conditions, seasons, day and night, it is difficult to assume a normal environment level in order to guarantee complete autonomous driving.

Therefore, there is a demand for a technique for predicting and correcting the steering angle in response to environmental conditions such as the actual driving environment.

The technology which is the background of the present invention is disclosed in Korean Patent No. 10-1490905 (published on Feb. 20, 2015).

An object of the present invention is to provide a driving environment-sensitive autonomous steering prediction apparatus and method for predicting a steering value in consideration of driving information and driving environment of a driving vehicle.

According to an embodiment of the present invention, a traveling environment-sensitive autonomous-steer-type predicting device is provided for extracting a characteristic point from an image of a camera mounted on a driving vehicle, and estimating a steering value using the extracted characteristic point The steering angle estimating unit may include a situation determining unit for determining the driving environment specificity of the driving vehicle by analyzing the driving information of the driving vehicle and the driving environment recognition result at the current point of time at the previous point of time, Calculating a reliability value of the predicted steering value by comparing the reliability value with a preset threshold value and if the reliability value is greater than the threshold value, And outputs a corrected steering value when the reliability value is smaller than the threshold value .

The situation determination unit may analyze an image photographed through a camera mounted on the driving vehicle to recognize environmental conditions by classifying the image according to a road surface state, a forward visibility state, a traveling road type, a driving neighboring object, have.

The situation determination unit can determine the driving environment specificity by applying the internal information including the steering angle and speed of the traveling vehicle and the environmental condition recognized by the classification to the Deep Neural Network.

The control unit may determine whether the correction value of the corrected steering value is included in the reference range, and when the correction value is included in the reference range, the corrected steering value may be determined by an ECU (Electronic Control Unit) And outputs the predicted steering value to an ECU (Electronic Control Unit) of the traveling vehicle or electric power steering when the correction value is not included in the reference range.

The basic steering value predicting unit estimates the driving vehicle lane using the extracted characteristic points and estimates the position of the driving vehicle and predicts the steering value from the steering angle value or the steering angle value from the photographing image through a learning algorithm including a deep- Can be predicted.

According to another embodiment of the present invention, there is provided a method of predicting autonomous steering using an autonomous steering system of a traveling environment-sensitive autonomous steering system, comprising the steps of: extracting a feature point from an image of a camera mounted on a driving vehicle; Calculating a reliability value of a predicted steering value, comparing the reliability value with a predetermined threshold value, and outputting a predicted steering value if the reliability value is greater than the threshold value, Determining a driving environment specificity of the driving vehicle by analyzing driving information of the driving vehicle and a current driving environment recognition result at a previous time point if the driving environment specificity is smaller than a threshold value, Correcting the predicted steering value, and outputting the corrected steering value.

According to the present invention, it is possible to predict and provide a steering value that is suitable for an actual driving environment and can be flexibly coped with in an unexpected situation by correcting the steering value in consideration of the state information of the traveling vehicle and the external driving environment.

In addition, according to the present invention, the ability to determine the autonomous driving of a traveling vehicle is improved through continuous learning using a deep network model.

FIG. 1 is a configuration diagram showing a traveling environment-sensitive autonomous steering prediction system according to an embodiment of the present invention.
FIG. 2 is a configuration diagram of a traveling environment-sensitive autonomous-steering prediction apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating a traveling environment-sensitive autonomous steering prediction method according to an embodiment of the present invention.
4 is an exemplary view showing various driving environments according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention.

First, an autonomous steering prediction system using a traveling environment-sensitive autonomous steering prediction apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a configuration diagram showing a traveling environment-sensitive autonomous steering prediction system according to an embodiment of the present invention.

1, the autonomous steering prediction system includes a camera 100, a traveling environment sensitive autonomous steering prediction device 200, a vehicle sensor 300, and an ECU / MDPS 400.

First, the camera 100 is mounted on the vehicle and photographs the surrounding environment of the vehicle including the front of the vehicle. At this time, at least one camera may be separately installed in the vehicle 100, and may represent a camera of a device such as a navigation device or a black box mounted on the vehicle.

The driving environment-sensitive autonomous steering apparatus 200 receives a photographed image from the camera 100, detects a lane, and predicts a steering value according to the detected lane direction

The driving environment-sensitive autonomous steering apparatus 20 derives a reliability value for the predicted steering value, compares the reliability value with a preset threshold value, and confirms whether the predicted steering value is suitable for the current situation.

If the reliability value of the predicted steering value is greater than a predetermined threshold value, the traveling environment-sensitive autonomous-steerable prediction apparatus 200 outputs the predicted steering value to a device for controlling the moving direction of the vehicle. On the other hand, if the reliability value for the predicted steering value is less than or equal to the preset threshold, the driving environment-sensitive autonomous steering apparatus 200 analyzes the driving information of the driving vehicle and the driving environment recognition result at the present time Thereby determining the driving environment specificity. The traveling environment-sensitive autonomous steering apparatus 200 corrects the predicted steering value in consideration of environmental factors corresponding to the driving environment specificity, and outputs the corrected steering value to a device for controlling the moving direction of the vehicle.

The vehicle sensor 300 represents a sensor and a device mounted on the vehicle for measuring information on the outside or inside of the vehicle.

For example, the vehicle sensor 300 may be a radar sensor, a radar sensor, a sensor for determining the road surface condition, a rear sensor, an ultrasonic sensor, a height sensor, a vehicle speed sensor (VSS) (Electronic Control Unit, ECU), and Electronic Stability Program (ESP).

The ECU / MDPS 400 represents a device for controlling the moving direction of the vehicle. Here, the electronic control unit (ECU) represents an electronic control unit that controls the state of an engine of an automobile, an automatic transmission, and an anti-lock brake system (ABS) by a computer, and includes an electric power steering , MDPS) refers to a device that assists in steering a handle with the help of an electric motor. MDPS can provide optimum steering power at low and high speeds depending on the speed so that it moves lightly at low speeds and heavily at high speeds.

For convenience of explanation, the ECU / MDPS is referred to as a device for controlling the moving direction of the vehicle by applying the steering value of the vehicle. However, the present invention is not limited to the specific device.

Next, FIG. 2 is a configuration diagram of a traveling environment-sensitive autonomous-steering prediction apparatus according to an embodiment of the present invention.

2, the traveling environment-sensitive autonomous steering apparatus 200 includes a basic steering value predicting unit 210, a situation determining unit 220, a steering value correcting unit 230, and a control unit 240 do.

First, the basic steering value predicting unit 210 may extract feature points from the photographed image, detect driving lanes using the extracted feature points, estimate the position of the driving vehicle, and calculate the following steering values.

Alternatively, the basic steering value predicting unit 210 may predict the next optimum steering value from the shot image using the end-to-end learning method. In this case, the end-to-end learning method can predict a steering value by applying a learning algorithm such as a deep-nerve network (DNN) from running data without performing a separate lane detection and a steering angle calculation process from the shot image.

The situation determination unit 220 analyzes the photographed image of the camera 100 or recognizes the state of the road surface and the traveling environment through the sensor 300 mounted on the vehicle. In addition, the situation determination unit 220 can recognize the control state of the traveling vehicle at the current time point by receiving the internal driving information of the traveling vehicle. Accordingly, the situation determination unit 220 can determine the traveling environment specificity by recognizing the traveling environment of the vehicle and the traveling environment of the vehicle.

Next, the steering value correcting unit 230 corrects the predicted steering value in consideration of the environmental factors corresponding to the driving environment specificity determined by the situation determining unit 220. [

The control unit 240 derives the reliability value of the predicted steering value, and outputs the predicted steering value to the electronic control unit (ECU) or the electric power steering (MDPS) of the vehicle if the reliability value is greater than a preset threshold value.

If the reliability value of the predicted steering value is equal to or smaller than a predetermined threshold value, the controller 240 outputs the corrected steering value in consideration of the external environment factor to the electronic control unit ECU or the electric power steering (MDPS) .

At this time, the controller 240 determines whether the corrected steering value corresponds to the normal range, and then outputs the corrected steering value only when the corrected steering value corresponds to the normal range. If the corrected steering value does not correspond to the normal range, And outputs the steering value predicted by the steering angle sensor 210.

Hereinafter, a driving environment-sensitive autonomous steering prediction method according to various driving environments will be described with reference to FIGS. 3 and 4. FIG.

FIG. 3 is a flow chart illustrating a method of predicting a driving environment-sensitive autonomous steering according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating various driving environments according to an embodiment of the present invention.

First, the basic steering predictor 210 extracts the feature points from the photographed images of the camera mounted on the driving vehicle, and predicts the steering values using the extracted feature points (S310). At this time, the basic steering predictor 210 detects the driving lane using the extracted feature points, estimates the position of the driving vehicle, and predicts the next steering value accordingly.

Alternatively, the basic steering predictor 210 may predict a steering value at a next time point from a photographed image through a learning algorithm including a depth-of-field network (DNN).

Meanwhile, the basic steering predictor 210 estimates one steering value by using both methods or predicts the steering values through two methods, and the controller 240 derives the reliability values of the respective estimated steering values, The predicted steering value of the predicted method with a higher value can be selected.

Next, the control unit 240 derives the reliability value of the predicted steering value (S320) and compares the reliability value with a predetermined threshold value (S330).

Here, the reliability represents the detection accuracy of the lane. For example, if lane markings are incomplete due to lane covered with snow or ice, or cracks in the lane or difficulty in securing an anterior visual field, the reliability value is lowered.

The criterion and threshold value for measuring such reliability can be changed and set later by the user.

If the reliability value is greater than the threshold value, the control unit 240 outputs the estimated steering value (S340).

If the reliability value is greater than the threshold value, the control unit 240 determines that the predicted steering value is suitable for the road condition, and controls the moving direction of the vehicle according to the steering value, (MDPS).

If the reliability value is smaller than the threshold value, the situation determination unit 220 determines the driving environment specificity of the driving vehicle by analyzing the driving information of the driving vehicle and the driving environment recognition result at the present time (S350) .

The situation determination unit 220 collects driving information such as a steering value and a speed of the driving vehicle at a previous point of time and notifies the state of the driving road surface, the size of the road, , Weather, visibility, and the like.

As shown in FIG. 4, various road conditions vary depending on season, weather, and situation even if the driving environment is the same road. That is, even if the same steering value is applied according to the road condition, the degree of movement of the actual vehicle is changed.

Therefore, the situation determination unit 220 analyzes the data collected through the camera 100 and the sensor 300 mounted on the traveling vehicle to determine whether the traveling state of the traveling road, the front view state, the traveling road type, It recognizes environmental conditions by classifying by obstacle distribution.

In more detail, the situation determination unit 220 may be classified into four types of detailed key recognition classifiers such as a road surface state recognition classifier, a roadway type recognition classifier, a forward visible state classifier, a driving neighbor object, or an obstacle recognition classifier The traveling environment can be recognized.

First, the situation determination unit 220 determines the state of the material applied on the road surface due to the clearness of the road marking state of the road surface, the road surface flatness and the state of the bouncing chute or hole, road surface moisture, snow, dust, And the degree.

Then, the situation determination unit 220 determines whether the lane of the road on which the vehicle is currently traveling is a multi-lane or a multi-lane or a lane on which the current vehicle is traveling through the traveling road type recognition classifier, The type of environment surrounding the road.

Next, the situation determination unit 220 recognizes the visibility of the forward view including the fog, the daytime solar light condition, the nighttime illumination condition, and the rainy condition through the forward visibility recognition classifier.

The situation determination unit 220 classifies the objects adjacent to the left, right, front and rear sides of the traveling vehicle through the running neighboring object or the obstacle recognition classifier, grasps the position of the nearby object, Recognize lanes alternatives in driving directions.

This recognition classifier is classified as a metrological classification for easy learning of Deep Neural Network (DNN). It can be appropriately re-classified according to the design of the neural network model, have.

In other words, in the embodiment of the present invention, there are two hierarchical classification hierarchies classified into four types of upper hierarchical classifiers and various sub-hierarchical classification groups of each detail, but later the user classifies the upper hierarchical classifiers in a single hierarchical structure Alternatively, the classification group may be further layered into three or more hierarchical levels rather than a two-level hierarchical level. In addition, the number of cases of lower hierarchical classification of the driving environment details may be added or reduced depending on the complexity of the driving environment scenario.

In this manner, the driving determination unit 220 determines the driving environment specificity by applying the internal information including the steering angle and speed of the driving vehicle and the environmental condition recognized by the classification to the DNN.

That is, the driving environment specificity may include both the external driving environment and the internal driving information.

Next, the steering value correcting unit 230 corrects the predicted steering value in consideration of environmental factors corresponding to the driving environment specificity (S360).

The steering value correcting unit 230 can correct the predicted steering value corresponding to the environmental factor using the data obtained by predicting and applying the steering value of the vehicle in various driving environments.

Next, the control unit 240 determines whether the correction value of the corrected steering value is included in the reference range (S370).

That is, the control unit 240 can determine whether the correction value, which is a difference value between the predicted steering value and the corrected steering value, is included in the normal range when considering the environmental factor.

If the correction value is included in the reference range, the control unit 240 outputs the corrected steering value to the electronic control unit ECU or the electric power steering (MDPS) (S380).

If the correction value is not included in the reference range, the controller 240 determines that the corrected steering value is not normal and outputs the estimated steering value to the electronic control unit (ECU) or the electric power steering (MDPS) of the traveling vehicle .

As described above, according to the embodiment of the present invention, the steering value is corrected in consideration of the state information of the traveling vehicle and the external traveling environment, so that it is possible to predict and provide a steering value that is suitable for the actual traveling environment and can be flexibly coped with in an unexpected situation .

In addition, according to the embodiment of the present invention, the self-driving determination ability of the traveling vehicle is improved through continuous learning using a deep network (DNN) model.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

100: camera 200: traveling environment-sensitive autonomous steering prediction device
210: basic steering prediction unit 220:
230: steering value correcting unit 240:
300: vehicle sensor 400: ECU / MDPS

Claims (10)

A basic steering value predicting unit for extracting a characteristic point from a photographed image of a camera mounted on a driving vehicle and predicting a steering value using the extracted characteristic point,
A situation determination unit for determining the driving environment specificity of the driving vehicle by analyzing the driving information of the driving vehicle and the driving environment recognition result at the present time,
A steering value correcting unit for correcting a predicted steering value in consideration of an environmental factor corresponding to the traveling environment specificity, and
Derives a reliability value of the predicted steering value, compares the reliability value with a preset threshold value, and outputs a predicted steering value if the reliability value is greater than a threshold value, and if the reliability value is smaller than the threshold value And a controller for outputting the corrected steering value. The method according to claim 1,
The situation determination unit may determine,
An image captured through a camera mounted on the traveling vehicle is analyzed to classify the traveling condition of the traveling road according to the road surface state, forward visibility state, running road type, running adjacent object or obstacle distribution, Prediction device. 3. The method of claim 2,
The situation determination unit may determine,
Wherein the driving environment specificity is determined by applying the internal information including the steering angle and the speed of the driving vehicle to the Deep Neural Network. The method according to claim 1,
Wherein,
And determines whether the corrected value of the corrected steering value is included in the reference range. If the corrected value is included in the reference range, the corrected steering value is outputted to the ECU (Electronic Control Unit) of the traveling vehicle or the electric power steering And outputs the predicted steering value to an electronic control unit (ECU) or electric power steering of the driving vehicle when the correction value is not included in the reference range. The method according to claim 1,
Wherein the basic steering value predicting unit comprises:
A traveling lane detecting unit for detecting a driving lane using the extracted feature points, estimating a position of the driving vehicle, predicting a steering value in advance, or estimating a steering value from a photographed image through a learning algorithm including a deep- Autonomous steering prediction device. A method for predicting autonomous steering using a traveling environment-sensitive autonomous steering prediction apparatus,
Extracting a feature point from a photographed image of a camera mounted on the driving vehicle, and estimating a steering value using the extracted feature point,
Deriving a reliability value of a predicted steering value, comparing the reliability value with a predetermined threshold value, and outputting a predicted steering value if the reliability value is greater than a threshold value,
Determining the driving environment specificity of the driving vehicle by analyzing the driving information of the driving vehicle and the driving environment recognition result at the current time when the reliability value is smaller than the threshold value,
Correcting the predicted steering value in consideration of environmental factors corresponding to the driving environment specificity, and
And outputting the corrected steering value. The method according to claim 6,
The step of determining the traveling environment specificity comprises:
An image captured through a camera mounted on the traveling vehicle is analyzed to classify the traveling condition of the traveling road according to the road surface state, forward visibility state, running road type, running adjacent object or obstacle distribution, Prediction method. 8. The method of claim 7,
The step of determining the traveling environment specificity comprises:
Wherein the driving environment specificity is determined by applying the internal information including the steering angle and the speed of the driving vehicle to the Deep Neural Network. The method according to claim 6,
Wherein the step of outputting the corrected steering value comprises:
And determines whether the corrected value of the corrected steering value is included in the reference range. If the corrected value is included in the reference range, the corrected steering value is outputted to the ECU (Electronic Control Unit) of the traveling vehicle or the electric power steering And outputting the predicted steering value to an electronic control unit (ECU) or electric power steering of the driving vehicle when the correction value is not included in the reference range. The method according to claim 6,
The step of predicting the steering value includes:
A traveling lane detecting unit for detecting a driving lane using the extracted feature points, estimating a position of the driving vehicle, predicting a steering value in advance, or estimating a steering value from a photographed image through a learning algorithm including a deep- Autonomous steering prediction method.

Images