WO2023167345A1 - Autonomous driving system and autonomous driving method capable of responding to traffic signal recognition failure - Google Patents

Abstract

The present invention relates to an autonomous driving method comprising the steps of: attempting to recognize a traffic signal from signal data received through a sensor; outputting a notification signal according to unclear recognition of the traffic signal; controlling the deceleration of an autonomous vehicle and recording an internal image of the autonomous vehicle, according to the output of the notification signal; receiving an input for the notification signal; and changing from deceleration control to autonomous driving control according to the input.

Description

Autonomous driving system and method for responding to traffic sign recognition failure

The present invention relates to an autonomous driving system and an autonomous driving method capable of responding to traffic signal recognition failure, and more specifically, when traffic signal recognition fails during autonomous driving, braking control is automatically performed and autonomous driving is performed according to traffic signal confirmation by a user. The present invention relates to an autonomous driving system and an autonomous driving method capable of responding to a failure of traffic sign recognition that can continue and record the situation.

Famous automobile companies around the world are focusing on research and development to create self-driving vehicles. For self-driving, artificial intelligence technology (deep learning technology) is most often used. For self-driving through deep learning technology, self-driving vehicles are equipped with numerous sensors and enable self-driving through the application of new technologies.

Self-driving vehicles utilize deep learning technology to distinguish roads, vehicles, people, motorcycles, road signs, and traffic lights (vehicle lights) and use them as data necessary for autonomous driving. In autonomous driving, if an object, object, or person that needs to be identified is not properly identified, it can lead to an accident, so the discrimination rate and reliability of discrimination of roads, vehicles, people, motorcycles, road signs, and traffic lights (vehicle lights) are very high. It is an important factor, and automobile companies are striving for research and development to increase this discrimination rate.

In particular, if an autonomous vehicle does not properly recognize a traffic light or a signal of a traffic light (hereinafter collectively referred to as a 'traffic signal'), it is highly likely to lead to a large-scale disaster. Recognition of traffic signals requires very high reliability, and if you misjudge it at a crossroad and enter while maintaining the driving speed, it can harm your life.

In order to prevent this risk, a project to prevent misrecognition of traffic signals through two-way communication between vehicles and roads by introducing C-ITS (Cooperative-Intelligent Transport System) is currently underway. However, before the establishment of this system or in places where this system is not applied, autonomous vehicles are still exposed to the risk of not properly recognizing traffic signals.

When autonomous vehicles fail to recognize traffic signs, automakers face serious legal problems. A specific vehicle manufacturer is mass-producing and selling a second-level autonomous driving model, and requires the driver to hold the steering wheel, forcing the driver to take responsibility in the event of an accident.

In this way, if an autonomous vehicle fails to recognize a traffic signal, it notifies the driver in advance so that the driver can be identified and subsequent autonomous driving continues, and it is preserved as evidence to prevent future legal disputes or traffic that can be used as evidence. There is a need for an autonomous driving system and autonomous driving method capable of responding to signal recognition failure.

The present invention has been made to solve the above-mentioned problems, and an autonomous driving system and autonomous driving capable of providing safe operation of an autonomous vehicle and continuing autonomous driving according to driver's confirmation when traffic signal recognition fails during autonomous driving. Its purpose is to provide a method.

In addition, an object of the present invention is to provide an autonomous driving system and an autonomous driving method capable of increasing a traffic signal recognition rate during autonomous driving by simultaneously using a lidar sensor and a camera sensor.

In addition, the present invention provides an autonomous driving system and autonomous driving method capable of improving the recognition rate of traffic signals according to the driver's driving environment by learning an artificial intelligence algorithm for traffic signal recognition using driver feedback for traffic signals. It has its purpose.

An autonomous driving method according to an aspect of the present invention includes attempting to recognize a traffic signal from signal data received through a sensor; outputting a notification signal according to unclear recognition of a traffic signal; Controlling the deceleration of the self-driving vehicle and recording an internal image of the self-driving vehicle according to the output of the notification signal; Receiving an input for the notification signal; and changing from the deceleration control to autonomous driving control according to the input.

In the above-described autonomous driving method, the step of attempting to recognize the traffic signal attempts to recognize a traffic light by using lidar signal data received through a lidar sensor and image signal data received through an image sensor, or the traffic light Try to recognize the output signal of

In the autonomous driving method described above, the step of attempting to recognize the traffic signal attempts to recognize a traffic light from lidar signal data received through a lidar sensor, and when the recognition of the traffic light is successful, the image signal received through the image sensor An attempt is made to recognize an output signal from data, and the LIDAR sensor and the image sensor are sensors that sense signals in an area in the same direction.

In the autonomous driving method described above, the step of outputting the notification signal outputs a voice message indicating a vehicle passage permission inquiry through a speaker and simultaneously outputs a message indicating the vehicle passage permission inquiry through a display.

In the autonomous driving method described above, in the receiving of the input, the vehicle passage permission input is received as a voice signal through a microphone or the vehicle passage permission input is received through a touch of a display.

The autonomous driving method described above further includes recognizing the traffic signal using the signal data and the input and learning an artificial intelligence algorithm installed in the autonomous vehicle according to the recognition of the traffic signal.

In the autonomous driving method described above, the step of learning the artificial intelligence algorithm outputs image signal data received through an image sensor to a display, recognizes a traffic light as a selection input on the display, and in the area of the recognized traffic light Recognizes the output signal as the image signal data of the recognized traffic light area data, the recognized output signal data and the image signal data of the image sensor are provided as learning data for the artificial intelligence algorithm, or the image signal data on which the traffic light is displayed Outputs to a display, recognizes the output signal as an input to the displayed traffic light, and provides area data of the traffic light, recognized output signal data, and image signal data of the image sensor as learning data for the artificial intelligence algorithm.

In addition, an autonomous driving system according to an aspect of the present invention includes a lidar sensor; a first image sensor; speaker; display; a second image sensor that photographs the inside of the self-driving vehicle; vehicle control module; and receiving signal data from the lidar sensor or the first image sensor, attempting to recognize a traffic signal from the received signal data, outputting a notification signal to the speaker and the display according to unclear recognition of the traffic signal, and controlling the vehicle. and an autonomous driving control module that performs deceleration control for the autonomous vehicle in conjunction with the module and records an image of the second image sensor, wherein the autonomous driving control module receives an input for the notification signal. The deceleration control is changed to autonomous driving control through control of the vehicle control module.

In the autonomous driving system described above, the self-driving control module attempts to recognize a traffic light from lidar signal data received through the lidar sensor, and when the traffic light is successfully recognized, the image received through the first image sensor An attempt is made to recognize an output signal from signal data, and the lidar sensor and the first image sensor mounted outside the self-driving vehicle are sensors that sense signals in an area in the same direction.

In the autonomous driving system described above, the autonomous driving control module outputs a voice message indicating a vehicle passage permission inquiry through the speaker and simultaneously outputs a message indicating the vehicle passage permission inquiry through the display.

In the autonomous driving system described above, an autonomous driving control module equipped with an artificial intelligence algorithm and attempting to recognize a traffic signal according to the artificial intelligence algorithm learns the artificial intelligence algorithm using the signal data and the input.

In the self-driving system, the self-driving control module outputs image signal data received through the first image sensor to the display, recognizes a traffic light as a selection input on the display, and displays the recognized traffic light. Recognizes an output signal as image signal data in an area and provides the area data of the recognized traffic light, the recognized output signal data, and the image signal data of the first image sensor as learning data for the artificial intelligence algorithm, or The displayed image signal data is output to the display, the output signal is recognized as an input to the displayed traffic light, and the area data of the traffic light, the recognized output signal data, and the image signal data of the first image sensor are converted into the artificial intelligence algorithm. provided as learning data.

The self-driving system and method according to the present invention as described above has an effect of providing safe driving of the self-driving vehicle and continuing autonomous driving according to driver's confirmation when traffic signal recognition fails during self-driving.

In addition, the autonomous driving system and the autonomous driving method according to the present invention as described above have an effect of increasing the recognition rate of traffic signals during autonomous driving by simultaneously using a lidar sensor and a camera sensor.

In addition, the autonomous driving system and autonomous driving method according to the present invention as described above can improve the recognition rate of traffic signals according to the driver's driving environment by learning an artificial intelligence algorithm for traffic signal recognition using driver feedback on traffic signals. There are possible effects.

The effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a diagram illustrating an exemplary block diagram of an autonomous driving system.

2 is a diagram illustrating a control flow for controlling an autonomous vehicle according to traffic signal recognition.

3 is a diagram illustrating a specific control flow for recognizing a traffic signal using a lidar sensor and a first image sensor.

The above objects, features and advantages will become more clear through the detailed description described later in detail with reference to the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs will understand the technical spirit of the present invention. can be easily carried out. In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an exemplary block diagram of an autonomous driving system 10 .

Referring to FIG. 1 , an autonomous driving system 10 includes a display 100, an input interface 200, a lidar sensor 300, one or more image sensors 400 and 500, a GPS module 600, and a speaker 700. , A vehicle control module 800 and an autonomous driving control module 900 are included. According to design examples, modifications, and other specific implementations, the autonomous driving system 10 may further include other sensors or control modules.

The autonomous driving system 10 according to the present invention is mounted in an autonomous vehicle and controls autonomous driving of the vehicle using signals sensed through various sensors.

Looking briefly at the components of the autonomous driving system 10 through FIG. 1 , the display 100 outputs various images or videos. The display 100 is installed on a dashboard in front of a driver's wheel or a center fascia of an autonomous vehicle to output various types of information or data visually recognizable by a user such as a driver. The display 100 may include an LCD display-module or an LED display-module.

The input interface 200 receives driver (or user) input. The input interface 200 may include a microphone and/or a touch screen to receive a voice signal from a microphone or a touch input from a touch screen coupled to the display 100 .

The lidar sensor 300 is mounted outside the self-driving vehicle (eg, on the upper outer roof of the self-driving vehicle) and senses the lidar signal. The lidar sensor 300 may emit light-based lidar (eg, laser) signals, sense the emitted lidar signals, and output the signals to the autonomous driving control module 900 . The lidar sensor 300 mounted on the roof of the self-driving vehicle makes it possible to recognize an object and its distance in all 360 directions outside the self-driving vehicle. Alternatively, the fixed lidar sensor 300 enables recognizing an object in front of the autonomous vehicle and its distance.

The autonomous driving system 10 includes one or more image sensors 400 and 500 (or camera sensors), and one image sensor 400 (hereinafter referred to as a 'first image sensor') detects the outside of the autonomous vehicle. It is photographed and the photographed image signal is output to the autonomous driving control module (900).

Preferably, the first image sensor 400 is a sensor capable of sensing an image of an area in the same direction (advancing direction) as the lidar sensor 300 . The first image sensor 400 is positioned adjacent to, in front of, or above the lidar sensor 300 and rotates 360 degrees or is fixed like the lidar sensor 300 in the same direction as the direction the lidar sensor 300 senses. The area of is sensed at the same viewpoint.

Another image sensor 500 (hereinafter referred to as 'second image sensor') is mounted inside the self-driving vehicle, takes pictures of the interior of the self-driving car, and outputs the captured image signal to the self-driving control module 900. The second image sensor 500 may capture a driver's motion (eg, pressing a button, pressing an accelerator, etc.) and output a captured image signal.

The GPS module 600 captures GPS signals and outputs them. The GPS module 600 may sense a signal from one or more GPS satellites and output a GPS signal indicating the detected signal to the autonomous driving control module 900 .

The speaker 700 converts an electrical signal, which is an audio signal, into an audible vibration signal. The speaker 700 is connected to the autonomous driving control module 900 and converts electrical signals of various information and inquiries into vibration signals and outputs them.

The vehicle control module 800 controls various devices inside the vehicle. The vehicle control module 800 controls the performance of functions such as acceleration, braking, shifting, and steering of the autonomous vehicle. The vehicle control module 800 may control driving of the autonomous vehicle according to a control signal received from the autonomous driving control module 900 .

The autonomous driving control module 900 controls driving of the autonomous vehicle using various sensing signals. The self-driving control module 900 receives signal data sensed by the lidar sensor 300 and/or the first image sensor 400, attempts to recognize a traffic signal from the received signal data, and determines whether the recognition of the traffic signal is unclear. Accordingly, a notification signal is output to the speaker 700 or the display 100 .

In addition, the self-driving control module 900 performs deceleration control for the self-driving vehicle in conjunction with the vehicle control module 800 (subsequently or together) following the output of the notification signal, and the internal second image sensor 500 Record the image of the image signal data received from the internal storage medium (hard disk or non-volatile memory, etc.).

The autonomous driving control module 900 stores artificial intelligence algorithms (programs) in an internal storage medium (hard disk or non-volatile memory) and recognizes traffic signals at least from signal data from sensors through the execution of artificial intelligence algorithms. Try it. The program of the artificial intelligence algorithm may be a program based on deep learning technology.

The autonomous driving control module 900 includes a display 100, a lidar sensor 300, an input interface 200, a first image sensor 400, a second image sensor 500, a speaker 700, a GPS module ( 600), a vehicle control module 800, etc., and a signal interface or bus interface for transmitting and receiving data or signals are provided, and various data or signals can be transmitted and received.

A detailed control flow of the autonomous driving control module 900 corresponding to traffic signal recognition failure will be reviewed in FIGS. 2 and 3 .

2 is a diagram illustrating a control flow for controlling an autonomous vehicle according to traffic signal recognition.

The control flow of FIG. 2 is performed by an autonomous vehicle and is preferably controlled by the autonomous driving control module 900 of the autonomous driving system 10 .

First, the self-driving vehicle performs autonomous driving based on signal data from various external sensors (S100), and the autonomous vehicle senses signals through the sensors (S200).

The lidar sensor 300 periodically senses lidar signals according to a set period or rotation period and outputs the sensed lidar signals to the autonomous driving control module 900. In addition, the first image sensor 400 capable of sensing an image of an area in the same direction as the lidar sensor 300 adjacent to the lidar sensor 300 transmits an external image through a lens provided to the lidar sensor 300. It captures at the same time point as the sensing time point, and outputs the captured image signal to the autonomous driving control module 900.

The self-driving vehicle tries to recognize a traffic signal from signal data received through a sensor (S300).

The autonomous driving control module 900 receives lidar signal data from the lidar sensor 300 and image signal data from the first image sensor 400 . LIDAR signal data is composed of LIDAR signals, and image signal data can be composed of image signals.

The self-driving control module 900, which attempts to recognize traffic signals through the execution of an artificial intelligence algorithm, lidar signal data received through the lidar sensor 300 and image signal data received through the first image sensor 400 Attempting to recognize a traffic light or recognizing an output signal of a recognized traffic light (for example, a vehicle passage permission signal (green signal) or a vehicle passage prohibition signal (red signal)) using . Traffic lights and their output signals may be collectively referred to as 'traffic signals'.

FIG. 3 is a diagram illustrating a specific control flow for recognizing a traffic signal using the lidar sensor 300 and the first image sensor 400 .

The control flow of FIG. 3 shows a specific example of the process S300 of FIG. 2 . The control flow of FIG. 3 is performed by the autonomous driving control module 900 and is preferably performed through an artificial intelligence algorithm that recognizes traffic signals.

First, the autonomous driving control module 900 attempts to recognize traffic lights from lidar signal data received from lidar sensor 300 (S301). The artificial intelligence algorithm that has learned the traffic light may output a recognition result including reliability (eg, discrimination rate) of the traffic light from LIDAR signal data.

According to the recognition attempt, the autonomous driving control module 900 determines whether a traffic light in the direction clearly exists in LIDAR signal data (S303). The autonomous driving control module 900 may determine whether a traffic light clearly exists based on whether the reliability probability output from the artificial intelligence algorithm is equal to or greater than a set threshold value (first threshold value). If the reliability probability is less than another set threshold (the second threshold significantly lower than the first threshold), the autonomous driving control module 900 may set the traffic light as non-existence and end the control flow of FIG. 3 .

When the probability of reliability calculated by the artificial intelligence algorithm is located between two thresholds and the presence or absence of a traffic light is unclear, the autonomous driving control module 900 calculates the traffic light from the image signal data from the first image sensor 400. The recognition of is reattempted (S313).

The self-driving control module 900 inputs image signal data from the first image sensor 400, which is located adjacent to the lidar sensor 300 and captures an image of an area in the same direction, into an artificial intelligence algorithm to detect traffic lights. Recognition results including reliability are received from the artificial intelligence algorithm.

Thereafter, the autonomous driving control module 900 determines whether a traffic light clearly exists in the image signal data (S315). The autonomous driving control module 900 may determine whether a traffic light clearly exists based on whether the reliability probability output from the artificial intelligence algorithm is greater than or equal to a set threshold.

In this way, the self-driving control module 900 according to the present invention attempts recognition of a traffic light again by using an image sensor looking in the same direction when recognition of a traffic light through the lidar sensor 300 fails. Accordingly, the recognition failure rate according to the surrounding environment, lighting, and brightness may be improved.

When the autonomous driving control module 900 determines that the traffic light clearly exists from the image signal, it can transition to step S305 and perform subsequent steps. If the reliability probability is less than the set threshold value, the autonomous driving control module 900 sets the recognition of the traffic light as unclear (S317) and ends the control flow of FIG. 3 .

Upon successful recognition of the traffic light according to the determination of whether the traffic light is clear, the self-driving control module 900 attempts to recognize the output signal of the traffic light from the image signal data received from the first image sensor 400 (S305). . The artificial intelligence algorithm of the autonomous driving control module 900 may output a recognition result including reliability of the output signal from image signal data. The output signal may be, for example, a vehicle passage permission signal (green signal) or a vehicle passage prohibition signal (red signal).

The autonomous driving control module 900 determines whether the output signal is clear from the image signal data (S307). The autonomous driving control module 900 may determine whether the output signal is clear based on whether the reliability probability output from the artificial intelligence algorithm is greater than or equal to a set threshold.

When the output signal is clear, the autonomous driving control module 900 sets the output signal (or traffic signal) clear (S309) and ends the control flow of FIG. 3 .

When the output signal is unclear, the autonomous driving control module 900 sets the output signal to be unclear (S311) and ends the control flow of FIG. 3 .

It is possible to dynamically recognize traffic signals by using image signal data and LIDAR signal data photographed from the same viewpoint targeting an area in the same direction. At least, the first image sensor 400 and the lidar sensor 300 photograph and sense an area in the same direction to achieve complementary improved recognition performance despite various external environment changes (resolution degradation during night driving, light noise, etc.) can provide

Referring to FIG. 2 again, referring to the subsequent control process, when the recognition of a traffic signal is clear, the self-driving vehicle proceeds with autonomous driving according to the recognized traffic signal (S100).

If the traffic signal is unclear (for example, the traffic signal is unclear or the output signal is unclear), the self-driving vehicle outputs a notification signal that can confirm this (S500).

According to the unclear recognition of the traffic light, the self-driving control module 900 outputs the image signal data received through the first image sensor 400 to the display 100 and furthermore sends a voice message indicating a traffic light confirmation query to the speaker 700. ) and simultaneously outputs a message indicating a traffic light confirmation query through the display 100. The traffic light confirmation query may include or imply a vehicle clearance query (eg, "Is the light green?").

According to the unclear recognition of the output signal, the self-driving control module 900 outputs the image signal data received through the first image sensor 400 to the display 100 and furthermore sends a voice message indicating a vehicle passage permission query to the speaker 700. ) and at the same time outputs a message indicating a vehicle passage permission inquiry through the display 100.

The vehicle passage permitting query may be, for example, an inquiry having meanings such as "Is it a green light" or "Is it a proceeding signal?". Alternatively, the vehicle passage permission query may be an query configured using a vehicle prohibition signal (eg, a red signal).

Following (or simultaneously with) the output of the notification signal, the self-driving vehicle performs deceleration control and records an internal image (S600). Autonomous vehicles can ensure or improve the safety of autonomous vehicles by deceleration control in situations where traffic signals are unclear.

The autonomous driving control module 900 outputs a control signal for vehicle deceleration to the vehicle control module 800 and stores image signal data received from the second image sensor 500 inside the vehicle in an internal storage medium. The video data of the image stored in this way can be used as various data thereafter. For example, video data can be used as data to check the driver's control or reaction status in the event of an accident, and it is used to clarify the legal responsibility of the manufacturer or driver.

The self-driving vehicle then receives an input for the notification signal (S700) and controls operation of the self-driving vehicle according to the input.

The autonomous driving control module 900 receives an input for a vehicle passage permission query as a voice signal through a microphone or receives an input for a vehicle passage permission query through a touch on the touch screen of the display 100 . The autonomous driving control module 900 receives an input for permitting or disallowing (stopping) vehicle passage through a microphone or a touch.

Furthermore, the autonomous driving control module 900 receives the driver's selection input (input for designating a specific location) on the display 100 displaying the image signal data from the first image sensor 400 according to the unclear recognition of the traffic light. Recognizes traffic lights and automatically recognizes output signals from image signal data in the area of the recognized traffic lights.

Alternatively (also), the self-driving control module 900 determines the position of a specific signal by the driver in the display 100 of the image signal data displaying the traffic light from the first image sensor 400 according to the unclear recognition of the output signal. Recognizes the output signal as the designated selection input. The input used for recognizing the output signal may be the same as the input for the vehicle passage permission inquiry.

The self-driving control module 900 is a driver's input, output signal data recognized according to the input, image signal data from the first image sensor 400 and lidar signal data from the lidar sensor 300, traffic signals, etc. Area data (data specifying the position of a traffic light) and the acquisition point of the data can be further stored in a storage medium. Here, the driver's input may be matched with the recorded internal image, stored, and provided as data.

The autonomous driving control module 900 terminates the storage of the video record according to the response to the vehicle passage permission query and changes from the deceleration control to the autonomous driving control through the control of the vehicle control module 800 (S100) to perform normal driving. can do.

Subsequently, as driver input data is accumulated due to unclear traffic signals, the self-driving vehicle can use the driver input data to learn the built-in artificial intelligence algorithm. The driver's input data includes information recognized according to the driver's input of an ambiguous traffic signal.

According to the arrival of a specified number or a set period, the self-driving control module 900 converts the area data of traffic lights recognized from the image signal data or LIDAR signal data of the storage medium (location of the recognized traffic light area), Output signal data (data representing the vehicle passage permission signal (green signal) or vehicle passage prohibition signal (red signal) recognized according to the driver's input), image signal data from the first image sensor 400, and/or LIDAR The signal data and the acquisition point of the data may be provided as learning data of the artificial intelligence algorithm.

Accordingly, artificial intelligence algorithms can improve recognition performance by using images obtained from the actual driving environment and driver inputs, and self-driving vehicles learn the driver's driving environment or external environment specialized for characteristics to autonomously drive specialized for the driver's environment. make this possible Furthermore, by having these learning data transmitted to the manufacturer's server, the recognition rate is improved for a specific traffic light with a low recognition rate (for example, a specific traffic light with a low recognition rate at night due to an advertising light or lighting that is overlapped with a traffic light or visible nearby) It can be used for updating software installed in autonomous vehicles.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention to those skilled in the art in the technical field to which the present invention belongs. It is not limited by drawings.

Claims (12)

1. As an autonomous driving method performed by an autonomous vehicle equipped with an artificial intelligence algorithm,

   attempting to recognize a traffic signal according to the artificial intelligence algorithm from signal data received through a sensor;

   outputting a notification signal according to unclear recognition of a traffic signal;

   Controlling the deceleration of the self-driving vehicle and recording an internal image of the self-driving vehicle according to the output of the notification signal;

   Receiving an input for the notification signal; and

   Including; changing from the deceleration control to autonomous driving control according to the input,

   The step of outputting the notification signal includes outputting a voice comment or message indicating a vehicle passage permission inquiry, wherein the vehicle passage permission query includes an inquiry asking whether the traffic signal is a green signal, a progress signal, or a red signal. and

   The input includes a response to the vehicle passage permission query,

   self-driving method.
2. According to claim 1,

   The step of attempting to recognize the traffic signal is to attempt to recognize a traffic light or recognize an output signal of the traffic light using lidar signal data received through a lidar sensor and image signal data received through an image sensor,

   self-driving method.
3. According to claim 2,

   The step of attempting to recognize the traffic signal attempts to recognize a traffic light from lidar signal data received through a lidar sensor, and attempts to recognize an output signal from image signal data received through an image sensor when the recognition of a traffic light is successful ,

   The lidar sensor and the image sensor are sensors for sensing signals in the same direction as each other,

   self-driving method.
4. According to claim 2,

   The vehicle passage permission query is output through a speaker or display,

   self-driving method.
5. According to claim 4,

   Receiving the input may include receiving it as a voice signal through a microphone or receiving it through a touch of a display,

   self-driving method.
6. According to claim 1,

   Recognizing the traffic signal using the signal data and the input and learning an artificial intelligence algorithm mounted in the autonomous vehicle according to the recognition of the traffic signal; further comprising,

   self-driving method.
7. According to claim 6,

   The step of learning the artificial intelligence algorithm,

   The image signal data received through the image sensor is output to the display, the traffic light is recognized as a selection input on the display, the output signal is recognized as the image signal data in the recognized traffic light area, the area data of the recognized traffic light, The recognized output signal data and the image signal data of the image sensor are provided as learning data of the artificial intelligence algorithm,

   The image signal data displayed by the traffic light is output to the display, the output signal is recognized as an input to the displayed traffic light, and the area data of the traffic light, the recognized output signal data, and the image signal data of the image sensor are converted into the artificial intelligence algorithm. Provided as learning data,

   self-driving method.
8. As an autonomous driving system mounted on an autonomous vehicle,

   LiDAR sensor or first image sensor; speaker; display; a second image sensor that photographs the inside of the self-driving vehicle; vehicle control module; and

   It is equipped with an artificial intelligence algorithm, receives signal data from the lidar sensor or the first image sensor, attempts to recognize a traffic signal from the received signal data according to the artificial intelligence algorithm, and if the recognition of the traffic signal is unclear, the speaker or an autonomous driving control module outputting a notification signal to the display, performing deceleration control of the autonomous vehicle in conjunction with the vehicle control module, and recording an image of the second image sensor;

   The autonomous driving control module changes from the deceleration control to autonomous driving control through control of the vehicle control module according to reception of an input for the notification signal,

   The output of the notification signal includes outputting a voice comment or message indicating a vehicle passage permission inquiry, wherein the vehicle passage permission query includes an inquiry asking whether the traffic signal is a green signal, a progress signal, or a red signal,

   The input includes a response to the vehicle passage permission query,

   autonomous driving system.
9. According to claim 8,

   The self-driving control module attempts to recognize a traffic light from lidar signal data received through the lidar sensor, and attempts to recognize an output signal from image signal data received through the first image sensor when the traffic light is successfully recognized. do,

   The lidar sensor and the first image sensor mounted outside the self-driving vehicle are sensors for sensing signals in the same direction as each other,

   autonomous driving system.
10. According to claim 8,

    The voice comment is output through the speaker, and the message is output through the display.

    autonomous driving system.
11. According to claim 8,

    The autonomous driving control module learns the artificial intelligence algorithm using the signal data and the input.

    autonomous driving system.
12. According to claim 11,

    The autonomous driving control module,

    The image signal data received through the first image sensor is output to the display, the traffic light is recognized as a selection input on the display, the output signal is recognized with the image signal data in the area of the recognized traffic light, and the recognized traffic is recognized. Provide the area data of the traffic light, the recognized output signal data, and the image signal data of the first image sensor as learning data of the artificial intelligence algorithm;

    The image signal data displayed by the traffic light is output to the display, the output signal is recognized as an input to the displayed traffic light, and the area data of the traffic light, the recognized output signal data, and the image signal data of the first image sensor are converted into the artificial light. Provided as learning data for intelligent algorithms,

    autonomous driving system.

Images