KR101977673B1 - Image analysis apparatus and operating method thereof - Google Patents

Abstract

An image analyzing apparatus according to an embodiment of the present invention includes an image obtaining unit for obtaining a surrounding image of a user vehicle, an object detecting unit for detecting a surrounding vehicle and a traffic light from a surrounding image, A user vehicle movement information extracting unit for extracting first movement information for the user vehicle, information on the sensed neighboring vehicle, and second movement information on the neighboring vehicle from the first movement information And a signal violation judging unit for judging whether the signal of the neighboring vehicle is violated by comparing the second vehicle movement information extracting unit and the second movement information with the current signal information.

Description

[0001] IMAGE ANALYSIS APPARATUS AND OPERATING METHOD THEREOF [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image analysis apparatus, and more particularly, to an image analysis apparatus for determining whether a signal of a neighboring vehicle is violated in a user vehicle and an operation method thereof.

The vehicle black box is a device for recording an image around the user's vehicle. The user can grasp the situation at the time of the accident through the image recorded in the black box. However, a video recording device such as a black box is utilized only in a specific situation, such as an accident, and is not utilized at normal times. Accordingly, various methods for utilizing images captured through a black box have been studied.

In the case where it is possible to judge whether or not a traffic law violation is simply the same as an illegal parking vehicle, a video recording device mounted on the vehicle is utilized. However, the violation of the vehicle signal at the intersection is interrupted by CCTV, and the video recording device mounted in the vehicle is not utilized.

Due to problems such as the cost of installing CCTV, it is difficult to control the signal violation of the vehicle at all the intersections using CCTV. Due to this, traffic accidents are occurring frequently due to signal violation at the intersection where CCTV is not installed.

It is an object of the present invention to provide an image analysis apparatus for determining whether a signal of a nearby vehicle is violated by using an image photographed by a user vehicle and an operation method thereof .

An image analyzing apparatus according to an embodiment of the present invention includes an image obtaining unit for obtaining a surrounding image of a user vehicle, an object detecting unit for detecting a surrounding vehicle and a traffic light from the surrounding image, A signal determination unit for determining current signal information, a user vehicle movement information extraction unit for extracting first movement information for the user vehicle, information about the sensed neighboring vehicle, and information about the neighboring vehicle from the first movement information, And a signal violation determiner for determining whether the signal of the neighboring vehicle is violated by comparing the second movement information with the current signal information.

In an exemplary embodiment, the user vehicle movement information extraction unit may extract the first movement information based on a change in size and position of the detected signal with time.

In an embodiment, the peripheral vehicle movement information extracting unit may extract the second movement information based on the first movement information and the size and position change of the sensed peripheral vehicle with respect to time.

The image analysis apparatus according to an embodiment of the present invention may further include a position information obtaining unit for obtaining position information of the user vehicle.

In an exemplary embodiment, the signal determination unit may collect signal system information corresponding to the position information of the user vehicle, and may determine the current signal information based on the information on the sensed signal and the signal system information.

In an embodiment, the signal system information may include at least one of traffic light information, U-turn information, unprotected signal information, and traffic light operation time information.

In an embodiment, the user vehicle movement information extracting unit may extract the first movement information based on the position information of the user vehicle.

In an embodiment, the first movement information and the second movement information may be classified into any one of a stop, a straight line, a left turn, and a right turn.

The image analysis apparatus according to an embodiment of the present invention may further include a communication unit for transmitting the peripheral image to an external server when the peripheral vehicle is determined to be a signal violation.

A method of operating an image analysis apparatus mounted on a user vehicle according to an embodiment of the present invention includes the steps of acquiring a peripheral image of the user vehicle, detecting peripheral vehicles and a traffic light from the peripheral image, Determining the current signal information based on the information about the current vehicle, extracting first movement information for the user vehicle, information about the sensed neighboring vehicle, and second movement from the first movement information to the neighboring vehicle And comparing the second movement information with the current signal information to determine whether the signal of the neighboring vehicle is violated.

The method of operating an image analysis apparatus according to an embodiment of the present invention may further include acquiring position information of the user vehicle and acquiring current signal system information based on the obtained position information.

In an embodiment, the first movement information may be extracted based on at least one of a change in size and position of the sensed signal with time, and position information of the user vehicle.

In an embodiment, the second movement information may be extracted based on the first movement information and a change in size and position with time of the sensed neighboring vehicle.

The image analysis apparatus according to the embodiment of the present invention can determine whether the signal of the neighboring vehicle is violated by considering the movement of the user vehicle. Thus, it can be judged whether or not the signal of the neighboring vehicle is violated in various situations.

In addition, the image analyzing apparatus according to the embodiment of the present invention can determine a signal-infringing vehicle at a range where the CCTV is not installed, and report a signal-infringing vehicle.

1 is a diagram illustrating an example in which an image analysis apparatus according to an embodiment of the present invention reports a signal-violating vehicle.
2 is a block diagram illustrating an image analysis apparatus and a server according to an embodiment of the present invention.
3 is a detailed block diagram of the image analysis apparatus of FIG.
4 is a block diagram showing an example of the operation of the signal determination unit of FIG. 3 in detail.
5 is an exemplary diagram illustrating signaling information stored in the memory of FIG.
FIG. 6 is a block diagram showing an example of the operation of the user vehicle movement information extracting unit of FIG. 3 in detail.
FIG. 7 is a diagram illustrating an example in which the user vehicle movement information extracting unit of FIG. 6 extracts movement information of a user vehicle.
8 is a block diagram showing an example of the operation of the peripheral vehicle movement information extracting unit of FIG. 3 in detail.
9 is a diagram showing an example in which the peripheral vehicle movement information extracting unit of FIG. 8 extracts movement information of a peripheral vehicle.
FIG. 10 is a block diagram showing an example of the operation of the signal violation determining unit of FIG. 3 in detail.
FIG. 11 is a diagram illustrating an example of determining whether the signal violation determination unit of FIG. 10 determines whether a signal is violated.
12 is a flowchart showing an exemplary operation of an image analysis apparatus according to an embodiment of the present invention.

In the following, embodiments of the present invention will be described in detail and in detail so that those skilled in the art can easily carry out the present invention.

1 is a diagram illustrating an example in which an image analysis apparatus according to an embodiment of the present invention reports a signal-violating vehicle. Referring to FIG. 1, a user vehicle may include an image analysis apparatus 100. Illustratively, the image analysis apparatus 100 may be mounted within the user vehicle.

The image analysis apparatus 100 can determine whether the signal of the surrounding vehicle is violated based on the surrounding image information. The image analyzing apparatus 100 can directly photograph a surrounding image or acquire a surrounding image from a separate image recording apparatus (for example, a black box). Hereinafter, for the sake of convenience of explanation, it is assumed that the image analysis apparatus 100 has a means for directly photographing an image.

The nearby vehicle may include all vehicles that can be photographed from the image analysis apparatus 100 in the user vehicle. Illustratively, as shown in FIG. 1, the nearby vehicle may be a vehicle ahead of the user vehicle.

The image analysis apparatus 100 can determine whether the signal of the neighboring vehicle is violated regardless of whether the user vehicle and the neighboring vehicle are moving or not. The image analysis apparatus 100 can determine whether the signal of the neighboring vehicle is violated based on the signal information (hereinafter, signal light signal information) indicated by the current signal or the like. Illustratively, as shown in FIG. 1, the image analysis apparatus 100 can determine a signal violation of a nearby vehicle at a range. For example, when the surrounding vehicle travels straight from the stop signal, the image analysis apparatus 100 may determine that the surrounding vehicle is a signal violation.

If it is determined that the neighboring vehicle has violated the signal, the image analysis apparatus 100 may transmit information on the signal violation to the external server 10. The information on the signal violation may include an image or an image (hereinafter referred to as a signal violation image) from which the fact of signal violation can be confirmed from the image captured by the image analysis apparatus 100. The image analysis apparatus 100 may perform wireless communication and transmit the signal violation image to the server 10. [

The server 10 may be provided in an organization for interrupting a traffic regulation violation. In the case where the server 10 receives a signal-infringing image, the agency that controls the violation of the traffic regulation may check the signal-infringing image and impose a penalty on the signal-violating vehicle. Illustratively, the agency that controls traffic violation violation extracts vehicle information (for example, a vehicle number listed on a license plate) from the signal infringement image received by the server 10, and then, based on the extracted vehicle information, You can impose a penalty on

In this manner, in the case of determining whether a signal of a nearby vehicle is violated directly in the vehicle and reporting a signal-infringing vehicle, the signal violation interruption is performed even in a place where a separate signal violation interrupter (for example, CCTV) It can be possible. This can reduce the installation cost for signal violation interrupters and reduce signal violations in locations where signal violations are frequent, such as at a range or at an intersection.

2 is a block diagram illustrating an image analysis apparatus and a server according to an embodiment of the present invention. 2, the image analysis apparatus 100 may include an image acquisition unit 110, an image analysis unit 120, a location information acquisition unit 130, and a communication unit 140.

The image acquiring unit 110 may acquire image information by capturing an image outside the user vehicle. The image information may include image information about objects such as nearby vehicles and traffic lights. For example, the image acquiring unit 110 may acquire image information about a nearby vehicle and a traffic light in front of the intersection.

The image analysis unit 120 may analyze the image provided from the image acquisition unit 110. The image analysis unit 120 may detect various objects included in the image. Illustratively, the image analysis unit 120 may detect nearby vehicles and traffic lights included in the image. The image analyzing unit 120 can detect movement information of nearby vehicles (for example, stop, straight ahead, left turn, right turn) of the surrounding vehicles by sensing the nearby vehicles. The image analysis unit 120 may detect a signal lamp and determine current signal light signal information (e.g., a straight signal, a left turn signal, a stop signal, etc.). The image analyzing unit 120 can determine whether the signal of the surrounding vehicle is violated based on the movement information of the surrounding vehicle and the signal light signal information.

The image analysis unit 120 may determine movement information (e.g., stop, straight, left, right turn) of the user vehicle from the objects included in the image. The image analysis unit 120 may determine the movement information of the neighboring vehicle based on the movement information of the user vehicle.

The location information obtaining unit 130 may obtain location information of the user vehicle. The position information obtaining unit 130 may include a Global Positioning System (GPS) module. The location information obtaining unit 130 may receive GPS information and obtain location information of the user vehicle. The location information acquisition unit 130 may provide the acquired location information to the image analysis unit 120.

The image analysis unit 120 may acquire the signal system information corresponding to the current location based on the location information provided from the location information acquisition unit 130. [ The signaling system information may include traffic light information, traffic light operation time information, U-turn information, unprotected signal information, and the like. The traffic light information may include information on the number of signals included in the traffic lights and the type of the traffic lights. The traffic light operation time information may include information on the time at which the traffic lights operate. The U-turn information may indicate whether or not a U-turn is possible at the current position. The unprotected signal information may include information indicating whether or not an unprotected left turn is possible.

The image analysis unit 120 may determine signal information of an image provided from the image acquisition unit 110 based on the obtained signal system information. The image analysis unit 120 may determine whether the signal of the surrounding vehicle is violated based on the obtained signal system information.

The communication unit 140 may perform wireless communication with the server 10. When the video analysis unit 120 determines that a signal violation is detected, the communication unit 140 may transmit the signal violation information to the server 10. For example, the signal violation information may include a signal violation image.

The communication unit 140 may receive the signaling information from the server 10. [ Illustratively, the communication unit 140 may receive the signaling information corresponding to the current position of the user vehicle obtained from the positional information obtaining unit 130. Alternatively, the communication unit 140 may receive the signal system information of various places in advance and then provide the signal system information to the image analysis unit 120.

3 is a detailed block diagram of the image analysis apparatus of FIG. 2 and 3, the image analysis apparatus 100 may include an image acquisition unit 110, an image analysis unit 120, a position information acquisition unit 130, and a communication unit 140. The image analysis unit 120 includes an object detection unit 121, a memory 122, a signal determination unit 123, a user vehicle movement information extraction unit 124, a peripheral vehicle movement information extraction unit 125, (126).

The object detection unit 121 may receive the image photographed by the image acquisition unit 110. [ The object detection unit 121 may detect objects included in the received image. For example, the object detection unit 121 can detect objects using deep learning.

Illustratively, the object detection unit 121 can classify the types of objects in each image included in the image, and determine the position and size of the objects. Accordingly, the object detection unit 121 can determine the positions and sizes of objects that may change with time in the image. For example, the object detection unit 121 can distinguish surrounding vehicles and traffic lights from each image, and can determine the location and size of nearby vehicles and traffic lights in each image. As the user vehicle approaches the traffic light, the position of the detected traffic light can be moved upwards, and the traffic light can be increased in size.

The object detection information detected by the object detection unit 121 may be transmitted to the signal determination unit 123, the user vehicle movement information extraction unit 124, and the peripheral vehicle movement information extraction unit 125.

The memory 122 may store an image photographed from the image acquiring unit 110. When the signal violation of the surrounding vehicle is determined from the photographed image, the image stored in the memory 122 may be transmitted to the server 10. Illustratively, an image at a specific time or at a specific time in the image stored in the memory 122 may be transmitted to the server 10. An image at a specific time or at a specific time may include an image that allows the traffic law regulator to visually confirm the signal violation.

Memory 122 may store signaling information. The signaling information may be transmitted from the server 10 according to the current position of the user vehicle. Illustratively, the signaling system information may include nearest signal light information, U-turn information, signal light operation time information, unprotected signal information, etc., based on the current position of the user vehicle.

Or memory 122 may receive and store signaling information of various regions from the server 10 in advance. The image analysis unit 120 may acquire the signal system information according to the current position among the signal system information stored in the memory 122. [ When the memory 122 stores signaling information of various regions in advance, the latest signaling information may be updated according to a predetermined period or a specific event. For example, when the start of the user vehicle is turned on, the latest signal system information can be updated.

Accordingly, since the image analysis apparatus 100 does not need to constantly receive the signal system information from the server 10 based on the current position, the load due to the communication can be reduced.

The signal determination unit 123 may determine the current signal information from the objects detected from the object detection unit 121 and the signal system information. For example, the signal determination unit 123 may determine the signal light signal information at the current position.

The user vehicle movement information extracting unit 124 may extract the movement information of the user vehicle based on the object information sensed by the object sensing unit 121 or the position information acquired from the position information acquiring unit 130. [

The neighboring vehicle movement information extracting unit 125 extracts movement information of the neighboring vehicle based on the movement information of the user vehicle extracted from the user vehicle movement information extracting unit 124 and the object information sensed by the object sensing unit 121 .

The signal violation judging unit 126 compares the current signal information judged by the signal judging unit 123 and the movement information of the neighboring vehicle extracted from the neighboring vehicle movement information extracting unit 125 to judge whether the signal of the neighboring vehicle is violated .

The signal determination unit 123, the user vehicle movement information extraction unit 124, the peripheral vehicle movement information extraction unit 125, and the signal violation determination unit 126 will be described in detail below with reference to the drawings.

4 is a block diagram showing an example of the operation of the signal determination unit of FIG. 3 in detail. 3 and 4, the image information acquired through the image acquisition unit 110 may be transmitted to the object detection unit 121. [ The object detection unit 121 may detect the objects included in the image and provide the object detection information to the signal determination unit 123. Illustratively, the object detection unit 121 may provide object detection information on the signal to the signal determination unit 123. [ The object detection information for a traffic light may include shape, color, and order information of each signal included in the traffic light.

The signal determination unit 123 can determine the current signal and the like based on the object detection information on the signal. For example, when the object detection unit 121 detects a blue signal, the signal determination unit 123 may determine the current signal and the like as the forward signal. When the object detection unit 121 detects a red signal, the signal determination unit 123 can determine the current signal and the like as the stop signal.

Alternatively, the signal determination unit 123 may determine the signal information by comparing the object signal information detected by the object detection unit 121 and the current signal system information stored in the memory 122. [ The signal determination unit 123 may obtain the current signal system information corresponding to the current position information of the user vehicle provided from the position information obtaining unit 130. [ For example, because the current signaling information may include the traffic light information, the signal determining unit 123 may determine the current signal information including the traffic light signal information more accurately.

The signaling system information may include signal light information, signal light operation time information, U-turn information, and unprotected signal information. Since the signal system information may include information that is difficult to be judged using only the object detection information, the signal determination unit 123 may determine the detailed current signal information through the signal system information.

5 is an exemplary diagram illustrating signaling information stored in the memory of FIG. 4 and 5, the memory 122 may store signaling information corresponding to location information. For example, as shown in FIG. 5, the memory 122 may store signaling information corresponding to the "A", "B", "C" positions.

As the signaling information corresponding to the " A " position, the memory 122 may include signal light information and U-turn information. The traffic light information indicates a red signal, a yellow signal, a left turn signal, and a blue signal, and the U-turn information may indicate that a U-turn is possible when turning left.

As signaling information corresponding to the " B " position, the memory 122 may include signal light information, unprotected signal information, and U-turn information. The traffic light information indicates a red signal, a yellow signal, and a blue signal, the unprotected signal information indicates an unprotected left turn, and the U-turn information indicates a U-turn prohibition.

As signaling information corresponding to the " C " location, the memory 122 may include traffic light information and traffic light operation time information. The traffic light information indicates the red signal, the yellow signal, the left turn signal, and the blue signal, and the traffic light operation time information may indicate from "05:00" to "23:00".

When the current position information obtained from the position information obtaining unit 130 is the " A " position, the signal determining unit 123 can extract the signal system information corresponding to the position " A " Accordingly, the signal determination unit 123 can determine the current signal information by comparing the extracted signal light information with the object detection information on the signal. Also, the signal determination unit 123 can determine that the signal can be turned on from the signal system information.

FIG. 6 is a block diagram showing an example of the operation of the user vehicle movement information extracting unit of FIG. 3 in detail. 3 and 6, the image information acquired through the image acquisition unit 110 may be transmitted to the object detection unit 121. [ The object detection unit 121 may detect the objects included in the image and provide the object detection information to the user vehicle movement information extraction unit 124. Illustratively, the object detection unit 121 may provide location and size information for a specific object to the user vehicle movement information extraction unit 124. For example, the object detection unit 121 may provide position and size information for a traffic light to the user vehicle movement information extraction unit 124. [

The location information obtained through the location information acquisition unit 130 may be provided to the user vehicle movement information extraction unit 124. [

The user vehicle movement information extracting unit 124 can extract the movement information of the user vehicle. The user vehicle movement information extraction unit 124 can extract the movement information of the user vehicle based on the object detection information detected by the object detection unit 121. [ Illustratively, the user vehicle movement information extracting unit 124 may extract the movement information of the user vehicle based on a change in size or position of the specific object with respect to time. For example, the user vehicle movement information extracting unit 124 can extract the movement information of the user vehicle based on the size or the position change of the signal lamp according to time in the image.

Or the user vehicle movement information extracting unit 124 may extract the movement information of the user vehicle based on the position information obtained from the position information obtaining unit 130. [

For example, the user vehicle movement information extracting unit 124 can extract the movement information of the user vehicle as "stop", "straight forward", "leftward turn", and "right turn".

FIG. 7 is a diagram illustrating an example in which the user vehicle movement information extracting unit of FIG. 6 extracts movement information of a user vehicle. Referring to FIGS. 6 and 7, the user vehicle movement information extracting unit 124 may extract the movement information of the user vehicle based on at least one of the object detection information and the position information.

7, when the traffic lights included in the object detection information are maintained at the same position and the same size with time, or when there is no change in the location information of the user vehicle, the user vehicle movement information extracting unit 124 extracts, Can be judged as being "stopped".

Alternatively, when the signal included in the object detection information moves upward and increases in size with time, or when the position information of the user vehicle changes in the straight forward direction, the user vehicle movement information extracting section 124 determines that the user vehicle is " "It can be judged that

Alternatively, when the signal included in the object detection information moves to the right according to the time, and the position information of the user vehicle changes in the leftward direction, the user vehicle movement information extraction unit 124 determines that the user vehicle is in the " can do.

Alternatively, when the signal included in the object detection information moves to the left according to time and the position information of the user vehicle changes in the direction of the right turn, the user vehicle movement information extracting unit 124 determines that the user vehicle is in the " can do.

As described above, the user vehicle movement information extracting unit 124 may extract the movement information of the user vehicle using at least one of the object detection information and the position information. When both the object detection information and the position information are used, the user vehicle movement information extracting unit 124 can more accurately extract the movement information of the user vehicle.

8 is a block diagram showing an example of the operation of the peripheral vehicle movement information extracting unit of FIG. 3 in detail. Referring to FIGS. 3 and 8, the image information obtained through the image acquisition unit 110 may be provided to the object detection unit 121. FIG. The object detection unit 121 may detect the objects included in the image and provide the object detection information to the neighboring vehicle movement information extraction unit 125. [ For example, the object detection unit 121 may provide the object detection information on the nearby vehicle to the neighboring vehicle movement information extraction unit 125. The object detection information for the surrounding vehicle may include information about the position or size of the surrounding vehicle.

The neighboring vehicle movement information extracting unit 125 can extract the movement information of the neighboring vehicle. The peripheral vehicle movement information extracting unit 125 extracts movement information of the surrounding vehicle based on the object detection information detected by the object detecting unit 121 and the movement information of the user vehicle extracted from the user vehicle movement information extracting unit 124 can do.

For example, the neighboring vehicle movement information extraction unit 125 may extract the movement information of the neighboring vehicle based on the size or position change of the neighboring vehicle with respect to time based on the movement information of the user vehicle. For example, the neighboring vehicle movement information extracting unit 125 can extract the movement information of the neighboring vehicle as "stop", "straight forward", "leftward turn", and "right turn".

9 is a diagram showing an example in which the peripheral vehicle movement information extracting unit of FIG. 8 extracts movement information of a peripheral vehicle. Referring to FIGS. 8 and 9, the neighboring vehicle movement information extracting unit 125 may extract the movement information of the neighboring vehicle based on the movement information and the object detection information of the user vehicle.

If the user vehicle is stationary according to the movement information of the user vehicle, and the surrounding vehicles included in the object detection information are maintained at the same position and the same size according to the time, the peripheral vehicle movement information extraction unit 125 extracts the movement information Is " stopped ".

Alternatively, when the user vehicle is moving forward according to the movement information of the user vehicle, and the neighboring vehicle included in the object detection information moves leftward or rightward according to time, the neighboring vehicle movement information extracting unit 125 extracts the movement information Is " stopped ". In this case, the distance traveled to the left or right may be a value smaller than the previously stored distance value.

Alternatively, when the user vehicle is turning left according to the movement information of the user vehicle, and the neighboring vehicle included in the object detection information moves rightward back in time, the neighboring vehicle movement information extracting unit 125 extracts the movement information of the neighboring vehicle as &Quot; stop "

Alternatively, when the user vehicle is turning right according to the movement information of the user vehicle and the neighboring vehicle included in the object detection information moves leftward in time, the neighboring vehicle movement information extracting unit 125 determines that the movement information of the neighboring vehicle is "Quot; stop "

When the user vehicle is stationary according to the movement information of the user vehicle, and the surrounding vehicle included in the object detection information moves upward and decreases in size, the peripheral vehicle movement information extracting unit 125 extracts the movement information Is " straight ahead ".

Alternatively, when the user vehicle is moving forward according to the movement information of the user vehicle, and the neighboring vehicles included in the object detection information are maintained at the same position and the same size with time, the neighboring vehicle movement information extracting unit 125 extracts It can be determined that the movement information is " straight ahead ".

Alternatively, when the user vehicle is turning left according to the movement information of the user vehicle, and the neighboring vehicle included in the object detection information moves to the right according to time, the neighboring vehicle movement information extracting unit 125 determines that the movement information of the neighboring vehicle is " Straight forward ".

Alternatively, when the user vehicle is turning right according to the movement information of the user vehicle, and the neighboring vehicle included in the object detection information moves to the left according to time, the neighboring vehicle movement information extracting unit 125 determines that the movement information of the neighboring vehicle is " Straight forward ".

When the user vehicle is stationary according to the movement information of the user vehicle and the neighboring vehicle included in the object detection information moves to the left in accordance with the time, the neighboring vehicle movement information extraction unit 125 extracts the movement information of the neighboring vehicle as " . ≪ / RTI >

Alternatively, if the user vehicle is traveling straight ahead in accordance with the movement information of the user vehicle, and the neighboring vehicle included in the object detection information moves leftward in time, the neighboring vehicle movement information extracting unit 125 extracts the neighboring vehicle movement information, Left turn ". In this case, the distance traveled to the left may be greater than a previously stored distance value.

Alternatively, when the user vehicle is turning left according to the movement information of the user vehicle, and the surrounding vehicles included in the object detection information are maintained at the same position and the same size with time, the peripheral vehicle movement information extracting unit 125 extracts It can be determined that the movement information is " left turn ".

If the user vehicle is turning right according to the movement information of the user vehicle and the surrounding vehicle included in the object detection information is not detected, the surrounding vehicle movement information extraction unit 125 may not determine the movement information of the surrounding vehicle.

When the user vehicle is stopped according to the movement information of the user vehicle and the neighboring vehicle included in the object detection information moves to the right according to time, the neighboring vehicle movement information extracting unit 125 extracts the movement information of the neighboring vehicle as " . ≪ / RTI >

Alternatively, if the user vehicle is traveling straight ahead according to the movement information of the user vehicle, and the nearby vehicle included in the object detection information moves rightward back in time, the neighboring vehicle movement information extraction unit 125 determines that the movement information of the nearby vehicle is & Right turn ". In this case, the distance to the right may be larger than the previously stored distance value.

When the user vehicle is turning left according to the movement information of the user vehicle and the surrounding vehicle included in the object detection information is not detected, the surrounding vehicle movement information extraction unit 125 may not determine the movement information of the surrounding vehicle.

Alternatively, when the user vehicle is turning right according to the movement information of the user vehicle, and the surrounding vehicles included in the object detection information are maintained at the same position and the same size with time, the peripheral vehicle movement information extracting unit 125 extracts, It can be determined that the movement information is " right turn ".

FIG. 10 is a block diagram showing an example of the operation of the signal violation determining unit of FIG. 3 in detail. 3 and 10, the signal violation determining unit 126 receives the current signal information from the signal determining unit 123 and receives movement information of the neighboring vehicle from the neighboring vehicle movement information extracting unit 125 have. The current signal information may include signal light signal information, U-turn information, unprotected signal information, traffic light operation time information, and the like. The movement information of the nearby vehicle may indicate any one of a stop, a straight ahead, a left turn or a right turn.

The signal violation judging unit 126 can judge whether the signal of the neighboring vehicle is violated by comparing the current signal information with the movement information of the nearby vehicle. The signal violation determining unit 126 may transmit the signal violation image stored in the memory 122 to the server 10 through the communication unit 140. [ For example, the signal violation determiner 126 may transmit the image of the specific period or the specific time of the image stored in the memory 122 to the server 10. For example, the signal violation judging unit 126 may transmit the image taken within a predetermined distance from the signal light or the range to the server 10. Alternatively, the signal violation judging unit 126 may transmit to the server 10 a video up to a point in time when any of the signal lamp and the surrounding vehicle is not sensed based on the point of time when the signal lamp and the surrounding vehicle are sensed.

FIG. 11 is a diagram illustrating an example of determining whether the signal violation determination unit of FIG. 10 determines whether a signal is violated. Referring to FIGS. 10 and 11, the signal violation determining unit may determine whether the signal of the neighboring vehicle is violated based on the current signal information and the movement information of the neighboring vehicle.

The signal violation judging section 126 judges that the signal information of the surrounding vehicle is in the vicinity of the vehicle 100. If the signal information signal included in the current signal information indicates the stop signal and the left turn signal, It can be determined that this signal violation has occurred.

Alternatively, if the signal information of the signal included in the current signal information indicates a straight signal, indicating that the unprotected left turn is possible, if the movement information of the nearby vehicle indicates a left turn, the signal violation determiner 126 does not violate the signal It can be judged that it is not.

Or, if the traffic light signal information included in the current traffic information indicates a yellow signal and the traffic light operation time indicates "05:00" to "23:00", the current time is "03:00" The signal violation judging unit 126 can judge that the neighboring vehicle has not violated the signal.

12 is a flowchart showing an exemplary operation of an image analysis apparatus according to an embodiment of the present invention. Referring to FIGS. 2 and 12, in step S101, the image analysis apparatus 100 can acquire an image around the user vehicle. In step S102, the image analysis apparatus 100 can acquire current signal system information based on the position information of the user vehicle. Illustratively, the image analysis apparatus 100 can acquire the current signal system information corresponding to the position information of the user vehicle from the database (for example, the information shown in FIG. 5) stored in advance in the image analysis apparatus 100 have. Or the image analysis apparatus 100 may obtain the current signaling system information corresponding to the position information of the user vehicle from the external server 10. [

In step S103, the image analysis apparatus 100 can sense the object from the acquired image. Illustratively, the image analysis apparatus 100 can detect nearby vehicles and traffic lights, and detect the location and size of nearby vehicles and traffic lights over time.

In step S104, the image analysis apparatus 100 can determine the current signal information by comparing the current signal system information with the object detection information detected in the image. For example, the image analysis apparatus 100 may compare the signal light information included in the signal system information with the object detection information on the signal light to determine the current signal light signal information.

In step S105, the image analysis apparatus 100 can extract the movement information of the user vehicle. Illustratively, the image analysis apparatus 100 may extract the movement information of the user vehicle based on the position information of the user vehicle or the position and size change of the signal light detected in the image.

In step S106, the image analysis apparatus 100 can extract the movement information of the nearby vehicle. Illustratively, the image analysis apparatus 100 may extract movement information of the neighboring vehicle based on the movement information of the user vehicle and the change in the position and size of the neighboring vehicle sensed in the image.

In step S107, the image analysis apparatus 100 can determine whether the signal of the neighboring vehicle is violated based on the current signal information and the movement information of the neighboring vehicle. If it is determined that the vehicle is a signal-violating vehicle, the image analysis apparatus 100 may transmit the signal-violation image to the external server 10 in step S108.

As described above, the image analysis apparatus 100 according to the embodiment of the present invention can determine whether a signal of a neighboring vehicle is violated by using an image photographed inside the user vehicle. Particularly, by determining the movement information of the neighboring vehicle based on the movement information of the user vehicle, the image analysis apparatus 100 can determine whether the signal of the neighboring vehicle is violated in various situations.

The above description is specific embodiments for carrying out the present invention. The present invention will also include embodiments that are not only described in the above-described embodiments, but also can be simply modified or changed easily. In addition, the present invention will also include techniques that can be easily modified and implemented using the embodiments. Therefore, the scope of the present invention should not be limited to the above-described embodiments, but should be determined by the claims equivalent to the claims of the present invention as well as the following claims.

10: server 100: image analysis device
110: Image acquisition unit 120: Image analysis unit
130: Position information obtaining unit 140:

Claims (13)

1. An image analysis apparatus mounted on a user vehicle,
An image acquiring unit acquiring a peripheral image of the user vehicle;
An object sensing unit for sensing a surrounding vehicle and a traffic light from the peripheral image;
A signal judging unit for judging current signal information based on the information about the sensed signal;
A user vehicle movement information extracting unit for extracting first movement information of the user vehicle in a first movement direction;
And a second movement information extracting unit that extracts second movement information on the second moving direction of the neighboring vehicle based on the size change and the positional change with time of the portion of the peripheral vehicle in the peripheral image and the first movement information, A movement information extracting unit; And
And a signal violation determiner for comparing the second movement information with the current signal information to determine whether the signal of the neighboring vehicle is violated,
Wherein the first moving direction and the second moving direction include a stop, a straight line, a left turn, and a right turn. The method according to claim 1,
Wherein the user vehicle movement information extracting unit extracts the first movement information based on a change in size of the portion of the traffic light with respect to time and a change in position with time in the surrounding image. delete The method according to claim 1,
Further comprising: a position information acquiring unit acquiring position information of the user vehicle. 5. The method of claim 4,
Wherein the signal determination unit collects signal system information corresponding to the position information of the user vehicle and determines the current signal information based on the information about the sensed signal and the signal system information. 6. The method of claim 5,
Wherein the signal system information includes at least one of traffic light information, U-turn information, unprotected signal information, and traffic light operation time information. 5. The method of claim 4,
Wherein the user vehicle movement information extraction unit extracts the first movement information based on the position information of the user vehicle. delete The method according to claim 1,
And a communication unit for transmitting the peripheral image to an external server when the peripheral vehicle is judged to be a signal violation. A method of operating an image analysis apparatus mounted on a user vehicle,
Obtaining a peripheral image of the user vehicle;
Detecting peripheral vehicles and a traffic light from the peripheral image;
Determining current signal information based on information on the sensed signal;
Extracting first movement information of the user vehicle in a first movement direction;
Extracting second movement information of the neighboring vehicle based on the change in size and time of the portion of the peripheral vehicle with respect to time and the first movement information; And
Comparing the second movement information with the current signal information to determine whether the signal of the neighboring vehicle is violated,
Wherein the first movement direction and the second movement direction include a stop, a straight line, a left turn, and a right turn. 11. The method of claim 10,
Obtaining location information of the user vehicle; And
And obtaining current signaling information based on the obtained location information. 12. The method of claim 11,
Wherein the first movement information is extracted based on at least one of a positional change with time and a positional change with time of the portion with respect to the signal lamp in the peripheral image. delete

Images