KR102321754B1 - System for detecting accident in road based on edge computing - Google Patents

Abstract

The present invention relates to an image loss detection system, which is a camera functioning as each node based on edge computing, which is installed at a predetermined interval on the roadside to photograph a preset loss detection area on the road, collects the captured images, and collects It is an edge camera and terminal installed in the situation room to analyze the recorded video to check whether an accident situation has occurred, to collect information related to the accident situation in the corresponding lane when an accident situation occurs, and to transmit the collected information to the situation room terminal. , and a situation room terminal for displaying information transmitted from the edge camera.
According to the present invention, by using an edge node camera provided with a multi-hop function, it is possible to operate an economical and high-accuracy error detection system, and there is an effect that the system can be continuously installed over a wide range.

Description

Edge computing-based image detection system {System for detecting accident in road based on edge computing}

The present invention relates to a technology for detecting an abnormal situation occurring on a road, and more particularly, to a technology for detecting an accident situation by analyzing an image taken of a road.

The video error detection system is a system for prompt post-processing by analyzing the video and delivering the accidental situation to the situation room when an abnormal situation occurs on the road.

Examples of typical disaster situations include a situation in which a falling object that has fallen from a vehicle is left on the road, a situation in which the vehicle is abnormally stopped on the road, a situation in which the vehicle is traveling in the opposite direction, and a moving object that should not be on the road, such as a person or a mountain animal, exists on the road. situations, etc.

The purpose of the accident monitoring system is to prevent a secondary accident in advance by promptly handling an accident situation when it occurs.

On the other hand, edge computing is a technology that processes data around the user's terminal or in the terminal itself rather than a central cloud server, and has a shorter data processing time and superior security compared to existing cloud computing. In particular, it is expected to be widely used in autonomous vehicles, smart factories, and the Internet of Things (IOT), where there is a large amount of data and requires real-time processing.

Edge computing implies processing data first at the edge of the network, as opposed to cloud computing where a central server processes all data.

In simple terms, cloud computing is a technology in which a high-performance computer performs work instead of a task. In other words, as the data provided by IoT devices exploded, it became difficult for the central computer to process all of them, and the time it took to process the data also emerged as a problem.

Edge computing is a technology that supplements this limitation with distributed processing technology, and the concept is that the data that can be analyzed by each IoT device is processed directly on the spot.

The maintenance detection system is a very necessary technology for road safety, but the existing maintenance detection system has a problem in that the detection range for each point of the unit system is limited. In fact, in the case of the longest radar, about 1 km is possible, and if additional detection is required, a separate additional system must be increased. In addition, in the case of radar, an image system must be additionally installed for the operator to make a final decision. When the system is expanded, the video is also expanded, which may cause network capacity problems. Also, in the case of an image system, the unit system distance range is about 200m, but there is a problem of accuracy when processing images from a long distance. As a result, it has disadvantages in terms of economic feasibility due to the construction of high-performance sensor systems and the increase in transport layer network facilities.

Republic of Korea Patent Registration 10-0435003

The present invention has been devised to solve the above problems, and by processing the collected images in the edge computer-based node camera, the image is not transmitted to the control room under normal circumstances, and when an abnormal situation occurs on the road. The purpose of this is to provide an image loss detection system that analyzes the image and delivers the incidental situation to the control room so that post-processing is carried out quickly.

Objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

The present invention for achieving the above object relates to an image loss detection system, which is a camera functioning as each node based on edge computing, which is installed at a predetermined interval on the roadside to photograph a preset loss detection area on the road, and photographing Edge for collecting images collected, analyzing the collected images to check whether an accident situation occurs, and when an accident situation occurs, collects information related to the accident situation of the corresponding lane, and transmits the collected information to the terminal in the control room As a terminal installed in the camera and the control room, it includes a control room terminal for displaying information transmitted from the edge camera.

When an accident occurs, the edge camera transmits information including the location of occurrence of an accident, camera ID, lane information, type by accident situation, and a scene photo when an accident situation occurs to the situation room terminal, and the situation room terminal sends the information When receiving, it is possible to display an alarm for an emergency situation and display the received information on the screen.

The image failure detection system may further include a CCTV camera for photographing the entire failure detection area, and the situation room terminal may normally display the image captured by the CCTV camera.

Upon receiving the accident situation occurrence information from the edge camera, the controller may further include a controller that enlarges and displays the image of the CCTV camera for photographing the corresponding accident occurrence location on the situation room terminal.

The controller may control the situation room terminal to display a map on which the location of occurrence of an accident is displayed together.

The edge camera may check whether an emergency situation has occurred according to the emergency situation setting information transmitted from the situation room terminal, and transmit information related to the emergency situation to the situation room terminal only in the case of the emergency situation.

The edge camera may perform image analysis using AI on the collected image, and determine whether it corresponds to the accidental situation by using the object attribute value derived by the image analysis.

According to the present invention, the edge node camera provided with the multi-hop function is an IoT-based communication/image processing/artificial intelligence module in which logic for processing a deep learning processing engine in a hardware manner is added to an image processing module that collects and encodes images. It has the advantage of high-speed image analysis as an integrated terminal of In addition, it is possible to operate an economical and high-accuracy oil leak detection system, and has the effect that the system can be continuously installed over a wide range.

In addition, according to the present invention, by significantly reducing the image transmission through self-processing using the edge node camera, there is an effect that can overcome the limitation of the network capacity and install continuous cameras and radars.

1 is a diagram showing the configuration of an image loss detection system according to an embodiment of the present invention.
2 is a diagram illustrating an installation example of an image loss detection system according to an embodiment of the present invention.
3 is a block diagram showing the internal structures of a camera and an image processing controller in a conventional accident detection system.
4 is a block diagram showing an internal structure of an edge camera in an image absence detection system according to an embodiment of the present invention.
5 to 7 are flowcharts of processing in an image loss detection system according to an embodiment of the present invention.
8 is a flowchart showing an excessive amount of pre-setting of an emergency situation according to an embodiment of the present invention.
9 is a flowchart illustrating a method for detecting an accidental situation in an edge camera according to an embodiment of the present invention.
10 is an exemplary diagram for explaining an AI image analysis technology for detecting an emergency situation on a road according to an embodiment of the present invention.

Advantages and features of the embodiments disclosed herein, and methods of achieving them will become apparent with reference to the embodiments described below in conjunction with the accompanying drawings. However, the embodiments to be proposed in the present disclosure are not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments are provided to those of ordinary skill in the art. They are only provided to fully indicate their categories.

Terms used in this specification will be briefly described, and the disclosed embodiments will be described in detail.

The terms used in this specification have been selected as widely used general terms as possible while considering the functions of the disclosed embodiments, but may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the detailed description part of the corresponding specification. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the content throughout the present specification, rather than the simple name of the term.

References in the singular herein include plural expressions unless the context clearly dictates that the singular is singular.

When a part "includes" a certain element throughout the specification, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. Also, as used herein, the term “unit” refers to a hardware component such as software, FPGA, or ASIC, and “unit” performs certain roles. However, "part" is not meant to be limited to software or hardware. A “unit” may be configured to reside on an addressable storage medium and may be configured to refresh one or more processors. Thus, by way of example, “part” includes components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functionality provided within components and “parts” may be combined into a smaller number of components and “parts” or further divided into additional components and “parts”.

In addition, in the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

FIG. 1 is a diagram showing the configuration of an image loss detection system according to an embodiment of the present invention, and FIG. 2 shows an installation example of the image loss detection system according to an embodiment of the present invention.

1 and 2 , the image loss detection system of the present invention includes an edge camera 100 , a Closed Circuit Television (CCTV) camera 200 , a controller 300 , and a situation room terminal 400 .

The edge camera 100 is a camera that functions as each node based on edge computing. It is installed on the roadside at a predetermined interval to photograph a preset detection area on the road, collects the captured image, and analyzes the collected image. It is checked whether an oil condition has occurred, and when an oil condition occurs, information related to the oil condition of the corresponding lane is collected, and the collected information is transmitted to the situation room terminal 400 .

The edge camera 100 detects vehicles, motorcycles, people, wild animals, and falling objects on the road to determine whether there is an accident, and transmits the point where the accident occurred and the contents of the accident to the situation room terminal 400 so that the operator can respond appropriately. make it possible

In addition, the edge camera 100 enables wireless communication between the edge cameras, and also provides a routing function to provide information even at a distant place.

The CCTV camera 200 is installed to photograph the entire accident detection area.

When the controller 300 receives the accident situation occurrence information from the edge camera 100 , the image of the CCTV camera 200 photographing the corresponding accident occurrence location is enlarged and displayed on the situation room terminal 400 .

The situation room terminal 400 is a terminal installed in the control room, and serves to display information transmitted from the edge camera 100 .

In the present invention, when an emergency occurs, the edge camera 100 transmits information including the location of the accident, camera ID, lane information, type by accident situation, and a scene photo when an accident situation occurs to the situation room terminal 400 . .

And, when the situation room terminal 400 receives information from the edge camera 100 , it displays an alarm of an emergency situation, and displays the received information on the screen.

The situation room terminal 400 may display an image captured by the CCTV camera 300 in normal times.

In the present invention, the controller 300 may control the situation room terminal 400 to display a map on which the location of the occurrence of the accident is displayed together.

In the present invention, the edge camera 100 checks whether an emergency situation has occurred according to the emergency situation setting information transmitted from the control room terminal 400, and, only in the case of the emergency situation, transmits information related to the emergency situation to the control room terminal ( 400) can be transmitted.

In the present invention, the edge camera 100 has its own error detection control unit built-in, and also includes a wireless network (routing) function, so that it can be freely expanded to a wide range without a separate limitation in the error detection range. If the field system exists in a network independent of the operator, it may be configured to transmit information to the operator through a gateway controller. This configuration can configure the network regardless of MANET (Mobile Adhoc Network), Internet network, and private network.

The main functions of the edge camera 100 of the present invention are as follows.

First, independent objects on the road, such as vehicles, people, motorcycles, animals, and falling objects, are detected and classified.

And, by tracking the movement of each object, the situation is detected, such as whether it is stagnant due to reverse driving or an accident. For example, the accident situation may include reverse driving of a vehicle, appearance of people and animals, occurrence of falling objects, and vehicle stop.

In addition, the location of each object on the road may be classified into lane units.

3 is a block diagram showing the internal structures of a camera and an image processing controller in a conventional accident detection system.

Referring to FIG. 3 , in the conventional accident detection system, the camera 10 and the image processing controller 20 are implemented as separate devices.

The camera 10 receives an image through the image sensor 11 , and the encoding unit 130 of the image collection board 12 encodes the image into a file or video.

Then, the encoded image is transmitted to the image processing controller 20 through the transmission unit 14, and the image processing controller 20 receiving the image processing board 21 for image processing for detection of errors. The decoding unit 22 performs a decoding process.

And, after the detection of the false information, the encoding unit 24 stores the data by encoding the image again in order to make the video data.

In this way, when error detection occurs, the error information is transmitted to the operator terminal, otherwise only the image is transmitted to the operator and the next image processing is continued.

4 is a block diagram showing an internal structure of an edge camera in an image absence detection system according to an embodiment of the present invention.

Referring to FIG. 4 , the edge camera 100 receives an image through the image sensor 110 , and performs image processing for error detection through its own image processing software 130 before encoding.

And, after image processing, in the encoding unit 140 of the image collection and processing board 120

Encoding and storing data. At this time, if an error condition is detected, the data is transmitted to the control room terminal 400. Otherwise, the data is discarded and the next image processing is continued.

As such, in the case of the edge camera 100 in the present invention, unnecessary repetition of transmission, encoding, and decoding can be reduced.

The present invention also supports a wireless multi-hop network to enable expansion of a service area.

In addition, in the present invention, by adding a CCTV camera 200 that shoots a long distance with high performance, it is possible to increase the operating effect of the operator in addition to the maintenance monitoring and real-time monitoring functions of the edge camera 100 . Due to the characteristics of edge cameras, there is a limit to support the zoom in/out function and high resolution, so it can be configured to add a separate high-resolution CCTV camera.

In the present invention, when an emergency situation is transmitted from an edge camera using a CCTV camera, the edge camera area is automatically enlarged to provide an emergency situation preset function so that the operator can observe immediately.

5 to 7 are flowcharts of processing in an image loss detection system according to an embodiment of the present invention.

Referring to FIG. 5 , the edge camera 100 collects captured images ( S501 ).

Then, the edge camera 100 periodically checks the state of the wireless network between each node (S503).

Then, by analyzing the collected images, it is checked whether an emergency situation has occurred (S505, S507).

When an oil condition occurs, information related to the oil condition of the corresponding lane is collected (S509). For example, information such as the location where the accident occurred, camera ID, lane information, the type of accident, and still image of the accident situation is collected. Here, the location where the accident occurred is used as information for displaying the map, and the camera ID is used to check the location where the accident occurred by camera, and in particular, when the CCTV is operated, it is used to set each camera preset in advance. . And, the type of remains means the type for each situation, and the still image is a photo of the site when the remains are found.

Then, the collected information is transmitted to the control room terminal 400 (S511).

When the CCTV camera 200 is installed (S513), information such as the ID of the CCTV camera and the location of the accident is transmitted to the situation room terminal 400 (S511).

Figure 6 is an embodiment of the present invention in the case of setting and operating a high-performance CCTV system to monitor the entire accident detection area.

Referring to FIG. 6 , the high-performance CCTV system periodically checks the network status with the situation room terminal 400 and the edge camera 100 ( S601 ).

If there is no emergency situation, the screen set by default in the CCTV camera 200 is transmitted to the situation room terminal 400 (S603).

Periodically inspect whether an accident occurs, and when a failure situation occurs (S605), an optimal CCTV image for monitoring the failure situation is collected by applying a preset value set in advance through the camera ID where the failure situation occurred (S607).

Then, the changed image is transmitted to the situation room terminal 400 (S609).

Referring to FIG. 7 , if there is no CCTV, the situation room terminal 400 displays a basic screen for facility monitoring (S701).

When an emergency situation alarm is transmitted from the edge camera 100 (S703), a still picture and the collected information are displayed (S705). In the case of operating the map, the controller 300 displays the map on the situation room terminal 400 so that the point of occurrence of the accidental situation is expressed through the location information.

When the CCTV is operated, the real-time CCTV image of the accident site is displayed on the screen of the situation room terminal 400 (S707).

8 is a flowchart showing an excessive amount of pre-setting of an emergency situation according to an embodiment of the present invention.

Referring to FIG. 8 , the edge camera 100 attempts to connect to the network with the control room terminal 400 (S801), and when the control room terminal 400 and the network are connected (S803), the function of setting an emergency situation in advance In order to do this, it requests the setting of an emergency situation to the situation room terminal 400 (S805).

Then, when the emergency situation setting information is received from the situation room terminal 400 (S807), the operation of the edge camera 100 is started accordingly (S809). That is, in the present invention, the edge camera 100 checks whether an emergency situation has occurred according to the emergency situation setting information transmitted from the control room terminal 400, and, only in the case of an emergency situation, transmits information related to the emergency situation to the control room. It can be transmitted to the terminal 400 .

In case of detection of loss through conventional image analysis or radar processing, a large amount of image and radar data processing must be processed by a separate control unit. (100) performs the role of a kind of loss detection sensor to perform image processing by itself. That is, the edge camera 100 of the present invention performs an image processing internally and at the same time performs a role as a sensor device for detecting an emergency situation.

Therefore, when a user sets an emergency situation in the edge camera 100 that performs the function of the accident detection sensor through the situation room terminal 400, the edge camera 100 provides an alarm and additional information to the user only when a corresponding emergency situation occurs. (point, time, still image, etc.) can be provided.

In the present invention, the setting of the emergency situation can be variously set according to the environment.

For example, in the case of a road, the emergency situation may be set as reverse running, stopping, falling objects, people appearing, animals appearing, and the like. In the case of hiking trails, it is possible to set the situation of an accident such as the appearance of a person in the entry control zone, no movement of the person for a certain period of time, or the occurrence of a forest fire. In the case of an exclusive road for automobiles, it is possible to set the accidental situation by the appearance of a motorcycle or the appearance of a person.

9 is a flowchart illustrating a method for detecting an accidental situation in an edge camera according to an embodiment of the present invention.

Referring to FIG. 9 , the edge camera 100 collects images captured in real time (S901), and performs image analysis using AI (Artificial Intelligence) on the collected images (S903).

Then, object attribute values derived by image analysis are collected (S905).

Then, it is determined whether there is an emergency situation using the object attribute value (S907).

If it is determined that an emergency situation has occurred (S909), the information on the emergency situation is propagated to the adjacent edge camera (S911). Alternatively, upon receiving information on occurrence of an emergency situation from an adjacent edge camera, it is propagated to the adjacent edge camera again (S911).

In the present invention, the emergency detection technology of the edge camera 100 is based on a video image classification technique using a deep learning algorithm. That is, each edge camera 100 installed on the road detects an object that changes in the monitoring area, makes it an object (given an ID), and tracks the movement of each object.

Through such object detection, it is possible to continuously detect and track an object moving from one edge camera to an adjacent edge camera.

Based on these functions, the edge camera's emergency detection technology defines each emergency situation through information such as the location, shape (size and color), movement speed, and presence of each object, and provides customized services can do.

The most representative oil condition on the road will be exemplified, and the algorithm that the edge camera of the present invention operates to detect the oil condition will be described.

The following four situations can be exemplified as a disaster situation on the road.

1. Vehicle stop: Detecting a situation in which an accident occurs on the road or the vehicle stops due to an abnormality in the driver

2. Reverse Driving: Detecting a vehicle traveling in the reverse direction of the road

3. Appearance of people or animals: the index finger of the appearance of people and animals on the road

4. Falling object detection: Detecting falling objects on the road

The AI image technology used for object detection in the present invention is an object detection technology based on image learning and an object detection technology using an image processing technology.

The object detection technology based on image learning conducts pre-learning through a deep learning algorithm for various object images classified into classes, and uses the algorithm obtained as a result of learning to detect the target object among the images collected from the road. will find it

In the present invention, it is possible to detect the size, shape, color, and position attribute values of each object through the object detection technology using the image processing technology. Among them, since the range for the falling object is too vast, it is possible to detect the falling object by using the image post-processing technique to extract the difference image of the image collected with the time difference.

10 is an exemplary diagram for explaining an AI image analysis technology for detecting an emergency situation on a road according to an embodiment of the present invention.

Referring to FIG. 10 , the edge camera 100 secures lane information of a point at which an object is detected by setting an area for each lane in a previously collected image.

Then, for the collected real-time image, an object is detected through AI image analysis technology, and as a result, each attribute value is derived.

Attribute values can be defined as 'Position_ID' indicating a lane number, 'Class_ID' indicating object classification, and 'speed' indicating object speed.

Position_ID may be set to a value in the range of 1 to n. For example, in the case of a secondary road, the value of Position_ID is 2.

In the case of Class_ID, object attribute values can be defined by classifying large vehicles into BC, medium vehicles as MC, small vehicles as SC, motorcycles as MV, humans as HO, animals as AO, and falling objects as EO.

In the case of speed, an object attribute value can be defined as -m ~ +n (km/h). For example, if the object speed is 60 km/h, the object property value may be defined as the speed 60. In the case of reverse driving, the object property value of speed may become negative.

In the example of FIG. 10, after securing the vehicle information through the detection coordinates of the object and tracking, if the object overlaps blocks A to F, the lane information of the corresponding block is determined, and the object attribute value of Postion_ID is derived. can

For example, if an object overlaps block C and block D, it is determined that the object is in the second lane, and the object attribute value of Position_ID can be derived as 2.

In addition, AI image analysis is performed on the detected object to classify the object type, and through this, an object attribute value of Class_ID can be derived.

For example, when the type of object is a medium-sized vehicle, the object attribute value of Class_ID may be derived as MC.

And, by calculating the speed of the object through the distance (AB, CD, EF) between blocks for each lane and the time detected in the block, it is possible to derive the object attribute value of the speed.

For example, if the distance between block A and block B is 20.0 m and an object moves from block A to block B, the time the object is detected in block A is 16:10:10, and the object in block B If the detected time is 16:10:12, the speed (m/s) is 20/2 = 10 (m/s). Converting this to speed (km/h), the speed is 36 (km/h). Accordingly, the object attribute value of velocity is 36 derived.

In the present invention, the edge camera 100 may derive such an object property value, and determine the existence of an object through the derived object property value.

For example, the edge camera 100 may recognize that the object is a small vehicle through the object property value, recognize that the object is in the first lane, and recognize that the speed of the small vehicle is 0, through which ' It can be detected that an accidental situation of 'vehicle stop' has occurred.

Also, the edge camera 100 may detect that a 'reverse running' accident situation has occurred when the object speed of the vehicle is - through the object attribute value.

In addition, the edge camera 100 may recognize that the object is a person or an animal through the object attribute value, and detect a situation in which the person or the animal appears on the road.

In addition, the edge camera 100 may detect the presence or absence of a falling object using an image post-processing technology through an object attribute value.

Although the present invention has been described above using several preferred embodiments, these examples are illustrative and not restrictive. Those of ordinary skill in the art to which the present invention pertains will understand that various changes and modifications can be made without departing from the spirit of the present invention and the scope of the appended claims.

100 Edge Camera 200 CCTV Camera
300 Controller 400 Control Room Terminal
110 Image sensor 120 Image acquisition and processing board
130 Image processing unit 140 Encoding unit

Claims (7)

As a camera that functions as each node based on edge computing, it is installed at a predetermined interval on the roadside to photograph a preset detection area on the road, collects the captured image, and analyzes the collected image to determine whether an accidental situation occurs. an edge camera for checking and collecting information related to the accident situation of the corresponding lane when an accident situation occurs, and transmitting the collected information to the situation room terminal;
CCTV camera for filming the entire oil and gas detection area; and
A terminal installed in a control room, comprising a control room terminal for displaying information transmitted from the edge camera,
The edge camera checks whether an emergency situation has occurred according to the emergency situation setting information transmitted from the situation room terminal, and transmits information related to the emergency situation to the situation room terminal only if it corresponds to the emergency situation,
When an accident occurs, the edge camera transmits information including the location of the accident, camera ID, lane information, type by accident situation, and a scene photo when an accident situation occurs to the situation room terminal,
When the situation room terminal receives the information, it displays an alarm of an emergency situation, and displays the received information on the screen,
The situation room terminal normally expresses the image captured by the CCTV camera,
Further comprising a controller that, when receiving the occurrence information of the occurrence of an accident from the edge camera, enlarges the image of the CCTV camera that captures the location of the occurrence of the accident and expands it to the terminal of the situation room,
The controller causes the situation room terminal to display a map showing the location of the occurrence of an accident,
The controller periodically inspects whether an accident occurs, and when an emergency situation occurs, a preset preset value corresponding to the camera ID in which the failure situation occurs is applied to collect the optimal CCTV image for monitoring the emergency situation,
The edge camera is equipped with a multi-hop routing function so that wireless communication between edge cameras is possible. propagate back to other adjacent edge cameras,
The edge camera performs image analysis using AI on the collected image, derives an object attribute value by image analysis, and determines whether it corresponds to an emergency situation using the derived object attribute value, but the object attribute value If it recognizes that the object is a vehicle through It is detected that this has occurred, and when it is recognized that the object is a person or an animal through the object property value, the status of the person or animal appearing on the road is detected, and the status of the falling object is detected using the image post-processing technology through the object property value. Image loss detection system, characterized in that for detecting the.
delete delete delete delete delete delete

Images